Category Archives: White Papers Exhibition

The technology of exhibition explained; from media players and projectors, to memory and local storage, from satellites and devices for the hearing and visual impaired…place or read it here.

Accessibility Technology Requirements for Cinema

A White Paper in which Harold Hallikainen of USL/QSC gives the critical information about the most recent United States Department of Justice rules for accessibility equipment in the cinema auditorium.

The deadlines (from last page of report) are:

  • Assistive Listening Systems are required to be operational now (rules requiring them are more than 20 years old). 
  • Staff Requirements are effective January 17, 2017, if a theater is providing closed captioning or audio description.
  • Advertising Requirements are effective January 17, 2017, if a theater is providing closed captioning or audio description.
  • Closed Captioning and Audio Description equipment is to be operational by June 2, 2018. However, if a theater converts from film to digital after December 2, 2016, closed captioning and audio description equipment must be installed within 6 months of the conversion or December 2, 2018, whichever is later.

Loudness in Cinema – IBC 2016 Presentation

A Complete Facility Inventory tool with RESTful hooks for an FLM interface is basically working. A Manufacturer’s Product Line Input Tool is in the works.

Loudness Intro Inventory System

 

The Audio Maintenance and Set-up sheets from the upcoming SMPTE Modern Calibration Procedures are laid out and working.

Loudness Intro SMPTE Audio Survey and Maintenance System

Daily/Weekly/Monthly Checklists are working, but need some detail added.

Loudness Intro Checklist System

These are all available for testing at the site: DCinemaCompliance.com

The Projectionist Training site DCinemaTraining.com needs 2 more chapters and a QA pass.

Loudness Intro Projectionist Training

Digital Test Tools, the hardware company, has a developed monitoring product waiting for production financing.


Today’s topic is that major tangent of Quality Assurance, Loudness in Cinemas.

Loudness In Cinema Intro MainSlide

We’re not dealing here today with Fletcher Munson Curve-like loudness. We’re dealing with what it is when the audience member says “It’s too loud.”

Loudness In Cinema Definition 0

We’ll start with the reminder that in a quiet room, the mosquito which generates 20/100,000th of a Pascal is too loud (20 x 10-6).

The attempt to create a clever Venn Diagram and a Loudness Matrix turned out to be a ridiculous proposition.

Loudness In Cinema DCPs - Definitions Venn

Too many tangents.

Every time I interviewed someone else, it was obvious: It’s tangents all the way around.

Loudness In Cinema Tangents of Goals and Purposes

One thing was clear throughout: The word of the year is “Annoyance”.

We’ll take up the tangents by Stakeholder segment, attempting to include a

  • What Can Be Done or
  • What Should Be Studied

for each stakeholder.

Loudness In Cinema Stakeholder Goals and Purposes

We start with this poignant quote from Hans Zimmer, who has taken a lot of abuse in the last couple years, along with Christopher Nolan.

Loudness In Cinema Hans Zimmer Quote

Creative Intent

Not much can be said about this – It is why we chose this field, this technology. For people in sound post production, it means that after years of getting a movie made and locked, music, dialog and effects are forced largely be created and assembled in a few weeks.

A few items on the To Be Studied List is:

  • whether mixes are actually louder and
  • why are mixes louder, and
  • how to communicate better that turning down a mix at the audio processor actually makes the critical dialog more unintelligible.
  • to separate fact from anecdote are stories that mixers in the EU are messing with the master gain to match what is happening in auditoriums.
    (From several interviews at major stages and mix rooms in LA, this isn’t happening there and most engineers sneer at the idea of it.)

Mixers go to a great deal of trouble to get the mix right then check it at other auditoriums – not just premier rooms but other auditoriums where us common people go. But they bring in their own projectors and tune the room. So it isn’t exactly like what us common people see and hear…note to self – Create studies to find:

  • If any producer or director sits for 15 minutes in dim-ish light with dim-ish music until their hearing sensitizes to quiet, then gets blasted by TASA Compliant but fully compressed, loud trailers (not their own).
  • Is it the loudness of the trailers that most people respond to as too loud and not the movies

Insert anecdote about how this is what my wife now trusts when I explain it to her while in the theater.

  • how many of the public is a complainer
  • how many of the public would choose to hear the movies like the Creatives intended.

A side study would be to

  • Find out the levels that people are playing these movies (or their music) at when they listen with ear buds on their phones and tablets.

David Monk makes the observation that once he knows what is being said, on his normal TV setup for example, the words are obvious. But sometimes he only finds these obvious words after replaying with subtitles on. Several people interviewed subsequently tell of doing and finding the same thing. David suggests that producers and mixers don’t know what we don’t know, that those words which are obvious to them would be obvious in any circumstance after hearing the words so many times during production and mixing.

For another view of the topic– one which verifies the theory of a prominent studio mixer/exec – I flew in with a women who trained at a major film university, and has subsequently mixed and directed several movies. She wasn’t trained and has never mixed with a VU meter. Instead of building a mix around dialog between -14 to -20 on a VU meter, it is done “at a comfortable level with the peak meter never hitting red.” She has never considered how this could create a big difference in movie loudness.

Tangents and Edge Cases

One horrible stat is that 60% of recent US war veterans have permanent hearing loss or chronic tinnitus ringing in the ears. That’s 600,000 of the former and 850,000 of the latter in the US alone. Add to that, 15% of baby boomers have significant hearing problems, 7.5% of 29-40year olds. UK military and civilian stats are similar in percentage and degree.

Loudness In Cinema Audience as a Stakeholder

The vets problems areorders of magnitude worse than imaginable; it isn’t rare for 25dB loss in one ear and 16dB in the other depending on how they held their rifle or what they typically sat next to. They were instructed to wear ear plugs under their helmets, but the military’s own studies show that finding a target in that condition doesn’t work.

Everyone you ask about cinemas has an opinion on loudness. Not all bad, e.g., one of the scientist interviewed for this segment said that:

The ability to control sound level while watching movies at home is the main reason people like me (no longer teen age) avoid the cinema (movie theaters) altogether.

His idea is to be able to bring and listen through his own bluetooth headphones so he can regulate the volume. Interesting concept.

I spoke to a friend about another friend whose hearing loss is suspiciously at the tone that his wife uses when she is upset. The 2nd friend says that he had the same issue – his wife insisted on tests and his doctor showed the frequency band on his chart where this occurred.

Their loss areas are one thing, but the edge frequencies leading to them are often ‘annoying’. Yet because speaker’s vertical frequency dispersion is nowhere near as smooth as their horizontal dispersion, we commonly place people with sub-prime hearing in sub-prime auditorium seats.

In the practical world, tangents are the all too common edge conditions. Later, we’ll look at some of the impacts that edge conditions might cause in our efforts at building a great room of sound.

Loudness In Cinema Zebrafish Do It

Loudness In Cinema Zebra Don't

Here’s the deal. All vertebrates can regenerate the damaged hair follicles that allow hearing and other sensing (such as the microscopic hairs on the bottom of a fish that senses variations in water currents.)

All vertebrates, except mammals.

Loudness In Cinema DCPs for Non-Except for Mammals

This is one view of the sets of hairs that are inside the Organ of Corti, which is part of the cochlea of the inner ear of mammals. We see the longer hairs and other views will show shorter adjacent hairs.

Chevron of Inner Ear Hairs

There are about 18-20,000 interconnected hairs. They all contain an even finer stereocilia that does the delicate touching on various parts of adjacent hairs, and which then help convert the stimulus into electrical signals using a transfer of potassium ions from the tip to the base of each hair.

Loudness In Cinema DCPs - Hair to Hair Transfer

This is an electron microscope view of a frogs hair which work on the same principles but when damaged by trauma, will regenerate. It is in the power of the nerve and its adjacent helper cells to recreate them using a gene factor called ATOH1. There is something in mammals blocking this function.

Loudness In Cinema – Frogs Inner Ear Hair

Another view of good working hairs. You’ll have noticed several in a chevron shape. These are tonotopically organized from high to low frequency.
If you start thinking of ⅓ octave EQ sets, you won’t be too far wrong.

Loudness In Cinema Chevron Hairs

Shown here (at the asterisk) are missing and damaged hairs. After damage the hairs and helper cells seem to maintain some viability for 10 days. Noise-deafened guinea pigs – given 60-70 dB hearing loss by simulated gunfire – can get substantial improvement if the Atoh1-based gene therapy is applied during that time period. Suffice to say, that’s a bumper-sticker statement for a complex decade of study.

Loudness In Cinema Missing Hairs

Again, good on the right and damaged in the center and left. We’ve all probably heard of missing limb phantom pain? There’s a working theory that these stragglers or the missing hairs themselves initiate a missing limb-style effect, which causes tinnitus.

Loudness In Cinema Good and Bad Hairs

Yehoash Raphael, the scientists who wants to use his bluetooth headphones in the cinema, makes the unequivocal statement:

There is no viable biological treatment for hearing loss yet.

Dr. Raphael also mentions that current science indicates an important negative outcome of acoustic trauma named Synaptopathy, that is, hidden hearing transmission issues at the nerve itself.

There are hundreds of researchers and many grants funding the laborious process of finding what works and what doesn’t, many using a friendly virus as a carrier for a gene factor.

Loudness In Cinema Gene Therapy ATOH-1

That’s as far as we’ll follow this tangent. Supporting documents will be put in the package of this presentation on the EDCF website. (Please acknowledge their copyrights if you republish this!)

So, what is the To Be Studied or done in the audience stakeholders domain. Education? Discovery?

What percentage actually complain, but this time, what is their history, and what exactly are they complaining of? Myself, if I get wax build-up, I will hear crackling sounds at loud piano recitals.

Instead of damaging the audio balance and intelligibility of the dialog by turning down the dial, could we map the auditorium for the audience? Would they understand if the cinema manager said that the sound won’t be turn down but that they would guide them to a seat that is less loud, or less loud at various frequencies?

Another tangent: The two reasons that broadcast world’s Loudness science doesn’t apply in cinema; the audience doesn’t have a remote control, and LUFS technology needs modifying for the length of movies. Thus the argument that goes: If I come to the theater and it is too cold, I put on a sweater. If it is always too loud, I put in my ear plugs (or in some pluperfect future, I shall have put on my bluetooth headphones).

Loudness In Cinema Exhibition as Stakeholder

Exhibition is very reticent of long drawn out studies becoming a red flag for sensationalists. They are the ones which stand to be most impacted by hyperbole and dissemination of partial truths. Recently, such hyperbolic betrayal came from within the technical community.

Here is what they are afraid of.

Loudness In Cinema Exhibitor House

Loudness In Cinema Focus on External Monitoring

Adding to the already complex structure of a cinema facility, the region of Barcelona passed a law that requires a back channel to the mayor’s office giving loudness data and the logs of the limiters as they kicked in during overages! Only because they were convinced by a certain company that the equipment is not available – such as a 64 channel limiters for an ATMOS system, or even an 8 channel for a 7.1 system, did the enforcement get dropped.

Just as there are no SMPTE or ISO police, there is no NATO or UNIC enforcers. The exhibition community response reverts to the basic premise that there are many commercial decisions that can’t be enforced by fiat. There are benefits and drawbacks to that. As an extreme example, as late as 2007, I installed digital cinema servers into rooms that were just converting from mono.

Loudness In Cinema Stakeholder 4 Technology

On the other hand, France, the largest EU market by many industry metrics, does have an enforcement arm that monitors exhibition facilities. The CNC normalized the ISO/SMPTE documents, and made them the law of the land. Alain Besse of the CST has begun taking his research project into Loudness all the way back to distortions created in production – microphone choices and placement among other things. He is planning a December symposium to study these and other matters.

There is one other important thing that Alain points out to those that grouse about loud sports and other entertainment venues. Communities are investigating sound not because of cinemas per se. There is fear of an epidemic of destroyed hearing from loud sound – especially low frequency sounds – in public venues, and cinemas are just another public space on the list.

On the To Be Studied List for exhibition is whether short term exposure to 85 and 90 and even 100 dB bursts of sound destroys ears. There is generalized info but not rigorous data. Mothers complain about children who come out with ringing ears, but are those kids also wearing ear buds listening to constant barrages of even louder sounds?

It should be clear that this is not an excuse for badly implemented sound in the auditorium, but what is the reality?

Loudness In Cinema Stakeholder Five – Standards Groups

Time, of course.

…and biting off more than can be handled.

The SMPTE group that developed the new digital pink noise standard started nearly 3 years ago. The documents were released many months ago. But for something so primary, there is little public knowledge and very little implementation. The SMPTE store still doesn’t have a standard tone package available for download. A pink standard was Building Block Number One of a list that needs to be done before Loudness can be tackled. Not pointing fingers, but rather pointing out that there is only so much that a volunteer group can do with their spare time. The long-term arc is great, but short-term progress is slow, and expecting engineers to be good at socializing is probably a good source for an oxymoron joke.

People new to the field always ask, “How about transcribing old standards for hearing loss in the workplace?” Well, it turns out that workplace laws – OSHA and the like – were draconian, not in the sense of being onerous for the facility owner, but onerous in the sense of only caring whether the worker still had the bandwidth available to hear a conversation at the end of their work day and at retirement. A worker was considered to have a material hearing impairment when his or her average hearing threshold levels for both ears exceeded 25 dB at 1, 2 and 3 kHz.

And finally, no group wants to walk the path that gets near the briar patch of liability. Societies become litigious for a reason, and despite the extremes that are used to mock the rules on either swing of the pendulum – with real or imagined anecdotes – we as practitioners of the technological arts can’t allow a vacuum to pull in a problem without being ready to correct it with real science in the face of legislators who hire an over ambitious engineering group to “Save The Children”. If we do allow our heads to be put in the sand, we’ll get laws like in Flanders mandating that children’s programs have to be played at essentially 4 on the dial, and adult fare at 5 on the audio processor dial.

Our new contribution to this Quality Assurance situation is a website that offers free DCPs and a comprehensive checklist for non-technical managers.

This Manager’s Walk Through Series gives the facility manager some method and knowledge against the impossible task of judging their auditoriums and communicating with their tech staff.

Each DCP is different, but each has high and low tones played in sequence around the room, with distorted and muted tones for comparison.

Loudness In Cinema DCPs for Non-Technical Manager w/Checksheet

There are graphics included to get the managers used to sensing the problems and the quality potential of their rooms. One DCP uses faces as the empirical standard to judge colors by. Another uses a cool educational graphic from the xkcd.com website, and there are more to come with lessons that fill them in on what they should expect as they build their talents. There’s also a nice dose of the new SMPTE pink noise for sweeping a room and a 2Pop DCP that puts a sync pop into different speakers every 2 seconds.

Download these from cinematesttools.com, password: QA_b4_QC

Loudness In Cinema DCPs for Non-Technical Manager

We look forward to helping you advance the trend of quality assurance in the cinemas. Thank you.

What there was no time to say in a 15 minute presentation:

A mythos has been created that there is a trend toward loudness laws because two, maybe 3, county level (not country level) groups have created laws that regulate audio levels in cinemas. In fact, the Flanders section of Belgium and the Municipality of Barcelona in Spain are the highlights and limits of that trend, and they did so 3 years ago (and only one is, or can be, implemented).

Not to say that there isn’t a good purpose in studying audio levels, but there is no need for the science, fact-based groups to use hyperbole any more than the “gee, we must do something to save the children” or the sensationalist press groups to use hyperbole. Likewise, there is no need to demean the low-knowledge groups as just did, since many are, in fact, properly working in a difficult area – clubs, concerts, sporting arenas, auto racing…) where there is a need to regulate due to entertainment industries that do deliver long and repeating exposure of +110dB levels. That cinemas, which might use brief periods of plus 100dB levels as part of the storytelling experience, get lumped into the same category is all the more reason that the area needs to be examined with the talents we have and not rely upon hope and namecalling.

If Annoyance is the buzzword, it is Distortion that is the hidden hole. In the field of projection we know that there was a long trend of installers specifying projectors down to the level that they just barely made the luminance levels for the size of the screen. This turned around to haunt the industry when exceedingly lower light level 3D became the norm, a norm that could only deliver a set of distortions, from horrible contrast to minimal stereoscopic separation. Those human visual system distortions developed horrible pictures that developed headaches and complaints and eventual collapse of a technology that should have improved but couldn’t due to under performing equipment.

Likewise, under-spec’d (and old) audio equipment delivers distortions of their own, and amplify distortions that are inherent but might go unheard in better systems that correctly play to the sensitivities of the human auditory system. From inexpensive first generation converters to speakers aimed above the heads of the audience, there are numerous potential points of failure that need to be put into a matrix and studied alongside the numerous potential points of failure in the hearing system of the varied audience members. The study that is required is one of a grand scale, and get even grander if there is any attempt to quantify subtle factors dialog intelligibility, loudness and room size (and image size?), their relation to annoyance, accommodation, and audience engagement, and differences between pre-show material and the movie itself, or (shudder to even type it) studying the actual limits for safe listening given the variations of human structure and past listening habits.

It will require a huge conclave of the various sciences. There are many existing groups which have different pieces to match the needed scope of the problem, many which are encumbered by the same time and access problems that SMPTE has, and the political expedient of self-regulation that is demonstrably incapable of reliably playing back movies at the level of artistic intent. Perhaps just creating a group that can generate a public venue to even create an outline of this kind of shared open project would be a first good step.

Finally, more at hand, there are methods that use current tools in the cinema field and variations of new tools developed as a solution for the broadcast field as a beginning for study and development of a valuable metric and algorithm and technique for use, instead of the silly and quite arbitrary Flander-based rules. Mr. Allen has developed a moving time window technique with Leq and Mr. Leem has put forth LUFS-based ideas. These studies, and others as they present themselves, should be open-sourced so that peer review can be done in a more modern and expeditious manner. The first step might be to describe these procedures well onto GitHub.

Good luck to us all.

States of Exhibition–Evolution

 

Snippet of Drawing: DCinema Facility

DCinema could be said to have started before the turn of the century, or possibly when George Lucas pushed so hard to get the first 100 systems installed to light up Attack of the Clones in 2002. My moment of ‘made it’ came during the set-up week prior to the Cannes Film Festival in 2004, when projector-derived subtitles were working on a 2K projector.

 

So choosing that date as when the R&D project stopped and real products began, we’re closing in on 10 years of evolution. Some tools came very recently, such as implemented standards and products for the deaf/blind/hard of hearing and visually impaired audience members. In North America there has been several chains which are now 100% covered for 100% of movies with closed caption and enhanced and descriptive audio. Obviously 3D got past the needles-in-the-eyes stage of Chicken Little, and it will only get better when laser light engines get integrated with HFR…and by that one presumes it will have to be after all the NIH leaves the projector manufacturers and 48fps leaves the field of HFR.

Here’s a drawing that tries to show the evolution to the degree that a 2D drawing can show such details. Anyone who wants to help map it into HTML for some obvious cool tricks is invited to write [email protected] – likewise, anyone who needs to strip the sponsored bits to make the drawing into a teaching device…please write. It is an evolving drawing as well, so if you’d like to be informed when revisions are made…you’ve got the address.

The full drawing is at: 
DCinema_Auditoria_and_Equipment_Network.pdf

States of Exhibition–Evolution

 

Snippet of Drawing: DCinema Facility

DCinema could be said to have started before the turn of the century, or possibly when George Lucas pushed so hard to get the first 100 systems installed to light up Attack of the Clones in 2002. My moment of ‘made it’ came during the set-up week prior to the Cannes Film Festival in 2004, when projector-derived subtitles were working on a 2K projector.

 

So choosing that date as when the R&D project stopped and real products began, we’re closing in on 10 years of evolution. Some tools came very recently, such as implemented standards and products for the deaf/blind/hard of hearing and visually impaired audience members. In North America there has been several chains which are now 100% covered for 100% of movies with closed caption and enhanced and descriptive audio. Obviously 3D got past the needles-in-the-eyes stage of Chicken Little, and it will only get better when laser light engines get integrated with HFR…and by that one presumes it will have to be after all the NIH leaves the projector manufacturers and 48fps leaves the field of HFR.

Here’s a drawing that tries to show the evolution to the degree that a 2D drawing can show such details. Anyone who wants to help map it into HTML for some obvious cool tricks is invited to write [email protected] – likewise, anyone who needs to strip the sponsored bits to make the drawing into a teaching device…please write. It is an evolving drawing as well, so if you’d like to be informed when revisions are made…you’ve got the address.

The full drawing is at: 
DCinema_Auditoria_and_Equipment_Network.pdf

Reflections on the Growth of 3D

Posted by Barry B. Sandrew Ph.D.               Wednesday, February 1, 2012

Copied from his Blog BSandrew.blogspot.com, with permission

Engaged in 2D and Immersed in 3D

This month, the multiple 3D films being recognized by the 84th Academy Awards cement the fact that 3D is here to stay. On the heels of the Best Visual Effects nominations for Hugo and Transformers: Dark of the Moon, Legend3D Founder, President, CCO/CTO Barry Sandrew dissects the appeal of 3D for the moviegoer in this two-part blog series on the neuroscience of 2D and 3D feature films. Relying on our current understanding of stereopsis, as described in scientific literature, Sandrew contemplates how our brains differentially perceive images and activate emotional responses when viewing a 2D movie versus a 3D movie.

The group dynamics of the 2D experience

When we attend a 2D movie, the group dynamics of the audience come into play as we direct our collective attention as observers to the images on the screen. Our personal space is infringed upon only tangentially by the other audience members that are sitting on either side of us, as well as by those behind and in front of us. The tangential influence upon us of the surrounding audience is a reminder that we are in a theater with strangers, all watching images being projected on the screen. However, in a 2D movie, our fellow audience members typically do not detract from the experience.  In fact, the audience in a 2D movie is intended to be an integral part of the total feature film experience and has been presented as one of the selling points of attending a 2D move in a theater rather than watching it on DVD or Blu-ray at home. As we become engaged in the movie, we can introspectively sense surprise, anger, sadness and happiness based on the perceived action on the screen and to a lesser degree, through the reactions of the audience members around us. We can empathize and sympathize with the characters being portrayed and we can experience anxiety and fear as we follow the storyline.

Disparity—A key element in the perception of stereo

Our brains receive 2D information in a theater as images presented sequentially at 24 frames per second. In a 2D theater experience, motion is perceived as changes in the up, down and sideways position of one object in relation to other 2D objects.  We perceive three dimensions in a 2D movie via single eye or monocular cues such as occlusion, motion, shading, size, parallax, texture gradient, perspective, saturation and brightness. However, lacking in a 2D movie is the most powerful 3D influence on the brain—disparity. Due to the separation of our eyes and the distance between them, we see a slightly different image from each eye.  This separation causes a horizontal displacement between the images, which is referred to as disparity. The best way to test this is to put your thumb in front of your eyes and alternately close one eye and then the other. You will see two separate perspectives of your thumb displaced horizontally and your thumb will appear to jump from side to side. When these separate images from our left and right eye reach our visual cortex, they are directed to highly specialized neurons that are tuned to critical parameters of disparity.  As a consequence, the two images are fused into one uniquely different image that exhibits the appearance of depth and volume. The result is 3D perception.

A lack of disparity contradicts monocular cues

In a 2D movie, each of our eyes receives precisely identical images from the movie screen with no horizontal displacement. When the two sets of identical images reach the visual cortex of our brain, they are directed primarily to neurons tuned to zero disparity. There, they are “fused” together into a single image that is exactly identical to each of the two images that originally comprised it. Other than the existence of monocular stereo cues, mentioned above, the audience does not perceive depth and volume due to the absence of disparity. This lack of disparity in a 2D movie creates a cue conflict situation that prevents us from being fully immersed in the story in the same manner as a 3D movie. The monocular cues in a 2D movie are telling us there is depth, but the lack of disparity contradicts the monocular cues—forcing the brain to try and reconcile the conflict.

The absence of disparity “pushes” the screen away from us

For some monocular cues, like motion and occlusion, there is a further complication that acts to distance us from the movie screen. As mentioned above, in a 2D movie the relative motion of objects on the screen (one 2D object moving relative to and possibly occluding another 2D object) can indicate depth in the absence of disparity. However, in the real world when there is this kind of relative motion in the absence of disparity, our brains normally interpret that information as indicating that the objects are far away. That’s because disparity in human vision drops off significantly at distances of several hundred yards.  You can prove this if you try the thumb trick above, alternately closing one eye and then the other, but this time, try the trick looking at a tree that is several hundred yards away.  You’ll see that the tree will not appear to jump from side to side like your thumb did. This is because the perspectives from our two eyes at that distance are identical. Our eyes do not converge on the tree, but rather are set to infinity.  The father away things are from us the less horizontal displacement we experience.  So our brain “comes up with” a solution in which we are literally “distanced” from the action on the screen. This distancing and lack of disparity in a 2D movie renders the spatial areas in front of and behind the theater screen irrelevant to the story and therefore irrelevant to each member of the audience. The storytelling, in its entirety, happens on the white screen at the end of the theater.

Skillful use of disparity in a 3D movie will always enhance the experience

In spite of the contradictions our brain must reconcile, a 2D movie is almost always comfortable to watch and with proper directing, cinematography and storytelling, it can evoke very strong emotions. Indeed, many of us remember the outpouring of emotions while watching the sinking of the Titanic or the building terror of T-Rex in Jurassic Park, as both movie experiences were presented in 2D. However, the influence of disparity on the audience in a 3D movie cannot be trivialized as a fad or unnecessary. Whether a 3D movie is captured with two cameras or converted from 2D-to-3D, disparity that is properly and skillfully stereographed will always enhance the storytelling experience by significantly amplifying the responses of disparity-tuned neurons and therefore more closely simulating reality.

3D a higher resolution medium

In the visual system, the absolute number of neurons activated in the brain is not as significant as the ratio of activation between subpopulations of neurons responding to zero disparity and those responding to a wide spectrum of tuned disparities, including zero.  This is analogous to an audio amplifier where the ratio between subpopulations of audio frequencies can vary and increasing the gain on the system increases the overall fidelity of the audio experience, meaning it becomes closer to the original source in resolution.  In the same manner, the ratio of activated zero disparity and disparity-tuned subpopulations of neurons in a 3D movie can vary, but it’s the intensity of the relative responses of those subpopulations that create a more accurate and higher resolution stereo image.  It appears that our brains interpret this higher resolution information as more closely simulating reality (“fidelity”) and therefore it immerses us in the movie to a greater degree, creating a heightened emotional investment in the story.  This is, of course, an oversimplification of an exquisitely complex process that has evolved from the earliest primates. However, the theater going experience is something we can all relate to and many of us recognize the profound difference between being engaged in a 2D movie versus being immersed in a 3D movie.

3D movies are a uniquely personal experience

Dynamic disparity (i.e. the relative amount of disparity, changes in disparity over time and the rate of disparity change) transforms the movie screen from a projection screen into a window that has both an interior and an exterior. Consequently, the storytelling environment is projected throughout the entire theater, actively becoming a unique part of the personal space of each member of the audience and resulting in the space both in front of and behind the screen becoming an integral part of the story. For example, an object that might fly out of the screen at us, traveling through what is called negative parallax, becomes very personal for each member of the audience because it affects and/or “intrudes” each audience members’ personal space equally.  Surround sound has an analogous effect in that we are “in it” rather than simply listening to stereo audio in front of us.  As a result, unlike the 2D movie experience, a 3D movie is more personal and the group dynamics of the larger audience are no longer in play to the same extent.  In fact, it’s my opinion that 3D glasses actually have the positive effect of helping to separate us from the other audience members, containing the experience as a more personal one for each of us, thereby potentially amplifying the immersive nature of the experience.

The Bottom Line

When we are engaged in a 2D movie, we are doing so as passive observers, watching the story take place “over there”, on the screen in front of us. However, when we are immersed in a 3D movie, we are doing so as active participants and the action can be happening behind and in front of us—as far as infinity. This is where the central difference lies. We cannot discount the uniquely active physical and emotional reactions that we have in the 3D feature film experience which are not experienced in our more passive reception of 2D movies. I believe, this is one of the reasons why we have seen 3D continue to flourish and be adopted so enthusiastically by both national and international movie exhibitors, as well as by the entertainment and consumer electronics industries at large. Consumers and moviegoers are hungry for visuals that offer a significantly greater sense of engagement and for visuals that spark an immersive, emotional response that has the ability to transport them into the heart of the action.  This year, we will have the opportunity to test the differential 2D and 3D theatrical experiences with three iconic movies that were originally released in 2D, being released in 3D. Top Gun has been converted by Legend3D and will be re-released on its 25th anniversary by Paramount on the heels of 3D converted re-releases of Titanic and Star Wars.  I am confident that all three films will be very successful at the box office, as each was already a proven success in 2D. Depending on how skillfully each of these films was converted, now in 3D they will give those who saw them originally in 2D an immersive and refreshingly unique experience; an experience that will hopefully help to solidify 3D as an essential part of filmmaking.

Next, we will continue to look at the neuroscience of 3D movies and further explore the uniquely tuned structures and neurons in the brain that respond selectively to dynamic disparity. We’ll look at concepts of visual processing that remain contentious within the scientific community, involving differential pathways within the brain that signal separately “where something is” and “what something is.” The evolutionary survival value of binocular vision will be discussed in the context of 3D movies and we will touch upon more primitive structures in the brain that are likely triggered by disparity to elicit powerful physical emotions and ‘flight’ or ‘fight’ reactions. For more detailed information and technical reviews on stereopsis, as well as in-depth information on the uniquely tuned disparity neurons in many parts of the brain, please see the following references:

(1) Cumming BG, De Angeles GC. (2001). The Physiology of Stereopsis. Annual Review of Neuroscience. 24, 203-238.

(2) Howard, IP, Rogers, BJ. (1995). Binocular Vision and Stereopsis.  New York: Oxford University Press.

(3) Poggio, GF, Poggio, T. (1984). The Analysis of Stereopsis.  Annual Review of Neuroscience. 7, 379-412.

(4) Born, R., Bradley, D. (2005). Structure and Function of Visual Area MT. Annual Review of Neuroscience. 28, 157-189.

I wish to acknowledge the review and insightful suggestions of my neuroscience colleague, David Heeger, Ph.D., Professor at New York University (http://www.cns.nyu.edu/~david/) where he is a member of the Center for Brain Imaging. The son of Nobel laureate and chemist, Alan J. Heeger, David Heeger is a contemporary neuroscientist who has been at the forefront in the field of functional magnetic resonance imaging (fMRI). Dr. Heeger was the first to bring together two separate and largely unrelated disciplines—cognitive neuroscience and film studies, opening the way for the exciting new interdisciplinary field of “Neurocinematic” Studies.

 

About the author (from his site):

This is the first in a series of blogs by Barry Sandrew, Ph.D., founder and CCO/CTO of Legend3D. Sandrew received his doctorate in neuroscience from State University of New York at Stony Brook in 1977. After winning a two year NIH Fellowship at Columbia University, College of Physicians and Surgeons, he joined the staff at Mass General Hospital and Harvard Medical School as staff Neuroscientist. In 1987, he left his academic and scientific career to found American Film Technologies, and invent the first all-digital colorization process for colorizing black and white feature films. He took that company public while converting hundreds of movies for clients such as Turner, Disney, Warner Bros., Fox, Gaumont, TF1, ABC, and CBS, among many others. Sandrew left colorization to start two new companies, one of which included Lightspan, one of the largest educational software companies in the world. That company went public and was later acquired by Plato Learning. In 2000, Barry then founded Legend Films and began the process of re-inventing colorization and developing a proprietary restoration process. Over the course of the next seven years, Legend Films converted approximately 145 black and white titles to color as well as several TV series. In 2006, he recognized that the future of entertainment would be 3D and, leveraging his proprietary colorization pipeline, he modified the entire R&D direction and business model to embrace 2D-to-3D conversion, changing the name of the company to Legend3D. Today, as the most innovative company in the conversion industry, Legend3D continues to lead the field, delivering the highest quality product with the fastest turn around time, at the most competitive pricing.


Reflections on the Growth of 3D

Posted by Barry B. Sandrew Ph.D.               Wednesday, February 1, 2012

Copied from his Blog BSandrew.blogspot.com, with permission

Engaged in 2D and Immersed in 3D

This month, the multiple 3D films being recognized by the 84th Academy Awards cement the fact that 3D is here to stay. On the heels of the Best Visual Effects nominations for Hugo and Transformers: Dark of the Moon, Legend3D Founder, President, CCO/CTO Barry Sandrew dissects the appeal of 3D for the moviegoer in this two-part blog series on the neuroscience of 2D and 3D feature films. Relying on our current understanding of stereopsis, as described in scientific literature, Sandrew contemplates how our brains differentially perceive images and activate emotional responses when viewing a 2D movie versus a 3D movie.

The group dynamics of the 2D experience

When we attend a 2D movie, the group dynamics of the audience come into play as we direct our collective attention as observers to the images on the screen. Our personal space is infringed upon only tangentially by the other audience members that are sitting on either side of us, as well as by those behind and in front of us. The tangential influence upon us of the surrounding audience is a reminder that we are in a theater with strangers, all watching images being projected on the screen. However, in a 2D movie, our fellow audience members typically do not detract from the experience.  In fact, the audience in a 2D movie is intended to be an integral part of the total feature film experience and has been presented as one of the selling points of attending a 2D move in a theater rather than watching it on DVD or Blu-ray at home. As we become engaged in the movie, we can introspectively sense surprise, anger, sadness and happiness based on the perceived action on the screen and to a lesser degree, through the reactions of the audience members around us. We can empathize and sympathize with the characters being portrayed and we can experience anxiety and fear as we follow the storyline.

Disparity—A key element in the perception of stereo

Our brains receive 2D information in a theater as images presented sequentially at 24 frames per second. In a 2D theater experience, motion is perceived as changes in the up, down and sideways position of one object in relation to other 2D objects.  We perceive three dimensions in a 2D movie via single eye or monocular cues such as occlusion, motion, shading, size, parallax, texture gradient, perspective, saturation and brightness. However, lacking in a 2D movie is the most powerful 3D influence on the brain—disparity. Due to the separation of our eyes and the distance between them, we see a slightly different image from each eye.  This separation causes a horizontal displacement between the images, which is referred to as disparity. The best way to test this is to put your thumb in front of your eyes and alternately close one eye and then the other. You will see two separate perspectives of your thumb displaced horizontally and your thumb will appear to jump from side to side. When these separate images from our left and right eye reach our visual cortex, they are directed to highly specialized neurons that are tuned to critical parameters of disparity.  As a consequence, the two images are fused into one uniquely different image that exhibits the appearance of depth and volume. The result is 3D perception.

A lack of disparity contradicts monocular cues

In a 2D movie, each of our eyes receives precisely identical images from the movie screen with no horizontal displacement. When the two sets of identical images reach the visual cortex of our brain, they are directed primarily to neurons tuned to zero disparity. There, they are “fused” together into a single image that is exactly identical to each of the two images that originally comprised it. Other than the existence of monocular stereo cues, mentioned above, the audience does not perceive depth and volume due to the absence of disparity. This lack of disparity in a 2D movie creates a cue conflict situation that prevents us from being fully immersed in the story in the same manner as a 3D movie. The monocular cues in a 2D movie are telling us there is depth, but the lack of disparity contradicts the monocular cues—forcing the brain to try and reconcile the conflict.

The absence of disparity “pushes” the screen away from us

For some monocular cues, like motion and occlusion, there is a further complication that acts to distance us from the movie screen. As mentioned above, in a 2D movie the relative motion of objects on the screen (one 2D object moving relative to and possibly occluding another 2D object) can indicate depth in the absence of disparity. However, in the real world when there is this kind of relative motion in the absence of disparity, our brains normally interpret that information as indicating that the objects are far away. That’s because disparity in human vision drops off significantly at distances of several hundred yards.  You can prove this if you try the thumb trick above, alternately closing one eye and then the other, but this time, try the trick looking at a tree that is several hundred yards away.  You’ll see that the tree will not appear to jump from side to side like your thumb did. This is because the perspectives from our two eyes at that distance are identical. Our eyes do not converge on the tree, but rather are set to infinity.  The father away things are from us the less horizontal displacement we experience.  So our brain “comes up with” a solution in which we are literally “distanced” from the action on the screen. This distancing and lack of disparity in a 2D movie renders the spatial areas in front of and behind the theater screen irrelevant to the story and therefore irrelevant to each member of the audience. The storytelling, in its entirety, happens on the white screen at the end of the theater.

Skillful use of disparity in a 3D movie will always enhance the experience

In spite of the contradictions our brain must reconcile, a 2D movie is almost always comfortable to watch and with proper directing, cinematography and storytelling, it can evoke very strong emotions. Indeed, many of us remember the outpouring of emotions while watching the sinking of the Titanic or the building terror of T-Rex in Jurassic Park, as both movie experiences were presented in 2D. However, the influence of disparity on the audience in a 3D movie cannot be trivialized as a fad or unnecessary. Whether a 3D movie is captured with two cameras or converted from 2D-to-3D, disparity that is properly and skillfully stereographed will always enhance the storytelling experience by significantly amplifying the responses of disparity-tuned neurons and therefore more closely simulating reality.

3D a higher resolution medium

In the visual system, the absolute number of neurons activated in the brain is not as significant as the ratio of activation between subpopulations of neurons responding to zero disparity and those responding to a wide spectrum of tuned disparities, including zero.  This is analogous to an audio amplifier where the ratio between subpopulations of audio frequencies can vary and increasing the gain on the system increases the overall fidelity of the audio experience, meaning it becomes closer to the original source in resolution.  In the same manner, the ratio of activated zero disparity and disparity-tuned subpopulations of neurons in a 3D movie can vary, but it’s the intensity of the relative responses of those subpopulations that create a more accurate and higher resolution stereo image.  It appears that our brains interpret this higher resolution information as more closely simulating reality (“fidelity”) and therefore it immerses us in the movie to a greater degree, creating a heightened emotional investment in the story.  This is, of course, an oversimplification of an exquisitely complex process that has evolved from the earliest primates. However, the theater going experience is something we can all relate to and many of us recognize the profound difference between being engaged in a 2D movie versus being immersed in a 3D movie.

3D movies are a uniquely personal experience

Dynamic disparity (i.e. the relative amount of disparity, changes in disparity over time and the rate of disparity change) transforms the movie screen from a projection screen into a window that has both an interior and an exterior. Consequently, the storytelling environment is projected throughout the entire theater, actively becoming a unique part of the personal space of each member of the audience and resulting in the space both in front of and behind the screen becoming an integral part of the story. For example, an object that might fly out of the screen at us, traveling through what is called negative parallax, becomes very personal for each member of the audience because it affects and/or “intrudes” each audience members’ personal space equally.  Surround sound has an analogous effect in that we are “in it” rather than simply listening to stereo audio in front of us.  As a result, unlike the 2D movie experience, a 3D movie is more personal and the group dynamics of the larger audience are no longer in play to the same extent.  In fact, it’s my opinion that 3D glasses actually have the positive effect of helping to separate us from the other audience members, containing the experience as a more personal one for each of us, thereby potentially amplifying the immersive nature of the experience.

The Bottom Line

When we are engaged in a 2D movie, we are doing so as passive observers, watching the story take place “over there”, on the screen in front of us. However, when we are immersed in a 3D movie, we are doing so as active participants and the action can be happening behind and in front of us—as far as infinity. This is where the central difference lies. We cannot discount the uniquely active physical and emotional reactions that we have in the 3D feature film experience which are not experienced in our more passive reception of 2D movies. I believe, this is one of the reasons why we have seen 3D continue to flourish and be adopted so enthusiastically by both national and international movie exhibitors, as well as by the entertainment and consumer electronics industries at large. Consumers and moviegoers are hungry for visuals that offer a significantly greater sense of engagement and for visuals that spark an immersive, emotional response that has the ability to transport them into the heart of the action.  This year, we will have the opportunity to test the differential 2D and 3D theatrical experiences with three iconic movies that were originally released in 2D, being released in 3D. Top Gun has been converted by Legend3D and will be re-released on its 25th anniversary by Paramount on the heels of 3D converted re-releases of Titanic and Star Wars.  I am confident that all three films will be very successful at the box office, as each was already a proven success in 2D. Depending on how skillfully each of these films was converted, now in 3D they will give those who saw them originally in 2D an immersive and refreshingly unique experience; an experience that will hopefully help to solidify 3D as an essential part of filmmaking.

Next, we will continue to look at the neuroscience of 3D movies and further explore the uniquely tuned structures and neurons in the brain that respond selectively to dynamic disparity. We’ll look at concepts of visual processing that remain contentious within the scientific community, involving differential pathways within the brain that signal separately “where something is” and “what something is.” The evolutionary survival value of binocular vision will be discussed in the context of 3D movies and we will touch upon more primitive structures in the brain that are likely triggered by disparity to elicit powerful physical emotions and ‘flight’ or ‘fight’ reactions. For more detailed information and technical reviews on stereopsis, as well as in-depth information on the uniquely tuned disparity neurons in many parts of the brain, please see the following references:

(1) Cumming BG, De Angeles GC. (2001). The Physiology of Stereopsis. Annual Review of Neuroscience. 24, 203-238.

(2) Howard, IP, Rogers, BJ. (1995). Binocular Vision and Stereopsis.  New York: Oxford University Press.

(3) Poggio, GF, Poggio, T. (1984). The Analysis of Stereopsis.  Annual Review of Neuroscience. 7, 379-412.

(4) Born, R., Bradley, D. (2005). Structure and Function of Visual Area MT. Annual Review of Neuroscience. 28, 157-189.

I wish to acknowledge the review and insightful suggestions of my neuroscience colleague, David Heeger, Ph.D., Professor at New York University (http://www.cns.nyu.edu/~david/) where he is a member of the Center for Brain Imaging. The son of Nobel laureate and chemist, Alan J. Heeger, David Heeger is a contemporary neuroscientist who has been at the forefront in the field of functional magnetic resonance imaging (fMRI). Dr. Heeger was the first to bring together two separate and largely unrelated disciplines—cognitive neuroscience and film studies, opening the way for the exciting new interdisciplinary field of “Neurocinematic” Studies.

 

About the author (from his site):

This is the first in a series of blogs by Barry Sandrew, Ph.D., founder and CCO/CTO of Legend3D. Sandrew received his doctorate in neuroscience from State University of New York at Stony Brook in 1977. After winning a two year NIH Fellowship at Columbia University, College of Physicians and Surgeons, he joined the staff at Mass General Hospital and Harvard Medical School as staff Neuroscientist. In 1987, he left his academic and scientific career to found American Film Technologies, and invent the first all-digital colorization process for colorizing black and white feature films. He took that company public while converting hundreds of movies for clients such as Turner, Disney, Warner Bros., Fox, Gaumont, TF1, ABC, and CBS, among many others. Sandrew left colorization to start two new companies, one of which included Lightspan, one of the largest educational software companies in the world. That company went public and was later acquired by Plato Learning. In 2000, Barry then founded Legend Films and began the process of re-inventing colorization and developing a proprietary restoration process. Over the course of the next seven years, Legend Films converted approximately 145 black and white titles to color as well as several TV series. In 2006, he recognized that the future of entertainment would be 3D and, leveraging his proprietary colorization pipeline, he modified the entire R&D direction and business model to embrace 2D-to-3D conversion, changing the name of the company to Legend3D. Today, as the most innovative company in the conversion industry, Legend3D continues to lead the field, delivering the highest quality product with the fastest turn around time, at the most competitive pricing.


Laser Projection Group Introduction

The Laser Illuminated Projector Association (LIPA) has released a pdf that outlines their purpose. Generally speaking, there are many international rules that require laser-based equipment to go through regulatory agencies which might make sense for other products (which might use focused light in the output), but don’t make sense for laser-based projectors (which use a diffuse light that substitutes for the xenon bulb.)

The pdf is here: Introducing LIPA

The LIPA website with much more information is: LIPA Website

Laser Projection Group Introduction

The Laser Illuminated Projector Association (LIPA) has released a pdf that outlines their purpose. Generally speaking, there are many international rules that require laser-based equipment to go through regulatory agencies which might make sense for other products (which might use focused light in the output), but don’t make sense for laser-based projectors (which use a diffuse light that substitutes for the xenon bulb.)

The pdf is here: Introducing LIPA

The LIPA website with much more information is: LIPA Website

Laser Projection Group Introduction

The Laser Illuminated Projector Association (LIPA) has released a pdf that outlines their purpose. Generally speaking, there are many international rules that require laser-based equipment to go through regulatory agencies which might make sense for other products (which might use focused light in the output), but don’t make sense for laser-based projectors (which use a diffuse light that substitutes for the xenon bulb.)

The pdf is here: Introducing LIPA

The LIPA website with much more information is: LIPA Website