Do We See the World Like a Movie?

We have the feeling that we experience the world like a continuously sampled data stream. If we perceive multiple objects of events seemingly at the same time, we may actually be multiplexing the several data streams; that is, we take a sample from one data stream, switch to take a sample from the next stream and so on—all on a millisecond time scale.

But another possibility is that we perceive objects and events like a movie frame, where the brain takes working-memory snapshots and plays them in succession. Like still frames in a movie, if played at a high-enough speed, the frames will blend in our mind to give the illusion of continuous monitoring.

In either case, we have to account for working memory. That is, we can only hold a small amount of information in our working memory at any one instant, as in being able to dial a seven-digit phone number you just looked up. In the phone number case, does our brain accumulate and buffer the representation of each integer until reaching the working memory holding capacity and then report it to consciousness as a set? Or is each integer transferred to consciousness and concantenated until the working memory capacity is filled?

A profound recent model of perception addresses the issue of continuous or movie-like perception, but unfortunately, it did not take working memory into consideration.  The model did address the issue of how consciousness integrates the static and dynamic aspects of the object of attention. For example, when viewing a white and moving baseball, consciousness apparently tracks both the static white color and shape of the ball and its movement at the same time. Are these two visual features bundled together and made available to consciousness on a continual basis or as a batch frame?

A related issue is the so-called flash-lag illusion. Displaying a moving object and a stationary light flash at the same time and location creates the illusion that the flash is lagging. There is some debate over why this happens, but it does argue against continuous monitoring of linked objects.

Another phenomenon that argues against continuous monitoring is the “color phi” phenomenon. Here, if two differently colored disks are shown at two locations in rapid succession, a viewer perceives just one disk that moves from the first location to the second, and the color of the first disk changes along the illusory path of movement. But the viewer cannot know in advance what the color and location of the second disk is. The brain must construct that perception after the fact.

Another way of studying fusion phenomenon is to show two different colored disks in rapid succession at the same location. In this case, an initial red disk followed by a green disk will be perceived as only one yellow disk. A viewer cannot consciously recognize the individual properties if there is not enough time between the two disks. This suggests that information is batched processed unconsciously and later made available to conscious awareness. Transcranial magnetic stimulation can disrupt the fusion, but only for about 400 milliseconds after the first stimulus when presumably the processing is unconscious. Since the presentation of the two disks only takes about 60 msecs, it means that unconscious processing of the fusion takes some 340 milliseconds before the results become available for conscious recognition.

Similar fusion can occur with other sense modalities. For example, the “cutaneous rabbit” effect is a somatosensory fusion illusion in which touch stimulation of first the wrist followed quickly by stimulation near the elbow produces the feeling of touch along the nerve pathway between the two points, as if a rabbit was hopping along the nerve. There is no way for conscious mind to know the pathway without the second touch near the elbow actually occurring. Perception of that pathway information is delayed until the information has been processed unconsciously.

So while these examples argue against continuous conscious monitoring of sensation, they don’t really fit well with the movie-frame idea either. We can distinguish two visual stimuli only 3 msecs apart, but a snapshot model that samples stimuli say every 40 msecs would miss the second stimulus. So to reconcile these conflicting possibilities, the authors advance a two-step model in which sensations are processed unconsciously at high speed, but the conscious percept is reported periodically or is read out when unconscious activity reaches a certain threshold or when there is top-down demand.. This fits the data from others that conscious awareness is delayed after the actual sensory event. For visual stimuli, this delay can be as long as 400 msecs.

Here the question of interest is why sensory awareness might require a mixture of continuous monitoring and periodic reporting of immediately prior data segments. Continuous monitoring and processing permits high-temporal resolution. Snapshot reporting conserves neural resources because information accumulates as a batch (a few bytes) before becoming available to consciousness. The really interesting question is what, if anything, happens to that string of movie-like snapshots that are captured in consciousness. How do these frames affect subsequent unconscious processing in the absence of further sensory input? Can unconscious processes capture and operate on the frames of conscious data? Or can successive frames of conscious data be processed batch wise in consciousness? A useful analogy might be whole-word reading. A beginning reader must sound out each letter in a word, which is comparable to the high-resolution time tracking of sensory input. However, whole word reading allows the more efficient capture of meaning because meaning has been batch pre-processed.

How do these ideas fit with the claim of other scholars that consciousness is just an observer witnessing the movie of life as it occurs? However, this assumption ignores the role that consciousness might have in reasoning, making decisions, and issuing commands. I argue this point elsewhere in my books, Mental Biology, and Making a Scientific Case for Conscious Agency and Free Will.

Research claimed as showing that free will is illusory needs reinterpretation in light of this two-step model of perception. Those experiments typically involved asking a subject to make a simple movement, like press a button, whenever they “freely” want to do so. They are to note when they made the decision by looking at a large, high-resolution clock. At the same time, their brain activity is monitored before, during, and after the chain of events.

The first event is the intention to button press. Intention is a conscious event. Was it preceded by unconscious high-resolution processing? If so, what was the need for high resolution? Or maybe this is just the way the brain is built to operate. The button press decision-making is a slow, deliberative process, which perhaps could be handled consciously as a slow progression of successive frames of conscious thought. Critics may say that there is no such thing as conscious processing, but there is no evidence for such conjecture. Once an intent is consciously realized, the subject is now thinking about when to make the press. This decision may well be determined unconsciously, but again there is no need for high temporal resolution. Moreover, there are intervening conscious steps, where the subject may think to himself, “I just did a press. Shouldn’t I wait? Is there any point in making many presses with short intervals? Or with long intervals? Or with some random mixture? Are each of these questions answered by the two-step model of sensory processing?” However the decision developed, corresponding brain electrical activity is available to be measured.

Then, there is the actual button press, the conscious realization that it has occurred, and the conscious registration of the time on the clock when the subject thought the decision to button press was made. Does the two-step model apply here? If so, there has to be a great deal of timing delays between what actually happened consciously in the brain and what the subject eventually realized the conscious thoughts.

The point is that the two-stage model of perception may have profound implications beyond sensation that involve ideation, reasoning, decision-making, and voluntary behavior. I have corresponded with the lead author to verify that I have a correct understanding of the publication. He said that his group does plan to study the implications for working memory and for free will.

Source:

Herzog, M. H., Kammer, Thomas, and Scharnowski, F. (2017). Time slices: What is the duration of a pecept? PLOS. April 12. Hrp://de.doi.org/10.1371/journal.pbio/1002433