Which of the following best illustrates the most predictable effect of schemas on perception?

Annu Rev Psychol. Author manuscript; available in PMC 2021 Dec 17.

Published in final edited form as:

PMCID: PMC8679009

NIHMSID: NIHMS1745764

Abstract

Events make up much of our lived experience, and the perceptual mechanisms that represent events in experience have pervasive effects on action control, language use, and remembering. Event representations in both perception and memory have rich internal structure and connections one to another, and both are heavily informed by knowledge accumulated from previous experiences. Event perception and memory have been identified with specific computational and neural mechanisms, which show protracted development in childhood and are affected by language use, expertise, and brain disorders and injuries. Current theoretical approaches focus on the mechanisms by which events are segmented from ongoing experience, and emphasize the common coding of events for perception, action, and memory. Abetted by developments in eye-tracking, neuroimaging, and computer science, research on event perception and memory is moving from small-scale laboratory analogs to facing the complexity of events in the wild.

Introduction

Event perception is the set of mechanisms by which an organism represents the activity in which it is immersed. Research on event perception has roots in Gestalt psychology (Köhler 1929), in Michotte’s (1946) studies of perceptual causality, in the ecological psychology of human behavior (Barker 1963), and in the social psychology of attribution (Newtson 1976). Miller and Johnson-Laird (1976), in their broad and original overview of language and perception, articulated what would come to be some of the central issues in the field of event cognition: the relationships between objects and events, the fact that events are ephemeral, disappearing even as they occur, and the tight relationships between goals in human action and the parts of events in perception.

In the 1970s and early 1980s there was a burst of interest in the visual perception of simple motion events. In 1980, Gunnar Johansson and his colleagues wrote a chapter for the Annual Review of Psychology titled “Event perception” (Johansson et al. 1980). Around the same time, Gibson’s (1979) volume on ecological perception discussed events in some detail, Cutting (1981) published six tenets for event perception, and the first International Conference on Event Perception took place (Warren & Shaw 1985). Interest then waned until the turn of the century, when a new wave of research took a broader view that encompassed the other sensory modalities, language perception, and the integration of perception and action. Since then, interest in event perception and its interaction with other aspects of psychology has grown steadily.

The mechanisms of event perception are fundamental organizers of conscious experience, of active behavior, of language use, and of the experience of remembering the past. The broader term event cognition refers to event perception and its interfaces with action control, language, and memory. In this review, I will first summarize some of the key empirical phenomena in event cognition, focusing on perception and memory. I will then give an overview of current theoretical models and situate them relative to other ideas in psychology and neuroscience. I will conclude by describing some of the methodological developments that are shaping the field.

The psychological study of event perception is broad, and the study of event memory even broader. This review will necessarily be selective, focusing on the understanding of everyday events across the lifespan.1 Some topics will receive short shrift here but have been recently been reviewed elsewhere (Radvansky & Zacks 2014): I will say little about how event perception and memory are affected by neurological and psychological disorders (for a review, see Zacks & Sargent 2010). I will also say little about the applications of event cognition to diagnosing and improving comprehension and memory (see Richmond et al. 2017, and commentaries). Finally, I will say little about event cognition in text comprehension—though it is important to note that text comprehension research has had a strong influence on current theories and research programs in event cognition (Radvansky & Zacks 2014; Zwaan & Radvansky 1998).

The structure of events in the world and in perception

Everyday experience can be described as chaotic—both in the usual sense of the word and in the technical sense of dynamical systems theory. The specific movements of leaves in the breeze or of people in a crowd are unpredictable and are sensitive to small fluctuations. At the same time, everyday experience can be described as highly structured, regular, and predictable. The larger patterns of blowing leaves or crowds of people are often quite straightforward; people’s actions can be predicted from their goals, from their stable characteristics, and from the situation around them. What is special, perhaps, about human activity is that the variables that organize the activity are removed from the relevant physical variables—they track actions and intentions rather than movements, and track objects rather than shapes or surfaces (Richmond et al. 2017). Human perceptual and conceptual systems are geared to track these regularities even when this requires abstracting considerably from the sensory input.

Continuous experience is segmented into continuous events

One thoroughgoing form of abstraction in perception is segmentation—dividing a continuum into parts. (Segmentation is sometimes described as unitization—grouping some of a continuum into a gestalt. These are two sides of the same coin.) Much research on event segmentation has utilized variants of a procedure in which observers press a key to mark boundaries between events while watching a movie or slide show, or while reading or listening to a story (Newtson 1973). Studies applying this procedure to depictions of everyday activities have revealed some key features of event segmentation behavior, summarized in Table 1. Recent studies of event segmentation have extended the range of presentation media and materials, exploring the segmentation of very simple displays and quite complex ones. On the simple end, when people viewed animations of flashing objects, the introduction of new objects is sufficient to induce event boundaries (Tauzin 2015), and minimal displays of human body motion are sufficient to support robust segmentation, even when the displays were inverted and object information was removed (Hemeren & Thill 2011). On the complex end, a recent study of the relationship between editing techniques in commercial film and segmentation revealed that both changes in the narrated situation (shifts in space, characters, and time) and editing transitions (dissolves, fades, and wipes) are associated with higher levels of segmentation (Cutting 2014).

Table 1:

Characteristics of behavioral event segmentation (see Radvansky & Zacks, esp. Ch. 5).

Most previous studies of visual event segmentation have presented activity from a third-person perspective. Two recent studies have explored the first-person perspective. Magliano and colleagues (2014) presented movies recorded from a first-person video game and found good segmentation agreement, replicating what is seen with other stimuli. Also replicating previous results, event boundaries tended to occur when actors’ goals changed. Swallow and colleagues (2018) directly compared first-person and third-person perspectives, using two cameras, one head-mounted and one tripod-mounted, to record everyday activities. They found strong agreement between segmentation from the two perspectives.

Event representations have internal structure

The events that are distilled from the stream of behavior are not themselves undifferentiated clumps. Rather, they have considerable structure within them. An event unfolds within a spatiotemporal framework and includes entities such as people and objects and the relations amongst them. The representation of this structured segment of experience is called a working event model (Radvansky & Zacks 2014). (I will reserve the term “working event model” for representations that are immediately accessible, whereas the broader term “event model” includes immediately accessible representations and representations in long-term memory.) Working event models are perceptual representations, because they represent currently-unfolding activity. At the same time, they are also working memory representation, because they maintain information throughout the duration of an event. There is strong converging evidence that event models are rapidly established during event comprehension and are supported by specialized neural mechanisms that are distinct from representations of event components (Stawarczyk et al. under review).

For events that are the perception of a human action, the configuration of an agent (who is doing the action), a patient (to whom or what is the action being done), and an action (what action is being performed) is particularly important. Observers can extract this configuration from pictures presented as briefly as 37 ms and subsequently masked (Hafri et al. 2012), and information about the roles of agent and patient is quickly bound to other features of the entities involved (Hafri et al. 2018). Information about an entity’s role (Hafri et al. 2012) and what it might do (Cohn et al. 2017) is often available from its pose: People acting as agents generally have more stretched-out body poses than those on whom they are acting, and the windup to a kick appears quite different from the windup to a throw. There appears to be a regularity to the time course of establishing an event configuration, with agents usually being identified first (Webb et al. 2010). A key feature of agents is that they are animate, and humans appear to have specialized perceptual mechanisms to identify animate agents based on static features and movements. Neurophysiologically, these mechanisms draw on brain areas including ones in the ventral temporal cortex specialized for representing face and body features, on areas in the lateral temporo-occipital cortex specialized for movement processing, and on networks associated with theory of mind (Heberlein 2008). In the perception of comic strips, viewers look at agents longer than they do patients, and are more willing to make predictions about what will happen in a subsequent frame based on pictures of agents than of patients (Cohn & Paczynski 2013). In short, event perception rapidly establishes the configuration of agent, patient, and action, with agents having primacy.

When comprehenders process information about the action itself, they appear to focus on the action’s goal rather than on the particulars of its surface structure. Goal focus is evident early in infancy; studies using imitation and habituation paradigms have shown that infants attend to adults’ goals at the expense of the specifics of their movements (Woodward 2009). By age four, children can better recognize pictures depicting the goal of an action than other pictures, and this advantage is present in adulthood (Papafragou 2010). Adults are better able to detect changes in movie clips when those clips change goal-relevant movement features than when they change features that are not associated with the action’s goal (Loucks & Pechey 2016). This goal processing contributes to observers’ ability to make predictions about upcoming action. For example, observers’ eye gaze anticipates which object is the goal of an actor’s reach, such that the eyes land on the target object well before the actor’s hand arrives; this is true both for observers and for actors themselves (e.g., Eisenberg et al. 2018; Flanagan & Johansson 2003; Hayhoe & Ballard 2005). One important aspect of goals is that they are a component of event structure to which observers and actors have quite different access. Someone performing an action has access to information about their goals and plans that observers cannot access (though such information is by no means without error). Differences in the information available to actors and to observers can lead to differences in event segmentation (Fournier & Gallimore 2013).

Building a working event model requires cognitive work. This can be seen in slower processing of event boundaries in reading (e.g., Pettijohn & Radvansky 2016) and self-paced slide shows (Hard et al. 2011). In language processing, priming studies suggest that reading a word naming one part of an event primes other components of that event. For example, verbs prime the agents, patients, and instruments of their associated events (Ferretti et al. 2001), and are in turn primed by those components (McRae et al. 2005). There is evidence that the effort to build event structures is re-used when possible: Successive sentences sharing a goal-subgoal decomposition were read more quickly than those that did not share goal-subgoal structure (Allen et al. 2010). Event representations can have more or less complex internal structure, and building more complex structures may place unique demands on the system. For example, if one reads that “The chef will weigh the onion. And then she will smell the onion,” the onion is in the same state throughout. However, if one reads that “The chef will chop the onion. And then she will smell the onion,” the onion changes from intact to chopped, requiring the representation of two object states. Solomon and colleagues (2015) compared the processing of these two types of passages during fMRI scanning. They focused on a region of the ventrolateral prefrontal cortex that is selectively active during instances of representational conflict, and found that it was more active when objects needed to be represented in two states. Data from eye-tracking and EEG measures show that during the processing of event descriptions, information from the visual environment is integrated rapidly and continuously (e.g., Altmann & Mirkovic 2009; Knoeferle et al. 2008). This supports the proposal that event comprehension operates on a common representational format that integrates information from multiple sources including language and perception.

Event perception is structured by partonomy and other relations

Event models not only have internal structure; they also form rich connections across events. Partonomic hierarchy (see Table 1) can be thought of as either an aspect of within-event structure or in terms of relationships across events: Viewed “downward” from an event to its sub-events, the parts are aspects of the within-event structure. Viewed “upward” from an event to the larger event of which it is a part, the sibling and parent events are cross-event relations. For goal-directed human activity, the part-subpart structure of events is conditioned strongly on the goal-subgoal structure that generates action. Hierarchical organization is ubiquitous in event segmentation data and in descriptions of events (Zacks et al. 2007). It appears to increase with familiarity with a particular event sequence (Hard et al. 2006) and with domain expertise (Bläsing et al. 2009).

Hierarchically organized event models are more complex than simple strings, but they retain the constraint of being continuous. An activity such as “walking a dog” may consist of sub-parts such as “putting on shoes,” “opening the door,” and “walking up the block,” but all of the temporal parts of the activity occupy a continuous stretch of time. Some activities, however, appear to form discontinuous events—for example “training for a marathon,” or “taking a calculus class.” Kubovy (2015) proposes that the mental representation such discontinuous events constitute projects that cohere despite their temporal discontinuity. Projects in turn are subcomponents of strands, which are large-scale structures such as “sports” or “school,” which are themselves temporally discontinuous. On this view, projects and strands run through lives as do strands through a weaving. An important question for future research is whether such structure is present during online perception of ongoing activity or is a construction established as part of the act of remembering.‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬

Event perception is heavily informed by knowledge

When someone experiences a particular event, they can bring to bear knowledge distilled from their many experiences with varying event types. A longstanding proposal is that event schemas or scripts capture knowledge about the temporal structure of events, entities, roles, and locations (Abelson 1981). How a perceiver understands a particular moment depends on how they can bring such knowledge to bear on the immediate stimulus. By monitoring eye movements while participants watched excerpts from a feature film, Loschky and colleagues (2015) showed that viewers’ looking patterns were more systematic when the clip they were viewing could be placed in a larger context. Knowledge about event categories can affect perception rapidly and automatically, as can be seen in a recent study of change detection (Strickland & Scholl 2015). Viewers were shown clips in which one object either went into another object or behind it. Participants were better able to detect a change an object’s width than its height when the object was shown going into a vertical container; when the object was going behind the other object there was no difference. This suggests that viewers’ visual systems tracked object width more carefully when viewing the event type—vertical containment—for which width is relevant.

The opportunity to deploy knowledge during event comprehension is associated with specific neural mechanisms. A study using multivoxel pattern analysis fMRI identified brain areas whose activity patterns distinguished between schemas for eating at a restaurant or flying from an airport (Baldassano et al. 2018). The analysis targeted areas that had consistently different patterns for the two schemas across multiple movie clips and story excerpts. Schema information was represented in medial components of the default mode network: medial PFC, superior frontal cortex, and medial posterior cortex. Another fMRI study of movie viewing found that presenting clips with a context that allowed for a schema to influence processing was associated with sustained fMRI responses in the inferior frontal cortex, middle temporal gyrus, and angular gyrus, all in the left hemisphere (Keidel et al. 2017). In contrast, when encoding under schema-poor conditions the parahippocampal gyrus and retrosplenial cortex were more strongly activated. One possibility is that the schema representations are stored in the medial network, and the left lateral network plays a role in deploying those representations to guide ongoing processing.

Event perception is predictive and inferential

One thing that knowledge does for event perception is to enable predictions about how events will unfold. If one is familiar with going to rock concerts, one can predict that after the lights go down the band will take the stage. Even in situations where perceivers have little relevant schema knowledge, their brains may make predictions based on more general knowledge such as knowledge about how bodies and objects move, about the relationships between goals and actions, and the like. In scene perception, Intraub and colleagues have shown that viewers quickly fill in information outside the field of vision, leading to systematic errors in picture memory (Intraub 2010). One kind of knowledge that can support strong inferences is knowledge about the causal consequences of actions. For example, kicking a soccer ball leads to the ball moving. When viewers see a sequence of video clips that show a player running up to a ball and then show the ball in flight, they often falsely recognize a clip showing the moment of contact that was not presented, after even quite brief delays or even when asked to monitor for the moment of contact while watching (Papenmeier et al. 2019; Strickland & Keil 2011).

The predictive nature of event perception can be seen in the previously-discussed fact that viewers look ahead to the goals of reaching motions (Eisenberg et al. 2018; Flanagan & Johansson 2003). Viewers can learn to predict even arbitrary, randomly-determined behavior sequences if they are given repeated experience, and this can allow them to look ahead to the targets of actions (Monroy et al. 2018). Predictive processing in event perception also can be seen in evoked EEG responses, particularly the N400 (for a review, see Amoruso et al. 2013; but see also Brouwer et al. 2012). In paradigms that use video editing, picture presentation, or language to tightly control the timing of event stimuli, researchers have shown that unpredicted happenings in event comprehension produce larger N400 responses.

There is a tight temporal relationship between prediction and event segmentation. The more unpredictable activity becomes, the more likely participants are to identify event boundaries (Huff et al. 2014). Making explicit predictions about the near future is more difficult near event boundaries, and attempting to do so is associated with increased fMRI activity in midbrain structures associated with signaling prediction error to the rest of the brain (Zacks et al. 2011). Predictive eye movements are also less prevalent near event boundaries (Eisenberg et al. 2018).

Events structure consciousness

The predictive nature of event perception illustrates a point highlighted by William James (1890): The conscious experience of the present is not an infinitesimal point but has temporal extent. The fact that event segmentation tasks are so easily learned, produce such reliable data, and correspond with ongoing neural events supports the conclusion that conscious experience is segmented into events (Zacks et al. 2007). The time course of other functions provides converging evidence. When participants are asked to monitor a complex situation for a visual target, the presence of an event boundary transiently disrupts performance (Huff et al. 2012). Consistent with this result, mind-wandering while watching a film was less likely at event boundaries than during the middles of events (Faber & D’Mello 2018). These findings suggest that model updating is resource-intensive: at event boundaries, those resources are less available for secondary tasks or off-topic wanderings of the mind.

Summary

In sum, perception is characterized by the segmentation of ongoing experience into meaningful events. Event representations have rich internal structure, and also rich relationships between them. The construction of these relationships is strongly influenced by knowledge, which enables online predictive inference about the unfolding of activity. The construction of working event models and the predictive nature of event perception highlight that the experience of “now” is smeared in time—there is no bright line between now, a moment ago, and a moment hence. This entails that there is no bright line between perceptual systems and memory systems (Christophel et al. 2017). A more fundamental distinction than that between perception and memory is that between the “expanded present” represented by a working model and events outside of that expanded present.

From perception to memory

Rubin and Umanath (2015) define event memory as “the mental construction of a scene, real or imagined, for the past or the future.” In other words, we can distinguish between representations of events that one is currently experiencing and representations of events that are based on other sources; event memories are the latter. One attractive feature of this conception of event memory is its emphasis on the constructive nature of memory. Another attractive feature is its acknowledgement that when one attempts bring an event to mind, the result depends on multiple signals and active strategic mechanisms; event memories are constructions rather than the mere reloading of previous event representations stored immutably in the brain’s vault.

This approach entails that experiencing a current event and remembering a previous event are underwritten by a common representational substrate: a working event model. The difference is that in perception we focus on how one’s current working model is populated by currently-available perceptual information, whereas in remembering we focus on how one’s event model is populated by retrieval of information from long-term memory. This analysis offers a novel take on the relationship between perception, working memory, and long-term memory.

Working memory representations are updated at event boundaries

Most theories of human memory distinguish between short-term (or primary or working) memory and long-term memory. An outstanding puzzle has been: what makes a memory long-term? The data presented in the previous section, in supporting the proposal that the brain maintains working event models and updates them periodically, suggests that in many cases the boundary between working memory and long-term memory may be the end of one event and the beginning of the next. Striking support for this proposal has come from a series of studies by Radvansky and colleagues on the memorial consequences of walking through doorways. In the initial experiments (Radvansky & Copeland 2006a; Radvansky et al. 2010), participants navigated a virtual reality environment in which they picked up objects and put them in a backpack, and later put down the objects. From time to time, they were probed to report which object was currently in their backpack. Controlling for distance traveled and time elapsed, participants were slower to respond if they had walked through a doorway after picking up an object. This effect has been found not just in virtual reality but also in real rooms (Radvansky et al. 2011) and in imagined ones (Lawrence & Peterson 2016). It does not appear to reflect simply a temporary disruption due to crossing the doorway because it persists when a constant delay is added to the retention interval (Pettijohn & Radvansky 2015); nor does it seem to be due to the dissimilarity in context between the encoding and retrieval conditions because it remains when participants return to the original room (Radvansky et al. 2011). These interactive paradigms converge with results from studies of narrative reading (e.g., Rinck & Bower 2000), and with studies of memory for recently-seen objects in movies (Swallow et al. 2009), which show that after a shift in location or of narrative time, memory retrieval is often less efficient.

Updating event models at boundaries would seem to be a resource-intensive operation. It may include comparing the contents of the old model with new perceptual information. Evidence supporting this idea comes from a study showing that, while viewing films, changes in actors’ clothing were noticed better at event boundaries than event middles (Baker & Levin 2015). If information that is present at event boundaries receives special processing, this may have consequences for later retrieval. Using fMRI, Swallow and colleagues (2011) showed that successful retrieval of visual object information encoded during an event boundary was associated with selective activity in the hippocampus and parahippocampal gyrus—structures associated with long-term memory in many studies in humans and other species. The delay in these studies was only 5 s, which supports the notion that what makes a memory long term may be whether it requires reaching back across an event boundary.

Thus, an event boundary may involve the segregation of the immediate present—subserved by working event models—from events in long-term memory. A natural implication of the hierarchical structure of activity, described in the previous section, is that event model updating happens on multiple timescales; some updates may only affect finer-grained event models, whereas others additionally update coarser-grained event models. If so, then updating at coarser-scale event boundaries should have larger effects on memory retrieval and brain activity. This hypothesis has yet to be tested.

Event boundaries also may condition what sorts of representations are formed in long-term memory. Recent studies of memory retrieval with longer delays have provided evidence that the experience of event structure in the present leaves heavy footprints on subsequent memory.

The structure of events in experience is mirrored in long-term memory

One idea that has become dominant in the psychology of human memory holds that the experience of remembering is fundamentally one of intentionally searching for information in the mind. While there is no doubt that people sometimes go into a mode of deliberate memory search (Tulving 1983), it may be that a more basic and more frequent mode of experiencing event memories results from involuntary associative cuing (Berntsen 2010). Features such as locations, people, or objects may quickly and automatically bring to mind events that are associated with those features. For example, a child, upon arriving in a classroom on Tuesday morning, might retrieve events of Monday’s class in that room, and upon encountering a friend one might retrieve features of the last conversation had with them. Such associative retrieval is adaptive: If a new event induces memories of similar previous events, those previous events are likely to provide valid predictive information about how the new event will proceed. But what if things change? What if there is a substitute teacher and as a result the events of yesterday’s lesson do not predict what will happen today? In such cases, event memory may impair rather than facilitate comprehension. However, if the child can register the discrepancy as part of the memory for Tuesday, this can result in a highly effective form of memory that embeds what happened on both days, the temporal relationship between them, and something about the variability associated with that context. A recent study of memory for changes in events provides evidence for such mechanisms (Wahlheim & Zacks in press).

Event memory is structured by the same dimensions as perception

What is the relationship between event structure in perception and in memory? There is strong evidence that the segments that are identified during event perception correspond to the representational units in subsequent memory. First, the boundaries themselves are remembered exceptionally well. For example, Huff and colleagues (2014) showed viewers episodes of a situation comedy, asked them to segment them, and tested their memory later. Event boundaries tended to occur at points in time when many features were changing, and those points were remembered better. Ezzyat and Davachi (2011) used a narrative priming paradigm to show that sentences cued memory retrieval of subsequent sentences more strongly when the two belonged to the same narrative event than when they did not, and that the binding of sentences within an event was associated with selective activity in brain areas including the ventromedial prefrontal cortex. Pettijohn and colleagues (2016) showed that breaking up the experience of studying a word list into two events (by walking through a doorway) increased the total number of studied words that were remembered later. They interpreted this manipulation as decreasing the degree to which words from the first half of the study experience interfered with those from the second half, and vice versa.

Such data support the existence of integrated event models in memory. However, event memory also depends on other representations—for example, representations of people and objects. Furthermore, even bound event models can be confused with similar events during memory retrieval. When one attempts to remember a past event, one constructs a working model that depends on these confusable event representations and other long-term memory representations, interference between similar representations can lead to confusions. For example, when people see a number of actors involved in a number of actions, confusions about which person performed which action are common (Earles et al. 2008; Kersten et al. 2013).

Just as there are structural relationships across events in perception, there are relationships across events in memory. One of the strongest organizers of relationships in event memory appears to be causal connection (Radvansky 2012). In fact, the organization of memory by causal relations is likely stronger in memory than in perception, because post-encoding processes elaborate causal relations. When people read narratives of event sequences, components with more causal connections are remembered better (Trabasso & Stein 1997). When people are asked to recall multiple events from distant periods in their lives, events that are causally related tend to cue each other (Brown 2005).

Spatial location is another powerful organizer of events in memory. It is much easier to remember the association between multiple objects and a single location than the association between multiple locations and a single object (e.g., Radvansky et al. 2017). For example, it is easier to remember that a potted plant, an ATM, and a poster are in a lobby than it is to remember that there is a potted plant in a lobby, a library, and a café. This suggests that a collection of objects in a location can be represented in a single event model, whereas representing the same object in multiple locations requires multiple event models.

Time is also a potential organizer of events in memory. Just as it is easier to associate multiple objects with a single location than multiple locations with a single object, it is easier to associate multiple objects with a single time period than multiple time periods with a single object (Radvansky et al. 1998). Hierarchical organization, prevalent in perception, is also prevalent in autobiographical memory. The same autobiographical cueing procedure that reveals causal relations in memory also shows clustering of events by membership in larger events (Brown 2005). If events are organized into part-subpart hierarchies in memory, this entails representing a certain amount of information about temporal distance, because two events that are both part of the same larger event will generally be closer in time than two events that are parts of different larger events. In addition, causal relations can scaffold memory for temporal order because causes precede their effects. However, some aspects of temporal organization, such as the order of events within a larger unit, can be quite weak in memory. For example, Wyer and Bodenhausen (1985) found that people recalling stories showed good memory for the order of actions within an event (possibly because these tend to be causally linked) but poor memory for the larger order of events.

For both space and time, there are a range of scales on which things can be organized. Temporal and spatial scales that have natural organization are probably represented—or reconstructed—better than those that are more arbitrarily organized, because those natural organizations facilitate prediction. For example, the arrangement of objects within a room is systematically related to the locations of doors and windows, and to the actions that take place within the room. However, the order of rooms on a hallway or of buildings on a street is much less consistently tied to objects and actions. This may be why, after watching many episodes of a television series, people have relatively good memory for the layout of rooms but poor memory for the spatial relations amongst rooms (Levin 2010). Similarly, the order of actions in making a sandwich is structured by causal and conventional relations that can facilitate predictions, but the order of which larger activities may precede or follow sandwich-making is less systematic (Wyer & Bodenhausen 1985). Gravina and Sederberg (2017) suggest that these relationships of systematicity and the predictions they allow for account for the temporal and spatial similarity gradients seen in memory representations in the medial temporal lobes (Nielson et al. 2015).

Memory for the duration of events is often important for planning future activities—and it is fascinatingly affected by features other than actual duration. In spatial navigation, routes that have more turns are remembered as being longer in space and also are mentally replayed more slowly (Bonasia et al. 2016). This appears to be an instance of a more general phenomenon: Intervals more filled with “stuff” are remembered as having taken longer when looking back on them. Wang and Gennari (2019) showed viewers animations, asked them to describe the animations, and then asked them to recall their duration. They found that those animations which elicited more extensive descriptions were remembered as having taken longer, controlling for actual duration. Similarly, routine events are usually described less richly than unusual ones, and such events are remembered as having been shorter (Avni-Babad & Ritov 2003). What is the “stuff” that goes into remembering duration? At least one source appears to be the number of sub-events in an interval. Jeunehomme and D’Argembeau (2018) asked participants to perform activities of daily living while wearing cameras, to segment their recordings, and then to estimate the durations of particular activities. They found that those activities which were segmented into more events were remembered as longer, controlling for actual duration. Finally, Bangert and colleagues (in press) observed a similar effect of event boundaries on judgments of the durations of intervals within movies of everyday activity.

Event memory, like perception, is heavily informed by knowledge

The fact that relationships within and amongst events are systematic means that they can be the subject of knowledge. We saw previously that knowledge affects event perception. The effects of knowledge on event memory are even more striking. Inspired by the influence of scripts and event schemas on memory for narrative text, Brewer and colleagues conducted several studies showing that memory for filmed events is influenced by knowledge about event categories (e.g., Brewer & Dupree 1983). Information that is congruent with an event schema is often remembered better than information that is irrelevant, in part because the schema can act as a bias during recall. For example, if one were asked to remember whether at a wedding the couple recited vows, one could respond based on a schema for weddings in general in addition to representations specific to that event. However, suppose the couple did not recite their vows, but instead had them tattooed onto their forearms. This might also be memorable, and indeed under many circumstances such discrepant information is also well remembered compared to schema-unrelated information—for example, the kind of car one took to the wedding as a guest. In a recent study, Bonasia and colleagues (2018) found that movie clips that were either congruent with a schema or discrepant were both remembered well compared to clips that were not strongly related to a script. Remembering schema-congruent clips led to more medial prefrontal fMRI activation, whereas remembering discrepant clips led to more medial temporal activation. This suggests that the influence of schemas on event memory may be mediated in part by the medial prefrontal cortex, and that the medial temporal lobe system is particularly taxed when memories need to bind arbitrary relations. In most studies of event comprehension and knowledge, the schemas in question are assumed to be well-learned. (We will see exceptions to this in the study of child development.) However, recent evidence suggests that event knowledge can be acquired from a few experiences (MacLean et al. 2018). In this study, participants tasted various foods in the laboratory, with a different experimenter conducting each tasting. After just a few of these events, a new food tasting experience in which the experimenter’s behavior deviated from the others was better remembered.

Remembering events is constructive and destructive

If event memory is the mental construction of a scene (Rubin & Umanath 2015), this suggests that the act of searching one’s memory for a particular episode is not so different from imagining a novel event or thinking about a counterfactual event. Indeed, neuroimaging and neuropsychological data strongly support a common mechanism for constructing events from memory, imagination, and reasoning (Addis et al. 2007; Schacter et al. 2012). A reasonable proposal is that all of these tasks depend on constructing an event model and their common neural correlates reflect either the representational medium of event models, the systems that are needed to construct such representations, or both. Barbey and colleagues (2009) used fMRI to examine the mechanisms of counterfactual reasoning and found that multiple dimensions of events were represented in the spatial locus of activation in the medial prefrontal cortex. This is consistent with the results of Baldassano et al. (2018), who found schema-specific patterns in this area during movie viewing. The coupling of this region with other default network regions during event encoding is associated with better subsequent memory of narrative sequences (Simony et al. 2016); This suggests that the common activations seen in event perception, event imagining, and event memory tasks reflect a functional, causal relationship.

The view of event memories as constructive also entails that remembering is reconstructive and destructive. That is, when one constructs an event memory, the operations of retrieval and the activation of knowledge during that retrieval affects one’s future remembering. For events, this can sometimes produce straightforward benefits to subsequent memory: For example, after viewing movies of everyday events, being reminded of them with pictures or titles reduces forgetting. However the same reminding also can impair memory for other events that were not reminded (Koutstaal et al. 1999). Eye witness memory studies show how a given retrieval attempt can produce negative effects on subsequent memory: For example, retrieving an event while trying to decide whether a mug shot matches the perpetrator of the event can lead the person pictured to be falsely incorporated into subsequent memories for the event (e.g., Kersten & Earles 2017). Post-encoding operations may affect not just the contents of long term memory event models, but also how activity is segmented in long term memory (Hohman et al. 2013).

Summary

In sum, although there is no bright line between perception and memory, there is good evidence for a distinction between information that is maintained in one’s current working event models and event information that is represented in other neural systems. Event memory can be conceived as the construction of a working model based on that other information—including episode-specific representations, knowledge, and new information generated during the construction process. The act of constructing a working event model creates new representations in these memory systems, which side-affect subsequent remembering. These operations have specialized neural mechanisms. The PM/AT framework (Ranganath & Ritchey 2012) summarizes and integrates current knowledge about these mechanisms. It proposes that one brain network, including the lateral temporal cortex and perirhinal cortex, supports the use of object knowledge and perceptual features in memory formation. Another network, including the medial prefrontal cortex, retrosplenial cortex, and parahippocampal cortex supports the organization of entities within a spatial framework into an event model, drawing on event schemas. This account fits well with the neurophysiological data reviewed here.

Group and individual differences in event cognition

There are substantial group and individual differences in event perception and event memory, which can inform theories of their mechanisms (Zacks & Sargent 2010). Here, I will focus on group-level effects of healthy aging and early Alzheimer’s disease, and of language, and on how individual differences in event perception relate to differences in memory.

The experience of events changes through adulthood, and is impaired in Alzheimer’s disease

We saw previously that event segmentation is characterized by agreement across observers and by hierarchical organization (“Segmentation of continuous experience into events”). Both of these features can be used to develop measures of group and individual differences. When asked to segment movies of everyday activities, older adults usually show lower segmentation agreement than younger adults (e.g., Kurby & Zacks 2018; but see Sargent et al. 2013). Older adults also show less hierarchical organization (Kurby & Zacks 2011). Interestingly, a recent study of the segmentation of filmed everyday events found no difference in segmentation agreement or alignment between young and older adults (Kurby & Zacks 2018). One possibility is that age differences in event segmentation may be reduced when older adults can use knowledge to construct richer event representations (Radvansky & Dijkstra 2007). Consistent with this proposal older adults, like younger adults, show robust updating at situational changes in narrative memory updating paradigms (Radvansky et al. 2003).

Early Alzheimer’s disease reflects a divergence from the path of healthy aging that is characterized by impairments in memory and thinking, which can initially be subtle but increase in severity and scope with disease progression. These include deficits in event segmentation and memory (e.g., Bailey et al. 2013b). They also include deficits in the ability to perform everyday activities, in and out of the laboratory (e.g., Giovannetti et al. 2008; Gold et al. 2015). One possibility is that these deficits result from disruption of event knowledge, or of the ability to use that knowledge effectively.

Language affects some aspects of event perception and memory—especially when using language

The language one speaks is a group difference that is of particular interest for perception and memory in general due to debates over whether and how language shapes thought. The effect of language on event perception and memory has attracted sustained attention because different languages represent aspects of event structure differently. For example, languages including English and Arabic encode in the form of their verbs whether an activity is viewed as ongoing over time (“is walking”) or as a whole (“walks”), whereas German does not. Because of this grammatical difference, German speakers are more likely to explicitly describe the locations of the endpoints of actions than are speakers of English or Arabic. (This is because whole actions tend to make less sense without their endpoints, whereas ongoing actions make sense with or without endpoints; for example “Stacey walks” sounds a little odd without a destination specified, whereas “Stacey is walking” sounds fine.) Corresponding with this linguistic difference, German speakers look more at action endpoints than do Arabic speakers (Flecken et al. 2014), and show larger P3 EEG responses to unexpected action endpoints than do English speakers (Flecken et al. 2015), even in tasks not involving language. However, such differences are not always observed. Papafragou and colleagues conducted several tests comparing Greek, which highlights path information more, to English, which highlights path information less, in designs similar to those of Flecken and colleagues. In these studies, no effects of language were found unless language was being used (Papafragou 2010; Papafragou et al. 2008; see also Gennari et al. 2002).

In addition to affecting attentional selection, language can affect segmentation. French verbs are more likely to represent motion paths more than do German verbs. For example, in a situation where a French speaker might say “La voiture a traversé le pont” (roughly, “The car crossed the bridge”), a German speaker would more likely say “Das Auto fuhr über die Brücke” (“The car drove over the bridge”). Whereas French verbs more often describe path (traverse/cross), German verbs more often describe manner of motion (fuhr/drive), conveying path information in a preposition (über/over). Given that information in a verb is obligatory, one might expect that changes in path would be more likely to lead to event model updating for speakers of French than of German; this has indeed been observed (Gerwien & von Stutterheim 2018).

The potential for language to influence event segmentation also can be seen in the gestures that speakers use to talk about events. For example, Avatime is a language that can use serial verbs to package a string of smaller actions into one event. For example, the sentence “lɛ̌ be-dzì e-mu-i” (roughly, “then return ascend”) means “Then they climbed up again.” When Avatime speakers use this construction, if they gesture while uttering a serial verb string the gesture spans the entire verb string, suggesting they conceive of it as a single unit (Defina 2016).

There are strong relationships between individual differences in event perception and in event memory

In addition to evidence for group differences, there is evidence for substantial individual differences in event perception—and individual differences in perception predict individual differences in memory. In a large-scale study of event segmentation and memory across the lifespan, people with higher segmentation agreement had better subsequent recall and recognition for events (Sargent et al. 2013). This held after controlling for individual differences in processing speed, working memory, crystallized knowledge, and laboratory episodic memory. Event knowledge also was a significant independent predictor of event memory. Within older adults, including those with early Alzheimer’s disease, better segmentation is associated with better event memory and with better ability to perform everyday actions (Bailey et al. 2013a,b; Kurby & Zacks 2011). Experimental interventions on event segmentation suggest that the relationship between event perception and event memory is causal: Interventions that improve event segmentation by instruction or by editing event stimuli improve subsequent event memory (Flores et al. 2017; Gold et al. 2016).

In short, event perception and event memory vary across individuals and groups. Moreover, differences in these two abilities are related to each other and also to other domains including everyday action performance. Intervening to improve event segmentation improves event memory, which is consistent with the previously-discussed evidence that segmentation during perception shapes the representational units of event memory. Differences between young and older adults highlight the importance of adult development in event perception and memory. But of course, these abilities do not emerge fully formed in 18-year-olds. How do they develop in childhood?

The development of event cognition reflects growth in knowledge, language use, and deliberate rehearsal

For many years, Piagetian accounts dominated the study of event understanding and event memory. These held that infants possess little of the conceptual structure to support adult-like performance, and that this conceptual structure emerges incrementally over years of experience in the world. Modern views, however, identify early competence in areas of both event perception and event memory, as well as extended growth. One important aspect of growth in perception and memory, which was noted by early theorists as well as current ones, is growth in schematic knowledge about events (Bauer 2006).

Psychology’s picture of the development of event perception is much hazier than its picture of the development of event memory. There is substantial evidence for early competence in infancy, but little data on how infant abilities develop into those of adults. Well within the first year of life, infants can individuate actions within a continuous stream of behavior (Sharon & Wynn 1998; Wynn 1996), and show evidence of segmenting activity at points that are identified by adults as event boundaries (Baldwin et al. 2001; Hespos et al. 2009, 2010; Saylor et al. 2007). Infants are sensitive to causal interactions such as in the Michotte launching effect (Cohen & Amsel 1998), and can use experiences with recent events (Nakano & Kitazawa 2017) and the statistics of extended experience with event types (Monroy et al. 2017) to guide their ongoing processing of current events. However, the infant studies have used almost completely different measures of event perception than have adult studies, and little is known about the developmental trajectory of event perception between infancy and young adulthood.

Much more is known about the development of event memory. This is true, in part, because of a strong applied interest in what children can remember for the sake of legal testimony. Whereas older views held that children lacked the ability to form event memories before the advent of language, a new generation of research has found evidence for early competence in event memory formation and continuity of development through the early phases of language acquisition (Bauer 2006). By 18–24 months, toddlers can recall elements of an otherwise-forgotten event if they are cued with features of that event (Sheffield & Hudson 1994); this indicates that the elements are bound into a coherent whole. At the same age, toddlers show better memory for event boundaries than for event middles, and show selective memory impairment from occluding event boundaries during encoding (Sonne et al. 2016, 2017). From ages 2 to 10, the development of event memory is linked to growth in event knowledge (Fivush 1997; Hudson et al. 1992). Young children tend to misremember events as having conformed to the schemas they have learned, but by 7 or 8 years, children have more ability to recall deviations from schemas and to recall schema-irrelevant details, especially with environmental support (Brown & Pipe 2003). One important aspect of children’s event knowledge is goal relations, which have strong early and continuing effects on event memory (Loucks et al. 2016). By age 10, children’s event memory looks qualitatively like that of adults, though encoding efficiency and completeness continue to increase with age. Together, the limited perceptual data and more extensive memory data indicate that very young children make use of event models that are in some ways quite adult-like, but that these become more elaborated over development, in part due to the development of knowledge structures.

Summary

In short, contrary to earlier views, there is good reason to think that infants construct event representations that are similar in form and content to those of adults, though more limited in many aspects. The development of event memory is conditioned strongly on the development of knowledge about event classes. An important issue for future research is the role of knowledge and other factors in the development of event perception.

Theoretical approaches to event cognition

In previous generations, theories of event cognition tended to deal with perception (Gibson 1979; Johansson et al. 1980; Michotte 1946), memory and inference (Abelson 1981), or action control (Miller et al. 1960). The memory and action control theories emphasized structure in mental representations, whereas the perceptual theories emphasized structure in the environment. In contrast, current theories tend to bridge at least two of these domains, and to consider structure both in the mind and in the world.

Common, multimodal coding of events and actions

One problem taken on by current theories is the relationship between people’s roles as perceivers and as actors in the stream of events. One such theory, which has been highly influential, is the Theory of Event Coding (TEC) proposed by Hommel and colleagues (2001). TEC is a qualitative account of how perception and action control are integrated in events on a short time-scale, from tens of milliseconds to a few seconds. It can be described in terms of a set of proposals about how immediate events are represented, listed in Table 2. The third and fourth proposals state that event representations are composites of feature codes. The range of possible codes is determined by previous experience encoded as knowledge. For example, most animals will have access to codes such as “red” and “short” to characterize a cup, but as one interacts with a cup one might add codes for its previous location and the level of liquid inside. The final proposal governs how event representations can be shaped by current interests: If one intends to pick up a cup to drink from it, shape features will be highly weighted; if one is selecting a cup to purchase in a gift shop, features related to attractiveness and desirability will receive more weight.

Table 2:

A key set of findings accounted for by TEC involves situations in which actions or intentions interact with perception. Planned actions can affect how visual or auditory stimuli are weighted, and irrelevant features of visual or auditory stimuli can affect action execution. TEC’s proposal that feature codes are first activated and then bound has the advantage that it can account for both facilitation from overlapping features and also for interference. However, in any given situation it is often not clear whether to predict facilitation or interference. As evidence has accumulated, it appears that additional mechanisms are needed to account for the complex patterns of facilitation and interference that are observed (Zwickel & Prinz 2012).

One of the most grounded ways to build a model of cognitive representation for perception and action is to explicitly build a controller for an agent. The REtrospective and PRospective Inference SchEme (REPRISE) model (Butz et al. 2018) does just that. REPRISE is a recurrent neural network that uses a bank of contextual neurons to represent which of a number of potential event states the network is currently experiencing, and to bias processing in the rest of the network based on the currently-inferred event state. The model identifies which state it is in by looking backwards at the recent dynamics of the system, and uses the current state and the values of the physical variables describing the environment to predict the consequences of its actions. Butz and colleagues applied REPRISE to a simulated environment in which the model learned to control a set of vehicles, and the vehicle it was driving could be changed without warning. Thus, at any given time, the model had to simultaneously infer which vehicle’s dynamics were in play and had to drive the vehicle. It could learn to identify vehicle changes (event boundaries) and to update its contextual neurons appropriately, improving driving performance.

Event segmentation and the formation of event memories

The application of REPRISE to detecting state changes illustrates the importance of modeling temporal structure in activity. Event segmentation theory (EST; Zacks et al. 2007) proposes that event segmentation occurs as a side-effect of constructing working event models that improve perceptual prediction. EST starts from a perceptual processing stream that takes a representation of the current state of the world (which may include perceptual information, language, and other sources) and produces predictions about what will happen a short time in the future. Such predictive processing is assumed to be an ongoing component of comprehension, which facilitates more effective and timely behavior. This predictive processing stream is modulated by a working event model that maintains a stable representation of “what is happening now.” The architecture retains its current working model as long as prediction error is low, and updates its working model when prediction error spikes. A simplified version of EST was implemented as a gated recurrent network by Reynolds and colleagues (2007). The model was trained using back-propagation on a series of inputs representing the position of an actor’s body over time while completing a sequence of goal-directed actions. The model was able to learn to predict the actor’s motions and to use spikes in prediction error to update its working models, improving prediction performance. EST proposes that the error-based updating mechanism could be implemented by phasic activity of the midbrain dopamine system, a system with broad projections throughout the cortex via direct connections to the prefrontal cortex and via the basal ganglia. This proposal has received support from neuroimaging (Zacks et al. 2011). Another neurophysiological hypothesis was that event model maintenance depended heavily on the lateral prefrontal cortex. This proposal has fared less well empirically; instead, components of the brain’s default network including the medial prefrontal and posterior cortex and parts of the lateral inferior parietal cortex are shaping up as better candidates (Stawarczyk et al. under review).

The event horizon model of Radvansky and colleagues incorporates EST as a front-end mechanism to account for a number of features in long-term memory for events (Radvansky 2012; Radvansky & Zacks 2014; see Table 3). The model provides an integrated descriptive account of many of the features of event memory described in “Event memory is structured by the same dimensions as perception” above. Segmentation during perception leads to units in event memory. The occurrence of similar event features in multiple events leads to better memory for those features, but to worse memory for identifying which particular features occurred in a specific event. Across events, causal relations are a major organizing feature that determines the likelihood of remembering a particular event and which other events that event will bring to mind. An important question left unresolved by EST and the event horizon model is how event structure evolves with forgetting, repeated retrieval, and subsequent experience. A recent study by Hohman and colleagues (2013) found that, when pressed, people retrieving information from autobiographical memory could retrieve additional sub-events from a remembered event and that doing so extended the remembered event’s boundaries. One possibility is that the temporal continuum that is sliced by perception can be resliced by the actions of memory retrieval. Another possibility is that such malleability of event boundaries in memory does not reflect the reslicing of a stored continuous experience, but rather a process in which the originally stored representations are discrete, and new discrete event representations are constructed through the act of retrieval.

Table 3:

The segmentation of events is one aspect of event structure; other aspects include the order of events, their hierarchical organization, and the roles that actors and objects play in an activity. All of these, we saw, are influenced by knowledge about different event types. Elman and McRae (in press) have recently proposed a model of the acquisition of event knowledge that takes on all of these aspects of structure. The model is a recurrent connectionist network that codes for information about agents, actions, patients, instruments, locations, and recipients. The model learns associations amongst these elements within and across timepoints. Both kinds of associations are subject to variability in natural experience; for example, when changing a tire, one may loosen the lug nuts before or after jacking up the car; and might pull the car into a driveway or not. Trained on a corpus of event sequences from people’s descriptions, the model learns the most typical patterns and also information about the alternatives and the degree of variability. It is able to use this information to make predictions about what will happen next in a novel sequence—a key feature of comprehension.

Perhaps the broadest-scope model of event cognition to date is the structured event memory (SEM) model (Franklin et al. 2019). SEM uses a hybrid architecture in which the dynamics within events are represented using recurrent neural networks and relations across events are modeled as a partially observable discrete process that generates a sequence of persisting states which correspond to event types. The model learns a library of possible event dynamics, stored in the weights of a set of recurrent networks, and learns at each point in time to apply a previously-stored weight set or to create a new one if none of the existing weight sets fit. The model segments ongoing activity into events, forms online event representations, and can retrieve event information later. It provides a novel account of the learning of event schemas: When the model first creates a new neural network to represent an event that is not well fit by existing learned dynamics, this representation looks like an episodic memory. As it encounters more instances that activate the same dynamics, the representation grows more and more schematic. Like the Reynolds et al. (2007) model, SEM can segment ongoing activity based on a dynamic video (in this case, full motion video rather than a body-tracking recording). It is sensitive to the statistical structure of activity subunits in a way similar to human observers. It shows memory updating at event boundaries in an adaptation of the “walking through doorways” paradigm (Radvansky & Copeland 2006b), the long-term memory benefit of splitting encoding across two events (Pettijohn et al. 2016), and stronger memory cueing within an event than across events (Ezzyat & Davachi 2011).

The hierarchical organization of activity in time is a central feature to be accounted for. In the models described thus far, hierarchical organization is addressed only implicitly. For example, in EST, segmentation on different timescales can be achieved by varying the time constant of integration of the prediction error signal: Longer time constants do more smoothing on the error signal and produce less frequent event model updating in response to large, slow error spikes; shorter time constants produce more frequent event model updating in response to quicker error spikes that need not be as large. The hierarchical process memory theory (Hasson et al. 2015) explicitly addresses how different time scales of representation might be implemented in the brain. Rather than view memory as a specialized system or set of systems, the authors note that all of the brain’s dynamical systems have characteristic timescales. In the retina and the cochlea, neural activity tracks the current state of the world closely, and only retains the influence of previous states of the world for tens or hundreds of milliseconds. This fast temporal fading carries through the earliest stages of cortical processing, but as sensory information is processed through successive cortical stages temporal dependencies grow longer and longer. Complementary patterns of time-dependence are seen in motor control: Cortical systems that are close to synapsing on muscles show fast-fading temporal integration, whereas earlier stages of motor control show longer timescales. Brain areas that show the longest temporal dependencies tend to be multimodal and tend to represent both perceptual and action-related features of activities; these areas overlap with the default network, and include the angular gyrus and areas in the medial posterior and medial frontal cortex.

Summary

In sum, recent theoretical approaches to event cognition have focused on two topics: the multimodal integration of representations for perception and for action, and the processing of temporal structure in events. Most current theories integrate computational descriptions with neurophysiological descriptions in accounting for behavior with respect to events.

New methodological developments shaping event cognition

A key to recent progress in event cognition has been the deployment of new methodological tools and approaches. One such development is eye-tracking techniques for working with complex, dynamic stimuli. Whereas for decades eye-tracking has made important contributions to reading, scene perception, and attention, improvements in hardware and software have now made it tractable to present naturalistic movies or live interactive experiences, record eye movements, and calculate features including looking to target objects (Hayhoe & Ballard 2005) and gaze synchronization across viewers (Loschky et al. 2015). These improvements have also made it tractable to track the characteristics of saccades over time in conjunction with stimulus features (Eisenberg et al. 2018). Eye tracking is particularly promising for studies of young children and others who are unable to perform complex tasks under instruction.

The current resurgence of interest in event perception has corresponded with the rise of fMRI as a means to study brain activity. Many of the key contributions of this method have already been discussed: fMRI initially provided a noninvasive means to test hypotheses about the operations of segmentation during ongoing visual event and narrative event comprehension (Speer et al. 2007; Whitney et al. 2009; Zacks et al. 2001). More recently, multivariate pattern analysis, intersubject synchronization measures, and functional connectivity analysis have allowed researchers to test sophisticated new hypotheses (Baldassano et al. 2017, 2018; Hasson et al. 2008). These methods have provided evidence that the brain segments ongoing experience at a range of temporal scales, with later perceptual processing stages specializing in longer time-scales. They also have shown that phasic activity at the boundaries of events is predictive of online memory updating and of the organization of subsequent long term memory (Baldassano et al. 2017; Ben-Yakov & Dudai 2011; Ben-Yakov et al. 2013, 2014; DuBrow & Davachi 2016; Ezzyat & Davachi 2011; Hsieh et al. 2014). New developments in EEG theory and methods also have made key contributions. An important theoretical development is a reinterpretation of the P600 component. Previous accounts held that this component reflected syntactic reanalysis, but a recent proposal is that it reflects integration of new information into an event model (Brouwer et al. 2012). New EEG methods include multivariate pattern analysis such as those that have been applied to fMRI. For example, Sols and colleagues (2017) recently used pattern-based EEG to provide evidence that sequential structure within an event is recapitulated at event boundaries, and that this replay predicts subsequent memory (see also, e.g., Knoeferle et al. 2008). Over time, neurophysiological studies have increasingly embraced naturalistic materials and experimental designs (Maguire 2012).

Meanwhile, in computer vision, a new generation of neurally-inspired machine learning models have transformed object and action recognition from naturalistic stimuli (Herath et al. 2017; see also Google Cloud Vision API; FaceBook Detectron 2019). These methods provide tools to test behavioral and neural hypotheses about human event perception at scale. Crucial for these investigations will be the creation of large corpora of coded event stimuli (McNamara et al. 2017). Finally, event cognition is starting to move outside the laboratory to confront the richness of perception and memory “in the wild.” One ongoing development is the use of wearable sensors and lifelogging devices to measure behavioral structure on the scale not just of seconds or minutes but of hours to weeks (Nielson et al. 2015; Zhuang et al. 2012). These data suggest that events over the course of the day have segmental structure consistent with that attested by observers’ segmentation of relatively brief events in the laboratory.

Looking ahead in event perception

So, where do we stand and how do things look? Since Johansson and colleagues (1980) canvassed the study of event perception in the Annual Review there has been a dramatic growth and widening in the empirical phenomena encompassed by the field. First, event perception has become more cognitive, embracing the investigation of mental representation as a complement to characterizing the structure of the stimulus. Second, event perception and event memory have converged on a number of features of event structure. Third, new measures—especially neurophysiological ones—have been brought to bear. Finally, the sheer volume of the empirical database on event perception and memory has grown dramatically. These developments warrant using the broader term “event cognition” to describe the new state of the art.

These empirical developments have been accompanied by new generations of theories. These theories have attempted to account for the interaction of perception, action, and memory, for the segmentation of ongoing activity, and for the temporal organization of events on multiple scales. Though the models reviewed here vary in their scope, assumptions, and format, they share a concern with the structure of event representations in the mind and brain. That is, like the empirical research programs, the theoretical programs have become very cognitive.

What comes next? One possibility is that the models will scale up to something more like the full complexity of event comprehension, as the empirical work has already started to do. We can look forward to theoretical and computational models that can “experience” the same environments that our participants confront and mechanistically account for their comprehension and memory for those environments in terms of representational form and neurophysiological instantiation. Ideally, a model would take in the same stimuli as a participant and produce a set of representations that would enable it track activity in real time, and also have event memories and event knowledge. Such a model should not be a passive perceiver but should also be able to act on its environment, closing the loop between perception and action, using event memory and knowledge as guides.

Another encouraging development is that event cognition appears to be drawing together research from multiple areas of psychology with neurophysiology, linguistics, and computer science. As the field of cognitive science has grown over the last several decades, its transdisciplinary strength has dissipated. One may hope that event cognition will develop into a point of newfound cognitive science consilience.

Acknowledgments

This research was supported by grants R21AG05231401 from the US National Institutes of Health and N00014–17-1–2961from the US Office of Naval Research. I would like to thank Susan Fiske for thoughtful and incisive editing, and the following for sharing feedback on the manuscript: Aya Ben-Yakov, Dorthe Berntsen, Neil Cohn, James Cutting, Johannes Gerwien, Bernhard Hommel, Helene Intraub, Alan Kersten, Peter Krøgjaard, Michael Kubovy, Christopher Kurby, Jeffrey Loucks, Ken McRae, Frank Papenmeier, Jesse Sargent, Brent Strickland, Martin Takac, Trine Sonne, Christiane von Stutterheim, and the members of the Dynamic Cognition Laboratory.

Terms and Definitions

Footnotes

1To identify relevant research, in January 2019 I searched Google Scholar, Web of Science, and PsycInfo with the terms “event perception” and “event memory.” This resulted in 1054 hits. Even after discarding duplicates, ephemera, and material that was off point, a substantial number of items remained. The final list of cited works reflects selection amongst those items and integration of other materials that either were known to me or came up in reading the retrieved papers and chapters.

Literature Cited

• Abelson RP. 1981. Psychological status of the script concept. American Psychologist 36:715–29 [Google Scholar]
• Addis DR, Wong AT, Schacter DL. 2007. Remembering the past and imagining the future: Common and distinct neural substrates during event construction and elaboration. Neuropsychologia 45(7):1363–77 [PMC free article] [PubMed] [Google Scholar]
• Allen K, Ibara S, Seymour A, Cordova N, Botvinick M. 2010. Abstract structural representations of goal-directed behavior. Psychological Science 21(10):1518–24 [PubMed] [Google Scholar]
• Altmann GTM, Mirkovic J. 2009. Incrementality and prediction in human sentence processing. Cognitive Science 33(4):583–609 [PMC free article] [PubMed] [Google Scholar]
• Amoruso L, Gelormini C, Aboitiz FA, Alvarez González Manes F, et al. 2013. N400 ERPs for actions: building meaning in context. Front. Hum. Neurosci 7: [PMC free article] [PubMed] [Google Scholar]
• Avni-Babad D, Ritov I. 2003. Routine and the perception of time. Journal of Experimental Psychology: General 132(4):543–50 [PubMed] [Google Scholar]
• Bailey HR, Kurby CA, Giovannetti T, Zacks JM. 2013a. Action perception predicts action performance. Neuropsychologia 51(11):2294–2304 [PMC free article] [PubMed] [Google Scholar]
• Bailey HR, Zacks JM, Hambrick DZ, Zacks RT, Head D, et al. 2013b. Medial temporal lobe volume predicts elders’ everyday memory. Psychological Science 24(7):1113–22 [PMC free article] [PubMed] [Google Scholar]
• Baker LJ, Levin DT. 2015. The role of relational triggers in event perception. Cognition 136:14–29 [PubMed] [Google Scholar]
• Baldassano C, Chen J, Zadbood A, Pillow JW, Hasson U, Norman KA. 2017. Discovering event structure in continuous narrative perception and memory. Neuron 95(3):709–721 [PMC free article] [PubMed] [Google Scholar]
• Baldassano C, Hasson U, Norman KA. 2018. Representation of real-world event schemas during narrative perception. J. Neurosci 38(45):9689–99 [PMC free article] [PubMed] [Google Scholar]
• Baldwin DA, Baird JA, Saylor MM, Clark MA. 2001. Infants parse dynamic action. Child Development 72(3):708–17 [PubMed] [Google Scholar]
• Bangert AS, Kurby CA, Zacks JM. in press. The influence of everyday events on prospective timing “in the moment.” Psychonomic Bulletin & Review [PMC free article] [PubMed]
• Bar M 2007. The proactive brain: using analogies and associations to generate predictions. Trends in Cognitive Sciences 11(7):280–89 [PubMed] [Google Scholar]
  Reviews Bar’s proposal for how predictive processing guides perception.
• Barbey A, Krueger F, Grafman J. 2009. Structured event complexes in the medial prefrontal cortex support counterfactual representations for future planning. Philosophical Transactions of the Royal Society B-Biological Science 364(1521):1291–1300 [PMC free article] [PubMed] [Google Scholar]
• Barker RG, ed. 1963. The Stream of Behavior: Explorations of Its Structure & Content Appleton-Century-Crofts [Google Scholar]
  Reviews ecological studies of the structure of human behavior by Barker’s field research station.
• Bauer PJ. 2006. Event Memory. In Handbook of Child Psychology: Cognition, Perception, and Language, Vol. 2, 6th Ed., eds. Kuhn D, Siegler RS, Damon W, Lerner RM, pp. 373–425. Hoboken, NJ: John Wiley & Sons Inc [Google Scholar]
  Describes theory and data on how event memory develops in children.
• Ben-Yakov A, Dudai Y. 2011. Constructing realistic engrams: Poststimulus activity of hippocampus and dorsal striatum predicts subsequent episodic memory. The Journal of Neuroscience 31(24):9032–42 [PMC free article] [PubMed] [Google Scholar]
• Ben-Yakov A, Eshel N, Dudai Y. 2013. Hippocampal immediate poststimulus activity in the encoding of consecutive naturalistic episodes. Journal of Experimental Psychology: General 142(4):1255. [PubMed] [Google Scholar]
• Ben-Yakov A, Rubinson M, Dudai Y. 2014. Shifting gears in hippocampus: Temporal dissociation between familiarity and novelty signatures in a single event. The Journal of Neuroscience 34(39):12973–12981 [PMC free article] [PubMed] [Google Scholar]
• Berntsen D 2010. The unbidden past: Involuntary autobiographical memories as a basic mode of remembering. Current Directions in Psychological Science 19(3):138–42 [Google Scholar]
  Proposes that many experiences of remembering events result from involuntary associative retrieval.
• Bläsing B, Tenenbaum G, Schack T. 2009. The cognitive structure of movements in classical dance. Psychology of Sport and Exercise 10(3):350–60 [Google Scholar]
• Bonasia K, Blommesteyn J, Moscovitch M. 2016. Memory and navigation: Compression of space varies with route length and turns. Hippocampus 26(1):9–12 [PubMed] [Google Scholar]
• Bonasia K, Sekeres MJ, Gilboa A, Grady CL, Winocur G, Moscovitch M. 2018. Prior knowledge modulates the neural substrates of encoding and retrieving naturalistic events at short and long delays. Neurobiology of Learning and Memory 153:26–39 [PubMed] [Google Scholar]
• Brewer WF, Dupree DA. 1983. Use of plan schemata in the recall and recognition of goal-directed actions. Journal of Experimental Psychology: Learning, Memory, and Cognition JEP:LMC 9(1):117–29 [Google Scholar]
• Brouwer H, Fitz H, Hoeks J. 2012. Getting real about Semantic Illusions: Rethinking the functional role of the P600 in language comprehension. Brain Research 1446:127–43 [PubMed] [Google Scholar]
• Brown D, Pipe M- E. 2003. Individual differences in children’s event memory reports and the narrative elaboration technique. Journal of Applied Psychology 88(2):195–206 [PubMed] [Google Scholar]
• Brown NR. 2005. On the prevalence of event clusters in autobiographical memory. Social Cognition 35–69
• Butz MV, Bilkey D, Humaidan D, Knott A, Otte S. 2018. Learning, planning, and control in a monolithic neural event inference architecture. arXiv:1809.07412 [cs] [PubMed]
• Christophel TB, Klink PC, Spitzer B, Roelfsema PR, Haynes J- D. 2017. The distributed nature of working memory. Trends in Cognitive Sciences 21(2):111–24 [PubMed] [Google Scholar]
• Cohen LB, Amsel G. 1998. Precursors to infants’ perception of the causality of a simple event. Infant behavior and development
• Cohn N, Paczynski M. 2013. Prediction, events, and the advantage of Agents: The processing of semantic roles in visual narrative. Cognitive Psychology 67(3):73–97 [PMC free article] [PubMed] [Google Scholar]
• Cohn N, Paczynski M, Kutas M. 2017. Not so secret agents: Event-related potentials to semantic roles in visual event comprehension. Brain and Cognition 119(Supplement C):1–9 [PMC free article] [PubMed] [Google Scholar]
• Cutting JE. 1981. Six tenets for event perception. Cognition 10(1):71–78 [PubMed] [Google Scholar]
• Cutting JE. 2014. Event segmentation and seven types of narrative discontinuity in popular movies. Acta Psychologica 149:69–77 [PubMed] [Google Scholar]
• Defina R 2016. Do serial verb constructions describe single events?: A study of co-speech gestures in Avatime. Language 92(4):890–910 [Google Scholar]
• DuBrow S, Davachi L. 2016. Temporal binding within and across events. Neurobiology of Learning and Memory 134, Part A:107–14 [PMC free article] [PubMed] [Google Scholar]
• Earles JL, Kersten AW, Curtayne ES, ... 2008. That’s the man who did it, or was it a woman? Actor similarity and binding errors in event memory. Psychonomic bulletin & … [PMC free article] [PubMed]
• Eisenberg ML, Zacks JM, Flores S. 2018. Dynamic prediction during perception of everyday events. Cognitive Research: Principles and Implications 3(1):53. [PMC free article] [PubMed] [Google Scholar]
• Elman JL, McRae K. in press. A model of event knowledge. Psychological Review
  Presents a recurrent connectionist model of the acquisition and deployment of event knowledge.
• Ezzyat Y, Davachi L. 2011. What constitutes an episode in episodic memory? Psychological Science 22(2):243–52 [PMC free article] [PubMed] [Google Scholar]
• Faber M, D’Mello SK. 2018. How the stimulus influences mind wandering in semantically rich task contexts. Cognitive Research: Principles and Implications 3(1): [PMC free article] [PubMed] [Google Scholar]
• FaceBook Detectron 2019. https://github.com/facebookresearch/Detectron
• Ferretti TR, McRae K, Hatherell A. 2001. Integrating verbs, situation schemas, and thematic role concepts. Journal of Memory and Language 44(4):516–547 [Google Scholar]
• Fivush R 1997. Event memory in early childhood. In The Development of Memory in Childhood, ed. Cowan N, pp. 139–61. Hove England: Psychology Press/Erlbaum (UK) Taylor & Francis [Google Scholar]
• Flanagan JR, Johansson RS. 2003. Action plans used in action observation. Nature 424:769–71 [PubMed] [Google Scholar]
• Flecken M, Athanasopoulos P, Kuipers JR, Thierry G. 2015. On the road to somewhere: Brain potentials reflect language effects on motion event perception. Cognition 141:41–51 [PubMed] [Google Scholar]
• Flecken M, von Stutterheim C, Carroll M. 2014. Grammatical aspect influences motion event perception: Findings from a cross- linguistic non-verbal recognition task. Language and Cognition: An Interdisciplinary Journal of Language and Cognitive Science 6(1):45–78 [Google Scholar]
• Flores S, Bailey HR, Eisenberg ML, Zacks JM. 2017. Event segmentation improves event memory up to one month later. Journal of Experimental Psychology: Learning, Memory, and Cognition 43(8):1183–1202 [PMC free article] [PubMed] [Google Scholar]
• Fournier LR, Gallimore JM. 2013. What makes an event: Temporal integration of stimuli or actions? Atten Percept Psychophys 75(6):1293–1305 [PubMed] [Google Scholar]
• Franklin N, Norman KA, Ranganath C, Zacks JM, Gershman SJ. 2019. Structured event memory: a neuro-symbolic model of event cognition. bioRxiv 541607 [PubMed]
• Gennari SP, Sloman SA, Malt BC, Fitch WT. 2002. Motion events in language and cognition. Cognition 83(1):49–79 [PubMed] [Google Scholar]
• Gerwien J, von Stutterheim C. 2018. Event segmentation: Cross-linguistic differences in verbal and non-verbal tasks. Cognition 180:225–37 [PubMed] [Google Scholar]
• Gibson JJ. 1979. The Ecological Approach to Visual Perception Boston: Houghton Mifflin [Google Scholar]
• Giovannetti T, Bettcher BM, Brennan L, Libron DJ, Kessler RK, Duey K. 2008. Coffee with jelly or unbuttered toast: Commissions and omissions are dissociable aspects of everyday action impairment in Alzheimer’s disease. Neuropsychology 22(2):235–45 [PubMed] [Google Scholar]
• Gold DA, Park NW, Troyer AK, Murphy KJ. 2015. Compromised naturalistic action performance in amnestic mild cognitive impairment. Neuropsychology 29(2):320–33 [PubMed] [Google Scholar]
• Gold DA, Zacks JM, Flores S. 2016. Effects of cues to event segmentation on subsequent memory. Cognitive Research: Principles and Implications 2: [PMC free article] [PubMed] [Google Scholar]
• Google Cloud Vision API. Google Cloud. https://cloud.google.com/vision/
• Gravina MT, Sederberg PB. 2017. The neural architecture of prediction over a continuum of spatiotemporal scales. Current Opinion in Behavioral Sciences 17(Supplement C):194–202 [Google Scholar]
• Hafri A, Papafragou A, Trueswell JC. 2012. Getting the gist of events: Recognition of two-participant actions from brief displays. Journal of Experimental Psychology: General [PMC free article] [PubMed]
• Hafri A, Trueswell JC, Strickland B. 2018. Encoding of event roles from visual scenes is rapid, spontaneous, and interacts with higher-level visual processing. Cognition 175:36–52 [PMC free article] [PubMed] [Google Scholar]
• Hard BM, Recchia G, Tversky B. 2011. The shape of action. Journal of Experimental Psychology: General 140(4):586–604 [PubMed] [Google Scholar]
• Hard BM, Tversky B, Lang D. 2006. Making sense of abstract events: Building event schemas. Memory & Cognition 34(6):1221–35 [PubMed] [Google Scholar]
• Hasson U, Chen J, Honey CJ. 2015. Hierarchical process memory: memory as an integral component of information processing. Trends Cogn. Sci 19(6):304–13 [PMC free article] [PubMed] [Google Scholar]
  Proposes, based on neuroimaging studies using naturalistic materials, that there is a hierarchy of ranges of time-dependence on the human cortex.
• Hasson U, Yang E, Vallines I, Heeger DJ, Rubin N. 2008. A hierarchy of temporal receptive windows in human cortex. Journal of Neuroscience 28(10):2539–50 [PMC free article] [PubMed] [Google Scholar]
• Hayhoe M, Ballard D. 2005. Eye movements in natural behavior. Trends in cognitive sciences 9(4):188–194 [PubMed] [Google Scholar]
• Heberlein AS. 2008. Animacy and intention in the brain: Neuroscience of social event perception. In Understanding Events: From Perception to Action, Vol. 4, eds. Shipley TF, Zacks JM, pp. 363–88. New York, NY: Oxford University Press [Google Scholar]
• Hemeren PE, Thill S. 2011. Deriving motor primitives through action segmentation. Front. Psychology 1: [PMC free article] [PubMed] [Google Scholar]
• Herath S, Harandi M, Porikli F. 2017. Going deeper into action recognition: A survey. Image and Vision Computing 60:4–21 [Google Scholar]
• Hespos S, Saylor M, Grossman S. 2009. Infants’ ability to parse continuous actions. Developmental Psychology 45(2):575–85 [PubMed] [Google Scholar]
• Hespos SJ, Grossman SR, Saylor MM. 2010. Infants’ ability to parse continuous actions: Further evidence. Neural Networks 23(8–9):1026–32 [PubMed] [Google Scholar]
• Hohman TJ, Peynircioğlu ZF, Beason-Held LL. 2013. Flexibility of event boundaries in autobiographical memory. Memory 21(2):249–60 [PMC free article] [PubMed] [Google Scholar]
• Hommel B, Muesseler J, Aschersleben G, Prinz W. 2001. The Theory of Event Coding (TEC): A framework for perception and action planning. Behavioral & Brain Sciences 24(5):849–937 [PubMed] [Google Scholar]
  Describes TEC and reviews a wide range of empirical support.
• Hsieh L- T, Gruber MJ, Jenkins LJ, Ranganath C. 2014. Hippocampal activity patterns carry information about objects in temporal context. Neuron 81(5):1165–78 [PMC free article] [PubMed] [Google Scholar]
• Hudson JA, Fivush R, Kuebli J. 1992. Scripts and episodes: The development of event memory. Applied Cognitive Psychology 6(6):483–505 [Google Scholar]
• Huff M, Meitz TGK, Papenmeier F. 2014. Changes in situation models modulate processes of event perception in audiovisual narratives. Journal of Experimental Psychology: Learning, Memory, and Cognition 40(5):1377–88 [PubMed] [Google Scholar]
• Huff M, Papenmeier F, Zacks JM. 2012. Visual target detection is impaired at event boundaries. Visual Cognition 20(7):848–64 [Google Scholar]
• Intraub H 2010. Rethinking scene perception: A multisource model. In Psychology of Learning and Motivation, Vol. 52, pp. 231–64. Elsevier [Google Scholar]
• James W 1890. The Principles of Psychology, Vol. 1. New York: Henry Holt [Google Scholar]
• Jeunehomme O, D’Argembeau A. 2018. Event segmentation and the temporal compression of experience in episodic memory. Psychological research [PubMed]
• Johansson G, Hofsten C von H, Jansson G. 1980. Event Perception. Annual Review of Psychology 31(1):27–63 [PubMed] [Google Scholar]
• Keidel JL, Oedekoven CSH, Tut AC, Bird CM. 2017. Multiscale integration of contextual information during a naturalistic task. Cerebral Cortex 1–9
• Kersten AW, Earles JL. 2017. Feelings of familiarity and false memory for specific associations resulting from mugshot exposure. Memory & Cognition 45(1):93–104 [PMC free article] [PubMed] [Google Scholar]
• Kersten AW, Earles JL, Upshaw C. 2013. False recollection of the role played by an actor in an event. Memory & Cognition 41(8):1144–58 [PMC free article] [PubMed] [Google Scholar]
• Knoeferle P, Habets B, Crocker M, Munte T. 2008. Visual scenes trigger immediate syntactic reanalysis: Evidence from ERPs during situated spoken comprehension. Cerebral Cortex 18(4):789–95 [PubMed] [Google Scholar]
• Köhler W 1929. Gestalt Psychology New York: H. Liveright [Google Scholar]
• Koutstaal W, Schacter DL, Johnson MK, Galluccio L. 1999. Facilitation and impairment of event memory produced by photograph review. Memory & Cognition 27(3):478–93 [PubMed] [Google Scholar]
• Kubovy M 2015. the deep structure of lives. Philosophia Scientiæ 19(3):153–76‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬ [Google Scholar]
• Kurby C, Zacks JM. 2011. Age differences in the perception of hierarchical structure in events. Memory & Cognition 39(1):75–91 [PMC free article] [PubMed] [Google Scholar]
• Kurby CA, Zacks JM. 2018. Preserved neural event segmentation in healthy older adults. Psychology and Aging 33(2):232–45 [PMC free article] [PubMed] [Google Scholar]
• Lawrence Z, Peterson D. 2016. Mentally walking through doorways causes forgetting: The location updating effect and imagination. Memory 24(1):12–20 [PubMed] [Google Scholar]
• Levin DT. 2010. Spatial representations of the sets of familiar and unfamiliar television programs. Media Psychology 13(1):54–76 [Google Scholar]
• Loschky LC, Larson AM, Magliano JP, Smith TJ. 2015. What would Jaws do? The tyranny of film and the relationship between gaze and higher-level narrative film comprehension. PLOS ONE 10(11):e0142474. [PMC free article] [PubMed] [Google Scholar]
• Loucks J, Mutschler C, Meltzoff AN. 2016. Children’s representation and imitation of events: How goal organization influences 3-year-old children’s memory for action sequences. Cognitive Science [PubMed]
• Loucks J, Pechey M. 2016. Human action perception is consistent, flexible, and orientation dependent. Perception 45(11):1222–39 [PubMed] [Google Scholar]
• MacLean CL, Coburn PI, Chong K, Connolly DL. 2018. Breaking script: Deviations and postevent information in adult memory for a repeated event. Applied Cognitive Psychology 32(4):474–86 [Google Scholar]
• Magliano JP, Radvansky GA, Forsythe JC, Copeland DE. 2014. Event segmentation during first-person continuous events. Journal of Cognitive Psychology 26(6):649–61 [Google Scholar]
• Maguire EA. 2012. Studying the freely-behaving brain with fMRI. NeuroImage 62(2):1170–76 [PMC free article] [PubMed] [Google Scholar]
• McNamara Q, De La Vega A, Yarkoni T. 2017. Developing a comprehensive framework for multimodal feature extraction. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1567–1574. New York, NY, USA: ACM [Google Scholar]
• McRae K, Hare M, Elman JL, Ferretti T. 2005. A basis for generating expectancies for verbs from nouns. Memory & Cognition 33(7):1174–1184 [PubMed] [Google Scholar]
• Michotte AE. 1946. The Perception of Causality New York: Basic Books [Google Scholar]
• Miller GA, Galanter E, Pribram KH. 1960. Plans and the Structure of Behavior New York: Holt [Google Scholar]
• Miller GA, Johnson-Laird PN. 1976. Language and Perception Cambridge, MA: Harvard University Press [Google Scholar]
• Monroy CD, Gerson SA, Domínguez-Martínez E, Kaduk K, Hunnius S, Reid V. 2017. Sensitivity to structure in action sequences: An infant event-related potential study. Neuropsychologia [PubMed]
• Monroy CD, Gerson SA, Hunnius S. 2018. Translating visual information into action predictions: Statistical learning in action and nonaction contexts. Mem Cogn 1–14 [PubMed]
• Nakano T, Kitazawa S. 2017. Development of long-term event memory in preverbal infants: an eye-tracking study. Scientific Reports [PMC free article] [PubMed] [Google Scholar]
• Newtson D 1973. Attribution and the unit of perception of ongoing behavior. Journal of Personality and Social Psychology 28(1):28–38 [Google Scholar]
• Newtson D 1976. Foundations of attribution: The perception of ongoing behavior. In New Directions in Attribution Research, eds. Harvey JH, Ickes WJ, Kidd RF, pp. 223–48. Hillsdale, New Jersey: Lawrence Erlbaum Associates [Google Scholar]
• Nielson DM, Smith TA, Sreekumar V, Dennis S, Sederberg PB. 2015. Human hippocampus represents space and time during retrieval of real-world memories. Proceedings of the National Academy of Sciences 112(35):11078–83 [PMC free article] [PubMed] [Google Scholar]
• Papafragou A 2010. Source-goal asymmetries in motion representation: Implications for language production and comprehension. Cognitive Science 34(6):1064–92 [PMC free article] [PubMed] [Google Scholar]
• Papafragou A, Hulbert J, Trueswell J. 2008. Does language guide event perception? Evidence from eye movements. Cognition 108:155–84 [PMC free article] [PubMed] [Google Scholar]
• Papenmeier F, Brockhoff A, Huff M. 2019. Filling the gap despite full attention: the role of fast backward inferences for event completion. Cogn. Research 4(1):3 [PMC free article] [PubMed] [Google Scholar]
• Pettijohn KA, Radvansky GA. 2015. Walking through doorways causes forgetting: Event structure or updating disruption? The Quarterly Journal of Experimental Psychology (just-accepted):1–27 [PubMed]
• Pettijohn KA, Radvansky GA. 2016. Narrative event boundaries, reading times, and expectation. Mem Cogn 44(7):1064–75 [PubMed] [Google Scholar]
• Pettijohn KA, Thompson AN, Tamplin AK, Krawietz SA, Radvansky GA. 2016. Event boundaries and memory improvement. Cognition 148:136–44 [PubMed] [Google Scholar]
• Radvansky G, Copeland D. 2006a. Walking through doorways causes forgetting: situation models and experienced space. Memory & Cognition 34(5):1150–56 [PubMed] [Google Scholar]
• Radvansky GA. 2012. Across the event horizon. Current Directions in Psychological Science 21(4):269–272 [Google Scholar]
• Radvansky GA, Copeland DE. 2006b. Walking through doorways causes forgetting: Situation models and experienced space. Memory & Cognition 34(5):1150–56 [PubMed] [Google Scholar]
• Radvansky GA, Copeland DE, Berish DE, Dijkstra K. 2003. Aging and situation model updating. Aging, Neuropsychology, and Cognition 10:158–66 [Google Scholar]
• Radvansky GA, Dijkstra K. 2007. Aging and situation model processing. Psychonomic Bulletin & Review 14(6):1027–42 [PubMed] [Google Scholar]
• Radvansky GA, Krawietz S, Tamplin A. 2011. Walking through doorways causes forgetting: Further explorations. The Quart. J. of Expt. Psych 1–14 [PubMed]
• Radvansky GA, O’Rear AE, Fisher JS. 2017. Event models and the fan effect. Mem Cogn 45(6):1028–44 [PubMed] [Google Scholar]
• Radvansky GA, Tamplin AK, Krawietz SA. 2010. Walking through doorways causes forgetting: Environmental integration. Psychonomic Bulletin & Review 17(6):900–904 [PubMed] [Google Scholar]
• Radvansky GA, Zacks JM. 2014. Event Cognition New York: Oxford University Press [Google Scholar]
  This book gives a comprehensive review of research on event cognition and describes EST and the Event Horizon model in depth.
• Radvansky GA, Zwaan RA, Federico T, Franklin N. 1998. Retrieval from temporally organized situation models. Journal of Experimental Psychology: Learning, Memory, & Cognition 24(5):1224–37 [PubMed] [Google Scholar]
• Ranganath C, Ritchey M. 2012. Two cortical systems for memory-guided behaviour. Nature Reviews Neuroscience 13(10):713–26 [PubMed] [Google Scholar]
  Presents the posterior-medial/anterior temporal (PMAT) account of how brain systems integrate knowledge-based associations and arbitrary episodic associations in memory.
• Reynolds JR, Zacks JM, Braver TS. 2007. A computational model of event segmentation from perceptual prediction. Cognitive Science 31:613–43 [PubMed] [Google Scholar]
• Richmond LL, Gold DA, Zacks JM. 2017. Event perception: Translations and applications. Journal of Applied Research in Memory and Cognition 6(2):111–20 [PMC free article] [PubMed] [Google Scholar]
• Rinck M, Bower G. 2000. Temporal and spatial distance in situation models. Memory and Cognition 28(8):1310–20 [PubMed] [Google Scholar]
• Rubin DC, Umanath S. 2015. Event memory: A theory of memory for laboratory, autobiographical, and fictional events. Psychological Review 122(1):1–23 [PMC free article] [PubMed] [Google Scholar]
• Sargent JQ, Zacks JM, Hambrick DZ, Zacks RT, Kurby CA, et al. 2013. Event segmentation ability uniquely predicts event memory. Cognition 129(2):241–55 [PMC free article] [PubMed] [Google Scholar]
• Saylor M, Baldwin D, Baird J, LaBounty J. 2007. Infants’ on-line segmentation of dynamic human action. Journal of Cognition and Development 8(1):113–28 [Google Scholar]
• Schacter DL, Addis DR, Hassabis D, Martin VC, Spreng RN, Szpunar KK. 2012. The future of memory: Remembering, imagining, and the brain. Neuron 76(4):677–94 [PMC free article] [PubMed] [Google Scholar]
• Sharon T, Wynn K. 1998. Individuation of actions from continuous motion. Psychological Science 9(5):357–62 [Google Scholar]
• Sheffield EG, Hudson JA. 1994. Reactivation of toddlers’ event memory. In Long-Term Retention of Infant Memories. Memory, Vol. 2, No. 4., ed. Fivush R, pp. 447–65. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc [Google Scholar]
• Simony E, Honey CJ, Chen J, Lositsky O, Yeshurun Y, et al. 2016. Dynamic reconfiguration of the default mode network during narrative comprehension. Nature Communications 7(1): [PMC free article] [PubMed] [Google Scholar]
• Solomon SH, Hindy NC, Altmann GTM, Thompson-Schill SL. 2015. Competition between mutually exclusive object states in event comprehension. Journal of Cognitive Neuroscience 27(12):2324–38 [PMC free article] [PubMed] [Google Scholar]
• Sols I, DuBrow S, Davachi L, Fuentemilla L. 2017. Event boundaries trigger rapid memory reinstatement of the prior events to promote their representation in long-term memory. Current Biology 27(22):3499–3504 [PMC free article] [PubMed] [Google Scholar]
• Sonne T, Kingo OS, Krøjgaard P. 2016. Occlusions at event boundaries during encoding have a negative effect on infant memory. Consciousness and cognition 41:72–82 [PubMed] [Google Scholar]
• Sonne T, Kingo OS, Krøjgaard P. 2017. Bound to remember: Infants show superior memory for objects presented at event boundaries. Scandinavian Journal of Psychology 58(2):107–13 [PubMed] [Google Scholar]
• Speer NK, Reynolds JR, Zacks JM. 2007. Human brain activity time-locked to narrative event boundaries. Psychological Science 18(5):449–55 [PubMed] [Google Scholar]
• Stawarczyk D, Bezdek MA, Zacks JM. under review. Constructing event representations: The role of the midline default network core [PMC free article] [PubMed]
• Strickland B, Keil F. 2011. Event completion: Event based inferences distort memory in a matter of seconds. Cognition 121(3):409–15 [PMC free article] [PubMed] [Google Scholar]
• Strickland B, Scholl BJ. 2015. Visual perception involves event-type representations: The case of containment versus occlusion. Journal of Experimental Psychology: General 144(3):570. [PubMed] [Google Scholar]
• Swallow KM, Barch DM, Head D, Maley CJ, Holder D, Zacks JM. 2011. Changes in events alter how people remember recent information. Journal of Cognitive Neuroscience 23(5):1052–64 [PMC free article] [PubMed] [Google Scholar]
• Swallow KM, Kemp JT, Candan Simsek A. 2018. The role of perspective in event segmentation. Cognition 177:249–62 [PubMed] [Google Scholar]
• Swallow KM, Zacks JM, Abrams RA. 2009. Event boundaries in perception affect memory encoding and updating. Journal of Experimental Psychology: General 138(2):236–57 [PMC free article] [PubMed] [Google Scholar]
• Tauzin T 2015. Simple visual cues of event boundaries. Acta Psychologica 158:8–18 [PubMed] [Google Scholar]
• Trabasso T, Stein NL. 1997. Narrating, representing, and remembering event sequences. In Developmental Spans in Event Comprehension and Representation: Bridging Fictional and Actual Events., Vol. Developmental Spans in Event Comprehension and Representation: Bridging Fictional and Actual Events, pp. 237–70. Mahwah, NJ, USA: Lawrence Erlbaum Associates, Inc [Google Scholar]
• Tulving E 1983. Elements of Episodic Memory New York: Oxford University Press [Google Scholar]
• Wahlheim CN, Zacks JM. in press. Memory guides the comprehension of event changes for older and younger adults. Journal of Experimental Psychology: General [PMC free article] [PubMed]
• Wang Y, Gennari SP. 2019. How language and event recall can shape memory for time. Cognitive Psychology 108:1–21 [PubMed] [Google Scholar]
• Warren WH, Shaw RE, eds. 1985. Persistence and Change: Proceedings of the First International Conference on Event Perception Hillsdale, N.J: L. Erlbaum Associates [Google Scholar]
• Webb A, Knott A, MacAskill MR. 2010. Eye movements during transitive action observation have sequential structure. Acta Psychologica 133(1):51–56 [PubMed] [Google Scholar]
• Whitney C, Huber W, Klann J, Weis S, Krach S, Kircher T. 2009. Neural correlates of narrative shifts during auditory story comprehension. NeuroImage 47(1):360–66 [PubMed] [Google Scholar]
• Woodward AL. 2009. Infants’ grasp of others’ intentions. Current Directions in Psychological Science 18(1):53–57 [PMC free article] [PubMed] [Google Scholar]
• Wyer RS, Bodenhausen GV. 1985. Event memory: The effects of processing objectives and time delay on memory for action sequences. Journal of Personality and Social Psychology 49(2):301–16 [PubMed] [Google Scholar]
• Wynn K 1996. Infants’ individuation and enumeration of actions. Psychological Science 7(3):164–69 [Google Scholar]
• Zacks JM, Braver TS, Sheridan MA, Donaldson DI, Snyder AZ, et al. 2001. Human brain activity time-locked to perceptual event boundaries. Nature Neuroscience 4(6):651–55 [PubMed] [Google Scholar]
• Zacks JM, Kurby CA, Eisenberg ML, Haroutunian N. 2011. Prediction error associated with the perceptual segmentation of naturalistic events. Journal of Cognitive Neuroscience 23:4057–66 [PMC free article] [PubMed] [Google Scholar]
• Zacks JM, Sargent JQ. 2010. Event perception: A theory and its application to clinical neuroscience. In The Psychology of Learning and Motivation: Advances in Research and Theory, Vol. 53, ed. Ross BH, pp. 253–99. San Diego, CA US: Elsevier Academic Press [Google Scholar]
• Zacks JM, Speer NK, Swallow KM, Braver TS, Reynolds JR. 2007. Event perception: A mind/brain perspective. Psychological Bulletin 133(2):273–93 [PMC free article] [PubMed] [Google Scholar]
• Zhuang Y, Belkin M, Dennis S. 2012. Metric based automatic event segmentation. International Conference on Mobile Computing, Applications, and Services, pp. 129–148. Springer [Google Scholar]
• Zwaan RA, Radvansky GA. 1998. Situation models in language comprehension and memory. Psychological Bulletin 123(2):162–85 [PubMed] [Google Scholar]
  Reviews work on event representations in discourse comprehension.
• Zwickel J, Prinz W. 2012. Assimilation and contrast: the two sides of specific interference between action and perception. Psychological Research 76(2):171–82 [PubMed] [Google Scholar]