Evaluating Confidence in Our Memories
Evaluating Confidence in Our Memories
Results and Implications from Neuroimaging and Eye Movement Monitoring Studies of Metamemory
Abstract and Keywords
Recognition confidence is a common metric used to assess the accuracy of eyewitness identifications. Consequently, it is critical that we understand what information individuals use to make confidence judgments about their memory. Drawing on research in the field of metamemory (i.e., knowledge of one’s own memory), this chapter examines findings from the behavioral, eye tracking, and neuroimaging literature to determine what factors influence subjective memory confidence, and their relationship to objective accuracy. Critically, confidence judgments may be based on factors other than direct retrieval of the original event, such as familiarity or fluency of the cue that serves to elicit the sought after memory. The chapter also evaluates the potential for techniques such as functional magnetic resonance imaging (fMRI) and eye tracking in distinguishing highly confident accurate and highly confident inaccurate memory.
Eyewitness misidentifications are the most common cause of wrongful convictions, playing a role in over 75% of cases that are overturned by DNA evidence (http://www.innocenceproject.org/understand/Eyewitness-Misidentification.php; Wells, et al., 1998). These misidentifications are likely the result of the fact that our memory is constructive in nature and not a verbatim recording of the past (Schacter & Dodson, 2001; Schacter, Norman, & Koutstaal, 1998) rather than a malicious intent to wrongfully accuse another individual. However, if eyewitness identifications are susceptible to distortions, then how do we evaluate the validity of an eyewitness’s identification? Confidence in the eyewitness identification is the most common way accuracy is assessed, and confident witnesses are very convincing to jurors (Wells & Olson, 2000). Indeed, several court cases have indicated that confidence is a good indicator of accuracy and should be used to evaluate the accuracy of the eyewitness’s identification (Manson v. Braithwaite, 1977; United States v. Telfaire, 1979). However, 87% of experimental psychologists surveyed would testify that confidence is not necessarily a good indicator of accuracy (Kassin, Ellsworth, & Smith, 1989; Kassin, Tubb, Hosch, & Memon, 2001). What do these psychologists know that the courts do not?
Let’s consider the following case: On the witness stand, Roberta Ingrado was 100% confident that Dror Goldberg was the man who brutally stabbed Manuel Silverio to death (Goldberg v. State, 2002). Most people would find (p.119) her certainty convincing (Cutler, Penrod, & Stuve, 1988; Wells & Olson, 2000). However, at the time of the lineup, which in this case occurred years earlier, she was only 80% confident in her identification (Goldberg v. State, 2002; Rubenzer, 2002). Many fewer people would be willing to convict on the basis of an identification made with only 80% confidence (Cutler et al., 1988). Would you trust her identification, especially given this change in confidence? At the very least, the change in confidence should raise a warning flag that something other than memory for the original event is contributing to her confidence. Thus, understanding what sources of information people use to make their confidence judgments is critical for being able to evaluate the validity of a confidence judgment. When individuals make a confidence judgment, they monitor the contents of their memory for relevant information and then make a final response. Researchers in the field of metamemory study people’s knowledge of their own memory, including the monitoring processes that lead to a final judgment. One purpose of metamemory research is to characterize the accuracies and inaccuracies of people’s introspective reports about their memory. In this chapter, I will discuss some of the theoretical and empirical work that focuses on how people make retrospective confidence judgments about their memory. I will then turn to whether particular methods, namely neuroimaging and eye tracking, might be useful in (1) distinguishing highly confident accurate and highly confident inaccurate memory and (2) determining what sources of information people use to make their confidence judgments.
How Do We Make Our Confidence Judgments?
The most common assumption is that memory confidence is strongly correlated with the strength or quality of the individual’s internal memory representation (Stretch & Wixted, 1998; Yonelinas, 1994). In some cases, this may be the dominant source for confidence, but numerous studies have shown that there are also other factors that influence memory (p.120) confidence (Bradfield, Wells, & Olson, 2002; Lindsay, Read, & Sharma, 1998; Perfect, Hollins, & Hunt, 2000; Sporer, 1993; Wells & Olson, 2000; Wells, Olson, & Charman, 2002, 2000). Almost everyone who has taken a multiple-choice test and used the process of elimination has experienced how confidence in an answer can be based on factors other than memory for the sought-after information. In such cases, even if an individual does not remember the correct answer to the question, but knows that all but one of the alternatives are wrong, that person should be confident that the remaining answer is correct, even though he or she does not have direct knowledge that the answer is correct. This is just one example of a reasoned, analytical process that can influence confidence judgments, and there (p.121) are many others. Theoretical conceptions of the types of influences on confidence judgments, as well as other metamemory judgments, have divided them into two types: experience-based and information-based (Koriat, Nussinson, Bless, & Shaked, 2008). Information-based judgments, as exemplified by but not limited to the process of elimination, are based on declarative knowledge and are the result of a deliberate, reasoned process. In contrast, experience-based judgments are made on the basis of a subjective feeling that arises from the mnemonic experience of the individual. Most people assume that memory confidence is experience-based and that the experience is directly tied to an accurate underlying memory representation. Of course, confidence judgments do not have to be, and are not likely to be, entirely experience-based or entirely information-based. Instead, they are likely to be based on a combination of experience- and information-related factors (Koriat, et al., 2008). However, it is useful to consider them separately to understand their influences on memory confidence.
Experience-based retrospective confidence judgments are based on the online experience of remembering (Koriat et al., 2008). The accuracy of experience-based judgments is often, but not always, similar to memory accuracy (Koriat et al., 2008; Yonelinas, 1994), especially if the mnemonic experience is based on the initial recollection. Research has shown that confidence judgments can be influenced by familiarity (Yonelinas, 1994), vividness of recollected details (Robinson, Johnson, & Robertson, 2000), and how quickly or easily the target information is retrieved (Kelley & Lindsay, 1993; Robinson, Johnson, & Herndon, 1997). In other words, memories that are retrieved more quickly and easily are often associated with higher confidence than memories that are more effortful to retrieve, and these factors also tend to be associated with the underlying memory trace. Thus, it is not surprising that in everyday situations memory confidence judgments based on these experience-based factors are often more accurate (Brewer, Keast, & Rishworth, 2002; Robinson et al., 2000; Sporer, Penrod, Read, & Cutler, 1995). Experience-based confidence judgments may be more desirable than information-based judgments because they tend to be more accurate and because they are thought to be related to the quality of the memory for the event.
However, it is important to note that most research procedures investigating experience-based judgments in the laboratory are very different from the circumstances under which eyewitnesses retrieve their memories, and the content is also quite different. Most laboratory studies on experience-based confidence judgments involve participants retrieving a single episode, but this is not the case with eyewitnesses, who are often repeatedly recalling the same event over a long period of time. It is well established that repetition enhances the ease with which information is retrieved (e.g., Kelley & Lindsay, 1993), so (p.122) an eyewitness’s experience of remembering the event may be related to factors other than the strength of the original memory. Indeed, repeated recall has been shown to increase false memories and the confidence in those memories (e.g., Roediger, Jacoby, & McDermott, 1996; Zaragoza & Mitchell, 1996). In other words, experience-based confidence judgments, although likely more accurate than information-based judgments, are still fallible and may not be able to distinguish between accurate memory and certain kinds of memory distortions, such as source misattributions. Source misattributions are a kind of memory error that occurs when the information remembered is associated with the wrong context (e.g., mistakenly remembering seeing someone in person instead of hearing someone else say he had seen someone in person).
Information-based confidence judgments, on the other hand, have a less consistent relationship to memory accuracy and the underlying memory trace for the event in question. As one might expect, the relationship of information-based confidence judgments to accuracy depends on the accuracy of the premises on which they are based (Koriat et al., 2008). As such, confidence judgments that are information-based, completely or in part, are more likely to be less accurate as a whole. Importantly for the legal system and eyewitness confidence, information-based judgments are less likely to be grounded in the individual’s memory for the event and may be contaminated by other individuals’ perspectives and beliefs (Luus & Wells, 1994; Shaw, Appio, Zerr, & Pontoski, 2007). However, there are types of information, such as the witness’s knowledge of how good or bad his or her view of the crime was, that are relevant for eyewitness confidence and are not necessarily sources of contamination.
Much of the eyewitness confidence research has focused on the impact of information-based judgments on the confidence-accuracy relationship. There has been particular interest in how factors called system variables, which can be controlled by the judicial system (Wells & Seelau, 1995), influence confidence. Indeed, based on eyewitness research, some researchers have even made recommendations on how to conduct lineups so as to get the best relationship between confidence and accuracy possible (Wells et al., 1998). One example is sequential versus simultaneous lineups (Lindsay & Wells, 1985). The simultaneous lineup is the traditional lineup, in which several individuals are viewed together. By seeing multiple individuals together, the eyewitness may make a relative judgment in which the eyewitness weighs the evidence for one suspect over another, much as in a multiple-choice test. In contrast, the sequential lineup, in which suspects are presented one after another, minimizes the ability to make comparisons among the suspects and encourages the eyewitness to make a more experience-based or absolute judgment. Indeed, several studies have shown that sequential lineups show reduced rates of mistaken (p.123) identifications (Clark & Davey, 2005; Kneller, Memon, & Stevenage, 2001; Sporer, 1993; Steblay, Dysart, Fulero, & Lindsay, 2001).
Another example of information-based confidence judgments causing inflated confidence in an identification comes from studies on postidentification feedback. Witnesses who were told, after making an identification from a lineup, “Good, you identified the actual suspect” showed increased confidence in their memory compared to those who heard no feedback or were given disconfirming feedback (Lindsay & Wells, 1985). Furthermore, positive feedback has been shown to inflate confidence in both accurate and mistaken identifications (Semmler, Brewer, & Wells, 2004). This effect is robust, and a recent meta-analysis of studies investigating postidentification feedback showed large effects of feedback on eyewitness confidence (Douglass & Steblay, 2006). In such cases, individuals are acknowledging the feedback and are incorporating this declarative knowledge into their confidence decisions. If this feedback is accurate, then confidence and accuracy will be related. However, in a courtroom setting, establishing a strong confidence-accuracy relationship is not the only goal. In the courtroom, a defendant deserves a fair trial; therefore, witness confidence should not be contaminated by someone who has a priori knowledge of the suspect. A proposed way to combat this issue is to have double-blind lineup procedures, in which neither the witness nor the administrator knows who the suspect is (Wells et al., 1998).
Another issue that often comes up in the eyewitness research literature is the malleability of confidence. Confidence malleability refers to the finding that an eyewitness becomes more or less confident about his or her identification based on events that occur after the identification (Wells & Seelau, 1995). Indeed, this was illustrated by the case of Roberta Ingrado going from 80% to 100% confident in her identification of Dror Goldberg (Goldberg v. State, 2002). One way to think about this issue is to think about the difference between “I am confident that I remember and saw that the suspect committed the crime” and “I am confident that the suspect committed the crime.” In the latter case, factors other than memory for the event come into play, and these may be biased against the accused. An example of a variable that can lead to changes in confidence is the identification of a co-witness (Luus & Wells, 1994) or even social pressure (Shaw et al., 2007). For example, Luus & Wells (1994) showed that when witnesses were told that a co-witness identified the same suspect, they increased their confidence; similarly, when the co-witness reported a conflicting identification, they decreased their confidence. What a co-witness does should not influence the original witness’ memory, but it does influence his or her confidence. This is related to several studied phenomena. For example, this tendency to change a previous judgment in light of new information has been referred to as the hindsight bias (for review, see Blank, (p.124) Musch, & Pohl, 2007), and has been studied extensively in terms of memory and metacognitive judgments (e.g., Mazzoni & Vannucci, 2007; Sanna & Schwarz, 2007). Another example is the illusion of knowledge, in which more knowledge can lead to increased confidence, even when accuracy is reduced (e.g., Hall, Ariss, & Todorov, 2007). New information based on a co-witness is one example of a factor that can change a witness’s confidence, and it is yet another example of an information-based judgment that can be dangerous in the courtroom. Based on the finding that confidence is malleable, one recommendation has been to record the witness’ confidence immediately after the identification is made (Wells et al., 1998) in the hope that this confidence judgment will be less contaminated by extraneous factors. Recording witness confidence immediately after the identification is made is also useful in combatting inflation in confidence due to repetition.
In addition to understanding what variables could lead to overconfidence in an identification, much research has been directed at finding markers or assessment variables to help distinguish between accurate and mistaken eyewitness identifications (Sporer, 1993). Examples of potential markers that have been investigated include confidence-accuracy calibration curves (Weber & Brewer, 2000) and response latency cutoffs (Weber, Brewer, Wells, Semmler, & Keast, 2004). Thus far, these curves have led to some statistically significant and insignificant findings, but they have not been found to be robust enough to use as a marker. In the remainder of the chapter, I focus on two methods, neuroimaging and eye tracking, and their potential use as a marker for (1) distinguishing confident accurate and inaccurate memory and (2) identifying the sources of information used to make confidence judgments.
What Can Neuroimaging Tell Us about Confidence and Accuracy in Memory?
Neuroimaging studies of metamemory are still in their infancy, especially those related to retrospective confidence judgments (for review, see Schwartz & Bacon, 2008). Although numerous neuroimaging studies have used confidence as a measure of memory quality or memory strength (Ranganath et al., 2004; Yonelinas, Otten, Shaw, & Rugg, 2005), we will focus here primarily on studies examining brain activations that inform the basis of retrospective confidence judgments and can distinguish between confident accurate and inaccurate memory. There are three main ways that neuroimaging studies have examined metacognition: (1) by investigating which brain regions are involved in the process of making the metacognitive judgment, (2) by determining those brain regions in which activity is modulated by the subjective level of confidence expressed for both accurate and inaccurate memory, and (p.125) (3) by examining those brain regions in which activity is modulated based on metacognitive accuracy.
A fundamental question to ask is whether the same cognitive processes are engaged when people make confidence judgments and when people make recognition judgments. If explicit confidence is tied to the recognition judgment and happens automatically as people are retrieving memory, or if the judgments were solely experience-based, then we might expect similar patterns of brain activity when people make confidence and recognition judgments. If people engage in additional cognitive processes beyond any automatic memory monitoring that occurs, such as incorporating information-related factors, then we would expect to see different patterns of brain activity when people make recognition and confidence judgments. As mentioned previously, behavioral evidence that confidence and accuracy can be dissociated suggests that there are additional influences on making confidence judgments besides just the recognition experience (Koriat, 2008a, 2008b; Koriat et al., 2008). In our own neuroimaging study, we directly contrasted brain activity, as measured by functional magnetic resonance imaging (fMRI), and showed that there were many brain regions that showed greater activity when participants made confidence judgments about their recognition performance than when they performed a forced-choice recognition task (Chua, Schacter, Rand-Giovannetti, & Sperling, 2006). The brain regions that showed greater activity when individuals made confidence judgments compared to recognition judgments, including medial prefrontal, medial parietal, and lateral parietal cortices, have been shown to be involved in self-reflection and self-related processing (Gusnard, Akbudak, Shulman, & Raichle, 2001; Raichle et al., 2001). This is consistent with the idea that when people are explicitly asked to make confidence judgments, they engage in additional internal and self-reflective processes to make their judgments.
The next question to ask is whether neuroimaging can distinguish between highly confident accurate and inaccurate memories, and in what circumstances it can distinguish between them. We used fMRI to compare brain activity for high-confidence and low-confidence decisions using a face-name relational memory task (Chua et al., 2006). Participants learned face-name pairs and then were tested on their memory while being scanned. They saw the face with three names written underneath it and were instructed to choose which of the three names had been paired with that face previously, and one of the names was always correct. They were then asked to indicate whether they had high or low confidence that they chose correctly. We then examined which brain regions showed greater activity for all high-confidence decisions (correct and incorrect name choices) compared to all low-confidence decisions (correct and incorrect name choices). We showed that the anterior and posterior (p.126) cingulate, as well as the medial temporal lobes (including the hippocampus, perirhinal cortex, and parahippocampal cortex) showed greater activity for high- versus low-confidence responses regardless of accuracy (Chua et al., 2006; Chua, Schacter, & Sperling, 2009). In contrast, a network of frontoparietal regions showed greater activity for low- compared to high-confidence responses (Chua et al., 2009). In another study, Moritz, Glascher, Sommer, Buchel, and Braus (2006) reported similar findings using a different paradigm. They used a modified version of the Deese-Roediger-McDermott (DRM) paradigm (Roediger & McDermott, 1995), in which participants saw several lists of 12 related words during study, and then at test saw the studied words as well as 4 additional semantically related words and 6 nonrelated, nonstudied words. This paradigm is known to elicit high rates of false recall and recognition, often accompanied by high confidence (Gallo, 2006; Roediger & McDermott, 1995). Moritz et al. (2006) were then able to use fMRI to examine which brain regions were modulated by confidence (high, medium, and low levels) for hits (correctly identifying a studied item as studied), false alarms (incorrectly endorsing a nonstudied item as studied), misses (incorrectly endorsing a studied item as nonstudied), and correct rejections (correctly identifying a nonstudied item as nonstudied). Similar to Chua et al. (2006, 2009), they found that the anterior and posterior cingulate, as well as the medial temporal lobe, showed greater activity for high- compared to low-confidence responses regardless of whether they were hits, false alarms, misses, or correct rejections. These findings suggest that in these two paradigms, the fMRI signal is dominated by subjective confidence signals rather than objective accuracy. This may be an example of how experience-based judgments can still be inaccurate under certain kinds of memory distortions, such as source misattributions.
However, subsequent work by Kim and Cabeza (2007) was able to distinguish between high and low confidence for both true and false recognition. In this study, they used a modified version of the DRM paradigm (Roediger & McDermott, 1995) to elicit high levels of false recognition. At study, participants viewed four words along with a category name and were asked to indicate whether three or four words on the list belonged to the category. At test, they viewed studied words and nonstudied words that belonged to the same category. They then compared high- and low-confidence responses for both true and false recognition. Similar to the Chua et al. (2006, 2009) and Moritz et al. (2006) studies, they showed greater activity in the posterior cingulate for high- and low-confidence responses for both true and false recognition. However, they showed greater anterior cingulate activity for high- compared to low-confidence responses for true recognition only. Within the medial temporal lobes, they observed differential patterns related to confidence for true and false recognition. The hippocampus, which is believed to be important (p.127) in recollecting an item and its context (Davachi, 2006; Davachi, Mitchell, & Wagner, 2000; Diana, Yonelinas, & Ranganath, 2007), showed greater activity for high-confidence compared to low-confidence true recognition. In contrast, the perirhinal cortex, another structure within the medial temporal lobes, which is thought to be important in item memory or familiarity-based memory (Davachi, 2006; Davachi et al., 2000; Diana et al., 2007), showed greater activity for high- compared to low-confidence false recognition. Furthermore, additional frontoparietal regions, which are also thought to subserve recognition based on a feeling of familiarity (Henson, Rugg, Shallice, & Dolan, 2000; Henson, Rugg, Shallice, Josephs, & Dolan, 1999; Yonelinas et al., 2005), also showed greater activity for high- compared to low-confidence false recognition. In such a case, we see brain-based evidence that confidence for true and false recognition is based on different sources of information. High confidence for true recognition appears to be based on recollective information, whereas high confidence for false recognition appears to be based on stronger feelings of familiarity. Although there are similarities between the above-mentioned studies, the differences merit discussion.
In order for fMRI to have some potential utility in distinguishing highly confident accurate and inaccurate memory, we need to know the circumstances in which it is capable of doing this. The above-mentioned studies suggest that there are limits to the potential of fMRI to detect differences, and the variety of paradigms raises some possibilities for future testing. One likely possibility is that fMRI is useful for distinguishing highly confident accurate and inaccurate recognition only when the confidence judgments are based on different enough information. In the Kim and Cabeza (2007) study, the authors suggested that the patterns of brain activity seen for high-confidence compared to low-confidence true recognition versus the pattern of brain activity seen for high-confidence compared to low-confidence false recognition indicated that confidence for true recognition was based on recollection and confidence for false recognition was based on familiarity. The question remains whether this was also the case in the Moritz et al. (2006) and Chua et al. (2006, 2009) studies. Chua et al. used a face-name relational memory task, in which performance is less likely to be based on familiarity (Cohen, Poldrack, & Eichenbaum, 1997; Cohen et al., 1999), and errors are likely due to source misattributions. If fMRI can only detect differences between confidence decisions based on different qualities of memory, we would not expect it to distinguish between highly confident accurate memory based on recollection and highly confident inaccurate memory based on misattributions. The Moritz et al. and Kim and Cabeza studies used similar procedures, though it is possible that the specific manipulation used by Kim and Cabeza is more likely to elicit familiarity-based responses. Kim and Cabeza presented a category name with (p.128) the individual list items, whereas Moritz et al. did not, and it is possible that this led to more familiarity-based false recognition responses made with high confidence. One possibility is that the category label made other exemplars from the category more familiar, and thus false recognition was based on this increase in familiarity. Of course, hearing many exemplars from the category (even without seeing the category label) could also lead to increased familiarity of the nonstudied exemplar in the Moritz et al. study. However, the longer lists used in the Moritz et al. study could also have led to increased generation of related exemplars during study. In such a case, participants may recall thinking of the word during study, but misattribute it to being studied rather than internally generated, and this would lead to a high-confidence response based on a false recollection. In summary, there are consistencies in the literature regarding which brain regions are modulated by confidence, and some promise that fMRI may be useful in distinguishing high-confidence responses based on different sources of information. However, further research is needed to precisely delineate when fMRI is and is not sensitive to different types of confidence judgments.
Although distinguishing highly confident accurate and highly confident inaccurate memory is useful, another potentially useful marker would be one for metacognitive accuracy. Metacognitive accuracy refers to whether or not the subjective confidence judgment is congruent with objective accuracy. In other words, correct recognition judgments made with high confidence and incorrect recognition judgments made with low confidence are metacognitively accurate, whereas incorrect recognition judgments made with high confidence and correct recognition judgments made with low confidence are metacognitively inaccurate. Broadly speaking, two kinds of measures are typically used to assess metacognitive accuracy, and they are referred to as absolute and relative measures (Nelson, 1996; Pannu & Kaszniak, 2005). Absolute measures, such as the Hamann index, measure whether the confidence judgment and recognition judgment are congruent and are tied to the scale used, such that a metacognitively accurate judgment is one made with, for example, 25% confidence and has a 25% likelihood of being a correct recognition judgment. Relative measures, such as the gamma statistic, provide a measure of whether the confidence judgment given to a recognition judgment is consistently ordered, such that higher-confidence judgments are more likely to be accurate than lower-confidence judgments but are not tied to the scale used. Few neuroimaging studies have examined whether there are brain regions whose activity correlates with metacognitive accuracy for retrospective confidence ratings, and the results have been somewhat mixed. In our own studies, we showed no brain regions in which activity correlated with metacognitive accuracy, using a measure of absolute accuracy (Chua et al., 2009). However, (p.129) recent work showed that activity in the right frontopolar cortex, as measured by fMRI, correlated with the gamma coefficient, indicating that this region was sensitive to metacognitively accurate confidence ratings (Yokoyama et al., 2010). Clearly, more research needs to be done and these findings need to be replicated to determine how robust they are before determining whether there are brain regions sensitive to the metacognitive accuracy of confidence judgments. However, there is reason to be optimistic because several studies using other types of metacognitive judgments that require participants to predict their future memory performance (e.g., feeling-of -knowing, tip-of-the-tongue, and judgments of learning) have suggested that various prefrontal brain regions are important for metacognitive accuracy (Pannu & Kaszniak, 2005; Schwartz & Bacon, 2008). By extension, it may be that similar prefrontal regions are also involved in metacognitive accuracy for confidence judgments.
The question of whether fMRI is capable of distinguishing highly confident accurate memories from highly confident inaccurate memories, however, is not the only question to be asked. Another important question is whether it should be used in a legal context. Indeed, there is some evidence that certain types of fMRI evidence may be too convincing and could potentially lead to prejudicial judgments (McCabe & Castel, 2008; Weisberg, Keil, Goodstein, Rawson, & Gray, 2008). Neuroimaging studies are often accompanied by striking visual images showing bright blobs of activity overlaid on a brain image, and research has shown that information accompanied by these types of images is rated as based on sounder scientific reasoning than the same information presented without these kinds of images (McCabe & Castel, 2008). Thus, it is possible that jurors could give more weight to the brain evidence than is warranted. Furthermore, explanations containing information about neuroscience are more satisfying to nonexperts, even when the neuroscience information is irrelevant (Weisberg et al., 2008). Thus, adding neuroscientific information has the potential to hinder jurors’ ability to evaluate the evidence accurately. Accordingly, alternative methods may be more appropriate for a legal setting, and the next section addresses relevant research using eye movement monitoring.
Eye Movements, Memory Accuracy, and Memory Confidence
Another method that may eventually prove useful for distinguishing highly confident accurate and highly confident inaccurate memory, and for understanding the basis for confidence judgments, is eye movement monitoring. Many studies have shown that viewers’ eye movements are indicative of their (p.130) previous mnemonic experiences (for review, see Hannula et al., 2010) and can be used to examine what information is being used to make confidence judgments (Chua, Hannula, & Ranganath, 2012). Eye movement monitoring studies of memory capitalize on the fact that we live in a visual world and that our eye movements are not random, but are influenced by our previous experiences, expectations, and knowledge. For example, if we are expecting a visitor, we might look out the window more often for his or her arrival. Alternatively, if we see something surprising, such as a pig walking down a city street, our eyes may be drawn to it because it is unexpected. Although eye movement monitoring may not have the same explanatory appeal to the layperson that fMRI does (McCabe & Castel, 2008; Weisberg et al., 2008), eye movement monitoring can provide an indirect measure of the underlying brain activity. In memory studies, amnesic patients with hippocampal damage do not show the same eye movement–based memory effects as intact controls (Hannula, Ryan, Tranel, & Cohen, 2007; Ryan, Althoff, Whitlow, & Cohen, 2000), and eye movement indices of relational memory correlate with hippocamal activity as measured by fMRI (Hannula & Ranganath, 2009). As such, eye movement monitoring has the potential to reveal information about our memory and cognition. In evaluating the utility of eye movement monitoring and fMRI, several benefits to eye movement monitoring exist. It has the virtue of being less costly than fMRI and potentially less prejudicial (McCabe & Castel, 2008). Both fMRI and eye movements provide indirect measurements of the underlying neural activity; fMRI is a technique that measures the downstream metabolic consequences of neural activity and is therefore considered indirect. However, fMRI is arguably a step closer to the underlying neural activity than eye movements and has been correlated with measures of neural activity (e.g., Goense & Logothetis, 2008; Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001). Nevertheless, the many benefits of eye movement monitoring may be particularly appealing for potential use in evaluating eyewitness identifications because it allows memory to be assessed without relying only on verbal reports or introspective judgments.
There is a growing body of research examining eye movement–based memory effects (for review, see Hannula et al., 2010). Although there have been many types of eye movement and memory studies, I will focus here on differential viewing behavior for item memory and relational memory. Relational memory is the memory for an item and its context. The classic paradigm for studying item memory using eye movements is to compare viewing behavior between familiar and unfamiliar items. There have been robust eye movement–based item memory effects, with participants showing fewer fixations and less constrained viewing for familiar compared to unfamiliar faces (e.g., Althoff & Cohen, 1999) and scenes (e.g., Ryan et al., 2000). Other studies have focused on (p.131) relational aspects of memory and have shown that eye movements were sensitive to the memory of spatial positions of elements within a scene (Ryan et al., 2000), that eye movements reflected the temporal sequence in which objects were studied (Ryan & Villate, 2009), and that participants spent more time viewing a face that had been presented with a specific scene compared to faces that were presented with other scenes (Hannula & Ranganath, 2009; Hannula et al., 2007). Thus, eye movement indices of item and/or relational memory have the potential to be used in eyewitness procedures. However, first we need to consider their relationship to memory confidence.
In a laboratory study, we examined the relationship between eye movements, recognition accuracy, and recognition confidence (Chua et al., 2012). We showed participants a scene, followed by a face superimposed on that scene, which formed a face-scene pair, and participants were instructed to try to remember these face-scene pairs for later testing. We then tested their memory for these face-scene pairs and monitored their eye movements during testing. The recognition test consisted of viewing a scene, which served as a cue that would elicit attempted retrieval of the face that had previously been paired with that scene. After viewing the scene cue, participants saw three faces superimposed on the scene (three-face display) and were asked to choose which of the three faces had been paired with the scene previously. One of these faces was the target face that had previously been paired with that scene, and if chosen, it meant that the participant correctly recognized the face-scene pair. The other two faces were considered distracters and had been previously paired with different scenes; choosing either of these faces meant that the participant forgot the face-scene pair or had a source misattribution. After choosing a face during the three-face display, participants indicated their confidence that they had correctly chosen the face paired with the scene. We monitored eye movements both when participants viewed the scene cue and when they viewed the three-face display. We therefore had independent measures of the cueing period and the target recognition period and could examine how eye movements during these different stages related to confidence and accuracy.
We investigated two main types of influences on confidence judgments in this study (Chua et al., 2012). The first is the target recognition experience, which is an experience-based influence on confidence. Again, this is probably what most people assume forms the entire basis of confidence judgments. After all, we make confidence judgments about our ability to recognize a target, so therefore our mnemonic experience and the quality of the memory retrieved during recognition should influence our confidence. Examples of things that fall under this category, and have been shown to influence confidence (and typically accuracy as well), are target familiarity (Yonelinas, 1994), vividness of retrieval (Robinson et al., 2000), and speed/ease of retrieval (Kelley & Lindsay, 1993; Robinson et al., 1997). (p.132) Many of these overlap with what would influence accurate memory as well; therefore, we expected that confidence judgments based on target recognition experience were more related to accuracy and that this was a relevant source of information for confidence. Another potential factor is relative evidence for one alternative over another; this is particularly important in forced-choice tasks, in which participants are shown many alternatives and are forced to choose one of them. This would be an information-based influence on confidence and is different from the target recognition experience, which involves considering a single item, whereas relative evidence involves weighing evidence among multiple alternatives and is a reasoned, analytic process (Koriat et al., 2008). Again, the most familiar example is the process of elimination in multiple-choice tests. One may not remember the answer but still succeed in answering the question correctly if choices can be eliminated. Relative evidence is expected to be less diagnostic of accuracy than target recognition experience, but its relation to accuracy really depends on the validity of the beliefs held and thus is potentially relevant. Finally, the other potential influence we investigated was cue familiarity or cue utilization. When an individual is given a cue to elicit retrieval of a sought-after target, there may be influences related to that cue that affect confidence. Of course, the confidence judgments are not made about the cue, they are made about the sought-after target, so it may be surprising that this could have an influence. However, it has been shown in the semantic memory domain that cue familiarity can influence confidence (e.g., a computer expert is more confident about giving answers to computer questions than a nonexpert; for review, see Koriat et al., 2008). However, given that the confidence judgments are about the target, we expected these influences to be irrelevant and nonoverlapping with accuracy.
The target recognition experience, as measured by eye movements, influenced confidence for accurate but not inaccurate memory (Chua et al., 2012). We compared the proportion of time spent viewing each of the faces during the three-face display. For accurate memory, in which participants correctly chose the face that was paired with the scene cue, there was a linear increase in viewing of the correctly chosen face with increasing confidence. This suggests that one factor influencing confidence in accurate memory is the target recognition experience. In contrast, there was no such effect of confidence on viewing of the incorrectly chosen face for inaccurate recognition trials. This suggests that something other than the target recognition experience is driving confidence for inaccurate memory.
We next examined effects of relative evidence assessment on confidence judgments (Chua et al., 2012). We used the number of transitions, or movement of the eyes, from one face to another as an index of relative evidence assessment (p.133) based on previous findings from the decision-making literature (Pochon, Riis, Sanfey, Nystrom, & Cohen, 2008; Reutskaja, Nagel, Camerer, & Rangel, 2011). We did not see evidence of an effect of relative evidence assessment, as indexed by eye movements, on confidence, but instead observed slightly more transitions during inaccurate trials compared to accurate trials. Thus, it does not appear that relative evidence assessment was driving confidence for inaccurate memory in this study.
We then examined viewing behavior during the scene cueing period (Chua et al., 2012). We used the number of fixations made during the scene cueing period as our index of cue familiarity, or other cue-related processing, because several studies have shown fewer fixations to repeated stimuli compared to novel stimuli (e.g., Althoff & Cohen, 1999; Ryan et al., 2000). Our results showed that for both highly confident accurate and highly confident inaccurate memory, fewer fixations were made during the scene cueing period compared to during their medium-confidence counterpart. Taken together with the analyses based on the target recognition experience, it appears that highly confident accurate memories show eye movement–based memory effects during both the scene cue and the three-face display, whereas highly confident inaccurate memories show eye movement–based memory effects only during the scene cueing period. This suggests that confidence in inaccurate memory may be related to overreliance on cue-related information without proper regard for the target recognition experience. Furthermore, these results show the power of eye movement monitoring in distinguishing memories of different qualities, particularly highly confident accurate and highly confident inaccurate memories, and in delineating what factors are influencing confidence judgments. These results are exciting and intriguing, with implications for evaluating confidence in eyewitnesses. However, these results are far from meeting the standard established in Daubert v. Merrell Dow Pharmaceuticals (1993) for admission of scientific evidence in courts. Further research is needed to replicate these findings and delineate limitations, or boundary conditions, in using this technique.
At least one other study has shown an effect of memory confidence on eye movements (Smith & Squire, 2008). In their paradigm, Smith and Squire showed novel and repeated scenes to younger and older adults while monitoring their eye movements. During test, participants indicated whether a scene was “definitely new,” “probably new,” “maybe new,” “maybe old,” “probably old,” or “definitely old.” Younger adults showed less sampling of repeated scenes compared to novel scenes (collapsed across correct and incorrect trials) that were given “definitely” and “probably” ratings but not “maybe” ratings. Older adults showed less sampling of repeated scenes only for those given a “definitely” rating. Both younger and older adults showed less sampling of (p.134) repeated scenes compared to novel scenes for correct trials but not incorrect trials (collapsed across confidence levels). Because they did not separately examine correct and incorrect trials across different levels of confidence, it is difficult to know if eye movements would be useful for distinguishing highly confident correct and incorrect recognition with this kind of setup and materials. Nevertheless, their findings, in combinations with those of Chua et al. (2012) mentioned earlier, suggest that there is a relationship between viewing behavior and confidence using multiple types of paradigms and that further delineation of these effects is warranted. Importantly, it is worth noting that in the Smith and Squire study, there are slightly different results from the two populations (younger and older adults). Given that eyewitnesses come from diverse age groups, future studies involving populations across a range of ages would be useful.
In addition to distinguishing highly confident accurate from highly confident inaccurate memory, another area of interest may be in distinguishing correct guesses from incorrect guesses. In other words, can eye movements reveal memory in the absence of awareness? We examined this issue in the above-mentioned face-scene paradigm (Chua et al., 2012) by examining viewing behavior during incorrect recognition trials. Specifically, we investigated whether or not participants spent more time viewing the correct face than would be expected by chance, even though participants failed to choose it during the explicit recognition task. In this paradigm, we did not see such an effect either collapsed across the entire trial or even in the earliest portions of the trial, suggesting that eye movements may be less useful for weak memories.
Whether or not eye movements reveal evidence of memory in the absence of awareness has been controversial, with some studies demonstrating effects in the absence of awareness (Ryan et al., 2000) and other studies showing eye movement–based memory effects only for aware memory (Smith, Hopkins, & Squire, 2006; Smith & Squire, 2008). The canonical paradigm in these studies involves comparing eye movements during repeated and manipulated scenes. In the manipulated scenes, an element of the scene was removed, and viewing behavior to this critical region, where the element was previously located, was measured. This was then compared to viewing of this same region in the repeated condition, when the scene was viewed both initially and at test without an element in the critical region. Some studies show that participants spend more time viewing the critical region in the manipulated condition only when they are aware of the change (Smith et al., 2006; Smith & Squire, 2008), and others show that this occurs regardless of awareness (Ryan et al., 2000). The major difference in these paradigms is the task instructions given to the participants. Although these results may have future applications, given that eye (p.135) movement–based memory effects in the absence of awareness are less robust and the emphasis of the legal system is on “innocent until proven guilty,” it would be imprudent to use eye movements for identifications that were not endorsed by the witness at this time. Further research is needed to determine the specific conditions under which eye movements reliably reveal memory effects in the absence of awareness.
General Limitations of Neuroimaging and Eye Movement Studies
Although there have been promising findings from neuroimaging (Chua et al., 2006, 2009; Kim & Cabeza, 2007; Moritz et al., 2006) and eye movement studies (Chua et al., 2012) of memory confidence that shed light on how people make confidence judgments on their memory, there are several limitations with these methods in terms of courtroom and lineup applications.
The first limitation is that the research findings reported are based on averaging across study participants and across trials; therefore, further work must be done to see if these findings are robust for eyewitness cases that would only involve a single trial and one individual. This is a greater problem for neuroimaging than for eye tracking because neuroimaging has a relatively poor signal-to-noise ratio. Indeed, researchers typically average over many trials across groups of subjects in order to have enough power to detect significant results. The eye movement monitoring study focused on in this chapter (Chua et al., 2012) also averages across subjects and trials; thus, it is unclear how robust the eye movement patterns related to confidence would be on an individual level. However, some eye tracking studies have reported results from single trials (Smith & Squire, 2008), suggesting that this method may be feasible for single trials. However, further testing is needed to determine the relationship between confidence, accuracy, and eye movements on a single-trial level.
Another limitation is that very few types of stimuli have been used to systematically examine confidence and accuracy in memory, and while these stimuli are appropriate for the laboratory, they are quite different from the experience of an eyewitness. First, the stimuli are quite simple and emotionally neutral. Although pictures of faces paired with names or scenes bear some similarity to photo lineups, the conditions under which study participants first encoded these faces is not. In the study, people viewed static images, but eyewitnesses viewed complex scenes and interactions. Indeed, there is some evidence that there are key differences in brain activation for autobiographical memories compared to laboratory memories (Cabeza et al., 2004). Furthermore, eyewitnesses likely viewed highly emotional events, whereas the studies here used (p.136) emotionally neutral events. Indeed, emotion has been shown to enhance the subjective feeling of remembering, which is related to confidence (Sharot, Delgado, & Phelps, 2004). There is substantial evidence that emotional memories show some differences in brain activation (Kensinger & Corkin, 2004; Kensinger & Schacter, 2005), although eye movement studies showed no significant difference in viewing behavior for emotional versus neutral scenes (Sharot, Davidson, Carson, & Phelps, 2008). Further work is needed involving testing participants under conditions that are more similar to eyewitness conditions.
Let’s return to the case of Dror Goldberg. After reading this chapter, would you convict him on the basis of Roberta Ingrado’s confident testimony? What would you want to know about her confidence in order to trust her reports? Dror Goldberg was convicted in 2000 and to date remains in prison. However, there is a task force consisting of people who believe he is innocent and are dedicated to appealing and overturning his conviction (http://drorgoldberg.org/). Eyewitness testimony was obviously not the only factor in his conviction, but it did have a role.
In 1996, the American Psychology/Law Society and Division 41 of the American Psychological Society appointed a subcommittee to review scientific research findings and make suggestions on the proper way to conduct lineups to minimize eyewitness misidentifications. The committee made three main recommendations: (1) conduct sequential lineups, (2) use a double-blind procedure, and (3) assess and record confidence at the time of the identification (Wells et al., 1998). These modifications to the traditional lineup have the potential to reduce overconfidence in faulty identifications, and although they were formed based on decades of applied eyewitness research, they are consistent with the metacognitive framework and evidence presented above, and they reduce potential contamination of confidence judgments from irrelevant sources. These recommendations have been implemented in some states, such as Wisconsin, New Jersey, and North Carolina (http://www.innocenceproject.org/fix/Eyewitness-Identification.php), which is a testimony to the benefits of research-based policy.
Efforts to reduce eyewitness misidentifications and overconfidence through policy are necessary early steps, and identifying markers of misidentifications and overconfidence is a good later step. In this chapter, I reviewed studies on memory confidence and accuracy using fMRI and eye tracking methods, and evaluated their potential utility for (1) distinguishing highly confident accurate and inaccurate memory and (2) identifying the factors that influence memory (p.137) confidence. Both fMRI and eye tracking show promising results, but significantly more research will need to be conducted over the next several years to validate their use in evaluation eyewitness confidence.
Althoff, R. R., & Cohen, N. J. (1999). Eye-movement-based memory effect: A reprocessing effect in face perception. Journal of Experimental Psychology: Learning, Memory and Cognition, 25(4), 997–1010.
Blank, H., Musch, J., & Pohl, R. F. (2007). Hindsignt bias: On being wise after the event. Social Cognition, 25(1), 1–9.
Bradfield, A. L., Wells, G. L., & Olson, E. A. (2002). The damaging effect of confirming feedback on the relation between eyewitness certainty and identification accuracy. Journal of Applied Psychology, 87(1), 112–120.
Brewer, N., Keast, A., & Rishworth, A. (2002). The confidence-accuracy relationship in eyewitness identification: The effects of reflection and disconfirmation on correlation and calibration. Journal of Experimental Psychology: Applied, 8(1), 44–56.
Cabeza, R., Prince, S. E., Daselaar, S. M., Greenberg, D. L., Budde, M., Dolcos, F., et al. (2004). Brain activity during episodic retrieval of autobiographical and laboratory events: An fMRI study using a novel photo paradigm. Journal of Cognitive Neuroscience, 16(9), 1583–1594.
Chua, E. F., Hannula, D. E., & Ranganath, C. (2012). Distinguishing highly confident accurate and inaccurate memory: Insights about relevant and irrelevant influences on memory confidence. Memory, 20(1), 48–62.
Chua, E. F., Schacter, D. L., Rand-Giovannetti, E., & Sperling, R. A. (2006). Understanding metamemory: Neural correlates of the cognitive process and subjective level of confidence in recognition memory. Neuroimage, 29(4), 1150–1160.
Chua, E. F., Schacter, D. L., & Sperling, R. A. (2009). Neural basis for recognition confidence in younger and older adults. Psychology of Aging, 24(1), 139–153.
Clark, S., & Davey, S. (2005). The target-to-foils shift in simultaneous and sequential lineups. Law and Human Behavior, 29(2), 151–172.
Cohen, N. J., Poldrack, R. A., & Eichenbaum, H. (1997). Memory for items and memory for relations in the procedural/declarative memory framework. Memory, 5(1–2), 131–178.
Cohen, N. J., Ryan, J., Hunt, C., Romine, L., Wszalek, T., & Nash, C. (1999). Hippocampal system and declarative (relational) memory: Summarizing the data from functional neuroimaging studies. Hippocampus, 9(1), 83–98.
Cutler, B., Penrod, S., & Stuve, T. (1988). Juror decision making in eyewitness identification cases. Law and Human Behavior, 12(1), 41–55.
Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 1993.
Davachi, L. (2006). Item, context and relational episodic encoding in humans. Current Opinion in Neurobiology, 16(6), 693–700.
Davachi, L., Mitchell, J. P., & Wagner, A. D. (2003). Multiple routes to memory: Distinct medial temporal lobe processes build item and source memories. Proceedings of the National Academy of Sciences of the United States of America, 100(4), 2157–2162. (p.138)
Diana, R. A., Yonelinas, A. P., & Ranganath, C. (2007). Imaging recollection and familiarity in the medial temporal lobe: A three-component model. Trends in Cognitive Science, 11(9), 379–386.
Douglass, A., & Steblay, N. (2006). Memory distortion in eyewitnesses: A meta-analysis of the post identification feedback effect. Applied Cognitive Psychology, 20(7), 859–869.
Gallo, D. A. (2006). Associative illusions of memory: False memory research in DRM and related tasks. New York: Psychology Press.
Goldberg v. State, 95 SW 3d.345—Tex: Court of Appeals, 2002.
Goense, J., & Logothetis, N. K. (2008). Neurophysiology of the BOLD fMRI signal in awake monkeys. Current Biology, 18(9), 631–640.
Gusnard, D. A., Akbudak, E., Shulman, G. L., & Raichle, M. E. (2001). Medial prefrontal cortex and self-referential mental activity: Relation to a default mode of brain function. Proceedings of the National Academy of Sciences of the United States of America, 98(7), 4259–4264.
Hall, C. C., Ariss, L., & Todorov, A. (2007). The illusion of knowledge: When more information reduces accuracy and increases confidence. Organizational Behavior and Human Decision Processes, 103(2), 277–290.
Hannula, D. E., Althoff, R. R., Warren, D. E., Riggs, L., Cohen, N. J., & Ryan, J. D. (2010). Worth a glance: Using eye movements to investigate the cognitive neuroscience of memory. Frontiers in Human Neuroscience, 4, 1–17.
Hannula, D. E., & Ranganath, C. (2009). The eyes have it: Hippocampal activity predicts expression of memory in eye movements. Neuron, 63(5), 592–599.
Hannula, D. E., Ryan, J. D., Tranel, D., & Cohen, N. J. (2007). Rapid onset relational memory effects are evident in eye movement behavior, but not in hippocampal amnesia. Journal of Cognitive Neuroscience, 19(10), 1690–1705.
Henson, R. N., Rugg, M. D., Shallice, T., & Dolan, R. J. (2000). Confidence in recognition memory for words: Dissociating right prefrontal roles in episodic retrieval. Journal of Cognitive Neuroscience, 12(6), 913–923.
Henson, R. N., Rugg, M. D., Shallice, T., Josephs, O., & Dolan, R. J. (1999). Recollection and familiarity in recognition memory: An event-related functional magnetic resonance imaging study. Journal of Neuroscience, 19(10), 3962–3972.
Kassin, S., Ellsworth, P., & Smith, V. (1989). The “general acceptance” of psychological research on eyewitness testimony. American Psychologist, 44(8), 1089–1098.
Kassin, S., Tubb, V., Hosch, H., & Memon, A. (2001). On the “general acceptance” of eyewitness testimony research. American Psychologist, 56(5), 405–416.
Kelley, C. M., & Lindsay, D. S. (1993). Remembering mistaken for knowing: Ease of retrieval as a basis for confidnece in answers to general knowledge questions. Journal of Memory and Language, 32(1), 1–24.
Kensinger, E. A., & Corkin, S. (2004). Two routes to emotional memory: Distinct neural processes for valence and arousal. Proceedings of the National Academy of Sciences of the United States of America, 101(9), 3310.
Kensinger, E. A., & Schacter, D. L. (2005). Retrieving accurate and distorted memories: Neuroimaging evidence for effects of emotion. Neuroimage, 27, 167–177.
Kneller, W., Memon, A., & Stevenage, S. (2001). Simultaneous and sequential lineups: Decision processes of accurate and inaccurate eyewitnesses. Applied Cognitive Psychology, 15(6), 659–671.
Koriat, A. (2008a). Subjective confidence in one’s answers: The consensuality principle. Journal of Experimental Psychology: Learning, Memory and Cognition, 34(4), 945–959.
Koriat, A. (2008b). When confidence in a choice is independent of which choice is made. Psychonomic Bulletin Review, 15(5), 997–1001.
Koriat, A., Nussinson, R., Bless, H., & Shaked, N. (2008). Information-based and experience-based metacognitive judgments: Evidence from subjective confidence. In J. Dunlosky & R. A. Bjork (Eds.), Handbook of metamemory and memory (pp. 117–135). New York: Psychology Press.
Lindsay, D. S., Read, J. D., & Sharma, K. (1998). Accuracy and confidence in person identification: The relationship is strong when witnessing conditions vary widely. Psychological Science, 9(3), 215–218.
Lindsay, R. C. L., & Wells, G. L. (1985). Improving eyewitness identifications from lineups: Simultaneous versus sequential lineup presentation. Journal of Applied Psychology, 70(3), 556–564.
Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature, 412(6843), 150–157.
Luus, C. A., & Wells, G. L. (1994). The malleability of eyewitness confidence: Co-witness and perseverance effects. Journal of Applied Psychology, 79(5), 714–723.
Manson v. Braithwaite, 432 U.S. 98 (1977).
Mazzoni, G., & Vannucci, M. (2007). Hindsight bias, the misinformation effect, and false autobiographical memories. Social Cognition, 25(1), 203–220.
McCabe, D. P., & Castel, A. D. (2008). Seeing is believing: The effect of brain images on judgments of scientific reasoning. Cognition, 107(1), 343–352.
Moritz, S., Glascher, J., Sommer, T., Buchel, C., & Braus, D. F. (2006). Neural correlates of memory confidence. Neuroimage, 33(4), 1188–1193.
Nelson, T. O. (1996). Gamma is a measure of the accuracy of predicting performance on one item relative to another item, not of the absolute performance on an individual item. Applied Cognitive Psychology, 10(3), 257–260.
Pannu, J. K., & Kaszniak, A. W. (2005). Metamemory experiments in neurological populations: A review. Neuropsychology Review, 15(3), 105–130.
Perfect, T. J., Hollins, T. S., & Hunt, A. L. (2000). Practice and feedback effects on the confidence-accuracy relation in eyewitness memory. Memory, 8(4), 235–244.
Pochon, J. B., Riis, J., Sanfey, A. G., Nystrom, L. E., & Cohen, J. D. (2008). Functional imaging of decision conflict. Journal of Neuroscience, 28(13), 3468–3473.
Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proceedings of the National Academy of Sciences of the United States of America, 98(2), 676–682.
Ranganath, C., Yonelinas, A. P., Cohen, M. X., Dy, C. J., Tom, S. M., & D’Esposito, M. (2004). Dissociable correlates of recollection and familiarity within the medial temporal lobes. Neuropsychologia, 42(1), 2–13.
Robinson, M. D., Johnson, J. T., & Herndon, F. (1997). Reaction time and assessments of cognitive effort as predictors of eyewitness memory accuracy and confidence. Journal of Applied Psychology, 82(3), 416–425.
Robinson, M. D., Johnson, J. T., & Robertson, D. A. (2000). Process versus content in eyewitness metamemory monitoring. Journal of Experimental Psychology: Applied, 6(3), 207–221.
Roediger, H. L., Jacoby, J. D., & McDermott, K. B. (1996). Misinformation effects in recall: Creating false memories through repeated retrieval. Journal of Memory and Language, 35, 300–318.
Roediger, H. L., 3rd, & McDermott, K. B. (1995). Creating false memories: Remembering words not presented on lists. Journal of Experimental Psychology: Learning, Memory and Cognition, 21, 803–814.
Rubenzer, S. J. (2002). Eyewitness identification: Challenging a confident witness and common misconceptions. Voice for the Defense (Journal for the Texas Criminal Defense Lawyers Association), 31(4), 20–23.
Ryan, J. D., Althoff, R. R., Whitlow, S., & Cohen, N. J. (2000). Amnesia is a deficit in relational memory. Psychology and Science, 11(6), 454–461.
Ryan, J. D., & Villate, C. (2009). Building visual representations: The binding of relative spatial relations across time. Visual Cognition, 17(1), 254–272.
Sanna, L. J., & Schwarz, N. (2007). Metacognitive experiences and hindsight bias: It’s not just the thought (content) that counts! Social Cognition, 25(1), 185–202.
Schacter, D. L., & Dodson, C. S. (2001). Misattribution, false recognition and the sins of memory. Philosophical Transactions of the Royal Society of London B: Biological Science, 356(1413), 1385–1393.
Schacter, D. L., Norman, K. A., & Koutstaal, W. (1998). The cognitive neuroscience of constructive memory. Annual Review of Psychology 49, 289–318.
Schwartz, B. L., & Bacon, E. (2008). Metacognitive neuroscience. In J. Dunlosky & R. A. Bjork (Eds.), Handbook of metamemory and memory (pp. 355–371). New York: Psychology Press.
Semmler, C., Brewer, N., & Wells, G. L. (2004). Effects of postidentification feedback on eyewitness identification and nonidentification confidence. Journal of Applied Psychology, 89(2), 334–345.
Sharot, T., Davidson, M. L., Carson, M. M., & Phelps, E. A. (2008). Eye movements predict recollective experience. PLoS One, 3(8), e2884.
Sharot, T., Delgado, M. R., & Phelps, E. A. (2004). How emotion enhances the feeling of remembering. Nature Neuroscience, 7(12), 1376–1380.
Shaw, J., III, Appio, L., Zerr, T., & Pontoski, K. (2007). Public eyewitness confidence can be influenced by the presence of other witnesses. Law and Human Behavior, 31(6), 629–652.
Smith, C. N., Hopkins, R. O., & Squire, L. R. (2006). Experience-dependent eye movements, awareness, and hippocampus-dependent memory. Journal of Neuroscience, 26(44), 11304–11312.
Sporer, S. L. (1993). Eyewitness identification accuracy, confidence, and decision times in simultaneous and sequential lineups. Journal of Applied Psychology, 78(1), 22–33.
Sporer, S. L., Penrod, S., Read, D., & Cutler, B. (1995). Choosing, confidence, and accuracy: A meta-analysis of the confidence-accuracy relation in eyewitness identification studies. Psychological Bulletin, 118(3), 315–327.
Steblay, N., Dysart, J., Fulero, S., & Lindsay, R. (2001). Eyewitness accuracy rates in sequential and simultaneous lineup presentations: A meta-analytic comparison. Law and Human Behavior, 25(5), 459–473.
Stretch, V., & Wixted, J. T. (1998). Decision rules for recognition memory confidence judgments. Journal of Experimental Psychology: Learning, Memory and Cognition, 24(6), 1397–1410.
United States v. Telfaire, 469 F.2d 552, 558-59 (D.C.Cir. 1979).
Weber, N., & Brewer, N. (2003). The effect of judgment type and confidence scale on confidence-accuracy calibration in face recognition. Journal of Applied Psychology, 88(3), 490–499.
Weber, N., Brewer, N., Wells, G. L., Semmler, C., & Keast, A. (2004). Eyewitness identification accuracy and response latency: The unruly 10–12-second rule* 1. Journal of Experimental Psychology: Applied, 10(3), 139–147.
Weisberg, D. S., Keil, F. C., Goodstein, J., Rawson, E., & Gray, J. R. (2008). The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience, 20(3), 470–477.
Wells, G. L., & Olson, E. A. (2003). Eyewitness testimony. Annual Review of Psychology, 54(1), 277–295.
Wells, G. L., Olson, E. A., & Charman, S. D. (2002). The confidence of eyewitnesses in their identifications from lineups. Current Directions in Psychological Science, 11(5), 151–154.
Wells, G. L., Olson, E. A., & Charman, S. D. (2003). Distorted retrospective eyewitness reports as functions of feedback and delay. Journal of Experimental Psychology: Applied, 9(1), 42–52.
Wells, G. L., & Seelau, E. (1995). Eyewitness identification: Psychological research and legal policy on lineups. Psychology, Public Policy, and Law, 1(4), 765.
Wells, G. L., Small, M., Penrod, S., Malpass, R., Fulero, S., & Brimacombe, C. (1998). Eyewitness identification procedures: Recommendations for lineups and photospreads. Law and Human Behavior, 22(6), 603–647.
Yokoyama, O., Miura, N., Watanabe, J., Takemoto, A., Uchida, S., Sugiura, M., et al. (2010). Right frontopolar cortex activity correlates with reliability of retrospective rating of confidence in short-term recognition memory performance. Neuroscience Research, 68(3), 199–206.
Yonelinas, A. P. (1994). Receiver-operating characteristics in recognition memory: Evidence for a dual-process model. Journal of Experimental Psychology: Learning, Memory and Cognition, 20(6), 1341–1354.
Yonelinas, A. P., Otten, L. J., Shaw, K. N., & Rugg, M. D. (2005). Separating the brain regions involved in recollection and familiarity in recognition memory. Journal of Neuroscience, 25(11), 3002–3008.
Zaragoza, M. S., & Mitchell, K. J. (1996). Repeated exposure to suggestion and the creation of false memories. Psychological Science, 7(5), 294–300.