Testing for Bias in a Photo Lineup
AbstractYou may have read about criminal cases where innocent people have been wrongly convicted of a crime. Sometimes, modern DNA analysis techniques have provided the evidence to exonerate these innocent people. In many cases, mistaken identification by eyewitnesses provided strong evidence for the original conviction. How can prosecutors and defense attorneys make sure that photo lineup procedures used to identify criminal suspects are unbiased? This project shows you how to conduct an objective test for bias in a simulated photo lineup.
Andrew Olson, Ph.D., Science Buddies
The procedure used for this project was adapted from:
- Malpass, R.S., date unknown. "A Lineup Evaluation 'Do-It-Yourself Kit' for Attorneys and Law Enforcement," Eyewitness Identification Research Laboratory, University of Texas at El Paso [accessed July 28, 2006] http://eyewitness.utep.edu/Documents/DIY%20Kit.pdf, and its accompanying Excel spreadsheet: http://eyewitness.utep.edu/Documents/bias-calc.xls.
The goal of this project is to test a photo "lineup" for bias. How can you tell if the "suspect" is being treated fairly?
Eyewitness accounts are continuously put into question in the court room. Photo lineups, from which a witness identifies a suspect in a criminal case, are one of the tools used for corroborating eyewitness accounts. A photo lineup is also designed to protect innocent people from being caught in a criminal investigation. If the witness cannot identify the suspect from a panel of six (or more) photographs, the reliability of their recall of the crime is called into question.
How can a prosecutor show that a photo lineup was fair (unbiased)? How can a defense attorney back up an argument that a photo lineup was biased against her client? This project shows you how to conduct an objective test of the fairness of a photo lineup.
Terms and Concepts
To do this project, you should do research that enables you to understand the following terms and concepts:
- forgetting curve,
- confidence intervals.
- What are some of the factors that affect the ability of an eyewitness to a crime to identify the suspect?
- The Wikipedia article on human memory is a good place to start, and also has many suggestions for further reading:
Wikipedia contributors, 2006. Human Memory, Wikipedia, The Free Encylopedia. Retrieved July 27, 2006.
- Campbell, T.W., 2005. Issues in Forensic Psychology: Eyewitness Recall,. Retrieved July 27, 2006.
- For basic information on lineup construction procedure and evaluating fairness of a lineup, see:
Staff, 2006. Eyewitness Identification Research Laboratory, University of Texas El Paso. Retrieved July 28, 2006.
- This project is based on the procedure described in this paper:
Malpass, R.S., date unknown. A Lineup Evaluation 'Do-It-Yourself Kit' for Attorneys and Law Enforcement, Eyewitness Identification Research Laboratory, University of Texas at El Paso. Retrieved July 28, 2006.
- Any introductory statistics textbook should contain an explanation of how confidence intervals are calculated. Here is an online source:
Staff, 1997. Confidence Intervals, Department of Statistics, Yale University. Retrieved July 28, 2006.
Materials and Equipment
To do this experiment you will need the following materials and equipment:
- mugshot gallery for creating photo lineups:
- All photos should be same size, similar viewpoint.
- You will need one "suspect" photo.
- You will need five "filler" photos for each lineup you wish to create.
- The people in the photos should be unfamiliar to your witnesses who create the description, and to the mock witnesses who view the photo lineups.
- You can make these yourself using volunteers, or
- perhaps you could use a camera to make copies of old yearbook photos.
- 2–3 witnesses to view and describe suspect photo, you will create a written description of the "suspect" from their composite description;
- 20–50 mock witnesses to view each lineup,
- lab notebook for recording results,
- calculator or computer and spreadsheet program to analyze the results.
Note: There are special considerations when designing an experiment involving human subjects. ISEF-affiliated fairs often require an Informed Consent Form for every participant who is questioned. In all cases, the experimental design must be approved by the fair's scientific review committee (SRC) prior to the commencement of experiments or surveys. Please refer to the ISEF rules for additional important requirements for studies involving human subjects: https://www.sciencebuddies.org/science-fair-projects/competitions/human-subjects-regulations.
- Instruct your witnesses that they will have, e.g., 20 seconds to examine a photograph and that they will then have to write a physical description of the person in the photograph.
- Separately for each witness, show photo of "suspect" and get their written description.
- Create a composite written description of the "suspect" using features identified in at least two of the written descriptions. This is the written description that your mock witnesses will use to try to identify the "suspect" from a photo lineup.
- For each photo lineup you want to test, you will need 5 "filler" photos. Try to predict which of your lineups will be unbiased, and which will be biased.
- Number the photos 1–6, so that your witnesses can select the "suspect" by number.
- Do the following steps individually with each mock witness:
- Have them read the composite written description of the "suspect."
- Take the written description away, and then show them the photo lineup.
- Ask them to identify the "suspect" (by number) and record the results.
- For each lineup, calculate the number of times each photo (1–6) was identified. Divide this number by the total number of choices made. If the "fillers" were chosen perfectly, you'd expect that each photo would have an equal probability of being chosen. That is, each photo would be chosen 1/6th of the time. How do your results compare to the ideal?
- You can also calculate the frequency with which the "suspect" was chosen from the lineup, and calculate the probability that this frequency differs from chance. We'll work an example to show you how:
- Write down the number of mock witnesses, n, who viewed the photo lineup: 29.
- Write down the number of mock witnesses who chose the "suspect": 14.
- Write down the number of photos used in the photo lineup: 6.
- Calculate the proportion, p, of mock witnesses who chose the "suspect": 14/29 = 0.483.
- Calculate the proportion, q, of mock witnesses who chose "fillers": q = 1 − p = 1− 0.483 = 0.517.
- Calculate the standard error, s.e., of p:
- Expected proportion for choosing suspect by chance: 1/n = 1/6 = 0.167.
- Critical ratio for difference from chance: (p − chance expectation)/s.e. = 0.483 − 0.167 / 29 = 3.406.
- For a 95% confidence interval (i.e., only a 5% chance that the simulated lineup is biased), the critical ratio must be less than 1.96.
- For a 99% confidence interval (i.e., only a 1% chance that the simulated lineup is biased), the critical ratio must be less than 2.58.
Ask an Expert
This project explores topics key to Peace, Justice and Strong Institutions: Promote just, peaceful and inclusive societies.
- Use the same photo lineup, but show it to two groups of mock witnesses. In one case, show the photos one-by-one (sequential photo lineup), in the other case, show all six photos at once (simultaneous photo lineup). Is there any difference in frequencies?
- Double-blind procedure (person showing photos does not know which is suspect) vs. suggestive feedback (confirmation when witness correctly identifies "filler" photos)
- For another experiment related to forensic psychology, see the Science Buddies project Testing the Accuracy of Eyewitness Testimony.
If you like this project, you might enjoy exploring these related careers: