First things first: Thanks for participating in the forensic perception experiment!
This private page just gives you a little bit of background to the experiment and its motivation, and a sneak preview of the results.
Important note: Please don’t share or quote this page or even discuss the experiment with anyone who has not done it themselves!
Two reasons:
- This page gives preliminary results intended only for participants, as thanks for doing the experiment. I will check and refine the results before publishing them officially.
- I will run a follow-up experiment soon, so will be seeking participants who don’t know the information provided below (maybe you can help me find some?).
Background
This audio is taken from the recent movie released to mark 20 years since the murder of 6-year-old beauty queen JonBenét Ramsey.
You can read a quick intro to the murder, the audio and the movie, as well as listen to original and ‘enhanced’ versions of the audio, at this link (highly recommended).
The audio you listened to in the experiment comes from the very end of the emergency phone call made by JonBenét’s mother, Patsy, when she discovered her daughter was missing. The movie makers allege that their ‘enhancing’ of this little snippet of audio has revealed utterances made by the family after they thought Patsy had hung up – and that the utterances incriminate the family as being involved in the murder.
One quick request
Hardly anyone in the experiment recognised the audio – which surprised me a bit because I thought the movie was quite well known.
If you have seen the movie (or otherwise heard the audio) but did not recognise it when you did the experiment – would you please let me know? That would be useful information to me.
Aim
Many many viewers were persuaded by the movie that the ‘enhancing’ had indeed revealed the incriminating utterances it suggested.
That matters – not just for this case (recall that I have no opinion as to the guilt of the family), but because similar things happen in many many other cases.
People really believe in the power of ‘enhancing’ to make unintelligible audio clear – with very poor understanding of what is actually involved in ‘enhancing’, or what effect it really has.
So one big aim of the experiment was to test the objective effects of the ‘enhancing’ carried out in the movie (this will make more sense if you have followed the link to the background info – still recommended!).
Materials
The experiment used the snippet of audio from the end of Patsy’s 911 call in ‘original’ and ‘enhanced’ versions (as shown in the movie).
Method
To achieve the aim, the experiment first asked participants to listen to each version ‘cold’, i.e. with no context or background information. This gives a baseline for how intelligible each version is. Participants then listened under various other conditions described below.
Summary results (preliminary)
Around 78 participants completed the experiment. Unbeknownst to you, you were randomised into two groups.
Group 1 (41 participants) started by listening ‘cold’ (i.e. without any background information) to the original audio (as portrayed in the movie).
Group 2 (37 participants) started by listening ‘cold’ to the ‘enhanced‘ audio (as portrayed in the movie – if you haven’t yet, you really need to follow that link to understand this distinction!).
Neither group was told the status of their audio as ‘original’ or ‘enhanced’.
So what did each group say they heard?
Of the 41 in Group 1 who listened cold to the ‘original’ audio, only 3 (7%) heard any words at all, and none of the words were remotely like those the movie makers allege are contained in the audio.
Of the 37 in Group 2 who listened cold to the ‘enhanced’ audio, 10 (27%) heard some words.
That is substantially more than Group 1 – and might at first suggest the ‘enhancing’ had been effective.
That’s until you realise that those 10 participants all heard completely different words, hardly any of which even made coherent phrases, and only one was remotely (not very) like any of the phrases suggested by the movie.
So conclusion number one is that the ‘enhancing’ techniques displayed by the movie did not reveal words objectively present in the audio.
Which raises the next important question …
… if the allegedly incriminating words were not revealed by the ‘enhancing’, how come so many people hear them in the movie?
This can be readily explained by the power of priming. For those who might not know, priming is a fascinating phenomenon whereby audio that is objectively unintelligible is heard as containing intelligible words when listeners have those words suggested to them – even if the words are not objectively present in the audio. Never heard of it? Try this 90-second video to experience it for yourself, and get an impression of how dangerous it can be when it affects indistinct audio used as forensic evidence in a criminal trial – a topic explored in detail by this site you are currently on.
Now let’s be clear: the power of priming has already been demonstrated quite extensively. We don’t need a whole experiment just to show that priming was at play in the JonBenét movie!
The present experiment aimed to use the JonBenét audio to go beyond what has already been shown, and the results below are quite interesting in that regard.
Had enough?
If you are new to this area, that might well be all you want to know about the results of this experiment! In that case, your next step should be to grab a cuppa and wander over to Rethink Speech – Forensic – Enhancing for some further background and information presented in a fairly light-hearted way (considering what a serious topic it is!). The rest of the experiment results will probably make more sense when you have some of that under your belt.
However, if you are familiar with priming and forensics, and/or have strength for some more in-depth discussion of the experiment results, by all means stick around here.
Recall that this experiment aimed to achieve results with significance far beyond the role of priming in this particular movie
The second aim was to look at the power of ineffective ‘enhancing’ to make a misleading transcript even more believable than it would have been just on the basis of normal priming.
So now that we’ve demonstrated the movie’s ‘enhancing’ was objectively ineffective, let’s go on to look quickly at what happened when each group was primed with the phrases suggested in the movie – and then beyond that to what happened in the final step. Note: Not everyone answered every question, so the numbers don’t always add up to 100%. And anyway I haven’t finished a detailed count of how many participants heard each phrase at each step.
Priming in Group 1
Recall that Group 1 started by listening cold to the original version of the audio – and that hardly anyone heard any words at all.
At the next step, Group 1 listened to the same, original, audio again, this time primed with the four phrases suggested by the movie-makers. The priming caused one participant to hear one of the phrases ‘clearly’. A few more heard one or other of the phrases, or something similar, but ‘not clearly’. Most continued to hear nothing at all.
This is a very weak priming effect, indicating that the words suggested by the movie are implausible as an account of what is contained in the audio.
What about priming in Group 2?
Group 2 also started by listening cold, but they (unwittingly) listened to the ‘enhanced’ version – and, as we saw above, without priming, no one heard the words the movie claimed were spoken, indicating the ‘enhancing’ had had no objective effect.
But what happened when Group 2 were primed with the movie’s alleged phrases?
Maybe you guessed it? Far more participants in Group 2 (listening to ‘enhanced’ audio) than in Group 1 (listening to ‘original’ audio) said they ‘clearly’ heard the phrases when they were suggested.
That might seem like good news for the movie makers: they could argue it shows their ‘enhancing’ really revealed those phrases (though recall even with the ‘enhanced’ version, the phrases were only ‘revealed’ by the priming).
But here’s the interesting thing. When we asked Group 2 to indicate the order they heard the phrases in, they all put them in different orders! And none were the order the movie suggested. So maybe we have to conclude they didn’t really hear the phrases quite as clearly as they thought they did.
So what made them think the phrases were clear when they actually weren’t?
Well, it’s the same effect as created the results we saw above for step 1. Recall that Group 2 participants, listening cold to the ‘enhanced’ version, were substantially more likely than Group 1 (27% as opposed to 7%) to hear some words (though not the words the movie makers intended, or even the same words as other participants).
Here, after priming, listening to the ‘enhanced’ version makes Group 2 substantially more likely than Group 1 to feel they hear the movie’s words – even though these words were not heard by anyone listening cold to either version (which isn’t surprising, as there is absolutely no acoustic evidence for their existence).
These results show that the ‘enhancing’ had made the audio seem clearer, without actually making it more reliably intelligible.
This apparent clarity makes listeners believe there are words to be heard – even if, objectively, there aren’t! And it makes them more susceptible to accepting suggested words even if the suggestion is highly misleading.
I think you’ll agree that is a big problem in a forensic context – and understand why it was an effect we wanted to explore with this experiment.
But wait, there’s more!
You might remember, after we suggested the movie’s phrases to you, we went on to give you an ‘enhanced’ version of the audio.
So what happened at that step?
In Group 1, who went from original to ‘enhanced’, three said they found this version ‘much clearer’ (that’s what the movie makers say: the enhanced version is much clearer, to the point it reveals the incriminating words). But sixteen of you said you only found it ‘somewhat clearer’ and twelve said it was ‘not clearer at all’.
So that looks like the ‘enhancing’ didn’t have much effect after all. Well, that’s until you look at perception of the phrases at this step. So let’s do that now.
Substantially more Group 1 participants now heard one or more of the movie phrases (listening to the ‘enhanced’ version) compared to when the phrases were first suggested in the previous step (with the original audio).
So now, that goes back to looking like a win for the movie-makers, by indicating their ‘enhancing’ made their phrases clearer. Well, that’s until you make another interesting comparison.
Hey, not so fast there!
Let’s review that last result. Group 1, having been primed with the movie’s phrases, and then given the ‘enhanced’ audio, heard the movie’s phrases more clearly (which seems like a win for the movie-makers).
But now compare this last result with what participants in Group 2 heard at step 1, when they listened to the audio cold. Recall that Group 2 started (unknowingly) with the ‘enhanced’ version, and that, listening cold, none of them heard the movie phrases (which were only ever heard, and only by some participants, after they were suggested).
That’s what gave us our conclusion number one (above): the movie’s ‘enhancing’ had no objective effect.
But now we are seeing the ‘enhancing’ might have had a pretty important subjective effect.
Participants in Group 1 heard the phrases in the ‘enhanced’ version more clearly than they had (after priming) in the original – even though we know from earlier results that it was not objectively clearer.
So where does this added clarity come from?
The comparison between groups shows that what made the ‘enhanced’ phrases seem ‘clearer’ to Group 1 was not the ‘enhancing’ on its own, but the ‘enhancing’ in conjunction with the priming effect of the phrases suggested at the previous step.
So the conclusion here is that the movie’s ‘enhancing’ made their suggested phrases more credible, without actually making the words objectively clearer.
Now – just one last twist (can you stand it?!)
It’s time to look at results for Group 2 at the step where they went from the priming to the enhanced version.
And I’m sorry now to have to confess that we tricked Group 2 a tiny bit here.
While Group 1 really went from original to ‘enhanced’, Group 2 just heard the same audio again (recall they had been listening to the ‘enhanced’ version all along, without being told it was ‘enhanced’). So for Group 2, the only real difference at this step was that you were led to believe you were listening to an enhanced version. On that belief, you might reasonably expect it would be clearer than what you had heard before – but in fact it was exactly the same.
Not all of Group 2 were fooled, not by a long way. Well over half of you found the audio to be ‘not clearer at all’. But a quarter of you said the audio was now ‘somewhat clearer’, and one said it was ‘much clearer’.
More importantly, the (non-existant) ‘enhancing’ led more of you to hear the movie phrases ‘clearly’ than at the previous step (where the audio was exactly the same, but you did not have the belief it was enhanced’).
Why is that important? Well, earlier on we showed that Group 1, hearing the ‘enhanced’ version after priming, were more likely to hear the movie phrases than Group 2 hearing the ‘enhanced’ version cold. That indicated the effect of the ‘enhancing’ might have been largely subjective.
Now with these Group 2 results, we know for sure that the improvement in clarity was brought about by belief the audio was enhanced. Because the audio was actually identical. So any effect of the ‘enhancing’ must be entirely subjective.
Sorry to have tricked you!
I hope you can see the reason for it. We need to show the world how easily people can be fooled by so-called ‘enhancing’, and what a dramatic effect it can have in making a completely misleading transcript seem quite credible, to the point where millions of listeners believe they ‘hear the words with their own ears’.
This is pretty important in a context where our courts routinely admit ‘enhanced’ audio to ‘assist’ juries in evaluating transcripts of indistinct covert recordings used as evidence in criminal trials.
Conclusion
So all in all, this experiment has shown three things:
- the ‘enhancing’ in the JonBenét movie had no objective effect on the clarity of the audio – the phrases they say were revealed by ‘enhancing’ were really suggested by priming;
- the ‘enhancing’ did have some effect on making the audio seem clearer, making the primed phrases more believable than they should have been;
- the mere belief that audio has been ‘enhanced’ boosts the credibility of the primed phrases.
While there is a lot more work that could be done on this audio, I think this demonstration is enough to add weight to existing evidence (see references below) that urges the courts to show far greater caution regarding admission and use of ‘enhanced’ audio (usually done in combination with a police transcript).
Enough for now!
Believe it or not there is actually more to say about this audio, but I think that’s about enough for now, don’t you agree?
If you are a beginner on this topic – congratulations on getting this far! You might now like to review some of the material at Rethink Speech – Forensic – Enhancing, but of course that is entirely optional!
If you are a scholar in this area – try my new paper: “Enhancing” forensic audio: false beliefs and their effect in criminal trials. Australian Journal of Forensic Sciences (2018), which describes two different experiments making similar points, and discusses ‘enhancing’ in more detail.
Looking for something kind of in-between? The video of my talk about ‘enhancing’ to the Australian Academy of Forensic Sciences is highly recommended, despite being quite alarming!
And finally – thanks again for participating in this forensic perception experiment!
I hope you find your effort has proved worthwhile. Any comments or questions – please feel welcome to contact me here.