This video investigates a compelling statistical anomaly observed in an online Zener card test designed by mathematician Ben Sparks. Initial analysis of tens of thousands of trials, particularly a 25-card version, revealed a significant deviation from the expected binomial distribution, with an underrepresentation of low scores and an overrepresentation of higher-than-random correct guesses, seemingly suggesting mild psychic ability.
However, after ruling out biases in the random number generator, coding errors, or overt cheating, the video demonstrates that this statistical "psychic" effect is a result of "rage quit survivor bias." Participants who performed poorly in the early stages of longer tests were significantly more likely to quit before completing the trials, leading to only the better-performing (or luckier) trials being recorded. Subsequent tracking of unfinished games confirmed that quitting participants consistently had lower-than-average scores.
The video concludes that while there's no evidence for human psychic abilities, there is robust statistical evidence for a human propensity to "rage quit." It uses this finding to highlight the broader importance of Ben Goldacre's "All Trials" campaign, advocating for the publication of all research data—positive or negative—to prevent skewed conclusions in fields like medical research. A final revelation exposes a fundamental flaw in the original test's programming: the "random" card was selected after the user's guess, meaning the test was inadvertently assessing the ability to predict a future random event rather than sensing a hidden card.
Summarizing the Statistical Evidence for "Mild Psychic Ability"
0:06 Initial Claim: The video opens by posing the question: Do we have mathematical proof that humans can be mildly psychic, based on statistical evidence?
0:49 Zener Cards & Probability: The Zener card test, using five distinct symbols, presents a 20% chance of guessing correctly for each card.
2:19 Ben Sparks' Online Test: Mathematician Ben Sparks created an online Zener card test (5, 10, or 25 guesses) for teaching statistics, which accidentally gained widespread popularity, accumulating over 27,000 trials.
6:44 Binomial Distribution Explained: The video explains the binomial distribution as the theoretical expected outcome for random guessing, which shows how often specific numbers of correct guesses should occur over many trials.
10:32 Results for 5 and 10-Card Trials: Data from the 5 and 10-card trials closely matched the expected binomial distribution for random chance, showing no evidence of psychic ability.
12:48 Anomaly in 25-Card Trials: The 25-card trial data, however, showed a significant deviation: fewer low scores (0-4 correct) and more medium-to-high scores (5-9 correct) than statistically expected, giving the impression of better-than-random performance.
14:10 Ruling Out Other Causes: Matt Parker and Ben Sparks ruled out biased random number generators, code bugs, and explicit cheating (as the distribution shape remained binomial, albeit shifted).
15:17 The "Rage Quit Survivor Bias" Revelation: The true cause was "rage quit survivor bias." The website only recorded data if a user completed all trials. People who started doing poorly in the longer 25-card game were more likely to quit before finishing, removing their low scores from the recorded data set.
16:09 Statistical Proof of Quitting Behavior: Ben started tracking unfinished games, finding that participants who quit had significantly lower average scores (8-12% correct) compared to those who completed games (20-21% correct). When all data (finished and unfinished) was combined, the average returned to the expected 20%.
18:38 Modeling Rage Quitting: Ben developed a Python model showing that a 3% chance of quitting on any wrong guess accurately replicated the skewed distribution observed in the 25-card test.
20:44 Broader Implications: "All Trials" Campaign: The findings underscore the importance of publishing all research data, including negative or unfavorable results, a concept championed by Ben Goldacre's "All Trials" campaign in medical research to prevent misleading conclusions.
21:35 Fundamental Flaw in Original Test Design: It's revealed that due to programming simplicity, the "random" Zener card was generated after the user made their guess. This meant the test wasn't assessing psychic ability (sensing a hidden card) but rather the ability to predict a future random event.
22:15 Sponsor Message: Jane Street Internships: The video is sponsored by Jane Street, promoting their global paid internships in quantitative fields for students with curious and collaborative minds.
24:07 Personal Anecdote: Matt Parker shares an anecdote about a game show where he sometimes got Zener card guesses right, playfully linking it to the video's theme.