In the mid-2010s, a well-regarded anxiety lab published a series of experiments suggesting that individuals with high trait anxiety showed overgeneralized threat responses. The measure was fear-potentiated startle, a reflex that grows when a person anticipates something aversive. The finding fit neatly into an emerging narrative: anxiety disorders involve a failure to distinguish safe from dangerous cues. Reviews cited it. Funding proposals leaned on it. Then, a few years later, the same lab tried to build on that work and hit a wall. When they adopted a standardized protocol borrowed from a consortium studying acoustic startle, the effect shrank. The original verdict, broad overgeneralization, did not survive the reanalysis. The result was not a dramatic null, but a quieter, more unsettling one: the conclusion had been partly an artifact of how the experiment was run.

A Single Protocol Flip Reversed a Threat Generalization Verdict

The original experiments used a classic differential fear conditioning design. Participants saw colored squares; one (the CS+) sometimes co-terminated with a loud noise or a mild shock to the wrist, while another (the CS-) never did. The next day, or sometimes within the same session, the lab measured startle eyeblink responses to a burst of white noise presented while the squares were on screen. The difference in startle magnitude between CS+ and CS- is the fear-potentiated startle effect. Generalization was assessed by presenting novel squares that varied in similarity to the CS+, from near-identical to completely different.

In the original data, the generalization gradient was flat. Participants with high trait anxiety startled strongly to all squares, even those that shared little with the danger cue. The lab interpreted this as a trait vulnerability marker. But the protocol had several features that later turned out to matter. The intertrial interval varied unpredictably, stimulus intensities were calibrated differently across participants, and the analysis pipeline was developed post hoc, after looking at the data.

When the same lab switched to a standardized protocol, the changes were subtle but systematic. Stimulus intensities were matched across participants using a pretest calibration, intertrial intervals were fixed, and the analysis steps were pre-registered. The result was a steeper generalization gradient: high trait anxiety individuals still showed some overgeneralization, but the effect size dropped by more than half. The original verdict, that generalization was broadly impaired, was revised to a more modest claim: generalization is slightly broader in anxious individuals, but not dramatically so.

The flip was not a debunking. It was a recalibration. The lab did not conclude that threat generalization is irrelevant to anxiety. Instead, they concluded that the magnitude of the effect depends heavily on procedural choices. The story illustrates a tension in neuroscience: subtle methodological differences can produce qualitatively different conclusions, and the field is only beginning to grapple with that.

Why Threat Generalization Tests Are Inconsistent Across Labs

Threat generalization is a core construct in anxiety research, but its measurement is anything but standardized. Labs differ in the number of stimuli, the similarity spacing between them, the duration of the conditioned stimulus, and the timing of the startle probe. Some use a 500-millisecond lead interval between the cue and the startle probe; others use 4 seconds. These parameters change the shape of the generalization gradient.

Sample sizes are often small, typically 20 to 40 participants per group. With such numbers, a single outlier can shift the mean. Effect sizes in this literature are modest, often in the range of Cohen's d = 0.3 to 0.5. Detecting those reliably requires larger samples than most labs run. Publication bias, where null results go unpublished, compounds the problem.

Species differences add another layer. Rodent studies often use fear-potentiated startle with a conditioned stimulus paired with footshock. Human studies use shock or loud noise, but the aversive intensity varies widely. A shock that is uncomfortable for one person may be painful for another, and calibration methods differ. Some labs use a subjective rating scale; others use a fixed intensity. These choices affect the baseline startle and the magnitude of potentiation.

Analysis pipelines vary too. Some labs compute a difference score, others a ratio, others a percent change. Some exclude non-responders, others do not. The lack of common pipelines means that two labs analyzing the same raw data could arrive at different conclusions. This is not a hypothetical concern; it has been demonstrated empirically in other domains of psychology. For instance, a well-known project that asked several teams to analyze the same fMRI dataset found that no two teams chose the same pipeline, and the resulting maps of brain activity varied substantially. A similar exercise with startle data would likely yield comparable variability.

The Original Verdict: Generalization Was Broad

The original finding had a clear narrative appeal. It suggested that anxiety is characterized by a failure to inhibit fear responses to safe cues, a mechanism that could explain why anxious individuals avoid many situations. The study was published in a high-impact journal and has been cited several hundred times. It became a cornerstone of the idea that threat generalization is a transdiagnostic endophenotype.

But the protocol had a flaw that only became apparent in hindsight. The intertrial interval varied between 15 and 30 seconds, which is common in the field. Yet this variability can affect the baseline startle and the degree of habituation. In the standardized protocol, the intertrial interval was fixed at 20 seconds, reducing this source of noise.

Another issue was the stimulus intensity calibration. In the original study, the shock level was set to each participant's tolerance, but the startle probe was a fixed 95-decibel burst. In the standardized protocol, the probe intensity was adjusted to produce a comparable baseline startle across participants. This reduced between-subject variance and increased the sensitivity to detect differences in generalization.

The original analysis also used a median split on trait anxiety scores, which can inflate effect sizes by capitalizing on noise. The reanalysis used the full continuous scale and found a weaker relationship. The lab acknowledged that the median split was a poor choice, but by then the damage was done: the broad-generalization claim had entered the literature.

Methodological Reanalysis: What Changed in Practice

The reanalysis was not a single change but a bundle of changes. The lab adopted a protocol that had been published by a consortium studying acoustic startle, which specified parameters in detail. The first change was stimulus calibration: each participant underwent a brief procedure to set the intensity of the startle probe to produce a reliable eyeblink response. This was done before the conditioning phase, and the same intensity was used throughout.

The second change was the intertrial interval. Instead of a variable interval, the lab used a fixed 20-second interval between trials. This reduced the influence of temporal uncertainty on startle reactivity. The third change was the number of trials: the standardized protocol included more trials per condition, which improved the reliability of the startle estimates.

The fourth change was pre-registration. The lab specified the analysis plan in advance, including the primary outcome measure (the difference in startle magnitude between CS+ and CS-) and the secondary measures (the generalization gradient slope). They also committed to reporting effect sizes and confidence intervals, not just p-values.

The fifth change was blinding. The researchers who scored the startle responses were unaware of the participant's trait anxiety level. In the original study, the scorers were not blinded, which could have introduced subtle bias. These changes, taken together, reduced noise and bias, and the result was a more precise estimate of the true effect.

The Flipped Result: Narrower Generalization Emerged

With the standardized protocol, the generalization gradient became steeper. The difference in startle magnitude between the CS+ and the most dissimilar novel stimulus was smaller, but still significant. The key finding was that high trait anxiety individuals did not show the same flat gradient as before; they showed a gradient that was only slightly shallower than that of low anxiety individuals.

The effect size for the group difference in generalization dropped from a Cohen's d of about 0.8 in the original study to about 0.3 in the reanalysis. A d of 0.8 is considered large; 0.3 is small to medium. The original finding was not a false positive in the strict sense, but it was an overestimate. The reanalysis suggested that the effect is real but much weaker than initially reported.

Statistical significance did not survive the reanalysis for some secondary measures. For example, the difference in generalization to a specific intermediate stimulus was no longer significant after controlling for multiple comparisons. The lab chose to report these nulls transparently, which is a departure from common practice.

The flipped result has implications beyond this one lab. It suggests that many published findings in this area may be inflated. A recent multi-lab replication effort in a related domain, fear conditioning, found that only about half of the effects replicated. The threat generalization literature has not been subjected to the same scrutiny, but the pattern is likely similar.

What This Means for Translational Anxiety Biomarkers

The threat generalization measure has been proposed as a biomarker for anxiety disorders, a tool to identify at-risk individuals and to track treatment response. The reanalysis does not invalidate this idea, but it complicates it. A biomarker that is sensitive to procedural details is less reliable for clinical use.

If the effect size is smaller than originally thought, the sample sizes needed to detect it in a clinical trial are larger. A study powered to detect a large effect would be underpowered to detect a small one, leading to false negatives or false positives depending on the direction of the bias. The field needs to reach a consensus on the protocol before moving forward with biomarker development.

Consortia efforts, such as the one that produced the standardized protocol, are a step in the right direction. They bring together multiple labs to agree on parameters and to share data. But these efforts are expensive and slow, and they face resistance from labs that have invested in their own protocols. The tension between innovation and standardization is real.

Some researchers argue that the field should embrace heterogeneity, because real-world anxiety is heterogeneous. They worry that over-standardization could lead to a narrow, artificial measure that does not capture the full spectrum of threat responses. This is a legitimate counterargument. The challenge is to find a balance between flexibility and comparability.

Practical Steps Toward Reproducible Threat Research

One practical step is to adopt shared protocols, at least for core measures. The fear-potentiated startle field has several published protocols; labs should choose one and report deviations. Another step is to publish analysis code. This allows other labs to re-run the analysis and to check for errors. The lab in this story made their code available on a public repository, which was a key factor in the reanalysis.

Pre-registration is another low-cost intervention. By specifying the analysis plan in advance, researchers reduce the temptation to analyze data in multiple ways and report only the results that are significant. Journals should encourage pre-registration by offering badges or by making it a condition for publication.

Effect sizes and confidence intervals should be reported alongside p-values. This allows readers to assess the magnitude of the effect and to compare results across studies. The lab's reanalysis reported effect sizes, which made the drop from 0.8 to 0.3 transparent.

Data sharing across labs is more difficult but potentially transformative. If raw data from multiple labs are pooled, the combined sample can be large enough to detect small effects reliably. The field is moving in this direction, with repositories like the Open Science Framework, but adoption is slow.

Finally, researchers should be willing to revisit their own conclusions. The lab in this story did not defend their original finding; they changed their protocol and reported the new result. This is how science should work, but it is rare. The threat generalization verdict is not settled, but it is now on firmer ground.

Broader Implications: Lessons for Other Domains

The lessons from this episode extend beyond threat generalization. Similar stories have unfolded in other areas of behavioral neuroscience. For example, in the study of fear extinction, a phenomenon closely related to generalization, early reports suggested that a specific genetic variant predicted extinction deficits. Later attempts to replicate found inconsistent results, and a meta-analysis concluded that the effect, if present, is much smaller than initially claimed. The same pattern of inflated effect sizes due to small samples and flexible analysis has been documented in studies of the startle response itself.

Consider the case of prepulse inhibition, a measure of sensorimotor gating often studied in schizophrenia research. A large multi-site consortium was formed to standardize the measurement across laboratories, because single-site studies showed wide variability in effect sizes. The consortium found that by carefully controlling stimulus parameters and analysis pipelines, they could achieve reliable effects, but only after substantial effort. This mirrors the experience of the anxiety lab described here.

Another relevant example comes from the field of cognitive neuroscience, where a widely cited study claimed that a particular brain region responded selectively to faces. Subsequent reanalyses with larger samples and more rigorous methods revealed that the effect was much weaker and more distributed than originally thought. The original study had used a small sample and a liberal analysis threshold, leading to an overestimate. These cases illustrate a systemic issue: the scientific reward structure often favors novel, positive findings over rigorous, reproducible ones.

The solution is not to abandon the pursuit of new discoveries, but to embed rigor into the process from the start. This means training researchers in open science practices, providing incentives for replication, and fostering a culture where admitting a mistake is seen as a strength, not a weakness. The lab in this story has become an example of this culture, and their willingness to share their journey has been influential.

As the field moves forward, it is crucial to remember that the ultimate goal is not to produce a single, universal protocol, but to understand the conditions under which threat generalization operates. By embracing methodological diversity, but also demanding transparency and reproducibility, the field can build a more solid foundation for translating basic research into clinical practice.