HARKing Explained: Why Hypothesizing After Results Can Mislead Research

A clinical trial wraps up. The primary outcome comes back null, but one subgroup responds better to the treatment than the control group did. Nobody had predicted that subgroup effect. Nobody wrote it into the protocol.

Then the paper gets revised. The Introduction frames the subgroup effect as the research hypothesis all along. The failed primary outcome fades into the background. By peer review, the study reads like a clean success.

This is HARKing: Hypothesizing After the Results are Known. Norbert Kerr coined the term in 1998 for cases where researchers present post-hoc hypotheses as if they had been set before the study began. Sometimes the original hypothesis is rewritten. Sometimes it is left out.

Exploration is normal. The problem is presenting an exploratory finding as a confirmatory one. Readers lose the warning label: early evidence, not proof.

What Is HARKing in Research?

Kerr's original definition is still the clearest starting point: HARKing occurs when a researcher presents "a post hoc hypothesis... in one's research report as if it were, in fact, an a priori hypothesis." An a priori hypothesis is formed before data collection. A post hoc hypothesis is formed after seeing the results. The problem is the label, not always the analysis itself.

Kerr also described a second form that gets less attention: leaving out original hypotheses that were tested and came back negative. Both forms hide what actually happened. Over time, theories can start to look more accurate than they are.

Types of HARKing

Researchers can split HARKing into a few types.

• CHARKing (Constructing HARKing) is the closest to what most people picture. A researcher finds an unexpected significant result, builds a new hypothesis to explain it, and presents that hypothesis as the starting point of the study. The theory is built to fit the data.
• SHARKing (Secretly HARKing) works differently. Instead of building a new hypothesis, the researcher removes evidence that an original prediction was made and failed. The paper appears to contain only positive results. Failed predictions disappear.
• THARKing (Transparent HARKing) is the exception. Hollenbeck and Wright used this term in 2017 for post-hoc hypotheses that are clearly labeled as exploratory. Readers can then judge the result for what it is. More on this shortly.

Why Researchers Engage in HARKing

Publication bias is one driver. Journals have often favored positive, statistically significant results over null findings. When tenure, funding, and department status depend on publication count and journal prestige, presenting data more favorably can start to look rational.

The pressure does not always start with the researchers. Passive HARKing happens when editors or reviewers ask authors to reframe a study around an unexpected finding, or to remove hypotheses that did not pan out. That leaves the researcher with a hard choice.

There is also an unconscious path. Hindsight bias can cause researchers to believe they expected a result they did not expect. Memory fades, and the post-hoc story fills the gap. Some cases are not deliberate deception.

Survey evidence from psychology gives some sense of the scale. John, Loewenstein, and Prelec found that roughly 27% of psychologists said they had claimed to predict an unexpected finding from the start. A review of several surveys put some form of HARKing at around 43%. These are self-reported figures, so they may be low.

HARKing vs. Exploratory and Confirmatory Research

This is where the article can easily sound harsher than it should. Exploration is not the issue. The issue is whether the paper treats exploration as if it were a planned test.

Exploratory analysis

Exploratory analysis is what happens when you look at data without a firm prior claim about what you expect to find. You examine distributions, spot patterns, compare variables, and get a feel for the dataset. The goal is to generate hypotheses, not to test them.

I find this distinction useful because it keeps the criticism in the right place. A lot of scientific progress begins with someone noticing something unexpected and deciding it deserves another look.

Confirmatory analysis

Confirmatory analysis is the testing stage. You start with a specific prediction formed before seeing the data, run the statistical test you planned, and check whether the data supports the prediction. The hypothesis came first. The data came second.

When a hypothesis is generated by looking at a dataset and then tested against the same dataset, the test is no longer independent. The hypothesis was shaped by the same noise present in that sample.

That is the warning from the introduction in more technical terms: the label changes, but the test does not.

Good tests put hypothesis before data. Image by Author.

The practical rule: the same dataset cannot fairly generate and then test the same hypothesis.

Examples of HARKing

The examples below are made-up scenarios, not documented cases. The point is the pattern, not accusing a particular study.

• The psychology experiment: A team tests whether mindfulness improves cognitive performance. The main effect is not significant, but participants over 50 show a strong positive effect. In the final paper, the age-group finding becomes the original hypothesis, and the null main effect becomes a side note.
• The marketing analysis: A company tests whether a loyalty program increases purchase frequency. The overall effect is marginal, but recent members show a clear increase. The final report presents that group as the planned focus, without saying it was found after the fact.
• The clinical trial: A medical study tests a new drug on a pre-specified primary outcome. The drug does not outperform the control, but a secondary biomarker improves. The study is revised to frame the biomarker as the primary hypothesis. As I will cover later, this is why trial registries ask researchers to record primary outcomes before data collection begins.

Why HARKing Can Distort Research Findings

The most direct result is an inflated false positive rate. When a hypothesis is formed by looking at the same data used to test it, the statistical test is not checking an independent prediction. It is checking how well a hypothesis built to fit the data fits the data. That result may not repeat because it is fitting the noise, not the signal.

Kerr listed twelve possible costs in the original paper. A few stand out. HARKed results get built into theory as if confirmed. Negative results get suppressed. Published work starts to look cleaner than the research process actually was.

Dorothy Bishop described HARKing as one of the "four horsemen of the reproducibility apocalypse," alongside publication bias, low statistical power, and p-hacking. None of these alone explains the state of published research. Together, they help explain why some findings are less reliable than their p-values suggest.

How HARKing Relates to the Replication Crisis

The replication crisis refers to the finding that many published results cannot be reproduced by independent teams. In the 2015 Reproducibility Project: Psychology, 97% of the original studies had statistically significant results, but only 36% of the replications did.

That gap is where HARKing can matter. A study built around a post-hoc hypothesis may fit the noise of one sample. In a new sample, the effect may shrink or disappear.

HARKing is a contributing factor, not the cause. It sits alongside p-hacking, publication bias, low statistical power, and small samples. Mark Rubin has argued that the evidence for HARKing as the main driver is weaker than often claimed. That view is debated, so I would keep the claim modest: HARKing is a plausible mechanism, not a proven single cause.

Either way, practices that hide the exploratory nature of results make the literature less reliable.

HARKing vs. P-Hacking

These two practices often show up together in discussions of research integrity. They are not the same thing.

P-hacking manipulates the analysis. It means changing data collection or analysis choices until a statistically significant result appears: stopping once p drops below 0.05, trying several analyses, dropping outliers, or testing many subgroups without correction. The hypothesis stays fixed; the analysis bends.

HARKing manipulates the hypothesis instead. The analysis might be standard. What changes is what the paper claims was predicted. The clinical trial from the introduction is a clean example: the biomarker finding was real, but the story around it was not.

The Texas Sharpshooter metaphor helps draw the boundary.

HARKing moves targets; p-hacking hides misses. Image by Author.

As covered earlier, both can inflate false positive rates. Both can also happen without conscious intent to deceive.

How Preregistration and Registered Reports Reduce HARKing

Most fixes for HARKing make the timing of the hypothesis visible. The two main approaches are preregistration and Registered Reports.

Preregistration

Preregistration means recording your research question, hypotheses, study design, and planned analysis before any data is collected. These documents go to time-stamped registries such as the Open Science Framework (OSF), ClinicalTrials.gov, or AsPredicted. Once posted, they show what was predicted before the results were known.

If the results differ from the plan, that change has to be acknowledged, not quietly erased. Unexpected results can still be reported, but labeled as exploratory.

Preregistration is not a complete fix. Many preregistered studies still change from the original plan, and vague registrations offer less protection. There is also a workaround called PARKing, pre-registering after already knowing the results. Preregistration makes that harder to hide, not impossible.

Registered Reports

Registered Reports go a step earlier by involving the journal before data collection begins.

In a standard pipeline, editors evaluate the completed result, so positive findings often have an easier path to publication.

Registered Reports split the process into two stages. In Stage 1, the journal reviews the research question and methods before data collection. If accepted, the journal promises to publish the results as long as the approved plan is followed. Stage 2 checks the results and adherence to the plan.

Review happens before results are known. Image by Author.

The effect shows up in the positive result rate. Scheel, Schijen, and Lakens found positive results in roughly 96% of traditional journal articles, compared with around 44% of Registered Reports in the same fields.

Over 300 journals now offer Registered Reports. Nature announced in June 2026 that it was extending the format across all the fields it publishes.

Transparent reporting practices

Not every research design suits strict preregistration, and not every journal offers Registered Reports. In those situations, clearer reporting can still reduce HARKing risk. The practices below are norms rather than requirements, but they are increasingly common in fields where replication problems have been most visible.

• Label any analysis that was not planned in advance as exploratory when reporting it
• Report all outcomes that were originally measured, including those that showed no effect
• Include a clearly marked exploratory findings section when unexpected results are worth reporting
• Share analysis code and data so others can check the work
• Report any deviation from a preregistered plan explicitly and with an explanation

These norms are not a replacement for preregistration or Registered Reports. They are the same basic principle applied without the formal process: make it harder to hide what was predicted and what was discovered.

HARKing-Like Problems in Data Science and Machine Learning

So far, I have used academic research examples. The same basic problem appears in machine learning and data science work, too, usually under different names.

The clearest parallel is data leakage. In machine learning, leakage occurs when information from the test set influences training. Common forms include selecting or engineering features using the full dataset before the train-test split, or tuning model hyperparameters by repeatedly peeking at test-set performance. The result is a model that looks good on the benchmark but fails in real use, because its score was partly built on data it should not have seen.

Kapoor and Narayanan at Princeton documented this problem across hundreds of studies in fields ranging from medicine to economics. The parallel to HARKing is direct: the thing being tested has been shaped by the same data it is then measured against.

Leakage enters long before model evaluation. Image by Author.

ML researchers also describe a practice sometimes called Grad Student Descent or SotA-hacking. Researchers run many experiments until the model edges out the current benchmark, then write the paper as if the winning setup came from a clean design argument. This is CHARKing applied to a machine learning pipeline.

Post-hoc metric selection is the same pattern in a different form: evaluate a model on multiple metrics, then decide after seeing all the results which one to present as the primary measure. What looks like a planned methods choice was made after the fact.

One caveat: the term HARKing itself is rarely used in machine learning. Similar problems are usually discussed under reproducibility, benchmark gaming, or evaluation methods. The parallel helps, but it is still an analogy.

Conclusion

HARKing is a labeling problem as much as a research problem. A post-hoc finding might be sound, but presenting it as a priori removes what readers need to judge it.

Preregistration, Registered Reports, and clear reporting all help by making the timing of the hypothesis visible. None is a complete fix. Preregistration can be vague, and Registered Reports still cover only a fraction of published research.

For researchers and data professionals, the practical rule is simple: write down what you predicted before you look at the data, report what you found, and label exploratory results as exploratory.

For more on the testing side of that workflow, see our course on Hypothesis Testing in Python.