Munim Technologies

Research/Medicine

Do published aflatoxin numbers recompute from their own open data?

We pre-registered checks of three aflatoxin studies from Kenya and Senegal against the data their authors deposited. 53 of 66 checks held, and most of the misses come down to definitions rather than errors.

Nov 15, 2026 · 7 min read

Aflatoxins are toxins made by moulds that grow on maize and groundnuts, and long-term exposure is linked to liver cancer and to poor child growth. Surveys and trials that measure them shape food-safety policy in much of Africa. We asked a simple question of three such studies whose authors had published their data openly: does each paper's numbers recompute from its own file? We wrote down every check and our predictions before reading the data, then ran sealed jobs to grade them. 53 of 66 checks held. The Kenyan maize-flour survey reproduced almost perfectly; the Kenyan trial reproduced its main arm means; the Senegal groundnut pilot did worst on our pre-registered rules, largely because we misread its file layout. We report that as a failure of our pre-registration, not of the data.

53 of 66

pre-registered checks held across three studies

28 of 29

Kenya maize-flour survey

20 of 25

Kenya cluster randomised trial

5 of 12

Senegal groundnut pilot

The question

Open data are meant to let anyone verify a result. In practice, few people try, and when they do, the attempt is rarely written down before the data are opened. Food-safety surveys are a good place to start: their headline numbers are mostly counts, shares and means that can be recomputed exactly, and they inform limits and interventions that affect what people eat.

What we did

For each study we filed a pre-registration listing every number we would check, the rule for whether it held (for shares, the computed value rounded to the decimals the paper prints must equal the paper's number), and our predictions. A sealed job waited until that file was public, downloaded the deposited data, checked its hash, ran the code and wrote out every statistic. Anything we looked at after grading is labelled post hoc and never changes a grade.

The three checks

StudyDeposited dataChecksHeld
Hoffmann et al. 2025, aflatoxin in 1,255 maize-flour samples, ten Kenyan townsIFPRI, Harvard Dataverse2928
Hoffmann, Jones and Leroy 2018, cluster trial of aflatoxin-free maize and child growth, 56 villagesIFPRI, Harvard Dataverse2520
Feed the Future pilot, groundnuts stored by 250 households, SenegalBauchet 2021, Harvard Dataverse125
Sources: results/kenya_aflatoxin_check.json, results/kenya_aflatoxin_rct_check.json, results/senegal_groundnut_check.json.

Kenya maize flour: one miss, and a likely reason

The open file reproduced the sample counts, all 60 cells of the paper's table by mill location, every mean and every share above Kenya's 10 ppb limit. The one miss was the headline share of samples with detectable aflatoxin: 97% in the paper, 94.4% at the paper's stated detection limit of 0.5 ppb. The share of samples with any non-zero reading is 96.97%, which rounds to the paper's figure. The difference is 32 samples recorded above zero but below 0.5 ppb. That is one reading that fits; the paper does not say how those values were counted.

Kenya maize flour: share of samples with detectable aflatoxin

1,255 analysed samples. The 0.5 ppb rule was pre-registered; the above-zero share was reported but not graded. Source: results/kenya_aflatoxin_check.json

Senegal groundnuts: our layout error

The survey answers and the coliform share reproduced, and every log10 column matched its raw counts. The two aflatoxin shares did not under our rule, which took one value per household. After grading we found the file has one row per sample, with two aflatoxin samples for most households. Counted per sample, as the published summary says it counted, both shares reproduce exactly: 267 of 308 samples below the EU limit of 4 ppb (86.7%, printed as 87%) and 25 of 308 above 100 ppb (8.1%). That reading is post hoc and changes no grade. Smaller open points remain: no count of the file's 184 samples gives the published 94.5% Enterobacteriaceae share, censored aflatoxin values are coded 101 where the codebook says 100, and five colony averages are not the mean of their two plates.

Senegal groundnuts: published shares against the pre-registered recomputation

CheckPublishedRecomputedHeld
Knew of aflatoxins20%20.00%yes
Knew of bacterial contamination22%22.04%yes
Aflatoxin below EU limit (per household)87%85.55%no
Aflatoxin above 100 ppb (per household)8%9.83%no
Coliforms above 1 log CFU/g97%96.74%yes
Enterobacteriaceae above 1 log CFU/g94.5%95.11%no
Households eating raw groundnuts7%7.58%no
Household rule as pre-registered. Counted per sample (post hoc), both aflatoxin shares reproduce exactly. Source: results/senegal_groundnut_check.json.

Kenya trial: a column labelled log10 that is not

The trial's file reproduced its randomisation (56 villages, 28 per arm, none mixed), its loss to follow-up, and 16 of the 17 arm means we checked from its descriptive tables, including child growth and serum aflatoxin at midline and endline. The misses were bookkeeping: arm counts off by one and two children, a mother-height mean just outside tolerance that the open file cannot reproduce because most heights are missing, and 17 non-detectable serum samples that were left out rather than imputed, split almost evenly between arms. The most useful finding for anyone reusing the data is a column. The mother serum column labelled "log10" holds log10 in survey waves 2 and 3 and the natural log in waves 4 to 6. The paper's own mother numbers use the natural log of the raw values and are unaffected.

Kenya trial: which logarithm the mother "log10" serum column holds, by survey wave

WaveMatches log10 onlyMatches natural log onlyMatches both (value near 1)
218100
313401
402874
503000
602981
1,206 mother rows with a serum value, each matched within 0.01. Post hoc diagnostic, not pre-registered. Source: results/kenya_aflatoxin_rct_posthoc.json.

What it means

These are well-documented open datasets, and their headline numbers largely stand. Most misses come down to definitions: which readings count as detectable, and whether a share is counted over samples or households. The rest are small labelling or entry slips that matter mainly to people who reuse the files. We did not check the trial's adjusted treatment effects, which need the authors' covariate coding. Each point is going to the authors and data publishers as a question, not a claim of error.

More research