Detecting Bots in Human Activity Logs

1. Overview

bot-detection/ holds five generators that each write the same {activity, subject, timestamp} record shape, and build_dataset.py mixes their output into one anonymized CSV with a separate answer key — that's the exercise students actually run detection against. This page is the explainer behind it: what each generator does mathematically, what its output looks like, and which statistical test catches it. The five tiers are built to fail in a specific order, from "regular enough that a single glance catches it" to "built specifically to survive that glance" to "calibrated against this page's own test battery so that glance doesn't even help."

2. Reading the diagrams

Every tier below gets the same two charts, generated from the actual generator code (not hand-drawn), so shape differences are real, not artistic license.

• Timeline scatter — 12 consecutive days as rows, hour-of-day (00–24) as the x-axis. Each dot is one logged event. The two shaded bands mark 00:00–06:00 and 22:00–24:00 as a rough "most people are asleep" reference window — a lot of dots inside those bands with no bias toward the daytime hours is itself a signal.
• Hour-of-day histogram — the same generator run for 365 days, all events pooled into 24 one-hour bins. Each histogram is scaled independently to its own tallest bar — compare the shape (flat vs. peaked vs. spiky) across tiers, not the raw bar heights, since the underlying event counts differ per tier.

All diagrams and every number quoted in this page use N = 3 events/day, a fixed start date, and a fixed seed, purely so every tier is compared under identical conditions. The pooled-histogram window is a full {rd.HIST_DAYS}-day year rather than a few months — Tier 5's holiday-calendar check (§7) needs at least one full year to see every US federal holiday once. The actual exercise dataset (build_dataset.py) randomizes N per subject from 1–8.

3. Tier 1 — Fixed interval

simulate_fixed_interval() in naive_bots.py places every event at (start_hour + k · 24/N) % 24 for k in 0..N-1, identically, every single day. There is no random term anywhere in the function — given the same N and start_hour, every day of output is byte-for-byte the same clock time.

Statistics theory: coefficient of variation

CV = σ / μ — the standard deviation of the inter-event gaps divided by their mean. Measured from a 365-day / N=3 run of this generator:

A CV of exactly zero isn't "unusually low," it's a mathematical impossibility for any continuous real-world timing process. No hypothesis test is even required here — a single stdev(gaps) == 0 check settles it.

4. Tier 2 — i.i.d. uniform

simulate_iid_uniform() draws each event's time-of-day i.i.d. — independent and identically distributed: every draw comes from the same Uniform(0, 24h) distribution, and knowing one event's time tells you nothing about the next. There's no memory of previous events at all. The inter-event gaps now have real variance (CV = 0.759) — close enough to the human tier's own gap variance (CV = 0.761) that Tier 1's test is no help here at all, so a gap-spread check alone won't catch this one — the tell is when events happen, not how far apart they are. (Tier 3's daily jitter is i.i.d. too, just Gaussian instead of uniform — the hard-mode tier's AR(1) drift in §5–6 is the one generator on this page that's explicitly not i.i.d., and that single difference is what its autocorrelation-based tests are built to detect.)

Statistics theory: chi-square goodness-of-fit

Bin the 365-day pooled events into 24 hourly bins and test against Uniform(0,24): χ² = Σ (Oᵢ − Eᵢ)² ⁄ Eᵢ, with 23 degrees of freedom (24 bins − 1). Each Oᵢ is itself approximately Poisson for a rate-based arrival process, and a sum of squared standardized differences of approximately-normal-for-large-counts bins converges to a chi-square distribution — the same reasoning that makes chi-square the standard tool for "does this histogram match an assumed shape." Under a true uniform null, E[χ²] = df: the expected value of the statistic equals the degrees of freedom, and the exact upper-tail probability (p-value) comes from the chi-square distribution's survival function, computed here with a stdlib-only regularized-incomplete- gamma routine (no scipy), validated against the textbook critical value χ²(df=23) = 35.17 → p = .05 before trusting it for anything else. Measured from real 365-day runs:

5. Tier 3 — Gaussian jitter, no drift

simulate_gaussian_no_drift() places N circadian-shaped anchor hours once, using the same build_daily_anchors() density-sampling code the hard-mode generator uses, then fires every anchor every single day with fresh, independent Gaussian jitter (σ = 9 minutes). Same realistic shape as a real routine — but with no memory from one day to the next, and never a missed occurrence.

How to detect it

Two signals survive, both about structure across days rather than within a single day:

1. Daily event count has zero variance. Real adherence isn't perfect — people miss a dose, skip a login, forget a check-in. This generator fires every anchor every day without fail: mean 3.000, sd 0.000 across 365 days (human tier: mean 2.436, sd 0.863). A subject who never once misses their routine, across hundreds of days, is itself a degenerate-distribution tell — the same CV = 0 idea from Tier 1, just applied to count per day instead of gap length.
2. No day-to-day autocorrelation. A real routine drifts in streaks — a busy week nudges everything twenty minutes later for several days running — while independent daily jitter resets to nothing each morning. Track one anchor's daily deviation from its mean time and compute the lag-1 sample autocorrelation: r₁ = Σ(xₜ − x̄)(xₜ₊₁ − x̄) ⁄ Σ(xₜ − x̄)².

Statistics theory: lag-1 autocorrelation and its significance band

From this illustrative 45-day run: AR(1) series r₁ = 0.588, i.i.d. white-noise series r₁ = 0.170. A standard rule of thumb (the same one behind the shaded bands on an ACF plot in any time-series package) draws a significance band at roughly ±2 ⁄ √n around zero — here, ±0.298 for n = 45 observations. The AR(1) series clears the band by roughly 2×; the white-noise series stays inside it.

Going deeper: one lag isn't the whole picture

Lag-1 autocorrelation only asks "does yesterday predict today?" An AR(1) process actually leaves a trace at every lag — the true autocorrelation at lag k is ρᵏ, decaying geometrically but never hitting exactly zero. Two tools use that more fully:

Q = n(n+2) Σₖ rₖ² ⁄ (n−k) for k = 1..10 — this is the formal, properly-calibrated version of "eyeball the ACF band": instead of one lag with an approximate normal band, it's a chi-square test that spends its power across all 10 lags simultaneously, which is why its p-value for the AR(1) series (6.37e-05) is far more decisive than the lag-1 test alone. The runs test throws away the magnitude of each day's deviation and looks only at whether it's above or below the median — a distribution-free check that doesn't lean on the ±2⁄√n heuristic's approximate-normality assumption at all, and it independently agrees: 14 runs in the AR(1) series is fewer than chance would produce (p=0.0042), while the white-noise series' 21 runs looks unremarkable (p=0.4527).

6. Tier 4 — Hard mode (human)

simulate_activity_log() in activity_simulator.py stacks nine sources of realism on top of Tier 3's circadian anchors: an AR(1) drift on each anchor (the same recurrence shown above), a weekend/weekday timing shift, per-slot adherence that's heterogeneous (each anchor gets its own fixed reliability, not one global rate — some slots are just flakier than others), whole days that get skipped entirely (a day-level miss correlated across every slot at once, not three independent coin flips landing tails together), higher miss odds on weekends at both the whole-day and per-slot level, a rare heavy-tail "life happened" delay, occasional bonus events, and a slow multi-week schedule creep. It was built, on request, to be as difficult as possible to separate from a real human — specifically by defeating each test on this page individually.

One more signal: does the week itself have a shape?

Every test so far treats each day as exchangeable. The hard-mode generator adds weekend_shift_hours = 1.1 to every anchor on Saturdays and Sundays — none of the three bot tiers know what a weekend is. But comparing a "weekday mean hour" to a "weekend mean hour" has a trap: hour-of-day is a circular quantity, and a plain linear average breaks the moment an anchor sits near midnight — 23:30 shifted 1.1h later becomes 00:36, which a linear mean reads as a huge swing in the wrong direction rather than the small forward shift it actually is. (An earlier draft of this page's numbers made exactly this mistake before it got caught by simply looking at the data.) The fix is a circular mean — average the unit vectors (cos θ, sin θ) for each event's angle, not the raw hour — and to only trust the result when the mean resultant length R (0 = no concentration, angles point every which way; 1 = perfectly clustered) says the mean means something:

The comparison that matters is gaussian vs. human: both have comparable concentration (R≈0.42 vs. R≈0.43), so the shift isn't an artifact of one being noisier than the other — gaussian genuinely shows none (0.02h) and human genuinely does (1.39h). Fixed and uniform's large-looking "shifts" are exactly the trap: R≈0 means their circular mean isn't measuring anything, the way an average compass bearing from directions pointing everywhere is not a bearing. (This page reports the shift as a descriptive effect size, not a formal p-value — a rigorous significance test for a difference between two circular means, e.g. Watson–Williams, needs more machinery than is built into these scripts.)

Is detecting this tier just "run the three tests"? No — and here's why that matters

Tier 1 fails interval-CV. Tier 2 fails χ² vs. uniform. Tier 3 fails count-variance, autocorrelation, and the runs test. This tier was built to pass all of those individually — so the instinct is to just run every test from this page and flag anything that fails any of them. That instinct is a reasonable starting heuristic, but it's statistically sloppy in a specific, fixable way.

The first problem is multiple comparisons. Run 5 independent tests, each individually calibrated to a 5% false-positive rate, and flag a subject if any one fires — the chance of wrongly flagging a genuine human is not 5%, it's 1 − (1 − 0.05)⁵ ≈ 22.6%. Every test you add to the pile makes false accusations more likely, not less, unless you correct for it — e.g. Bonferroni: test each signal at α ⁄ 5 = 0.01 instead of 0.05 to hold the combined false-positive rate near the original 5%.

The second, deeper problem is that voting throws away information. An unweighted "3 of 5 signals" rule treats a decisive Ljung-Box p-value of 6.37e-05 identically to a borderline chi-square reading, and treats a test with real discriminating power (autocorrelation, which Tier 3 is specifically built to fail) the same as one with almost none against this particular opponent (χ², which Tier 3 passes by construction). The statistically principled version of "combine the evidence" is a likelihood ratio: the Neyman–Pearson lemma says that for two fully specified hypotheses, the likelihood-ratio test is the most powerful test achievable at any given false-positive rate — no voting scheme beats it, even in principle. In the wild you rarely know the bot's exact generating distribution, so the practical approximation is a discriminative model — a logistic regression is the simplest one — fit on a feature vector (interval CV, χ² p-value, daily-count sd, Ljung-Box Q, runs-test z, weekday/weekend shift) against labeled examples. That's exactly what build_dataset.py's answer key is for: it turns "which combination of these signals actually separates human from bot" from a guess into something you fit.

One more wrinkle worth naming, not solving here: several of these tests need real history per subject to have any power (§11's cold-start problem). The principled fix for that isn't a fixed-sample test at all — it's a sequential one, like Wald's SPRT, which accumulates the log-likelihood ratio as each new day of data arrives and stops the moment the evidence crosses a threshold, rather than waiting for a fixed n before testing anything.

7. Tier 5 — Ultra-hard (adversarially calibrated)

simulate_ultra_hard_log() in ultra_hard.py starts from Tier 4's persona and adds five defenses. Four are aimed at a specific technique from the bot-detection literature; the fifth is a direct, common-sense response to a specific question worth asking about every tier on this page: what happens on a US federal holiday?

1. Adversarial self-calibration. Rather than hand-picking constants, a short search tries 12 candidate parameter sets on a 90-day trial run and keeps whichever lands every statistic in this page's own battery (interval CV, χ²/df, daily-count sd, weekday/weekend shift) inside a human-plausible band at once — the practical version of what Davis et al.'s multi-feature Botometer and Jin & Offutt's evasion research both demonstrate: tuning one axis at a time isn't enough against a real detector.
2. Per-subject noise-distribution diversity. Tier 4 always dithers with the same Gaussian+log-normal-tail shape. This subject drew gamma as its primary jitter family (Gaussian, Weibull, or Gamma, chosen per subject) — so a detector can't fingerprint the generator itself by noise shape across many accounts, the gap Jin & Offutt's evasive-bot framework targets.
3. Complexity floor. Pan et al. found bot accounts have measurably lower complexity in their posting-time sequences than humans. This generator compresses its own rolling inter-event gap sequence with zlib and treats a too-low compressed/raw ratio as a signal to inject more incompressible jitter. This run's final ratio: 1.092 (comfortably above the 0.5 floor). This guard is a genuine backstop, not inert decoration — a sequence of near-constant gaps (every gap within about a minute of the same value) verifiably crosses the floor at a ratio of ~0.475 and would trigger it. It just never fires here, because this generator's own parameters (multiple circadian anchors, 5-14 minute jitter) never produce gaps that regular to begin with — a real run typically lands around 1.05-1.09, not because the guard is doing the work, but because it was already true before the guard checked.
4. Non-reuse guard. Nagaraja & Shah's Clicktok catches clickspam by detecting reused click timing. A generative, continuous-noise simulator like this one has a measure-zero chance of reusing a pattern by accident, so this guard has limited practical bite here — it's included because it's cheap and because the honest failure mode (a purely generative model can't really be caught the way a traffic-replay bot can) is worth naming rather than glossing over.
5. US federal holiday awareness. None of the other four tiers know what a calendar is. A real 9-to-5 office worker misses federal holidays outright, frequently takes the "bridge day" next to one (the Friday after a Thursday holiday, the Monday before a Tuesday one), and is a little more likely to be away the adjacent weekend when a holiday already forms a natural 3-day weekend. This one isn't from the literature — it's a direct answer to a specific, sharp question worth asking of every tier on this page: what does this subject's miss rate look like strictly on the federal holiday calendar?

How close is it, really?

The sixth — weekday/weekend shift — differs by 5.4% here, and that number deserves more suspicion than the other five, not less. Run Tier 4 alone across 8 different seeds, with no Tier 5 involved anywhere: its own shift ranges from 0.39h to 1.30h — a swing of 234% within a single generator, before Tier 5 enters the comparison at all. This is an inherently noisy statistic at single-subject scale — one weekend-vs-weekday split, driven by AR(1) drift interacting with wherever a randomly-placed circadian anchor happens to land — so a one-seed-vs-one-seed percentage on this axis isn't a fair measure of calibration quality the way the other five are. It's reported here anyway, honestly, rather than left out because it's inconvenient.

Does IS/IV actually mean anything at this event density?

Lin et al. computed IS and IV from densely-sampled smartphone data — 288 five-minute epochs a day. This page's tiers produce a few discrete events a day, binned into 24 hourly buckets — a much sparser signal, and worth asking directly whether these clinical metrics still measure anything real at that density, rather than assuming they do because the formula runs without error. Computed for every tier at once:

IV turns out flat across every tier (2.28–2.08) and doesn't discriminate much at this density — fair to be skeptical of it here. IS is a genuine surprise: it separates the tiers cleanly, just not along the axis this section originally assumed. Fixed interval scores exactly 1.000 — the maximum possible value, because "identical clock time every day" is literally what interdaily stability measures, and Tier 1 is definitionally perfect at it. Gaussian (0.81) is also high — no drift means the daily profile really is nearly identical from one day to the next. Human and ultra-hard sit far lower (0.18 and 0.17) because AR(1) drift is, by construction, a day-to-day profile that doesn't stay the same. Uniform sits near zero (0.003), correctly reflecting that it has no daily pattern to be stable or unstable about.

Put plainly: this section originally treated IS as a diagnostic that "can't discriminate anything" at this event density, extrapolated from a single Tier 5 run in isolation rather than tested against the other tiers. That claim doesn't survive contact with the actual cross-tier numbers above — IS is quietly one of the cleanest single-number separators of "has real day-to-day drift" from "doesn't" on this whole page, and Tier 3's own detection story (§5) could have used it directly instead of, or alongside, autocorrelation. Worth remembering generally: an untested claim about what a metric "would" show is a hypothesis, not a finding, even when it sounds plausible — this page tries to hold itself to that standard everywhere else, and this was a place it briefly didn't.

What does this subject do on a federal holiday?

None of the tests above ever look at the calendar. Bin miss rate by what kind of day it actually was — a US federal holiday, the office-worker "bridge day" next to one, a weekend a holiday extends into a 3-day weekend, or an ordinary day — across the full 365-day run, for every tier at once:

Ultra-hard's own numbers land close to what the mechanism was built to produce: a 0.727 holiday miss rate against a 0.030 baseline — roughly a 24.2× jump — and the single observed bridge day landed a miss too. The holiday-weekend "lesser effect" category (0.200, n=20) came in below the ordinary-day rate this run, despite the underlying per-day probability genuinely being higher (≈0.18 vs. ≈0.07 — verified directly against the generator's own calculation, not just read off this one sample) — with only 20 such days in a single year, that's within ordinary sampling noise, not a bug. It's also a fair preview of why the federal-holiday column (n=11) is the only one of these four with enough events to trust as a real per-subject signal within one year at all — the smaller categories would need several years of history before a single subject's rate means much on its own.

How to detect it: what the calibration search didn't check

Two more honest gaps remain, both because the search only scores four of this page's statistics — it never re-ran Ljung-Box or the runs test on each candidate's own generated series (extracting a clean per-anchor offset series from a 90-day trial is noisy enough that doing so reliably was out of scope here), so those two tests were never directly optimized against, only indirectly protected by keeping ρ and the jitter scale inside literature-plausible ranges. And second: aggregate enough events for one subject and a normality test (Shapiro–Wilk or Anderson–Darling) on that subject's own dithering residuals would catch it outright two-thirds of the time — Tier 4 always dithers Gaussian; two of Tier 5's three noise families aren't Gaussian at all. Neither gap is exploitable with a handful of days of data, which is exactly the cold-start problem in §11 again, just one level further out — and now joined by a third, calendar-shaped version of the same problem: the holiday categories above need a year of history before they mean anything, not a handful of days.

Does Tier 5's population cluster too tightly? A population-level test

Everything above compares exactly one Tier-5 subject to exactly one Tier-4 subject. A detector with access to many accounts at once could look for something no single-subject test can see: the calibration search always picks parameters that land safely inside the human-plausible bands — so maybe every Tier-5 account clusters suspiciously close to the "plausible middle," while a real population has natural heterogeneity. Worth testing directly rather than left as a hypothesis.

Generated 60 independent subjects per tier (fixed target_n = 3, so posting-rate differences can't leak into the result — only behavioral-shape differences can), and compared the across-subject standard deviation of four statistics already used elsewhere on this page. A ratio near 1 means no detectable difference; well below 1 would mean Tier 5 really is unusually uniform. Because a variance ratio from a couple dozen samples per side is itself a noisy estimate, each ratio ships with a 95% bootstrap confidence interval — a claim here without one would repeat exactly the mistake the IS/IV correction above was about.

Worth showing the small-n instability directly, since it's the exact trap this test is designed to avoid: at n=8 subjects per tier, the daily-count-sd ratio came back 1.65 — a 65% gap that reads like a real effect. At n=60, that same ratio settles to 1.120 with a CI comfortably containing 1. The n=8 number wasn't wrong arithmetic, it just wasn't a big enough sample to mean anything — the same shape of trap as this section's IS/IV correction and finding #2's seed-sweep follow-up above.

8. Comparison table

9. Statistical toolbox

10. Regenerating this page

Every diagram and every number above is read out of a real run of the generators in bot-detection/, not typed in by hand. If the generators' parameters change, regenerate both:

cd bot-detection
python3 render_diagrams.py   # writes diagrams_out/*.svg, prints all stats
python3 build_page.py        # reads them, writes ../bot-detection.html

render_diagrams.py uses a fixed seed and start date, so re-running it without touching the generators reproduces this exact page byte-for-byte.

11. Limits of this exercise

• Parameters are illustrative, not fitted. Jitter sigmas, the adherence rate, the AR(1) ρ — these are engineering choices tuned to look plausible, not calibrated against measured human telemetry. A determined adversary tuning against real data would do better than this baseline.
• These are worked examples, not a tuned classifier. No ROC curve, no precision/recall, no chosen decision threshold — that's the natural next exercise once build_dataset.py produces a labeled dataset to test a real rule against.
• All five generators are known-source here. Real-world bot detection is usually black-box; having the generating code, as this page does, is a favorable case that won't always hold.
• Cold start. Every day-to-day test above needs history to work. A five-day-old account has no usable autocorrelation estimate yet, regardless of whether it's real.
• Who calibrates the calibrator? Tier 5's target bands (§7) were set by hand, the same way Tier 4's constants were — "adversarial calibration" only searches within ranges a person picked as plausible. It's a real improvement over guessing each parameter independently, but it isn't escaping the underlying limitation, just moving it up one level.
• The holiday calendar is US-only, federal-only, and one persona archetype. Real subjects observe religious holidays, company-specific closures, regional/state holidays, or none of the above (shift workers, always-on services) — an office-worker archetype was the one requested, and it's the only one modeled. It also only added the holiday mechanism to Tier 5, which is why the human (Tier 4) row in §7's table shows zero holiday awareness too — a real, now-documented gap in this page's own "human" baseline, not something this update tried to quietly paper over.

12. Further reading

Six sources, each mapped to the tier or technique it directly informed. Four are quoted verbatim from the primary source; two (marked) were paywalled or blocked on fetch, so those are summarized from search-indexed abstracts rather than quoted — worth knowing the difference when you go verify these yourself.

• Pan, Liu, Liu & Hu (2016), "Discriminating bot accounts based solely on temporal features of microblog behavior," Physica A: Statistical Mechanics and its Applications, vol. 450, p. 193. secondary-sourced, paywalled
  Classified Sina Weibo accounts into human/bot/hybrid using nothing but posting-time burstiness and sequence "complexity" — bots score higher burstiness, lower complexity. Direct ancestor of Tier 1–2's interval-CV idea, and the basis for Tier 5's zlib-compression complexity floor (§7).
• Davis, Varol, Ferrara, Flammini & Menczer (2016), "BotOrNot: A System to Evaluate Social Bots," WWW 2016 Companion, pp. 273–274 (arXiv:1602.00975).
  "Here we present BotOrNot, a publicly-available service that leverages more than one thousand features to evaluate the extent to which a Twitter account exhibits similarity to the known characteristics of social bots." Timing is one feature category among a thousand — the basis for Tier 4–5's "no single test, and no single axis of calibration, is enough" framing (§6–7).
• Lin, Chen, Chuang, Wang, Lin & Lin (2023), "Examining Human-Smartphone Interaction as a Proxy for Circadian Rhythm in Patients With Insomnia," JMIR, 25:e48044.
  "The 'app-count' was defined as the sum of minute-by-minute use counts in nonoverlapping 5-minute epochs (288 epochs/d)." Derives clinical circadian metrics — interdaily stability (IS), intradaily variability (IV) — from raw usage timestamps, validated against actigraphy. The direct source for Tier 5's IS/IV diagnostics (§7).
• "Unobtrusive inference of circadian rhythms from smartphone data" (medRxiv preprint, 2025). secondary-sourced, blocked on fetch
  A framework for extracting circadian structure passively from usage timestamps — supporting evidence that hour-of-day density (the basis of every tier's anchor model, §2) is a real, extractable biological signal, not just a convenient modeling choice.
• Nagaraja & Shah (2019), "Clicktok: Click Fraud Detection using Traffic Analysis," University of Strathclyde (arXiv:1903.00733).
  "We argue that legitimate click activity has copy-resistance properties, owing to the uncertainty of inter-click times... an attacker would need to model the timing behaviour of the user, which has some uncertainty that forms the basis for our passive defences." Their mimicry defense catches clickspam by detecting reused click timing (81% detection at a 110-per-100k false-positive rate on stealthy attacks, per the abstract) — the basis for Tier 5's non-reuse guard (§7), including its honestly-limited practical bite against a purely generative model.
• Jin, Offutt, Zheng, Mao, Koehl & Wang (2013), "Evasive Bots Masquerading as Human Beings on the Web," DSN 2013.
  "The design rationale is that web bots behave intrinsically differently from human beings, allowing them to be detected... Our results show that the proposed evasive system can effectively mimic human behaviors and evade detectors by achieving high similarities between human users and evasive bots." Built and evaluated using a redefined Kullback–Leibler distance against real human behavioral data — the closest thing to an attacker's-eye mirror of this whole page, and the direct basis for Tier 5's per-subject noise-distribution diversity and adversarial-calibration approach (§7).