EXPERIMENTS · BUILT
Is the ranking just the clock?
Every story loses score as it ages. How much of what leaves the top 25 is that, rather than news?
- Question
- How much of the top 25's churn is the clock rather than the news?
- Method
- Replay each run's score with the recency half-life at 3 to 48 hours, and with none, from the snapshots' own score parts; measure agreement between consecutive runs.
- Result
- The clock causes 0% of turnover between six-hourly runs and 0% day to day.
- Decision
- Keep the 12-hour half-life: recency is not what moves the top 25.
How it was measured
A story's score is 3 for each outlet that covered it, plus 2 × ln(1 + articles), plus up to 4 for recency, which halves every 12 hours. Every run keeps each story's score and its parts, so the run can be ranked again as if the half-life were different. For each half-life the charts show how much consecutive runs agree on the top 25:
- Rank-biased overlap: 1 is the same list in the same order, with the top weighted most.
- Kendall's τ: how well the order agrees, over the stories both runs share.
- Turnover: the share of the previous top 25 that left.
"None" drops recency altogether. Whatever churn remains there is news; the gap between it and the current half-life is the clock.
The numbers
Live runs, four a day · 25 consecutive pairs
| Half-life | Overlap | τ | Turnover |
|---|---|---|---|
| 3 h | 0.861 | 0.802 | 13% |
| 6 h | 0.862 | 0.805 | 12% |
| 12 h (now) | 0.863 | 0.808 | 12% |
| 24 h | 0.863 | 0.81 | 12% |
| 48 h | 0.864 | 0.816 | 12% |
| none | 0.864 | 0.816 | 12% |
The clock accounts for 0% of turnover here; the rest would happen with no recency at all.
Backfilled days, one a day · 368 consecutive pairs
| Half-life | Overlap | τ | Turnover |
|---|---|---|---|
| 3 h | 0.355 | 0.583 | 62% |
| 6 h | 0.354 | 0.585 | 62% |
| 12 h (now) | 0.355 | 0.585 | 62% |
| 24 h | 0.355 | 0.588 | 62% |
| 48 h | 0.355 | 0.587 | 62% |
| none | 0.355 | 0.59 | 62% |
The clock accounts for 0% of turnover here; the rest would happen with no recency at all.
Run by run at the current half-life · overlap between each run and the one before
Limits
- The live series covers the runs kept since the snapshots began; a longer record will narrow the figures.
- Backfilled days are one run a day, clustered afresh each day and matched to the day before by shared articles, so they churn far more than live runs. Compare the two series by their shape, not their level.