Koryu · 黄龍
Koryu
← Research library
Research · 研究 · 115 · Methodology

Backtest the point-in-time universe. Not the one in your download.

5 Oct 20264 min readMethodologyKoryu Research

This article explains a data-construction discipline central to our own research standards. Nothing here is investment advice. Decisions are yours.

A point-in-time universe answers, for every date in a backtest, one question with archival honesty: which assets qualified on that day, by the rules, using only what was knowable that day? It is the difference between simulating the market that existed and simulating the one that happened to survive into your download.

The fiction the default produces

The convenient backtest starts from today's top-100 list and fetches its history: every asset in the test is certified, by construction, to have survived until now. The strategy never meets the top-ten asset that went to zero in a week, the case the LUNA collapse made famous, never holds the exchange tokens that died with their venues, never touches the crypto graveyard at all. The inflation this buys is not subtle, and our own equity audit put a number range on its equity-market cousin; crypto's death rate is higher, so its flattery is larger. A momentum strategy is the worst case: it ranks on strength, and yesterday's doomed assets were often strong on the way up, so the current-list version systematically deletes exactly the trades that would have hurt most.

What PIT construction actually requires

Three archival ingredients. Historical rankings: market capitalization and volume as of each date, for every asset alive then, including the dead ones under their original identities. Rule evaluation in the past tense: every eligibility test the published universe rules impose, liquidity floors, listing age, exclusions, computed from that date's data only, with no peeking at what later happened. And event resolution: delistings, renames, splits of chains and tickers, each mapped so the asset that died is distinguishable from the asset that took its name. This is precisely the battery our vendor gate runs before any historical data is trusted, a commitment the build-in-public roadmap made in writing: a vendor that cannot enumerate the top-150 as of an arbitrary 2021 date, corpses included, cannot support an honest crypto backtest, whatever its marketing says.

Our specification, stated

The derivation program's research universe is point-in-time by construction: the top-150 by the published rules, re-evaluated at each rebalance date from that date's data. The number is deliberately wider than the monitor's current top-100, because a research universe must contain tomorrow's entrants and yesterday's exits to model turnover honestly: assets enter the test when they qualified, not when they became famous, and leave it when they failed the rules, not when they became embarrassing. The monitor's current-list top-100 is a different instrument doing a different job, live measurement under the monitor methodology, where survivorship is not a concept: the present has no survivors, only members.

What survives even PIT discipline

Honesty requires listing the residual biases, because PIT is necessary, not sufficient. Venue survivorship: exchange consolidation kills venues, and bars sourced from the survivors still miss the pairs that died with them. Fill fiction: an asset can be present in the universe while being untradeable in practice, halted chains, delisted pairs, and books too thin for the modeled size, so presence must be paired with honest cost and fill assumptions, and stops in backtests live or die on exactly those conventions. And data vintage: prices as recorded today are not always prices as displayed then; revisions, backfills, and symbol surgery leave scars only vendor archaeology reveals.

The honest posture is layered: PIT membership first, because it is the largest single correction, then each residual named and either modeled or disclosed, which is why the transplant experiment carries its own caveat list, and why every number this site publishes travels with its data-quality footnote until the vendor gate closes. Backtests do not fail from missing sophistication; they fail from simulating a kinder market than anyone was ever offered. Decisions are yours.

Related reading
MethodologyPaper trading in public: what it proves and what it can't5 min readMethodologyWhy we publish our formulas: the case against secret sauce5 min readMethodologyA 365-day market: how 24/7 changes annualized numbers5 min read
Frequently asked

What is a point-in-time universe?

A universe reconstructed for every backtest date: which assets qualified that day, by the published rules, using only that day's data, dead assets included under their original identities. It simulates the market that existed rather than the one that survived into your download.

Why are current-list backtests biased?

Every asset in them is certified by construction to have survived until today: the strategy never meets the top-ten asset that went to zero, never holds tokens that died with their venues. Momentum strategies are the worst case, since the doomed were often strong on the way up.

What does PIT construction require?

Historical rankings as of each date (corpses included), eligibility rules evaluated in the past tense with no peeking, and event resolution mapping delistings, renames, and chain splits so the asset that died is distinguishable from the one that took its name.

What is this site's research universe?

Point-in-time top-150 by the published rules, re-evaluated at each rebalance from that date's data: wider than the monitor's live top-100 because a research universe must contain tomorrow's entrants and yesterday's exits to model turnover honestly.

Does PIT fix everything?

No: venue survivorship (bars from surviving exchanges), fill fiction (present but untradeable assets: halted chains, delisted pairs), and data vintage (revisions and symbol surgery) all survive PIT discipline. Necessary, not sufficient; each residual gets named and modeled or disclosed.