We spent this series building a strategy factory in public. A fair question: is our process anything like what a professional quant desk does — or a hobbyist cosplay of one? So we did the research properly: primary sources where they exist (firms' own publications, researcher podcasts, published papers), books where they don't, and honest tags where the truth is folklore. Every claim below carries its confidence level, because a page about research discipline should practice some.
The clearest published blueprint is López de Prado's "meta-strategy paradigm": a quant firm's real product is not any strategy — it's the process that manufactures strategies, staffed as specialized stations. His claim: "every successful quantitative firm I am aware of applies the meta-strategy paradigm."
Where firms differ is upstream — where ideas come from — and at the top — how survival is governed:
The published methodology converges on a short list of hard gates — most of them born from the embarrassing discovery that with enough trials, pure noise produces beautiful backtests (about 14 independent tries manufactures an in-sample Sharpe of 1.0 from nothing):
| gate | prescription | source |
|---|---|---|
| significance | t > 3.0, not the classical 2.0 — the field has run too many tests | Harvey, Liu & Zhu, RFS 2016 |
| Sharpe haircut | as a first cut, halve the backtest Sharpe; formally, a multiple-testing haircut that hits marginal strategies hardest | Harvey & Liu, JPM 2014/2015 |
| deflated Sharpe | DSR > 95% vs the Sharpe pure luck would produce given N trials | Bailey & López de Prado, JPM 2014 |
| overfit probability | PBO: odds the in-sample winner ranks below median out-of-sample; want single digits | Bailey, Borwein, LdP, Zhu 2017 |
| trial disclosure | a backtest reported without its trial count is void | Notices of the AMS 2014 |
| test design | purged CV with embargo; CPCV for a distribution of paths; walk-forward alone insufficient | AFML ch. 7/12 |
| backtest length | must grow with trials: ~5 years supports at most ~45 independent configs | Notices of the AMS 2014 |
| cost gate | expected costs ≤ ⅓ of pre-cost Sharpe, checked before evaluating the backtest | Carver, Systematic Trading 2015 |
| sizing | volatility targeting; forecasts capped; half-Kelly at most | Carver 2015; Chan |
| believability | persistent · pervasive · robust · investable · intuitive | AQR/Ilmanen; Berkin & Swedroe |
And on transaction costs, one number worth framing: AQR published the analysis of $1.7 trillion of its own live executions — median cost ~6 bps on US large caps — because they consider the cost model important enough to be research in its own right.
Here is our actual pipeline — the one that produced one champion and nineteen graveyard entries — with every gate drawn:
Scoring ourselves against the canonical lifecycle, stage by stage, no grade inflation:
| canonical stage | us | the honest note |
|---|---|---|
| economic hypothesis first | PARTIAL | we write expectations down pre-test, but some families were pattern-first (crude's short bias is a regime bet we flagged, not a theory) |
| pre-registration + trial log | STRONG | grids declared in committed code before results; 19 negatives logged with evidence; this is our best subject |
| data curation as a discipline | STRONG | learned the hard way: a calendar-roll artifact voided a verdict, so every cache now passes an automated QC gate |
| feature research before backtests | DIVERGES | AFML says "backtesting is not a research tool; feature importance is" — we are backtest-first; tried meta-labeling once, dropped it |
| cost/capacity gate before evaluation | PARTIAL | real costs are baked into the engine from bar one, but as an input, not a pre-rejection gate; capacity is moot at micro size |
| independent backtest adjudication | PARTIAL | the strategist and backtester are the same entity here — mitigated by frozen certification scripts, pre-committed verdicts, and adversarial red-team audits that have twice found bugs in our own favor |
| multiple-testing adjudication | STRONG | CPCV, PBO, deflated Sharpe with effective-trial clustering — the exact published machinery, reproducible from committed scripts |
| measurement resolution | STRONG | our bar-size rule (finale) is a lesson some of the literature doesn't even state explicitly |
| risk overlay / sizing | STRONG | vol-scaled sizing certified at DSR 98%; ratchets clamped; the eval math published honestly |
| portfolio construction | THIN | one certified strategy; the uncorrelated-second-signal search is recorded but the seat is empty |
| incubation / paper | PENDING | designed (champion vs adaptive A/B) but blocked on broker API access — the pros would not skip this and neither will we |
| monitoring / decay / retirement | PARTIAL | nightly regression re-certifies the champion on fresh data; live decay tracking doesn't exist yet because live doesn't exist yet |
| research culture | STRONG | the graveyard IS the product; negative results are first-class citizens; audits attack our own conclusions |
Three institutional practices this research convinced us to adopt, in order:
1. The cost gate, moved to the front. Carver rejects a strategy whose trading costs exceed a third of its pre-cost Sharpe before looking at any backtest. Our RT Levels v1 — 3,000 trades of costs against a thin edge — would have been flagged at the door.
2. Feature research as the discovery tool. Our order-flow dataset has only ever been tested as a bolt-on filter. The AFML prescription is to ask what the features predict under leakage-proof CV first, and only then build strategies from what survives.
3. Pre-set retirement criteria. Before the champion trades live, we will write down — in advance, like everything else — the live-vs-backtest divergence that forces a size-down and the one that forces retirement. Renaissance let validated models run "without interference"; Millennium fires on a number. Both work because the rule predates the drawdown.