[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"article-body-the-league-table-rescored-daily":3},"\nAsk a vendor why their model is right for your problem and you\nwill receive an architecture story. Transformer this, foundation\nthat, proprietary the other. Architecture stories are ideology.\nWhich model best predicts a given target, on given data, at a\ngiven horizon, is an empirical fact, and an unstable one. It\nchanges as the data changes, sometimes within a quarter.\n\nThe forecasting field has known this for decades, most famously\nthrough the M-competitions, where simple methods repeatedly\nembarrassed sophisticated ones on real series and no single method\ndominated across question types. A vendor whose product is one\nmodel, however capable, has answered an empirical question with a\ncommitment made before your data was seen.\n\n## What evidence-driven selection requires\n\nIf model choice is an empirical question, the machinery for\nanswering it follows directly, and none of it is exotic.\n\nSelection has to be a standing competition, not a coronation. Many\nmodel families compete on each question, because the winner for\nsteady weekly demand is routinely the wrong choice for\nintermittent spares or regulatory case timing. One question, one\ncompetition, one current champion.\n\nThe scoring has to happen on unseen data, in rolling backtests,\nwith [a placebo check](/insights/the-placebo-test) behind it so a\nlucky fit cannot take a seat it did not earn.\n\nThe table has to be re-scored continuously as outcomes arrive. A\nchampion that drifts loses its seat to the contender that has not,\nwithout a meeting, without a migration project, without anyone\ndefending last year's architecture choice.\n\nAnd every competition needs one entrant that does not care about\nelegance. The naive baseline, last year plus trend, sits in every\ntable, and a champion must beat it or there is no champion.\n[Research keeps finding](/insights/the-naive-baseline-is-beating-you)\nthat a large share of sophisticated forecasts fail exactly that\ntest. When nothing beats naive, the honest output is a sentence.\nThis target is not predictable yet with this data, and here is\nwhat would change that.\n\nThat sentence prevents the quiet catastrophe of enterprise\nforecasting, which is not bad models but confident automation of\nguesswork.\n\n## What this replaces\n\nEvidence-driven selection retires two familiar failure modes. The\nfirst is the data science queue, where each question waits months\nfor a hand-built model that then ossifies because nobody has time\nto revisit it. The second is the platform monoculture, where every\nquestion is answered by the vendor's one architecture, at whatever\nquality that architecture happens to achieve on it.\n\nAgainst both, the competitive mechanism is boring, continuous and\nauditable. Every model's history is on the table, every\nsubstitution has a scoring reason, and the answer to \"why this\nmodel?\" is never a belief. It is a row.\n\nInside Prophesee's Foresight engine this runs as a standing league\ntable of 62 models from 13 families, re-scored daily, naive\nbaseline enforced as the floor. The table is the evidence for the\nargument, not the other way round. To see who wins on your data,\n[start here](/contact).\n",1786984937499]