Companies spend heavily on forecasting. Software licences, analysts, planner adjustments, consensus meetings. Almost none of them measure whether any of that activity adds value, because there is nothing to measure against.
There should be. A control group is available, it is free, and it takes a week to set up. Compute the naive forecast, same as last year adjusted for trend, for every item you plan, and measure your actual process against it over the two years of history you already possess.
The published evidence on what this reveals is uncomfortable. Steve Morlidge's research in Foresight, the International Institute of Forecasters' journal, studied 300,000 real forecasts across eight supply-chain companies and found 52% were less accurate than the naive forecast. Follow-up work across more than 20 companies found 30 to 50% of forecasts routinely losing to naive. Half the forecasting effort in industry is spent losing to a number that costs nothing to produce.
The question is the process, not the forecast
The field calls the discipline forecast value added, and the name holds the insight. The interesting question is not whether the forecast is accurate. It is which step of the process improved it.
- The model against naive. For products that sell steadily, the statistical layer usually wins. For items that sell rarely or in unpredictable bursts, frequently not. Knowing which is which redirects the effort and the spend.
- Each human step against its input. Did the planner's adjustment improve on the model? Did the consensus meeting improve on the planner? The human half of the chain has its own uncomfortable evidence base (half the forecast is a person, and nobody scores that half).
- The whole chain against naive. The final number. If the end-to-end process loses to the free baseline for a third of the portfolio, that third is being planned at a premium price for negative value.
None of this is a judgement about forecasters. It is a management failure of the ordinary kind. An activity that consumes real money runs for years without anyone checking whether it works, because nobody installed the control group.
Run it before the next purchase
AI forecasting claims are everywhere, and accuracy improvements of 8 to 20% are routinely cited without stating the baseline. If you have never computed your own naive benchmark, you cannot price those claims. An "85% accurate" system may be brilliant or may be losing to last year plus trend, and the sales material will not volunteer which.
Run the audit first and the conversation changes. You can demand proof on your own hard items, and a vendor must beat your reality, not your anxiety. The same test then governs the system after purchase, and the steps that keep failing are retired without sentiment.
Some items should stop consuming forecasting effort
The most useful finding is often the least expected one. Some demand cannot be forecast better than naive, by anyone, because the signal is not in the data. Those items should stop consuming forecasting effort altogether. The right response to them is a different operating posture (e.g. faster response and honest buffers), not a better model, and only the control group tells you which items they are.
Continuous forecast value added, per item and per process step, with the naive baseline enforced as the floor, is standard in the Prophesee Supply Chain Suite. Install the control group on your own history. Start here.