How to audit a JMM forecast record

Start with the resolution rule

The question headline is shorthand. The resolution rule is the contract. It names the exact series or filing, defines strict boundaries such as “greater than,” and explains how revisions are handled. If two reasonable readers could score the same outcome differently, the rule is not finished.

• Identify the official source.
• Check the comparison operator and unit.
• Confirm whether the first release or a later revision controls.
• Confirm the deadline and time zone.

Then inspect the frozen forecast

A forecast admitted to the JMM public ledger stores the issue probability and model inputs with the publication proof. A source refresh may add new observations or settlement evidence, but it cannot replace the original probability. That separation is what makes a track record possible.

A simple model is not automatically a weak record. Hidden discretion is the larger problem, and it is the thing a published calculation removes. Every probability currently on the JMM ledger is one formula: hits plus one, over observations plus two. That is Laplace’s rule of succession, and the added one and two are there so a model that has watched twenty-two releases without seeing an event cannot report zero. The result is then bounded before publication, which is why one record reads 0.05 when its own arithmetic gives 0.0417. Print the calculation without the bound and that record looks like a mistake; print both and it is the honest fact that this desk cannot claim certainty.

Look for the comparator too. Each record freezes a naive baseline in the same hashed fields as the probability, so it cannot be chosen afterwards to flatter the result: assume the quarter repeats the last one, or assume a series stays on the side of a threshold it is already on. The gap between the two is a disagreement, not an edge and not an expected return, and it means nothing at all until outcomes exist.

Score outcomes without erasing revisions

The initial published source value determines the score when the rule says so. Later official revisions remain useful context and should be displayed separately. That prevents a later data revision from silently changing whether an earlier call was right.

Probability forecasts should eventually be evaluated across a set, not by celebrating one confident hit. JMM will publish Brier scores only after eligible public calls have settled, and none has: every record on the ledger is open, so there is no score to read and nothing to average. What is inspectable before that gate is the whole issue record, which is the part that cannot be added later.