Evidence, Bayes factors, and complexity
The Occam factor via Laplace, model comparison scales, Bayesian complexity derived.
Parameter inference asks "given this model, what are the parameters?" Model inference asks the question this lab was built for: "which parametrization of the proton does the data actually support?" The evidence $Z = \int L\,\pi\,d\theta$ — the normalizing constant we so carefully computed in §4.2 — is the answer, and this chapter derives why. We derive the Laplace approximation to expose the Occam factor hiding inside $\ln Z$, put honest error bars of interpretation on the Jeffreys-style verdict scales, work out exactly when a Bayes-factor verdict is prior-proof and when it is bookkeeping, derive the Bayesian complexity that tells us how many directions the data constrain, and check all of it against the lab's real evidence ladder: $-808$, $-109$, $+1453$.
From parameter inference to model inference
Apply Bayes' theorem one level up, to models $M_1, M_2$ themselves:
With equal model priors, the posterior odds are the Bayes factor, and $\ln B_{12} = \Delta \ln Z$ — the single number our result cards report. Note what $Z = p(D \mid M)$ is: the probability the model assigned to the data before seeing it, averaged over its own prior. A model gets no credit for containing a good fit somewhere in its parameter space; it gets credit for predicting the data with the prior mass it actually committed. That distinction — likelihood maximized vs likelihood averaged — is the entire content of the Occam factor, which the following derivation makes explicit.
Assume a single dominant peak at $\hat\theta$ and expand to second order in $\Delta = \theta - \hat\theta$: since $-\ln L = \chi^2/2 + \text{const}$, we get $L(\theta) \approx L_{\max}\, e^{-\Delta^{\mathsf T} A\, \Delta/2}$ with $A = \tfrac12 \nabla\nabla \chi^2 \big|_{\hat\theta}$ — the Hessian of $-\ln L$. For a uniform prior $\pi = 1/V_\pi$ containing the peak with room to spare, the Gaussian integral is standard:
$$ Z \;=\; \frac{1}{V_\pi} \int L\, d\theta \;\approx\; \frac{L_{\max}\, (2\pi)^{d/2}\, |A|^{-1/2}}{V_\pi} $$
Diagonalize $A$: its eigenvalues define posterior widths $\sigma_i = (\text{eigenvalue})^{-1/2}$, and $|A|^{-1/2} = \prod_i \sigma_i$. With a box of per-axis widths $W_i$ (so $V_\pi = \prod_i W_i$):
$$ \ln Z \;\approx\; \ln L_{\max} \;+\; \sum_{i=1}^{d} \ln \frac{\sqrt{2\pi}\,\sigma_i}{W_i} $$
$\blacksquare$
Substitute $\ln L_{\max} = -\chi^2_{\min}/2 + \ln G$ (the shared Gaussian constant of §4.1) and suppose for orientation that all $d$ constrained directions share a typical scale ratio $W/\sigma$:
$$ \ln Z \;\approx\; -\frac{\chi^2_{\min}}{2} \;-\; d \,\ln\!\frac{W}{\sqrt{2\pi}\,\sigma} \;+\; \ln G $$
Two competing terms. The first rewards fit quality. The second — the Occam factor — is rent charged per constrained direction: $\ln(\text{prior width}/\text{posterior width})$, the log of the compression the model needed to explain the data. A parameter the data never constrain ($\sigma_i \to W_i/\sqrt{12}$-ish, posterior $=$ prior) pays essentially no rent — the evidence does not punish parameters, it punishes compressed parameters that failed to buy fit quality. Adding a parameter is worth it iff it lowers $\chi^2_{\min}/2$ by more than its rent, typically a few units. This is Occam's razor derived, not assumed. $\blacksquare$
Define the complexity $C \equiv \langle \chi^2 \rangle_{\text{post}} - \chi^2_{\min}$ (the posterior-mean misfit above the best). In the Gaussian regime the posterior is $p(\Delta) \propto e^{-\Delta^{\mathsf T} A \Delta/2}$ with covariance $\Sigma = A^{-1}$, and $\chi^2 - \chi^2_{\min} = \Delta^{\mathsf T} A\, \Delta$. The expectation of a quadratic form under its own matched Gaussian:
$$ C = \mathbb E\big[\Delta^{\mathsf T} A\, \Delta\big] = \operatorname{tr}\!\big(A\, \mathbb E[\Delta \Delta^{\mathsf T}]\big) = \operatorname{tr}(A\, A^{-1}) = d $$
— exactly the number of parameters, because $\Delta^{\mathsf T} A \Delta \sim \chi^2_d$, whose mean is $d$. Now the punchline for flat directions: where the prior, not the likelihood, cuts off the posterior, that direction's contribution to $\mathbb E[\Delta^{\mathsf T} A \Delta]$ is far below 1 (the likelihood curvature $A$ is tiny there). So in general $C$ counts effectively constrained directions: $0 \le C \le d$, with each data-dominated direction contributing $\approx 1$ and each prior-dominated one $\approx 0$. $\blacksquare$
Interpretation scales — useful, and honestly conventional
The community reads $|\Delta \ln Z|$ against Jeffreys-style bands (in the form popularized for cosmology by Trotta): below $1$, "inconclusive"; $1$–$2.5$, "weak"; $2.5$–$5$, "moderate/substantial"; above $5$, "strong/decisive" ($\ln B = 5$ means odds of $e^5 \approx 150{:}1$). Two honest caveats. First, the band edges are conventions — round numbers someone found reasonable, not theorems; nothing changes in nature at $\ln B = 4.9$ vs $5.1$. Second, the scale presumes you believe both priors; a Bayes factor is only as meaningful as the prior volumes that went into it (next section). What is not conventional is the number itself: $\Delta\ln Z$ is a well-defined functional of models, priors, and data, with a sampling error we control (§4.2's $\pm 0.29 < 0.5$ budget). Our ladder lives so far beyond these bands — $|\Delta\ln Z|$ of $109$, $808$, $1453$ — that band-edge philosophy never matters in this lab; what matters is the prior-sensitivity audit.
When is a verdict prior-proof?
Split $\Delta \ln Z$ using Derivation 2 (models $a$, $b$ on the same data, so $\ln G$ cancels):
Prior choices touch only the ledger, and each width enters logarithmically: rescaling one prior width by a factor $f$ moves $\Delta\ln Z$ by $\ln f$ — about $2.3$ per decade per parameter. That gives a crisp robustness criterion: a verdict is prior-proof when the likelihood gap dwarfs any plausible ledger movement, $\tfrac12 |\Delta\chi^2_{\min}| \gg d \cdot \ln(\text{any defensible volume ratio})$. With $d \sim 50$ parameters and a generous one-decade uncertainty in every single width, the ledger moves by at most $\sim 115$; a likelihood gap of thousands is untouchable, a gap of tens is not. Worked Example 2 runs both of our headline verdicts through this test — one passes with twelve orders of magnitude to spare, the other is worth auditing.
Complexity in the field, and the health identity
The complexity $C$ of Derivation 3 is measured, not assumed: compute $\langle \chi^2 \rangle$ as a weighted average over the posterior samples of §4.2, subtract the best $\chi^2$ found. Our real-data MSHT20 run (PPDF-12) reports exactly this arithmetic:
Read it with Derivation 3: of 52 parameters, the data constrain $\approx 14$ directions; the other $\approx 38$ sit prior-dominated (their marginals are the prior, §4.1's flat-direction regime). The identity $\langle\chi^2\rangle = \chi^2_{\min} + C$ is definitional, so its diagnostic power is in the values being simultaneously sane: $C$ must land in $[0, d]$ and be stable under reruns and population changes; a $C$ near zero says the sampler never resolved the posterior bulk (weights collapsed onto one point), a $C$ far above $d$ says the "best point" is not actually near the mode or the weights are corrupted. We recompute the triplet on every run as a sampler health check before any verdict is read — it is the cheapest test that catches the most failure modes, and it is exact in our runs precisely because all three numbers come from the same weighted sample set. The same $C$ also calibrates closure expectations ($d_{\text{eff}}$ in §4.4's $N - d_{\text{eff}}$ law) — one number, three jobs.
$\Delta\ln Z$ answers "which of these two models predicted the data better?" — a strictly relative question. Both can be wrong: a model can win by $+1453$ and still be inadequate, and a model can fit perfectly ($\chi^2/N \approx 1$) yet lose the evidence race to a leaner rival. In our ladder the two questions get two separate numbers: $\Delta\ln Z$ ranks models, while $\chi^2_{\min}/N = 1.002$ (PPDF-12) certifies adequacy against the absolute yardstick $\mathbb E[\chi^2_{\min}] = N - d_{\text{eff}}$ of §4.4. Never let one impersonate the other: the toy model's problem is not that it loses by 1453, it is that its own $\chi^2/N \approx 5.6$ floor says it cannot describe the proton at all.
Model averaging — the consistent endgame
When no model decisively wins, Bayes does not force a choice: predictions can be averaged with evidence weights, $p(g \mid D) = \sum_m p(g \mid D, M_m)\, p(M_m \mid D)$, with $p(M_m \mid D) \propto Z_m$ at equal model priors. The PDF bands from such an average inflate to include parametrization uncertainty — the between-model spread that single-model bands structurally miss. Our ladder's gaps are so large that averaging degenerates to selection (weights $e^{-109}$ and below are zero for any purpose), but the concept sets the target for the multi-parametrization program: the honest band is the average over every parametrization that survives, weighted by its evidence — see the roadmap notes in the wiki.
Worked example 1 — Laplace vs exact on the §4.1 toy
Setup (from §4.1, Worked Example 1): $L(\theta) = e^{-(\theta-2)^2/(2\times 0.25)}$, flat prior on $[0,4]$, $V_\pi = 4$. Exact answer there: $Z = 0.3133$, $\ln Z = -1.161$.
Laplace ingredients: $\hat\theta = 2$, $\chi^2_{\min} = 0$, so $L_{\max} = 1$; Hessian of $-\ln L$ is $1/\sigma^2$ with $\sigma = 0.5$, so $d = 1$ and $|A|^{-1/2} = 0.5$.
Laplace evidence: $Z \approx L_{\max} \sqrt{2\pi}\,\sigma / V_\pi = 2.5066 \times 0.5 / 4 = 0.3133$ — identical to exact, because a Gaussian likelihood is the one case where the quadratic expansion is not an approximation. The only neglected piece is the prior truncation at $\pm 4\sigma$, worth a relative $-6\times10^{-5}$ on $Z$ — invisible at our precision. Occam anatomy: $\ln Z = -1.161 = 0$ (perfect fit) $+\ \ln(\sqrt{2\pi}\times 0.5/4) = \ln(1.2533/4)$ — all of it rent. And the 10×-wider prior of §4.1 raises the rent by exactly $\ln 10$: Laplace reproduces the $-3.463$ on the nose. For our real 52-dim posteriors Laplace is a cross-check, not a substitute — flat directions violate the single-Gaussian-peak assumption — but wherever $C \approx$ (number of Gaussian directions) it lands within a few nats of the nested-sampling $Z$, and we use that agreement as yet another health test.
Worked example 2 — the real ladder under the prior-proof test
Case A — the grid loses by 109 (PPDF-9). The 36-parameter grid parametrization trails the rigid LH form by $\Delta\ln Z = -109$ on the same data. Could priors flip it? Shrinking every grid-parameter width by a common factor $f$ credits the grid $+36 \ln f$; solving $36 \ln f = 109$ gives $f = e^{3.03} \approx 21$. A 21× shrink of all 36 widths is not obviously absurd — until you recall §4.1: in the grid's many prior-dominated directions the posterior fills the box, so a 21× shrink clips real posterior mass, changes the fit itself, and forfeits the assumed "free" volume gain. Verdict: robust under any prior change that leaves the posterior intact, but the margin is audit-worthy — we report the volume arithmetic next to the number rather than calling it untouchable.
Case B — MSHT20 beats the toy by 1453 (PPDF-12 vs PPDF-7). $\ln Z$: $-381.9$ vs $-1835.1$, gap $+1453.2$. Decompose it: the toy floors at $\chi^2/N \approx 5.6$ while MSHT20 reaches $1.002$, so with $N = 648$ the likelihood gap alone is $\tfrac12(5.6 - 1.002) \times 648 \approx 1490$. The residual $\approx -37$ is the Occam ledger — a few dozen nats of extra rent paid by the bigger parametrization, consistent with a few-dozen extra compressed directions at a few nats each. To flip the verdict on volume grounds you would need the ledger to move by $1453$: with 52 parameters that is $f = e^{1453/52} = e^{27.9} \approx 1.3\times10^{12}$ per parameter width. No defensible prior — no prior a human has ever written down — moves twelve orders of magnitude per direction. This verdict is likelihood-dominated and prior-proof in the strict sense of the criterion above: the toy model does not lose on Occam grounds, it loses because it cannot fit the proton.