Inference from Data and Models · §4.6 Controllability and Observability
Carl Wunsch · MIT OpenCourseWare · Course notes · MIT OCW
Read §4.6, printed p.249, for the stacked rank test; derive the rotation-specific noise bound here.
Open the sourceCapstone III · Recover the hidden world
Design an observation, prove its limits, and test your intelligence claim.
Prerequisites: 25 · Hidden states and observable shadows, 26 · Harmony as a path, not a frozen shape, 27 · PCA and the geometry of compression, 28 · Probability: uncertainty is not geometry alone, 29 · What would “intelligence in the patterns” mean?
Understand
Objectives
- Compare exact recovery, singular failure and near-singular noise amplification.
- Derive a bounded-error inequality with its assumptions.
- Use a third reading to distinguish two specified models.
Start with something you can see
In the recovery lab compare θ=π/2, θ=0.01 and θ=0 for the fixed x₀=(2,1). Predict whether two noiseless readings uniquely recover the state, then enter observation noise (not state noise). Reveal each line before the estimate. Which case has a unique but fragile answer? The companion lecture treats inverse problems more broadly; our rotation and noise bound are separate calculations. [1]
Give the idea a precise name
The two-reading observation matrix O has first row C=[1,0] and second row CA=[cosθ,−sinθ] for the known rotation A. Write noisy readings r₀=y₀+e₀ and r₁=y₁+e₁ with |e₀|,|e₁|≤ε. The inverse formula is defined only when sinθ≠0.
det O=−sinθ; â=r₀; b̂=(r₀cosθ−r₁)/sinθ
Subtract the exact formula b=(y₀cosθ−y₁)/sinθ: b̂−b=(e₀cosθ−e₁)/sinθ, hence |b̂−b|≤ε(|cosθ|+1)/|sinθ|. Full rank gives unique noiseless recovery, not a uniform stability guarantee. At θ=0 the rows coincide, so no unique b exists; with conflicting noise the two constraint lines may not even intersect.
Work one small world
For x₀=(2,1) and θ=π/2, the first three noiseless readings are 2, −1, −2. A linear trend through the first two values instead predicts 2, −1, −4. The third reading separates these two specified models, not every model consistent with three points.
| Recovery worksheet | Two noiseless readings | Rank and effect of noise |
|---|---|---|
| θ=π/2 | (2,−1) | rank 2; |b̂−b|≤ε |
| θ=π/3 | (2,1−√3/2) | rank 2; |b̂−b|≤√3 ε |
| θ=0.01 | (2,2cos(0.01)−sin(0.01)) | rank 2; bound ε(1+cos(0.01))/sin(0.01), near 200ε |
| θ=0 | (2,2) | rank 1; b unobservable, no estimate |
Counterexample to “invertible means stable”: at θ=0.001, take e₀=0.001 and e₁=−0.001. The second-coordinate error is 0.001(cos(0.001)+1)/sin(0.001), nearly 2 although both observation errors are tiny. At θ=0, dividing by sinθ is invalid rather than a very large finite estimate.
THE BRIDGEA reconstruction formula, a failure case and a held-out test make the model claim inspectable.
WHERE THIS IDEA STOPSSuccessful recovery in this chosen system is not evidence of general intelligence or a unique physical explanation.
Experiment
Choose θ = π/2 and Reveal next step twice; compare noiseless readings to the estimate. Use θ = 0 and repeat: no estimate is drawn. Choose θ = 0.01, enter 0.001 and −0.001 as observation noise, and compare error; Use seed for noise pair is an optional reproducible alternative. Reset this lab restores θ=π/3 and zero noise.
What is the smallest extra observation that would distinguish the two explanations you currently cannot separate?
Check & explain
Which angle makes the two-reading observation matrix singular?
- π/2
- π/4
- π
For a quarter-turn of x₀=(5,3), what is the third first-coordinate reading?
Teach back
In the teach-back field, submit a noisy-recovery comparison with the four worksheet rows. Show O, determinant, inverse, error bound, singular limit, and the third-reading model comparison. End with one held-out test, a baseline and a falsifier for a claim about a predictor—not a claim of intelligence.
- If a rank proof is missing, compute det O for each angle and reject θ=0.
- If noise is confused with state change, write rᵢ=yᵢ+eᵢ and the error bound.
- If one matched model is called unique, add the linear-trend rival and the third reading.
- If the empirical claim lacks a baseline, specify held-out whole trajectories and an error comparison.
Read deeper
Mathematics for Machine Learning
Deisenroth, Faisal & Ong · Freely readable textbook
Read §3.9, pp.91–94 for rotations and §8.6, pp.283–287 for model comparison.
Open the sourceLecture 29: Duality Puzzle / Inverse Problem / Integral Equations
Gilbert Strang · MIT OpenCourseWare · Lecture · 51 min
Start at 13:03 for the longer 13:03–28:08 passage or 23:04–26:09 short core. The full 51-minute lecture is available from the same card; our two-state calculation is separate.
Open the source