Three Ways to Think You've Validated a World Model
Debugging a driving world model's trajectory conditioning, building a geometric checker for its rollouts, and a silent-selection bug that surfaced three times across independent validation code.
I set out to test a distillation question. I ended up writing a paper about why that question was premature.
The plan was simple, on paper. Vista, OpenDriveLab’s action-conditioned driving world model, generates a rollout: give it a starting frame and a trajectory, and it imagines the next 25 frames of driving. It ships with 50 sampling steps by default. Distill it down to 4, or 2, or 1, and you get a much cheaper rollout, if the quality holds. The question I wanted to answer was whether a policy-ranking verdict survives that distillation. Rank two driving policies with Vista: does the ranking stay the same at 50 steps and at 2?
That’s still an interesting question. I haven’t answered it. Instead I spent most of this project discovering that the question has two silent prerequisites nobody in the driving-world-model literature seems to state out loud, and neither one held up when I actually checked. This is the story of finding that out, what it cost to check properly instead of assuming, and what I’m building next because of it.
Prerequisite one: does the model even listen to you?
Vista takes an 8-number trajectory: four future waypoints, each a (lateral, forward) pair in
meters, at half-second intervals. Feed it a left turn, it should generate a left turn. Feed it a
right turn, a right turn. That’s the entire premise of using it for closed-loop policy evaluation.
The model’s response has to be attributable to the policy’s actual decision.
First bug, and if you’re adapting Vista to anything other than nuScenes you will hit this too: the
axis order is (lateral, forward), not (forward, lateral). The failure mode is nasty precisely
because it doesn’t look like a bug. You get blurry, indecisive turning behavior, output that reads
as “the model doesn’t turn very well” instead of “the input is wrong.” I only caught it by
triangulating three independent sources: the paper, the original author’s own preprocessing
script, and a direct statistical decode of the nuScenes annotation data Vista was trained on. If
your Vista turns look weak or wrong, check your axis order before you conclude anything about the
model.
Fixing that revealed a second problem, and this one isn’t a bug. It’s arithmetic. Vista encodes
each trajectory value through a sinusoidal embedding built for diffusion timesteps, which run in
the hundreds to thousands. Real driving trajectories are single-digit meters, three orders of
magnitude smaller than what the embedding was tuned for. I checked whether that actually matters
instead of assuming it either way, and it does. Cosine similarity between the embeddings of +3m
and -3m (same magnitude, opposite direction, exactly the pair a left-vs-right decision depends
on) is 0.735. Cosine similarity between +3m and +15m (same direction, five times the
magnitude) is 0.662. Lower. At realistic driving scale, Vista’s own conditioning discriminates
magnitude more reliably than direction. Behavioral tests agreed: no reliable directional response
at realistic magnitude, and only a real but seed-dependent effect at ten times the scale.
I didn’t introduce that bug. It’s a property of Vista’s own embedding design, and you can verify it yourself with ten lines of numpy and no GPU. It means the first prerequisite, that a generated rollout is attributable to the policy’s action, doesn’t cleanly hold at the magnitudes a real policy actually produces.
Prerequisite two: can anyone tell if a rollout is any good?
Say the conditioning problem didn’t exist. There’s a second prerequisite underneath the ranking question: something has to look at a generated rollout and say whether it’s a fair test of the policy or an artifact of the generator falling apart. A rollout can look plausible frame-to-frame while a static pole bends, or the ego vehicle’s implied speed does something a real car can’t do. So I built a geometric checker: VGGT, a feed-forward 3D reconstruction model, validated first against real Waymo pose (11.5cm mean position error, 0.987 heading correlation on a real 49-degree turn) and cross-checked against real lidar depth.
Then I tried to use it, and this is where the project turned into a different paper. Four times a check looked like it had found a real problem. Three of those four turned out, once I built the control I should have built from the start, to be measuring something else entirely.
The sharpest one: a scale-consistency score checking whether a rollout’s implied camera speed evolves smoothly. On a real, dead-straight clip it looked clean. On a real 49-degree turn, at a 5-frame averaging window, it reported 4.6% deviation, comfortably under the 5% threshold I’d pinned in advance. A clean pass. The actual ground truth, on the exact same clip, was 13.3%. A clear fail. The checker wasn’t being cautious. It was confidently wrong: it reported smoother implied camera motion than the real clip actually had, and widening the averaging window made things worse, which ruled out “just needs more data” as the explanation.
Then I ran the same statistic on the straight clip and found it was wrong there too, just in the opposite direction: jumpier than reality on the easy clip, smoother than reality on the hard one. Same estimator, two opposite failure directions. I checked this wasn’t a coincidence by sweeping the averaging window from 3 to 15 frames on both clips. On the straight clip, the gap to ground truth shrank as the window widened, the signature of high-frequency noise averaging itself out. On the turning clip, the gap grew as the window widened, the signature of a low-pass filter smoothing away real, slower variation instead of noise. That crossover, not just the two headline numbers, is why I call it a bandpass distortion rather than a simple too-strict or too-lax bias.
My best explanation, and I want to be clear it’s an explanation rather than something I isolated with an ablation: a joint-attention model is trained to build one temporally consistent story out of the whole clip, and that’s exactly the property that would make it bad at flagging temporal inconsistency, since flagging it means contradicting the story it just built.
The near-miss that would have been a confidently wrong published result
Before I trusted that finding, I hit a smaller, uglier version of the same problem. It’s the one I’d want another engineer to see before they publish a negative result of their own. Extracting the recovered heading change on that same turning clip, my first pass returned -1.6 degrees against a ground truth of 49.0. Wrong sign, off by an order of magnitude. Read on its own, that’s a near-total failure to recover the turn, and it would have been easy to believe, because it matched a limitation I half-expected going in.
It wasn’t real. The position trajectory, computed completely independently, visibly curved and matched ground truth to 11.5cm. A straight predicted path can’t be rigidly aligned onto a genuinely curved one and look that good, so the heading number and the position number couldn’t both be telling the truth. The bug was a coordinate-convention mismatch. I’d extracted heading by reading yaw directly off VGGT’s rotation matrix under the wrong axis convention: Waymo’s vehicle-frame assumption, applied to OpenCV’s camera-frame output. The number that came out wasn’t heading with some noise on it. It was a completely different physical quantity wearing heading’s name.
The generalizable lesson isn’t “check your axis conventions,” though that’s true here too. It’s about what to do when two independent measurements of the same thing disagree: trust the one with fewer assumptions baked in. Position, computed from consecutive frame deltas, has no rotation-matrix convention to get wrong. Heading, decoded from a rotation matrix, does. A plausible-looking negative result that happens to confirm something you already suspected is exactly the shape of finding that turns into a confidently wrong published claim, if you have no independent check to catch it.
The pattern that showed up three times
By the time I’d chased down why a sparse-point rigidity score kept passing rollouts it shouldn’t have, I’d found the same design mistake twice in my own code. The first version treated any point that never reached enough tracking confidence as simply excluded from scoring. That’s the wrong default for this metric specifically, because low confidence isn’t independent of rigidity. Thin, high-frequency structure like poles and wires is low-confidence by construction, so the filter removes exactly the structure most likely to reveal a problem. A rollout falling apart geometrically and a rollout with no data at all look identical under a pooled pass/fail statistic. The fix: report the unscoreable fraction as a first-class output.
The second version, one level deeper, evaded the exact fix I’d just built. It used an adaptive confidence threshold, set to each frame’s own median. That sounds principled. It generalizes across clips with different absolute confidence scales. It’s actually the same bug wearing a disguise: the threshold gets set by whatever surface happens to dominate a given frame, so a perfectly trackable point can fail simply by sharing a frame with something more confident elsewhere. Reporting the unscoreable fraction within one run doesn’t catch this, because the bias doesn’t look like missing data. It looks like a systematic skew toward whichever surface is nearest or most reflective, invisible unless you go looking for it specifically.
Here’s what that subtle deformation actually looks like, next to the real, unwarped frame and the large positive control used to confirm any checker in this story could still fire on something obvious:
The third time is the one that convinced me it isn’t just my code. I ran the same subtle, carefully controlled deformation against GeCo, a published geometric consistency checker I had no hand in writing. It missed the deformation completely: no separation from the unwarped clip. Before trusting that null result, I ran a positive control, a large, obvious deformation, the kind GeCo’s own published validation uses to confirm the metric fires at all. It fired clearly. The pipeline worked, and the subtle warp genuinely sat inside the range GeCo reads as unremarkable. Then, instead of just accepting “not much signal,” I checked why the response was smaller than the visual deformation suggested, and found a clean, monotonic dose-response: pixels displaced under 5px lost almost no coverage, pixels displaced over 90px lost twenty times as much. The most-deformed pixels weren’t scoring badly. They were disappearing from the denominator, in proportion to how deformed they were. Silent selection again, in a system I didn’t write, using a motion model I didn’t train.
Twice in code I control, found by fixing them. Once in code I don’t control, found only because I ran a positive control to rule out a wiring bug and got a dose-response instead. If there’s one operational takeaway from this whole project, it’s this: a scoring pipeline should never report a number without also reporting what fraction of the data that number is actually based on. “Zero problems detected” and “the checker had no opinion” look identical in a summary table. They are not the same claim.
What honoring a pre-committed stopping rule actually costs
Partway through, I pinned a rule before I had any results to argue about: any geometric score gets at most two failed validation attempts. Fail twice, and it drops from a pass/fail gate to a reported diagnostic. I wrote that down because the rigidity score had already burned most of a week: five separate validation attempts, each one individually reasonable, each one turning up one more confound worth ruling out before the next. That’s exactly how a week disappears one week into a fourteen-week plan. Not one bad decision, just a chain of locally justified “one more check.”
The rule did its job, in a way I didn’t love in the moment. Scale consistency’s redesigned, self-contained version, the one that doesn’t need ground truth, which is the actual production requirement, got exactly two attempts. The first found a false pass on the turning clip. The second, a window sweep, confirmed it wasn’t a window-size artifact: VGGT’s own reported deviation trended toward zero as the window widened, while the real deviation stayed substantial. Two strikes. Demoted. No third redesign, no “but this version is different.” Writing the rule down before I had a result to protect is the only reason I actually stopped there instead of finding a reason this particular case was special.
Where this leaves the project
I did not answer “does a policy-ranking verdict survive distillation.” I found that the question presupposes two things, attributable conditioning and a trustworthy geometric checker, and neither one holds without real, load-bearing work first. That’s the paper I ended up writing, Three Ways to Think You’ve Validated a World Model. It argues that building trustworthy automatic evaluators, not sampling cost and not generation fidelity, is the actual binding constraint on using generative world models for driving evaluation. I also put together a companion notebook repo so the numbers above are checkable rather than just asserted: no GPU required for four of the five notebooks, and the fifth needs nothing more than a laptop and 60MB of bundled data. I’m not releasing either publicly yet, but if you want to read the paper or run the notebooks yourself, reach out and I’ll send them over.
What’s next follows directly from what killed this project’s original scope. The driving geometric checker never got the one thing it actually needed: ground truth for the geometric coherence of a generated scene. The scene never existed, so there was nothing to validate the instrument against except another model’s opinion, and that instrument turned out to be structurally blind to exactly what it was asked to measure. A synthetic environment solves this by construction. Control the world, and you know its exact geometry, poses, and depth for every frame, so a geometric reward can be validated directly against truth instead of against another instrument’s opinion.
The plan is staged, so it produces a result at every stage instead of betting everything on the last one. Stage one: build a geometric-consistency score in a synthetic environment and validate it against known ground truth, the step the driving version never reached, and report its sensitivity floor, the smallest deformation it can detect. Stage two: use that score as a rejection filter on a batch of generated rollouts and check whether downstream conclusions actually change between the filtered and unfiltered library. That second half might be the whole result on its own. If filtering changes nothing downstream, that’s a real finding about how much geometric coherence actually matters, and it costs no training at all. Stage three, only if the first two justify it: use the validated score as an RL reward and fine-tune a generative model against it directly.
I don’t know yet whether a synthetic result transfers to driving. Thin structures, weak parallax, and sparse texture are domain-specific failure modes a clean synthetic environment won’t reproduce, and I’m not pretending otherwise going in. But proving the mechanism works somewhere you can actually check it beats assuming it works and finding out the hard way, which is more or less the summary of everything above.