Real-Time Feedback Framework for VR Training: From Ungoverned Alerts to a Scalable Decision System.
Established a feedback framework for Axon’s VR training platform — partnering with Engineering and Art to define how real-time feedback and after-action reporting are categorized, prioritized, and delivered across headset, tablet, and web.
IMPACT
A reusable feedback framework — replacing ad hoc, per-exercise decisions with a system, shortening the path to shipped feedback for PMs, Designers, and Engineers.
The feedback list kept growing, and nothing was governing it
Real-time feedback is the signal a trainee gets in the seconds between an action and the system’s response — while the dummy is still in range. It shipped one feature at a time, with no framework governing any of it: a lag indicator here, a probe-justification prompt there, each one correct in isolation.
By the time the pattern was audited, a single drill could surface up to eight overlapping callouts at once, with three concrete failure modes running underneath the clutter:
Two systems were straining under that same lack of structure at once
Realtime feedback and After Action Report were both built on the same ungoverned feedback system. Different screens, different drills, the identical root problem underneath both: no framework deciding what got shown, when, or how urgently.
Every feedback type got sorted by what it actually was, before anything else got decided
The fix started as a category exercise, not a visual one. Every existing feedback tag: Ready, Arm, Threat, Full Spread, Wire Break, Justify Each Probe, and the rest got sorted into one of five functions: Instructional, Shot-Specific, System Event, Result, and Mixed-Use.
Every category then got a place on the actual timeline of a drill
A drill, by definition, runs from the moment the range is clear to the moment the dummy disappears and the range clears again, typically ten of them make up one exercise. Each category landed in one of three periods:
Before the dummy appears (preparation — instructional feedback, so nothing distracts from the action about to start)
While the dummy is active (the action phase — shot-specific and system-event feedback, delivered immediately because that’s when a correction is still useful)
After the dummy disappears (reflection — results only, so the user’s full attention goes to the outcome without the dummy still competing for it).
Everything, regardless of category or timing, was still defaulting to the same delivery
Categorizing what and when didn’t touch the actual complaint driving all three original issues: delivery primarily relied on a single method: world-locked UI, with no variation in how loudly something announced itself. An instruction and a warning looked and felt the same, because the system had never been asked to distinguish between them at the delivery layer.
Five modalities got ranked by how much attention they’re allowed to take
Default pairings followed from there:
informational and system-event feedback got Voice Over with optional lighting;
shot-specific feedback got either HUD or Attached UI, never both, to avoid stacking two competing visual sources;
results got Standalone UI, because a final outcome deserves the user’s full attention and nothing else competing for it.
The pattern across all three axes collapsed into three tenets, named only once the pattern was real
Mapping every feedback item against every modality it actually used surfaced the same three questions worth asking of any new feedback type going forward, and each one turned into a tenet:
Minimize Cognitive Load.
Show only what’s needed, when it’s needed. In practice: a hard cap of two pieces of information delivered at once — grounded in working memory holding roughly 3 to 5 chunks, less under stress — and alignment across modalities when more than one fires on the same event, so two channels never send two different messages about the same moment.
Design for Trainee’s Performance.
Every piece of feedback has to drive improvement, not just report a fact. The test that fell out of testing sessions was blunt and three-tiered before it got simplified: does this feedback help the trainee improve in this drill (give it immediately, dummy still present), in the next drill (give it in the mini-AAR), or in the next exercise (give it in the full AAR)? Collapsed to its essence: if it helps now, say it now — if it helps later, wait.
Drive Consistent Mental Models.
Establish a language of feedback so users know where to look and what to expect, every time. Consistent mappings (Instructional always gets Voice Over plus Light, for instance), and when more than one modality fires at once, both need to agree with each other rather than compete.
