From Figma to Interactive Unity UI
Hand-building every Figma screen into Unity was slow, drifted from the source, and produced throwaway work twice: once when the screen changed, and again when the UI stack changed. I designed and built an AI orchestration pipeline that treats Figma as the input, matches what already exists in the design system, and only creates what a human approves. After two live projects, we split that pipeline so designers own UI in code and engineering owns business logic.
Ship now, on a stack you already plan to leave
The roadmap had a tension a lot of product teams will recognize. New screens had to ship on a UI framework everyone already knew was transitional.
That created two layers of throwaway risk. Any hand-built screen hierarchy would be rebuilt when the framework switched. Any custom infrastructure on top of it, tokens, binding code, design-system packages, would go with it.
Several teams were going to build the same class of screens with different AI tools. The workflow had to live in the skill, not in one team’s favorite setup.
Days per screen, so prototypes barely existed
Every Figma file was translated by hand. Design intent and what shipped drifted. Turning a screen into a working, inspectable prototype took days, which meant prototypes rarely got built at all.
Iteration speed was the bottleneck, not only implementation quality. A one-off import demo would not survive a real project. The pipeline had to be safe to run twice.
One component, then one pipeline, then two jobs
DECISION 1 · PROVE IT ON ONE COMPONENT
Pull values from Figma, check tokens, generate styles and behavior, drop the result in a showcase, compare to source. It worked. It was still one component, one agent, one person driving every step.
DECISION 2 · SCALE TO A SCREEN
A 13-step orchestrator reads a Figma URL, matches the design system, pauses for a human on every gap, and binds the screen to an MVVM ViewModel that never touches Unity.
DECISION 3 · SPLIT DESIGN AND DEV
After two live projects, one thread was doing two jobs. The live system is a Design Orchestrator, a handoff agent, and a Dev Orchestrator, so both sides can work in parallel.
A pipeline that refuses to invent the system twice
Give it a Figma link and an ID. Phase 1 is read-only: describe, then match. Checkpoint 1 is a human list of gaps. Phase 2 builds only what was signed off. Phase 3, owned by engineering, plans the ViewModel shape and wires the screen so the data is editable.
If a file would change, stop and ask. The orchestration spine is never written by the AI. Existing design-system files are never modified. Business logic stays in ViewModels. Binding and view prefabs can be regenerated when the UI framework changes.
What moved
The designer’s job changed. Figma is the input, not a spec to re-interpret. Designers approve every gap and own the UI in code. Engineering owns the business logic.
A shared package of components, built true to Figma with designers in the loop, replaced one-off screen builds. Three new projects shipped on the UI Orchestrator at 71% under budget (321 hours actual vs. 1,105 hours budgeted).
3
projects shipped
71%
under budget