Source:
AI Automation Engineer, Real-World Test Lab
Niantic Spatial
San Francisco, United States
Publication date: 04/11/2026
We're hiring an AI Automation Engineer, reporting to the Director of the Real-World Test Lab, to build the system that produces our evidence. Today our evaluations are manual, inconsistent, and slow. You'll design and own the automation that runs them end to end on every relevant release, with no human driving it, turning one-off experiments into an always-on service the whole company relies on.
Responsibilities
Build the evaluation machine: own the automation that executes Lab evaluations end to end, including environment setup, run orchestration, artifact capture, and result collection; Make results comparable: instrument scorecards so results can be compared across product versions, devices, and capture conditions; Automate the agent-driven layer: build agent workflows that exercise priority customer use cases at realistic scale across the difficulty spectrum; Kill manual work permanently: convert one-off experiments into standing protocols that run on every relevant release; Make failures actionable: produce diagnostics that reach owners with a reproducible case and data attached; Keep data from being the bottleneck: work with the AI Data Manager so every run is reproducible from a known dataset stateRequirements
Built and maintained production-grade automation or test infrastructure that other engineers relied on daily; Experience evaluating systems where correctness is graded rather than binary, meaning quality, accuracy, or latency thresholds rather than pass/fail assertions; Strong Python, with fluency in orchestration, CI-style pipelines, cloud storage, and reproducible environments; Worked with backend services and APIs you did not own, integrating against them without becoming a bottleneck for their team; Written up a technical finding clearly enough that a non-author could act on it without a meeting; A bachelor's degree in a relevant field, or equivalent experienceNice to have
Built agent-based or LLM-driven automation for a task that previously required human judgment; Worked on evaluation or benchmarking for computer vision, 3D reconstruction, or spatial systems; Instrumented dashboards or scorecards that leadership used for release decisions; Operated data provisioning or registry infrastructure across multiple environments and access modelsBenefits
Annual bonus; Equity; Comprehensive benefits package including medical, dental, and vision coverage; 401(k)How to apply
Apply online: https://www.jobs.pl/en/oferta/103762549/aplikuj
JOBS.pl · https://www.jobs.pl/en/ai-automation-engineer-real-world-test-lab-oferta-103762549





