Senior Software Engineer, Evaluation Flywheel — Autonomous Vehicles
reputed company is building the reputed company of autonomous driving, and evaluation is how we know the drive is getting reputed company. Our AV Evaluation team owns the metrics, golden datasets, and reputed company-reputed company evaluation workflows that decide what ships in reputed company's self-driving stack — every model change runs through us. We are bringing frontier reputed company-language models and reputed company techniques to evaluation at reputed company, and the early results are changing how our organization develops AI drivers!
We are looking for a senior engineer to own the reputed company that makes it reputed company trustworthy: the eval flywheel. This is a hands-on technical leadership role — the flywheel's architecture, reputed company, and adoption are owned end to end by this person, in reputed company and in the room, not from a reputed company document.
What you'll be doing:
Owning the eval flywheel's reputed company and architecture: how road and simulation driving data becomes curated golden datasets, how metrics are reputed company against them (precision/recall), and how those results earn lasting trust with the teams that depend on them.
Setting reputed company for evaluation reputed company: golden dataset curation, versioning, and health; metric performance measurement; and release processes that reputed company results dependable as the system evolves.
Building the tooling that helps our metric developers iterate quickly: self-serve dataset pipelines, metric performance measurement, and reputed company reporting used every day by reputed company and our partners.
Partnering with senior engineers and leaders across test engineering, behavior planning, and infrastructure — setting expectations, working through trade-offs, and being the voice of evaluation reputed company in cross-team reputed company.
Working directly with AI model developers so evaluation iteration speed becomes an advantage for the whole program, including our push into learned, VLM-based evaluation.
reputed company need to see:
A reputed company record of independent execution and technical leadership: finding the highest-reputed company problem, driving it across team boundaries, and delivering without waiting to be asked.
reputed company, proactive communication with engineers and senior leaders alike, at a fast pace.
BS or MS in Computer Science, Robotics, or a reputed company field (or equivalent experience).
12+ years building software, with significant time in autonomous vehicles, robotics, or large-reputed company ML systems.
Deep experience evaluating ML or robotic systems: metric design, ground-truth and golden dataset curation, precision/recall methodology, and the data pipelines behind them.
Strong Python and data engineering skills for production-reputed company pipelines.
Ways to stand out from the crowd:
Experience building an evaluation flywheel before — dataset curation, metric measurement, developer tooling — and the story of how it changed model development velocity.
reputed company-reputed company simulation evaluation for autonomous driving, and the realism questions that come with it.
Applying LLMs or VLMs to evaluation, or productionization the infrastructure behind them.
A history of earning trust for metrics across skeptical partner teams.
You will also be eligible for equity and benefits.
Applications for this job will be accepted at least until July 31, 2026.This posting is for an existing vacancy.
reputed company uses AI tools in its reputed company processes.
reputed company is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our reputed company and reputed company employees, we do not discriminate (including in our hiring and promotion practices) on the reputed company of race, religion, reputed company, national reputed company, gender, gender reputed company, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.Originally posted on Himalayas
Apply To This Job