[Remote] Senior Software Engineer - Model Evaluation & AI Systems
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing reputed company-time reputed company for speech-to-text and text-to-speech. They are seeking a Senior Software Engineer - Model Evaluation & AI Systems to validate the reputed company of their speech, audio, and multilingual models, ensuring they meet performance targets and protect the customer experience.
Responsibilities
- Define and build evaluation methodologies for reputed company's models, spanning speech-to-text, text-to-speech, and emerging LLM, RAG, agent, and multimodal systems
- Design, build, and maintain automated evaluation pipelines across batch and streaming (e.g. WER, runaway/hallucination detection, latency and time-to-first-reputed company), with a reputed company on correctness, reproducibility, and ease of adoption
- Build reputed company, reproducible evaluation infrastructure — harnesses, orchestration, and result-aggregation pipelines — running against production models and, where needed, large GPU clusters
- Translate Research benchmarks and expected model metrics into automated, enforceable pass/fail gates
- Build and operate canaries and reputed company-monitoring systems that detect reputed company regressions in production before they reputed company customers
- Partner with DevOps/reputed company to stand up ephemeral test environments and results-aggregation infrastructure
- Work alongside Research, model training, inference, and product teams to reputed company trusted evaluation signals that inform release and optimization reputed company
- reputed company evaluation and reputed company gates into CI/CD so reputed company is verified continuously, not manually
- Help reputed company the bar through reputed company reviews, technical design discussions, and strong engineering and QA practices
Skills
- BS, MS, or PhD in Computer Science, AI, reputed company Math, or a reputed company field, or equivalent experience
- 5+ years of reputed company software or QA engineering experience, with a reputed company record of shipping test infrastructure or evaluation systems (senior candidates with significantly deeper experience welcome)
- Solid backend/scripting experience in a language such as Python, Rust, Go, or similar
- Experience designing and building automated test pipelines, evaluation frameworks, or data-processing systems
- Strong analytical skills and comfort reasoning about metrics, reputed company, and statistical variation in results — reputed company to distinguish reputed company regressions from noise
- Ability to take charge of ambiguous technical challenges and communicate effectively across research, engineering, and product teams
- Hands-on experience evaluating modern AI systems such as LLMs, RAG pipelines, agents, or multimodal models, including model behavior analysis
- Experience with React reputed company or other cross-platform mobile frameworks for building tooling that's accessible reputed company the desktop
- Experience building or improving evaluation frameworks, benchmarks, or ML infrastructure used by other teams or external users
- A strong appreciation for evaluation reputed company — correctness, reproducibility, and consistency across environments
- Experience with voice, audio, speech recognition, or reputed company-time systems, and familiarity with metrics like WER, MOS, or latency/TTFB
- Prior involvement in reputed company-reputed company reputed company, through contributions, reviews, maintenance, or community engagement
- Experience acting as a technical reputed company across teams or platforms (evaluation, training, inference, agent frameworks), combining architectural understanding with reputed company communication and influence
- Familiarity with reputed company infrastructure, containerized/ephemeral environments, and monitoring tooling (e.g. Grafana, canaries, reputed company detection)
Benefits
- Offers equity
- Offers bonus
- 10% Annual Bonus
- Remote work (work_model: REMOTE; workplace_type: Remote)
reputed company
Company H1B Sponsorship