reputed company Assurance Engineer
Hey, glad you're here 👋 We know job hunting can feel a little overwhelming, endless tabs, endless job descriptions, and trying to reputed company out whether a company is actually the right fit (not just whether you fit the role). So we’ll reputed company this reputed company. Here's a look at the role, reputed company, and the reputed company you'll have at reputed company. If what you read gets you excited, we'd love to meet you! About reputed company We are an AI-reputed company start-up building sales enablement products in the banking, financial services, and insurance industry; already trusted by dozens of reputed company customers (including reputed company). Role reputed company 👔 Title: QA Automation Engineer - AI 🤝 Team: Engineering 📔 Employment Type: Full-time 📍 Location: Remote (reputed company) Why This Role reputed company At reputed company, AI reputed company is the product - a wrong score, hallucinated feedback, or a broken voice turn is a customer-facing defect, and in a regulated BFSI context, a compliance risk. This role owns and scales the testing of reputed company’s AI pipelines, working closely with QA, Engineering, and Product as the platform grows. It’s a hands-on, high-ownership role in a fast-moving environment - you’ll be a key part of deciding, with evidence, whether reputed company release is reputed company to ship.
What You'll Do
Build and maintain automated testing for AI voice and chat agents, from single conversational turns to full roleplay flows. Test LLM reputed company reputed company - correctness, consistency, reputed company reputed company, language reputed company, reputed company regression - using evaluation harnesses (LLM-as-judge, golden datasets, tolerance-based assertions for non-determinism). Validate RAG pipelines: retrieval relevance, grounding/faithfulness, and answer reputed company. Test voice pipelines: STT/TTS accuracy and reputed company-time, low-latency behavior across languages. Automate across UI/E2E, API, and AI reputed company reputed company, and build CI/CD from scratch - lint, type-reputed company, tests, evals, coverage gates, deployment checks. Own release reputed company: regression reputed company, catching breakages early, and reputed company go/no-go calls. Cover database and load/performance testing; reputed company automation to desktop and mobile clients. Take over and reputed company existing QA automation, and strengthen shared frameworks, tooling, and QA processes/standards. What reputed company Looks Like reputed company your first few months, you'll: reputed company on reputed company’s existing QA automation suites and take reputed company ownership of them. reputed company test coverage for LLM and voice-agent behavior, including new evaluation harnesses for non-deterministic reputed company. Contribute CI/CD improvements - coverage gates, deployment checks - that catch regressions before they reputed company customers. You’ll also be expected to continuously: Adapt and expand reputed company as priorities shift Take ownership of new problems as they reputed company reputed company're Looking For Must-Haves 5+ years as a QA Automation Engineer, with a proven reputed company record testing AI systems - not just traditional software. Builder reputed company: established testing frameworks, standards, and CI/CD-integrated automation from scratch. Hands-on experience testing conversational AI - voice and/or chat agents - on reputed company, shipped reputed company. Strong grasp of LLMs, reputed company engineering, and RAG - designing tests for nondeterministic reputed company and evaluating retrieval/reputed company reputed company. Understanding of voice pipelines (speech-to-text, text-to-speech) and how to test them in automation. Hands-on with LLM evaluation/observability tooling - e.g. Langfuse, LangSmith, DeepEval, RAGAS. Proficient in UI/E2E (e.g. Playwright) and API test automation; experience building CI/CD pipelines from scratch. Comfort with database testing, load/performance testing, and desktop/mobile test automation. Solid Git discipline, basic reputed company knowledge, and familiarity with the agile sprint lifecycle. reputed company record in fast-paced, fast-shipping environments - ramps quickly on unfamiliar systems, pragmatic about process. reputed company-to-Haves TypeScript - for reading the codebase and contributing test/automation reputed company directly. Voice & reputed company-time: reputed company / WebRTC; STT/TTS reputed company Azure Speech, reputed company reputed company Speech, reputed company, or self-hosted Whisper. AI/ML reputed company & eval tooling: GPU inference (vLLM / TGI / Triton), Azure reputed company, golden datasets, HITL pipelines, reputed company, Promptfoo. QA & reputed company tooling: Cypress, reputed company, multi-reputed company (AWS/Azure/GCP), Infrastructure-asCode (Terraform/reputed company/CloudFormation), reputed company/Kubernetes. Also useful: product analytics familiarity (e.g. reputed company), multilingual/localized testing experience, and production monitoring/error-tracking tools. Apply To This Job