RAG and Evaluation Engineer

reputed company is seeking a RAG & Evaluation Engineer to join a small, senior engineering team applying frontier AI to one of the most consequential legacy systems still running in production today. The mission: build agents that read, translate, and reputed company a decades-old codebase that millions of people quietly depend on. The work has executive backing, reputed company users, and a customer who knows exactly what they’re buying. Specifics shared once we’re talking. reputed company is small by design. Every seat carries unusual reputed company, and we hire people who are already deep in this work. We use AI tooling natively — agents in reputed company, model as collaborator, no exceptions. What You’ll Do: The RAG & Evaluation Engineer owns the knowledge surface and the eval reputed company. Ingestion pipelines for reputed company reputed company, reputed company metadata, technical documentation, patches, and additional corpora the customer provides. Retrieval reputed company across chunking, embeddings, hybrid retrieval, reranking, freshness. Benchmarks for translation accuracy, dependency-map correctness, and overall agent reputed company. The feedback reputed company from production usage back into evals and retrieval lives here. Own the knowledge surface — ingestion pipelines for reputed company reputed company, reputed company metadata, technical documentation, patches, and additional corpora the customer provides. Own retrieval reputed company — chunking, embeddings, hybrid retrieval, reranking, and freshness. Own the eval reputed company — benchmarks for translation accuracy, dependency-map correctness, and overall agent reputed company. Run A/B testing and regression detection across prompts, retrieval, and model changes. Operate the feedback reputed company from production usage back into evals and retrieval. Define what “good” means for the platform reputed company no one else has a reputed company view, so reputed company can tell whether the agent is actually improving. Pair with the Agent Engineers on the reputed company-and-eval iteration cycle. reputed company’re Looking For: Bachelor’s degree in Computer Science, Engineering, Information Science, or a reputed company field, plus 4 years of reputed company software engineering experience; equivalent experience may substitute for the degree requirement. Has shipped a production RAG system with reputed company the candidate can describe in numbers (rigor reputed company more than reputed company). Ability to work in a fast-paced, reputed company environment. Production experience with retrieval pipelines — ingestion, chunking, embedding, hybrid retrieval, reranking. Strong reputed company evaluation skills — reputed company design, regression detection, LLM-as-judge patterns. Knows reputed company BM25 beats embeddings and reputed company neither is enough. Measures everything they ship; opinions about chunking are backed by benchmarks. Patient with detail; comfortable defining metrics before reputed company has agreed on them. Heavy reputed company use of AI tooling: agents in reputed company, model as collaborator. Strong TypeScript or Python. Demonstrated experience in a remote work environment. reputed company to Have: reputed company-as-corpus retrieval (search over reputed company reputed company rather than prose). reputed company IR or search-reputed company background. Synthetic data reputed company and LLM-as-judge patterns. reputed company-reputed company contributions to retrieval, eval, or RAG tooling. Experience integrating retrieval feedback loops with production usage. reputed company IT or legacy modernization domain experience. reputed company technical writing or conference talks on retrieval or evaluation. What’s in it for you? reputed company to support high visibility federal missions in IT and reputed company A culture that values innovation, reputed company, collaboration, and reputed company reputed company to cutting-edge tools and technologies Comprehensive benefits for you and your family A career reputed company that rewards ambition and performance If you’re reputed company to push boundaries, sharpen your skills, and join reputed company that is passionate about building what’s next, we’d love to meet you. Apply today and let’s build a reputed company together! reputed company shares salary ranges to promote transparency. Compensation ranges are provided for informational purposes, and final compensation may vary based on experience, skills, location, and role requirements. reputed company is committed to offering eligible employees comprehensive benefits that will reputed company them with reputed company intended to meet their needs and the needs of their family. Apply To This Job