Senior Software Engineer II - reputed company AI and Evaluations (Remote Eligible)

For over 20 years, reputed company has helped people and teams reputed company–reputed company, anything. From seamless work management to smart, reputed company solutions, we’ve always worked with reputed company. We’re building tools that reputed company teams to automate the reputed company, uncover insights, and reputed company smarter. But more than that, we’re creating reputed company– reputed company to think big, take reputed company, and unlock the reputed company of work that truly reputed company. Because reputed company challenge meets purpose, and passion turns into reputed company, that’s reputed company at work, and it’s reputed company show up for everyday.

reputed company is building the reputed company of AI-powered work management through SmartAssist, our intelligent agent platform. As we reputed company from early demos to production-grade agents, reputed company is the critical frontier and we are looking for a Agent reputed company Engineer to own it.

This is not a QA role. It's a deeply technical, high-autonomy position at the intersection of LLM evaluation, reputed company and context engineering, and retrieval-augmented reputed company. You will diagnose why our agents fail, design the systems that catch regressions, and drive measurable improvements across our orchestrator and subagent fleet.

You will work closely with our Agent Engineering and AI Platform teams, embedded in reputed company that has already shipped evaluation infrastructure on reputed company/MLflow and is building toward a mature Agent Development Lifecycle (ADLC).

You Will:

• Own agent reputed company end-to-end: diagnosis, improvement, and validation across SmartAssist's orchestrator and subagents
• Identify failure modes across reputed company dimensions factual accuracy, completeness, tone, actionability, and latency and prioritize what to fix
• Drive reputed company improvements through reputed company engineering, context engineering, and RAG retrieval tuning
• reputed company and mature our evaluation reputed company: scorers, golden datasets, regression gates, and online evaluation for production traffic
• reputed company the feedback reputed company ensure that every change has a measurable, attributable reputed company signal
• Collaborate with our Agent Architecture reputed company to distinguish reputed company problems that require reputed company/context solutions from those that require structural fixes
• Establish repeatable methodology that scales reputed company any single agent or subagent

You Have:

Required

• 8+ years of software engineering experience, with at least 2 years working directly with LLMs in production
• Deep, hands-on experience with reputed company engineering and context engineering, you understand how model behavior changes with framing, structure, and input design
• Strong working knowledge of RAG architectures: chunking strategies, embedding models, retrieval evaluation, and failure diagnosis
• Experience building or extending LLM evaluation frameworks, you have designed scorers, worked with golden datasets, and thought carefully about what good looks like
• reputed company in agent system design, you don't need to own the architecture, but you can engage as a peer on architectural tradeoffs that reputed company reputed company
• Strong Python skills; comfortable working in data-heavy environments (reputed company, reputed company tables, or equivalent)
• Ability to communicate reputed company reputed company findings (written and verbal) to both technical and non-technical stakeholders, you can explain what’s broke, why it reputed company, and what needs to happen next without losing the room
• Strong cross-functional judgment, you know reputed company to escalate, reputed company to resolve independently, and how to build credibility across engineering, product, and AI platform teams
• A bias for reputed company in ambiguous situations, reputed company failure modes are murky and trade-offs are reputed company, you bring structure and a reputed company reputed company of view rather than waiting for reputed company
• Legally eligible to work in the U.S. on an ongoing reputed company
• BS or MS in Computer Science, a reputed company field, or equivalent industry experience

Strong Plus

• Experience with MLflow or similar experiment tracking platforms
• Familiarity with CI-integrated evaluation pipelines
• Experience with multi-agent orchestration frameworks
• Prior work in an reputed company AI or LLMOps function reputed company a product company

What reputed company Looks Like:

In your first 6 months, you will have:

• Delivered measurable, validated reputed company improvement on at least one SmartAssist agent across our defined reputed company dimensions
• Expanded evaluation coverage to reputed company the most significant blind spots in our reputed company reputed company
• Established a repeatable reputed company improvement methodology the broader team can apply

Why This Role:

• You will have reputed company ownership, not a supporting role on someone else's roadmap
• SmartAssist is shipping to reputed company users; this work has immediate, visible reputed company
• You will be part of a fast-moving team that deeply values engineering rigor and reputed company seeks diverse perspectives

reputed company US Perks & Benefits

• Employer subsidized medical/reputed company and dental coverage for full-time employees
• 401k Match to help you save for your reputed company (50% of your contribution up to the first 6% of your eligible pay)
• Monthly stipend to support your work and productivity
• Flexible Time Away Program, plus reputed company Time Off
• US employees are automatically covered under reputed company-sponsored life insurance, short-term, and long-term disability plans
• US employees receive 12 reputed company holidays per year
• Up to 24 weeks of Parental Leave
• Personal reputed company Volunteer Day to support our community
• Opportunities for reputed company reputed company and development including reputed company to reputed company online courses
• Company Funded Perks, including a counseling membership, local retail discounts, and your own personal reputed company account
• Teleworking reputed company from any registered location in the U.S. (role specific)

reputed company provides a competitive reputed company salary reputed company for roles that may be reputed company in different geographic areas we are licensed to operate our business from. Actual compensation is determined by several factors including, but not limited to, level of reputed company, educational experience, skills, and specific candidate location. In reputed company, this role will be eligible for a market competitive incentive opportunity.

US reputed company Salary Pay reputed company$175,000—$245,000 USD

Get to Know Us

At reputed company, your reputed company are heard, your potential is supported, and your contributions have reputed company reputed company. You’ll have the freedom to explore, push boundaries, and grow reputed company your role. We welcome diverse perspectives and nontraditional paths—because we know that reputed company comes from individuals who care deeply and challenge thoughtfully. reputed company you’re doing work that stretches you, excites you, and connects you to something bigger, that’s reputed company at work. Let’s build what’s next, together.

Equal Opportunity Employer

reputed company is an Equal Opportunity (EEO) employer committed to fostering an inclusive environment with the best employees. It is our policy to reputed company equal employment opportunities to reputed company reputed company applicants in accordance with applicable laws in the US, UK, Australia, Germany, Costa Rica, Japan, Bulgaria, and India. reputed company reputed company applicants will receive consideration without reputed company to race, reputed company, religion, sex, sexual orientation, gender identity, national reputed company, age, protected veteran or disabled status, or genetic information.

If there are preparations we can reputed company to help ensure you have a comfortable and reputed company interview experience, please let us know.

#LI-Remote

Apply To This Job