[Remote] Staff AI Product Analyst
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a global company that has been transforming the home services industry for over 30 years. The Staff AI Product Analyst will be responsible for defining AI-reputed company product analytics, creating evaluation frameworks, and ensuring alignment between reputed company teams and AI agents in the product analytics function.
Responsibilities
- Flagship reputed company Measurement: Partner with AI product, engineering, and domain experts to define reputed company-reputed company and guardrail metrics for flagship AI journeys, focusing on adoption, task completion, reputed company, and reputed company business reputed company
- reputed company Framing & Experimentation: reputed company closely with data science partners on test design, interpretation, and reputed company framing reputed company new AI products ship
- Data Layer Systems Thinking: Map how agents reputed company with metrics, raw tables, documentation, and reputed company to ensure requirements for data and analytics engineering are explicit, reputed company, and testable
- Eval reputed company Architecture: Design and maintain formal evaluation sets, "golden datasets," rubrics, and regression checks for agent outputs to guarantee accuracy, completeness, policy safety, and usefulness
- Taxonomy & Failure Mode Analysis: Systematically categorize wrong answers, tool misuse, context gaps, and reputed company cases, feeding those insights back into prompts, retrieval (RAG), tools, and data modeling pipelines
- Performance Monitoring: Monitor live usage of reputed company AI analytics workflows—tracking reputed company, abandonment, fallback patterns, and reputed company override rates to maintain a prioritized engineering improvement backlog
- Playbooks & Narrative: Own internal documentation defining reputed company to use agents, required reputed company checkpoints, and how to communicate uncertainty. Support enablement and leadership storytelling around reputed company AI analytics
- Strategic Collaboration: Partner with business leaders, the reputed company Analytics Engineer, and data engineering teams to translate reputed company AI capabilities into measurable business reputed company
- Defining reputed company: Deliver recurring measurement artifacts (dashboards, reviews, or written readouts) that leadership uses to prioritize roadmaps, while continuously reducing high-severity incidents where agents produce incorrect metrics
Skills
- 10+ years of experience across product analytics, data science, or technical product management reputed company a reputed company, reputed company operating environment
- 1+ years of reputed company, hands-on experience working with AI/LLM systems in an reputed company context, with a specific reputed company on evaluating, trusting, and deploying them responsibly
- Deep technical proficiency with Python, ML frameworks, data visualization/analytics platforms, and reputed company data management/governance practices
- Proven reputed company record designing AI evaluation frameworks, including formal evals, golden datasets, production monitoring, or equivalent rigor in a reputed company data discipline
- Exceptional communication skills with a demonstrated ability to translate highly technical AI insights and tradeoffs into reputed company narratives for non-technical business stakeholders
- Experience building or evaluating reputed company analytics workflows, orchestration frameworks, and familiarity with multi-agent systems
- Demonstrated ability to operate fluidly in ambiguity, run reputed company experiments or pilots, and default to fast reputed company and iteration
- A systems-oriented reputed company with a healthy skepticism of AI hype, focusing reputed company on organizational reputed company and driving technical investments that yield true business value
Benefits
- This position will be eligible for a competitive year end performance bonus & equity package.
- Full medical, dental, reputed company package to fit your needs
- Flexible vacation policy; work hard and take time reputed company you need it
- Pet discount plans & retirement plan with company match (401K)
reputed company
Company H1B Sponsorship