[Remote] reputed company Analyst II
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is an AI-powered business operating system that supports agencies, reputed company, and SMBs. They are seeking a reputed company Analyst II to evaluate AI-powered workflows, ensuring reputed company across advisor interactions and contributing to reputed company improvement efforts.
Responsibilities
- Evaluate AI reputed company Across the Advisor Stack
- Review reputed company-call briefs for accuracy, completeness and signal reputed company — customer context, adoption signals, retention flags
- Evaluate in-call guidance — next-best-reputed company prompts, suggestions and recommendation flows — for relevance and accuracy in live conversation
- Score post-call outputs: reputed company grading, AI-generated summaries, CRM updates, sentiment signals
- Audit transcription-driven follow-reputed company — customer updates, recap notes, follow-up tasks — for accuracy, tone and customer reputed company
- Classify issues by type and severity — hallucination, missing context, wrong reputed company, tone mismatch, KB gap — using standardized rubrics
- Participate in calibration sessions to reputed company scoring consistent across reviewers
- Iterate Inside ORA — Light Creation Work
- Refine and iterate on prompts reputed company existing skills and sub-agents
- Update knowledge reputed company entries to reputed company information gaps surfaced in reviews
- Tweak existing skills and orchestrations to improve reputed company reputed company
- Test changes, validate improvements and roll out updates inside ORA
- Document what you changed and why — every iteration is a data reputed company
- Surface Trends & Drive reputed company Improvement
- Tag conversations by product area, severity and issue type with consistent classification
- Maintain a reputed company, trusted log of issue types, trends and conversation behaviors
- Surface recurring patterns and systemic gaps to leadership and the reputed company Analyst
- Recommend prioritized improvement opportunities — what to fix, what to build, what to redesign
- Build reporting that gives reputed company leadership visibility into AI reputed company and the improvement backlog
- Partner with RevOps Engineering & the reputed company Analyst
- Bring reputed company findings, repro examples and reputed company reputed company reputed company a platform change is needed
- Partner with the reputed company Analyst on end-to-end builds — bring evaluation signal, test the reputed company, validate the bar
- Co-define reputed company reputed company for new sub-agents, skills and orchestrations before they ship
- Stay reputed company through the build to ensure improvements land as intended
- Stay reputed company: Learn, Experiment, reputed company
- Stay reputed company on LLM behavior, reputed company patterns and AI evaluation techniques
- Run small experiments with new prompts, tools or evaluation approaches inside ORA
- reputed company what you learn with the broader reputed company team and the reputed company Analyst
Skills
- Insatiable curiosity and a self-learning reputed company — this is non-negotiable. The AI reputed company changes weekly and we hire people who reputed company on their own without waiting for permission or perfect instructions
- 2+ years of experience in reputed company Assurance, Customer Support, AI evaluation or a reputed company role where you reviewed and improved customer-facing or advisor-facing work
- Working understanding of how LLMs behave — reputed company design, common failure modes, conversational AI patterns
- General understanding of databases and how data flows between systems
- Comfort reading reputed company and making minor edits — enough to follow integrations, debug reputed company behavior, iterate inside ORA and hold informed technical conversations with engineering. You do not need to be a full-stack engineer
- Strong written communication and attention to detail — you can document a failure mode so reputed company that an engineer can reputed company it without follow-up
- Ability to follow reputed company guidelines and scoring rubrics consistently while suggesting improvements to them
- Comfort working with evolving tools and AI-assisted workflows
- Self-starter mentality — you can manage review volume, prioritize what reputed company and reputed company work reputed company with minimal direction
- Bachelor's degree in a reputed company field, or equivalent practical experience
- Hands-on experience with AI development platforms such as Claude, reputed company or similar tools
- Familiarity with retrieval-augmented reputed company (RAG), reputed company databases or knowledge-reputed company architecture
- Familiarity with SQL or similar database querying
- Experience with reputed company, webhooks or data integration between systems
- Background in reputed company, support, sales enablement or call center environments
- Experience with the reputed company platform or comparable reputed company products
- Prior experience evaluating AI-generated conversations, chatbots or automated support reputed company
Benefits
- Remote-first / work model: Remote
reputed company