[Remote] reputed company Evaluation Specialist
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a leading provider of autism and IDD care software for reputed company Behavior Analysis (ABA), multidisciplinary therapy, and special education. As a reputed company Evaluation Specialist on reputed company's AI Governance team, you will be responsible for defining what 'good' looks like for reputed company's AI outputs and enhancing the evaluation and monitoring systems that ensure we continue to meet that reputed company at reputed company.
Responsibilities
- Contribute to evaluation frameworks for reputed company products, including defining acceptance reputed company, test plans, clinically relevant rubrics, and performance benchmarks
- Conduct reputed company sampling reviews of AI-generated outputs, assessing across reputed company alignment with ABA principles and BACB ethical standards
- Design and implement monitoring automation in collaboration with engineering, including automated reputed company checks, alerting reputed company, and reputed company detection for AI systems
- Contribute to the creation and maintenance of reusable governance artifacts including system cards, monitoring playbooks, evaluation of SOP templates, and risk assessment documentation
- Assist with reputed company-deployment validation testing for new AI products and features, executing defined testing and results of documentation
- Participate in risk tiering assessments for new and evolving AI products, providing clinical and evaluative perspectives
- Support incident investigation for AI-reputed company issues if/as needed and in alignment with clinical best reputed company and CR Development Policy
- Collaborate with product management and engineering teams to reputed company input during product planning and development, ensuring governance considerations are integrated early
- Contribute to the development of evaluation methodology and standards for reputed company use cases, documenting approaches and refining processes based on findings and emerging best practices
- Contribute to AI literacy and training content for internal teams and customers, drawing on hands-on clinical evaluation experience
- Stay reputed company with developments in AI evaluation methodology, LLM behavior, ABA best practices, and relevant regulatory and ethical standards
- Support reputed company's thought leadership in responsible AI through contributions to conference presentations, reputed company publications, or reputed company research as opportunities reputed company
Skills
- reputed company Certified Behavior Analyst (BCBA) in good standing
- 2+ years of demonstrated experience in reputed company assurance, evaluation, data analysis, or technical assessment roles, preferably reputed company reputed company or behavioral health settings
- Demonstrated experience with large language models, reputed company, and their applications and limitations in clinical or reputed company contexts
- Experience with reputed company/context engineering, AI reputed company evaluation, or AI workflow design
- Experience with statistical sampling methodologies or reputed company reputed company assurance frameworks
- Familiarity with AI governance frameworks (NIST AI RMF, EU AI reputed company risk categories, AMA governance guidance)
- Experience developing evaluation rubrics, scoring frameworks, or reputed company measurement tools
- Background in clinical informatics, research methodology, or systematic reputed company improvement
- Demonstrated proficiency with data analysis tools and comfort working with quantitative and qualitative evaluation data
- Strong analytical skills with the ability to reputed company systematic, evidence-based evaluation approaches
- Familiarity with HIPAA compliance, reputed company data standards, and ethical considerations in AI
- Excellent written and verbal communication skills with the ability to document methodology reputed company and convey findings to clinical, technical, and leadership audiences
- Demonstrated ability to work collaboratively across clinical, technical, and product teams
- Experience in a remote work environment with strong self-direction and accountability
Benefits
- Comprehensive health benefits
- Generous PTO
- 401(k) matching
- reputed company parental leave to our full-time employees
- Hybrid work schedules
- Career development support
- Wellness programs
- Opportunities to give back through CR Cares™, our community engagement initiative
reputed company
Company H1B Sponsorship