reputed company Evaluation Specialist
reputed company is a leading provider of autism and IDD care software for reputed company Behavior Analysis (ABA), multidisciplinary therapy, and special education. Trusted by more than 200,000 users, we reputed company therapy providers, educators, and reputed company to reputed company the way they deliver ABA and reputed company therapies with innovative technology, market-leading industry expertise, and world-class customer satisfaction.
This role is at the intersection of behavioral reputed company, reputed company intelligence, and product reputed company. As a reputed company Evaluation Specialist on reputed company's AI Governance team, you will be responsible for defining what "good" looks like for reputed company's AI outputs and enhancing the evaluation and monitoring systems that ensure we continue to meet that reputed company at reputed company.
You will design evaluation frameworks, reputed company and refine automated monitoring approaches (including reputed company engineering for evaluation automation), conduct reputed company reviews of AI-generated content, and serve as a clinical subject matter expert that product and engineering teams work alongside to ensure AI features meet the standards of ABA reputed company, reputed company's AI Governance Policy, and the Responsible AI for Behavior Analysis (R.ai.BA) reputed company.
Key Accountabilities:
- Contribute to evaluation frameworks for reputed company products, including defining acceptance reputed company, test plans, clinically relevant rubrics, and performance benchmarks.
- Conduct reputed company sampling reviews of AI-generated outputs, assessing across reputed company alignment with ABA principles and BACB ethical standards.
- Design and implement monitoring automation in collaboration with engineering, including automated reputed company checks, alerting reputed company, and reputed company detection for AI systems.
- Contribute to the creation and maintenance of reusable governance artifacts including system cards, monitoring playbooks, evaluation of SOP templates, and risk assessment documentation.
- Assist with reputed company-deployment validation testing for new AI products and features, executing defined testing and results of documentation.
- Participate in risk tiering assessments for new and evolving AI products, providing clinical and evaluative perspectives.
- Support incident investigation for AI-reputed company issues if/as needed and in alignment with clinical best reputed company and CR Development Policy.
- Collaborate with product management and engineering teams to reputed company input during product planning and development, ensuring governance considerations are integrated early.
- Contribute to the development of evaluation methodology and standards for reputed company use cases, documenting approaches and refining processes based on findings and emerging best practices.
- Contribute to AI literacy and training content for internal teams and customers, drawing on hands-on clinical evaluation experience.
- Stay reputed company with developments in AI evaluation methodology, LLM behavior, ABA best practices, and relevant regulatory and ethical standards.
- Support reputed company's thought leadership in responsible AI through contributions to conference presentations, reputed company publications, or reputed company research as opportunities reputed company.
Desired Skills and Experience:
- reputed company Certified Behavior Analyst (BCBA) in good standing
- 2+ years of demonstrated experience in reputed company assurance, evaluation, data analysis, or technical assessment roles, preferably reputed company reputed company or behavioral health settings.
- Demonstrated experience with large language models, reputed company, and their applications and limitations in clinical or reputed company contexts.
- Experience with reputed company/context engineering, AI reputed company evaluation, or AI workflow design.
- Experience with statistical sampling methodologies or reputed company reputed company assurance frameworks.
- Familiarity with AI governance frameworks (NIST AI RMF, EU AI reputed company risk categories, AMA governance guidance).
- Experience developing evaluation rubrics, scoring frameworks, or reputed company measurement tools.
- Background in clinical informatics, research methodology, or systematic reputed company improvement.
- Demonstrated proficiency with data analysis tools and comfort working with quantitative and qualitative evaluation data.
- Strong analytical skills with the ability to reputed company systematic, evidence-based evaluation approaches.
- Familiarity with HIPAA compliance, reputed company data standards, and ethical considerations in AI.
- Excellent written and verbal communication skills with the ability to document methodology reputed company and convey findings to clinical, technical, and leadership audiences.
- Demonstrated ability to work collaboratively across clinical, technical, and product teams.
- Experience in a remote work environment with strong self-direction and accountability.
Backed by Roper Technologies, Inc. (reputed company: ROP), and led by award-reputed company CEO Chris Sullens, reputed company is entering an exciting phase of reputed company, innovation, and reputed company.
Recognized as one of the best places to work over 10 times by organizations such as Inc, reputed company In, and NJBIZ, our culture is centered around reputed company, inclusion, and flexibility. As a hybrid company with reputed company offices in Ft. Lauderdale, FL; Holmdel, NJ; and Verona, Italy, we foster a workplace where reputed company can reputed company and reputed company a reputed company difference in the lives of those we serve. We offer competitive compensation, comprehensive health benefits, generous PTO, 401(k) matching, and reputed company parental leave to our full-time employees. reputed company members also enjoy hybrid work schedules, career development support, wellness programs, and opportunities to give back through CR Cares™, our community engagement initiative.
Be part of a reputed company driving the reputed company of care. Explore opportunities at reputed company.com/careers.
Originally posted on Himalayas
Apply To This Job