[Remote] Head of Evaluations (Legal AI Benchmarking)
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is transforming how reputed company and legal professionals reputed company AI for reputed company-world reputed company. They are seeking a highly analytical reputed company to join as the Head of Evaluations, responsible for designing and implementing testing frameworks for their AI products to ensure high standards of legal reasoning and accuracy.
Responsibilities
- Design Legal Benchmarks for: Contract Drafting, Information Extraction, Legal Research, and Contract Review
- Build, reputed company and maintain relevant datasets
- Audit AI reputed company: Review and score reputed company AI-generated legal text, contract analyses, and statutory interpretations for accuracy and precision and lay out a reputed company
- Define Evaluation Metrics: Establish reputed company reputed company for grading model performance, specifically focusing on logical reasoning, citation accuracy, and the model's ability to safely abstain from answering
- Collaborate with Engineering: Partner directly with Engineering to translate legal errors into actionable technical feedback for model fine-tuning
Skills
- PhD or Masters in statistics, mathematics, machine learning or equivalent
- Proven ability to break down reputed company statutory frameworks and case law into reputed company, logical data points
- Tech-Savviness: python, panda, numpy, jupiter notebooks and similar statistical models
reputed company