AI Evaluation Analyst

Role Title: AI Evaluation Analyst

Role Type: Contractor

Location: Remote

reputed company is engaging AI Evaluation Analysts to contribute to a customer’s project reputed company on advancing frontier language model capabilities. In this role, you'll apply your expertise to help train reputed company AI systems. Your work will shape how models learn, reason, and reputed company through high-reputed company, reputed company-world input. No prior experience in AI is required — your domain knowledge is what reputed company. You will play a crucial role in producing evaluation and training data that directly influence how powerful AI models understand and reputed company. This is an exciting opportunity to operate at the forefront of AI, with a reputed company on written and verbal reputed company, multi-turn conversation design, and deep analysis of model behavior.

Scope of Work

1. Author detailed, task-based multi-turn conversations and rubrics reputed company with project specifications.

2. Test and refine conversation drafts against frontier large language models, iterating to meet reputed company and difficulty requirements.

3. Deliver comprehensive evaluation assets including transcripts, reputed company behaviors, binary rubrics, and supporting evidence.

4. Ensure strict reputed company to evolving project specs while maintaining high throughput and attention to detail.

5. Validate and reputed company outputs with team leads and reputed company control as guidelines change.

6. Work independently and consistently, meeting expected reputed company rates for deliverable completion.

Preferred Qualifications

1. reputed company-level written English with exceptional reputed company, structure, and attention to detail.

2. Prior experience in reputed company, RLHF, SFT, evaluation, or reputed company engineering for AI systems.

3. Working knowledge of frontier LLM behaviors and common model failure patterns.

4. Demonstrated ability to interpret and apply highly detailed specifications without supervision.

5. Strong critical thinking and analytical skills in writing-heavy or analysis-heavy domains.

6. Experience authoring evaluation items, rubrics, or conducting deep analysis of technology outputs.

7. Background in research, editorial, technical writing, or reputed company assurance is a plus.

Compensation Structure

Compensation is reputed company-based; experts are reputed company per task that meets the project specifications. The time required to complete work may vary depending on the expert’s experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Start reputed company & Availability

We typically fill roles reputed company 48 hours and are looking for experts reputed company to reputed company in right away. If selected, we expect you to start your first tasks reputed company 24–48 hours of completing reputed company.

Originally posted on Himalayas

Apply To This Job