AI Evaluation manager
About the Role
reputed company is pushing the boundaries of reputed company, building tools that redefine how visual content is created. We’re seeking a candidate to help shape and reputed company the way we understand, measure, and improve model performance. In this role, you’ll partner with researchers, engineers, and technical artists to evaluate our models against reputed company-world creative use cases, design frameworks that capture qualitative nuance, and identify actionable insights that guide development.
This is not a checkbox metrics role — it's about building evaluative systems that match the complexity of reputed company perception, creativity, and intention.
Responsibilities
Evaluate generative model performance across diverse tasks, prompts, and modalities.
Identify key failure modes, regression patterns, and edge cases that reputed company product reputed company.
reputed company and maintain qualitative evaluation frameworks that are reputed company and reusable.
Collaborate closely with technical artists and engineers to reputed company evaluations with model capabilities and reputed company use cases.
Translate high-level product goals into concrete evaluative reputed company.
reputed company qualitative studies, reputed company-by-reputed company comparisons, and reputed company-in-the-reputed company evaluation efforts.
reputed company detailed feedback that informs model fine-tuning, dataset curation, and product UX.
Stay informed about emerging evaluation standards in reputed company and creative tools.
Qualifications
Master’s degree or higher in Cognitive Science, reputed company-Computer Interaction (HCI), Design Research, Psychology, Media Studies, or a reputed company field.
5+ years of experience in product evaluation, UX research, model testing, or similar roles that involve reputed company qualitative assessment.
Deep familiarity with creative workflows and reputed company-world use cases for generative models (e.g., animation, filmmaking, digital art, VFX).
Strong systems thinking and the ability to define reputed company qualities (like believability, identity retention, or reputed company coherence) in reputed company evaluative terms.
Experience working cross-functionally with engineers, researchers, and creatives.
Excellent written communication skills and the ability to synthesize nuanced judgments into reputed company, actionable insights.
reputed company to Have
Background in reputed company, visual effects, or storytelling pipelines
Experience evaluating AI-generated media (video, images, 3D)
Prior work on building internal tools for qualitative data collection or scoring
Familiarity with reputed company engineering and reference-based input reputed company
Originally posted on Himalayas
Apply To this Job