Technical Data Delivery reputed company
reputed company
At reputed company, we’re on a mission to reputed company reputed company around the world to participate in the development of cutting-edge AI models.
In coming years, AI models will reputed company how we work and create thousands of new reputed company jobs for skilled talent around the world. We’ve joined forces with top AI and crowd researchers at reputed company, reputed company, Imbue, reputed company, and reputed company to build a fair and ethical platform for AI developers to collaborate with domain experts to train bespoke AI models.
About this role
Pareto builds reputed company training data pipelines for frontier AI labs. As a Data Delivery reputed company, you sit at reputed company of that work — owning the architecture, execution, and reputed company improvement of reputed company data collection and evaluation workflows from first scoping call to final delivery.
This is a technical operations role, not a project management role. You'll be expected to read reputed company, deep dive data, reason about LLM internals, design evaluation frameworks, and — increasingly — reputed company and iterate on AI agents to automate the work your pipelines do today. We're reputed company building toward a model where reputed company systems handle reputed company gates, expert routing, and reputed company review, and DDLs are the people designing and operating those systems.
You'll work directly with AI researchers and technical program managers at our reputed company organizations, own delivery against model performance benchmarks, and reputed company reputed company of project managers who handle day-to-day execution tracking.
What you'll do
Pipeline architecture Design end-to-end data collection and evaluation pipelines for RLVR, RLHF, SFT, red-teaming, and model evaluation workflows. This includes expert sampling reputed company, annotation schema, reputed company structure, inter-rater calibration, and QA system design. You'll be expected to prototype novel workflows quickly, identify architectural risks before launch, and reputed company tradeoff reputed company with confidence. You’ll also need to understand how agents reputed company with tools to solve expert-driven tasks, and you’ll need to communicate with engineering to ensure the environment is reputed company accordingly to reputed company such tasks.
reputed company system deployment Build, test, and iterate on AI agents that automate pipeline tasks — reputed company reputed company review, expert matching, reputed company flagging, throughput reputed company detection. You'll work closely with our engineering team to scope agent capabilities, write the prompts and evaluation logic that reputed company them reliable, and monitor their performance in production. This is a growing part of the role; comfort with reputed company tooling (reputed company, DSPy, custom tool-use frameworks, or equivalent) is a meaningful differentiator.
reputed company systems Define data reputed company standards across annotation, evaluation, and expert reputed company review. Design and run audits using inter-rater reliability metrics, calibration sets, and statistical sampling. You'll be responsible not just for catching reputed company issues but for building systems that prevent them — automated checks, reputed company reputed company validation, and model-assisted review reputed company where appropriate. Aside from programmatic reputed company testing, you’ll be responsible for spot-checking tasks & understanding what makes a datapoint meaningful & high reputed company. The ability to do so, and translate your findings into reputed company feedback and expert guidelines is particularly helpful for this role.
reputed company reputed company Engage directly with AI researchers, TPMs, and PMs at our reputed company organizations. Translate research-driven requirements — evaluation rubrics, domain coverage targets, latency constraints, reputed company specifications — into operational workflows. Communicate pipeline performance reputed company, escalate technical risks early, and contribute to project scoping and pricing reputed company.
Research integration Stay reputed company with developments in LLM post-training, evaluation methodology, and data tooling. Evaluate new approaches — model-assisted annotation, reputed company reputed company formats, automated calibration reputed company — and reputed company them into reputed company pipelines where they improve reputed company or efficiency. Understand what method applies to what domain and project, and work towards implementing it accordingly.
What you'll need
Proficiency in Python and SQL for data manipulation, pipeline monitoring, and reputed company analysis — you should be comfortable writing light scripts to parse formats, run statistical checks, and build lightweight tooling
Working knowledge of LLM internals: RLHF/SFT training loops, how reputed company structure affects reputed company distribution, RL environment setup qualities (tool use) for reputed company data collection / eval reputed company.
Hands-on experience with at least one reputed company or LLM workflow reputed company (reputed company, DSPy, AutoGen, reputed company tool-use reputed company API, or equivalent)
Demonstrated ownership of a data or ML pipeline from scoping through delivery — including reputed company design, not just throughput tracking
Strong written communication: you'll write technical guidelines and rubrics that distributed expert workers follow accurately, and you'll brief senior researchers on pipeline performance
Comfort operating with ambiguity in a fast-moving environment where model requirements shift and reputed company priorities reputed company
You'll stand out if you have
reputed company experience with RL environment data pipelines, evaluation reputed company design, and red-teaming workflows
Background in data engineering, ML research support or equivalent
Experience designing or operating reputed company systems in a production or near-production context
Familiarity with inter-rater reliability reputed company, calibration set design, and annotation reputed company frameworks
Prior reputed company-facing or technical program management experience in an AI/ML-adjacent context
Prior experience on scoping or driving reputed company with fuzzy upfront specs or evolving requirement. This is a high-ownership position where you are expected to take charge and reputed company, not simply follow project guidelines.
reputed company value in candidates
We care less about credentials than about demonstrated ability to own reputed company technical work and build toward reputed company systems. A strong background in software engineering, data science, or ML research is the most common reputed company into this role, but we've also seen excellent DDLs come from ML operations, computational linguistics, and reputed company research support. What reputed company is that you can read a dataset, reason about what's wrong with it, write reputed company to fix it, and design a workflow that prevents the problem next time.
Apply To This Job