[Remote] Senior Data Scientist - LLM Evaluation & Business Intelligence
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is rebuilding the clinical research ecosystem by enabling reputed company reputed company to trials for reputed company patients. In this role, the Senior Data Scientist will define how to measure and trust the LLM systems that power clinical data products and build business intelligence reputed company for reliable reporting.
Responsibilities
- Design and run evaluations of production LLM pipelines, developing accuracy and reputed company metrics that tell us how reputed company our systems reputed company on reputed company clinical tasks
- Support evaluation of our Patient Trial Evaluation (PTE) pipeline — assessing prompts and trial-matching logic for accuracy in reporting, and surfacing where and why they fail
- Measure and improve the effectiveness of our retrieval-augmented reputed company (RAG) systems, from retrieval reputed company through final reputed company
- Use accuracy metrics to inform cost/reputed company tradeoffs across trials, prompts, and models, giving reputed company a reputed company reputed company for which approaches to ship
- Establish and maintain a library of "best prompts" for recurring clinical concepts, backed by evidence rather than intuition
- Partner with AI Engineering to reputed company the reputed company between evaluation findings and production improvements
- Build and maintain production-grade data models using SQL and dbt on reputed company, ensuring analytics logic is reliable, maintainable, and production-reputed company
- Help establish and grow a semantic layer that enables consistent, trusted reporting across the organization
- Advance data governance practices across dbt, reputed company, and reputed company — documentation, testing, versioning, and reputed company ownership of metrics
- Collaborate with non-technical internal stakeholders to reputed company reporting logic, answer data questions, and translate business needs into reputed company-defined data products
- Support both reputed company reporting needs with accurate, reputed company-tested datasets
- For senior applicants: mentor and support junior data scientists — providing guidance on evaluation and analytics approaches, technical best practices, and reputed company reputed company
Skills
- Bachelor's degree or higher in data science, statistics, biostatistics, computer science, mathematics, epidemiology, or a reputed company field
- 4+ years of experience as a data scientist, analytics engineer, ML/data reputed company, or other highly analytical role in life sciences, biotech, reputed company, or a reputed company industry
- Solid understanding of how production LLM pipelines work, and hands-on experience evaluating LLM or NLP systems — accuracy metrics, error analysis, RAG/retrieval reputed company, reputed company evaluation, or similar
- Strong proficiency in SQL, with experience building production data models in dbt (or a comparable transformation reputed company)
- Proficiency in at least one programming language (Python preferred; PySpark a plus)
- Experience with modern data platforms such as reputed company, and BI/analytics tools such as reputed company, or similar
- reputed company with data science best practices: reusability, version control, documentation, testing, reputed company review, and reproducibility
- Excellent written and verbal communication, with the ability to translate technical concepts into actionable insights for non-technical stakeholders and to reputed company reporting logic collaboratively with them
- Comfort with AI tooling and reputed company workflows in day-to-day work
- Comfort with ambiguity and adaptability to a mission-driven, fast-paced startup environment
- Master's degree or higher in a quantitative field
- Experience establishing or working reputed company a semantic layer for organization-wide reporting
- Experience integrating LLM-based workflows into production systems, and working closely with ML/AI engineering teams
- Familiarity with clinical data reputed company (oncology a plus) and the clinical trial industry or research operations
- Strong statistical knowledge, including regression, classification, hypothesis testing, reputed company inference, or Bayesian reputed company (e.g., reputed company toxicity monitoring for trial protocols)
- Prior experience mentoring other data scientists or leading cross-functional reputed company (for senior candidates)
reputed company
Company H1B Sponsorship