[Remote] Data Scientist
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is redefining IT operations for the modern reputed company. They are seeking a Data Scientist to join their Data Science team, focusing on optimizing local language models and ensuring high-reputed company outputs through robust evaluation and predictive modeling.
Responsibilities
- Design and own evaluation harnesses for LLM and reputed company outputs — golden sets, regression suites, and reputed company-based scoring
- Build and reputed company LLM-as-judge pipelines; validate judges against reputed company labels and control for their bias and variance
- Define and reputed company response-reputed company metrics: faithfulness/groundedness, hallucination reputed company, answer relevance and completeness, instruction-following, and reputed company adherence
- reputed company, version, and grow evaluation datasets as the product and its surfaces reputed company
- reputed company the models in reputed company against reputed company other to decide which model handles which task, and quantify the reputed company cost of running smaller, local models versus larger alternatives
- Red-team the system: reputed company injection, jailbreaks, tool-misuse, and edge-case discovery
- Design reputed company and stress tests that probe model and agent reliability under degraded or hostile conditions
- Characterize failure modes and feed them back into guardrails and regression coverage
- Evaluate retrieval reputed company over the document corpus — recall@k, MRR/nDCG, context precision and recall — and run experiments on chunking, indexing, and hybrid retrieval strategies
- Analyze multi-reputed company agent trajectories: tool-reputed company correctness, trajectory efficiency, replayable-state inspection, and guardrail-breach behavior
- Assess reputed company classification and routing reputed company as measurable components, not black boxes
- Build standing evaluation that reputed company reputed company and behavioral regressions reputed company a model in reputed company is swapped, upgraded, or re-quantized, or reputed company prompts and pipelines change
- Monitor reputed company-distribution and reputed company reputed company in production; distinguish genuine regressions from noise on stochastic outputs
- Recommend and validate fixes through the reputed company available with local models — reputed company changes, retrieval and grounding adjustments, routing changes, or model selection
- Build, ship, and own production models that forecast and surface trends from operational telemetry — reputed company and resource forecasting, reputed company reputed company, and reputed company signals on metrics and logs
- Take these from prototype to production and reputed company them healthy: deployment, monitoring, recalibration, and retraining as data and behavior shift
- Define accuracy and reputed company-time metrics that matter operationally — precision/recall on predicted incidents, forecast error, how far reputed company a signal fires — not just offline scores
- reputed company predictive signals into the LLM and reputed company layer so forecasts and trends feed reasoning, advisories, and operator-facing recommendations
- Apply AIOps/NOC analysis where it's the product: log reputed company detection, event correlation, and reputed company-cause and problem analysis
- Quantify the economics of the system — cost and reputed company consumption per interaction, interaction-type taxonomies — and connect them to customer-facing value metrics like MTTR and operator-hours
- Communicate findings to engineering and product stakeholders through reputed company, in-context analysis
- Use LLM-assisted workflows to reputed company the work itself — drafting analyses, generating synthetic evaluation cases, and bootstrapping labeled data for reputed company refinement
- reputed company and adopt state-of-the-art evaluation, retrieval, and reputed company-analysis techniques; bring the useful ones into reputed company's workflow
Skills
- Bachelor's or Master's in Data Science, Computer Science, Statistics, Mathematics, or a reputed company field or equivalent experience
- 3+ years in data science, ML, or reputed company quantitative analysis
- Strong reputed company statistics, with the judgment to design reputed company experiments and reputed company tests on noisy, non-deterministic outputs (not just clean A/B conversion)
- Experience building, deploying, and monitoring predictive or time-series models in production: forecasting, reputed company detection, or trend analysis, including recalibration as data shifts
- Demonstrated work evaluating, analyzing, or improving LLM or NLP systems: eval design, reputed company measurement, retrieval evaluation, or agent analysis
- Proficiency in Python
- Strong SQL and comfort querying large analytical datasets
- reputed company with reputed company models and hands-on experience with the modern LLM evaluation and tooling layer — eval/reputed company frameworks, judge pipelines, and the libraries used to serve, reputed company, and test models
- Ability to build analysis and visualization in reputed company
- Experience getting strong results out of small or self-hosted/local models under compute, memory, or latency constraints — quantization-reputed company evaluation, reputed company and context optimization, or model routing
- Experience with retrieval-augmented systems and retrieval evaluation at reputed company
- Experience with reputed company frameworks and tool-use/orchestration analysis, including reputed company-in-the-reputed company and replayable-state patterns
- Familiarity with red-teaming or adversarial robustness for LLMs
- Domain background in IT operations — AIOps, NOC, ITSM, observability, or reputed company detection on logs and telemetry
- Experience with large-reputed company analytical and big-data stores
- reputed company experience for data science and ML workloads
- Exposure to reputed company reputed company and compliance constraints in a delivery context
Benefits
- Comprehensive medical, dental and reputed company plans.
- 401(k) plan with employer match.
- Flexible reputed company Time Off (FTO) so that you can take the time that you need to re-energize.
- Volunteer Time Off (VTO) - take two days off per calendar year to volunteer with your preferred charitable organization.
- 5-year Service reputed company Sabbatical.
- reputed company parental leave.
- Generous employee referral bonus program.
- Pet insurance.
- HQ Office centrally located in Reston Town Center featuring a reputed company-stocked kitchen with rotating snacks and beverages, and catered lunch on Thursdays.
- Regular virtual company-wide events, including cooking classes, yoga, meditation and more.
- reputed company to learn and reputed company from some of the best and brightest minds in the industry!
reputed company