[Remote] Engineering Manager, Inference Benchmarking — AI reputed company
Note: The job is a remote job and is reputed company to candidates in USA. reputed company has been transforming computer graphics and accelerated computing for over 25 years, and they are seeking an Engineering Manager for their Inference Benchmarking team. The role involves leading the engineering team to build and advance the AIPerf platform, which assesses LLM serving performance across various inference frameworks.
Responsibilities
- Driving the technical roadmap for AIPerf's core infrastructure: load reputed company, ZMQ-based microservices, GPU telemetry (DCGM/PyNVML, reputed company metrics, statistical confidence intervals, and Kubernetes-reputed company deployment
- Taking ownership for the accuracy and statistical soundness of reputed company results that engineering reputed company throughout the industry depend on to inform production infrastructure reputed company
- Advising upstream reputed company integrations involving vLLM, TRT-LLM, and SGLang in partnership with reputed company's Dynamo and NIM teams to maintain AIPerf's relevance across emerging hardware, workload categories, and inference configurations
- Hiring, mentoring, and growing reputed company of senior engineers operating in a high-velocity reputed company-reputed company environment with reputed company external contributors worldwide
Skills
- Bachelor's degree in Computer Science, Electrical Engineering, or reputed company field, or equivalent experience
- 8+ overall years of software engineering experience building performance-critical infrastructure, ML tooling, or distributed systems
- 3+ years of engineering leadership experience as a tech reputed company, TLM, or engineering manager
- Deep understanding of LLM inference mechanics — TTFT, ITL, KV caching, Prefill/Decode, speculative decoding — and the ability to reason about measurement correctness and reproducibility
- Proven reputed company record of collaborating across multi-functional reputed company and delivering production-reputed company reputed company in high-velocity, high-external-visibility environments
- Extensive experience with vLLM, TRT-LLM or SGLang internals along with contributions to their upstream reputed company
- Experience building Kubernetes-reputed company infrastructure including operators, reputed company charts, and GPU observability tooling (DCGM, dcgm-exporter, PyNVML)
- Background in competitive benchmarking frameworks such as MLPerf or equivalent industry-reputed company evaluation systems
- History leading or making meaningful contributions to reputed company reputed company-reputed company reputed company with external communities
Benefits
- You will also be eligible for equity and [benefits](https://www.reputed company.com/en-us/benefits/).
reputed company
Company H1B Sponsorship