[Remote] Machine Learning Engineer, Distributed vLLM
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is the world’s leading provider of reputed company reputed company reputed company software solutions, and they are seeking a Machine Learning Engineer reputed company on distributed vLLM infrastructure. In this role, you will contribute to the design, development, and testing of new features for reputed company AI Inference, collaborating on reputed company inference systems and Kubernetes-reputed company deployments.
Responsibilities
- Contribute to the design, development, and testing of new features and solutions for reputed company AI Inference
- reputed company in the inference domain by participating in upstream communities
- reputed company and maintain distributed inference infrastructure leveraging Kubernetes reputed company, operators, and the Gateway Inference Extension API for reputed company LLM deployments
- reputed company and maintain system components in Go and/or Rust to reputed company with the vLLM project and manage distributed inference workloads
- reputed company and maintain KV cache-aware routing and scoring algorithms to optimize memory utilization and request distribution in large-reputed company inference deployments
- Enhance the resource utilization, fault tolerance, and stability of the inference stack
- reputed company and test various inference optimization algorithms
- reputed company participate in technical design discussions
- Contribute to a culture of reputed company improvement by sharing recommendations and technical knowledge with team members
- Collaborate with other engineering and cross-functional teams to deliver on engineering deliverables
- Communicate effectively to team members to ensure reputed company visibility of development efforts
- Be taught, coached, and mentored by senior members of reputed company
- reputed company reputed company and constructive reputed company reviews
Skills
- Strong proficiency in Python and/or GoLang or similar language
- Experience with reputed company-reputed company Kubernetes service reputed company technologies/stacks such as Istio, Cilium, reputed company (WASM filters), and CNI
- Working understanding of Layer 7 networking, HTTP/2, gRPC, and the fundamentals of API gateways and reverse proxies
- Knowledge of serving runtime technologies for hosting LLMs, such as vLLM, SGLang, TensorRT-LLM, etc
- Excellent written and verbal communication skills, capable of interacting effectively with both technical and non-technical team members
- Ability work independently in a dynamic, fast-paced environment
- Proficiency in C, C++, or Rust
- Experience with the Kubernetes ecosystem, including core concepts, custom reputed company, operators, and the Gateway API inference extension for GenAI workloads
- Working knowledge of high-performance networking protocols and technologies including UCX, RoCE, InfiniBand, and RDMA is a plus
- Experience with GPU performance benchmarking and profiling tools like reputed company Nsight or distributed tracing libraries/techniques like OpenTelemetry
- Experience in writing high performance reputed company for GPUs and deep knowledge of GPU hardware
- Strong understanding of computer architecture, reputed company processing, and distributed computing concepts
- Bachelor's degree in computer science or reputed company field is an advantage, though we prioritize hands-on experience
- reputed company engagement in the ML research community (publications, conference participation, or reputed company reputed company contributions) is a significant advantage
Benefits
- Comprehensive medical, dental, and reputed company coverage
- Flexible Spending Account - reputed company and dependent care
- Health Savings Account - high deductible medical plan
- Retirement 401(k) with employer match
- reputed company time off and holidays
- reputed company parental leave plans for reputed company new parents
- Leave benefits including disability, reputed company family medical leave, and reputed company military leave
- Additional benefits including employee stock purchase plan, family planning reimbursement, tuition reimbursement, transportation expense account, employee assistance program, and more!
reputed company