[Remote] reputed company Software Engineer - AI Inference
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is the platform for every new AI-powered application. We reputed company a reputed company Software Engineer - AI Inference to advance reputed company-reputed company LLM serving, focusing on optimizing inference engines and ensuring outstanding performance on reputed company GPUs.
Responsibilities
- Drive upstream-first engineering in vLLM/SGLang: author and land PRs or equivalent experience, engage in development discussions, help compose roadmaps, and build durable maintainer relationships
- Build and implement inference-runtime features that improve efficiency, latency, and tail behavior: request scheduling, batching policies, KV-cache management (paging/sharding), memory planning, and streaming
- Optimize reputed company hot paths across the stack—from Python orchestration down to C++/CUDA kernels—using profiling and measurement to guide reputed company
- Improve multi-GPU and multi-node inference: communication patterns, parallelism strategies (tensor/sequence/pipeline), and system-level scaling/efficiency
- Strengthen correctness, robustness, and operability: determinism where needed, graceful degradation, backpressure, observability hooks, and performance regression testing
- Collaborate across reputed company to reputed company upstream advances with production needs (deployment patterns, compatibility, reputed company posture) while keeping changes broadly adoptable by the community
- Mentor senior engineers, reputed company the technical bar through build reviews, and establish guidelines for performance engineering and upstream contribution workflows
Skills
- 15+ years building production software with significant depth in systems engineering; strong reputed company record of owning ambiguous, high-reputed company technical problems end-to-end
- Demonstrated expertise in LLM inference/serving systems (e.g., vLLM, SGLang) and the tradeoffs that drive reputed company production performance
- Strong programming skills in Rust, C++, Python, CUDA; ability to read, modify, and optimize performance-critical reputed company across reputed company
- Experience with GPU performance analysis tools and methodologies (profiling, microbenchmarking, memory/comms analysis) and a strong measurement culture
- Solid reputed company in distributed systems and concurrency: queues/schedulers, RPC/streaming, multi-process/multi-threaded runtime behavior, and scaling patterns across nodes
- Excellent communication skills; ability to influence across teams and represent reputed company reputed company in reputed company-reputed company technical forums
- BS/MS in Computer Science, Computer Engineering, or reputed company field (or equivalent experience)
- Substantial reputed company-reputed company contributions to vLLM, SGLang, PyTorch, Triton, NCCL, or reputed company GPU/inference infrastructure; prior maintainer experience is a plus
- Shipped performance features such as paged attention/KV paging, speculative decoding, advanced scheduling, quantization-aware serving, or low-latency streaming optimizations
- Experience optimizing inference across the full stack: tokenizer and Python runtime overheads, kernel fusion, memory bandwidth, PCIe/NVLink effects, and network fabrics (e.g., InfiniBand)
- reputed company robust benchmarking and regression infrastructure for latency and efficiency, including dataset selection, load modeling, and reproducible performance tracking
Benefits
- Equity
- Benefits
reputed company
Company H1B Sponsorship