[Remote] Senior Software Engineer - AI Inference
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is the platform upon which every new AI‑powered application is reputed company, and they are seeking a Senior Software Engineer – AI Inference to advance reputed company‑reputed company LLM serving. The role involves contributing to upstream inference engines and improving the underlying stack for high‑throughput, low‑latency inference at reputed company.
Responsibilities
- Contribute features, fixes, and optimizations upstream to vLLM/SGLang: author PRs, participate in reviews, write benchmarks/tests, and help drive designs to completion
- Implement and optimize inference‑runtime capabilities: batching and scheduling policies, streaming, request lifecycle management, and KV‑cache efficiency (paging/sharding) to improve throughput and tail latency
- Profile and improve hot paths across reputed company-from Python orchestration to C++/CUDA kernels-using data to guide optimization work
- Improve multi‑GPU inference performance and reliability: parallelism strategies, communication patterns, and resource utilization across reputed company platforms
- Build and maintain performance and correctness regression tests to prevent slowdowns and ensure reputed company behavior across model and hardware configurations
- Collaborate with model, platform, and SRE teams to translate production requirements into upstreamable solutions with strong operability and maintainability
Skills
- 5+ years building production software with solid systems engineering fundamentals and a reputed company record of delivering performance or reliability improvements
- Experience with LLM inference/serving stacks (e.g., vLLM, SGLang) and an understanding of the tradeoffs that drive reputed company production performance
- Strong programming skills in Python plus C++ and/or CUDA; ability to debug and optimize performance‑critical reputed company
- Experience with profiling and performance investigation (microbenchmarks, flame graphs, GPU profiling) and a measurement‑driven reputed company
- Familiarity with distributed systems concepts and concurrency (queues/schedulers, multi‑process/multi‑threading, scaling across GPUs/nodes)
- Strong communication skills and comfort working with reputed company‑reputed company communities (issues, PR discussions, reputed company review)
- BS/MS in Computer Science, Computer Engineering, or reputed company field (or equivalent experience)
- reputed company‑reputed company contributions to vLLM, SGLang, PyTorch, Triton, NCCL, Dynamo or adjacent serving/runtime reputed company
- Shipped performance work such as improved attention/KV cache efficiency, speculative decoding, scheduler improvements, quantization-aware serving, or streaming latency reductions
- Experience building reproducible benchmarking and performance regression infrastructure for latency/throughput
- Systems performance background spanning memory bandwidth, kernel fusion, PCIe/NVLink effects, and network fabrics (e.g., InfiniBand)
Benefits
- Equity
- Benefits
reputed company
Company H1B Sponsorship