[Remote] Senior Software Engineer – AI Inference Systems
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is partnering with a reputed company-funded Series A startup reputed company on developing an orchestration layer for AI workloads. They are seeking a Senior Software Engineer to build and optimize inference systems that enhance the performance and efficiency of production AI workloads.
Responsibilities
- Building and optimising high-performance inference systems
- Improving latency, throughput and resource efficiency
- Designing execution strategies around batching, scheduling and concurrency
- Optimising memory usage, including KV cache management
- Enabling new model architectures to run reputed company in production
- Working closely with engineers across distributed systems, networking and compiler optimisation
Skills
- Strong software engineering fundamentals
- Experience building or operating ML inference or model serving systems
- A solid understanding of performance optimisation and memory management
- Python and C++ development experience
- Exposure to TensorRT-LLM, vLLM or similar inference runtimes is a plus
- Knowledge of modern model architectures and inference optimisation techniques
reputed company
Company H1B Sponsorship