Machine Learning Systems Engineer

About the Role: You will own the serving path for Newton and related multimodal models. Much of our inference stack is Rust-native: model nodes in our agent runtime, built on Rust ML stacks (candle, Burn) with custom GPU kernels, plus the routing layer that streams real-time inference to GPU nodes. You will drive GPU utilization, numerical precision, and low-latency serving from the kernel up. What You'll OwnBuild and own model nodes in our Rust inference runtime: loading, warmup, batching, streaming, GPU memory pools. Optimize kernels and the GPU path: custom CUDA kernels, mixed precision, quantization, parity against research. Own inference routing and serving: streaming path API to GPU node, request batching, SLO-backed latency and cost. Productionize research checkpoints: export, compilation, quantization, parity evals, and rollout. Build the observability inference needs: latency histograms, GPU metrics, OOM signatures, replayable traces.

Key Qualifications: 6+ years software engineering, several of them in ML systems, inference, or high-performance GPU computing. Has owned a production serving path end to end, not only benchmarked models. Expert in PyTorch with models shipped to production; knows what will be slow before the profiler runs. Strong CUDA or equivalent GPU depth: memory hierarchy, occupancy, Nsight or equivalent profiling. Rust or C++ alongside Python, Linux performance, production ops; ready to work in Rust daily. Works with researchers: can translate an architecture change into serving work.

Nice to Have: Rust ML stacks: candle, Burn, or comparable GPU compute in Rust. Custom kernels and compiler stacks: Triton, CUTLASS, Torch Inductor, TensorRT. Quantization and mixed precision in production with a numerical-correctness suite. Multimodal, video, embedding or time-series serving, not only decoder-only chat LLMs. High-performance serving stacks (vLLM, SGLang, TensorRT-LLM): continuous batching, paged KV cache.