Skip to content

Machine Learning Performance Engineer

Jane Street
🇺🇸New YorkMediorFull-timePosted May 20, 2024

About the Role

Join Jane Street's ML team as a systems engineer focused on optimizing the performance of machine learning models for both training and inference. You'll work on a variety of optimization challenges including efficient large-scale training, low-latency inference in real-time systems, and high-throughput inference in research. Your work will involve a whole-systems approach across storage systems, networking, and host- and GPU-level considerations, from improving CUDA performance to analyzing cache behavior at the lowest level.

Requirements

  • Experience in low-level systems programming and optimization
  • An understanding of modern ML techniques and toolsets
  • The experience and systems knowledge required to debug a training run's performance end to end
  • Low-level GPU knowledge of PTX, SASS, warps, cooperative groups, Tensor Cores, and the memory hierarchy
  • Debugging and optimization experience using tools like CUDA GDB, NSight Systems, NSight Compute
  • Library knowledge of Triton, CUTLASS, CUB, Thrust, cuDNN, and cuBLAS
  • Intuition about the latency and throughput characteristics of CUDA graph launch, tensor core arithmetic, warp-level synchronization, and asynchronous memory loads
  • Background in Infiniband, RoCE, GPUDirect, PXN, rail optimization, and NVLink, and how to use these networking technologies to link up GPU clusters
  • An understanding of the collective algorithms supporting distributed GPU training in NCCL or MPI
  • An inventive approach and the willingness to ask hard questions about whether we're taking the right approaches and using the right tools