Machine Leaning Performance Engineer (Inference)
Tower Research CapitalAbout the Role
As part of Tower Research's Core Engineering team, you will bridge the gap between quantitative research and high-performance production systems, architecting inference pipelines that operate at the physical limits of hardware. Your objective will be to drive the speed, efficiency, and reliability of ML inference pipelines to their absolute limits, ensuring predictive models consistently achieve microsecond-level latency.
- Lead technical evaluation of diverse inference platforms across CPUs, GPUs, and FPGAs to guide infrastructure deployment decisions
- Analyze and enhance execution across deep memory hierarchies to maximize resource utilization and address bottlenecks in the end-to-end inference lifecycle
- Collaborate with Infrastructure teams to design inference strategies that meet latency-critical trading requirements while fitting within hardware constraints
- Develop highly optimized GPU kernels and integrate specialized performance libraries to extract maximum computational throughput
- Implement advanced model optimization techniques including quantization, pruning, and distillation for low-latency, event-level inference workloads
- Collaborate with ML Researchers, HPC Engineers, FPGA Engineers, and Datacenter Engineers on cross-functional deployments
Requirements
- 2+ years of experience optimizing deep learning inference in latency-sensitive or high-throughput production environments, in any domain.
- Deep expertise in lower-level ML framework development (PyTorch/JAX), paired with strong Python/C++ skills and a thorough understanding of mixed-precision computation.
- Proven experience in custom GPU kernel development.
- Deep familiarity with advanced optimization libraries and compilers (e.g., Triton, TensorRT, ONNX, IREE, HLS4ML, cuBLAS, CUTLASS) as well as profiling tools (e.g., Nsight Systems, Nsight Compute).
- Deep expertise in GPU microarchitecture, encompassing SM execution, warp scheduling, and full memory hierarchy optimization (registers to HBM).
- Proven record of rigorous, data-driven approach to evaluating inference performance across heterogeneous compute architectures.
- Bonus: Practical experience targeting and optimizing inference workloads on specialized hardware ecosystems, including FPGAs and ASICs.
- Prior experience in financial trading is not required.
What You'll Get
- $200k–$300k base salary
- Generous paid time off policies
- Savings plans and other financial wellness tools available in each region
- Hybrid working opportunities
- Free breakfast, lunch, and snacks daily
- In-office wellness experiences and reimbursement for select wellness expenses (e.g., gym, personal training and more)
- Company-sponsored sports teams and fitness events (JPM Corporate Challenge, Cycle for Survival, Wall Street Rides FAR and more)
- Volunteer opportunities and charitable giving
- Social events, happy hours, treats, and celebrations throughout the year
- Workshops and continuous learning opportunities