Senior GenAI Research Engineer - Optimization and Kernels

Tech Stack

Responsibilities

• Drive performance improvements through advanced optimization techniques including kernel fusion, mixed precision, memory layout optimization, tiling strategies, and tensorization for training-specific patterns.
• Design, implement, and optimize high-performance GPU kernels for training workloads (e.g., attention mechanisms, custom layers, gradient computation, activation functions) targeting NVIDIA architectures.
• Design and implement distributed training frameworks for large language models, including parallelism strategies (data, tensor, pipeline, ZeRO-based) and optimized communication patterns for gradient synchronization and collective operations.
• Profile, debug, and optimize end-to-end training workflows to identify and resolve performance bottlenecks, applying memory optimization techniques like activation checkpointing, gradient sharding, and mixed precision training.
• Advance the scientific frontier by creating new techniques that go beyond the state of the art in deep learning.

Benefits

• Equity

Culture

Requirements