Design and implement performance benchmarks and testing
methodologies to evaluate application performance
Build tools to automate workload analysis, workload
optimization, and other critical workflows
Triage system issues and identify bottleneck and inefficiencies
by analyzing the sources of issues and the impact on hardware,
network and propose solutions to enhance GPU utilization
Support the team to develop appropriate kernels and systems for
new model architectures and algorithms
Participate in, or lead design reviews with peers and
stakeholders to decide amongst available technologies.
Review code developed by other developers and provide feedback
to ensure best practices (e.g., style guidelines, checking code in,
accuracy, testability, and efficiency).
Contribute to existing documentation or educational content and
adapt content based on product/program updates and user
feedback.
Represent MBZUAI at industry conferences and events, showcasing
the institution’s cutting-edge HPC and deep learning capabilities
and establishing MBZUAI as a global leader in AI research and
innovation.
Perform all other duties as reasonably directed by the line
manager that are commensurate with these functional
objectives.
Validate CUDA kernel outputs and gradients against reference
implementations, and benchmark representative shapes, dtypes, and
model workloads.
Technical Qualifications
Must-Haves:
- Strong C++ skills and hands-on CUDA kernel development and
optimization for deep-learning workloads.
- Understanding of GPU memory hierarchy, warp/block execution,
and compute-memory trade-offs, with demonstrated profiling-driven
optimization.
- Strong Python skills and experience integrating kernels with
PyTorch or an equivalent framework, including numerical and
gradient validation where needed.
Nice-to-Haves:
- Experience with Triton, CUTLASS, or PTX/SASS analysis.
- Experience with multi-node distributed training or inference
systems.
- Experience validating mixed-precision computations, such as
BF16 or FP8.
$150,000 - $450,000 a year
Salary Range
The posted salary range represents the company’s good faith
estimate of the compensation for this position upon hire. The
actual compensation offered may vary within this range depending on
individual qualifications, including but not limited to relevant
skills, experience, education, certifications, geographic location,
and specific business needs.
The posted salary range represents the company’s good faith
estimate of the compensation for this position upon hire. The
actual compensation offered may vary within this range depending on
individual qualifications, including but not limited to relevant
skills, experience, education, certifications, geographic location,
and specific business needs.
Benefits Include
*Comprehensive medical, dental, and vision benefits
*Bonus
*401K Plan
*Generous paid time off, sick leave and holidays
*Paid Parental Leave
*Employee Assistance Program
*Life insurance and disability
We may use artificial intelligence (AI) tools to support parts of
the hiring process, such as reviewing applications, analyzing
resumes, or assessing responses and identifying potential
inconsistencies or verification signals in application materials
based on available information. These tools assist our recruitment
team but do not replace human judgment. Final hiring decisions are
ultimately made by humans. If you would like more information about
how your data is processed, please contact us.
About the Institute of Foundation Models
We are a dedicated research lab for building, understanding, using,
and risk-managing foundation models. Our mandate is to advance
research, nurture the next generation of AI builders, and drive
transformative contributions to a knowledge-driven economy.
As part of our team, you’ll have the opportunity to work on the
core of cutting-edge foundation model training, alongside
world-class researchers, data scientists, and engineers, tackling
the most fundamental and impactful challenges in AI
development. You will participate in the development of
groundbreaking AI solutions that have the potential to reshape
entire industries. Strategic and innovative problem-solving skills
will be instrumental in establishing MBZUAI as a global hub for
high-performance computing in deep learning, driving impactful
discoveries that inspire the next generation of AI pioneers.
The Role
The GPU Kernel Engineer will play a role at the forefront of
optimizing performance for the machine learning software stacks,
especially at training and inference, and support the team to
develop new and cutting-edge systems. The ideal candidate will have
a strong background in parallel computing, and hands-on experience
in system level coding, debug methodologies, and large-scale
machine learning experience.
This role focuses on CUDA kernel development and optimization.
Distributed training experience is a plus.
Key Responsibilities
- Understand, analyze, profile, optimize, and provide guidance to
the team on deep learning workloads on state-of-the-art hardware
and software platforms to improve their efficiency with different
levels of optimization
- Design and implement performance benchmarks and testing
methodologies to evaluate application performance
- Build tools to automate workload analysis, workload
optimization, and other critical workflows
- Triage system issues and identify bottleneck and inefficiencies
by analyzing the sources of issues and the impact on hardware,
network and propose solutions to enhance GPU utilization
- Support the team to develop appropriate kernels and systems for
new model architectures and algorithms
- Participate in, or lead design reviews with peers and
stakeholders to decide amongst available technologies.
- Review code developed by other developers and provide feedback
to ensure best practices (e.g., style guidelines, checking code in,
accuracy, testability, and efficiency).
- Contribute to existing documentation or educational content and
adapt content based on product/program updates and user
feedback.
- Represent MBZUAI at industry conferences and events, showcasing
the institution’s cutting-edge HPC and deep learning capabilities
and establishing MBZUAI as a global leader in AI research and
innovation.
- Perform all other duties as reasonably directed by the line
manager that are commensurate with these functional
objectives.
- Validate CUDA kernel outputs and gradients against reference
implementations, and benchmark representative shapes, dtypes, and
model workloads.
Technical Qualifications
Must-Haves:
- Strong C++ skills and hands-on CUDA kernel development and
optimization for deep-learning workloads.
- Understanding of GPU memory hierarchy, warp/block execution,
and compute-memory trade-offs, with demonstrated profiling-driven
optimization.
- Strong Python skills and experience integrating kernels with
PyTorch or an equivalent framework, including numerical and
gradient validation where needed.
Nice-to-Haves:
- Experience with Triton, CUTLASS, or PTX/SASS analysis.
- Experience with multi-node distributed training or inference
systems.
- Experience validating mixed-precision computations, such as
BF16 or FP8.
$150,000 - $450,000 a year
Salary Range
The posted salary range represents the company’s good faith
estimate of the compensation for this position upon hire. The
actual compensation offered may vary within this range depending on
individual qualifications, including but not limited to relevant
skills, experience, education, certifications, geographic location,
and specific business needs.
The posted salary range represents the company’s good faith
estimate of the compensation for this position upon hire. The
actual compensation offered may vary within this range depending on
individual qualifications, including but not limited to relevant
skills, experience, education, certifications, geographic location,
and specific business needs.
Benefits Include
*Comprehensive medical, dental, and vision benefits
*Bonus
*401K Plan
*Generous paid time off, sick leave and holidays
*Paid Parental Leave
*Employee Assistance Program
*Life insurance and disability
We may use artificial intelligence (AI) tools to support parts of
the hiring process, such as reviewing applications, analyzing
resumes, or assessing responses and identifying potential
inconsistencies or verification signals in application materials
based on available information. These tools assist our recruitment
team but do not replace human judgment. Final hiring decisions are
ultimately made by humans. If you would like more information about
how your data is processed, please contact us.
