Senior MLOps Engineer

Build and maintainKubernetes deploymentsfor LLM and TTS inference usingHelm,ArgoCD, andPrometheus/Grafanamonitoring.

Implement and optimizemodel serving pipelinesusingvLLM,SGLang,TensorRT, or similar frameworks for high-throughput inference.

Develop CI/CD and MLOps automation for data versioning, model validation, and deployment (GitHub Actions, Jenkins, or AWS CodePipeline).

IntegrateOpenWebUI,Gradio, or similar UIs for user-facing model demos and internal evaluation tools.

Collaborate with ML researchers to productize models — including TTS (e.g., ElevenLabs API), ASR (Whisper), and LLM-based chat systems.

Ensureobservability, cost optimization, and reliabilityof cloud resources across multiple environments.

Contribute to internal tools fordataset curation, model monitoring, and retraining pipelines.

Maintain infrastructure-as-code usingTerraformandHelm chartsfor reproducibility and governance.

Support real-time multimodal workloads (voice, text, vision) across inference clusters.

Academic Qualifications

• 4+ years of experience inMLOps,DevOps, orCloud Infrastructure Engineeringfor ML systems.
• Strong proficiency inKubernetes,Helm, andcontainer orchestration.
• Experience deploying ML models viavLLM,SGLang,TensorRT, orRay Serve.
• Proficiency withAWS services(EKS, EC2, S3, RDS, CloudWatch, IAM).
• Solid experience withPython,Docker,Git, andCI/CD pipelines.
• Strong understanding ofmodel lifecycle management,data pipelines, andobservability tools(Grafana, Prometheus, Loki).
• Excellent collaboration skills with ML researchers and software engineers.

Professional Experience – Preferred

• Extensive Experience withvLLM, K8s, Elevenlabs,Whisper,Gradio/OpenWebUI, or custom TTS/ASR model hosting.
• Familiarity withmulti-GPU scheduling,NCCL optimization, andHPC cluster integration.
• Knowledge ofsecurity,cost management, andnetwork policyin multi-tenant Kubernetes clusters andcloudflaresystems.
• Prior work inLLM deployment,fine-tuning pipelines, orfoundation model research.
• Exposure todata governanceandresponsible AI operationsin research or enterprise settings.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

About the Institute of Foundation Models (IFM)

The Institute of Foundation Models is a dedicated research lab for building, understanding, deploying, and risk-managing large-scale AI systems. We drive innovation in foundation models and their operationalization, empowering research, education, and industry adoption through scalable infrastructure and real-world applications.
As part of our engineering team, you will operate at the intersection of machine learning and systems design — building the cloud, orchestration, and deployment layers that power the next generation of intelligent applications at MBZUAI. You’ll work alongside world-class AI researchers and engineers to productionize LLMs, voice models, and multimodal systems at scale.

The Role

As a Senior MLOps Engineer, you will design, build, and maintain robust ML(Machine Learning) infrastructure across training, inference, and deployment pipelines. You will take ownership of the model lifecycle — from data ingestion to real-time serving — and ensure our LLM and speech models are deployed efficiently, securely, and reproducibly in Kubernetes-based environments.
This position requires deep hands-on experience with Kubernetes (EKS), Helm, AWS cloud infrastructure, and modern MLOps toolchains (e.g., vLLM, SGLang, OpenWebUI, Weights & Biases, MLflow). Familiarity with speech/voice AI frameworks like ElevenLabs, Whisper, and RVC is also valuable.

Key Responsibilities

• Design and manage scalable ML infrastructure on AWS using EKS, EC2, RDS, S3, and IAM-based access control.
• Build and maintain Kubernetes deployments for LLM and TTS inference using Helm, ArgoCD, and Prometheus/Grafana monitoring.
• Implement and optimize model serving pipelines using vLLM, SGLang, TensorRT, or similar frameworks for high-throughput inference.
• Develop CI/CD and MLOps automation for data versioning, model validation, and deployment (GitHub Actions, Jenkins, or AWS CodePipeline).
• Integrate OpenWebUI, Gradio, or similar UIs for user-facing model demos and internal evaluation tools.
• Collaborate with ML researchers to productize models — including TTS (e.g., ElevenLabs API), ASR (Whisper), and LLM-based chat systems.
• Ensure observability, cost optimization, and reliability of cloud resources across multiple environments.
• Contribute to internal tools for dataset curation, model monitoring, and retraining pipelines.
• Maintain infrastructure-as-code using Terraform and Helm charts for reproducibility and governance.
• Support real-time multimodal workloads (voice, text, vision) across inference clusters.

Academic Qualifications

• 4+ years of experience in MLOps, DevOps, or Cloud Infrastructure Engineering for ML systems.
• Strong proficiency in Kubernetes, Helm, and container orchestration.
• Experience deploying ML models via vLLM, SGLang, TensorRT, or Ray Serve.
• Proficiency with AWS services (EKS, EC2, S3, RDS, CloudWatch, IAM).
• Solid experience with Python, Docker, Git, and CI/CD pipelines.
• Strong understanding of model lifecycle management, data pipelines, and observability tools (Grafana, Prometheus, Loki).
• Excellent collaboration skills with ML researchers and software engineers.

Professional Experience – Preferred

• Extensive Experience with vLLM, K8s, Elevenlabs, Whisper, Gradio/OpenWebUI, or custom TTS/ASR model hosting.
• Familiarity with multi-GPU scheduling, NCCL optimization, and HPC cluster integration.
• Knowledge of security, cost management, and network policy in multi-tenant Kubernetes clusters and cloudflare systems.
• Prior work in LLM deployment, fine-tuning pipelines, or foundation model research.
• Exposure to data governance and responsible AI operations in research or enterprise settings.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.