Senior Technical Lead

LLMOps Engineer 

Role Summary 

The LLMOps Engineer is responsible for operationalizing, scaling, monitoring, and governing Large Language Model (LLM)–based systems in production. This role focuses on ensuring LLM applications are reliable, performant, cost-efficient, observable, and secure across their lifecycle—from deployment to continuous optimization. 

Key Responsibilities 

• Deploy and manage LLM-based applications across cloud and hybrid environments. 
• Build and maintain production-grade inference pipelines for LLM workloads. 
• Design and implement LLMOps pipelines for model, prompt, and configuration lifecycle management. 
• Implement monitoring for latency, throughput, cost, errors, drift, and hallucinations. 
• Build dashboards and alerts to ensure SLA and reliability targets. 
• Optimize inference cost and performance using caching, batching, routing, and autoscaling strategies. 
• Enforce security, access control, audit logging, and governance controls for LLM usage. 
• Collaborate with Responsible AI teams to operationalize safety and compliance guardrails. 
• Enable AI Engineers and Fullstack teams through reusable LLMOps frameworks and platforms. 

Required Skills & Experience 

• Strong understanding of LLMs, Generative AI architectures, and inference workflows. 
• Hands-on experience running LLM-based systems in production environments. 
• Strong experience with cloud platforms (Azure, AWS, or GCP). 
• Hands-on experience with Docker, Kubernetes, and container orchestration. 
• Experience with CI/CD for ML or AI applications. 
• Programming and automation skills in Python or similar languages. 
• Familiarity with MLOps / LLMOps tooling, monitoring, and logging systems. 

Experience Level 

• 6–10 years of overall experience in DevOps, Platform, or ML/AI Engineering roles. 
• 2–4+ years of experience operating ML or GenAI systems in production. 

Success Measures 

• Stable, scalable, and cost-efficient LLM systems running in production. 
• Reduced production incidents and faster recovery for AI services. 
• Faster and safer enterprise adoption of GenAI solutions enabled through robust LLMOps practices.