EL3 – Databricks MLOps Engineer (Contract)

Domain: Claims Payment Integrity | M&R, C&S, E&I Claims (preferred)
Actuarial & Forecasting Analytics Exposure is an Added Advantage
Tech Stack: Databricks, Spark, Python, Scala, Azure, GitHub Actions, Terraform
AI/LLM Capabilities: Embedding Models, LLM Integration, LangChain Agentic Frameworks

Role Summary

The EL3 Databricks MLOps Engineer is a senior hands-on role responsible for enabling end-to-end machine learning lifecycle automation on Databricks. This includes building and maintaining the CI/CD infrastructure, environment configuration, packaging and deploying ML models, supporting reproducible experiments, and ensuring scalable job orchestration for AI/ML workloads, including LLM-based applications.

The role partners closely with Data Scientists, AI/ML Engineers, platform teams, and business stakeholders within Claims Payment Integrity to ensure robust, reliable, and automated ML delivery.

Key Responsibilities

• Enable and automate the end-to-end ML lifecycle on Databricks (environment setup, model workflow automation, job scheduling, monitoring hooks).

• Build frameworks, templates, and utilities that make ML development and experimentation reproducible and scalable.

• Implement CI/CD pipelines using Git, GitHub Actions, Jenkins, Azure DevOps, or similar tools.

• Package, version, and deploy ML models into Databricks-managed execution environments.

• Set up automated workflows for training, retraining, evaluation, and scheduled job execution.

• Support creation and integration of machine learning models including classification, forecasting, anomaly detection, NLP, and PI models.

• Enable LLM/GenAI-driven solutions by integrating: 

  • Embedding model generation

  • RAG architectures

  • Vector databases

  • LangChain agentic workflows

• Optimize resource usage, runtime configurations, and code execution patterns for ML workloads.

• Collaborate with Data Scientists to translate experimental notebooks into production-ready pipelines.

• Implement platform-level controls for environment consistency, dependency management, access control, and model versioning.

• Support troubleshooting, debugging, and performance improvements for ML workloads.

• Document standards, templates, guidelines, and best practices for MLOps teams.

• Work cross-functionally with product, engineering, and analytics teams across PI.

Required Qualifications

• Bachelor’s/Master’s degree in Computer Science, Engineering, or related field

• 6–9 years of relevant experience in ML Engineering, MLOps, or platform engineering

• Strong hands-on experience with Databricks, Spark (batch/streaming), Python, Scala

• Experience enabling ML lifecycle tools such as MLflow (tracking, packaging, model registration)

• Strong CI/CD experience using Git, GitHub Actions, Jenkins, or Azure DevOps

• Experience deploying AI/ML models into cloud environments (Azure preferred)

• Ability to create and integrate embedding models, semantic vectors, and LLM-driven components

• Experience with LangChain for agentic workflows and integration of tools/functions

• Strong problem-solving, debugging, and collaboration skills

Preferred Qualifications

• Experience with Azure OpenAI or OpenAI-compatible LLM APIs

• Familiarity with healthcare claims workflows, PI, FWA, provider billing, or pricing

• Experience in Agile/Scrum environments

• Strong understanding of software engineering best practices, packaging, dependency management

Good-to-Have Data Knowledge

• Call Center datasets (member & provider interactions)

• Provider RCM datasets (billing, coding, authorizations)

• EHR/clinical datasets for cross-domain validation