Job description

Job Summary

We are seeking a highly skilled Azure Data Engineer to design, develop, and optimize enterprise-scale data platforms on Microsoft Azure. The ideal candidate will have strong expertise in building modern data lakes and lakehouse architectures, implementing batch and streaming data pipelines, data modeling, performance tuning, and cost optimization. The role involves working with large-scale datasets using Azure-native services and Databricks technologies to deliver reliable, scalable, and high-performance data solutions.

Key Responsibilities

Design, develop, and maintain scalable data ingestion and transformation pipelines using Azure Data Factory (ADF) and Azure Databricks.
Build and manage enterprise data platforms using ADLS Gen2, Medallion Architecture (Bronze, Silver, Gold layers), and Lakehouse principles.
Develop robust ETL/ELT solutions using PySpark, Spark SQL, and Python.
Implement and optimize batch and streaming data processing solutions.
Design and maintain Delta Live Tables (DLT) for dependable and automated data transformation workflows.
Build streaming pipelines using Auto Loader, Structured Streaming, and Delta Lake technologies.
Establish and govern data assets through Unity Catalog for centralized security, governance, and metadata management.
Create and maintain dimensional and enterprise data models to support analytics and reporting requirements.
Develop and optimize Power BI datasets and semantic models for business intelligence and reporting.
Implement CI/CD pipelines for data engineering solutions using Azure DevOps or similar tools.
Monitor and improve data platform performance through workload optimization, partitioning strategies, caching, and query tuning.
Drive cloud cost optimization initiatives across Azure Data Lake, Databricks, storage, and compute resources.
Ensure adherence to data governance, security, compliance, and best practices.
Collaborate with architects, business stakeholders, analysts, and application teams to understand requirements and deliver data solutions.

Skill Requirements

Azure Data Services

Azure Data Factory (ADF)
Azure Data Lake Storage Gen2 (ADLS)
Azure Databricks
Delta Lake
Unity Catalog

Data Engineering & Processing

Medallion Architecture
Batch Processing
Streaming Data Processing
Delta Live Tables (DLT)
Auto Loader
ETL/ELT Frameworks

Programming & Analytics

PySpark
Spark SQL
Python
Data Modelling
SQL

Reporting & Visualization

Power BI
Data Visualization
Semantic Models

DevOps & Automation

CI/CD Pipelines
Azure DevOps
Git
Infrastructure Automation

Performance & Optimization

Query Optimization
Spark Performance Tuning
Partitioning Strategies
Workload Management
Cost Optimization

Other Requirements

Experience with large-scale Azure Data Platform implementations.
Strong understanding of Lakehouse Architecture and Delta Lake.
Experience working in Agile/Scrum environments.
Knowledge of data governance and security frameworks.

Microsoft Azure Data Engineering certifications are highly desirable.

Someone shared one job with you. ASAI keeps finding the rest.

Scored, not searched. Every role ranked against your actual profile.

Alerts as often as hourly. Reach new roles while the pile is still small.

Skill gaps, spelled out. See exactly which requirements you don't meet yet.

Verified jobs, only. Say no to ghost jobs. Your time deserves respect.