Assistant Manager - Data Engineering

Designation : Assistant Manager - Data Engineering

Level : L3

Location : Chennai

Experience : 5 to 8 Years

Job Role :

We are seeking a highly skilled Lead Fabric Data Engineer to drive the technical execution of our enterprise Fabric Data Strategy project. In this hands-on engineering lead role, you will be responsible for translating high-level data strategy and architectural blueprints into production-ready pipelines, Delta Lake structures, and high-performance analytical storage systems across Microsoft Fabric. You will lead a pod of data engineers, standardizing design patterns across ingestion, Medallion Architecture, CI/CD, and semantic serving layers.

Responsibilities :

1. Engineering Leadership & Technical Execution

Lead the hands-on design, development, and deployment of end-to-end data pipelines using Microsoft Fabric (Data Factory pipelines, Dataflows Gen2, Synapse Data Engineering Spark Notebooks).

Standardize Medallion Architecture (Bronze/Silver/Gold) patterns using Delta Lake formats, optimizing for storage efficiency, Schema Evolution, and V-Order indexing in OneLake.

Establish domain-driven data patterns (Data Mesh concepts) and leverage OneLake shortcuts to eliminate data duplication across enterprise platforms.

Manage capacity utilization (F-SKUs), monitoring compute usage, job performance, and query execution times to ensure cost governance.

2. Migration & System Integration

Design and execute data migration scripts to shift legacy workloads (Azure Synapse, SQL Server, Databricks, Snowflake) into Microsoft Fabric Lakehouse and Warehouse destinations.

Build streaming and event-driven data ingestion pathways using Real-Time Intelligence (Eventstreams & KQL Database/Eventhouse).

Build scalable semantic models and optimize Direct Lake, Import, and DirectQuery modes for Power BI integration without compromising performance.

3. Data Governance, Security & DevOps

Configure workspace access controls, object-level security, Row-Level Security (RLS), and Column-Level Security (CLS) across Fabric Lakehouses and Data Warehouses.

Integrate Microsoft Purview for automated data lineage, data cataloging, and sensitivity labeling.

Establish robust software engineering best practices for data teams: enterprise Git integration, CI/CD deployment pipelines, automated testing, and code review processes.

4. Mentorship & Team Guidance

Provide day-to-day technical leadership, code reviews, and architectural guidance to data engineers and developers.

Maintain clear engineering documentation, pipeline runbooks, and architectural patterns for broader team consumption.

Collaborate with Data Architects, Strategy Leads, and Business Analysts to unblock engineering dependencies and refine sprint backlogs.

Required skills : 

Experience: 7+ years in Data Engineering, Data Platform Development, or ETL Architecture, with at least 1.5+ years of direct hands-on build experience within the Microsoft Fabric ecosystem.  Core Tech Stack: Expert proficiency in Microsoft Fabric (OneLake, Lakehouse, Warehouse, Data Factory, Real-Time Intelligence), PySpark/Apache Spark, T-SQL, Delta Lake, and DAX.  Languages & Scripting: Strong programming skills in Python/PySpark, SQL, M, and PowerShell or Azure CLI.  Data Modeling: Expert understanding of Kimball dimensional modeling, star schemas, Data Lakehouse patterns, and partition strategies.  DevOps & Governance: Hands-on experience with Azure DevOps / GitHub Actions, Fabric Deployment Pipelines, and Microsoft Purview integration.

Microsoft Certified: Fabric Data Engineer Associate (DP-700) (Highly Preferred)  Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)  Microsoft Certified: Azure Data Engineer Associate (DP-203)