Lead Data Engineer
Bengaluru · Senior
Applicants who checked fit first are 3.1× more likely to hear back
Your score for this role already exists
ASAI compared this JD against 41 signals - skills, seniority, domain, stack overlap etc. Add a resume and it unlocks in about 30 seconds.
Always free · no credit card · 1 tap with Google
What we know about this role
LOW
Our tracked job listings for nxp show limited recent activity.
First 72 hours
Window passed - posted 51d agoEarly applicants get seen before the pile builds.
Not a repost
The first time we've seen this listing - it hasn't been closed and reopened.
Skills required
8 listedYou almost certainly match several of these already. Unlock your skill map to see the matches, the gaps, and what to fix first.
Job description
Position Summary:
We are seeking an experienced Senior Data Engineer to drive the performance, governance, and AI-native maturity of our enterprise Data Platform in Databricks. This is a Databricks-focused Data Engineering role with a working understanding of DevOps practices — designing scalable pipelines, tuning workloads for performance and cost, and operationalizing modern data and AI capabilities on Lakehouse.
The ideal candidate has deep, hands-on Databricks expertise, a strong performance-engineering instinct, and a builder's mindset for AI-assisted operations. You'll own the Databricks performance and governance standards for the platform, mentor engineers, and shape the direction for AI-native operations.
Key Responsibilities:
- Design and develop scalable data pipelines and Lakehouse solutions on Databricks.
- Tune Databricks workloads for performance and cost, including cluster sizing, query optimization, and Delta Lake table design.
- Establish and enforce best practices for partitioning, clustering, and workload isolation.
- Track performance trends, identify high-cost queries, and partner with source teams and end users to resolve long-running loads.
- Design and operationalize Unity Catalog for data governance — access control, lineage, and security.
- Build monitoring and self-healing automation using Databricks-native AI and agentic capabilities.
- Drive CI/CD workflows for Databricks assets, setting DevOps best practices for deployment and release management.
- Lead design reviews and mentor Data Engineers on Databricks best practices and AI-native features.
- Own Databricks vendor coordination — case management, escalations, and release adoption strategy.
What Success Looks Like (First 6–12 Months)
- Within 6–12 months, you'll define the platform's tuning and governance standards, lead design reviews, mentor junior engineers, and shape the AI-native operations roadmap.
Required Qualifications:
- Bachelor's or Master's degree in Computer Science, Information Technology, or equivalent relevant experience.
- 6+ years of data engineering experience with 2+ years hands-on Databricks in enterprise settings.
- Deep understanding of Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and Workflow orchestration.
- Proven ability to tune Spark workloads for cost and performance at production scale.
- Advanced Python (PySpark) and SQL skills.
- Working knowledge of CI/CD practices and DevOps principles applied to data workloads.
- Experience with observability tooling for Databricks.
Preferred Qualifications:
- Experience with Databricks-native AI capabilities and agentic frameworks.
- Familiarity with Databricks Serverless Compute and DBSQL performance tuning.
- A Databricks Certified Professional.
- Exposure to Infrastructure-as-Code is a plus.
Competencies:
- Performance-engineering mindset — measures, tunes, and re-measures.
- Curiosity for AI-native operations and continuous automation.
- Strong sense of platform ownership — quality, cost, and reliability.
- Effective communication with engineering peers, vendors, and business stakeholders.
- Influence outcomes across source teams, vendors, and business stakeholders without direct authority.
Tools to land this role
PremiumYour match score, skill map and tracker stay free.
Ground Reality
PremiumSpot the dealbreakers in this 413-word post before you apply
Skip jobs with hidden dealbreakers. See what they actually need, what's easy to miss, and how they describe the team.
Ground Reality on every job, plus Insider Path, practice interviews and resume tailoring.
Insider Path
PremiumFind someone who can refer you here
We help you find people there who worked or studied where you did, and draft your first message and follow-up.
Free · no signup
Get tomorrow's jobs before you have to search
Daily job drops, skill trends and free resources - posted straight to the group. Leave any time.
No spam. Just jobs and resources.
Why people use ASAI
Someone shared one job with you. ASAI keeps finding the rest.
Scored, not searched. Every role ranked against your actual profile.
Alerts as often as hourly. Reach new roles while the pile is still small.
Skill gaps, spelled out. See exactly which requirements you don't meet yet.
Verified jobs, only. Say no to ghost jobs. Your time deserves respect.
More Data Engineer roles in Bengaluru
See all
Keep browsing