Senior Consultant | Azure Databricks | Bengaluru | Engineering | Data Modernization & Migration

Databricks Engineer

Your potential, unleashed.

India’s impact on the global economy has increased at an exponential rate and Deloitte presents an opportunity to unleash and realize your potential amongst cutting edge leaders, and organizations shaping the future of the region, and indeed, the world beyond.

At Deloitte, your whole self to work, every day. Combine that with our drive to propel with purpose and you have the perfect playground to collaborate, innovate, grow, and make an impact that matters.

The Team

Deloitte’s Technology & Transformation practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.

We are seeking highly skilled Databricks Data Engineers to join our data modernization team. You will play a pivotal role in designing, developing, and maintaining robust data solutions on the Databricks platform. Your experience in data engineering, along with a deep understanding of Databricks, will be instrumental in building solutions to drive data-driven decision-making across a variety of customers.

Experience Band: 7 – 12 years

Band:  Sr. Consultant/ Manager

Location: Bangalore.

Relevant Experience: 5+ Years

Mandatory skills:

§ Strong hands-on experience with Databricks, PySpark, and SQL.

§ Experience with Azure, AWS, or GCP cloud platforms.

§ Expertise in Delta Lake, Delta Live Tables (DLT), Unity Catalog/Data Governance, Databricks Workflows, and Serverless Compute.

§ Knowledge of Genie, Mosaic AI, and Databricks Data Intelligence Platform capabilities.

§ Strong understanding of Data Architecture, Data Modelling, Lakehouse Architecture, Data Warehousing, and Medallion Framework.

§ Experience with GitHub, CI/CD pipelines, Agile delivery models, and DevOps practices.

§ Strong analytical, troubleshooting and performance optimization skills.

§ Excellent stakeholder management and business requirement gathering capabilities.

§ Ownership to bring initiatives on performance & cost optimization as required.

Responsibilities

§ Develop and maintain scalable data engineering solutions using Databricks, PySpark, and SQL for enterprise data platforms.

§ Design, implement, and optimize data architectures that are secure, scalable, reliable, and performance-driven.

§ Build and manage data pipelines for processing structured, semi-structured, and unstructured data from multiple source systems.

§ Develop and support both batch and near real-time data processing solutions using Databricks and Spark technologies.

§ Implement and manage Databricks capabilities including Delta Lake, Delta Live Tables (DLT), Data Workflows, Serverless Compute, Unity Catalog/Data Governance, Genie, Mosaic AI, and the Databricks Data Intelligence Platform.

§ Design and develop data solutions following Lakehouse, Medallion (Bronze-Silver-Gold), Data Warehouse, and Data Mart architectural standards.

§ Optimize Spark jobs, SQL queries, clusters, and data pipelines to improve performance, scalability, reliability, and cost efficiency.

§ Manage and monitor Databricks clusters, resource utilization, and workload performance across cloud environments.

§ Collaborate with business stakeholders, product owners, and architects to understand business requirements and translate them into scalable technical solutions.

§ Define and implement enterprise data governance, security, access controls, lineage, and compliance standards within Databricks.

§ Leverage cloud-native services across Azure, AWS, or GCP to build integrated and scalable data solutions.

§ Develop and deploy code using GitHub, CI/CD pipelines, and Agile development methodologies, ensuring adherence to coding standards and best practices.

§ Work with large-scale datasets while implementing robust security models, data quality frameworks, and performance optimization techniques.

§ Perform system integration and end-to-end testing to ensure data accuracy, completeness, and reliability across platforms.

§ Analyze, troubleshoot, and resolve defects, data issues, and production incidents in a timely manner, ensuring minimal business impact.

§ Facilitate discussions with business and technical teams to proactively address data, integration, and platform-related challenges.

§ Ensure effective management of resources, dependencies, and deliverables related to data engineering and integration initiatives.

§ Document technical designs, data models, architecture decisions, and operational procedures to support long-term platform sustainability.  

Qualifications

§ Bachelor’s or master’s degree in any field

§ Your experience in designing, implementing, and maintaining data solutions on Databricks

§ Experience with at least one of the popular cloud platforms – Azure, AWS or GCP

§ Experience with ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes

§ Knowledge of data warehousing and data modelling concepts

§ Experience with Python or SQL

§ Experience with Delta Lake

§ Understanding of DevOps principles and practices

§ Excellent problem-solving and troubleshooting skills

§ Strong communication and teamwork skills

 Your role as a leader

At Deloitte India, we believe in the importance of leadership at all levels. We expect our people to embrace and live our purpose by challenging themselves to identify issues that are most important for our clients, our people, and for society and make an impact that matters.

In addition to living our purpose, Consultant across our organization:

Develop high-performing people and teams through challenging and meaningful opportunities

Deliver exceptional client service; maximize results and drive high performance from people while fostering collaboration across businesses and borders

Influence clients, teams, and individuals positively, leading by example and establishing confident relationships with increasingly senior people

Understand key objectives for clients and Deloitte; align people to objectives and set priorities and direction.

Acts as a role model, embracing and living our purpose and values, and recognizing others for the impact they make

How you’ll grow

At Deloitte, our professional development plan focuses on helping people at every level of their career to identify and use their strengths to do their best work every day. From entry-level employees to senior leaders, we believe there’s always room to learn. We offer opportunities to help build world-class skills in addition to hands-on experience in the global, fast-changing business world. From on-the-job learning experiences to formal development

Programs at Deloitte University, our professionals have a variety of opportunities to continue to grow throughout their career. Explore Deloitte University, The Leadership Centre.

Benefits

At Deloitte, we know that great people make a great organization. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you.

Our purpose                                                                                                                                                                                                                                                                              

Deloitte is led by a purpose: To make an impact that matters.

Every day, Deloitte people are making a real impact in the places they live and work. We pride ourselves on doing not only what is good for clients, but also what is good for our people and the communities in which we live and work—always striving to be an organization that is held up as a role model of quality, integrity, and positive change. Learn more about Deloitte's impact on the world

Recruiter tips

We want job seekers exploring opportunities at Deloitte to feel prepared and confident. To help you with your interview, we suggest that you do your research: know some background about the organization and the business area you’re applying to. Check out recruiting tips from Deloitte professionals.