Deputy Manager | Data Engineer | Hyderabad | Regulatory & Financial Risk

The team

Deloitte Strategy, Risk & Transaction helps entities mitigate risk while discovering new opportunities to create value. Our end-to-end risk services span all domains, from managing strategic risks in the C-Suite to improving board oversight, and from balancing financial and environmental policies to addressing cyber threats. Learn more about Risk, Regulatory & Forensic)

Your work profile

• We are looking for a skilled Senior Developer with strong expertise in PySpark to design, build, and maintain scalable data processing solutions.
• The ideal candidate will have experience working across the data engineering stack, strong knowledge of distributed computing, and hands-on exposure to SDLC-based project development. 
• We are seeking an experienced Python Backend Developer to design, build, and maintain scalable backend systems and APIs.
• The ideal candidate has strong expertise in Python frameworks, database design, and backend performance optimization. 
• We are looking for a DevOps Engineer to build, automate, and maintain scalable, secure, and highly available infrastructure.
• The ideal candidate has strong experience in cloud platforms, CI/CD automation, and modern DevOps practices.

Key Responsibilities: 

• Design, develop, and optimize large-scale data pipelines using PySpark. 
• Build and maintain ETL/ELT workflows to process structured and unstructured data. 
• Work with big data technologies such as Hadoop ecosystem, Spark, and Impala. 
• Collaborate with cross-functional teams including data scientists, analysts, and product teams. 
• Ensure code quality through best practices, code reviews, and version control. 
• Optimize job performance, memory management, and execution time in Spark jobs. 
• Work on data modeling, transformation, and pipeline orchestration. 
• Participate in end-to-end SDLC activities including requirement analysis, design, development, testing, and deployment. 
• Troubleshoot and resolve data processing issues and production incidents. 
• Contribute to automation, monitoring, and logging frameworks. 
• Required Skills and Qualifications: 
• Strong hands-on experience with Python and PySpark (mandatory). 
• Strong SQL skills including advanced querying and performance tuning. 
• Experience with distributed data systems such as Hadoop and Spark ecosystem. 
• Experience with Impala, Hive, and SQL Server or other relational databases. 
• Strong understanding of data structures, algorithms, and performance optimization. 
• Experience with ETL pipeline development and data warehousing concepts. 
• Knowledge of SDLC methodologies (Agile/Scrum preferred). 
• Familiarity with version control systems such as Git. 
• Experience working with large datasets in production environments. 
• Knowledge of C# or other programming languages. 
• Experience with cloud platforms such as Azure, AWS, or GCP. 
• Exposure to workflow orchestration tools like Airflow or Oozie. 
• Understanding of data lakes, data governance, and data quality frameworks. 
• Experience with CI/CD pipelines. 
• Strong analytical and problem-solving abilities. 
• Good communication and collaboration skills. 
• Ability to work independently and in a fast-paced environment. 
• Proactive mindset with attention to detail. 
• Preferred Candidate Profile: 
• 5-8 Years of Hands-on PySpark developer with real-time project delivery experience. 
• Bacheler’s degree in any specialization
• Exposure to end-to-end data pipeline lifecycle. 
• Comfortable handling large-scale data processing and optimization challenges.