Job Role: Senior Data Engineer
Location: Hyderabad, India
Experience: 6–8 Years
Work Mode: Hybrid
About Anblicks
Anblicks is a Great Place to Work® Certified Data & AI company helping enterprises build modern, intelligent, and scalable data platforms. As a Snowflake Elite Services Partner, Databricks Consulting Partner, AWS Advanced Consulting Partner, and Microsoft Azure Gold Partner, we work with leading organizations to solve complex data, cloud, analytics, and AI challenges.
We are looking for an experienced Senior Data Engineer to design and build scalable data platforms and high-performance data solutions using Databricks, PySpark, Python, SQL, and cloud technologies.
Role Summary
As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing modern data engineering solutions and Lakehouse platforms. You will work on large-scale data processing, data pipelines, data modeling, performance optimization, governance, and cloud-native technologies while contributing to technical design and engineering best practices.
Key Responsibilities
- Design and develop scalable, reliable, and high-performance data pipelines using Databricks, Spark, PySpark, and cloud technologies.
- Design and implement complex ETL/ELT workflows using Databricks, PySpark, SQL, and cloud services.
- Establish engineering standards, design patterns, and best practices for data engineering solutions.
- Make technical and architectural recommendations focused on scalability, performance, maintainability, security, and cost optimization.
- Develop reusable frameworks, libraries, and accelerators to improve engineering productivity.
- Design and implement optimized data models for analytics, operational, and AI/ML use cases.
- Identify and resolve performance bottlenecks through Spark optimization, query tuning, partitioning, and resource management.
- Implement and support data quality, monitoring, observability, governance, and security capabilities.
- Follow software engineering best practices including code reviews, automated testing, CI/CD, Infrastructure as Code, and DevSecOps.
- Lead root-cause analysis and resolution of complex production issues.
- Drive improvements in platform reliability, resiliency, scalability, and operational efficiency.
- Contribute to modern DataOps and Data Product practices.
- Collaborate with architects, developers, data scientists, business stakeholders, and cross-functional teams.
Required Skills & Experience
- Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.
- 6–8 years of overall Data Engineering experience, including 5+ years of hands-on Databricks experience.
- Strong experience building large-scale data platforms on public cloud environments.
- Expert-level proficiency in Python, PySpark, SQL, and distributed data processing.
- Strong understanding of Lakehouse architecture, data warehousing, and modern data engineering concepts.
- Hands-on experience with Spark optimization, performance tuning, partitioning strategies, and workload management.
- Strong experience with AWS services such as S3, EMR, Glue, Lambda, ECS/EKS, Redshift, and IAM.
- Experience designing scalable data models and data architectures for analytical and operational workloads.
- Strong understanding of CI/CD, automated testing, and software engineering best practices.
- Hands-on experience with Databricks Asset Bundles (DABs).
- Experience implementing security, governance, privacy, and compliance controls within data platforms.
- Ability to drive technical initiatives and contribute to architectural decisions.
- Strong problem-solving, communication, and stakeholder management skills.
- Experience working in Agile, cross-functional, and globally distributed teams.
Good to Have
- Experience with Infrastructure as Code tools such as Terraform or CloudFormation.
- Experience implementing data quality, observability, and metadata management solutions.
- Experience with streaming technologies such as Kafka, Azure Event Hubs, or Amazon Kinesis.
- Experience deploying and operationalizing Machine Learning and Generative AI solutions.
- Strong knowledge of Delta Lake optimization, Unity Catalog, and Databricks governance.
- Experience leading technical design reviews and mentoring engineering teams.
Why Anblicks?
- Work on large-scale Data, Cloud, AI & Analytics projects.
- Opportunity to work with leading cloud and data technologies including Databricks, Snowflake, AWS, Azure, and AI/ML.
- Exposure to enterprise-scale transformation and modernization initiatives.
- Collaborative, technology-driven engineering environment.