Job description

Senior Lead Data Engineer

Location: Remote/ Bangalore

Experience: 11+ years

About the Role:

Are you passionate about building high-performance data systems? Do you thrive at the intersection of Data Engineering, Cloud, and Big Data technologies?

We’re looking for a Senior Lead Data Engineer to join our growing team! If you have deep expertise in ETL/ELT pipelines, Data Warehousing, Cloud platforms (GCP, AWS), and Hadoop ecosystems, we want to hear from you!

ETL Tools: primarily use Pentaho Data Integration (PDI) for designing and managing ETL workflows, including data extraction, transformation, and loading from various sources into target systems.

Databases:

MySQL for transactional data handling and intermediate data staging.
Amazon Redshift as the primary data warehouse for analytics and reporting workloads.

Workflow Monitoring & Orchestration:

Azkaban is used for monitoring and managing data pipeline executions and ensuring task dependencies are handled correctly.

Job Scheduling & Scripting:

Crontab and Unix shell scripting are utilized for scheduling recurring jobs and automating data processes on a daily or hourly basis.

Cloud Services (AWS):

Hands-on experience with:
RDS for managing relational database services.
Amazon Redshift for handling analytical queries and data warehousing.
EC2 for running compute workloads.
Basic IAM (Identity and Access Management) for managing user roles and access permissions.

CI/CD & Deployment:

Jenkins is used for automating the deployment of ETL scripts and scheduling code pushes to production environments.

What You'll Do:

Architect, design, and build robust ETL/ELT pipelines using Informatica PowerCenter and cloud-native solutions.
Work hands-on with Data Warehousing concepts and Data Modeling to power analytics and insights.
Build large-scale Hadoop workflows with tools like Hive, Pig, Spark, and HDFS.
Collaborate closely with business and technical teams to translate requirements into scalable solutions.
Produce high-quality technical documents: design docs, mapping specs, and unit test plans.
Optimize big data processing and cloud infrastructure for performance, scalability, and cost.
Mentor junior engineers and contribute to a culture of technical excellence.

Tech Stack You'll Work With:

Informatica PowerCenter
GCP (BigQuery, Dataflow, Pub/Sub)
AWS (Redshift, S3, Glue, Lambda)
Hadoop (Hive, Pig, Spark, HDFS)
SQL, Python (bonus!)
DevOps, CI/CD, Docker/Kubernetes (nice to have!)

What You Bring:

11+ years of IT experience with a strong focus on Data Engineering.
Solid foundation in Data Warehouse architecture and Data Modeling principles.
Deep understanding of Cloud and Big Data technologies.
Excellent communication, documentation, and troubleshooting skills.
Ability to thrive in a fast-paced, agile environment.

Your Background:

Bachelor's or Master’s in Computer Science, Information Technology, or related field.

Someone shared one job with you. ASAI keeps finding the rest.

Scored, not searched. Every role ranked against your actual profile.

Alerts as often as hourly. Reach new roles while the pile is still small.

Skill gaps, spelled out. See exactly which requirements you don't meet yet.

Verified jobs, only. Say no to ghost jobs. Your time deserves respect.