Senior Technical Lead
Bengaluru · Staff/Principal
Applicants who checked fit first are 3.1× more likely to hear back
Your score for this role already exists
ASAI compared this JD against 41 signals - skills, seniority, domain, stack overlap etc. Add a resume and it unlocks in about 30 seconds.
Always free · no credit card · 1 tap with Google
What we know about this role
HIGH
Our tracked job listings for HCLTech show strong recent activity.
First 72 hours
Window passed - posted 48d agoEarly applicants get seen before the pile builds.
Not a repost
The first time we've seen this listing - it hasn't been closed and reopened.
Skills required
16 listedYou almost certainly match several of these already. Unlock your skill map to see the matches, the gaps, and what to fix first.
Job description
Job Summary
Data Engineer
Role: Core Data Engineer (AWS)
Summary: You will design, build, and operate secure, reliable, and cost efficient data pipelines on AWS, covering batch, streaming, and CDC ingestion
through to curated datasets ready for analytics. You will embed testing, governance, observability, and CI/CD across the data lifecycle.
Key Responsibilities
Key responsibilities
• Ingestion and CDC: Implement pipelines using AWS Lambda , EMR for batch/streaming( future need); manage schema evolution and backfills.
• Orchestration: Build workflows with Step Functions and/or MWAA (Airflow), including retries, alerts, and SLAs.
• Lakehouse and warehouse: Model datasets on S3 (Parquet/Iceberg), Glue Data Catalog, Athena; partitioning, compaction, and performance tuning.
• Data quality and testing: Add automated tests (Great Expectations/Deequ,PySpark unit tests), data contracts, and CI gates.
• Security and governance: Implement IAM, KMS, VPC endpoints, Lake Formation policies, PII handling, audit trails (CloudTrail), and RBAC/RLS.
• Observability and FinOps: CloudWatch metrics/alerts, lineage, usage dashboards; cost optimisation (S3 lifecycle, compression, job sizing).
• CI/CD and IaC: Provision with Terraform/CloudFormation/CDK; build pipelines with GitHub Actions; environment promotion and rollbacks.
• Documentation and runbooks: Maintain pipeline diagrams, SLAs/SLOs, and incident playbooks.
• Adoption: be able to adopt capabilities from other teams instead of building duplicate capabilities and be able to contribute back to community.
Outcomes (first 60–90 days)
• Productionise one end to end pipeline with tests, monitoring, and alerting.
• Establish governance baseline (Lake Formation, tagging/classification, encryption) and data contracts for two key sources.
• Stand up CI/CD and IaC for data services; reduce at least one cost driver via storage/compute optimisation.
Skill Requirements
Skills and experience
• 10+ years building data platforms on AWS; strong Python (incl. PySpark) and SQL.
• Hands on with DMS, Kinesis/MSK, Glue/EMR, Lambda, Step Functions/MWAA,S3/Parquet/Iceberg, Glue Catalog, Athena, Redshift/Serverless.
• IaC (Terraform) and CI/CD (GitHub Actions).
• Data quality frameworks (Great Expectations/Deequ), schema evolution,backfills.
• Security and compliance: IAM/KMS, private networking, Lake Formation,audit/lineage.
• Performance and cost tuning across storage and compute.
Nice to have
• Snowflake, Apache Flink, Redshift Spectrum, OpenLineage, OPA/policy as code.
Other Requirements
Tools to land this role
PremiumYour match score, skill map and tracker stay free.
Ground Reality
PremiumSpot the dealbreakers in this 340-word post before you apply
Skip jobs with hidden dealbreakers. See what they actually need, what's easy to miss, and how they describe the team.
Ground Reality on every job, plus Insider Path, practice interviews and resume tailoring.
Insider Path
PremiumFind someone who can refer you here
We help you find people there who worked or studied where you did, and draft your first message and follow-up.
Free · no signup
Get tomorrow's jobs before you have to search
Daily job drops, skill trends and free resources - posted straight to the group. Leave any time.
No spam. Just jobs and resources.
Why people use ASAI
Someone shared one job with you. ASAI keeps finding the rest.
Scored, not searched. Every role ranked against your actual profile.
Alerts as often as hourly. Reach new roles while the pile is still small.
Skill gaps, spelled out. See exactly which requirements you don't meet yet.
Verified jobs, only. Say no to ghost jobs. Your time deserves respect.
More Data Engineer roles in Bengaluru
See all
Keep browsing