Apache Spark Technical Specialist - Scala, Python
Chennai · Mid Level
Applicants who checked fit first are 3.1× more likely to hear back
Your score for this role already exists
ASAI compared this JD against 41 signals - skills, seniority, domain, stack overlap etc. Add a resume and it unlocks in about 30 seconds.
Always free · no credit card · 1 tap with Google
What we know about this role
HIGH
Our tracked job listings for HCLTech show strong recent activity.
First 72 hours
Window passed - posted 31d agoEarly applicants get seen before the pile builds.
Not a repost
The first time we've seen this listing - it hasn't been closed and reopened.
Skills required
22 listedYou almost certainly match several of these already. Unlock your skill map to see the matches, the gaps, and what to fix first.
Job description
Job Summary
Scala Data Engineer
Lloyds Banking Group is the UK’s largest Retail, Digital and Mobile bank, supporting over 30 million customers. Our products and services touch the lives of millions of people every day, giving us a unique opportunity to Help Britain Prosper. Technology, data and engineering are central to our strategy, with continued investment in cloud, data platforms, artificial intelligence and automation.
The Finance Platform has an opportunity within its Data Application Solutions Lab function for an individual to support the delivery of modern data solutions and reporting capabilities across Insurance Finance. Working within a cloud-first environment, you will help build, enhance and support strategic data products that enable critical Finance processes and decision-making.
Our engineering teams develop data solutions primarily using Scala and Apache Spark on Google Cloud Platform (GCP). We leverage modern cloud-native technologies including BigQuery, DataProc, GCS and automated CI/CD pipelines to deliver scalable and resilient data platforms. Whilst experience in Scala is beneficial, we welcome applications from engineers with strong software engineering fundamentals gained in other languages or technologies who are keen to develop their skills in modern data engineering. Data Engineering experience with problem solving in industry is particularly important.
You will begin by contributing to enhancements and support activities across our existing platforms. We work using Agile delivery practices, including Scrum, Kanban, sprint planning, backlog refinement and continuous improvement.
This is a hands-on role where you will work closely with Finance, Accounting, Risk and Engineering colleagues to solve complex business problems through practical and scalable data engineering solutions. The role requires strong analytical thinking, problem-solving skills and a passion for delivering high-quality technology outcomes.
We are committed to creating an inclusive environment where colleagues can thrive and be themselves. We actively support diversity across all backgrounds and experiences and are proud to offer reasonable adjustments throughout our recruitment process.
Technical Proficiencies
- Computer Science, Software Engineering or related degree, or equivalent industry experience.
- Experience developing and supporting data solutions using Apache Spark and distributed processing frameworks.
- Experience developing applications in Scala or another object-oriented/JVM-based programming language.
- Strong understanding of software engineering principles including testing, code quality and maintainability.
- Experience with unit testing frameworks such as Xray or equivalent.
- Experience delivering solutions on Google Cloud Platform (GCP).
- Strong SQL \ Python skills for data analysis, troubleshooting, optimisation and data profiling.
- Experience working with BigQuery and large-scale analytical datasets.
- Familiarity with CI/CD and DevOps tooling including Maven, Git, Nexus, SonarQube, Harness and automated deployment pipelines.
- Experience using JIRA and Confluence to manage Agile delivery and technical documentation.
- Experience working within Agile teams using Scrum, sprint planning and iterative delivery practices.
- Strong communication and stakeholder management skills.
- Demonstrable ability to apply practical problem-solving techniques to real-world business and data challenges.
Nice to Have
- Experience with DataProc, Cloud Composer, GCS or other GCP data services.
- Experience within Financial Services, particularly Finance, Insurance, Regulatory Reporting or Risk data domains.
- Knowledge of modern data platform architectures and cloud migration programmes.
- Experience supporting production platforms and investigating data or operational issues.
- Exposure to DevOps, platform engineering or infrastructure-as-code practices.
- Previous experience with SAS, particularly in reporting, analytics or legacy platform support.
Key Responsibilities
Scala Data Engineer
Lloyds Banking Group is the UK’s largest Retail, Digital and Mobile bank, supporting over 30 million customers. Our products and services touch the lives of millions of people every day, giving us a unique opportunity to Help Britain Prosper. Technology, data and engineering are central to our strategy, with continued investment in cloud, data platforms, artificial intelligence and automation.
The Finance Platform has an opportunity within its Data Application Solutions Lab function for an individual to support the delivery of modern data solutions and reporting capabilities across Insurance Finance. Working within a cloud-first environment, you will help build, enhance and support strategic data products that enable critical Finance processes and decision-making.
Our engineering teams develop data solutions primarily using Scala and Apache Spark on Google Cloud Platform (GCP). We leverage modern cloud-native technologies including BigQuery, DataProc, GCS and automated CI/CD pipelines to deliver scalable and resilient data platforms. Whilst experience in Scala is beneficial, we welcome applications from engineers with strong software engineering fundamentals gained in other languages or technologies who are keen to develop their skills in modern data engineering. Data Engineering experience with problem solving in industry is particularly important.
You will begin by contributing to enhancements and support activities across our existing platforms. We work using Agile delivery practices, including Scrum, Kanban, sprint planning, backlog refinement and continuous improvement.
This is a hands-on role where you will work closely with Finance, Accounting, Risk and Engineering colleagues to solve complex business problems through practical and scalable data engineering solutions. The role requires strong analytical thinking, problem-solving skills and a passion for delivering high-quality technology outcomes.
We are committed to creating an inclusive environment where colleagues can thrive and be themselves. We actively support diversity across all backgrounds and experiences and are proud to offer reasonable adjustments throughout our recruitment process.
Technical Proficiencies
- Computer Science, Software Engineering or related degree, or equivalent industry experience.
- Experience developing and supporting data solutions using Apache Spark and distributed processing frameworks.
- Experience developing applications in Scala or another object-oriented/JVM-based programming language.
- Strong understanding of software engineering principles including testing, code quality and maintainability.
- Experience with unit testing frameworks such as Xray or equivalent.
- Experience delivering solutions on Google Cloud Platform (GCP).
- Strong SQL \ Python skills for data analysis, troubleshooting, optimisation and data profiling.
- Experience working with BigQuery and large-scale analytical datasets.
- Familiarity with CI/CD and DevOps tooling including Maven, Git, Nexus, SonarQube, Harness and automated deployment pipelines.
- Experience using JIRA and Confluence to manage Agile delivery and technical documentation.
- Experience working within Agile teams using Scrum, sprint planning and iterative delivery practices.
- Strong communication and stakeholder management skills.
- Demonstrable ability to apply practical problem-solving techniques to real-world business and data challenges.
Nice to Have
- Experience with DataProc, Cloud Composer, GCS or other GCP data services.
- Experience within Financial Services, particularly Finance, Insurance, Regulatory Reporting or Risk data domains.
- Knowledge of modern data platform architectures and cloud migration programmes.
- Experience supporting production platforms and investigating data or operational issues.
- Exposure to DevOps, platform engineering or infrastructure-as-code practices.
- Previous experience with SAS, particularly in reporting, analytics or legacy platform support.
Skill Requirements
Other Requirements
Tools to land this role
PremiumYour match score, skill map and tracker stay free.
Ground Reality
PremiumSpot the dealbreakers in this 1070-word post before you apply
Skip jobs with hidden dealbreakers. See what they actually need, what's easy to miss, and how they describe the team.
Ground Reality on every job, plus Insider Path, practice interviews and resume tailoring.
Insider Path
PremiumFind someone who can refer you here
We help you find people there who worked or studied where you did, and draft your first message and follow-up.
Free · no signup
Get tomorrow's jobs before you have to search
Daily job drops, skill trends and free resources - posted straight to the group. Leave any time.
No spam. Just jobs and resources.
Why people use ASAI
Someone shared one job with you. ASAI keeps finding the rest.
Scored, not searched. Every role ranked against your actual profile.
Alerts as often as hourly. Reach new roles while the pile is still small.
Skill gaps, spelled out. See exactly which requirements you don't meet yet.
Verified jobs, only. Say no to ghost jobs. Your time deserves respect.
More Data Engineer roles in Chennai
See all
Keep browsing