AES - Delivery - Digital Test Automation Engineer
Bengaluru · Senior
Applicants who checked fit first are 3.1× more likely to hear back
Your score for this role already exists
ASAI compared this JD against 41 signals - skills, seniority, domain, stack overlap etc. Add a resume and it unlocks in about 30 seconds.
Always free · no credit card · 1 tap with Google
What we know about this role
LOW
Our tracked job listings for fa-etvl-saasfaprod1-cx1 show limited recent activity.
First 72 hours
Window passed - posted 7d agoEarly applicants get seen before the pile builds.
Not a repost
The first time we've seen this listing - it hasn't been closed and reopened.
Skills required
33 listedYou almost certainly match several of these already. Unlock your skill map to see the matches, the gaps, and what to fix first.
Job description
You will own quality for data pipelines, microservices, APIs, GUIs, and LLM/ML integrations running on Databricks and Microsoft Azure. The role focuses on test strategy, test case design and execution, automation, ticket management, and agentic QA for LLM workflows. Minimum experience: 5 years
Key Responsibilities
Test Strategy and Planning
Define and maintain QA strategy for data, API, GUI, microservices, and LLM/ML features; prioritize test coverage based on risk and business impact.
Test Case Design and Execution
Create clear, reusable test cases for functional, integration, regression, performance, and pricing scenarios; execute manual and automated tests and maintain test artifacts.
Data and Database Testing
Validate ETL/ELT pipelines on Databricks and Delta Lake; write and run T‑SQL validation queries; verify data lineage, completeness, and correctness.
API Testing
Design and run contract, integration, and negative tests for RESTful APIs; use Postman, OpenAPI/Swagger, and automated API test suites.
GUI Testing
Create and execute UI test plans for web applications; perform exploratory testing and maintain automated UI tests using Selenium, Playwright, or equivalent.
Microservices Validation
Test service contracts, message flows, resilience, and backward compatibility across containerized microservices; validate observability and error handling.
LLM and ML QA
Design evaluation metrics and test harnesses for ML models and LLM integrations; validate inference correctness, hallucination rates, latency, and cost tradeoffs.
Prompt Engineering and Agentic QA
Create minimalistic, cost‑aware prompts and prompt templates; implement agentic QA processes for autonomous agents including safety checks, guardrails, and evaluation pipelines.
Pricing and Business Logic Testing
Validate pricing engines, discount rules, billing flows, and edge cases; create scenario matrices and automated checks for pricing correctness.
Automation and CI/CD
Integrate tests into CI/CD pipelines; author automated test suites, test runners, and validation scripts; collaborate with SRE/DevOps on test environments.
Ticketing and Incident Management
Raise, triage, and manage tickets in Azure DevOps or Jira; document reproduction steps, root cause analysis, and remediation.
Documentation and Collaboration
Produce test plans, runbooks, release checklists, and postmortems; mentor junior QA engineers and participate in cross‑functional reviews.
Required Skills and Experience
Experience: Minimum 5 years in QA, software testing, or related roles.
Databricks and Data Testing: Hands‑on experience validating Databricks pipelines, Delta Lake, and Spark outputs.
Microsoft SQL Server: Medium proficiency in T‑SQL for validation queries and database testing.
Azure: Practical experience with Azure Data Factory, ADLS/Blob Storage, Azure DevOps, and Azure Functions.
API Testing: Strong experience with REST APIs, Postman, OpenAPI/Swagger, and automated API testing.
GUI Testing: Experience with Selenium, Playwright, or equivalent UI automation tools and exploratory testing.
Microservices: Familiarity testing containerized services, message queues, and service orchestration.
LLM and ML Fundamentals: Understanding of ML lifecycle, model evaluation, prompt engineering basics, and LLM failure modes.
Prompt Engineering: Ability to craft minimalistic prompts and maintain prompt libraries for reproducible tests.
Agentic QA Awareness: Knowledge of agentic QA concepts, safety checks, and evaluation metrics for autonomous agents.
Testing Tools and Automation: Experience with test frameworks, CI/CD integration, and test data management.
Ticketing Tools: Experience using Azure DevOps, Jira, or similar.
Soft Skills: Strong analytical thinking, clear written communication, and cross‑team collaboration.
Nice to Have
Certifications: ISTQB, Azure Data Engineer, or Databricks certifications.
Programming: Python for test automation and data validation; familiarity with Scala or Java.
Observability: Experience with Application Insights, Datadog, Prometheus, or Grafana.
Performance Testing: Experience with JMeter, k6, or similar tools.
LLM Ops: Experience with prompt versioning, cost monitoring, and model evaluation frameworks.
Infrastructure as Code: Familiarity with Terraform or ARM templates for test environment provisioning.
Interview and Evaluation Guide
Technical screen: SQL validation problem and API testing scenarios.
Practical exercise: Create test cases and automated checks for a sample Databricks ETL and an API that returns pricing.
LLM prompt exercise: Provide minimal prompts for a given LLM task and design evaluation metrics for correctness and safety.
System design: Design an agentic QA pipeline for autonomous agents including test harness, safety gates, and monitoring.
Behavioral: Examples of ticket triage, incident postmortems, and cross‑team collaboration.
Success Criteria at 6 Months
Test Coverage: Comprehensive, automated test suites for data pipelines, APIs, and core GUI flows.
LLM Readiness: Reproducible prompt templates, evaluation metrics, and agentic QA checks integrated into CI.
Reliability: Reduced production incidents and faster mean time to resolution through clear runbooks and test automation.
Pricing Confidence: End‑to‑end validated pricing flows with scenario tests and regression guards.
Collaboration: Trusted QA partner to engineering and product teams with documented processes and mentoring outcomes.
Key Responsibilities
Test Strategy and Planning
Define and maintain QA strategy for data, API, GUI, microservices, and LLM/ML features; prioritize test coverage based on risk and business impact.
Test Case Design and Execution
Create clear, reusable test cases for functional, integration, regression, performance, and pricing scenarios; execute manual and automated tests and maintain test artifacts.
Data and Database Testing
Validate ETL/ELT pipelines on Databricks and Delta Lake; write and run T‑SQL validation queries; verify data lineage, completeness, and correctness.
API Testing
Design and run contract, integration, and negative tests for RESTful APIs; use Postman, OpenAPI/Swagger, and automated API test suites.
GUI Testing
Create and execute UI test plans for web applications; perform exploratory testing and maintain automated UI tests using Selenium, Playwright, or equivalent.
Microservices Validation
Test service contracts, message flows, resilience, and backward compatibility across containerized microservices; validate observability and error handling.
LLM and ML QA
Design evaluation metrics and test harnesses for ML models and LLM integrations; validate inference correctness, hallucination rates, latency, and cost tradeoffs.
Prompt Engineering and Agentic QA
Create minimalistic, cost‑aware prompts and prompt templates; implement agentic QA processes for autonomous agents including safety checks, guardrails, and evaluation pipelines.
Pricing and Business Logic Testing
Validate pricing engines, discount rules, billing flows, and edge cases; create scenario matrices and automated checks for pricing correctness.
Automation and CI/CD
Integrate tests into CI/CD pipelines; author automated test suites, test runners, and validation scripts; collaborate with SRE/DevOps on test environments.
Ticketing and Incident Management
Raise, triage, and manage tickets in Azure DevOps or Jira; document reproduction steps, root cause analysis, and remediation.
Documentation and Collaboration
Produce test plans, runbooks, release checklists, and postmortems; mentor junior QA engineers and participate in cross‑functional reviews.
Required Skills and Experience
Experience: Minimum 5 years in QA, software testing, or related roles.
Databricks and Data Testing: Hands‑on experience validating Databricks pipelines, Delta Lake, and Spark outputs.
Microsoft SQL Server: Medium proficiency in T‑SQL for validation queries and database testing.
Azure: Practical experience with Azure Data Factory, ADLS/Blob Storage, Azure DevOps, and Azure Functions.
API Testing: Strong experience with REST APIs, Postman, OpenAPI/Swagger, and automated API testing.
GUI Testing: Experience with Selenium, Playwright, or equivalent UI automation tools and exploratory testing.
Microservices: Familiarity testing containerized services, message queues, and service orchestration.
LLM and ML Fundamentals: Understanding of ML lifecycle, model evaluation, prompt engineering basics, and LLM failure modes.
Prompt Engineering: Ability to craft minimalistic prompts and maintain prompt libraries for reproducible tests.
Agentic QA Awareness: Knowledge of agentic QA concepts, safety checks, and evaluation metrics for autonomous agents.
Testing Tools and Automation: Experience with test frameworks, CI/CD integration, and test data management.
Ticketing Tools: Experience using Azure DevOps, Jira, or similar.
Soft Skills: Strong analytical thinking, clear written communication, and cross‑team collaboration.
Nice to Have
Certifications: ISTQB, Azure Data Engineer, or Databricks certifications.
Programming: Python for test automation and data validation; familiarity with Scala or Java.
Observability: Experience with Application Insights, Datadog, Prometheus, or Grafana.
Performance Testing: Experience with JMeter, k6, or similar tools.
LLM Ops: Experience with prompt versioning, cost monitoring, and model evaluation frameworks.
Infrastructure as Code: Familiarity with Terraform or ARM templates for test environment provisioning.
Interview and Evaluation Guide
Technical screen: SQL validation problem and API testing scenarios.
Practical exercise: Create test cases and automated checks for a sample Databricks ETL and an API that returns pricing.
LLM prompt exercise: Provide minimal prompts for a given LLM task and design evaluation metrics for correctness and safety.
System design: Design an agentic QA pipeline for autonomous agents including test harness, safety gates, and monitoring.
Behavioral: Examples of ticket triage, incident postmortems, and cross‑team collaboration.
Success Criteria at 6 Months
Test Coverage: Comprehensive, automated test suites for data pipelines, APIs, and core GUI flows.
LLM Readiness: Reproducible prompt templates, evaluation metrics, and agentic QA checks integrated into CI.
Reliability: Reduced production incidents and faster mean time to resolution through clear runbooks and test automation.
Pricing Confidence: End‑to‑end validated pricing flows with scenario tests and regression guards.
Collaboration: Trusted QA partner to engineering and product teams with documented processes and mentoring outcomes.
Tools to land this role
PremiumYour match score, skill map and tracker stay free.
Ground Reality
PremiumSpot the dealbreakers in this 1399-word post before you apply
Skip jobs with hidden dealbreakers. See what they actually need, what's easy to miss, and how they describe the team.
Ground Reality on every job, plus Insider Path, practice interviews and resume tailoring.
Insider Path
PremiumFind someone who can refer you here
We help you find people there who worked or studied where you did, and draft your first message and follow-up.
Free · no signup
Get tomorrow's jobs before you have to search
Daily job drops, skill trends and free resources - posted straight to the group. Leave any time.
No spam. Just jobs and resources.
Why people use ASAI
Someone shared one job with you. ASAI keeps finding the rest.
Scored, not searched. Every role ranked against your actual profile.
Alerts as often as hourly. Reach new roles while the pile is still small.
Skill gaps, spelled out. See exactly which requirements you don't meet yet.
Verified jobs, only. Say no to ghost jobs. Your time deserves respect.
More Quality Assurance Engineer roles in Bengaluru
See all
Keep browsing