Site Reliability Engineer II

The Enterprise Technology Services organization partners with every part of the American Express business to power the company’s growth and innovation with trust and efficiency, and drive competitive differentiation with speed. We support the delivery and operations of technology, digital, and data capabilities, platforms, and services globally. Specifically, our team is responsible for the company’s technology engineering, architecture, and infrastructure, providing 24x7 support to ensure an uninterrupted, high-quality experience for customers and colleagues.

We are looking for a talented Site Reliability Engineer (SRE) with strong Java/Kotlin software engineering skills to join our team in Bengaluru, India. In this role, you will help enhance the reliability, resiliency, and operational excellence of mission-critical systems through software engineering, automation, observability, and proactive failure prevention. You will leverage modern engineering practices and emerging AI capabilities to improve incident diagnostics, accelerate recovery, automate operational workflows, and reduce service disruptions. Working alongside experienced engineers, you will contribute to building scalable, highly available systems that deliver exceptional experiences for our customers.

• Function as a member of an Agile engineering team, contributing to the design, development, deployment, and operational reliability of highly available, scalable, and resilient applications.
• Develop high-quality, maintainable software and automation solutions using Java, Kotlin, or Python, leveraging AI-assisted coding and Spec-Driven Development (SDD) practices to improve engineering productivity and code quality.
• Contribute to Non-Abstract Large System Design (NALSD), supporting the design and implementation of scalable, fault-tolerant distributed systems for mission-critical payment processing and customer-facing applications.
• Participate in Failure Mode Analysis to identify potential system vulnerabilities, assess failure scenarios, and implement mitigation strategies to improve system resilience.
• Design and execute controlled chaos engineering experiments to validate system resilience, identify weaknesses, and improve fault tolerance and recovery capabilities.
• Design and enhance application observability using OpenTelemetry (OTEL), distributed tracing, metrics, logging, and monitoring frameworks to improve visibility into system health and operational reliability.
• Contribute to defining, monitoring, and improving Service Level Indicators (SLIs), Service Level Objectives (SLOs), and error budgets to drive data-informed reliability improvements.
• Develop and maintain automation for application deployments, infrastructure operations, incident response, and system recovery, leveraging resilient workflow orchestration technologies such as Temporal where appropriate.
• Explore and apply Generative AI capabilities, including RAG, context engineering, and agentic workflows using frameworks such as LangGraph, to improve operational automation, incident investigation, troubleshooting, and engineering efficiency.
• Support the reliability and observability of AI-enabled workflows, including automated evaluation using Promptfoo, safety controls using NeMo Guardrails, and AI observability practices.
• Troubleshoot application and infrastructure issues, participate in incident response and root cause analysis, and implement corrective and preventive actions to reduce recurring failures.
• Develop and maintain operational runbooks, troubleshooting guides, and automated remediation procedures to improve operational readiness.
• Implement and enhance CI/CD pipelines, application scaling, automated testing, and infrastructure automation across development, testing, and production environments.
• Participate in code reviews, performance optimization, and continuous improvement initiatives while collaborating with software engineering, infrastructure, security, and platform teams.

• 4+ years of professional experience in software engineering, Site Reliability Engineering, DevOps, or related technical disciplines, or equivalent practical experience.
• Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
• Strong programming skills in Java, Kotlin, or Python, with experience developing production-quality applications, automation tools, and distributed services.
• Strong understanding of data structures, algorithms, object-oriented programming, system design, and software engineering principles.
• Experience designing, developing, testing, and supporting RESTful APIs, microservices, and distributed applications using technologies such as Java/J2EE, Spring Boot, Kotlin, or Python.
• Working knowledge of multithreading, concurrency, application performance tuning, and troubleshooting.
• Experience with CI/CD tools and workflows such as Jenkins, Maven, Git, or similar technologies.
• Experience with relational or NoSQL databases such as DB2, PostgreSQL, Redis, or Couchbase.
• Hands-on experience with application observability, including OpenTelemetry, metrics, logs, distributed tracing, and monitoring platforms such as Prometheus, Grafana, or Splunk.
• Familiarity with resilience engineering practices such as NALSD, failure mode analysis, chaos engineering, SLOs, SLIs, and error budgets.
• Experience investigating production issues, performing root cause analysis, and implementing corrective actions.
• Familiarity with AI-assisted software development, including SDD, prompt engineering, and AI-supported coding, testing, and debugging.
• Understanding of Generative AI concepts such as LLMs, RAG, context engineering, and agentic orchestration using LangGraph or similar frameworks.
• Awareness of AI evaluation, safety, and observability concepts, including tools such as Promptfoo, NeMo Guardrails, and AI tracing frameworks.
• Ability to independently deliver moderately complex engineering tasks and collaborate effectively with cross-functional teams.
• Strong analytical, problem-solving, and communication skills.