Role Overview
The Principal Site Reliability Engineer will be a senior technical expert responsible for driving
end-to-end resilience, reliability, and scalability
across our mission-critical payments platform. This role focuses on
front-to-back payment flows
, ensuring systems are designed for
fault tolerance, observability, and operational excellence
.
You will perform deep technical reviews, troubleshoot complex issues, and define patterns for
resiliency by design
. As a hands-on engineer, you will collaborate with development and production support teams, advocate
chaos engineering
, and build a culture of
designing for failure
. This position requires strong technical breadth across
infrastructure, applications, networks, databases, and integrations
, combined with expertise in modern reliability engineering practices.
Key Responsibilities
- Reliability Engineering Leadership:
- Drive strategies to improve reliability, maintainability, and scalability across payment flows and platform components.
- Architecture & Design Reviews:
- Conduct deep technical assessments of system architectures, identifying risks and recommending improvements for fault tolerance and disaster recovery.
- Incident Management & Root Cause Analysis:
- Act as a senior escalation point for production incidents, lead RCA, and implement permanent fixes to prevent recurrence.
- Define and enforce reliability patterns, frameworks, and best practices; ensure adoption across engineering teams.
- Chaos Engineering & Failure Testing:
- Advocate and implement chaos engineering principles to validate system resilience under real-world failure scenarios.
- Observability & Monitoring:
- Design and implement full-stack observability solutions, including metrics, logging, distributed tracing, and alerting.
- Develop automation for failover, capacity management, and self-healing mechanisms to reduce operational risk.
- Partner with development, infrastructure, and production support teams to embed reliability into the SDLC.
- Analyze service risk assessments and production incidents to identify systemic issues and drive long-term improvements.
- Promote operational excellence and a mindset of designing for failure across all engineering teams.
Required Skills & Experience
- 12+ years in software engineering or infrastructure roles, with at least 5 years focused on reliability engineering or SRE.
- Proven experience building and operating fault-tolerant, highly available systems at scale.
- Strong knowledge of distributed systems, resiliency patterns (circuit breakers, retries, failover), and disaster recovery strategies.
- Expertise across infrastructure (compute, storage, networking), application architecture, databases, and integration patterns.
- Ability to troubleshoot complex technical issues across distributed systems and perform deep root cause analysis.
- Collaboration & Influence:
- Skilled at working with development, operations, and architecture teams to embed reliability into design and delivery.
Purpose of the role
To drive technical excellence and innovation by leading the design and implementation of robust software solutions, providing mentorship to engineering teams, fostering cross-functional collaboration, and contributing to strategic planning to ensure the delivery of high-quality solutions aligned with business objectives.
Accountabilities
- Provision of guidance and expertise to engineering teams to ensure alignment with best practices and foster a culture of technical excellence.
- Contribution to strategic planning by aligning technical decisions with business goals, anticipating future technology trends, and providing insights to optimize product roadmaps.
- Design and implementation of complex, scalable, and maintainable software solutions, considering long-term viability and business objectives.
- Mentoring and coaching to junior and mid-level engineers to foster professional growth and knowledge sharing, elevating the overall skillset and capabilities of the organization.
- Collaboration with business partners, product managers, designers, and other stakeholders to translate business requirements into technical solutions and ensure a cohesive approach to product development.
- Innovation within the organization by identifying and incorporating new technologies, methodologies, and industry practices into the engineering process.
Director Expectations
- To manage a business function, providing significant input to function wide strategic initiatives. Contribute to and influence policy and procedures for the function and plan, manage and consult on multiple complex and critical strategic projects, which may be business wide..
- They manage the direction of a large team or sub-function, leading other people managers and embedding a performance culture aligned to the values of the business. Or for an individual contributor, they lead organisation wide projects and act as deep technical expert and thought leader, identifying new ways of working and collaborating cross functionally. They will train, guide and coach less experienced specialists and provide information affecting long term profits, organisational risks and strategic decisions..
- Provide expert advice to senior functional management and committees to influence decisions made outside of own function, offering significant input to function wide strategic initiatives.
- Manage, coordinate and enable resourcing, budgeting and policy creation for a significant sub-function.
- Escalates breaches of policies / procedure appropriately.
- Foster and guide compliance, ensure regulations are observed that relevant processes in place to facilitate adherence.
- Focus on the external environment, regulators, or advocacy groups to both monitor and influence on behalf of Barclays, when appropriate.
- Demonstrate extensive knowledge of how the function integrates with the business division / Group to achieve the overall business objectives.
- Maintain broad and comprehensive knowledge of industry theories and practices within own discipline alongside up-to-date relevant sector / functional knowledge, and insight into external market developments / initiatives.
- Use interpretative thinking and advanced analytical skills to solve problems and design solutions in often complex/ sensitive situations.
- Exercise management authority to make significant decisions and certain strategic decisions or recommendations within own area.
- Negotiate with and influence stakeholders at a senior level both internally and externally.
- Act as principal contact point for key clients and counterparts in other functions/ businesses divisions.
- Mandated as a spokesperson for the function and business division.
All Senior Leaders are expected to demonstrate a clear set of leadership behaviours to create an environment for colleagues to thrive and deliver to a consistently excellent standard. The four LEAD behaviours are: L – Listen and be authentic, E – Energise and inspire, A – Align across the enterprise, D – Develop others.
All colleagues will be expected to demonstrate the Barclays Values of Respect, Integrity, Service, Excellence and Stewardship – our moral compass, helping us do what we believe is right. They will also be expected to demonstrate the Barclays Mindset – to Empower, Challenge and Drive – the operating manual for how we behave.