Founding the Program

Embarking on the journey to establish a Site Reliability Engineering (SRE) program in an environment devoid of executive sponsorship or a dedicated budget demands a blend of ingenuity, determination, and strategic foresight. Throughout this chapter, you will discover pragmatic strategies to lay the groundwork for SRE in settings where resources are limited.

Start Where You Are

In your quest to enhance reliability, you don’t need anyone’s permission to begin. Take the initiative to track metrics related to system reliability, refine your runbooks, conduct informal postmortems, automate repetitive tasks, and share your findings with your team. These actions speak volumes. By demonstrating value through tangible efforts rather than mere proposals, you set the stage for future success.

The Grassroots Approach

1. Pick One High-Impact Problem

Resist the temptation to solve every issue at once. Instead, focus on one significant and visible problem. This could be frequent pager alerts for the same issue, a manual deployment process causing outages, a lack of visibility into production health, or slow incident response due to inadequate documentation. By concentrating on a single pain point, you can make meaningful progress.

2. Fix It Measurably

Begin by documenting the current state of the problem, including the time spent, frequency, and impact. Implement a solution and measure the improvements. Once you have concrete results, share them widely to highlight the success. For instance, consider the example of automating a database backup process. Previously, this task consumed two hours of manual work per week, but with automation, it now requires none, freeing up over 100 hours annually.

3. Build Momentum

Success is a powerful catalyst for support. After securing your first victory, it’s time to address another high-impact issue. Involve others in crafting solutions, document your approach for easy replication, and present your findings to leadership. This momentum can drive further buy-in and resource allocation for your SRE initiatives.

Getting Organizational Buy-In

Speak the Language of Business

Technical excellence alone rarely garners attention. To secure organizational buy-in, frame SRE concepts in business terms. Instead of stating, “We need better observability,” articulate it as, “We’re losing $X per incident due to slow detection. Improved monitoring could reduce our Mean Time to Recovery (MTTR) by 50%.” Similarly, replace “We should implement SLOs” with “We can’t prioritize reliability work without clear targets. Service Level Objectives (SLOs) provide an objective means to balance features and stability.”

Build Relationships

SRE cannot thrive in isolation. Forge cross-functional relationships to demonstrate the benefits of SRE to various stakeholders. Show engineering teams how SRE simplifies their tasks, highlight the reliability impact on user experience to product managers, quantify the cost of incidents and automation return on investment for finance teams, and present reliability as a competitive advantage to leadership.

Creative Tactics

When traditional budget requests fall short, employ creative tactics to advance your SRE goals:

Team Dinners and Informal Talks

Host brown-bag lunch sessions centered on SRE topics. Share postmortems and insights over team dinners to foster a community focused on reliability.

Borrow Time from Existing Projects

Integrate reliability improvements into ongoing feature work, treating SRE efforts as “infrastructure” or “tech debt.” Build automation alongside new features to maximize efficiency.

Leverage Open Source Tools

Utilize free tiers of commercial tools like PagerDuty and Datadog, deploy self-hosted open source alternatives, and construct internal tools only when necessary.

Start with Volunteers

Identify engineers with a keen interest in reliability, form an informal working group, and hold regular meetings to discuss practices and share tools.

Defining Your Mission

A well-defined mission guides your efforts and communicates value effectively. Here are some example SRE mission statements:

Example SRE Mission Statements

For a startup, the mission might be, “Ensure customer-facing services maintain 99.9% uptime while enabling rapid feature development.” In an enterprise setting, it could be, “Reduce incident impact on business operations by 50% through automation, monitoring, and cultural change.” A platform team’s mission may focus on providing “reliable, self-service infrastructure that enables product teams to deploy confidently.”

Key Elements

• Measurable outcomes: Set specific reliability targets.
• Stakeholder alignment: Clearly demonstrate value to the business.
• Cultural component: Outline how you’ll transform work practices.
• Scope boundaries: Define what’s included and what’s outside your purview.

Common Obstacles and Solutions

“We don’t have budget”

Start with free or open source tools and demonstrate return on investment through pilot projects. Quantify the cost of incidents to justify further investment.

“We don’t have headcount”

Begin by allocating 20% of time from interested engineers, proving value before seeking dedicated roles. Rotate SRE responsibilities across teams to manage workload.

“Leadership doesn’t see the value”

Track and share the costs of incidents, including downtime, engineering hours, and customer impact. Present case studies from similar companies and start small, letting results speak for themselves.

“Our culture resists change”

Identify allies and champions in other teams. Make SRE practices optional yet appealing, and lead by example rather than enforcing mandates.

Measuring Early Success

To validate your efforts, track key metrics such as incident frequency, MTTR, toil reduction, deploy frequency, and team satisfaction. These indicators help prove the value of your SRE initiatives.

Timeline for Bootstrapping

Month 1-3: Foundation

Begin by identifying a critical reliability problem, implementing a solution, and measuring its impact. Document your approach and share the results with stakeholders to build credibility.

Month 4-6: Expansion

Tackle 2-3 additional high-impact issues, recruit volunteers for an SRE working group, establish basic on-call and incident response practices, and start tracking Service Level Indicators (SLIs) for critical services.

Month 7-12: Formalization

Present a business case for formal SRE investment, define team structure and responsibilities, implement a comprehensive SLO framework, and develop self-service tools for common needs.

Next, we’ll explore how to hire and build your SRE team as the program matures.