Checklists

Practical checklists for common SRE activities. Print them, adapt them, and use them to ensure nothing gets missed.

Production Readiness Checklist

Use this before launching any new service:

Monitoring & Observability

  • Golden signals instrumented (latency, traffic, errors, saturation)
  • Application logs structured and centralized
  • Distributed tracing implemented
  • Dashboards created for key metrics
  • SLIs defined and measured
  • SLOs documented with error budgets

Alerting

  • Critical alerts configured and tested
  • Alert runbooks written
  • On-call rotation assigned
  • Alert thresholds validated (not too noisy)
  • Escalation procedures documented
  • Test alert sent and verified

Reliability & Resilience

  • Load tested at 2x expected peak traffic
  • Failure modes identified and documented
  • Circuit breakers implemented for dependencies
  • Graceful degradation tested
  • Rate limiting configured
  • Retry logic with exponential backoff
  • Timeout policies set on all external calls

Deployment & Rollback

  • Automated deployment pipeline tested
  • Canary deployment strategy implemented
  • Rollback procedure tested successfully
  • Feature flags for major changes
  • Database migrations are reversible
  • Zero-downtime deployment verified

Capacity & Scaling

  • Autoscaling configured and tested
  • Resource limits set (CPU, memory, connections)
  • Capacity plan documented for next 6 months
  • Database connection pooling configured
  • Caching strategy implemented

Security

  • Authentication and authorization implemented
  • Secrets managed securely (not in code)
  • Input validation on all endpoints
  • Rate limiting to prevent abuse
  • Security scan passed
  • Least privilege access configured

Documentation

  • Architecture diagram up to date
  • API documentation complete
  • Runbooks for common operations
  • Disaster recovery plan documented
  • Dependencies mapped
  • Contact information for dependencies

Disaster Recovery

  • Backup strategy implemented and tested
  • Recovery Time Objective (RTO) defined
  • Recovery Point Objective (RPO) defined
  • DR drill completed successfully
  • Backup restoration tested

Weekly SRE Team Health Check

Run this with your team every week:

Toil Assessment

  • Tracked time spent on toil this week
  • Toil percentage: ____% (target: <50%)
  • Top 3 toil sources identified
  • Action items to reduce toil created

Incident Review

  • All incidents from last week have postmortems
  • Action items from postmortems assigned
  • Overdue action items escalated
  • Patterns across incidents identified

On-Call Health

  • On-call load acceptable (< 2 pages/shift average)
  • All pages were actionable and necessary
  • Runbooks updated based on this week’s pages
  • On-call engineers have time to recover

SLO Compliance

  • All services within error budget
  • Services approaching budget limits flagged
  • Error budget policy followed for decisions
  • SLO dashboard reviewed with team

Team Morale

  • Team members feel supported
  • Workload is sustainable
  • Learning opportunities available
  • Work-life balance maintained

New SRE Onboarding Checklist

Week 1: Foundation

  • Access to all necessary systems granted
  • Laptop setup and tooling installed
  • Meet the team (1:1s scheduled)
  • Read last 10 incident postmortems
  • Review service architecture diagrams
  • Shadow on-call engineer for a shift
  • Join incident response as observer

Week 2-4: Learning

  • Complete production access training
  • Deploy a test change to production
  • Write or update a runbook
  • Participate in a postmortem as scribe
  • Shadow 3+ incident responses
  • Review SLOs for all critical services
  • Complete security training

Week 5-8: Contributing

  • Join on-call rotation as secondary
  • Lead a postmortem
  • Complete a toil reduction project
  • Update or create monitoring dashboard
  • Improve alerting for one service
  • Present learning to team

Month 3+: Independent

  • On-call as primary (with backup)
  • Own a service or system area
  • Propose and execute SRE improvement
  • Mentor next new hire
  • Contribute to SRE strategy

Change Review Checklist

Use this for production changes:

Before Deployment

  • Code reviewed by at least 2 people
  • All tests passing
  • Deployment plan documented
  • Rollback plan documented and tested
  • Monitoring confirms current system health
  • Error budget status checked
  • High-risk change identified (if applicable)
  • Stakeholders notified of change window

During Deployment

  • Monitoring dashboard open
  • Communication channel active
  • Canary deployed and healthy
  • Gradual rollout (10% → 50% → 100%)
  • Metrics watched for anomalies
  • Ready to rollback if needed

After Deployment

  • All metrics within normal ranges
  • Error rates unchanged or improved
  • Latency unchanged or improved
  • No new alerts fired
  • Deployment marked successful
  • Change documented in change log