Designing Distributed Systems - Brendan Burns (англ.)

Designing Distributed Systems Брендана Бернса - практическое руководство по проектированию масштабируемых и надежных распределенных систем в облачной среде. Автор, один из создателей Kubernetes, систематизирует ключевые паттерны и архитектурные подходы, используемые в cloud-native разработке.

В книге подробно рассматривается, как с помощью контейнеров и оркестраторов строить отказоустойчивые сервисы, повторно использовать архитектурные шаблоны и эффективно управлять сложными системами. Особое внимание уделяется паттернам sidecar, ambassador и adapter, а также масштабированию, асинхронной обработке, очередям задач и event-driven архитектурам.

Издание ориентировано на backend-разработчиков, DevOps-инженеров, архитекторов и технических лидеров, которые работают с микросервисами, Kubernetes и распределенными системами. Книга помогает выстроить общее архитектурное мышление в команде и служит практическим справочником при создании надежных сервисов в продакшене.

Designing Distributed Systems - Table of Contents 

Preface

---

1. Introduction

• A Brief History of Systems Development

• A Brief History of Patterns in Software Development

• Formalization of Algorithmic Programming

• Patterns for Object-Oriented Programming

• The Rise of Open Source Software

• The Value of Patterns, Practices, and Components

• Standing on the Shoulders of Giants

• A Shared Language for Discussing Our Practice

• Shared Components for Easy Reuse

• Summary

---

Part I. Single-Node Patterns

2. The Sidecar Pattern

• An Example Sidecar: Adding HTTPS to a Legacy Service

• Dynamic Configuration with Sidecars

• Modular Application Containers

• Hands On: Deploying the topz Container

• Building a Simple PaaS with Sidecars

• Designing Sidecars for Modularity and Reusability

• Parameterized Containers

• Define Each Container’s API

• Documenting Your Containers

• Summary

3. Ambassadors

• Using an Ambassador to Shard a Service

• Hands On: Implementing a Sharded Redis

• Using an Ambassador for Service Brokering

• Using an Ambassador to Do Experimentation or Request Splitting

• Hands On: Implementing 10% Experiments

4. Adapters

• Monitoring

• Hands On: Using Prometheus for Monitoring

• Logging

• Hands On: Normalizing Different Logging Formats with Fluentd

• Adding a Health Monitor

• Hands On: Adding Rich Health Monitoring for MySQL

---

Part II. Serving Patterns

5. Replicated Load-Balanced Services

• Stateless Services

• Readiness Probes for Load Balancing

• Hands On: Creating a Replicated Service in Kubernetes

• Session Tracked Services

• Application-Layer Replicated Services

• Introducing a Caching Layer

• Deploying Your Cache

• Hands On: Deploying the Caching Layer

• Expanding the Caching Layer

• Rate Limiting and Denial-of-Service Defense

• SSL Termination

• Hands On: Deploying nginx and SSL Termination

• Summary

6. Sharded Services

• Sharded Caching

• Why You Might Need a Sharded Cache

• The Role of the Cache in System Performance

• Replicated, Sharded Caches

• Hands On: Deploying an Ambassador and Memcache for a Sharded Cache

• An Examination of Sharding Functions

• Selecting a Key

• Consistent Hashing Functions

• Hands On: Building a Consistent HTTP Sharding Proxy

• Sharded, Replicated Serving

• Hot Sharding Systems

7. Scatter/Gather

• Scatter/Gather with Root Distribution

• Hands On: Distributed Document Search

• Scatter/Gather with Leaf Sharding

• Hands On: Sharded Document Search

• Choosing the Right Number of Leaves

• Scaling Scatter/Gather for Reliability and Scale

8. Functions and Event-Driven Processing

• Determining When FaaS Makes Sense

• The Benefits of FaaS

• The Challenges of FaaS

• The Need for Background Processing

• The Need to Hold Data in Memory

• The Costs of Sustained Request-Based Processing

• Patterns for FaaS

• The Decorator Pattern: Request or Response Transformation

• Hands On: Adding Request Defaulting Prior to Request Processing

• Handling Events

• Hands On: Implementing Two-Factor Authentication

• Event-Based Pipelines

• Hands On: Implementing a Pipeline for New-User Signup

9. Ownership Election

• Determining If You Even Need Master Election

• The Basics of Master Election

• Hands On: Deploying etcd

• Implementing Locks

• Hands On: Implementing Locks in etcd

• Implementing Ownership

• Hands On: Implementing Leases in etcd

• Handling Concurrent Data Manipulation

---

Part III. Batch Computational Patterns

10. Work Queue Systems

• A Generic Work Queue System

• The Source Container Interface

• The Worker Container Interface

• The Shared Work Queue Infrastructure

• Hands On: Implementing a Video Thumbnailer

• Dynamic Scaling of the Workers

• The Multi-Worker Pattern

11. Event-Driven Batch Processing

• Patterns of Event-Driven Processing

  • Copier

  • Filter

  • Splitter

  • Sharder

  • Merger

• Hands On: Building an Event-Driven Flow for New User Sign-Up

• Publisher/Subscriber Infrastructure

• Hands On: Deploying Kafka

12. Coordinated Batch Processing

• Join (or Barrier Synchronization)

• Reduce

  • Hands On: Count

  • Sum

  • Histogram

• Hands On: An Image Tagging and Processing Pipeline

13. Conclusion: A New Beginning?