Notes on Distributed System

Terms

• FLP Impossibility, Impossibility of Distributed Consensus with One Faulty Process
• CAP Theorem
• BASE: An Acid Alternative
• Fallacies of Distributed Computing
• Distributed systems theory for the distributed engineer
  Chinese version: 面向分布式系统工程师的分布式系统理论

Books

• Distributed Systems for fun and profit
• Distributed Systems Concepts and Design(Fifth Edition) - George Coulouris
• Distributed Systems Principles and Paradigms - Andrew Tanenbaum
• 走向分布式

Courses

• Cloud Computing Concepts, University of Illinois
• Distributed Systems - CMU fall2016 in Go Programming Language
• Principles of Distributed Computing - ETHZ
• Distributed Systems for Information Systems Management - CMU
• Distributed Systems - MIT
• Introduction to Distributed System Design - Google Code University

Papers & Implementations

Storage

Papers

• The Google File System
• MapReduce: Simplified Data Processing on Large Clusters
• Bigtable: A Distributed Storage System for Structured Data
• Dynamo: Amazon’s Highly Available Key Value Store
• Cassandra: A Decentralized Structured Storage System
• Availability in Globally Distributed Storage Systems
• Ceph: A Scalable, High-Performance Distributed File System
• Sheepdog: Distributed Storage System for KVM
• PacificA: Replication in Log-Based Distributed Storage Systems
• Finding a needle in Haystack: Facebook’s photo storage
• Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency

Implementations

• Ceph
• Sheepdog

Databases

• Spanner: Google’s Globally-Distributed Database
• Tera - 高性能、可伸缩的结构化数据库
• Salt: Combining ACID and BASE in a Distributed Database

Resource Management

• Apache Hadoop YARN: Yet Another Resource Negotiator
• Mesos: A Platform for Fine-Grained Resource Sharing in the Data Centeri
• Borg, Kubernetes an open-source implementation of Borg by Google

  Data Processing

• MapReduce: Simplified Data Processing on Large Clusters
• Pregel: A System for Large-Scale Graph Processing

Query Processing

• Design and Implementation of a Query Processor for a Trusted Distributed Data Base Management System

Messaging Systems

• The Log: What every software engineer should know about real-time data’s unifying abstraction
  Chinese version 日志：每个软件工程师都应该知道的有关实时数据的统一概念
• Kafka: a Distributed Messaging System for Log Processin

Consensus Algorithms

Papers

=======

Distributed Consensus Algorithms

• The Part Time Parliament, Lamport’s original Paxos paper, but it’s a bit difficult to understand.
• Paxos Made Simple, a more terse readable Paxos paper by Lamport himself.
• The Chubby Lock Service for loosely coupled distributed systems, an Implementation of Fast-Paxos
• Paxos made live - An engineering perspective
• Using Paxos to Build a Scalable, Consistent, and Highly Available Datastore
• Raft Consensus Algorithm
• Paxos vs Raft

Implementations

• Apache ZooKeeper, ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services, an opensource Implementation of Google Chubby
• etcd, Distributed reliable key-value store for the most critical data of a distributed system based on Raft

Others

=======

Service Discovery

• ZooKeeper, paper Zab
• Eureka!
• Consul
• etcd

Consistency and Ordering in Distributed Systems

• Time, Clocks, and the Ordering of Events in a Distributed System
• The Byzantine Generals Problem

Distributed Hash Tables

• Consistent hashing and random trees
• Web caching with consistent hashing
• Chord
• Pastry
• Tapestry

Replication in Distributed Databases

• Notes on Distributed Databases
• Epidemic algorithms for replicated database maintenance
• Principles of Computer System Design, Chapter 10: Consistency

Blogs and other reading links

• What are some good resources for learning about distributed computing? Why?

reference

• awesome-distributed-systems
• service-discovery
• flp
• CAP理论
• THE BASE METHODOLOGY VERSUS THE ACID MODEL

Related Posts

• 06 Jul 2017 » Notes on Stroage
• 23 May 2017 » Ceph RBD 块设备的快照机制
• 17 May 2017 » Hello World