The Journey to Amazon DynamoDB: From Scaling by Architecture to Scaling by Commandment

One of the lessons Amazon has learned from operating Amazon’s platform is that the reliability and scalability of a system is dependent on how its application state is managed. To run applications at massive scale, requires one to operate datastores that can scale to operate seamlessly across thousands of servers and can deal with various failure modes such as server failures, datacenter failures and network partitions.

In this talk, we will talk about some of the common challenges in running datastores at high scale. We will discuss our journey of scalability, and discuss potential solutions for these challenges and how different datastores (both relational and non-relational) address them with some real customer case studies. Most importantly, we will discuss our journey from Dynamo to DynamoDB.

Swami Sivasubramanian works as a General Manager and Architect for NoSQL Database services in Amazon Web Services where he builds large scale cloud computing platforms and also manages different groups within AWS Database services. He manages the Amazon DynamoDB, Elasticache, SQS, SNS and SimpleDB services. Swami has built several large scale systems in the past. Some of the well-known ones (that is externally visible) include Amazon Dynamo, Amazon CloudFront and Amazon RDS. He has also built other large scale systems that is used for building Amazon service infrastructure. Swami obtained his Ph.D. from Vrije Universiteit, Amsterdam from the Computer Systems Group headed by Andrew S. Tanenbaum and Maarten van Steen. Swami has authored more than 40 refereed journals and conference papers. He also holds more than 10 patents and has more than 60 pending patent applications. He also serves in program committees of different ACM/USENIX/IEEE conferences, and also as guest editors for different IEEE/ACM magazines.