This presentation introduces Aurora DSQL, Amazon Web Services' newest relational SQL database offering, optimized for transactional workloads. The speaker details its core features, including serverless operation, high scalability both up and down, and active-active multi-region support with strong consistency and symmetric latency, built upon AWS's decade-plus experience with services like Aurora and DynamoDB. A significant focus is placed on its Postgres compatibility, allowing developers to leverage existing code while benefiting from simplified, high-availability architectures. The session also provides crucial scalability tips, such as avoiding hot write keys, and delves into DSQL's strong snapshot isolation model and its linearizability, highlighting how these design choices simplify application development by offloading complex distributed system challenges to the database service.
Summarizing Aurora DSQL: Architectures, Scalability, and Consistency
0:30 Aurora DSQL Overview: This talk covers getting started with Aurora DSQL, architectural patterns, its design principles, and its isolation and consistency model.
1:14 Genesis of Aurora DSQL: The speaker's journey from the Lambda team revealed customer demand for serverless-compatible relational databases and multi-region active-active databases, leading to the four-year development of DSQL.
A relational SQL database optimized for transactional workloads (e.g., microservices, mobile backends), not analytics.
Scalable: Designed to scale from a few transactions a day to millions per second.
Serverless: No visible infrastructure to manage (patching, monitoring, compliance are handled by AWS).
Multi-Region Active-Active: Offers optional active-active support across multiple regions without compromising ACID properties, crucial for critical applications.
Postgres Compatible: Connects with Postgres clients and supports most existing SQL code, emphasizing ease of migration for "crown jewel" business logic.
Experienced Foundation: Built on a decade of AWS's experience running large-scale services like Aurora (launched 2014) and DynamoDB.
Even a simple setup (DNS, ALB, single app instance) can achieve hundreds to thousands of requests/sec with 99.9% availability and no data loss, showcasing AWS's goal to simplify user architectures.
A Lambda function with DSQL can be the simplest multi-thousand requests/sec architecture, offering multi-AZ resilience and scalability without infrastructure management.
A multi-AZ application with DSQL provides 99.95% availability, surviving entire Availability Zone failures with minimal customer impact.
15:10 Scalability Tip 1: Avoid Hot Write Keys: Concurrent transactions attempting to update the same key (e.g., a single counter) will lead to optimistic concurrency control (OCC) errors, requiring application-level retries. Schema design should aim to spread updates across multiple keys. Reads, however, never conflict.
21:43 Scalability Tip 2: Avoid Hot Key Ranges: Unlike traditional databases that favor data locality, DSQL performs optimally with workload dispersion across the key space, especially for writes. Using strategies like Version 7 UUIDs for primary keys can help naturally spread out writes.
23:54 Multi-Region Active-Active Applications: This architecture provides benefits for latency (local clients) and critical availability/compliance needs. It's achieved by simply deploying the multi-AZ application architecture in multiple regions and connecting them via a DSQL linked cluster.
AZ Failure: DSQL ensures continuous availability within milliseconds in the remaining AZs, with no data or consistency loss.
Region Failure: Unlike traditional manual failovers that incur minutes/hours of downtime, DSQL's active-active design can achieve recovery within milliseconds, remaining strongly consistent and preserving all committed transactions.
30:00 Why Prefer Active-Active? Active-active architectures are preferred over active-failover due to continuous operation (caches warm, easier monitoring), simpler capacity planning, and the ability to route customers to the closest region for symmetric, low-latency reads and writes.
Reads and writes in DSQL are always processed locally within the region, ensuring fast operations and avoiding cross-continent/world latency.
This is enabled by AWS's multi-region clock architecture (EC2 time service).
35:58 The Cost of Strong Consistency: Commit Time Latency: While reads and writes are local, commits incur a cross-region penalty (typically 15-100ms) to ensure synchronous durability across regions.
Transactions Improve Performance: Writing multi-statement transactions amortizes this commit cost, leading to lower latency and higher throughput compared to auto-commit. Read-only transactions are always fast and local, even on commit.
DSQL currently offers Strong Snapshot Isolation, a strong isolation level where each transaction commits atomically at a specific time and sees a consistent snapshot of the database at its start time.
It is Linearizable: Every transaction appears to happen atomically between the start and end of its commit, even across regions, ensuring immediate visibility of writes.
This combination simplifies programming by abstracting the distributed nature of the system, addressing the challenge developers face with eventual consistency.
47:52 Isolation Level Comparison: DSQL's Strong Snapshot Isolation is stronger than MySQL/Postgres READ COMMITTED and MySQL REPEATABLE READ, and equivalent to Postgres' REPEATABLE READ mode (which uses snapshots).
50:09 Why Strong Snapshot Isolation? It strikes a "sweet spot" by providing strong consistency (simplifying application logic) without the performance downsides and application complexity of serializable isolation. DSQL's architecture (MVCC, hardware clocks, reduced coordination) also allows for significant isolation benefits "for free."
53:32 Key Takeaways: Simple architectures can achieve great scalability, availability, and durability with cloud databases like DSQL. Active-active is a recommended default for multi-region designs. Avoiding hot write keys is crucial for high scalability. Using transactions can improve performance, latency, and throughput.
56:33 Next Steps: Explore DSQL documentation, attend related re:Invent talks (internals, hands-on workshop, chalk talk), and provide feedback during the preview phase to shape the General Availability product.