5 database announcements that stole the show at AWS re:Invent
Each year, everyone who attends Amazon’s re:Invent walks away with a different set of “greatest hits”– announcements they think are going to be important, impactful changes. There’s a lot to choose from–containers, machine learning, serverless computing, a number of new instance types, virtual reality, even hardware.
After listening to the first full day’s rollouts, I think the biggest impacts are going to come from something that sounds less sexy: databases. Unlike some of the more avant-garde, trendy rollouts in new areas, the database offerings underpin most of the systems you need to serve your customers today.
Several years ago, software development writer Martin Fowler wrote about the shift away from a one-size fits all data storage model, where everything went into a relational database, to “ployglot persistence” in which organizations have a rich portfolio of options, and can choose different tools for different problems. You can certainly store a graph in a relational database, but it may not be the best or most performant way.
AWS has embraced this view with gusto, talking about giving builders their choice of purpose-built tools tuned for different uses, including relational, NoSQL, and now a graph DB, all within the same platform and ecosystem.
At re:Invent yesterday, the company rolled out significant improvements to many of these products. Here are five that caught my eye:
The AWS Aurora relational database gives organizations a way to run performant, cost-effective, resilient databases on the platform. I saw two impressive changes here:
1. Multi-master replication
AWS Aurora has supported Read Replicas for some time now; this gave you failover support and the ability to “scale out” horizontally by launching up to 15 read-only replicas of your database. AWS is taking this to the next level by supporting writeable, multi-master support, which dramatically improves the resiliency and flexibility of your platforms. The single region, multi-AZ version of this is available today.
The upgrade opens the door to use Aurora in some applications where you may have been considering Oracle RAC. Both are designed to support resiliency and performance for relational databases at scale. Aurora may be simpler and more cost effective for organizations moving forward, opening a new front in the battle between Oracle and AWS for cloud-scale relational databases.
2. Aurora Serverless
The new serverless mode for Aurora promises a database that will spin up when you hit it, scales dynamically based on load without your intervention, and then shuts down when there’s no load. Because you pay by the second for the capacity you’re actually using, this Aurora Serverless a great solution for those applications you may only use intermittently.
For example, I have a system that automatically tracks and loads incoming datasets. It already has a lambda function that fires when data arrives in S3. This will allow it to seamlessly connect to the database, which will be provisioned as it’s hit, log the metadata in the database, and then let that database shut down until I need to access it again.
Dynamo has always been one of AWS’s favorite things to talk about, with good reason. A fully managed NoSQL solution that can scale well if you use it properly, it is heavily used in Amazon itself and in a host of “born-in-the-cloud” applications. Dynamo has received steady investment from AWS, including capabilities like the DAX accelerator, released earlier this year. This week, Amazon unveiled two new features customers have been asking for.
1. DynamoDB Global Tables
Global Tables gives organizations multi-master database support across regions. In the past, customers could use Lambda and Dynamo Streams to address some of the need for multi-region, but this appears to be a dramatically simpler solution.
2. Dynamo Backup and Restore
One thing that always gave me pause about Dynamo was the inability to back up the database. I began my career as an Oracle DBA, and the discipline of backing up the database and testing my ability to restore that database was like breathing. A database you couldn’t back up and restore seemed like an odd thing indeed. While AWS previously provided resiliency with redundant copies of the data to protect against hardware issues, it didn’t protect you from application logic errors that damaged the data, or provide the kind of long-term backup and storage many customers need.
DynamoDB Backup provides on-demand backups for long-term storage, and promises to roll out point-in-time recovery in 2018, all without compromising the performance of the database at scale. This will improve many customers’ comfort level with the product, and address legitimate needs for the kind of backup and restore semantics we’re used to having in our databases.
AWS launched a preview this week of Neptune, its new graph database service. A graph database is designed to model the connections between things–an increasingly important space for many modern applications. Platforms like LinkedIn or Facebook are driven by the interconnectedness of people and subjects, and many internal applications are being built to model similar relationships between users and products, products and capabilities, compatibility information, and more.
A graph DB platform such as Neptune offers a more flexible, performant way of modelling and interacting with this kind of data than rolling your own support using a traditional relational database or Dynamo. It understands that you’re modeling these kinds of connections between entities.
The Neptune platform feels a bit like an Aurora for graph databases, with support for industry standards to access the data, support for up to 15 read-replicas, and more. I’m excited to be able to kick the tires on this platform, and see what we can build.
For more information about these AWS database updates, contact your account executive. And stay tuned to the blog for further updates from AWS re:Invent.