My top Picks from re:Invent 2020

This year the AWS re:Invent time was virtual and free, but as every year, developers, architects, business leaders, and everyone in between are faced with the daunting task of selecting from thousands of hours of re:Invent content.

So I’ve picked out and reviewed the announcements that look the most interesting to me, from compute, database & storage to networking, machine learning and development. Hopefully you’ll find my suggestions useful!

1.Compute

On the very first day of the conference, Amazon announced EC2 Mac instances for macOS, adding after many years a new operating system to EC2. This is mainly targeted at processes that only run on Mac OS, like building and testing applications for iOS, MacOS, tvOS and Safari. The first part of Andy Jassy’s keynote was focused on announcements related to compute options and serverless technologies. AWS introduced new instance types on different processors and EC2 families, including Intel Xeon M5zn instances, Graviton2-powered C6gn instances, Intel-powered D3/D3en instances, memory-optimized R5b instances and AMD-powered G4ad GPU instances. See InfoQ’s coverage here.

My comments: The ability to run Mac instances on the public cloud is exciting. Best benefit will be for software companies building iOS applications, which can now be tested directly on AWS at any time, without having to buy and maintain dedicated hardware. AWS gives a big advantage to cross-platform development here. But also for the individual developer and technology enthusiast, this is a cost-effective way to try out how your scripts and programs would run on Mac platform.

Lambda and serverless

There were announcements about Lambda and serverless deployments: billing granularity is reduced from 100ms to 1ms which reduces costs for every single Lambda function automatically, the availability of functions with up to 10 GB of memory and 6 vCPUs was added. One more new feature is the support for container images as a packaging format, to simplify the transition from current container-based workloads to serverless functions. To know more about AWS Lambda updates, see InfoQ’s article.

My comments: The reduced billing granularity for Lambda is impressive cost reduction, some customers use a lot of small and simple Lambda functions that complete in a few milliseconds. Before, even for the smallest function, the minimum cost of 100ms had to be paid. Now when only the actual number of ms is billed, these customers see massive reductions in their Lambda costs

Adding Amazon ECS Anywhere and Amazon EKS Anywhere

AWS will make the container orchestration software used in ECS and EKS freely available to deployments outside AWS, including other cloud providers. This will provide increased integration and lower latency and will follow the path of Microsoft and Google that already offer Azure AKS and Google Anthos for free.

My comments: These releases take AWS to the same level as other major public cloud providers when it comes to managed Kubernetes offering. As EKS can now be run outside AWS, it will make customers more ready to adopt this platform, because they don’t have to worry about vendor lock-in anymore. A potentially even more attractive offering is ECS Anywhere; with ECS the customers get the powerful ways of container orchestration without the complexity of Kubernetes. Before, ECS could only run on AWS which limited its adoption, but now many more customers will consider it as an option.

AWS Proton

During the first keynote, the public preview of AWS Proton, a new managed deployment service for container and serverless applications, was announced. With AWS Proton, customers can automate and manage infrastructure provisioning and code deployments for serverless and container-based applications. See InfoQ’s coverage here. ECR Public Repositories is a public container registry to store, manage, share, and deploy container images globally.

My comments: ECR Public Repositories make it possible to easily share your containerized services to the public. This is a good way to share open-sourced software back to the community so that everyone can benefit.

2. Storage

There were three major announcements for EBS, the block-storage service designed for use with EC2, including a new EBS gp3 volume type that is 20% cheaper than the previous gp2 type. Additionally, it provides a higher baseline performance and it is the first general purpose volume that allows to configure IOPS independently from disk size. Given the improvements and the simplicity of the upgrade, Corey Quinn, cloud economist at The Duckbill Group, suggests to switch immediately to the new volume type:

“EBS gp3 is a game changer, full stop. 80% of the cost of gp2, it can be converted in place, and there’s no downside. Do it immediately.”

My comments: The new gp3 volume type for EBS is a great improvement, no question about that. We suggest all customers to review and convert gp2 to gp3 now.

A new io2 Block Express

A new io2 Block Express volume type is available in preview, helping small-size high-IOPS workloads, and a tiered IOPS pricing for the io2 volume type is now in place.

The main update for object storage was that S3 now delivers strong read-after-write consistency automatically for all applications. AWS announced other improvements to S3, including replication with multiple destination buckets, two-way replication across regions to improve support for multi-master and multi-region applications and new bucket keys.

My comments: The strong consistency of S3 is one of the major highly advanced technical improvements of this re:Invent event. It removes the need to worry about common errors that could happen when an application tried to read an S3 object shortly after it was updated, and could have received an old version instead. The strong consistency means that the reading application will now always get the newest version.

3. Database

There were important new launches for databases, including the previews of Babelfish for Aurora, a translation layer for Amazon Aurora PostgreSQL that enables Aurora to understand commands from applications written for Microsoft SQL Server. Aurora Serverless v2 is a new serverless relational database, compatible with MySQL, and AWS Glue Elastic Views builds materialized views that combine and replicate data across multiple data stores. There are separate coverages for both Aurora Serverless v2 and Babelfish for Aurora on InfoQ. Amazon Aurora PostgreSQL now integrates with AWS Lambda.

Amazon Redshift

Different improvements and new features were introduced for Amazon Redshift, the data warehouse service, for example the ability to move clusters between availability zones, automatic table optimization and the previews of data sharing and support for native JSON data processing.

My comments: This year, AWS did not announce any new database product, unlike most previous re:Invent events. Still, the improvements from this year help companies and DBAs to work more productively with data from multiple sources. This is especially useful for enterprises which host many different types of data stores: with the new features, consolidating the business data from different platforms to a single data lake or data warehouse becomes more feasible and cost-effective.

4. Networking and IoT

AWS Local Zones, single zone extensions of regions that are located near densely populated areas to provide lower latency, were introduced in 2019. During the conference, AWS announced that three new Local Zones will be GA this month in Boston, Houston, and Miami and twelve more in 2021, including New York City and Chicago.

Smaller Outpost options will follow next year as well, allowing customers to deploy AWS hardware in smaller offices, factories, and sites that are space-constrained but need access to low-latency compute capacity.

In the IoT space, the new AWS IoT Greengrass 2.0 provides an open source edge runtime and tools for local software development and managing software on large fleets of devices.

My comments: For Vietnam and the rest of the Southeast Asia, the major focus should be in the better IoT and edge support which allows new business cases and proof-of-concept projects with IoT-capable devices. The scaled-down version of Outpost will make this product more attractive for Vietnamese customers, and we expect the first pilot projects of Outposts in Vietnam and other Southeast Asian markets within a year.

5. Machine Learning

Many machine learning new features and products were discussed during the keynote of Swami Sivasubramanian, vice president for AI and machine learning at AWS, and were focused around SageMaker. The new Amazon SageMaker Feature Store is a fully-managed, purpose-built repository to store, update, retrieve, and share machine learning features. See InfoQ’s coverage here.

SageMaker Clarify, SageMaker Debugger, SageMaker Managed Data Parallelism and SageMaker Model Parallelism were other new services and features announced. Among many positive reactions, Jeremy Edberg, CEO at MinOps, highlights the benefits of Amazon SageMaker Clarify, the service focused on bias detection and explainability:

“It helps you to detect biases in your dataset. I think it’s fantastic; it’s just surfacing that this problem exists. A lot of people don’t even realize this is a problem at all. It’s pretty amazing!”

Coney Quinn finds the new approach confusing:

At launch, Amazon SageMaker was an easy onramp to machine learning for folks without formal data science training. Today, SageMaker Autopilot, SageMaker Studio, SageMaker Feature Store, SageMaker DataWrangler, SageMaker Ground Truth, SageMaker Notebook, SageMaker Neo, SageMaker RL, SageMaker Marketplace, SageMaker Experiments, SageMaker Debugger, SageMaker Model Monitor, and whatever they’re released between the time I write this and the time I publish it mean that a neophyte is going to pull up the service page, get freaked out, shut their laptop and walk away.

AWS Trainium

AWS Trainium, a machine learning training chip designed by AWS and Habana Gaudi based EC2 built for machine learning were announced at the beginning of the conference, together with Amazon Monitron, an end-to-end system that detects abnormal behavior in industrial machinery, and AWS Panorama, a machine learning appliance and SDK.

Amazon Lookout for Metrics is one more addition in the anomaly detection space, adding a flexible service for time series analysis, together with Amazon Lookout for Equipment and Amazon Lookout for Vision.

My comments: The Machine Learning space is vastly expanding both in services width and breadth, with more specialized solutions coming to market. All the new releases from AWS target specific use cases, which are worth exploring for companies already having some experience on AI/ML projects. For those who are still new to Machine Learning, it may be worth waiting for some months, as AWS does recognize the challenge of having so many services options to choose from, and will surely come up with a solution that simplifies getting started in this exciting technology.

6. Monitoring, Architecture and Coding

A significant part of Werner Vogels’ keynote was dedicated to the importance of logging, monitoring and how to improve deployments. A few new services and improvements are now available: CloudTrail provides more granular control of data event logging, while Amazon Managed Service for Prometheus (AMP) and Amazon Managed Service for Grafana (AMG) are now in preview.

My comments: We believe that better visibility in how your applications and infrastructure are working, is a key differentiator in having quality software and services that support your business. All improvements to AWS monitoring tools functionality are therefore directly benefiting your business processes. This year, there’s a special focus to make Kubernetes easier to understand, the announcement of AWS Managed Service for both Prometheus and Grafana means that monitoring your Kubernetes deployments in EKS and other platforms becomes more well-organized and straightforward.

AWS Fault Injection Simulator

AWS Fault Injection Simulator is a managed chaos engineering service that will be available in 2021 and introduces disruptive events across a range of AWS services, including EC2, EKS, ECS and RDS.

My comments: As companies become more cloud-native and adapt the mindset of proactive approach to reliability and stability, chaos engineering practices become widespread. When you introduce random disruptive events to your systems, such as suddenly shutting down an instance, you will be able to test how your system recovers from such failures gracefully. With a managed chaos engineering service from AWS, it will be possible to organize and monitor all these efforts in one service for better results.

Cloudshell

Cloudshell, a browser-based shell to interact with AWS resources, is already available and makes it simpler to work with the CLI without running an instance and handling credentials. Python is now supported by Amazon CodeGuru, the managed service for automated code reviews and application performance recommendations. Amazon Location, a service to integrate maps, location awareness, and other location-based features to web and mobile applications was announced in preview at the end of the conference.

My comments: Cloudshell is one of the most exciting product announcements from AWS for cloud solutions architects, who now can access the shell and command line interface (CLI) directly from AWS console. This makes running our scripts much faster and improves productivity in everyday work with AWS services. Although Cloudshell is initially available only in Europe and US regions, we expect AWS to add it in Singapore region within a short time, then the Vietnamese customers can benefit of this also.

We invite everyone who is enthusiastic to learn about new AWS product launches and best practices to join the re:Invent January 2021 sessions in Ho Chi Minh City and Hanoi as announced by AWS.

Source of information:

Infoq

reinvent.awsevents