AWS Trainium2 Chip. Source: AWS

Every year, scores of startups find themselves to be replaced by feature announcements at AWS re:Invent 2024. Not this year — it was less about innovation but about closing the gaps in their GenAI stack and more importantly, doubling down on their core strength — platform.

Amazon is known for emphasizing avoiding the tyranny of OR during decision-making processes, which was also brought up in the keynote. This paradigm has guided AWS in enabling more choices with their services — be it compute, storage, analytics, or more. In the same vein, AWS appears to try to be both — the best platform to run GenAI workloads and the best GenAI platform. Here are key announcements from AWS re:Invent 2024 that caught our attention:

  1. Trainium2 Instances: Amazon EC2 Trn2 instances based on the new Trainium2 AI chip are generally available. AWS claims these offer 30–40% better performance than the current generation of GPU-based EC2 P5e/ P5en instances. AWS also unveiled Trainium3 chips, the next-generation machine learning chips. These chips offer 4x faster performance compared to their predecessors, catering to the growing demand for high-performance AI training infrastructure.
  2. Amazon Nova: AWS launched a new family of foundation models integrated into Amazon Bedrock, designed to excel in creating text, images, and videos.
  3. Amazon SageMaker AI, the next-generation Amazon SageMaker provides a unified platform for data, analytics, and AI. Its core, SageMaker Unified Studio, helps streamline the development, training, and deployment of custom GenAI models.
  4. Amazon SageMaker HyperPod now provides flexible training plans, allowing users to automatically reserve capacity, set up clusters, and create model training jobs. HyperPod Recipes help users get started training and fine-tuning popular publicly available foundation models in minutes without extensive expertise and quickly switch between GPU-based and Trainium-based instances.
  5. Amazon Bedrock Agent and Amazon Q Developer enhancements: Upgrades to Amazon Bedrock Agent included support for Automated Reasoning checks (AI safeguard), multi-agent orchestration, and model distillation. Enhancements to Amazon Q Developer include support for unit test generation, streamlined code review process, and documentation generation.

After having said and done everything, innovations in strengthening the platform outweighed the announcements in the GenAI space, which were essentially AWS trying to catch up with the competition. The messaging was the same at the AWS Generative AI Analyst Summit (held in October) as well.

Quick Summary of AWS re:Invent 2024 Announcements

Too many choices could also be confusing for customers, resulting in decision paralysis. With the launch of a homegrown foundational model, it is true that now customers have one more foundational model to pick from. But will that move the needle?

We can expect to see a consolidation in this space as LLMs are already hitting performance limits with the size of training parameters. Just like how witnessed with multi-cloud pitches and distro wars of Kubernetes/ OpenStack projects, only a handful of LLMs will matter at the end. Yet another LLM this late in the game is just a self-fulfilling vanity affair, no matter how good it performs in the benchmark.

We are already seeing some of these trends. According to Artificial Analysis AI Review, multiple models caught up with OpenAI’s GPT-4 in performance this year and inference pricing declining overall. Model reasoning quality and price are the primary drivers for selecting models, with the demand for models concentrated on a handful of top models.

Source: Artificial Intelligence AI Review 2024 Highlights

Though OpenAI continues to lead the market share, we can expect to see more GenAI applications built on AWS using Anthropic or open-source LLMs. AWS has data gravity and service gravity to their advantage. AWS also includes platform capabilities across the stack to attract new GenAI workloads. We can expect AWS to rapidly increase their market share of end-user GenAI applications/ revenue from GenAI applications.

CloudDon Take

Most things in life are non-binary, but decisions need to be.

AWS walking on the tightrope to balance between their foundation (cloud infrastructure) and their aspirations (Generative AI platform) is far too skewed towards the foundations that AWS might as well focus just on that. Or AWS needs to swing it towards the aspirations by going big, not just checking off items from the list.

AWS should decide what they want to be — the best platform to support Generative AI workloads or the best Generative AI platform. Not both.