In the tradition of robust civil engineering, where reliable foundations enable all subsequent progress, Amazon Web Services is executing a sweeping generational upgrade across its core infrastructure and platform services. The recent wave of performance benchmarks reveals a coordinated strategy centered on the Graviton5 processor, touching compute, analytics, database, and serverless offerings. These improvements are not about flashy revolutions but about measurable, practical gains: higher throughput per dollar, lower latency, reduced operational burden, and more predictable cost structures. As in the evolution of road networks, the focus is on durability, efficiency, and the removal of friction points that impede the flow of data.
Redshift: Rebuilding the Data Warehouse
The most pervasive enhancements have arrived in Amazon Redshift. The introduction of Redshift RG instances—powered by Graviton processors—represents a fundamental re-engineering of the data warehouse. Internal benchmarks show that data warehouse workloads run 50–60% faster and data lake analytics 45% faster on RG compared to the prior RA3 generation 11. These numbers translate directly into customer outcomes: Southwest Airlines reported exactly those gains along with a 30% cost reduction from the elimination of per-TB Spectrum scanning charges 11. Qoala experienced 60–70% faster query processing times, noting that performance gains exceeded the cost of the incremental investment 11. Tombola achieved 1.8x–2x faster write throughput, up to 2.2x faster read speeds, and processed 40% more data within the same window, all while reducing compute spend by 30% 11. For data loading, RG provides 2.0x faster load speeds than RA3 11, and at 10TB scale, ingestion performance is 2x faster for TPC-DS and 1.4x faster for TPC-H 11. Economically, the per-vCPU cost is 30% lower than RA3 11, and the complete removal of the $5/TB scan charge for data lake queries 11 removes a significant toll on the information highway. Architecturally, RG eliminates the separate Spectrum fleet overhead that RA3 required for data lake queries 11, streamlining the path from storage to insight.
But hardware is only part of the story. A new query compilation optimization tackles the cold-start problem that has long made interactive work challenging. Now, only 1% of query segments miss the compilation cache 13, and for those that do, the "FastCompose" approach reduces compilation time by a factor of 25.7x, to a P50 of 170 milliseconds 13. This optimization is enabled by default for all users at no additional cost, requiring no configuration changes 13. Combined with concurrency scaling—which automatically handles spikes for ETL and analytics workloads 12 and now triggers on materialized view refreshes 12—Redshift is becoming a more resilient, always-on data thoroughfare. A gaming company’s migration from Vertica to a 32-node ra3.16xlarge cluster with an additional consumer cluster for workload isolation 12, and the challenges around resize operations noted in another case 12, remind us that these systems, like any complex infrastructure, require careful planning and maintenance.
Graviton5: A Step Change in Compute Throughput
The Graviton5 processor is the load-bearing beam of this infrastructure refresh. Across instance families, it delivers substantial improvements in core performance, memory latency, and cache architecture. The general-purpose m9g instance, for example, doubles PassMark CPU scores relative to m6g (10.87K vs 5.22K) and sees memory latency scores improve from 71.56 to 30.71 24. Single-core SCore improves 16.5% and multi-core 17.5% at the same vCPU count 24. The design places a heavy emphasis on cache: L3 cache per core is 2.6x larger than Graviton4 24, and overall L3 cache is 5x larger 4, a boon for analytics engines and JVM-based workloads. For LLM inference, smaller models achieve over 20 tokens per second prompt throughput and over 15 tokens per second generation throughput, with more than a 30% improvement over m8g 24. An architectural artifact causes memory latency to increase abnormally at block sizes matching the L3 cache 24—a quirk that performance engineers will need to navigate, much like understanding the load rating of a bridge.
The compute-optimized C9g and C9gd instances leverage the same chip, providing up to 100 Gbps network bandwidth and doubling EBS bandwidth to 72 Gbps, with support for up to 128 attached EBS volumes 4. C9gd adds NVMe instance storage with up to 30% higher capacity 4 and, notably, high-resolution I/O metrics and latency histograms via CloudWatch or nvme-cli at no extra cost 4. The integration of the Nitro Isolation Engine into the security architecture 4 underscores that speed does not come at the expense of safety.
Memory-optimized X8i instances deliver equally dramatic gains. Workday reported a 44% reduction in latency and a 50% increase in throughput for in-memory workloads after migrating from X2i to X8i 23. Orion cut SQL Server core usage from 128 to 64 while maintaining performance, halving licensing costs 23. Memcached performance is up to 88% faster on X8i compared to X2i 23. The new Instance Bandwidth Configuration (IBC) allows users to shift up to 25% of bandwidth between network and EBS, optimizing for specific workload patterns 4,23—akin to dynamism in routing traffic across a network. The flagship x8i.metal-96xl instance packs 384 vCPUs, 6,144 GiB of memory, 100 Gbps network, and 80 Gbps EBS bandwidth 23. For those still on older instance types, a 12-month committed EC2 m6 instance offers superior cost-efficiency compared to legacy types like a1 or m4 26.
OpenSearch: A New Pavement for Log Analytics
The new "Optimized for Log Analytics" engine in Amazon OpenSearch Service represents a step-change in price-performance for log and observability workloads. By employing columnar storage and an append-only write path, it delivers up to 4x better price-performance on internal benchmarks 15,16. It provides up to 2x higher ingestion throughput on the same hardware 15 and up to 2x faster analytical query execution 15,16. Storage costs are reduced by up to 70% for aggregation workloads, and users can retain up to 3x more data at the same cost 15,16. In a benchmark loading 24.4 billion documents, the optimized engine achieved 80% storage savings versus raw JSON and 52% versus Lucene default storage, with a peak ingestion throughput of 1.78M docs/sec versus 647K docs/sec for Lucene 16. CPU utilization at peak was 62% for the optimized engine versus 72% for Lucene 16, indicating more efficient use of resources. For selective lookups, the query planner routes to the retained Lucene inverted index, completing a single traceId lookup across 7.4 billion spans in 165 milliseconds 16. Importantly, there are no additional charges for the optimized engine 16. This is a well-engineered system that simply does the job better, without fuss.
Lambda MicroVMs: Extending the Reach of Serverless
AWS Lambda MicroVMs enable new serverless patterns by supporting suspend and resume execution for up to 8 hours 1,2,7,20, with a maximum idle duration of 900 seconds 20. They boot from memory and disk snapshots in approximately 2 seconds 19 and can vertically scale to 4x baseline capacity during peak load 19. Unlike container-based architectures, no kernel or resources are shared between sessions 20. This shift toward session-aware execution 10 broadens the addressable use cases for serverless, though operational constraints remain: CloudFormation custom resources face a 4096-byte response limit when used with Lambda MicroVMs 19, a reminder that every new road has its weight limits.
Bedrock: Promising Platform, Growing Pains
Amazon Bedrock, the managed AI service, exhibits the unevenness of a road still under construction. Under load, tail latency can be severe: p95 InvocationLatency of 45 seconds exceeded the 30-second SLA, with 48% of average latency attributed to time-to-first-token (TTFT) routing and scheduling 18. A migration to Nova Pro with prompt caching, despite a 99.9% system prefix cache hit ratio, resulted in a surprising 29x increase in latency and a 6–8.5x increase in cost 18. In high-load testing, throttle rates reached 12% for Nova Pro, 70% for Anthropic Claude 3.5 Sonnet, and 149% for Amazon Haiku 4.5 18. The prompt caching mechanism itself introduces a cache-propagation lag of roughly 5 seconds 18, but subsequent cache reads within the 5-minute TTL cost only 10–25% of standard input pricing 18. For latency-sensitive GPT OSS models, low reasoning effort is advised 9. These metrics reflect a maturing but still evolving platform; careful capacity planning and architecture choices are essential.
Other Infrastructure Tuning
A number of smaller but notable improvements round out the picture. Amazon SQS can now deliver up to 10 messages per Lambda invocation 21, increasing batching efficiency. IAM database authentication on Amazon RDS scales with instance resources and can support high-volume connection patterns 17. Graviton5 also yields up to 35% runtime improvements for Synopsys EDA tools compared to Graviton4 14, and early testing of Siemens Calibre shows 30%+ performance boost 14. The broader architectural shift from request-level load balancing to session-aware execution 10 is evident across services.
However, infrastructure always requires vigilance. S3 buckets lacking explicit public access blocks are vulnerable to rapid automated enumeration 5. Amazon EKS can experience HTTP 429 throttling due to API Priority and Fairness seat exhaustion 8; resource quotas are necessary to prevent unbounded compute consumption 3. Cloud database systems face inherent trade-offs around write-ahead logging latency, durability, and cost 6; BtrLog achieves lower WAL latency than EBS baselines 6. Even the Free Tier can be a trap: users may rapidly exceed credit limits for certain services 25, and budget notification latency can lead to overspend 22.
Strategic Implications
For AWS, these claims are not just isolated performance leaflets; they represent a multi-front platform refresh that strengthens the foundational layers of the cloud. The internalization of silicon innovation via Graviton5 reduces dependency on x86 processors and potentially improves margins over time. The price-performance gains—particularly Redshift RG’s elimination of scan tolls and X8i’s ability to halve SQL Server licensing—directly reduce customer total cost of ownership and increase platform stickiness. Customer validations from Qoala, Tombola, and Southwest Airlines provide the empirical evidence that encourages migration.
The Redshift compilation optimization and OpenSearch log analytics engine position AWS to capture more real-time and observability workloads, pulling them onto a unified platform. Graviton5’s emphasis on large caches and NUMA topology aligns with enterprise trends toward data-intensive processing, while detailed I/O metrics at no extra cost address operational observability needs. The session-aware execution pattern (Lambda MicroVMs, concurrency scaling) reflects a broader industry trend toward stateful, long-running serverless compute, expanding the addressable use cases for cloud functions.
On the AI front, the Bedrock challenges illustrate that generative AI services are still finding their footing. The throttling and latency spikes are like bottlenecks on a newly opened highway; they will be addressed through iterative engineering, but they demand careful routing today. The introduction of prompt caching and reasoning-level controls signals that AWS is actively tuning, but performance engineering in this space remains a work in progress.
Key Takeaways
- AWS is leveraging its Graviton5 chip to deliver a generational performance leap across compute, analytics, and specialized workloads, offering materially lower cost and higher throughput—much like a new road surface that reduces rolling resistance.
- Amazon Redshift RG and its compilation optimization modernize the data warehouse for real-time and data lake use cases, backed by strong customer validation and compelling economics.
- The OpenSearch log analytics engine disrupts the logging-as-a-service market by delivering order-of-magnitude storage efficiency and throughput gains, potentially consolidating observability workloads onto AWS.
- While Bedrock AI services exhibit real-world scaling challenges, AWS is actively tuning performance with techniques like prompt caching and reasoning-level controls, indicating a maturing but still evolving generative AI platform.
- Operational discipline remains paramount: from mitigating EKS throttling to securing S3 buckets and monitoring Free Tier usage, the well-maintained infrastructure requires constant attention.