APACHE KAFKA® STREAMING COMPRESSION PERFORMANCE ON AMAZON EC2 M8A.4XLARGE VERSUS AMAZON EC2 M8G.4XLARGE INSTANCESWHITEPAPER | FEBRUARY 2026 Authors: Akash Kumar, SMTS, CSG, AMD Other Contributors: Ravi Jangra, SMTS, CSG, AMD Zhiyi Sun (Director of Cloud Solutions Architect at AMD), Anand Kumar Jaiswal, SSE, CSG, AMD Ajay Ratnam (CSG, AMD) WHITEPAPER: APACHE PERFORMANCE | FEBRUARY 2026 1

KEY TAKEAWAYS • Apache® Kafka® on Amazon EC2 M8a.4xlarge instances with AMD AOCL 5.2 Compression library deliver ~51% higher throughput for producer benchmark compared to Amazon EC2 M8g.4xlarge instances with zlib-ng 2.2.1 library (with 512 KB records size). • AOCL 5.2 Compression provides ~15% performance advantage over zlib-ng 2.2.1 on the Amazon EC2 M8a.4xlarge, demonstrating the impact of AMD processor-specific library optimizations. • Amazon EC2 M8a.4xlarge instances can achieve ~11% lower cloud opex for Apache® Kafka® workloads against Amazon EC2 M8g.4xlarge instances. • Advanced AOCL 5.2...

together we advance. SOFTWARE CONFIGURATION Performance benchmarking focused on Kafka producer throughput with gzip compression enabled-a common production configuration. Benchmarks used 512 KB record sizes to simulate real-world data streaming workloads and processed 10,000 records across five iterations to establish statistical validity. The geometric mean of throughput measurements provides a robust performance metric that accounts for run-to-run variation. A software-level performance gain, combined with AMD EPYC 9005 Series processor architecture benefits, contributes to the overall ~51%...

Figure: AOCL 5.2 compression (zlib) vs zlib-ng-2.2.1 on Amazon EC2 M8a.4xlarge Instance (Software Comparison - 14.7% advantage) unit is records processed per second (Higher is better). AOCL 5.2 COMPRESSION OPTIMIZATIONS The ~51% combined hardware+software performance advantage is driven in part by advanced compression optimizations in AMD Optimizing C/C++ Libraries (AOCL 5.2), specifically engineered for AMD EPYC processor microarchitecture. Two key innovations differentiate AOCL from standard compression libraries: Enhanced Hash Function Implementation: AOCL 5.2 uses hash key generation using...

Cost Efficiency Comparison Records Processed per Dollar Spent650,618 AMD EPYC 9005 AW5 Graviton4 Amazon EC2 m8a.4xlarge instances offer compelling price performance advantages for Apache Kafka® workloads by combining high throughput with lower cloud opex.). A measured ~51% performance advantage, driven by both AMD processor hardware architecture and AOCL Compression 5.2 optimizations, enables organizations to process more records per second thus using fewer compute resources, helping reduce overall infrastructure costs. Software-to-software benchmarks confirm that AOCL 5.2 compression(zlib) provides...

APPENDIX: KAFKA PERFORMANCE BENCHMARKING METHODOLOGY This appendix provides step-by-step instructions for reproducing the Apache Kafka® 4.1.0 performance benchmarks presented in this white paper. The methodology enables consistent, reproducible results across different benchmark environments and enables validation of the reported performance advantages. STEP-BY-STEP PROCEDURE Step 1: Install Java® OpenJDK 17for rep Apache Kafka 4.1.0 requires Java 17 or later. Install OpenJDK 17 using the following command: sudo apt-get update sudo apt-get install openjdk-17-jdk Verify installation by checking...

Step 5: Format Kafka Storage Format the Kafka log directories with the cluster ID: bin/kafka-storage.sh format -t -c config/server.properties --standalone Replace with the ID generated in Step 4. Expected output: Formatting dynamic metadata voter directory /tmp/kraft-combined-logs with metadata.version 4.1-IV1 Step 6: Start Kafka Server with Compression Library Start the Kafka server with the specific compression library preloaded using LD_PRELOAD: LD_PRELOAD=/path/to/compression_lib/libz.so ./bin/kafka-server-start.sh config/server.properties & For AOCL testing: Use /path/to/aocl-5.2/lib/libz.so...

--record-size 524288: 512 KB per record (524288 bytes) simulates realistic data streaming workloads --num-records 10000: Process 10,000 records per benchmark iteration --throughput -1: Unlimited throughput (maximum sustainable rate) compression.type=gzip: Enable gzip compression for all messages acks=all: Full acknowledgment ensures message durability and replication LD_PRELOAD: Dynamically loads the specified compression library (AOCL or zlib-ng) • Execute 5 iterations per configuration to establish statistical validity • Calculate geometric mean of throughput across all iterations • Ensure...