Express brokers for Amazon MSK

Express brokers for Amazon Managed Streaming for Apache Kafka (Amazon MSK) are a new broker type that make Apache Kafka workloads simpler to manage, more cost effective to run at scale, and more elastic. With the same low latency that you expect from Apache Kafka, Express brokers include unlimited elastic storage capacity and intelligent rebalancing. Express brokers natively deliver streaming data to Apache Iceberg tables on Amazon S3 Tables, a fully managed capability that continuously materializes your streaming data as queryable Apache Iceberg tables enabling real-time analytics without building or operating a separate ETL pipeline.

Express brokers for Amazon MSK simplify Apache Kafka management by removing the need for deep technical expertise. With fully managed storage, automated maintenance, and flexible scaling with intelligent rebalancing, Express brokers remove the need for complex storage planning, manual partition balancing, making it easy to adjust capacity as workloads change. This allows your organization to focus on core business initiatives rather than spending time and resources on Apache Kafka operations.

Express brokers for Amazon MSK provide up to three times more sustainable throughput performance than standard brokers. Additionally, scaling Express brokers is up to 20 times faster, safer and require 50% fewer brokers than scaling standard brokers. With pay-per-use storage, which includes three-way replication by default, you can increase your price for performance.

Express brokers for Amazon MSK is continuously monitored for resource imbalance and recover brokers up to 90% faster than standard Apache Kafka brokers. This minimizes the impact of data overload and configuration errors within cluster availability and performance. With defaults for intelligent rebalance, critical cluster configurations and reserved bandwidth for background operations, such as replication and recovery, cluster availability is highly predictable and stable.

​Delivery to streaming tables with Express brokers continuously materializes Apache Kafka topics as Apache Iceberg tables on Amazon S3 Tables. Amazon MSK data delivery to streaming tables can reduce the cost of ingesting and delivering Apache Kafka data into Amazon S3 Tables by up to 60% versus self-managed deployments and reduces downstream query costs by up to 30% versus self-managed Apache Kafka deployments. With this capability, intelligent inline compaction eliminates the performance impact of small files and keeps query performance predictable without sacrificing data freshness, while built-in coordination resolves concurrent writer conflicts across high-throughput consumers. Amazon MSK supports throughput of up to 10 GB/s for delivery to Apache Iceberg on Amazon S3 Tables, and because this native capability adds no broker egress throughput, customers avoid the incremental infrastructure costs of scaling connector pipelines and match capacity to actual demand rather than peak. Customers deliver data to streaming tables and query or transform the data with any engine of their choice, including Apache Spark, Trino, or Apache Flink. You can also deliver streaming data in the source format to general purpose S3 buckets for backup or downstream data processing use cases.