Apache Kafka rebuilt for S3
We rewrote Apache Kafka from scratch on top of S3, resulting in some unique capabilities.
Never worry about Partitions again
Properly partitioning your topics is the most major issue running Kafka at scale. Your produce/consume workload may change at any time causing hotspots to appear. Astradot allows instant, dynamic, re-partitioning of your topics. Scale producers and consumers hetrogenously as your workload changes. Need 10 producers and 100 consumers, you can do that! This results in no performance hotspots ever!
No network costs
At scale, the cross-az network costs dominate all other costs of running Kafka. Astradot requires no cross-az replication since it writes directly to S3. Since traffic to S3 is free, Astradot has no network costs at all.
No replication costs
Data is stored directly in S3. No need for replicas.
Durability and Availability of S3
Astradot has only 1 dependency - S3. If any node can in the cluster can reach S3, your cluster is up. Individual nodes are completely stateless.
Performance
Kafka Producers use a combination of S3 Express and S3 Standard to achieve similar latency as your existing disk-based Kafka. To achieve lower cost for specific use cases, S3 Standard-only clusters can also be created.
Kafka Consumer performance is similar to regular Kafka, as Astradot is able to use pre-fetching of data.
Astradot is written in Rust, so you get the performance of native code with no GC.
100% Kafka compatibility
Fully api-compatible with Kafka, including all security features of Kafka.
Tested to the Extreme
Not just Unit and Integration tested. We run Deterministic Simulation Testing on every single commit to build the most reliable Kafka ever. Read More
Completely Diskless Kafka
Not just Serverless, Diskless. Complete separation of Storage and Compute allows instant scale up/down of your cluster. Your costs for both storage and compute are exactly what you use. You are never over or under provisioned.
Enterprise Ready
Multi-region
AWS PrivateLink support
Cluster level RBAC
SSO/SAML support
SOC2 Type2 audited
Available on AWS and Azure
About us
We have been operating in stealth mode out of San Francisco. We are backed by True Ventures, with participation from GGV Capital, Uncorrelated Ventures and other prominent investors.