Apache Kafka rebuilt for S3
We rewrote Apache Kafka from scratch (in Rust) on top of S3, resulting in some unique capabilities.
No Hotspots Ever
Regular Apache Kafka can hotspot on broker nodes which are assigned to specific paritions. Astradot's unique distributed object system distributes the load of Kafka across the entire compute of the cluster, regardless of partitions. Adding/removing compute instantly scales up and down the capacity of the cluster. Thus instead of a complex distributed system, it feels like just a single computer than you can upgrade/downgrade on demand.
No network costs
At scale, the cross-az network costs dominate all other costs of running Kafka. Astradot requires no cross-az replication since it writes directly to S3. Since traffic to S3 is free, Astradot has no network costs at all.
No replication costs
Data is stored directly in S3. No need for replicas.
Durability and Availability of S3
Astradot has only 1 dependency - S3. If any node can in the cluster can reach S3, your cluster is up. Individual nodes are completely stateless.
Performance
Kafka Producers use a combination of S3 Express and S3 Standard to achieve similar latency as your existing disk-based Kafka. To achieve lower cost for specific use cases, S3 Standard-only clusters can also be created.
Kafka Consumer performance is similar to regular Kafka, as Astradot is able to use pre-fetching of data.
Astradot is written in Rust, so you get the performance of native code with no GC.
100% Kafka compatibility
Fully api-compatible with Kafka, including all security features of Kafka.
Tested to the Extreme
Not just Unit and Integration tested. We run Deterministic Simulation Testing on every single commit to build the most reliable Kafka ever. Read More
Completely Diskless Kafka
Not just Serverless, Diskless. Complete separation of Storage and Compute allows instant scale up/down of your cluster. Your costs for both storage and compute are exactly what you use. You are never over or under provisioned.
Enterprise Ready
Multi-region
AWS PrivateLink support
Cluster level RBAC
SSO/SAML support
SOC2 Type2 audited
Available on AWS and Azure
About us
We have been operating in stealth mode out of San Francisco. We are backed by True Ventures, with participation from GGV Capital, Uncorrelated Ventures and other prominent investors.