.png)
.png)
.png)
.png)
.png)
A few weeks back, we hinted at the possibility of a ZK proof system faster than Plonky3, the proof system used by a number of other projects, including Succinct’s SP1. Recently, the Valida team published independent benchmarks, revealing some interesting insights. Now, it’s time to share our data comparing RISC Zero zkVM and the SP1 zkVM.
At RISC Zero, we’re excited to share that zkVM 1.0 outperforms competitors by a significant margin. Our zkVM delivers high performance and is well-positioned for future improvements. Here’s a high-level comparison of RISC Zero’s zkVM against the SP1 zkVM:
Overview and Rationale
Our recent benchmarks showcase RISC Zero's clear advantage over SP1, reaffirming our position as the leader in zkVM performance. This context ensures a fair comparison and highlights the real-world implications of our performance lead. We seek to investigate the following questions:
Methodology:
We used a variety of hardware setups, ranging from consumer devices to cloud instances, and focused on common features:
In previous benchmarking efforts by Succinct, comparisons often included dissimilar features, such as accelerated versus unaccelerated Keccak, which obscured the performance of the underlying proof systems. Additionally, prior benchmarks compared a fine-tuned SP1 deployment against a sub-optimal RISC Zero setup. In our tests, we aimed for the best “out of the box” performance from both systems, using recommended compiler flags and preferred AWS instances.
To ensure fairness, we enabled all recommended performance features for both RISC Zero’s zkVM and Succinct’s SP1. Our zkVM is optimized for GPU proving, a feature not utilized in earlier comparisons. This is like comparing the speeds of an airplane and a car by making the airplane stay on the ground.
Lastly, for these tests, we chose not to include data from Jolt or Valida:
These questions are relevant because, over time, VMs will converge on similar features. The performance of the underlying proof system will then determine which VM leads in cost and speed.
By standardizing our testing environment, we ensured the results were directly comparable and fair.
We tested R0 zkVM 1.0.0-rc.5 against SP1 1.0-testnet.
We conducted experiments on 4 cloud instances:
We also tested on a variety of consumer (end-user) devices:
Notes:
We conducted several experiments. Each of these experiments is parameterized by the scale of the workload. This allows us to compare the scaling properties of the underlying proof systems for different types of workloads:
Lastly, we also wanted to gather some data about critical blockchain use cases. For this we revisited Succinct’s prior benchmarks for Tendermint Light Client, updated it to use the latest Tendermint library, and enabled GPU proving for RISC Zero.
Experiment sizes range from 2k-167M cycles (approximate). For each task, we measured the total time required to create a single ZK proof. For RISC Zero and Succinct, this is a two step process: first a segmented (resp. sharded) proof is generated, then it is compressed (resp. reduced) into a single proof. In certain use cases, additional reduction steps might be performed; we did not measure those additional steps in these tests.
For the raw data see here.
Across the board, we found that a properly configured RISC Zero zkVM outperforms a similarly configured Succinct SP1 deployment in both cost and speed. This holds true:
In certain specific use cases, which heavily rely on specific cryptographic operations, Succinct's SP1 currently demonstrates faster performance due to their use of accelerators. RISC Zero is actively working on integrating similar accelerators to boost performance in these scenarios. While Succinct's use of accelerated Keccak provided an advantage in specific benchmarks, RISC Zero's zkVM delivers strong overall performance and cost-efficiency across a wide range of workloads compared to SP1. As we continue to enhance our zkVM with additional accelerators and optimizations, we expect to see further improvements in real-world applications.
Our Tendermint light client test results, illustrated in the chart below, showcase R0's industry-leading performance.
.png)
We utilized Succinct's performance benchmark suite, which includes the Tendermint light client test, and made updates to re-run the test. The methodology and full details of the benchmark suite and our updated fork can be found HERE.
Our testing was conducted on the MacBook Pro M3. For shard size 20, R0 achieved a proof duration of 3.85 minutes, significantly faster than SP1's 7.54 minutes. For shard size 22, R0 completed in 3.08 minutes compared to SP1's 4.23 minutes. This demonstrates R0's superior efficiency, being 48% and 27% faster than SP1, respectively. These results clearly highlight our lead in the industry.
We also collected data from AWS. As with our other experiments, we tested SP1 on their recommended instance: r7i.16xlarge ($2.8005/hr, shard size 22). The proof was completed in 4.27 minutes, roughly equivalent to their performance on M3. Meanwhile, we tested RISC Zero on our recommended instances: g4dn.xlarge ($0.331/hr, shard size 20) and g6.xlarge ($0.5239/hr, shard size 21). On these instances we observed times of 3.43 minutes and 1.30 minutes, respectively.
.png)
To ensure transparency and reproducibility, we have made our benchmarking scripts and data available. To replicate the results, you can check out our guide HERE.
For complete raw data, check out the spreadsheet HERE.
Our experiments highlighted that RISC Zero zkVM not only leads in performance and cost-efficiency but also provides a robust foundation for future enhancements. In many head-to-head comparisons between similar systems, RISC Zero demonstrates strong performance and cost-efficiency over SP1. This demonstrates not only a fundamental difference between these VMs but also between the underlying proof systems.
It's important to note that performance may vary depending on specific workloads and hardware configurations. While RISC Zero shows strong overall performance, there may be certain scenarios where other solutions, such as Succinct's SP1, excel due to factors like the use of accelerators for specific tasks.
All these systems are likely to continue improving over the coming year. RISC Zero and Succinct are both actively working on the next set of performance enhancements. Thus, one can expect more head-to-head comparisons to be published in the coming months.
This competition, along with pressure from Jolt and Valida, will benefit consumers and drive advancements in the ZK space as a whole. With ongoing enhancements and a strong foundation, RISC Zero is positioned to remain the best zkVM for the foreseeable future.
To stay updated on RISC Zero benchmarks, check out our dedicated benchmarking website.
