Arbiter-Core Release v0.5.0
This week we released a new minor version of the arbiter-core crate which is now published to crates.io as v0.5.0 and our docs are hosted on docs.rs. Feel free to check out the GitHub release notes to see more specifics! We also gained three new contributors!
What We, and the Community, Have Added
Crates.io
One big change was the ability to post our crate to crates.io. We were blocked by the fact that we had a custom ethers-rs fork, but we were able to work around it with the help of a community member via this PR which allowed us to bypass the transaction receipt polling mechanism with some unsafe Rust. We hope that this can help arbiter-core be more portable and accessible to the community. For instance, now our docs are hosted on docs.rs and we can use the crate in other projects without having to use a git dependency.
Block and Gas Settings
We also added the ability to set the block number and block timestamp in our Environment. Fundamentally, this comes in two flavors: UserControlled or RandomlySampled.
UserControlledallows the user to set the block number and timestamp to whatever they want via a method defined on theRevmMiddleware(which is made available to clients (Arc<RevmMiddleware>)). Similarly, the user can set the gas price for any and all transactions with a similar method. We also provide a methoddeal()to distribute raw ether to any address.RandomlySampledallows the user to choose a rate parameter for the amount of transactions (on average) that should appear in blocks. We draw the actual sizes from a Poisson distribution (which can also be seeded for replayability). To get the gas price to correspond to this, the user can choose a multiplier, e.g., if the number of transactions per block is 1000, then the multiplier could be set to 10^6 to get a 1Gwei gas price. Thinking ahead, we can model this more like EIP-1559, but this was a good first step.
More Middleware Methods
A large part of why we implemented block and gas settings was due to requests from the community to include more data in our transaction receipts. This has been added in so that proper gas analysis of contracts can be done. Alongside this, we added in other methods for querying the status of the environment, e.g., get_block_number(), get_gas_price(), get_block_timestamp(). We also added in a method to get the balance of an address, get_balance().
Some of this was necessitated in Primitive's Portfolio strategy contracts, we need to be able to move time forward. We are not alone in this as Philogy circuit breaker contracts also require time as an explicit contract parameter. We were happy to assist in getting a simulation for this contract started, and we are sure that we will see more later!
Benchmarks
Part of what we promised with Arbiter was speed well beyond simulations ran on Anvil. We have also produced some basic benchmarks that compare the arbiter-core RevmMiddleware to an Anvil chain. Of course, there are tradeoffs for using our environment versus Anvil, but we hope to allow for more seamless integration of tools like Anvil so our end users have a choice for which type of environment they want to emulate. For now, here are some basic benchmarks:
| Operation | RevmMiddleware | Anvil | Relative Difference |
|---|---|---|---|
| Deploy | 241.819µs | 8.215446ms | ~33.97x |
| Lookup | 480.319µs | 13.052063ms | ~27.17x |
| Stateless Call | 4.03235ms | 10.238771ms | ~2.53x |
| Stateful Call | 843.296µs | 153.102478ms | ~181.55x |
You can see more details in the bench code.
Repository Cleanup
We wanted to provide a better experience in contributing to Arbiter so we have implemented templates for issues and pull requests as well as some standardization. We'll continue to improve these and add more streamlined ways to contribute to Arbiter. Our milestones and labels now organize our releases and what we have planned for them.
Example simulation
As soon as our last notable release v0.3.0, we began simulations for our own Portfolio protocol. You can see how we do it by visiting the portfolio_simulations repository. Even after audits from amazing groups such as Spearbit, we found errors in our NormalStrategy that may not have really possible to find in traditional audits.
- First was Issue #436 which we closed quite easily.
- Second was Issue #439 which is more difficult to fix. We are currently working on a fix for this issue.
Finding bugs like this show the advantages of the type of simulation we developed with Arbiter. Not only do you get the benefit of testing your contract performance (which was our primary goal with portfolio_simulations), but you get more ways to verify there is no long tail risk (e.g., unintended behavior of the contract only seen after thousands of swaps!).
This simulation also gave us more insight on what the developer experience with Arbiter is really like. We have a lot to improve in this respect and will aim to reduce the cognitive load for the smart contract developer. We will carry forward testing other strategies inside of Portfolio and develop some of the developer experience improvements we have planned in this environment.
What We Have Planned
As we just said, we have learned a lot more about what we can do to improve the experience of using Arbiter.
Improved User Interface
Arbiter CLI was not a focus over the past few weeks as we pushed for the features we listed above. We will spend more time here to provide the user better generation of contract bindings, better templates, and more code generation for things such as config creation and data collection. Perhaps we can also integrate more of an interface for post processing or even anomaly detection in the Arbiter binary or arbiter-core.
More Middlware
We need to track the nonce of accounts properly as these are necessary in creating proper transaction receipts and can be necessary for certain simulations. We will then implement a get_nonce() method on the middleware.
Cheatcodes
For those who use Foundry, we would like to give you access to more capabilities that you know and love. In the next release we plan to give you access to the full list of Environment Cheatcodes. Furthermore, these will combine some of the API we gave in this v0.5.0 release such as the update_block into a more buildable cheatcode interface.
Forking
One of the most hotly requested features is the ability to fork a chain. This will be a core focus for the next release and it should come far more easily now that we have the RevmMiddleware interface. Please stay tuned for this!
Data Output
Collecting data from simulations was one of the largest hurdles we faced in our own simulations. We will be working on a more generic way to collect data from simulations and output it in a way that is easy to use for most users in post. It will be tough to make something that is perfect for everyone, but that's why we are also considering...
Different DBs
At the moment we have been using revm's CacheDB for our database, but it is not necessarily the best option for everyone. We would like to include other options that allow for writing to disk. This can be useful for those who want to use other analysis tools for postprocessing. In a way, this would make tools developed for node clients such as Reth more workable with Arbiter.
Config Generators
Finally, another tough part of the simulation developer experience is the continuous need to recompile your simulation into the debug level binary. We would like to reduce the amount of compilation needed for you and allow you to run in the release profile earlier. To do so, we need to make config generation much easier and require less boilerplate for users to create simulations.
If you would like to contribute to Arbiter in any way, be it suggestions, features, bug fixes, or management, please feel free to reach out to us on Telegram or open issues/PRs on Github. We are very open to community feedback and if there is consensus on desired features we will be sure to implement them as soon as we can.