Improvement to the compile time of a crate
For one of my projects, I need to use LLVM so I tried this cool inkwell crate that provides a mostly safe wrapper over LLVM.
To my dismay, though, compiling this crate takes… a lot of time:
Debug build: 1m 05s Release build: 3m 34s
By the way, I write this article for the sole purpose of trying to fix some problems there is in the crate ecosystem and by no mean I want to incriminate the author of this crate (or any other). I’ve been guilty of doing the same mistakes, but I learned from them and want other people to learn from them as well.
Procedural macros
So, I started to remove some crates to see how it would improve the compile time.
I decided to start by removing the procedural macro included in a sub-crate of inkwell.
After removing it, we get the following build times:
Debug build: 26.56s Release build: 57.19s
Wow, just by removing this procedural macro, we get 2x to 3x improvement in compile time!
Even more worrying is this:
304 insertions(+), 986 deletions(-)
This means the procedural macros were actually bigger than the code it generated! So please, think about the complexity you’re adding to your projects. Is it really worth it to write a procedural macro in your case? Think about it, because they can quickly become a mess (hum, hum…) and increase a lot compile time.
Regex
The next crate I removed was the regex crate, which gave this new build time:
Debug build: 25.85s
It’s not much faster because regex is still a transitive dependency: don’t worry, it will be removed completely later.
One thing I do want to mention, though, is to avoid pulling a crate, especially if it has many dependencies and increase compile time, for a one-time usage that you could write with a few more lines of code. Here’s the patch, which removed the usage of the crate, which was only used in a single function:
impl StringRadix {
/// Create a Regex that matches valid strings for the given radix.
- pub fn to_regex(&self) -> Regex {
+ pub fn is_match(&self, slice: &str) -> bool {
+ if slice.is_empty() {
+ return false;
+ }
+ let mut chars = slice.chars();
+ if slice.starts_with(&['-', '+'] as &[char]) {
+ if slice.len() == 1 {
+ return false;
+ }
+ chars.next();
+ }
match self {
- StringRadix::Binary => Regex::new(r"^[-+]?[01]+$").unwrap(),
- StringRadix::Octal => Regex::new(r"^[-+]?[0-7]+$").unwrap(),
- StringRadix::Decimal => Regex::new(r"^[-+]?[0-9]+$").unwrap(),
- StringRadix::Hexadecimal => Regex::new(r"^[-+]?[0-9abcdefABCDEF]+$").unwrap(),
- StringRadix::Alphanumeric => Regex::new(r"^[-+]?[0-9[:alpha:]]+$").unwrap(),
+ StringRadix::Binary => chars.all(|char| char.is_digit(2)),
+ StringRadix::Octal => chars.all(|char| char.is_digit(8)),
+ StringRadix::Decimal => chars.all(|char| char.is_digit(10)),
+ StringRadix::Hexadecimal => chars.all(|char| char.is_digit(16)),
+ StringRadix::Alphanumeric => chars.all(|char| char.is_ascii_alphanumeric()),
}
}
}Is that really clearer with regexes? In my opinion, the clarity is roughly the same, even though the code is a bit longer without regexes.
Remove some other crates
After that, I removed the enum-methods crate, which resulted in a noticeable compile time improvement:
Debug build: 22.11s
Then, I removed the either crate, which didn’t really improved compile time:
Debug build: 22.07s
I’ve never really seen a use case for this crate, because I used the Result type for errors and I simply create a new type when I need a Either type, which will have a clearer name than Either anyway.
Regex (continued)
Finally, I truly and completely removed the regex crate from llvm-sys.
That was in another crate and again, we can see that the parsing done was simple enough to do it without regexes.
The improvement in compile time is huge, though:
Debug build: 9.97s Release build: 16.57s
Conclusion
First of all, the code might be slightly different than the original, might miss some feature attributes and may even be buggy, but all the tests pass and it would be easy to fix these issues.
Overall, it’s a 13x improvement in compile time in a release build. That’s enormous.
I know sometimes it’s hard to remove dependencies from a project. But, one thing I want to mention is that it only took me a couple of hours, in a code base I didn’t even know, in order to vastly improve the compilation time and reduce the complexity. Which means that sometimes it’s really easy and you, as a crate author, will be able to do it even more quickly for your own crate.
One thing to remember is to avoid procedural macros in order to reduce compile time and code complexity.
Also, avoid using crates with many dependencies (like regex), especially if you only use it once or twice.
Moreover, some crates, like either are so simple you can recreate it in your code in a few minutes.
People tell me to not reinvent the wheel, but I vastly prefer to reinvent a small wheel than to create a whole new big truck.
Why am I telling you that? Simply because the resulting code is smaller than the original, as GitHub says:
680 additions and 1,110 deletions.
So please, stop using many dependencies for the sake of not reinventing the wheel: that might not result in you writing less code, which is the goal of libraries, right?
So how to become aware of changes that could increase compile time by adding new dependencies?
Well, the first thing I look at when I review a pull request is the Cargo.lock file.
Why?
Because it lists all transitive dependencies, so that’s a quick way to see how many new dependencies were added by this pull request.
One might not even notice this file, because some people decided to hide it by default in GitHub pull requests, so it’s important to remember to take a look at this file.
Perhaps we should show it by default…
Maybe we could think of some goals for a 2020 roadmap for Rust… One step was attempted in the past, but that was rejected. Perhaps it’s time to bring it back…