Running Rust Tasks With “xtask” and “xtaskops”
- Pop quiz: How do you write and run your daily tasks?
- How do we do this in Rust projects?
- The workflow problem
- The productivity problem
- What are the solutions out there?
- Setting up xtask
- Using xtaskops
- Using xtask for every project
Pop quiz: How do you write and run your daily tasks?
If you have a moderately large React project or a project in Go, or any other kind of language, you probably found yourself needing your own build, tests or CI tasks.
For example, for
yarn in Node.js, you'd write your own
test:integration task which run only your integration tests:
$ yarn test:integration
test:integration might be any series of shell commands which tell your test runner what to filter in.
And sometimes, these jobs have dependencies, so you order them in the right way:
$ yarn clean && yarn build && yarn test:integration
And then, you drop those into your
"ci": "yarn clean && yarn build && yarn test:integration"
Which eventually become:
$ yarn ci
make or similar tools, for all kinds of tasks such as build, lint, test, coverage, CI, release, it looks pretty similar.
Sometimes you have to write your own “CI tooling” shell scripts:
How do we do this in Rust projects?
It’s not really possible to do ad-hoc tasks with Cargo like it is with
npm, by adding some shell script to your manifest file (for Rust -
People usually solve this with custom shell scripts invoked for a specific need, or by using (abusing?)
make, similar to how the Go community did it.
There are a few reasons why we’re going the easy route like this. They are:
- It’s easy, just name the task and go
- We all know shell commands
- Running order and failure modes are inherited from the shell, too
- Easy to experiment & iterate
The workflow problem
Trying out a shell command, and pasting it as a
make task is great for simple cases.
But, for any moderately large project, ask youself these questions:
- How do you supply custom parameters to
npmtasks for that matter)
- How about supplying parameters to a shell script? and flags or options? and defaults?
The answer is: you go back to what ever shell script voodoo you are familiar with at that moment.
Prompting for user input is even worse, and running tasks in a specific order (resolving a graph of tasks) is left to you, hand-coding the order.
The productivity problem
Like in “normal” software, to be productive you need to:
- Do some programmatic logic
- Reuse your existing codebase
- Import other people’s libraries or tasks
- Ability to debug, reason about, or test tasks
All of these are not possible in
make or similar tools, and it's a painfully bad experience with authoring shell scripts.
What are the solutions out there?
Obviously, there are a few solutions for this challenge.
Declarative “Linear” build tools
There are your typical tools that were grown to “run my shell command” such as
You get to use everything you know about environment variables, shell command ordering and logic (
&&, etc), use positional params
$1, $2, etc. and hope for the best.
Smarter ‘make’ tools, semi-declarative
cargo-make are purpose-built to solve task-related pains, which is a good thing.
For example building tasks with
cargo-make looks like this:
[tasks.A] dependencies = ["B", "C"]
[tasks.B] dependencies = ["D"]
[tasks.C] dependencies = ["D2"]
[tasks.D] script = "echo hello"
So we get:
- Task-first tooling with dependencies, order resolution
- Convenience for parameters and naming
But, we’re ultimately limited by running commands on the shell, or the tool we’re using.
Programmatic language DSL tooling
In this category, you have a library or an infrastructure, and you’re left to build tasks on your own.
All you get is a way to run your fully-programming-language capable tasks in a convenient way, and some glue.
A great example for this was
rake in Ruby. It looked like
make but you build all of your tasks in your native language: Ruby.
At some point in time, a similar solution (which is more of a guideline) emerged for Rust:
xtask is a convention, a project set up practice.
If you follow the set up steps, you end up with a
cargo xtask command and a new Rust project called
xtask in which you build all your custom stuff.
xtask means you use cargo as you're used to.
It also means you are able to use Rust to its full potential: import libraries, reuse project logic, test, lint and run your build tasks safely.
We’re trying to move from a single crate project to a multiple crate workspace, where you have two members:
[ your-project, xtask ]
If your project is called
acme move your code to a new
acme/ inner folder.
Then, within your project:
$ cargo new --bin xtask
Set up your workspace (root
[workspace] members = [ "acme", "xtask" ]
Add an alias for
cargo to know
[alias] xtask = "run --package xtask --"
That’s it. Now the
xtask project is yours to build. Treat it like any other binary that should perform build tasks.
This is where you’d want to:
- Take commands and parameters easily
- Route the commands to tasks your wrote easily
- Author tasks easily
Right now, there’s no recommended way to author
xtask tasks easily.
This is why
xtaskops was born.
Using xtaskops, here is a fully loaded
xtask binary, with more tasks than you'd need. It fits a single screen more or less:
By using the
xtaskops library, you get all of the common tasks in Rust for free, and then some cool ones too like
bloat-deps (show biggest dependencies, by size) and
powerset (runs tests over combinations of features).
Just wire them to your CLI framework of choice (here, we’re using
Here’s some tasks that are available today in
- bloat_deps Show biggest crates in release build
- bloat_time Show crate build times
- dev Run cargo check followed by cargo test for every file change
- ci Run typical CI tasks in series: fmt, clippy, and tests
- coverage Run coverage
- docs Run cargo docs in watch mode
- install Instal cargo tools
- powerset Perform a CI build with powerset of features
For running your own commands, use the
xtask for every project
Instead of manually setting up
xtask over and over for every project, you can use rust-starter which is
For existing projects, you can copy the
xtask files or use Backpack which can grab parts of repositories easily.
Of course, you may want to make a starter project of your own: set up
xtask once for yourself, and reuse your project for all of your work. Then use Backpack to kick off every new project from your repository address.
Before you add another
make task, or write another custom shell build script, remember:
- Your build, ci, workflow, errands code — is still maintained code
- Don’t underestimate maintenance overhead for a task using shell commands
- Your shell voodoo better be bug free, tested, and reasoned about
- It’s better to reuse team investment in knowledge, code, standards, and practices
makecan be tricky and require forgotten knowledge (e.g.
- package.json tasks often require additional packages (
You can use
xtask to alleviate all these pains, and write more Rust code.