Why I Chose Rust for a Workflow Engine

Most workflow engines work the same way: define a step graph in a YAML file or DSL, let an engine interpret it, and hope the execution matches what you imagined. I used this approach for years with n8n, Airflow, and even Temporal in an earlier version, before hitting the limits of declarative definitions when business logic gets complex.

I built IronFlow to solve this. It's a workflow engine where workflows are imperative Rust code: no YAML, no DSL. The engine persists every step, tracks costs, and exposes everything through a REST API.

The problem with declarative workflows

A YAML workflow file works for simple cases: A then B then C. The trouble starts when you need:

• Nested conditions: if step 2 fails and step 1 produced a certain flag, skip step 3 but run step 4 with different parameters
• Conditional parallelism: fan-out on N items, but some items need different processing
• Granular error handling: retry with backoff on some steps, fail-fast on others, structured logs for debugging

In YAML, these scenarios produce condition trees that become unreadable. You end up writing code in "hooks" or "scripts" embedded in the YAML, which is coding in a template language instead of a real one.

Temporal solved this by offering workflows as code (Go, Java, TypeScript, Python). Their approach is right. But Temporal requires heavy infrastructure: a cluster with Cassandra or MySQL, a frontend service, a history service, a matching service. For a project that needs to orchestrate AI agents and shell commands, it's overkill.

Why Rust and not Go or Node

The language choice for a workflow engine isn't neutral. The engine is an infrastructure component that runs continuously, manages concurrency, and manipulates state machines. Here's what motivated the Rust choice.

The type system for state machines

IronFlow's core is an FSM (finite state machine) managing each run's lifecycle. A run passes through precise states - Pending, Running, AwaitingApproval, Retrying, Completed, Failed, Cancelled - and transitions between these states are constrained.

In Rust, this constraint is expressed in the type system. The FSM rejects invalid transitions at compile time, not at runtime:

pub enum RunEvent {
    PickedUp,
    AllStepsCompleted,
    StepFailed,
    StepFailedRetryable,
    RetryStarted,
    MaxRetriesExceeded,
    CancelRequested,
    ApprovalRequested,
    Approved,
    Rejected,
}

Each event can only be applied from certain states. Approved is only valid from AwaitingApproval. PickedUp is only valid from Pending. The transition table is explicit in the code:

In Go, this same logic would use switch on string or int. An invalid transition error would only appear at runtime. In Node, you wouldn't even have guarantees on event types.

Tokio and zero-compromise parallelism

A workflow engine must handle concurrency everywhere: multiple runs in parallel, concurrent steps within a single run, pending HTTP calls, workers polling the API. Tokio provides all of this with near-metal performance.

IronFlow uses an API + workers model. The API owns persistence and never executes anything. Workers poll the API for pending runs, execute them locally, and stream steps and logs back. Scaling out means starting more workers.

Concretely, a workflow can fan-out on parallel steps with ctx.parallel():

let checks = ctx
    .parallel(
        vec![
            ("test", StepConfig::Shell(ShellConfig::new("cargo test"))),
            ("lint", StepConfig::Shell(ShellConfig::new("cargo clippy"))),
            ("audit", StepConfig::Shell(ShellConfig::new("cargo audit"))),
        ],
        true, // fail-fast: stop everything if one step fails
    )
    .await?;

Each step runs in its own Tokio task. The ? propagates errors naturally. No callback hell, no promise chaining.

A single binary, zero dependencies

Go also produces a static binary, and it's a common argument in its favor. But Rust goes further with lto = true, strip = true and codegen-units = 1 in the release profile: the binary is more compact and starts faster.

For IronFlow, this means trivial deployment: a single file to copy to the server. No Node runtime, no JVM, no Python with its virtualenvs. The worker runs with a few MB of RAM, even under load.

What Rust makes natural in IronFlow

Imperative code workflows

An IronFlow workflow is a WorkflowHandler trait implementation. You receive a WorkflowContext and chain operations:

struct Deploy;

impl WorkflowHandler for Deploy {
    fn name(&self) -> &str {
        "deploy"
    }

    fn execute<'a>(&'a self, ctx: &'a mut WorkflowContext) -> HandlerFuture<'a> {
        Box::pin(async move {
            ctx.shell("build", ShellConfig::new("cargo build --release"))
                .await?;

            let checks = ctx
                .parallel(
                    vec![
                        ("test", StepConfig::Shell(ShellConfig::new("cargo test"))),
                        ("lint", StepConfig::Shell(ShellConfig::new("cargo clippy"))),
                        ("audit", StepConfig::Shell(ShellConfig::new("cargo audit"))),
                    ],
                    true,
                )
                .await?;

            if checks.is_empty() {
                return Ok(());
            }

            ctx.approval("gate", ApprovalConfig::new("Ship to production?"))
                .await?;

            ctx.shell("deploy", ShellConfig::new("./deploy.sh")).await?;
            Ok(())
        })
    }
}

The control flow is standard Rust. If tests fail, ? propagates the error and the run transitions to Failed. The approval gate suspends the run until a human acts. No DSL to learn.

Interchangeable AI providers

IronFlow supports 10 AI providers, all behind the same AgentProvider trait. A workflow written for Claude runs on any other provider without modification:

let router = ProviderRouter::new(claude)
    .route(ProviderMatcher::ModelPrefix("nvidia/".into()), nvidia);

let a = Agent::new().prompt("Review").model(Model::SONNET).run(&router).await?;

let b = Agent::new().prompt("Review").model("nvidia/deepseek-v4-flash").run(&router).await?;

The ProviderRouter dispatches on the model name. A workflow can mix vendors within the same execution. Providers include Claude Code (local), SSH, Docker, Kubernetes (ephemeral and persistent), Anthropic API, OpenAI, Gemini, Mistral, and NVIDIA NIM.

In Go, this routing-by-trait pattern would use interfaces - similar on the surface, but without the lifetime guarantees Rust provides. In Node, you'd use duck typing and discover errors in production.

Architecture in 12 crates

The workspace is split into 12 crates, each with a clear responsibility:

Cargo's feature system lets you include only what you need. ironflow-core works as a standalone library without a server or database. Providers like SSH, Docker, and Kubernetes are behind feature flags (transport-ssh, transport-docker, transport-k8s).

Honest comparison with alternatives

Temporal is the obvious choice for teams that need durable execution at scale and accept the operational complexity. Temporal actually chose Rust for their Core SDK, citing "fearless concurrency" and the fact that "if it compiles, it probably works."

Windmill chose Rust for its backend and reports 10 to 13x better performance than Airflow thanks to the PostgreSQL + Rust combination. Same stack as IronFlow.

n8n is perfect for no-code and quick automation, but its Node.js base and GUI model limit infrastructure use cases.

IronFlow sits between Temporal (too heavy for AI agent workflows) and no-code tools (too limited for complex logic). The bet is simple: if the workflow is complex enough to need conditions, loops, and granular error handling, it should be code. And if it's code, you should use a language that guarantees at compile time that states are correct.

What I'd do differently

Rust is not without trade-offs. Build time for a 12-crate workspace is significant - several minutes for a release build. And the Rust developer pool is smaller than Go's or TypeScript's.

If IronFlow were an internal enterprise tool with a 10-person team, Go would probably be a better choice. But for an open-source project where engine correctness is critical and performance matters, Rust is the right trade-off.

The code is open source

IronFlow is published under the MIT license on GitLab (mirror on GitHub). All crates are on crates.io. The project is part of a tooling ecosystem I'm building around automation and Claude Code, with MCP RTK (MCP filtering proxy) and the Claude Code skills I use daily. The complete setup tying all these tools together is detailed in a previous post. The project page has installation links and documentation. For a deep dive into the API's internal architecture (entities, store traits, Axum handlers), see Layered Architecture for a REST API in Rust with Axum and SQLx.