The Mainline NPU Driver on RK3588, and What It Cannot Do Yet

The Rockchip NPU has had an in-tree Linux driver since 6.18, and it is deliberately small: it powers the block, maps buffers and submits jobs, and nothing else. Everything that makes a model run lives in user space, where Mesa currently accelerates convolutions and little else, so most of a network that is not MobileNet-shaped falls back to the CPU. The vendor stack does far more — INT4, FP16, all three cores on one model, LLMs — but ties you to a Rockchip BSP kernel. That trade is the actual decision.

For years the answer to "how do I use the NPU on this SoC" was the same: take the vendor BSP kernel, load an out-of-tree module, link a closed runtime, and accept the kernel version that comes with it. That is changing. There is now a mainline NPU driver for the RK3588 in drivers/accel, and it is worth knowing how much of the problem it solves before planning a product around it.

Two drivers for the same silicon

The RK3588 NPU has two Linux drivers, and they are mutually exclusive. Rockchip's rknpu lives in the BSP kernel with its own DRM interface. The upstream one, rocket, was written by Tomeu Vizoso and first shipped in Linux 6.18 in November 2025. Both drive the same three cores, and the choice determines your kernel, your runtime and your maintenance burden together.

Upstream, the NPU is not a special case any more. As of Linux 7.2 drivers/accel holds six drivers: AMD's XDNA, Arm's Ethos-U, Habana, Intel's NPU, Qualcomm's Cloud AI 100, and rocket. The Ethos-U merge commit states that its job submission is based on rocket, so the newest accelerator driver in the tree was modelled on the Rockchip one.

What the mainline NPU driver actually does

The kernel documentation is unusually direct about scope. The driver "just powers the hardware on and off, allocates and maps buffers to the device and submits jobs to the frontend unit. Everything else is done in userspace." That is the whole design. There is no compiler in the kernel, no operator library, no model format.

It exposes four ioctls, unchanged since the merge: create a buffer object, submit a job, and two for cache maintenance. The interface is still version 1.0. Points worth knowing if you read the code:

Each open file gets its own IOMMU domain, created on open and attached per job — isolation added during review, not in the first posting.
Buffers are shmem GEM objects mapped into the NPU address space, and the device is not cache-coherent with the CPU, so user space brackets access with the two cache ioctls.
Job dependencies are implicit only, taken from each buffer's reservation object. No explicit fences, no syncobj.
The kernel does not inspect the command stream: a task is a DMA address of a register-command buffer and a count.
The driver does not implement dma-buf import, so a buffer produced by the camera or GPU cannot be handed straight to the NPU. Export works through the shmem helpers.

The device tree side is one node per core, not one per NPU. Each of the three cores has its own IOMMU and power domain, and all three ship disabled in the SoC include file, so a board must enable them and supply the two regulators the binding requires — regulators that are required by the binding and read by nothing in the tree.

On a board running a mainline kernel, this is what to check first:

raghu@techveda.org:~$ grep -E "CONFIG_DRM_ACCEL(_ROCKET)?=" /boot/config-$(uname -r)
raghu@techveda.org:~$ ls /dev/accel/
raghu@techveda.org:~$ dmesg | grep -i rocket

Where the work actually happens

Because the kernel is a submission shim, capability is decided in Mesa. The user-space side is a Gallium driver, also called rocket, reached through Mesa's Teflon TFLite delegate, first shipped in Mesa 25.3.0 in November 2025.

Its operator support is narrower than the delegate's. The driver accepts convolutions, and only with per-tensor quantisation and no dilation, plus additions it fuses into the preceding convolution. Everything else runs on the CPU. Mesa's documentation lists three fully supported models on RK3588: MobileNet V1 at about 18 ms, MobileNet V2 at about 21 ms, and SSDLite MobileDet at about 48 ms. On an Inception network, a measurement posted upstream this month put roughly 79 percent of the inference back on the CPU.

The limits you will hit

Four of these matter for planning, and none is hidden.

The clock is pinned at 200 MHz. The SoC device tree assigns that rate and the driver has no devfreq support. Measurements posted upstream on MobileNet V1 give about 91 inferences per second at 200 MHz against 242.8 at 1000 MHz — roughly 2.6 times the throughput sits behind unwritten work.
Adding it is not trivial. An NPU power domain cannot be switched while the compute clock is above 200 MHz; doing so wedges the power-domain code and the next register access panics the board. The maintainer wants DVFS upstream, but the shape of the fix is unsettled.
Reset does not reliably recover the block. After a job timeout, set at 500 ms, the IOMMU can fail to come back and later inferences return a blank output surface — described upstream as unresolved.
One job runs on one core. The scheduler spreads separate jobs across the three cores, but a single model cannot be split across them upstream.

For anyone planning around a different part: in-tree support is RK3588 only. RK3576 enablement reached its eighth revision on the list this month but is not merged, and neither is the matching Mesa work.

What the vendor stack still gives you

The gap is real and not small. Rockchip's runtime supports INT4, FP16 and BF16 alongside INT8, gangs cores for a single model through a core-mask call, and has a separate SDK for language models with no upstream counterpart. It also exposes controls for on-chip SRAM and priority that upstream does not model.

The cost is the kernel. The vendor driver ships in Rockchip's BSP kernel, whose maintained branches are 5.10, 6.1 and 6.6, so you take that kernel and its security backport story for the life of the product. Check how current each half of the vendor stack is, too: the conversion toolkit last released in April 2025, while the language-model SDK moved in June 2026.

How to choose

The decision separates by what the product needs. For quantised convolutional vision of the MobileNet or MobileDet kind, the mainline NPU driver is a serious option today, and it buys a kernel you can move forward on your own schedule — the argument in upstream-first BSP. If you need transformers, low-bit weights, all three cores on one model, or an on-device language model, upstream cannot do it.

The more useful position is to be able to read both. The upstream driver is about 13 files and a generated register header; the vendor driver is a different DRM interface over the same registers. Diffing two drivers for one block, and judging which limitation is hardware and which is unwritten software, is the substance of TECH VEDA's Linux Device Drivers training.

Key takeaways

The Rockchip NPU has been in drivers/accel since Linux 6.18, a subsystem now holding six drivers.
The mainline NPU driver is a submission shim by design: four ioctls, per-file IOMMU domains, no command-stream inspection, no dma-buf import.
Capability lives in Mesa, where only convolutions and fused additions are accelerated.
The clock is pinned at 200 MHz with no DVFS, worth roughly 2.6 times in throughput.
The vendor stack gives INT4, FP16, core ganging and LLMs, but ties the product to a 5.10, 6.1 or 6.6 BSP kernel.

Frequently asked questions

Which kernel version do I need for the in-tree Rockchip NPU driver?
Linux 6.18 or newer. The driver first shipped in 6.18, released in November 2025, and at 7.2 it still supports only the RK3588. RK3576 support has been posted upstream but is not merged.

Can I run any TFLite model on it through Mesa?
No. The rocket Gallium driver accelerates convolutions with per-tensor quantisation and no dilation, plus additions fused into a convolution. Other operators fall back to the CPU, which is why MobileNet-shaped networks work well and larger networks mostly do not.

Why is the NPU slower than the datasheet suggests?
The device tree pins the NPU clock at 200 MHz and the driver has no devfreq support. Measurements posted upstream show about 91 inferences per second at 200 MHz against 242.8 at 1000 MHz on MobileNet V1.

When should I still use Rockchip's vendor stack?
When you need INT4 or FP16 weights, all three NPU cores working on one model, or on-device language models. None of those exist upstream. The cost is committing to a Rockchip BSP kernel, currently 5.10, 6.1 or 6.6.