The Tiny AI Pocket: Inside Tiiny AI's 120-Billion-Parameter Pocket Supercomputer

What Shipped at CES

"Tiny AI pocket" stopped being a figure of speech in January 2026. At CES in Las Vegas, Tiiny AI, a startup headquartered in Dover, Delaware, per business records compiled by Dealroom, showed a device small enough to fit in a jacket pocket that could run a 120-billion-parameter language model entirely offline, with no cloud API and no per-token bill, as audioXpress reported from the show floor.

Guinness World Records had already certified the device two months earlier, in December 2025, under the category "The Smallest MiniPC (100B LLM Locally)," according to Tiiny AI's own launch announcement.

The device is called the Pocket Lab, and by the time it reached CES it was already positioned less as a novelty and more as a category-defining product.

Yahoo Tech's hands-on from the show noted that it responded to prompts as fast as many cloud AI services and, despite running all day, never became noticeably warm, which stood out against the thermal complaints typical of compact AI PCs. That combination of small size, sustained load, and low heat is the entire engineering pitch of the product.

The Hardware Inside a 300-Gram Box

Strip away the marketing and the Pocket Lab is a dense piece of ARM-based silicon. It pairs a 12-core ARMv9.2 CPU with a custom dedicated NPU, together delivering roughly 190 TOPS of combined AI throughput, according to specifications compiled by Israeli tech outlet Gadgety.

It ships with 80GB of LPDDR5X memory, of which 48GB is reserved specifically for the NPU, plus a 1TB SSD, packed into a chassis measuring 14.2 by 8 by 2.53 centimeters and weighing about 300 grams, per measurements published by Live Science.

Power draw is the number that raises eyebrows. Notebookcheck's specification breakdown lists a 30-watt typical TDP inside a 65-watt power envelope, low enough that the device runs from a laptop-style adapter rather than a desktop power supply.

For context, a single NVIDIA RTX 4090 alone can draw over 450 watts under load, so the Pocket Lab is doing 120-billion-parameter inference at roughly a fifteenth of the power budget of one consumer GPU, before even counting the rest of a desktop system.

Software support is broad by design. Out of the box, the device offers one-click installation for more than 50 open-source models, including GPT-OSS, Llama, Qwen, GLM, Mistral, and Phi, plus deployment for over 100 agent frameworks such as OpenClaw, OpenCode, and Flowise, according to a company announcement distributed via PR Newswire.

Tiiny AI has also said it plans to support user-imported GGUF models from Hugging Face through a conversion tool, extending the device beyond its officially adapted model list.

The Trick: TurboSparse and PowerInfer

Running a 120-billion-parameter model on a phone-class chip should be impossible with standard dense inference, where every neuron in every layer fires for every token. Tiiny AI's answer rests on two pieces of research-derived software: TurboSparse and PowerInfer.

TurboSparse replaces a model's standard activation function with a variant called dReLU, which pushes models like Mistral and Mixtral to roughly 90 percent activation sparsity without materially hurting output quality, according to the technique's original writeup. In practice, a Mixtral-class model that would normally activate all of its parameters ends up firing only a few billion of them per token.

PowerInfer is the engine that acts on that sparsity. It exploits what its creators call activation locality: a small set of "hot" neurons fire on nearly every input, while a much larger set of "cold" neurons only fire occasionally, per the project's own technical description.

Hot neurons get preloaded onto fast, cache-like memory close to the NPU, while cold neurons are computed on the CPU cores as needed, cutting both memory footprint and the amount of data shuttled between compute units. The original research version of PowerInfer reported up to an 11.69x speedup over the popular llama.cpp runtime on a single consumer GPU.

The diagram below shows the shape of that hybrid path at inference time, from an incoming token through the predictor that decides which neurons must fire, out to the two compute paths, and back to a merged result.

Tiiny AI did not build PowerInfer from scratch. The project began at Shanghai Jiao Tong University's IPADS lab and was first released in December 2023 under paper authors Yixin Song, Zeyu Mi, Haotong Xie, and Haibo Chen.

What changed is ownership: the same repository, carrying its original commit history and roughly 9,800 GitHub stars, now lives under a GitHub organization called Tiiny-AI, whose own changelog credits the January 2026 CES launch of the Pocket Lab directly, per the project's current repository.

That changelog entry is also the most precise source for how the 120-billion-parameter claim actually works: it states the device runs GPT-OSS-120B in int4, meaning 4-bit quantization, at 20 tokens per second. That detail matters, because 4-bit quantization is what makes an 80GB memory budget arithmetically possible for a 120-billion-parameter model in the first place, not an incidental detail.

What Tiiny AI built on top is the productized layer: a phone-class ARM chip, an NPU tuned for the hot-neuron cache, and firmware that lets a non-developer install and run a model with one click rather than compiling an inference engine from source.

From Demo to Funded Hardware

A CES demo is not a shipping product, and Tiiny AI's next move was to prove demand before mass production. The company launched a Kickstarter campaign on March 11, 2026, and crossed one million dollars in pledges within five hours, from 728 backers, according to StartupHub's coverage of the launch.

By the time the campaign closed, it had pulled in $3,069,202 from 2,181 backers against an original goal of just $10,000, per the campaign's own Kickstarter page.

Pricing sat at $1,399 for the Super Early Bird tier, with a $1,299 price available to anyone who had placed an earlier $9.90 website deposit, and a broader retail estimate of roughly $1,500 once the device leaves crowdfunding, per audioXpress's CES coverage.

Tiiny AI targeted mass production for May 2026 and an August 2026 delivery window, explaining in its own Kickstarter FAQ that the gap between production and shipping existed specifically to allow time for FCC and similar certifications.

As of this writing, that window looks to be slipping. The campaign's most recent public update, posted August 28, 2026 and titled "Production Update: Inside Tiiny's Assembly Line," per its listing on Kickstarter, was still describing assembly-line production rather than confirming shipments to backers. That does not mean the device will not ship, but it does mean the promised August delivery had not been confirmed complete by month's end.

Reviewers who covered the campaign were consistently blunt about the risk profile. Cech Tech Reviews' breakdown pointed out that backers were funding unproven hardware with no independent performance validation, no established company track record, and a multi-month wait typical of crowdfunded hardware delays, while noting that a DIY Ryzen 7 5700X3D machine with 128GB of DDR5 could be assembled today for a comparable $1,200 to $1,500.

That "no independent validation" gap did not last long. On March 24, 2026, while the Kickstarter campaign was still open, YouTuber Alex Ziskind published a hands-on test of a pre-production unit, reported in Geeky Gadgets' writeup of the review.

Ziskind measured 18 tokens per second running GPT-OSS 120B, slightly under Tiiny AI's own 20-tokens-per-second marketing figure, and flagged a real capacity limit: running the full 120-billion-parameter model can require unloading a smaller model first to free memory rather than running both at once.

How It Stacks Up Against NVIDIA's Pocket Rival

Tiiny AI is not alone in the pocket-supercomputer category. NVIDIA's DGX Spark, built around a GB10 Grace Blackwell superchip, launched as a Founders Edition on October 15, 2025, and NVIDIA raised its price by 18 percent to $4,699 on February 23, 2026, citing global memory supply constraints, according to NVIDIA's own developer forum price-change notice.

It is a different machine aimed at a different budget, but the comparison is useful for placing the Pocket Lab's engineering tradeoffs in context.

A caveat on that TOPS row: the two figures are not measured the same way. Tiiny AI's roughly 190 TOPS blends CPU and NPU throughput, while NVIDIA's roughly 1,000 TOPS is GPU-only at FP4 precision, so the ten-fold gap reflects different measurement bases, not a clean multiplier on real-world speed.

Both companies also lean on 4-bit quantization to reach their headline parameter counts: Tiiny AI's 120B figure and NVIDIA's 200B figure both assume 4-bit weights rather than full precision, per a technical breakdown of the DGX Spark's NVFP4 support.

NVIDIA's 1,000-TOPS figure carries its own asterisk, too. ServeTheHome's independent testing measured combined CPU-plus-GPU power draw topping out just under 200 watts rather than the 240-watt rated ceiling, with typical LLM inference workloads pulling only 60 to 90 watts, while id Software co-founder John Carmack separately reported roughly half the advertised compute in his own testing.

NVIDIA attributes part of the gap to its own monitoring tool showing GPU-only power rather than total system draw, per its developer forum clarification.

The two devices are not really competing for the same buyer, but a same-model comparison is revealing. LMSYS, the research group behind the SGLang inference engine, reported around 50 tokens per second running GPT-OSS 120B on the DGX Spark, per its own optimization writeup, roughly two and a half times the 18 to 20 tokens per second Tiiny AI and Ziskind each reported for the Pocket Lab on the same model.

That gap is real, but it is far narrower than the roughly ten-fold difference in the two devices' raw TOPS figures would suggest. The DGX Spark is better understood as a development kit whose code ports directly to NVIDIA's full server line than as a pure speed champion, per LMSYS's own hands-on review of the hardware.

The Pocket Lab trades NVIDIA's software depth and raw TOPS for a tenth of the weight, a quarter of the power draw, and a third of the price, betting that most personal and small-team workloads do not need petaFLOP-class compute to be useful.

A Bigger Trend Than One Startup

The Pocket Lab is a specific product, but it is riding a broader shift toward pocket-scale AI hardware. OpenAI announced in May 2025 that it would ship 100 million AI "companion" devices, describing a family of pocket-size, screen-free, contextually aware devices developed with former Apple design chief Jony Ive, whose startup io it had just acquired for $6.5 billion, according to the Wall Street Journal's original report.

That timeline has since slipped. In a February 2026 court filing tied to a trademark dispute with audio startup iyO, OpenAI confirmed its first hardware device will not ship before the end of February 2027, later than the second-half-2026 window it had cited only weeks earlier, and said it would drop the "io" name entirely, according to TechRepublic's coverage of the filing.

The slip is a useful reminder that pocket-AI hardware is easier to announce than to ship, a test Tiiny AI's own delivery record is still working through too.

CES 2026 itself carried the same theme past Tiiny AI alone, with exhibitors showing everything from AI-powered pocket pets to compact humanoid robots built around locally run models.

The common thread across these products is a rejection of the assumption that useful AI requires a live connection to a hyperscaler's data center. Whether that bet pays off depends on model efficiency research continuing to outpace the hunger of frontier-scale models, since a sparsified 120-billion-parameter model on a pocket device is only competitive with cloud AI for as long as cloud models are not moving the goalposts even faster.

What This Means for Developers

For a developer, the practical appeal of the Pocket Lab is not raw benchmark speed but the removal of two constraints: internet dependency and per-token billing. A device drawing 30 watts and running fully offline is well suited to agentic workflows that need to stay private, to fieldwork or travel where connectivity is unreliable, or to cost-sensitive prototyping where a cloud API bill would otherwise scale with every test run.

Reported throughput of around 18 to 21 tokens per second on 120-billion-parameter models, consistent between Tiiny AI's own figures and Ziskind's independent test, is workable for iterative use, though still short of what a well-provisioned cloud endpoint delivers.

The honest caveat is the one every reviewer of this device has repeated: a startup founded in 2024, however credentialed its team, does not carry the multi-year track record of NVIDIA, and a first-generation crowdfunded product should be evaluated with that in mind before it becomes a dependency in a production pipeline.

Treat the Pocket Lab as what it is: a genuinely novel piece of hardware, backed by real published research in TurboSparse and PowerInfer, that is still in its first commercial generation.

Published via ZyVOP — Write once in Markdown, auto-backup to GitHub, and syndicate to Dev.to, Medium & Hashnode in 1 click.