Local LLM Break-Even Math [2026]: Power, Idle, Depreciation

Originally published at kunalganglani.com — read it there for inline code, hero image, and live links.

Local LLM break even cost model power depreciation is the phrase I wish existed back when I first started seeing people justify $2–5k hardware purchases with “GPU price ÷ tokens.” That math is comforting. It’s also usually wrong.

The uncomfortable truth is that local LLM economics are dominated by what your GPU does when it’s not generating tokens. Idle power. Low utilization. Depreciation. Random failures. The fact that you could have left the money in something boring and liquid instead.

If you want a real answer to “Should I buy a GPU or just pay an API?”, you need a total cost of ownership model. One you can copy into a spreadsheet and tweak with your own kWh price, duty cycle, and resale assumptions.

Key takeaways

• A local LLM cost model is mostly a utilization problem, not a “tokens per second” problem.
• Idle power can be a first-order cost if your rig sits on for 16–24 hours/day.
• Depreciation and resale value matter more than people admit, especially if you upgrade every 12–24 months.
• Break-even vs cloud depends on your workload shape (bursty vs steady), not your peak throughput.
• Opportunity cost is the adult version of “but I already bought the GPU.”

If your GPU isn’t running hot most of the day, your “local is cheaper” math is probably a fantasy.

What this post is (and isn’t)

This is not another simplistic calculator.

It’s a parametric framework for a local LLM total cost of ownership model that includes:

• Fixed costs (hardware, depreciation, capital)
• Variable costs (electricity, maintenance)
• Real-world penalties (idle time, downtime, failure risk)
• A clean way to compare against cloud/API pricing

I’ll also give you a ready-to-copy spreadsheet structure and three worked scenarios (hobbyist, indie, small team).

I’m going to keep the numbers illustrative because this is meant to be 2026-ready. You swap in current pricing and rerun. Don’t hardcode 2024-era prices and pretend it’s “analysis.”

For related baseline break-even thinking, I already have a simpler calculator-style post at Local LLM Cost vs Cloud API: 2026 Break-Even Math [Calculator]. This post is the grown-up version.

The naive model everyone uses (and why it fails)

The naive model is basically:

• Buy GPU for $X
• GPU generates Y tokens/sec
• Cloud costs $Z per 1M tokens
• Therefore break-even occurs after N tokens

The failure modes are obvious once you say them out loud:

1. It assumes 100% utilization. Your GPU is not generating tokens 24/7 unless you’re actually running a service.
2. It ignores idle power. Many rigs sit powered on all day because it’s your workstation, not a headless inference box.
3. It ignores depreciation/resale. GPUs are not servers you keep for 5 years. In local LLM land, people flip hardware quickly.
4. It ignores failure risk. A dead GPU fan or PSU turns your “cheap inference” into downtime and emergency spend.
5. It ignores opportunity cost. Capital isn’t free, even if you paid cash.

If you’re building local AI seriously, you need to think like FinOps, not like a Reddit comment.

The TCO model: separate fixed vs variable costs

Here’s the structure I use.

Step 1: Define your time horizon

Pick a horizon that matches reality:

• 12 months if you upgrade often (common for local LLM hobbyists)
• 24–36 months if you’re a small team buying “workhorse” machines

Let’s call it H months.

Step 2: Fixed costs (hardware + capital)

Fixed costs are things you pay regardless of utilization.

Hardware capital

• GPU cost
• CPU/mobo/RAM/SSD cost (if this is a dedicated box)
• Networking/UPS (often ignored, sometimes critical)

Call total upfront cost CapEx.

Depreciation (with resale)

This is the key: the cost is not what you paid. It’s:

DepreciationCost = CapEx - ResaleValue

Model ResaleValue as a percentage of CapEx after H months.

• Best case: 70% resale after 12 months (hot market)
• Typical: 50%
• Worst: 20% (new generation drops, or your card is beat up)

Opportunity cost

If you want to be honest, add a cost of capital. Even 5% annual matters on a $4,000 build.

OpportunityCost ≈ CapEx × AnnualRate × (H/12)

This is the “I could have done literally anything else with that money” line item.

Step 3: Variable costs (electricity + maintenance)

Electricity is two numbers: idle draw and load draw.

Define:

• P_idle watts (system on, not generating)
• P_load watts (system generating tokens)
• kWh_price ($/kWh)

Then:

EnergyCost = kWh_price × (P_idle×IdleHours + P_load×LoadHours) / 1000

This is where utilization finally shows up.

Maintenance / incidentals

• Fans
• Thermal paste
• A replacement SSD
• That one power supply you regret buying

Call it MaintCost over the horizon. For a single rig, even $100–$300/year is realistic.

Step 4: Downtime and failure risk

For hobbyist rigs, downtime is “annoying.” For a small team, downtime is “you’re blocked.”

A simple expected-value model works:

FailureRiskCost = FailureProbability × ReplacementCost

You can add a downtime penalty if this is production:

DowntimeCost = DowntimeHours × CostPerHour

If you’re doing AI in production style workloads, you should price downtime honestly.

Step 5: Total local cost over horizon

Put it together:

LocalTCO = DepreciationCost + OpportunityCost + EnergyCost + MaintCost + FailureRiskCost

Now you need an output metric.

How do you calculate local LLM cost per 1M tokens when GPU sits idle most of the day?

You calculate it by dividing TCO by the tokens you actually generate, not your theoretical maximum.

Define:

• Throughput = tokens/sec when generating
• DutyCycle = fraction of time you’re generating tokens (0 to 1)
• HoursOnPerDay = how long the machine is powered on

Then:

LoadHours = HoursOnPerDay × 30 × H × DutyCycle

IdleHours = HoursOnPerDay × 30 × H × (1 - DutyCycle)

TotalTokens = Throughput × LoadHours × 3600

Finally:

LocalCostPer1M = LocalTCO / (TotalTokens / 1,000,000)

This is the whole point: if DutyCycle is 5%, your GPU is basically a space heater with a side hustle.

If you want to sanity-check throughput assumptions, I publish reproducible local inference runs at LLM benchmarks. One thing I’ve learned maintaining that database is that quantization quality cliffs are model-family-specific. A blanket “just run Q4” recommendation is wrong, and it will change your real throughput and thus your cost per token.

Utilization is the break-even lever (not peak performance)

Here’s a simple intuition table. Assume:

• Horizon: 24 months
• Machine on: 16 hours/day
• Idle draw: 80W
• Load draw: 450W
• Electricity: $0.15/kWh
• CapEx: $3,500
• Resale: 50% after 24 months
• Opportunity cost: 5% annual
• Maintenance + failure EV: $400 over horizon
• Throughput: 150 tok/s average

You can swap every number. The point is the shape.

Why the huge swing?

• Depreciation is mostly fixed.
• Opportunity cost is fixed.
• Idle energy is fixed-ish.

So your denominator (tokens) is what moves.

This is also why “my GPU does 2x tok/s” is often less important than “my GPU is actually used.”

What utilization rate makes buying a GPU cheaper than renting cloud GPUs?

Break-even is when:

LocalCostPer1M ≤ CloudCostPer1M

Cloud cost depends on whether you’re comparing to:

1. LLM APIs (pay per token)
2. Cloud GPUs (pay per hour)

APIs are easier to compare. Cloud GPUs require you to build your own serving stack.

API comparison (token pricing)

You plug in your current provider’s $/1M input and output tokens. For a concrete anchor, Anthropic publicly listed Claude 3.5 Sonnet at $3 per million input tokens and $15 per million output tokens in their announcement (Anthropic).

But your blended $/1M depends on your ratio of input:output. A coding agent might generate a lot of output. A RAG Q&A flow might be input-heavy.

If you want help thinking about token shapes and retries, see AI Agent Cost Per Task [2026]: Token Budgets & Break-Even Math and Agent Per-Task Cost Calculation [2026]: Retries, Tools, Caching. In production, retries dominate bills. I’ve seen that first-hand building the Firework short-video generation platform where the AI feature’s bill was dominated by retries and regeneration, not first-pass tokens.

Cloud GPU comparison (hourly pricing)

Cloud GPUs can win if your workload is bursty. If you only need heavy inference 2 hours/day, renting is rational.

For instance-type discovery and current on-demand rates, AWS’s instance family pages are the right canonical source, even if prices shift by region. Start with AWS EC2 G5 instances.

Your cloud TCO will include:

• Instance-hours × $/hour
• Storage
• Egress (often ignored)
• Ops time (absolutely ignored, but real)

If you’re a small team, ops time is the silent killer. Every hour you spend babysitting drivers is an hour you’re not shipping.

How should you account for depreciation and resale value of GPUs used for LLM inference?

You should account for it explicitly, and you should do sensitivity analysis.

I use three scenarios:

• Best: You resell quickly, market stays hot. 60–70% resale.
• Typical: You keep it 2 years, sell when the next gen is out. 40–55% resale.
• Worst: You keep it 3 years or the card is effectively unsellable. 10–25% resale.

Two practical notes:

1. Depreciation is not linear. GPUs often cliff when a new generation drops.
2. Local LLM usage can hurt resale. If you ran it hot 24/7, buyers can tell.

If you’re building a spreadsheet, make Resale% a single input cell so you can slide it from 20% to 70% and watch your break-even move.

How to model downtime, failures, and replacement costs for hobbyist rigs?

Don’t overcomplicate it. Use expected value.

For a hobbyist rig:

• Failure probability per year: 5–10% for “something annoying” (fan, PSU, SSD)
• Replacement cost: whatever you’d actually pay, not MSRP fantasies

So:

FailureRiskCost = p_fail × cost_replace × years

For a small team, add a downtime penalty:

• If a broken rig blocks 2 engineers for half a day, that’s 8 engineer-hours.

This is where local “cheap tokens” gets exposed. Tokens aren’t the expensive part. People are.

If you’re serious about reliability, treat your local inference service like production AI. You need observability, limits, and safe deployment patterns. (And yes, this is why I write so much about that side of the house.)

Opportunity cost: the line item everyone pretends doesn’t exist

Opportunity cost is the difference between:

• Buying a $4k box today
• Keeping that $4k liquid

You can model it as:

• A conservative annual rate (4–6%)
• Or your company’s cost of capital

Even at 5% annual, $4,000 over 2 years is about $400. That’s not nothing. It’s also not the biggest cost. But it’s the difference between “local wins” and “local barely ties” in a lot of real spreadsheets.

If you hate the idea of opportunity cost, fine. Put the rate at 0%. Just don’t pretend you did math.

A spreadsheet template you can copy in 10 minutes

Create these input cells:

• Horizon months H
• CapEx
• Resale %
• Annual opportunity rate
• kWh price
• Hours on per day
• P_idle (W)
• P_load (W)
• Duty cycle
• Throughput (tok/s)
• Maintenance cost (over horizon)
• Failure probability/year
• Replacement cost

Then compute:

• DepreciationCost
• OpportunityCost
• LoadHours, IdleHours
• EnergyCost
• FailureRiskCost
• LocalTCO
• TotalTokens
• LocalCostPer1M

That’s it. No magic.

If you want a simpler version with fewer knobs, start with LLM cost style API optimization first. If you can cut your API bill by 60% with caching and routing, that changes the whole break-even picture.

Worked scenarios: hobbyist, indie, small team

These are not “the numbers.” They’re examples to show how the model behaves.

Scenario A: Hobbyist workstation (low utilization)

• Machine on: 10 hours/day
• Duty cycle: 5–10%
• Throughput: 80–150 tok/s
• Horizon: 12 months

Outcome: local cost per 1M tokens often looks worse than mid-tier APIs, unless you have heavy daily usage.

This is where most local rigs sit. People buy hardware for privacy and fun. That’s valid. Just don’t call it “cheaper.”

Scenario B: Indie builder shipping an agent (moderate utilization)

• Machine on: 16 hours/day
• Duty cycle: 20–40%
• Horizon: 24 months

Outcome: break-even becomes plausible. Your tokens amortize depreciation fast enough that local can beat “premium” APIs, especially if you’re generating lots of output.

If you’re building AI agents that run continuously, you’re closer to a service profile. Utilization climbs.

Scenario C: Small team with a shared inference box (high utilization)

• Machine on: 24/7
• Duty cycle: 50–80%
• Horizon: 24–36 months

Outcome: local inference can get very cheap per 1M tokens. At this point, your main risks are reliability and ops overhead, not raw cost.

This is also where you start caring about serving stacks (vLLM, batching, queueing) and governance. A shared box is a shared blast radius.

For serving tradeoffs, see vLLM vs Ollama 2026: Production Power or Developer Ease?.

Local vs cloud isn’t just cost: the hidden “soft” variables

A purely numeric model still misses two real-world things:

1. Privacy and data control. If you can’t send data to a third party, cost doesn’t matter. Local wins by constraint.
2. Latency and UX. Local can be faster for tight loops. Cloud can be more consistent at scale.

If you want to reason about latency properly, don’t use averages. Use streaming metrics. I wrote up how I think about that in LLM Latency Benchmark Methodology: Streaming UX Metrics [2026].

And if you’re doing retrieval work, remember this: model choice is rarely the dominant factor. When I built the Walmart conversational commerce chatbot at Firework, retrieval quality dominated answer quality at scale, and we were handling millions of queries daily with sub-second responses. That kind of system has a very different cost profile than a single local box.

Conclusion: the break-even question you should actually ask

Most people ask: “When does my GPU pay for itself?”

The better question is: “What workload do I need to justify keeping this GPU busy?”

If your answer is “I’ll use it sometimes,” you’re buying it because you want it. Own that. It’s a hobby. That’s fine.

If your answer is “This will run daily, for real users, with real uptime expectations,” then do the spreadsheet, include depreciation and idle power, and make the call like an adult.

My prediction for 2026: local LLM adoption will keep climbing, but the winners won’t be the people with the biggest GPUs. They’ll be the people who can keep them utilized, reliable, and boring. That’s where the actual cost advantage lives.

Originally published on kunalganglani.com