Measuring Local LLM Energy Costs on Apple Silicon M3 Ultra

Running LLMs on your own hardware is often described as free after the initial purchase. A previous Towards Data Science piece put a number on that assumption by sampling the power draw of an RTX 3090 and pricing it against a real electricity tariff. Its finding: the most expensive model to run wasn’t the biggest one. Cost per token tracked throughput, not parameter count.

Running everything on an M3 Ultra Mac Studio with 96 GB of unified memory — no discrete GPU, no VRAM, just one pool of memory the CPU and GPU share — raises the question of whether the same surprise holds on Apple Silicon, and by how much. Five models, sustained generation, real wall-socket energy, priced at a flat utility rate of $0.31/kWh produced a clear answer: the surprise not only holds, it gets bigger. A 120-billion-parameter model comes in about five times cheaper per token than a model a quarter its size.

Here’s how the numbers were obtained, and why they land where they do.

How the Measurement Works

The instrument is a small tool called TokenWatt. It’s an OpenAI-compatible proxy built for Apple Silicon: you point it at whatever local inference server you already run, and it forwards every request byte-for-byte while bracketing it with an energy measurement. It reads the whole-SoC power rails through Apple’s IOReport interface — the same counters Activity Monitor’s energy tab uses — which means no sudo, no external instrumentation, just the numbers the chip reports about itself. It subtracts a rolling idle baseline so you’re measuring the marginal cost of the request, not the machine sitting idle, and it prices the result at your electricity rate.

The honesty problem with any measurement like this is that on-die counters can drift, with no way for a reader to know whether to trust them. The solution is calibration against ground truth. Every measurement here was taken with the Mac plugged into a Shelly Plug US Gen4 that meters actual energy at the wall, and TokenWatt’s built-in calibration fits the SoC counters to it — no hand-tuning, and the same procedure anyone with a metering plug can rerun. Every number below carries a real error band from that fit, landing between ±2.6% and ±4.5%. When you see “$0.109 per million tokens,” that’s a wall-verified figure with a stated uncertainty, not a hopeful reading off an internal register.

For the headline numbers, each model was run in a sustained generation loop — the machine doing nothing but decoding tokens as fast as it can — at three durations (120, 360, and 720 seconds), three passes each, with a 15-minute idle baseline between runs. That’s the cleanest, most reproducible signal available: a saturated GPU, no gaps, no cold starts. It’s a best case, and the implications of messier real traffic are addressed later.

The Five Models

Everything here fits in 96 GB, which matters (more on that near the end). The models span from a 4-billion-parameter dense model up to a 120-billion-parameter mixture-of-experts, across different quantizations. “Footprint” is what the weights occupy in unified memory, and “tok/s” is sustained decode throughput. Cost is per million output (generated) tokens, at $0.31/kWh.

Model	Params	Type	Footprint	tok/s	$/1M out
Qwen3.5-4B	4B	dense	2.9 GB	133.8	$0.063
Qwen3.6-35B-A3B	35B	MoE (3B)	35 GB	76.0	$0.087
Qwen3-Coder-Next	~80B	MoE (3B)	60 GB	65.0	$0.103
gpt-oss-120b	120B	MoE (5B)	59 GB	74.0	$0.109
Qwen3.6-27B	27B	dense	28 GB	21.5	$0.554

Sorting that table by cost rather than size reveals the key finding. The cheapest model is the smallest, but the most expensive model — by a factor of five to nine — is the 27-billion-parameter dense one, which is smaller than three of the four models it’s more expensive than. The 120B model undercuts it five to one. Parameter count simply doesn’t predict what a model costs to run: the most expensive option is a mid-size dense model, and every model larger than it is cheaper.

Why the Big Model Wins

The mechanism is the same one the RTX 3090 piece identified, and it’s worth stating plainly: the electricity cost of a token is watts divided by throughput. Two things set it — how much power the machine draws while generating, and how many tokens it produces per second — and the 27B dense model loses on both.

It draws 138 watts, the most of any model in the table, and generates only 21.5 tokens per second, the fewest. The reason is what governs single-stream decode on this hardware: to produce one token, the machine has to read every active weight out of memory, and generation speed is set by how many bytes that is divided by the chip’s memory bandwidth. A dense 8-bit model activates all 27 billion parameters for every token — it streams its entire ~28 GB of weights per token. That’s the most data movement in the table, which is why it’s both the slowest and the most power-hungry: high bytes-per-token, low speed, the worst of both worlds.

The mixture-of-experts models break that link between size and bytes-per-token. gpt-oss-120b has 120 billion parameters sitting in memory, but for any given token a router activates only about 5 billion of them — so it reads a small fraction of its 59 GB per token, even though the whole model is resident. Less data moved per token means more speed (74 tok/s) and less power (94 W). Quantization compounds it: the MoEs here run at 4 to 6 bits against the dense model’s 8, shrinking the bytes-per-token further. The result is a 120B model that is genuinely cheaper and faster to run than a 27B one, on the same machine, measured at the same wall socket — because what you pay for per token is bytes streamed, and parameter count only sets the bytes if the model activates all of them.

The 4B model wins on pure cost because it’s tiny and fast (134 tok/s), but it represents a different quality tier. The real lesson is in the middle of the table: among models you’d actually reach for on a hard task, the well-quantized MoEs beat the dense mid-size model outright. If you were picking by parameter count — “27B is smaller, it must be cheaper to run than a 120B” — you’d pick the single most expensive option on the board.

Does It Hold When the Traffic Is Real?

A sustained loop is a benchmark, not a workload. It keeps the GPU pinned at 100% with no idle gaps, no cold starts, no waiting on a user. Real serving is lumpier, and lumpier is more expensive per token.

A second data source addresses this directly: TokenWatt logging of real inference traffic over a month — roughly 6,300 requests across these same models. These numbers are the tool’s estimated (not wall-calibrated) tier, so the ordering is more reliable than the third decimal place, but the ordering is the whole point:

Model	Real traffic, 30 days	$/1M total tokens
Qwen3.6-27B (dense)	3,788 req	~$0.78
Qwen3.6-35B-A3B (MoE)	1,270 req	~$0.071
Qwen3-Coder-Next (MoE)	933 req	~$0.057

The gap is, if anything, wider in the wild than on the bench: in day-to-day use the dense 27B costs roughly ten times as much per token as the MoEs. Two things move these numbers relative to the controlled table — they’re priced per total token (prompt plus output, not just generation), and real traffic runs the machine well below saturation, which raises the per-token cost by roughly two to three times versus the best-case loop. But the shape is identical. The dense mid-size model is the expensive one no matter how the data is sliced.

The Apple Silicon Asterisks

A few things are specific to this machine, and they matter if you want to reproduce this or reason about your own setup.

Holding a big model resident costs power even between tokens. On a unified-memory machine the whole model lives in the same RAM the system uses, and keeping tens of gigabytes of weights resident carries a standing overhead — memory bandwidth, regulators, fans — separate from the arithmetic of generation. This overhead climbs with model size and then flattens out at roughly 21 watts once a model is large enough; the calibration itself also shifts slightly for the largest, memory-bound models. It’s a second-order effect against the throughput story, but it’s real — one more reason to keep only the models you’re actively using resident.

96 GB is a ceiling, and it’s why the table stops where it does. The largest model tested, gpt-oss-120b at MXFP4, occupies about 59 GB resident. Add the OS and a working KV cache and you’re near the practical limit of a 96 GB machine — which is exactly why there’s no 200B model in the table. A MoE has to hold all its experts in memory even though it only uses a few per token, so the footprint is the full model size, not the active-parameter size. A 192 GB or 256 GB Mac Studio could run substantially larger models, and the same watts-over-throughput logic predicts where they’d land. The results also reproduce downward: on a 16, 32, or 64 GB Mac, the smaller rows in this table should show the same ordering, because the mechanism doesn’t depend on this particular machine.

This is marginal electricity only. It is not the cost of the Mac, which is by far the larger number and depends entirely on how hard you use it. Amortizing the machine — a Mac Studio that Apple currently sells for $5,299 (M3 Ultra, 96 GB, 1 TB) — over a few thousand requests dwarfs a tenth of a cent of electricity. The measurement here is the energy floor beneath that equation: useful for comparing models against each other and for knowing the true variable cost of a token, but not a total cost of ownership figure.

The broader takeaway is that on Apple Silicon, as on discrete GPU hardware, the metric that governs token economics is throughput per watt — and throughput per watt is determined by bytes streamed per token, not by parameter count. Choosing a model by size alone is a reliable way to pick the wrong one.