Field note R-002 13 November 2026 Measured, one run per row

Memory ceiling · speed

A 22 GB model wrote 30 tokens a second under a 12 GB ceiling.

The same 35B mixture-of-experts model answered a 10,000-token prompt three ways: fully in memory, under a 16 GB ceiling, and under a 12 GB ceiling. Reading the prompt took the same time in all three. Writing the reply slowed as the ceiling came down.

Watch it

One question, one ceiling, both screens.

On the left, the chat window sends a short coding question to Ornith-1.5-35B-A3B-OptiQ-4bit. On the right, Observatory shows the same moment: no model loaded, then about 10.6 GB in use under a 12 GB ceiling, with the line "This reply has read experts from the SSD." The clip runs at real speed and shows the model's reasoning as it streams. It was recorded on a development build.

Measured

Ten thousand tokens in, five hundred out.

Each row is one request of 10,077 prompt tokens and 500 written tokens, after a small warmup, with the prompt cache off.

Ornith-1.5-35B-A3B-OptiQ-4bit, 21.9 GB on disk
Memory ceilingExpertsPeak model memoryReading the promptWriting the reply
36 GBAll in memory20.9 GB4.8 s · 2,085 tokens/s66.5 tokens/s
16 GBRead from the SSD15.4 GB4.5 s · 2,228 tokens/s40.3 tokens/s
12 GBRead from the SSD11.1 GB4.5 s · 2,235 tokens/s30.2 tokens/s

The largest model in the catalog, an experimental build, ran under the same two ceilings:

Ornith-1.5-122B-MLX-mixed-2bit, about 38 GB on disk
Memory ceilingExpertsPeak model memoryReading the promptWriting the reply
16 GBRead from the SSD15.1 GB17.7 s · 570 tokens/s16.1 tokens/s
12 GBRead from the SSD10.8 GB12.8 s · 789 tokens/s14.2 tokens/s

Peak model memory stayed under the ceiling in every row. On the 122B model the prompt was read faster under the lower ceiling; with one run per row, that difference is within what a second run could reverse.

How it was measured

The workload, the build, and the clock.

  • Mac

    Apple M5 Pro with 48 GB of memory.

  • Build

    An optimized development build of version 0.2.103, serving through the local API the way any client would.

  • Prompt

    The opening of Romeo and Juliet followed by a request for a detailed summary: 10,077 prompt tokens. The reply was capped at 500 tokens.

  • Warmup

    One request of 602 prompt tokens and 50 written tokens before each measured request, which also loads the model.

  • Cache

    The persistent prompt cache was turned off, so no saved prompt work counted as speed.

  • Clock

    Reading the prompt is the time from sending the request to the first streamed token. Writing speed is 500 tokens divided by the time after that first token.

  • Memory

    Peak model memory is the peak the app reports for the model's own memory, read after each request.

What it does not show

A ceiling on a large Mac is not a small Mac.

The ceiling limits the memory the model itself may use. This Mac had 48 GB, so macOS could keep recently read model files in its own file cache, outside that ceiling. A Mac with 16 GB has far less room for that, and its SSD reads will be slower than the ones measured here. These rows show how the app behaves under a ceiling. They do not predict speed on a Mac with that much memory.

The model files had been read earlier in the session, so every row is a warm run. Cold reads from the SSD were not measured. Each row is a single run, so small differences between rows are not meaningful.

Read it as

A 21.9 GB model stayed under a 12 GB ceiling and kept writing at a usable speed. Not as a promise for any other Mac, model, or prompt.

Repeat it

Four steps on your own Mac.

  1. In Observatory, download the model from the Models page.
  2. On the Settings page, set Maximum model RAM to the ceiling you want and press Apply. Turn the prompt cache off and save.
  3. Send a long prompt to /v1/chat/completions on the local API with "stream": true and "max_tokens": 500, after one short warmup request.
  4. Time the first streamed token and the last, and read peak memory on Observatory's Overview page.

Your numbers will differ with your Mac, your SSD, the model and the prompt.