# How to Run a 744B-Parameter Model on Hardware You Already Own: A Colibrì Walkthrough

- Published: 15 September 2026
- Updated: 15 September 2026
- Author: RenderBob team
- Category: Guides
- Canonical: https://renderbob.io/blog/colibri-744b-model-limited-hardware-walkthrough

Colibrì runs GLM-5.2, a 744-billion-parameter Mixture-of-Experts model, on a 12-core laptop with 25GB of RAM and no GPU by streaming experts from disk.

Every article on this blog about the GPU shortage has argued the same thing: capacity is scarce, so extract more from what you own and burst to the cloud for the rest. Colibrì, a project by Italian developer Vincenzo (JustVugg), tests a harder version of that idea. It runs GLM-5.2, a 744-billion-parameter frontier Mixture-of-Experts model, on a 12-core laptop with 25GB of RAM and no GPU. It treats storage, RAM and VRAM as one inference hierarchy and streams the model's experts from disk exactly when they are needed.

## The idea in one paragraph

A Mixture-of-Experts model like GLM-5.2 does not use all 744 billion parameters for every token. It activates only around 40 billion, and of those, only about 11GB actually changes from token to token: the routed experts a router selects. Colibrì keeps the small, constant dense part of the model (attention, shared experts, embeddings, roughly 17B parameters) resident in RAM at int4, about 9.9GB. The other 19,456 routed experts, roughly 19MB each at int4, live on disk (around 370GB total) and are streamed on demand, layer by layer, guided by a learning cache that tracks which experts your workload actually uses and keeps the hottest ones pinned.

## Getting started

Download a prebuilt release for Linux, macOS or Windows from the project's GitHub releases. No compiler is needed, only Python 3 for the launcher. Get the pre-converted GLM-5.2 int4 model container from Hugging Face: the gs64 group-scaled build with the int8 MTP head, at roughly 372GB. Plan disk space on a fast drive, then:

- COLI_MODEL=/nvme/glm52_i4 ./coli chat  (RAM, cache and MTP auto-detected)
- COLI_MODEL=/nvme/glm52_i4 ./coli doctor  (read-only readiness check)
- COLI_MODEL=/nvme/glm52_i4 ./coli tune  (measures and saves your fastest safe profile)
- ./coli web --model /nvme/glm52_i4  (API and dashboard, opens a browser)

## Set expectations, because the project does

On the honest baseline, the 25GB dev box the project started on, speed is roughly 0.05 to 0.1 tokens per second, which means running it overnight for one response. On better hardware it climbs: a 128GB CPU-only desktop reports around 1.8 tok/s warm; a single RTX 5070 Ti laptop-class box hits about 1.07 tok/s via the GPU-resident pipeline; six RTX 5090s with full expert residency reach 5.8–6.8 tok/s. Hardware requirements for frontier-scale inference are more negotiable than they look.

## A few things to get right from the start

Use the gs64 container specifically. Older per-row int4 mirrors measured meaningfully worse on quality and were the root cause of early repetition and never-terminating generation bugs. The MTP speculative-decoding head must be int8, not int4: an int4 head collapses to near-zero draft acceptance. If you have a second SSD, mirror the model across both and let Colibrì stripe reads across them; a 9GB/s plus 3GB/s pair reads roughly a third faster than the fast drive alone. Test O_DIRECT mode (DIRECT=1) on real NVMe. It bypasses the page cache and was measured up to 34% faster in some configurations, though it is drive-dependent and can be neutral or worse on QLC or DRAM-less drives.

Colibrì is an LLM inference engine. For the hardware-constrained problems this blog covers constantly, it shows that a frontier model can still run if you place its weights across RAM, VRAM and disk.