To run the Flint local coding model, pick the file that fits your hardware: the roughly 23 GB NVFP4 GGUF for a 48 GB Mac or a single NVIDIA GPU, or the roughly 80 GB BF16 weights for a data-centre GPU. Then serve it with Ollama, llama.cpp, rec or vLLM and point your coding agent at the local endpoint.
📺 Watch: NEW Flint : A Fast Local Coding Model!
🔥 Get the Agent OS as a free bonus: AI Profit Boardroom members get the full Agent OS zip, prompt libraries, daily tutorials and weekly live coaching calls. → Get inside
This is the hands-on guide. Files, hardware, commands, and how to wire it into an agent stack. If you want the background on who built Flint and how the benchmarks stack up, read my Flint coding model explainer first.
I run Hermes daily inside my Agent OS with a profile per model. Local models get their own profile so I can swap them in for heavy, repetitive coding work without touching a paid API. Flint is built for exactly that job.
Inside the AI Profit Boardroom I show members exactly how I set those profiles up.
Step 1: check your hardware
Before you download anything, be honest about your machine. Here's what Recursive's docs and model cards say you need.
- Mac with rec (fastest Mac path). An M4 Pro, M5 Pro or M5 Max with at least 48 GB of unified memory and about 30 GB of free disk.
- NVIDIA with the NVFP4 GGUF. The model card recommends at least 48 GB of GPU memory, or the unified memory on a DGX Spark. Recursive tested it on B300, RTX PRO 6000 Blackwell, RTX Spark and DGX Spark.
- Full BF16 weights. One B300 or H200 GPU per replica, per the BF16 model card.
The download size isn't the whole story. Recursive's docs point out you also need room for the KV cache, compute buffers, your operating system and whatever else you're running. Bigger contexts and parallel requests use more. If you run out of memory, lower the context first.
If you're not sure which route makes sense for your team, book a free AI strategy session and we'll work it out with you.
Step 2: choose your download
BF16 is the full-precision model. NVFP4 is a 4-bit version that Recursive trained with quantisation in mind, and they say it lands within 0.7 points of the 16-bit model across their benchmarks. For almost everyone running locally, the 4-bit file is the one.
| File or tag | Format | Size | Hardware | Best for |
|---|---|---|---|---|
| RecursiveSI/flint | BF16 | About 80 GB | One B300 or H200 | GPU servers, SGLang, vLLM |
| RecursiveSI/flint-nvfp4-gguf (flint-nvfp4.gguf) | NVFP4 GGUF | About 22.9 GB | 48 GB+ NVIDIA GPU or DGX Spark | llama.cpp, vLLM, OpenCode, Pi |
| Ollama flint:40b-a2b-qat-int4 | 4-bit packaged | 23 GB | Desktop running Ollama | The easiest one-command start |
| rec (Recursive's Mac engine) | 4-bit MLX | About 30 GB disk needed | M4 Pro, M5 Pro, M5 Max, 48 GB+ | Fastest on a Mac |
Ollama's library lists every Flint tag with a 256K context window. One more note from Recursive: NVFP4 names a quantisation format, not a file type. The GGUF used by llama.cpp is separate from the MLX model that rec manages, so grab the format your runtime expects.
📺 Watch: Maple AI: FREE Local Model on Your iPhone
Step 3: run Flint locally
Pick one of these. You don't need all of them.
Ollama, the easy path
Install Ollama, open the app, then run:
ollama run flint:40b-a2b-qat-int4
It downloads the model on first use and drops you into a chat. Type /bye to leave. From there, Ollama's launcher can start a coding harness from your project folder: ollama launch opencode, ollama launch pi, ollama launch codex or ollama launch vscode.
rec on a Mac
Recursive's own engine. The GitHub README says it delivers 1.4 to 2.6 times the throughput of llama.cpp on a 48 GB M4 Pro. Install it with the script from the recursive-oss/rec repo:
curl -fsSL https://raw.githubusercontent.com/recursive-oss/rec/main/install.sh | bash
Then run rec launch pi (Recursive's recommended harness) or rec launch opencode from your project folder. Use rec pull to download without starting, rec status to check it and rec stop to shut it down.
llama.cpp on NVIDIA
Download the file:
hf download RecursiveSI/flint-nvfp4-gguf flint-nvfp4.gguf pi-models.json opencode.json --local-dir flint-nvfp4-gguf
Then start the server with the settings from the model card:
llama-server -m flint-nvfp4-gguf/flint-nvfp4.gguf --alias flint -ngl all -c 262144 -np 1 -fa on -ctk f16 -ctv f16 --jinja --temp 1 --top-p 0.95 --top-k 20 --min-p 0 --host 127.0.0.1 --port 8080
That runs the full 262,144-token context. If memory is tight, lower the -c value and match your client's context limit to it. You need a llama.cpp build that supports Flint.
📺 Watch: Hermes: New FREE Local AI Model
Step 4: serve it with vLLM
vLLM shipped day-0 support with a dedicated Docker image, vllm/vllm-openai:flint, for x86_64 and arm64 with CUDA 13.0. Native support is being merged upstream, so the image is the simplest route for now. The vLLM team says one Blackwell GPU is enough, and they measured about 500 tokens per second for a single request on a GB200, even at 100K context.
For the NVFP4 checkpoint:
docker run --gpus all --ipc=host -p 8000:8000 -v ~/.cache/huggingface:/root/.cache/huggingface vllm/vllm-openai:flint RecursiveSI/flint-nvfp4-gguf --hf-config-path RecursiveSI/flint --tokenizer RecursiveSI/flint --enable-auto-tool-choice --tool-call-parser glm47 --reasoning-parser glm47 --exclude-tools-when-tool-choice-none
For BF16, swap the model name for RecursiveSI/flint and drop the two config and tokenizer flags. Those are only needed for the GGUF, because it borrows its config and tokenizer from the BF16 repo.
Don't skip the parser flags. Flint emits GLM-4.7-style tool calls and thinking blocks, and without them your agent sees raw text instead of tool calls.
Want a team that sets this up with you every week? That's what the AI Profit Boardroom is for: the Agent OS, a 30-day AI agent roadmap, daily tutorials and 4 coaching calls a week, for $69/mo.
Step 5: plug Flint into OpenHands
OpenHands is the agent I'd pair with this for full repo work. Recursive's guide walks through it.
- Install Docker Desktop and uv.
- Run uv tool install openhands --python 3.12
- Copy your project to a throwaway folder, cd into it, and run openhands serve --mount-cwd
- Open http://localhost:3000 in your browser.
- Go to Settings, then LLM, then Advanced. Set Provider Connection to None.
- For Ollama, set Custom Model to openai/flint:40b-a2b-qat-int4, Base URL to http://127.0.0.1:11434/v1 and API key to ollama. For llama.cpp, use openai/flint-local (match your alias) and http://127.0.0.1:8080/v1.
- Save it as Flint (local), make it active and start a new conversation.
One gotcha that catches people. If OpenHands is making requests from inside Docker, 127.0.0.1 points at the container, not your machine. Use http://host.docker.internal:11434/v1 for Ollama instead. And give it at least 32K of context so the system prompt and tools fit.
Use a disposable copy of your repo the first time. OpenHands can edit every file in that folder.
Step 6: wire Flint into your agent stack
Once Flint is running, it's just an OpenAI-compatible endpoint. That's the beauty of it. Anything that talks to that API can use it.
rec goes further and also exposes an Anthropic-compatible endpoint at /v1/messages on http://127.0.0.1:11500, with model flint and the key from rec auth token. That's handy if your tools expect Anthropic's format. For more on running Anthropic's coding agent against local models, see my guide to Claude Code local.
Here's the old way versus the new way. The old way: every agent call goes to a paid API, and you ration usage because the bill scares you. The new way: a fast local model takes the grinding work, and you save the premium model for the hard calls.
Set these sampling defaults on any manually configured endpoint, per Recursive's docs:
- Temperature 1.0, top-p 0.95, top-k 20.
- An output budget of 65,536 tokens where supported, and at least 32,768 for hard problems. Reasoning counts toward it.
- Leave reasoning effort unset. Flint's reasoning is always on.
- Keep the returned reasoning content in conversation history.
In my setup, Flint gets its own Hermes profile next to the others. I test every new model on real tasks and log the results at Goldie Bench, and the whole system is laid out in the Agent OS guide.
Want this mapped onto your business? Grab a free AI strategy session.
Flint local coding model FAQ
How much memory do I need to run Flint locally?
Recursive recommends at least 48 GB. That's 48 GB of unified memory on an M4 Pro, M5 Pro or M5 Max for rec, or 48 GB of GPU memory for the NVFP4 GGUF on NVIDIA. The full BF16 model needs a B300 or H200.
Should I download BF16 or NVFP4?
NVFP4 for local use. It's about 22.9 GB against roughly 80 GB for BF16, and Recursive says it stays within 0.7 points of the 16-bit model.
What's the fastest way to start?
On a supported Mac, rec launch pi. Everywhere else, ollama run flint:40b-a2b-qat-int4, then ollama launch opencode.
Does Flint work with OpenHands?
Yes. Recursive has an OpenHands guide that connects it to a local Ollama, Unsloth or llama.cpp server through a custom LLM profile.
Ready to build a local AI stack that does real work? Join the AI Profit Boardroom or book a free AI strategy session and we'll set up your plan.
