Instructions to use SoAIHQ/gpt-oss-120b-VISION-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use SoAIHQ/gpt-oss-120b-VISION-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4
# Run inference directly in the terminal:
llama cli -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4
# Run inference directly in the terminal:
llama cli -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4
# Run inference directly in the terminal:
./llama-cli -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Use Docker

docker model run hf.co/SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

LM Studio
Jan

vLLM

How to use SoAIHQ/gpt-oss-120b-VISION-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SoAIHQ/gpt-oss-120b-VISION-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SoAIHQ/gpt-oss-120b-VISION-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Ollama
How to use SoAIHQ/gpt-oss-120b-VISION-GGUF with Ollama:
```
ollama run hf.co/SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4
```
Unsloth Desktop

How to use SoAIHQ/gpt-oss-120b-VISION-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use SoAIHQ/gpt-oss-120b-VISION-GGUF with Docker Model Runner:
```
docker model run hf.co/SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4
```

Lemonade

How to use SoAIHQ/gpt-oss-120b-VISION-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Run and chat with the model

lemonade run user.gpt-oss-120b-VISION-GGUF-MXFP4

List all available models

lemonade list

Hermes Agent

How to use SoAIHQ/gpt-oss-120b-VISION-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use SoAIHQ/gpt-oss-120b-VISION-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

gpt-oss-120b-VISION · GGUF

OpenAI's larger open-weight reasoning model with its released weights unchanged, plus experimental image input.

OpenAI's original gpt-oss-120b weights, unchanged, packed as GGUF for llama.cpp and SoAI.

At a glance

Text	Images	Audio	Reasoning	Tool calling	Context
Yes	Experimental	No	Yes	Yes	128K

Specification	Details
Parameters	117B total, 5.1B active
Architecture	Mixture of experts (128 experts, 4 active)
Context window	131,072 tokens
Reasoning	Always on, depth set by `reasoning_effort` (`low`, `medium`, `high`)
Languages	Mainly English
Made by	OpenAI

Choosing a file

Download	Size	Quality	Best for
MXFP4 (part 1 + part 2)	65.4 GB	Original	The released weights, unchanged. Download both parts and load the first one
mmproj (F16)	651.1 MB	Add-on	Needed only for image input

This repository has one model build because OpenAI post-trained the model with its expert weights already in MXFP4 and ran its published evaluations in that format. A Q4_K_M build would expand those 4-bit experts and round them again into another format, which adds error on top of the original rounding. A Q8_0 build would roughly double the size without recovering any precision.

The model needs memory for the file and for the context, and the context part grows with the context length you set. llama.cpp can run the model on the GPU, on the CPU, or with its layers split between them.

Images need the mmproj-gpt-oss-120b-f16.gguf add-on as well, loaded with --mmproj. llama-server -hf downloads it automatically.

Quick start

With llama.cpp, this command starts a local server with a built-in chat UI and an OpenAI-compatible API:

llama-server -hf SoAIHQ/gpt-oss-120b-VISION-GGUF:MXFP4

Open http://localhost:8080 to chat, or send requests to the API:

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "Explain GGUF in one sentence."}],
  "chat_template_kwargs": {"reasoning_effort": "high"}
}'

Download the files to a folder first

hf download SoAIHQ/gpt-oss-120b-VISION-GGUF --include "gpt-oss-120b-MXFP4-*.gguf" "mmproj-*" --local-dir gpt-oss-120b-VISION-GGUF
llama-server -m gpt-oss-120b-VISION-GGUF/gpt-oss-120b-MXFP4-00001-of-00002.gguf \
  --mmproj gpt-oss-120b-VISION-GGUF/mmproj-gpt-oss-120b-f16.gguf

The model only works with its harmony prompt format. The original chat template that produces it is embedded in the GGUF, so runtimes apply it automatically, and llama.cpp parses tool calls back into OpenAI-compatible tool_calls.

Recommended settings

Mode	`temperature`	`top_p`
All tasks	1.0	1.0

OpenAI recommends these values in the original model card. Reasoning cannot be switched off, so use low effort for quick answers. Allow at least 16K output tokens to leave room for long answers after the reasoning.

In llama-server, you set reasoning per request with "chat_template_kwargs": {"reasoning_effort": "high"}.

How these files are made

The mixture-of-experts weights are copied in the MXFP4 format that the model was post-trained and evaluated in, and every other tensor keeps its released BF16 precision. Nothing is requantized, so the conversion changes the container and leaves the weights as released. Some other GGUF builds of this model compress part of it further, which this build does not do.

Before any quantization, the build checks that the chat, thinking and tool-call markers are stored as special tokens and that the embedded chat template matches the original. A conversion that imports those markers as plain text breaks turns, thinking and tool calls without any error message, so the build stops instead of publishing the file.

The original model reads text only. This repository adds experimental image input through the vision encoder and projector from OpenGVLab/InternVL3_5-GPT-OSS-20B-A4B-Preview (revision aaabe6aa48), which its authors trained for this model family. The mmproj-*.gguf file contains only those vision tensors and the language model is the original release, so text, reasoning and tool calling work the same with or without it. The image input has not been benchmarked, and the vision weights keep the donor's license, which the donor model card describes. The donor was trained for gpt-oss-20b, so with gpt-oss-120b the image input is less accurate, especially on dense interface text.

All files and SHA-256 checksums

File	Type	Size	SHA-256
gpt-oss-120b-MXFP4-00001-of-00002.gguf	MXFP4	44.6 GB	`1bbe3017cb60ac95…`
gpt-oss-120b-MXFP4-00002-of-00002.gguf	MXFP4	20.8 GB	`b160e767d5f4111b…`
mmproj-gpt-oss-120b-f16.gguf	mmproj (F16)	651.1 MB	`29b69220b8fa8da9…`

The full checksums are in SHA256SUMS.

Provenance

Item	Details
Original model	openai/gpt-oss-120b by OpenAI, revision `b5c939de8f`
Converted with	llama.cpp `v0.5.0`, commit `7fe450e19`
Precision	MoE experts in native MXFP4 and all other tensors in BF16, both as released
Weights	The original release, with no fine-tuning, merging or retraining. Only the storage format differs.
Built	2026-09-26

License and credits

These files are distributed under the original model's license (apache-2.0), and its full terms continue to apply. The license and the original model card also describe the model's capabilities, training data, benchmarks and limitations.

OpenAI designed and trained the model, and SoAI only repackages the published weights. The conversion uses llama.cpp by Georgi Gerganov and the ggml contributors.

Run it with SoAI

Your models, your hardware, one software suite. Download SoAI for Windows, macOS, Linux or Android.

_{Packaged by SoAI.}

Downloads last month: 390

GGUF

Model size

117B params

Architecture

gpt-oss

Hardware compatibility

4-bit

Model tree for SoAIHQ/gpt-oss-120b-VISION-GGUF

Base model

openai/gpt-oss-120b

Quantized

(133)

this model