---
title: MiniGPT-4
emoji: 🐙
colorFrom: gray
colorTo: gray
sdk: gradio
sdk_version: 6.26.0
python_version: "3.10"
app_file: app.py
short_description: Uncensored captioning with the original 2023 MiniGPT-4
fullWidth: true
startup_duration_timeout: 1h
pinned: false
license: other
---

# MiniGPT-4

Uncensored captioning with the original April 2023 MiniGPT-4, running again. Every public demo of it broke
years ago. This one uses the model the paper describes: a frozen EVA ViT-G vision
encoder, a frozen BLIP-2 Q-Former, one trained linear projection, and a frozen
**Vicuna v0** language model.

Not MiniGPT-v2. That one runs on LLaMA-2-chat, which behaves very differently.

## Why the other demos are dead

Three separate causes, all fixed here.

**Vicuna v0 was undistributable.** It shipped as a delta that had to be applied to
LLaMA-1 base weights, and LLaMA-1 was never released publicly. `Vision-CAIR/vicuna-7b`
holds the authors' own prepared weights, which removes the blocker entirely.

**The vision encoder download 403s.** MiniGPT-4 fetches `eva_vit_g.pth` from a Google
Storage bucket that no longer serves it. This Space pulls the same file from a Hub
mirror instead.

**The dependency pins rotted.** The reference environment asks for
`transformers==4.30.0` and `gradio==3.47.1`, which cannot coexist with a modern
`huggingface_hub`. Rather than pin backwards, the vendored model code was patched
forward and now runs on current `transformers`. Four changes were needed:

| what | why |
|---|---|
| `apply_chunking_to_forward`, `prune_linear_layer` | moved to `transformers.pytorch_utils` |
| `find_pruneable_heads_and_indices` | dropped upstream, vendored back in |
| custom `LlamaForCausalLM` | only added a training-time `reduction` kwarg, replaced with stock |
| `prepare_model_for_int8_training` | renamed in newer `peft`, imported defensively |

## Self-test

The CPU ggml port of MiniGPT-4 has a defect where two completely different images
encode to a bit-identical embedding, so every answer is confabulated from the prompt
alone while looking plausible. This Space runs a check at startup: it encodes a black
image and a patterned one and asserts the embeddings differ. The result is printed to
the logs and stated in the header, so a broken build cannot masquerade as a working one.

## Credit

Model by [Vision-CAIR](https://github.com/Vision-CAIR/MiniGPT-4), KAUST.
Vicuna by [LMSYS](https://lmsys.org/).