Instructions to use zai-org/CogVideoX1.5-5B-I2V with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use zai-org/CogVideoX1.5-5B-I2V with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("zai-org/CogVideoX1.5-5B-I2V", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
zR commited on
Commit ·
53543e5
1
Parent(s): 35e3f31
test
Browse files- README.md +3 -2
- README_zh.md +2 -2
- vae/config.json +2 -1
README.md
CHANGED
|
@@ -41,7 +41,7 @@ Below is a table listing information on the video generation models available in
|
|
| 41 |
<tr>
|
| 42 |
<td style="text-align: center;">Video Resolution</td>
|
| 43 |
<td colspan="1" style="text-align: center;">1360 * 768</td>
|
| 44 |
-
<td colspan="1" style="text-align: center;">
|
| 45 |
</tr>
|
| 46 |
<tr>
|
| 47 |
<td style="text-align: center;">Inference Precision</td>
|
|
@@ -53,7 +53,8 @@ Below is a table listing information on the video generation models available in
|
|
| 53 |
</tr>
|
| 54 |
<tr>
|
| 55 |
<td style="text-align: center;">Multi-GPU Inference Memory Consumption</td>
|
| 56 |
-
<td colspan="2" style="text-align: center;"><b>BF16:
|
|
|
|
| 57 |
</tr>
|
| 58 |
<tr>
|
| 59 |
<td style="text-align: center;">Inference Speed<br>(Step = 50, FP/BF16)</td>
|
|
|
|
| 41 |
<tr>
|
| 42 |
<td style="text-align: center;">Video Resolution</td>
|
| 43 |
<td colspan="1" style="text-align: center;">1360 * 768</td>
|
| 44 |
+
<td colspan="1" style="text-align: center;"> Min(W, H) = 768 <br> 768 ≤ Max(W, H) ≤ 1360 <br> Max(W, H) % 16 = 0 </td>
|
| 45 |
</tr>
|
| 46 |
<tr>
|
| 47 |
<td style="text-align: center;">Inference Precision</td>
|
|
|
|
| 53 |
</tr>
|
| 54 |
<tr>
|
| 55 |
<td style="text-align: center;">Multi-GPU Inference Memory Consumption</td>
|
| 56 |
+
<td colspan="2" style="text-align: center;"><b>BF16: 24GB* using diffusers</b><br></td>
|
| 57 |
+
|
| 58 |
</tr>
|
| 59 |
<tr>
|
| 60 |
<td style="text-align: center;">Inference Speed<br>(Step = 50, FP/BF16)</td>
|
README_zh.md
CHANGED
|
@@ -27,7 +27,7 @@ CogVideoX是 [清影](https://chatglm.cn/video?fr=osm_cogvideo) 同源的开源
|
|
| 27 |
<tr>
|
| 28 |
<td style="text-align: center;">视频分辨率</td>
|
| 29 |
<td colspan="1" style="text-align: center;">1360 * 768</td>
|
| 30 |
-
<td colspan="1" style="text-align: center;">
|
| 31 |
</tr>
|
| 32 |
<tr>
|
| 33 |
<td style="text-align: center;">推理精度</td>
|
|
@@ -39,7 +39,7 @@ CogVideoX是 [清影](https://chatglm.cn/video?fr=osm_cogvideo) 同源的开源
|
|
| 39 |
</tr>
|
| 40 |
<tr>
|
| 41 |
<td style="text-align: center;">多GPU显存消耗</td>
|
| 42 |
-
<td colspan="2" style="text-align: center;"><b>BF16:
|
| 43 |
</tr>
|
| 44 |
<tr>
|
| 45 |
<td style="text-align: center;">推理速度<br>(Step = 50, FP/BF16)</td>
|
|
|
|
| 27 |
<tr>
|
| 28 |
<td style="text-align: center;">视频分辨率</td>
|
| 29 |
<td colspan="1" style="text-align: center;">1360 * 768</td>
|
| 30 |
+
<td colspan="1" style="text-align: center;"> Min(W, H) = 768 <br> 768 ≤ Max(W, H) ≤ 1360 <br> Max(W, H) % 16 = 0 </td>
|
| 31 |
</tr>
|
| 32 |
<tr>
|
| 33 |
<td style="text-align: center;">推理精度</td>
|
|
|
|
| 39 |
</tr>
|
| 40 |
<tr>
|
| 41 |
<td style="text-align: center;">多GPU显存消耗</td>
|
| 42 |
+
<td colspan="2" style="text-align: center;"><b>BF16: 24GB* using diffusers</b><br></td>
|
| 43 |
</tr>
|
| 44 |
<tr>
|
| 45 |
<td style="text-align: center;">推理速度<br>(Step = 50, FP/BF16)</td>
|
vae/config.json
CHANGED
|
@@ -35,5 +35,6 @@
|
|
| 35 |
"CogVideoXUpBlock3D"
|
| 36 |
],
|
| 37 |
"use_post_quant_conv": false,
|
| 38 |
-
"use_quant_conv": false
|
|
|
|
| 39 |
}
|
|
|
|
| 35 |
"CogVideoXUpBlock3D"
|
| 36 |
],
|
| 37 |
"use_post_quant_conv": false,
|
| 38 |
+
"use_quant_conv": false,
|
| 39 |
+
"invert_scale_latents": true
|
| 40 |
}
|