Spaces:
Paused
Paused
Port Marigold V2 to CPU-only
Browse filesRuns the 20B Qwen-Image-Edit-2509 base from a memory-mapped GGUF instead of
bitsandbytes NF4, which is CUDA-only. diffusers copies each GGUF tensor out of
its memory map, which peaked near twice the file size and was OOM-killed; the
mapping is kept instead, so the weights are file-backed and evictable.
Everything computes fp32, since bf16 kernels can deadlock on a CPU-only build.
The four 1.85 GB LoRA checkpoints stream in one at a time rather than all four
resident. Runs are serialized on a lock, because Gradio's concurrency limit is
per handler and DualVisionApp registers three.
Measured on the free tier: 3.5 to 5 min per modality at 512 px in 14.7 GB.
This view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +2 -0
- .gitignore +4 -0
- LICENSE.txt +177 -0
- README.md +111 -7
- app.py +311 -0
- assets/shields/code.svg +1 -0
- assets/shields/follow.svg +1 -0
- assets/shields/paper.svg +1 -0
- assets/shields/website.svg +1 -0
- assets/shields/weights.svg +1 -0
- example_images/00_lobby.jpg +3 -0
- example_images/01_harbor_window.jpg +3 -0
- example_images/02_shop_window.jpg +3 -0
- example_images/03_garden_bed.jpg +3 -0
- example_images/04_brick_corridor.jpg +3 -0
- example_images/05_forest_stump.jpg +3 -0
- example_images/06_cows.jpg +3 -0
- example_images/07_lakeside_window.jpg +3 -0
- example_images/08_thistle.jpg +3 -0
- example_images/09_cat_garden.jpg +3 -0
- example_images/10_ornate_room.jpg +3 -0
- example_images/11_cats_sleeping.jpg +3 -0
- example_images/12_church_ironwork.jpg +3 -0
- example_images/13_glass_door.jpg +3 -0
- example_images/14_dogs.jpg +3 -0
- example_images/15_kitten.jpg +3 -0
- example_images/16_hallway.jpg +3 -0
- example_images/17_lynx.jpg +3 -0
- example_images/18_dog_in_car.jpg +3 -0
- example_images/19_burano.jpg +3 -0
- example_images/20_sculpture_archer.jpg +3 -0
- example_images/21_lion.jpg +3 -0
- example_images/22_portrait_blossoms.jpg +3 -0
- example_images/23_glass_facade.jpg +3 -0
- example_images/24_tulips.jpg +3 -0
- example_images/25_train.jpg +3 -0
- example_images/26_bull.jpg +3 -0
- example_images/27_church_nave.jpg +3 -0
- example_images/28_snow_pines.jpg +3 -0
- example_images/29_rooftop_city.jpg +3 -0
- example_images/30_heron.jpg +3 -0
- example_images/31_lilies.jpg +3 -0
- example_images/32_goat.jpg +3 -0
- example_images/33_cat_fluffy.jpg +3 -0
- example_images/34_van_field.jpg +3 -0
- example_images/35_garden_palms.jpg +3 -0
- example_images/36_sculpture_bust.jpg +3 -0
- example_images/37_vase_branches.jpg +3 -0
- example_images/38_portrait_fence.jpg +3 -0
- example_images/39_penguins.jpg +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
*.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
*.png filter=lfs diff=lfs merge=lfs -text
|
.gitignore
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
.venv/
|
| 2 |
+
.gradio/
|
| 3 |
+
__pycache__/
|
| 4 |
+
local/
|
LICENSE.txt
ADDED
|
@@ -0,0 +1,177 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
Apache License
|
| 3 |
+
Version 2.0, January 2004
|
| 4 |
+
http://www.apache.org/licenses/
|
| 5 |
+
|
| 6 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 7 |
+
|
| 8 |
+
1. Definitions.
|
| 9 |
+
|
| 10 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 11 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 12 |
+
|
| 13 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 14 |
+
the copyright owner that is granting the License.
|
| 15 |
+
|
| 16 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 17 |
+
other entities that control, are controlled by, or are under common
|
| 18 |
+
control with that entity. For the purposes of this definition,
|
| 19 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 20 |
+
direction or management of such entity, whether by contract or
|
| 21 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 22 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 23 |
+
|
| 24 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 25 |
+
exercising permissions granted by this License.
|
| 26 |
+
|
| 27 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 28 |
+
including but not limited to software source code, documentation
|
| 29 |
+
source, and configuration files.
|
| 30 |
+
|
| 31 |
+
"Object" form shall mean any form resulting from mechanical
|
| 32 |
+
transformation or translation of a Source form, including but
|
| 33 |
+
not limited to compiled object code, generated documentation,
|
| 34 |
+
and conversions to other media types.
|
| 35 |
+
|
| 36 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 37 |
+
Object form, made available under the License, as indicated by a
|
| 38 |
+
copyright notice that is included in or attached to the work
|
| 39 |
+
(an example is provided in the Appendix below).
|
| 40 |
+
|
| 41 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 42 |
+
form, that is based on (or derived from) the Work and for which the
|
| 43 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 44 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 45 |
+
of this License, Derivative Works shall not include works that remain
|
| 46 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 47 |
+
the Work and Derivative Works thereof.
|
| 48 |
+
|
| 49 |
+
"Contribution" shall mean any work of authorship, including
|
| 50 |
+
the original version of the Work and any modifications or additions
|
| 51 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 52 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 53 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 54 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 55 |
+
means any form of electronic, verbal, or written communication sent
|
| 56 |
+
to the Licensor or its representatives, including but not limited to
|
| 57 |
+
communication on electronic mailing lists, source code control systems,
|
| 58 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 59 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 60 |
+
excluding communication that is conspicuously marked or otherwise
|
| 61 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 62 |
+
|
| 63 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 64 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 65 |
+
subsequently incorporated within the Work.
|
| 66 |
+
|
| 67 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 68 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 69 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 70 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 71 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 72 |
+
Work and such Derivative Works in Source or Object form.
|
| 73 |
+
|
| 74 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 75 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 76 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 77 |
+
(except as stated in this section) patent license to make, have made,
|
| 78 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 79 |
+
where such license applies only to those patent claims licensable
|
| 80 |
+
by such Contributor that are necessarily infringed by their
|
| 81 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 82 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 83 |
+
institute patent litigation against any entity (including a
|
| 84 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 85 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 86 |
+
or contributory patent infringement, then any patent licenses
|
| 87 |
+
granted to You under this License for that Work shall terminate
|
| 88 |
+
as of the date such litigation is filed.
|
| 89 |
+
|
| 90 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 91 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 92 |
+
modifications, and in Source or Object form, provided that You
|
| 93 |
+
meet the following conditions:
|
| 94 |
+
|
| 95 |
+
(a) You must give any other recipients of the Work or
|
| 96 |
+
Derivative Works a copy of this License; and
|
| 97 |
+
|
| 98 |
+
(b) You must cause any modified files to carry prominent notices
|
| 99 |
+
stating that You changed the files; and
|
| 100 |
+
|
| 101 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 102 |
+
that You distribute, all copyright, patent, trademark, and
|
| 103 |
+
attribution notices from the Source form of the Work,
|
| 104 |
+
excluding those notices that do not pertain to any part of
|
| 105 |
+
the Derivative Works; and
|
| 106 |
+
|
| 107 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 108 |
+
distribution, then any Derivative Works that You distribute must
|
| 109 |
+
include a readable copy of the attribution notices contained
|
| 110 |
+
within such NOTICE file, excluding those notices that do not
|
| 111 |
+
pertain to any part of the Derivative Works, in at least one
|
| 112 |
+
of the following places: within a NOTICE text file distributed
|
| 113 |
+
as part of the Derivative Works; within the Source form or
|
| 114 |
+
documentation, if provided along with the Derivative Works; or,
|
| 115 |
+
within a display generated by the Derivative Works, if and
|
| 116 |
+
wherever such third-party notices normally appear. The contents
|
| 117 |
+
of the NOTICE file are for informational purposes only and
|
| 118 |
+
do not modify the License. You may add Your own attribution
|
| 119 |
+
notices within Derivative Works that You distribute, alongside
|
| 120 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 121 |
+
that such additional attribution notices cannot be construed
|
| 122 |
+
as modifying the License.
|
| 123 |
+
|
| 124 |
+
You may add Your own copyright statement to Your modifications and
|
| 125 |
+
may provide additional or different license terms and conditions
|
| 126 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 127 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 128 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 129 |
+
the conditions stated in this License.
|
| 130 |
+
|
| 131 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 132 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 133 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 134 |
+
this License, without any additional terms or conditions.
|
| 135 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 136 |
+
the terms of any separate license agreement you may have executed
|
| 137 |
+
with Licensor regarding such Contributions.
|
| 138 |
+
|
| 139 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 140 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 141 |
+
except as required for reasonable and customary use in describing the
|
| 142 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 143 |
+
|
| 144 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 145 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 146 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 147 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 148 |
+
implied, including, without limitation, any warranties or conditions
|
| 149 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 150 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 151 |
+
appropriateness of using or redistributing the Work and assume any
|
| 152 |
+
risks associated with Your exercise of permissions under this License.
|
| 153 |
+
|
| 154 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 155 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 156 |
+
unless required by applicable law (such as deliberate and grossly
|
| 157 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 158 |
+
liable to You for damages, including any direct, indirect, special,
|
| 159 |
+
incidental, or consequential damages of any character arising as a
|
| 160 |
+
result of this License or out of the use or inability to use the
|
| 161 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 162 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 163 |
+
other commercial damages or losses), even if such Contributor
|
| 164 |
+
has been advised of the possibility of such damages.
|
| 165 |
+
|
| 166 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 167 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 168 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 169 |
+
or other liability obligations and/or rights consistent with this
|
| 170 |
+
License. However, in accepting such obligations, You may act only
|
| 171 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 172 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 173 |
+
defend, and hold each Contributor harmless for any liability
|
| 174 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 175 |
+
of your accepting any such warranty or additional liability.
|
| 176 |
+
|
| 177 |
+
END OF TERMS AND CONDITIONS
|
README.md
CHANGED
|
@@ -1,13 +1,117 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version:
|
| 8 |
-
python_version:
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Marigold V2 CPU
|
| 3 |
+
emoji: 🌼
|
| 4 |
+
colorFrom: green
|
| 5 |
+
colorTo: indigo
|
| 6 |
sdk: gradio
|
| 7 |
+
sdk_version: 5.29.0
|
| 8 |
+
python_version: "3.10"
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
| 11 |
+
license: apache-2.0
|
| 12 |
+
fullWidth: true
|
| 13 |
+
short_description: Depth, normals, albedo on free CPU, ~4 min per output
|
| 14 |
+
startup_duration_timeout: 1h
|
| 15 |
+
models:
|
| 16 |
+
- Qwen/Qwen-Image-Edit-2509
|
| 17 |
+
- QuantStack/Qwen-Image-Edit-2509-GGUF
|
| 18 |
+
- huawei-bayerlab/marigold-v2-0
|
| 19 |
+
preload_from_hub:
|
| 20 |
+
- QuantStack/Qwen-Image-Edit-2509-GGUF Qwen-Image-Edit-2509-Q3_K_M.gguf
|
| 21 |
+
- Qwen/Qwen-Image-Edit-2509 transformer/config.json,vae/config.json,vae/diffusion_pytorch_model.safetensors
|
| 22 |
+
- huawei-bayerlab/marigold-v2-0 depth/Log-stage2/trainables.safetensors,depth/Log-layered/trainables.safetensors,normals/trainables.safetensors,albedo/trainables.safetensors
|
| 23 |
+
- huawei-bayerlab/marigold-v2-0 qwen_text_embeddings/qwen_edit_2509_qwen_depth_realimg512_prompt_embeds.pt,qwen_text_embeddings/qwen_edit_2509_qwen_depth_realimg512_prompt_mask.pt,qwen_text_embeddings/qwen_edit_2509_qwen_normals_dummy512_prompt_embeds.pt,qwen_text_embeddings/qwen_edit_2509_qwen_normals_dummy512_prompt_mask.pt,qwen_text_embeddings/qwen_edit_2509_qwen_albedo_rgb_dummy512_prompt_embeds.pt,qwen_text_embeddings/qwen_edit_2509_qwen_albedo_rgb_dummy512_prompt_mask.pt
|
| 24 |
---
|
| 25 |
|
| 26 |
+
# Marigold V2, CPU only
|
| 27 |
+
|
| 28 |
+
A CPU port of [toshas/Marigold-V2](https://huggingface.co/spaces/toshas/Marigold-V2).
|
| 29 |
+
Same weights, same single flow step, no GPU anywhere.
|
| 30 |
+
|
| 31 |
+
Live demo: https://luminia-marigold-v2-cpu.hf.space/
|
| 32 |
+
|
| 33 |
+
## What had to change
|
| 34 |
+
|
| 35 |
+
The upstream Space runs a 20B `Qwen-Image-Edit-2509` transformer through bitsandbytes NF4 on
|
| 36 |
+
an A100. Three things in that path are GPU only.
|
| 37 |
+
|
| 38 |
+
**NF4 does not exist on CPU.** bitsandbytes needs CUDA, so the weights come from a GGUF build
|
| 39 |
+
instead, loaded through `diffusers`' `GGUFQuantizationConfig` and dequantized per layer at
|
| 40 |
+
fp32. `GGUFLinear` subclasses `nn.Linear`, so peft still wraps it and the four Marigold LoRAs
|
| 41 |
+
attach exactly as before.
|
| 42 |
+
|
| 43 |
+
There is a second trap underneath it. `diffusers` reads each GGUF tensor as
|
| 44 |
+
`torch.from_numpy(tensor.data.copy())`, and `GGUFReader` hands it a memory map, so that
|
| 45 |
+
`.copy()` turns a mapped file into anonymous RAM and the load peaks near twice the file size.
|
| 46 |
+
Q2_K, the smallest build there is, peaked at 13.5 GB and was OOM-killed. `enable_gguf_mmap()`
|
| 47 |
+
drops the copy, the pages stay file-backed on the 50 GB disk, and the identical run then held
|
| 48 |
+
5.18 GB anonymous with 7.33 GB mapped. Base weights are read-only at inference, so sharing the
|
| 49 |
+
mapping is safe. Set `MARIGOLD_GGUF_MMAP=0` to go back to upstream behaviour.
|
| 50 |
+
|
| 51 |
+
Because anonymous memory no longer tracks the quant, the quant only decides how much page
|
| 52 |
+
cache it wants on top of a fixed ~5.2 GB:
|
| 53 |
+
|
| 54 |
+
The tier reports `MEMORY=16.0G`, not the 18 GB the docs quote, so the margin is about 1 GB
|
| 55 |
+
rather than 3. Anon is roughly 4.9 GB once the dead VAE decoder backup is gone:
|
| 56 |
+
|
| 57 |
+
| quant | on disk | + ~4.9 GB anon | verdict |
|
| 58 |
+
|---|---|---|---|
|
| 59 |
+
| bf16 (upstream) | 40.9 GB | n/a | needs a GPU |
|
| 60 |
+
| Q4_K_M | 13.1 GB | 18.0 GB | over 16 GB, so it re-reads from disk every pass |
|
| 61 |
+
| **Q3_K_M (default)** | **9.8 GB** | **14.7 GB** | fits 16 GB with about 1 GB spare |
|
| 62 |
+
| Q2_K | 7.2 GB | 12.1 GB | the floor, measured working |
|
| 63 |
+
|
| 64 |
+
**bf16 deadlocks on a CPU-only torch build.** The first forward hangs with no crash and no log
|
| 65 |
+
line (pytorch#75458, pytorch#111456). Everything runs fp32 here. The one exception is the
|
| 66 |
+
timestep, still divided in bf16 so the rounding matches training, then widened before it
|
| 67 |
+
reaches a kernel.
|
| 68 |
+
|
| 69 |
+
**Four checkpoints will not fit at once.** Upstream holds all four 1.85 GB `trainables.safetensors`
|
| 70 |
+
resident, which is 7.4 GB on top of the transformer. Here they are read per request and dropped,
|
| 71 |
+
so only one is in memory at a time.
|
| 72 |
+
|
| 73 |
+
## Cost
|
| 74 |
+
|
| 75 |
+
Measured on the Space itself, free tier, two shared cores, at 512 px (~196k pixels). Three runs,
|
| 76 |
+
each the first after a restart:
|
| 77 |
+
|
| 78 |
+
| run | Depth | See-through Depth | pair |
|
| 79 |
+
|---|---|---|---|
|
| 80 |
+
| 1 | 204.8 s | 218.4 s | 7.2 min |
|
| 81 |
+
| 2 | 244.8 s | 253.5 s | 8.4 min |
|
| 82 |
+
| 3 | 306.0 s | 270.0 s | 9.6 min |
|
| 83 |
+
|
| 84 |
+
So **3.5 to 5 minutes per modality, 7 to 10 minutes for the default pair**. The spread is the
|
| 85 |
+
cores being shared, not the input: runs 2 and 3 processed the same photo at the same size. Treat
|
| 86 |
+
any single number here as a sample, not a spec.
|
| 87 |
+
|
| 88 |
+
Cost is dominated by dequantizing 20B weights once per pass rather than by pixel count, so raising
|
| 89 |
+
the long side is cheaper than it looks. A cold page cache costs more than the resolution does: the
|
| 90 |
+
same forward took 749.8 s cold against 99.2 s warm on a 4-core box.
|
| 91 |
+
|
| 92 |
+
Every run prints its real per-modality timing to the container log, so read `logs/run` for the
|
| 93 |
+
number on your own input.
|
| 94 |
+
|
| 95 |
+
One job runs at a time (`default_concurrency_limit=1`), because on two shared cores a second
|
| 96 |
+
concurrent request makes both slower.
|
| 97 |
+
|
| 98 |
+
## Run it locally
|
| 99 |
+
|
| 100 |
+
Faster than the Space on any machine with more than two cores, and your images never leave it.
|
| 101 |
+
|
| 102 |
+
```bash
|
| 103 |
+
git clone https://huggingface.co/spaces/Luminia/Marigold-V2-CPU
|
| 104 |
+
cd Marigold-V2-CPU
|
| 105 |
+
pip install -r requirements.txt
|
| 106 |
+
python app.py
|
| 107 |
+
```
|
| 108 |
+
|
| 109 |
+
Set `MARIGOLD_GGUF_FILE=Qwen-Image-Edit-2509-Q8_0.gguf` for the best quality if you have the RAM
|
| 110 |
+
for it, or point `MARIGOLD_MAX_LONG_SIDE` higher than 512.
|
| 111 |
+
|
| 112 |
+
## Credit
|
| 113 |
+
|
| 114 |
+
Model and original demo by the Marigold V2 authors: [paper](https://arxiv.org/abs/2609.08084),
|
| 115 |
+
[code](https://github.com/huawei-bayerlab/marigold-v2),
|
| 116 |
+
[weights](https://huggingface.co/huawei-bayerlab/marigold-v2-0). GGUF conversion by
|
| 117 |
+
[QuantStack](https://huggingface.co/QuantStack/Qwen-Image-Edit-2509-GGUF). Apache 2.0, as upstream.
|
app.py
ADDED
|
@@ -0,0 +1,311 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import base64
|
| 2 |
+
import os
|
| 3 |
+
import time
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
|
| 6 |
+
import gradio as gr
|
| 7 |
+
from gradio_dualvision import DualVisionApp
|
| 8 |
+
from huggingface_hub import login
|
| 9 |
+
from PIL import Image
|
| 10 |
+
|
| 11 |
+
from marigoldv2_inference import (
|
| 12 |
+
BASE_MODEL_URI,
|
| 13 |
+
CHECKPOINTS,
|
| 14 |
+
GGUF_FILE,
|
| 15 |
+
MODEL_URI,
|
| 16 |
+
PROCESSING_MAX_LONG_SIDE,
|
| 17 |
+
MarigoldV2,
|
| 18 |
+
)
|
| 19 |
+
|
| 20 |
+
DEFAULT_RIGHT_MODALITY = "See-through Depth"
|
| 21 |
+
DEFAULT_MODALITIES = ["Depth", "See-through Depth"]
|
| 22 |
+
|
| 23 |
+
uri_base = os.environ.get("QWEN_IMAGE_EDIT_URI", BASE_MODEL_URI)
|
| 24 |
+
uri_model = os.environ.get("MARIGOLD_V2_URI", MODEL_URI)
|
| 25 |
+
|
| 26 |
+
if "HF_TOKEN_LOGIN" in os.environ:
|
| 27 |
+
login(token=os.environ["HF_TOKEN_LOGIN"])
|
| 28 |
+
|
| 29 |
+
model = MarigoldV2(uri_base, uri_model)
|
| 30 |
+
QUANT = GGUF_FILE.replace("Qwen-Image-Edit-2509-", "").replace(".gguf", "")
|
| 31 |
+
# Backend and throughput on one line, the guide's `## Name | CPU (int8) | ~1.5s per 1s audio`.
|
| 32 |
+
# The minutes are measured on the Space's two cores and stay prose on purpose: scaling them by
|
| 33 |
+
# thread count would be a confident lie, since 4 cores at a quarter the pixels still took 134.6 s.
|
| 34 |
+
BACKEND = f"CPU ({QUANT}, {model.threads} threads) | 3.5 to 5 min per modality on 2 cores"
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
SHIELDS_DIR = Path(__file__).parent / "assets" / "shields"
|
| 38 |
+
LINKS = [
|
| 39 |
+
("website", "Website", "https://hf.co/spaces/huawei-bayerlab/marigold-v2-web"),
|
| 40 |
+
(
|
| 41 |
+
"paper",
|
| 42 |
+
"Paper",
|
| 43 |
+
"https://arxiv.org/abs/2609.08084",
|
| 44 |
+
),
|
| 45 |
+
("code", "Code", "https://github.com/huawei-bayerlab/marigold-v2"),
|
| 46 |
+
("weights", "Weights", f"https://hf.co/huawei-bayerlab/marigold-v2-0"),
|
| 47 |
+
("follow", "Follow", "https://twitter.com/antonobukhov1"),
|
| 48 |
+
]
|
| 49 |
+
# Badges are the committed SVGs, shared with the README, model card, and website. Inlined
|
| 50 |
+
# as data URIs so the HTML header needs no static file route.
|
| 51 |
+
BADGES = {
|
| 52 |
+
name: "data:image/svg+xml;base64,"
|
| 53 |
+
+ base64.b64encode((SHIELDS_DIR / f"{name}.svg").read_bytes()).decode()
|
| 54 |
+
for name, _, _ in LINKS
|
| 55 |
+
}
|
| 56 |
+
HEADER_CSS = """
|
| 57 |
+
.mv2-header { max-width: 70vw; margin: 0 auto; display: grid; grid-template-columns: minmax(0, 6fr) minmax(0, 7fr); gap: 28px; align-items: center; color: var(--body-text-color); }
|
| 58 |
+
.mv2-identity { display: flex; flex-direction: column; gap: 6px; border-right: 1px solid var(--border-color-primary); padding-right: 28px; }
|
| 59 |
+
.mv2-titles { display: flex; flex-wrap: wrap; align-items: baseline; gap: 4px 14px; }
|
| 60 |
+
.mv2-title { font-family: 'Instrument Serif', Georgia, serif; font-weight: 400; font-size: 36px; line-height: 40px; color: var(--body-text-color); }
|
| 61 |
+
.mv2-subtitle { font-size: 12px; line-height: 16px; letter-spacing: 0.16em; text-transform: uppercase; color: var(--body-text-color-subdued); }
|
| 62 |
+
.mv2-links { display: flex; flex-wrap: wrap; align-items: center; gap: 6px 8px; margin-top: 8px; }
|
| 63 |
+
.mv2-link-group { display: inline-flex; gap: inherit; white-space: nowrap; }
|
| 64 |
+
.mv2-links a { display: inline-flex; line-height: 0; }
|
| 65 |
+
.mv2-links img { height: 22px; width: auto; transition: transform 0.15s ease; }
|
| 66 |
+
.mv2-links a:hover img { transform: translateY(-1px); }
|
| 67 |
+
.mv2-intro { margin: 0; font-size: 14px; line-height: 20px; }
|
| 68 |
+
.mv2-cost { font-weight: 600; }
|
| 69 |
+
@media (max-width: 900px) {
|
| 70 |
+
.mv2-header { max-width: none; grid-template-columns: minmax(0, 1fr); gap: 10px; }
|
| 71 |
+
.mv2-identity { display: grid; grid-template-columns: max-content minmax(0, 1fr); column-gap: 16px; row-gap: 4px; align-items: center; border-right: 0; border-bottom: 1px solid var(--border-color-primary); padding: 0 0 10px 0; }
|
| 72 |
+
.mv2-titles { display: contents; }
|
| 73 |
+
.mv2-title { grid-column: 1; grid-row: 1; font-size: 30px; line-height: 34px; }
|
| 74 |
+
.mv2-links { grid-column: 2; grid-row: 1; justify-content: flex-end; gap: 6px 8px; margin: 0; }
|
| 75 |
+
.mv2-subtitle { grid-column: 1 / -1; grid-row: 2; }
|
| 76 |
+
.mv2-intro { font-size: 13px; line-height: 18px; }
|
| 77 |
+
}
|
| 78 |
+
@media (max-width: 529px) {
|
| 79 |
+
.mv2-identity { grid-template-columns: minmax(0, 1fr) max-content; }
|
| 80 |
+
.mv2-subtitle { grid-column: 1; font-size: 10px; letter-spacing: 0.12em; }
|
| 81 |
+
.mv2-links { grid-row: 1 / span 2; align-self: center; flex-direction: column; align-items: flex-end; gap: 6px; }
|
| 82 |
+
.mv2-link-group { gap: 6px; }
|
| 83 |
+
.mv2-links img { height: 20px; }
|
| 84 |
+
}
|
| 85 |
+
@media (max-width: 370px) {
|
| 86 |
+
.mv2-title { font-size: 26px; line-height: 30px; }
|
| 87 |
+
}
|
| 88 |
+
/* Each modality group centered inside its half of the selector row, with a
|
| 89 |
+
hairline on the midline separating the left pane's choices from the right's. */
|
| 90 |
+
#selector_right > .wrap,
|
| 91 |
+
div:has(> #selector_right) > fieldset:first-child > .wrap { justify-content: center; }
|
| 92 |
+
div:has(> #selector_right) { position: relative; }
|
| 93 |
+
div:has(> #selector_right)::before {
|
| 94 |
+
content: "";
|
| 95 |
+
position: absolute;
|
| 96 |
+
top: 3px;
|
| 97 |
+
bottom: 3px;
|
| 98 |
+
left: 50%;
|
| 99 |
+
width: 1px;
|
| 100 |
+
background: var(--border-color-primary);
|
| 101 |
+
}
|
| 102 |
+
/* Footer links wrap as whole items; outside .contain, so plain rules only. */
|
| 103 |
+
footer.svelte-1byz9vf { flex-wrap: wrap; justify-content: center; gap: 2px 24px; }
|
| 104 |
+
footer.svelte-1byz9vf a { white-space: nowrap; }
|
| 105 |
+
footer.svelte-1byz9vf .divider { display: none; }
|
| 106 |
+
/* Examples block as wide as the drop zone, tiles in a full rectangle (column counts that divide 12). */
|
| 107 |
+
.block:has(> .gallery.svelte-p5q82i) { max-width: 70vw; margin-left: auto; margin-right: auto; }
|
| 108 |
+
.gallery.svelte-p5q82i.svelte-p5q82i.svelte-p5q82i { display: grid; grid-template-columns: repeat(6, minmax(0, 1fr)); gap: var(--spacing-lg); }
|
| 109 |
+
.gallery-item .gallery.svelte-a9zvka { width: 100%; min-width: 0; max-width: none; height: auto; min-height: 0; aspect-ratio: 1; }
|
| 110 |
+
.gallery.svelte-p5q82i .gallery-item img { width: 100%; min-width: 0; max-width: none; height: 100%; min-height: 0; object-fit: cover; }
|
| 111 |
+
@media (max-width: 900px) {
|
| 112 |
+
.block:has(> .gallery.svelte-p5q82i) { max-width: none; }
|
| 113 |
+
.sliderrow .slider { max-width: none; width: 100%; }
|
| 114 |
+
.slider .wrap.half-wrap { width: 100%; }
|
| 115 |
+
}
|
| 116 |
+
@media (max-width: 829px) { .gallery.svelte-p5q82i.svelte-p5q82i.svelte-p5q82i { grid-template-columns: repeat(4, minmax(0, 1fr)); } }
|
| 117 |
+
@media (max-width: 529px) { .gallery.svelte-p5q82i.svelte-p5q82i.svelte-p5q82i { grid-template-columns: repeat(3, minmax(0, 1fr)); } }
|
| 118 |
+
/* Keyboard focus stays visible wherever the theme above removes an outline. */
|
| 119 |
+
:where(button, input, textarea, select, [role="button"], [tabindex]):focus-visible {
|
| 120 |
+
outline: 2px solid var(--color-accent) !important;
|
| 121 |
+
outline-offset: 2px;
|
| 122 |
+
}
|
| 123 |
+
@media (prefers-reduced-motion: reduce) {
|
| 124 |
+
*, *::before, *::after { animation-duration: 0.01ms !important; transition-duration: 0.01ms !important; }
|
| 125 |
+
}
|
| 126 |
+
"""
|
| 127 |
+
HEADER_HEAD = '<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Instrument+Serif&display=swap">'
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
class MarigoldV2App(DualVisionApp):
|
| 131 |
+
def make_header(self):
|
| 132 |
+
links = "".join(
|
| 133 |
+
'<span class="mv2-link-group">'
|
| 134 |
+
+ "".join(
|
| 135 |
+
f'<a href="{href}" target="_blank" rel="noopener noreferrer">'
|
| 136 |
+
f'<img src="{BADGES[name]}" alt="{label}"></a>'
|
| 137 |
+
for name, label, href in group
|
| 138 |
+
)
|
| 139 |
+
+ "</span>"
|
| 140 |
+
for group in (LINKS[:3], LINKS[3:])
|
| 141 |
+
)
|
| 142 |
+
gr.HTML(
|
| 143 |
+
f"""
|
| 144 |
+
<div class="mv2-header">
|
| 145 |
+
<div class="mv2-identity">
|
| 146 |
+
<div class="mv2-titles">
|
| 147 |
+
<div class="mv2-title">Marigold V2</div>
|
| 148 |
+
<div class="mv2-subtitle">{BACKEND}</div>
|
| 149 |
+
</div>
|
| 150 |
+
<div class="mv2-links remove-elements">{links}</div>
|
| 151 |
+
</div>
|
| 152 |
+
<p class="mv2-intro">
|
| 153 |
+
A CPU port of the authors' <i>Marigold V2</i> demo, no GPU anywhere. Each modality is one
|
| 154 |
+
pass through a 20B transformer, so
|
| 155 |
+
<span class="mv2-cost">the default Depth plus See-through Depth pair takes 7 to 10 minutes</span>
|
| 156 |
+
at 512 px on two shared cores. Set the modalities in Settings first, because the run starts
|
| 157 |
+
the moment you drop a photo. One job runs at a time, so you may wait behind another visitor.
|
| 158 |
+
Examples ignore Settings and take the same 7 minutes on their first click, then they are cached.
|
| 159 |
+
</p>
|
| 160 |
+
</div>
|
| 161 |
+
""",
|
| 162 |
+
padding=False,
|
| 163 |
+
)
|
| 164 |
+
|
| 165 |
+
def make_advanced_settings(self):
|
| 166 |
+
"""Open the settings panel and put the cost in its header.
|
| 167 |
+
|
| 168 |
+
DualVisionApp starts the first run on upload, with no Start button, so a dropped photo
|
| 169 |
+
commits seven minutes immediately. Upstream hides these controls and the Apply button
|
| 170 |
+
behind `gr.Accordion(..., open=False)`, which means the labels carrying the cost are
|
| 171 |
+
never read before that happens. This mirrors upstream's bookkeeping, so re-check it if
|
| 172 |
+
the pinned gradio_dualvision commit ever moves.
|
| 173 |
+
"""
|
| 174 |
+
with gr.Accordion(
|
| 175 |
+
f"Settings: {len(DEFAULT_MODALITIES)} modalities at {PROCESSING_MAX_LONG_SIDE} px, 7 to 10 min",
|
| 176 |
+
open=True,
|
| 177 |
+
visible=self.advanced_settings_visible,
|
| 178 |
+
):
|
| 179 |
+
user_components = self.build_user_components()
|
| 180 |
+
with gr.Row():
|
| 181 |
+
btn_clear, btn_submit = self.make_buttons()
|
| 182 |
+
self.input_keys = list(user_components.keys())
|
| 183 |
+
self.input_cls = list(v.__class__ for v in user_components.values())
|
| 184 |
+
self.input_kwargs = [
|
| 185 |
+
{k: v for k, v in c.constructor_args.items() if k != "value"}
|
| 186 |
+
for c in user_components.values()
|
| 187 |
+
]
|
| 188 |
+
return user_components, btn_clear, btn_submit
|
| 189 |
+
|
| 190 |
+
def build_user_components(self):
|
| 191 |
+
return {
|
| 192 |
+
"modalities": gr.CheckboxGroup(
|
| 193 |
+
choices=list(CHECKPOINTS),
|
| 194 |
+
value=DEFAULT_MODALITIES,
|
| 195 |
+
label="Modalities to compute (each is one full 20B pass, 3.5 to 5 min at 512 px)",
|
| 196 |
+
info="Depth and See-through Depth share one colour scale when both are on.",
|
| 197 |
+
),
|
| 198 |
+
"max_long_side": gr.Slider(
|
| 199 |
+
minimum=256,
|
| 200 |
+
maximum=1024,
|
| 201 |
+
step=128,
|
| 202 |
+
value=PROCESSING_MAX_LONG_SIDE,
|
| 203 |
+
label=f"Max long side ({PROCESSING_MAX_LONG_SIDE} px default, 3.5 to 5 min per modality)",
|
| 204 |
+
info=(
|
| 205 |
+
"Only downscales, so raising it past your photo's long side does nothing. "
|
| 206 |
+
"Time goes on dequantizing the 20B weights rather than on pixels, so raising "
|
| 207 |
+
"this costs much less than the pixel count suggests, and lowering it saves little."
|
| 208 |
+
),
|
| 209 |
+
),
|
| 210 |
+
}
|
| 211 |
+
|
| 212 |
+
def process_components(
|
| 213 |
+
self, image_in, modality_selector_left, modality_selector_right, **kwargs
|
| 214 |
+
):
|
| 215 |
+
if modality_selector_right is None:
|
| 216 |
+
modality_selector_right = DEFAULT_RIGHT_MODALITY
|
| 217 |
+
return super().process_components(
|
| 218 |
+
image_in, modality_selector_left, modality_selector_right, **kwargs
|
| 219 |
+
)
|
| 220 |
+
|
| 221 |
+
def process(self, image_in: Image.Image, **kwargs):
|
| 222 |
+
modalities = kwargs.get("modalities")
|
| 223 |
+
# An explicitly emptied box is a mistake worth naming. The examples path passes no
|
| 224 |
+
# kwargs at all, so None still has to fall through to the defaults.
|
| 225 |
+
if modalities is not None and not modalities:
|
| 226 |
+
raise gr.Error(
|
| 227 |
+
"Pick at least one modality in Settings. Depth alone is the cheapest at "
|
| 228 |
+
"3.5 to 5 min.",
|
| 229 |
+
duration=None,
|
| 230 |
+
)
|
| 231 |
+
modalities = modalities or DEFAULT_MODALITIES
|
| 232 |
+
max_long_side = int(kwargs.get("max_long_side") or PROCESSING_MAX_LONG_SIDE)
|
| 233 |
+
|
| 234 |
+
progress = gr.Progress()
|
| 235 |
+
began = time.perf_counter()
|
| 236 |
+
# One forward per modality, 60 transformer blocks inside each, so hooking the blocks
|
| 237 |
+
# moves the bar every ~3.5 s with a real ETA instead of twice in seven minutes.
|
| 238 |
+
# These hooks mutate the shared model, and default_concurrency_limit=1 does NOT prevent
|
| 239 |
+
# a second run entering (it is per handler, and there are three). MarigoldV2 holds its
|
| 240 |
+
# own lock, so only the run that owns it is inside model(). The hooks still must come
|
| 241 |
+
# off in finally or they leak onto the next run.
|
| 242 |
+
blocks = model.transformer.transformer_blocks
|
| 243 |
+
total = len(modalities) * len(blocks)
|
| 244 |
+
state = {"done": 0, "name": modalities[0]}
|
| 245 |
+
|
| 246 |
+
def tick(*_):
|
| 247 |
+
state["done"] += 1
|
| 248 |
+
elapsed = time.perf_counter() - began
|
| 249 |
+
fraction = state["done"] / total
|
| 250 |
+
left = elapsed / fraction - elapsed
|
| 251 |
+
progress(
|
| 252 |
+
fraction,
|
| 253 |
+
desc=f"{state['name']} [+{elapsed / 60:.1f} min, about {left / 60:.0f} min left]",
|
| 254 |
+
)
|
| 255 |
+
|
| 256 |
+
handles = [block.register_forward_hook(tick) for block in blocks]
|
| 257 |
+
try:
|
| 258 |
+
outputs = model(
|
| 259 |
+
image_in,
|
| 260 |
+
modalities=modalities,
|
| 261 |
+
max_long_side=max_long_side,
|
| 262 |
+
progress_cb=lambda fraction, message: state.update(name=message),
|
| 263 |
+
)
|
| 264 |
+
except gr.Error:
|
| 265 |
+
raise
|
| 266 |
+
except Exception as exc:
|
| 267 |
+
raise gr.Error(
|
| 268 |
+
f"Run failed at {max_long_side} px with {len(modalities)} modalities: {exc}. "
|
| 269 |
+
f"Try Max long side 512 and one modality.",
|
| 270 |
+
duration=None,
|
| 271 |
+
) from exc
|
| 272 |
+
finally:
|
| 273 |
+
for handle in handles:
|
| 274 |
+
handle.remove()
|
| 275 |
+
progress(1.0, desc=f"done in {(time.perf_counter() - began) / 60:.1f} min")
|
| 276 |
+
# One entry per build_user_components key, or DualVisionApp rejects the result AFTER
|
| 277 |
+
# the whole run has completed. The values come back as the restored control state.
|
| 278 |
+
return outputs, {"modalities": modalities, "max_long_side": max_long_side}
|
| 279 |
+
|
| 280 |
+
|
| 281 |
+
with MarigoldV2App(
|
| 282 |
+
title="Marigold V2 CPU",
|
| 283 |
+
key_original_image="Input",
|
| 284 |
+
examples_path="example_images",
|
| 285 |
+
examples_per_page=12,
|
| 286 |
+
# NOT False. DualVisionApp hardcodes cache_examples=True and forwards this to cache_mode,
|
| 287 |
+
# so False is not "off", it is an invalid mode that falls back to caching every example
|
| 288 |
+
# eagerly at startup. At CPU speed that never finishes. "lazy" computes an example on its
|
| 289 |
+
# first click and caches it from then on.
|
| 290 |
+
examples_cache="lazy",
|
| 291 |
+
left_selector_visible=True,
|
| 292 |
+
advanced_settings_visible=True,
|
| 293 |
+
squeeze_canvas=True,
|
| 294 |
+
# Default 75vh puts the examples and settings below the fold, worse with the panel open.
|
| 295 |
+
squeeze_viewport_height_pct=58,
|
| 296 |
+
spaces_zero_gpu_enabled=False,
|
| 297 |
+
css=HEADER_CSS,
|
| 298 |
+
head=HEADER_HEAD,
|
| 299 |
+
) as demo:
|
| 300 |
+
# Cores are the bottleneck on the 2-vCPU tier, so a second concurrent job makes both slower.
|
| 301 |
+
demo.queue(
|
| 302 |
+
api_open=False,
|
| 303 |
+
# Per concurrency group, and gradio derives the group from str(id(fn)), so this does
|
| 304 |
+
# not by itself stop three handlers running at once. MarigoldV2's lock is what does.
|
| 305 |
+
default_concurrency_limit=1,
|
| 306 |
+
# Unbounded, visitor six waits 40 minutes staring at a queue position. Refuse instead.
|
| 307 |
+
max_size=4,
|
| 308 |
+
).launch(
|
| 309 |
+
show_error=True,
|
| 310 |
+
ssr_mode=False,
|
| 311 |
+
)
|
assets/shields/code.svg
ADDED
|
|
assets/shields/follow.svg
ADDED
|
|
assets/shields/paper.svg
ADDED
|
|
assets/shields/website.svg
ADDED
|
|
assets/shields/weights.svg
ADDED
|
|
example_images/00_lobby.jpg
ADDED
|
Git LFS Details
|
example_images/01_harbor_window.jpg
ADDED
|
Git LFS Details
|
example_images/02_shop_window.jpg
ADDED
|
Git LFS Details
|
example_images/03_garden_bed.jpg
ADDED
|
Git LFS Details
|
example_images/04_brick_corridor.jpg
ADDED
|
Git LFS Details
|
example_images/05_forest_stump.jpg
ADDED
|
Git LFS Details
|
example_images/06_cows.jpg
ADDED
|
Git LFS Details
|
example_images/07_lakeside_window.jpg
ADDED
|
Git LFS Details
|
example_images/08_thistle.jpg
ADDED
|
Git LFS Details
|
example_images/09_cat_garden.jpg
ADDED
|
Git LFS Details
|
example_images/10_ornate_room.jpg
ADDED
|
Git LFS Details
|
example_images/11_cats_sleeping.jpg
ADDED
|
Git LFS Details
|
example_images/12_church_ironwork.jpg
ADDED
|
Git LFS Details
|
example_images/13_glass_door.jpg
ADDED
|
Git LFS Details
|
example_images/14_dogs.jpg
ADDED
|
Git LFS Details
|
example_images/15_kitten.jpg
ADDED
|
Git LFS Details
|
example_images/16_hallway.jpg
ADDED
|
Git LFS Details
|
example_images/17_lynx.jpg
ADDED
|
Git LFS Details
|
example_images/18_dog_in_car.jpg
ADDED
|
Git LFS Details
|
example_images/19_burano.jpg
ADDED
|
Git LFS Details
|
example_images/20_sculpture_archer.jpg
ADDED
|
Git LFS Details
|
example_images/21_lion.jpg
ADDED
|
Git LFS Details
|
example_images/22_portrait_blossoms.jpg
ADDED
|
Git LFS Details
|
example_images/23_glass_facade.jpg
ADDED
|
Git LFS Details
|
example_images/24_tulips.jpg
ADDED
|
Git LFS Details
|
example_images/25_train.jpg
ADDED
|
Git LFS Details
|
example_images/26_bull.jpg
ADDED
|
Git LFS Details
|
example_images/27_church_nave.jpg
ADDED
|
Git LFS Details
|
example_images/28_snow_pines.jpg
ADDED
|
Git LFS Details
|
example_images/29_rooftop_city.jpg
ADDED
|
Git LFS Details
|
example_images/30_heron.jpg
ADDED
|
Git LFS Details
|
example_images/31_lilies.jpg
ADDED
|
Git LFS Details
|
example_images/32_goat.jpg
ADDED
|
Git LFS Details
|
example_images/33_cat_fluffy.jpg
ADDED
|
Git LFS Details
|
example_images/34_van_field.jpg
ADDED
|
Git LFS Details
|
example_images/35_garden_palms.jpg
ADDED
|
Git LFS Details
|
example_images/36_sculpture_bust.jpg
ADDED
|
Git LFS Details
|
example_images/37_vase_branches.jpg
ADDED
|
Git LFS Details
|
example_images/38_portrait_fence.jpg
ADDED
|
Git LFS Details
|
example_images/39_penguins.jpg
ADDED
|
Git LFS Details
|