LuffyTheFox

/

Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-GGUF

Like 689

Found a bug in .safetensors version of my pipeline. I don't have motivation to continue debugging process for big models. What's the point for me to process large models (DeepSeek-V4.1-Flash) that I will never be able to run with current resources? DeepSeek-V4.1-Flash-Genesis project is frozen. Perhaps I will change my decision in the future, but for now I don’t see any point in continuing doing anything with DeepSeek.

@LuffyTheFox

Hello!

Have you ever considered "REAP" or possibly stripping layers/weights out and then running a healing LORA that you then merge to it?
It was just a thought to possibly resolve the large model issue as it is indeed frustrating to watch these GIANT models get released that no one
can run unless they have a damn server rack in their room.

• EnlistedGhost (Jon Z)

@LuffyTheFox

Hello!

Have you ever considered "REAP" or possibly stripping layers/weights out and then running a healing LORA that you then merge to it?
It was just a thought to possibly resolve the large model issue as it is indeed frustrating to watch these GIANT models get released that no one
can run unless they have a damn server rack in their room.

• EnlistedGhost (Jon Z)

Nope I not checked that, because it would damage data in the model and make it feel lobotomized.

@LuffyTheFox

It would if it's done with a baseball bat and not healed with an ALORA that is merged to it afterwards...
Tell you what. Let me prune the DeepSeek model and provide it to you. Who knows- maybe it'll be something you like.

If not? Then I will gladly offer my sincerest apologies for wasting your time.

I'd just hate to see you give up. You have seriously provided fantastic models to the community and I use your specific Qwen model for a friend of mine that would otherwise never be able to have the inference quality they currently acheive thanks to your hard work.

• EnlistedGhost (Jon Z)

@LuffyTheFox

It would if it's done with a baseball bat and not healed with an ALORA that is merged to it afterwards...
Tell you what. Let me prune the DeepSeek model and provide it to you. Who knows- maybe it'll be something you like.

If not? Then I will gladly offer my sincerest apologies for wasting your time.

I'd just hate to see you give up. You have seriously provided fantastic models to the community and I use your specific Qwen model for a friend of mine that would otherwise never be able to have the inference quality they currently acheive thanks to your hard work.

• EnlistedGhost (Jon Z)

Thank you very much for kind words. Okay, create a pruned version of DeepSeek V4.1 Flash model and upload it to Hugging Face in .safetensors or GGUF format. Share to me link and I will take a look.

Can we get some ternary releases?

@LuffyTheFox

Will do!

I have one more question and I will leave you alone until I have the model provided to you:
Would you mind both a DeepSeek-4.1-Flash pruned and a DeepSeek-4-Flash-Vision pruned?

I ask this because DeepSeek-4-Flash-Vision is a very capable model and would be higher quality regardless of pruning methodology used.
If not- no worries I will focus solely on 4.1

Thank you so much for entertaining my concept and for your time you have taken responding to me!

• EnlistedGhost (Jon Zaretsky)

@LuffyTheFox

Will do!

I have one more question and I will leave you alone until I have the model provided to you:
Would you mind both a DeepSeek-4.1-Flash pruned and a DeepSeek-4-Flash-Vision pruned?

I ask this because DeepSeek-4-Flash-Vision is a very capable model and would be higher quality regardless of pruning methodology used.
If not- no worries I will focus solely on 4.1

Thank you so much for entertaining my concept and for your time you have taken responding to me!

• EnlistedGhost (Jon Zaretsky)

Since I am 3D character artist I need only DeepSeek-4.1-Flash-Vision pruned. In addition, many people are involved in agentic workflows, and they need vision too for computer usage.

Can we get some ternary releases?

People with computing resources can try to do it after Noise Gate supression in DeepSeek V4.1 Flash Vision pruned tensor weight spectrum, because otherwise ternary output quality will be garbage.

@LuffyTheFox

Understood! A pruned DeepSeek-4.1-Flash is what I will create.

Please allow me 48 hours (I do not expect it to take so long, but I wanted to ask for this amount of time just in case I hit any roadblocks or hurdles while producing it).
Thanks btw, I really appreciate the opportunity and your willingness to entertain this possible avenue.

I will report back asap!

• EnlistedGhost (Jon Zaretsky)

PS: I'm using a proprietary pruning method. This provides me an excellent stress test. If it works out, you wind up with a usable model and I would have validated several days worth of effort 😃 ... The point being; If my math and theory is correct- the model will NOT suffer damage or only suffer ~0.X% damage which will be negligable. Off I go!

@LuffyTheFox Quick Update:

So far I have managed to do a test-run of my pruning method before applying it to such a large model as DeepSeek-4.1-Flash.

I was very pleasantly surprised that so far I have been successfull in downscaling a model by -32% (Dense Weight Model, but I easily will next apply this to the DeepSeek-4.1-Flash MoE). The resulting model after pruning/downscaling it by 32% was still coherent and capable of quite decent quality responses for the first paragraph or two of a response. Of course, it will require a small LoRA to restore proper capabilities but this is already MUCH better than I had expected the theory to work.

As such, I am very excited to apply this to your desired model over the course of the next day. I'll stay in touch- wish me luck!

• EnlistedGhost (Jon Zaretsky)

@LuffyTheFox Quick Update:

So far I have managed to do a test-run of my pruning method before applying it to such a large model as DeepSeek-4.1-Flash.

I was very pleasantly surprised that so far I have been successfull in downscaling a model by -32% (Dense Weight Model, but I easily will next apply this to the DeepSeek-4.1-Flash MoE). The resulting model after pruning/downscaling it by 32% was still coherent and capable of quite decent quality responses for the first paragraph or two of a response. Of course, it will require a small LoRA to restore proper capabilities but this is already MUCH better than I had expected the theory to work.

As such, I am very excited to apply this to your desired model over the course of the next day. I'll stay in touch- wish me luck!

• EnlistedGhost (Jon Zaretsky)

Wow congratz. Good luck with it 😉

@LuffyTheFox Quick Update:

So far I have managed to do a test-run of my pruning method before applying it to such a large model as DeepSeek-4.1-Flash.

I was very pleasantly surprised that so far I have been successfull in downscaling a model by -32% (Dense Weight Model, but I easily will next apply this to the DeepSeek-4.1-Flash MoE). The resulting model after pruning/downscaling it by 32% was still coherent and capable of quite decent quality responses for the first paragraph or two of a response. Of course, it will require a small LoRA to restore proper capabilities but this is already MUCH better than I had expected the theory to work.

As such, I am very excited to apply this to your desired model over the course of the next day. I'll stay in touch- wish me luck!

• EnlistedGhost (Jon Zaretsky)

Important moment. My Genesis pipeline after almost a year of work is calibrated only for GGUF now. So, after processing please upload mixed precision Q8_0 GGUF first using the same quantization type (F16 and F32) for other tensors as DeepSeek team already did for original model, it's really popular format for LLMs, because it's easy to setup to make things up and running for people.

After GGUF upload .safetensors too if you can.

@LuffyTheFox

I absolutely agree and prefer to use GGUF format myself as well!

No problem, I originally had planned: download safetensors --> prune model --> heal model --> convert to F16 GGUF --> dynamic quant (mixed precision) Q8 GGUF

It's definitely going to take most of my day (which has just rolled over to 01:20 (am), and might possibly bleed over a little into the next day. (I'm not certain, it should be much faster than that but I would rather not quote a sooner time and then fail to produce a result within that limit).

Anyway, I'll definitely post with progress and results 😃

• EnlistedGhost (Jon Zaretsky)

@LuffyTheFox

Good day to you!

Quick update: Currently I have ran extensive testing on the best iteration of my downscaling/pruning method and have a agreeable solution for what I will use. I am waiting for all the safetensors to finish downloading for DeepSeek-4.1-Flash (damn... it's massive... I had to bust out a dedicated drive for this- hahahaha).

Once that finishes up it will take me easily a couple, to possibly a few hours to to let my system churn through the math and apply its decisions to the model layers, weights, and keys. Afterwhich I will make the biggest swap partition on my NVME m.2 I have ever done (lol, I normally don't use swap partitions) and I'll run a raw damage assessment inference with the modified model after converting it to GGUF and quantizing it down so that I can realistically run it. If the results look as good as the tests I did with the few dense models I tried my method with- I will then proceed to running a healing post-train/fine-tune to restore any damage that may have occured.

The healing LoRA post-training/fine-tune will regrettably take at least a day with my modest hardware- but I'll let it run.

Okay, I'm off to continue working on it

• EnlistedGhost (Jon Zaretsky)

@LuffyTheFox

Good day to you!

Quick update: Currently I have ran extensive testing on the best iteration of my downscaling/pruning method and have a agreeable solution for what I will use. I am waiting for all the safetensors to finish downloading for DeepSeek-4.1-Flash (damn... it's massive... I had to bust out a dedicated drive for this- hahahaha).

Once that finishes up it will take me easily a couple, to possibly a few hours to to let my system churn through the math and apply its decisions to the model layers, weights, and keys. Afterwhich I will make the biggest swap partition on my NVME m.2 I have ever done (lol, I normally don't use swap partitions) and I'll run a raw damage assessment inference with the modified model after converting it to GGUF and quantizing it down so that I can realistically run it. If the results look as good as the tests I did with the few dense models I tried my method with- I will then proceed to running a healing post-train/fine-tune to restore any damage that may have occured.

The healing LoRA post-training/fine-tune will regrettably take at least a day with my modest hardware- but I'll let it run.

Okay, I'm off to continue working on it

• EnlistedGhost (Jon Zaretsky)

Good luck with it. Don't rush and take your time😃. Some people are already waiting this model.

@LuffyTheFox

Understood! I am making steady progress!

Of course, downloading the model alone took an entire day even with 50MB/s down speed (kept having my connection "blip" and sever a large download chunk... Sigh... But that's not important lol I got that handled.

While I was downloading the model and altering my novel (prototye) pruning method- I have continued to attempt to refine it to properly (well, as properly as possibly) "stitch back together" the layers that are left so as to shave off post-training/finetuning time as much as possibly. I admittedly don't want to be sitting here staring at my computer for two days commenting on it like a sports announcer (>.<)

I'll Pop back into this convo in like 2-4 hours but if the testing goes well maybe I'll show some spoilers while the gigantic model gets modified and saved onto my drive(s). I still can't get over what a giant whale this thing is! But hey, this is also the most fun I've had in a while- and I like a challenge. 😃

• EnlistedGhost (Jon Zaretsky)

Hey Luffy!

My apologies for taking a day before getting back to you here on this thread.

So far I have been testing my novel pruning method code like a posessed maniac (lol, sorry trying to use a little humor). There were a lot of things that sounded like good ideas in theory but were terrible in practice as I continued to run pre-DeepSeek-4.1-Flash testing on smaller models in order to acheive the highest zero-shot quality possible post-pruning.

And even though I make habitual almost obsessive backups; I nearly lost the best iteration due to constant changing and tuning of the working files I was developing.

It's not an excuse for how long things have taken- merely an explanation.

Seriously though: The DeepSeek-4.1-Flash prune attempt is going to start shortly. At least within the next 12 hours and I will let you know how everything goes. Sorry for all the waiting and once again, thank you for providing the opportunity.

• EnlistedGhost (Jon Zaretsky)

Hey Luffy!

My apologies for taking a day before getting back to you here on this thread.

So far I have been testing my novel pruning method code like a posessed maniac (lol, sorry trying to use a little humor). There were a lot of things that sounded like good ideas in theory but were terrible in practice as I continued to run pre-DeepSeek-4.1-Flash testing on smaller models in order to acheive the highest zero-shot quality possible post-pruning.

And even though I make habitual almost obsessive backups; I nearly lost the best iteration due to constant changing and tuning of the working files I was developing.

It's not an excuse for how long things have taken- merely an explanation.

Seriously though: The DeepSeek-4.1-Flash prune attempt is going to start shortly. At least within the next 12 hours and I will let you know how everything goes. Sorry for all the waiting and once again, thank you for providing the opportunity.

• EnlistedGhost (Jon Zaretsky)

Hello. No worries. No need to rush. Take your time 😉

There is no need to apologize, either. What matters most to people is getting a high-quality result that will work better than Qwen3.8-27B, and avoid buying multiple DGX Sparks / GPUs to run DeepSeek V4.1 Flash locally.

@LuffyTheFox

Wow, thank you so much for your patience and understanding!
I'm honestly not used to that.

Still, I wanted to provide a progress update really quick before I continue working:

• I had to replace a componenet in my Lenovo P920, it literally took quite a bit of my day yesterday but was worth it. (She's an older rig, but a capable one)
• I'm making final adjustments for MoE pruning specifically and then will begin providing files, quants, and metrics regarding the work that is concluded.
• Compiled a very recent version of Llama.cpp (b11552) in order to provide best possible results during this excursion
• Installed a 1TB m.2 drive dedicated just to this project and model work

Thank you again, very much. I look forward to posting results.

• EnlistedGhost (Jon Zaretsky)