九原客 (@9hills) “Qwen3.8-27B 本地部署指南 1. Q4 基本没有质量损失，甚至可以用Q3 2. 用Unsloth的GGUF” — TopicDigg
九原客
@9hills
加入 February 2010
0 正在关注 0 粉丝
Qwen3.8-27B 本地部署指南
1. Q4 基本没有质量损失，甚至可以用Q3
2. 用Unsloth的GGUF。
3. 思考开low就行了。
4. 开 dflash2
Main result. At xhigh every quant landed between 88.0 and 90.0% pass@1 on the full suite:
AWQ INT4: 90.0% NVFP4: 89.3% GGUF Q4_K_M: 89.3% FP8: 88.7% NInfer: 88.0%
Yes, the 4 bit quants scored above the FP8 baseline. McNemar says its a statistical tie, first and last place differ by three tasks out of 150.
I started this run to show you how quantization eats quality. There is nothing to show. The gap between quants is smaller than the gap between reasoning presets.
显示更多
