Unified local model
Qwen3.8-27B-Perplexing-Pegasus-Heretic
An Apache-2.0 abliterated release of AveLabs’ Qwen3.8-27B-Perplexing-Pegasus (Qwen3.8-27B + Perplexity pplx-decider-v1-27b). It packs vision, thinking, tool calling, structured decision logprobs and all 15 Multi-Token Prediction tensors into one checkpoint — down to 16 GB consumer GPUs.
Why it matters
Peak local agent setups usually juggle two or three models in memory at once: a vision model to inspect the page, a typed-decision router for scoring, and an uncensored model that won’t stall on harmless workflows. Unifying them into one 27B checkpoint lets a single 16 GB or 24 GB GPU run the whole stack.
Built for 16 GB and 24 GB machines
For 16 GB cards (such as RTX 4060 Ti 16 GB, RTX 5060 Ti 16 GB, RTX 4080, RTX 5070 Ti or RTX 5080), the GGUF repo provides imatrix quantizations calibrated directly on the abliterated BF16 weights across general knowledge, reasoning, code, multimodal chat and pplx-decider prompts.
IQ3_XXS (11.09 GB) plus the F16 vision projector mmproj (0.86 GB) totals 11.95 GB, leaving ~4 GB on a 16 GB GPU for KV cache and runtime overhead; Q3_K_M (13.00 GB) also fits in 16 GB with tighter context. On 24 GB GPUs, Q4_K_M (15.84 GB) is the recommended default. Across IQ4_XS, Q3_K_M and IQ3_XXS, the MTP layer (blk.64) is explicitly kept at Q8_0.
Abliteration without breaking typed decisions
We ran heretic-llm Arbitrary-Rank Ablation (ARA, lora_rank=50, preserve_row_magnitudes=true, full BF16 precision, 150 Optuna TPE trials) across layers 16–52, leaving lm_head.weight untouched (cosine similarity 1.000001), preserving the decision/ readout (decision_config.json, readout.safetensors) and keeping all 15 mtp.* tensors intact.
Trial 7 cut classifier refusals from 102/104 on unmodified Pegasus to 6/104 (and 0/8 on refusal probes with thinking off) at 0.0618 KL divergence, while keeping 100.0% (30/30) typed-decision agreement and 90.0% → 90.0% probe accuracy.
Four hardware targets under Apache-2.0
Alongside llama.cpp, LM Studio and Ollama (GGUF), the release ships AutoRound W4A16 INT4 (18.21 GB) for vLLM on Intel Arc Pro B70 (32 GB XPU) and NVIDIA CUDA GPUs, mixed NVFP4 + FP8 (22.99 GB) with unquantized BF16 lm_head, vision and MTP tensors for NVIDIA Blackwell (sm_100 / sm_120), and the full BF16 reference checkpoint.
Formats, VRAM targets and verification
| Build | Size | Target hardware | Verification & notes |
|---|---|---|---|
GGUF IQ3_XXS + mmproj | 11.09 + 0.86 GB | 16 GB GPU (llama.cpp / LM Studio) | imatrix; blk.64 MTP at Q8_0 |
GGUF Q3_K_M + mmproj | 13.00 + 0.86 GB | 16 GB GPU (llama.cpp / LM Studio) | imatrix; blk.64 MTP at Q8_0 |
GGUF IQ4_XS + mmproj | 14.69 + 0.86 GB | 16–24 GB GPU / offload | imatrix; blk.64 MTP at Q8_0 |
GGUF Q4_K_M + mmproj | 15.84 + 0.86 GB | 24 GB GPU recommended default | imatrix |
| AutoRound W4A16 INT4 | 18.21 GB | Intel Arc Pro B70 32 GB / CUDA (vLLM) | 0/8 refused · 100.0% match (0.06364 mean TVD) |
| Mixed NVFP4 + FP8 | 22.99 GB | NVIDIA Blackwell sm_100/sm_120 (vLLM + MTP) | 0/8 refused · 100.0% match (0.06123 mean TVD) |
| Base BF16 | Full precision | H100 / B200 / RTX PRO 6000 | 0/8 refused · 100.0% match (0.0192 mean TVD) |
Sizes report weight files on disk (excluding KV cache). AutoRound INT4 and NVFP4 metrics come from their model cards (NVFP4 measured on calibrated QDQ weights prior to packing); GGUF inherits the base BF16 verification and has not been separately re-scored on the 30-probe decision suite.
Models
-
Qwen3.8-27B-Perplexing-Pegasus-Heretic-GGUFimatrix GGUF (IQ3_XXS to Q8_0 + split BF16) and mmproj F16 for 16–24 GB GPUs 102 downloads · 4 likes -
Qwen3.8-27B-Perplexing-Pegasus-Heretic-AutoRound-INT4AutoRound W4A16 INT4 symmetric for vLLM on Intel Arc Pro B70 and CUDA 20 downloads · 0 likes -
Qwen3.8-27B-Perplexing-Pegasus-Heretic-NVFP4Mixed NVFP4 + FP8 with MTP for NVIDIA Blackwell vLLM 15 downloads · 0 likes -
Qwen3.8-27B-Perplexing-Pegasus-HereticBase BF16 weights with intact lm_head, decision/ readout and 15 MTP tensors 31 downloads · 0 likes
Caveats
- Weight sizes exclude KV cache and runtime buffers; longer contexts on 16 GB cards need
IQ3_XXSorQ3_K_Mrather thanQ4_K_M. - End-to-end scores on WebBrain’s frozen 100-case planner and vision benchmarks have not been published for this checkpoint yet.
- Built on Alibaba’s Qwen3.8-27B, Perplexity’s
pplx-decider-v1-27band AveLabs’Qwen3.8-27B-Perplexing-Pegasus.