Unified local model

Qwen3.8-27B-Perplexing-Pegasus-Heretic

An Apache-2.0 abliterated release of AveLabs’ Qwen3.8-27B-Perplexing-Pegasus (Qwen3.8-27B + Perplexity pplx-decider-v1-27b). It packs vision, thinking, tool calling, structured decision logprobs and all 15 Multi-Token Prediction tensors into one checkpoint — down to 16 GB consumer GPUs.

11.95 GBIQ3_XXS (11.09 GB) + mmproj F16 (0.86 GB) for 16 GB GPUs
0 / 8refusal probes refused (102/104 → 6/104 classifier)
30 / 30typed-decision agreement vs Pegasus (0.0192 mean TVD)
15 MTPMulti-Token Prediction tensors kept (blk.64 at Q8_0 in IQ/3-bit GGUFs)

Why it matters

Peak local agent setups usually juggle two or three models in memory at once: a vision model to inspect the page, a typed-decision router for scoring, and an uncensored model that won’t stall on harmless workflows. Unifying them into one 27B checkpoint lets a single 16 GB or 24 GB GPU run the whole stack.

Built for 16 GB and 24 GB machines

For 16 GB cards (such as RTX 4060 Ti 16 GB, RTX 5060 Ti 16 GB, RTX 4080, RTX 5070 Ti or RTX 5080), the GGUF repo provides imatrix quantizations calibrated directly on the abliterated BF16 weights across general knowledge, reasoning, code, multimodal chat and pplx-decider prompts.

IQ3_XXS (11.09 GB) plus the F16 vision projector mmproj (0.86 GB) totals 11.95 GB, leaving ~4 GB on a 16 GB GPU for KV cache and runtime overhead; Q3_K_M (13.00 GB) also fits in 16 GB with tighter context. On 24 GB GPUs, Q4_K_M (15.84 GB) is the recommended default. Across IQ4_XS, Q3_K_M and IQ3_XXS, the MTP layer (blk.64) is explicitly kept at Q8_0.

Abliteration without breaking typed decisions

We ran heretic-llm Arbitrary-Rank Ablation (ARA, lora_rank=50, preserve_row_magnitudes=true, full BF16 precision, 150 Optuna TPE trials) across layers 16–52, leaving lm_head.weight untouched (cosine similarity 1.000001), preserving the decision/ readout (decision_config.json, readout.safetensors) and keeping all 15 mtp.* tensors intact.

Trial 7 cut classifier refusals from 102/104 on unmodified Pegasus to 6/104 (and 0/8 on refusal probes with thinking off) at 0.0618 KL divergence, while keeping 100.0% (30/30) typed-decision agreement and 90.0% → 90.0% probe accuracy.

Four hardware targets under Apache-2.0

Alongside llama.cpp, LM Studio and Ollama (GGUF), the release ships AutoRound W4A16 INT4 (18.21 GB) for vLLM on Intel Arc Pro B70 (32 GB XPU) and NVIDIA CUDA GPUs, mixed NVFP4 + FP8 (22.99 GB) with unquantized BF16 lm_head, vision and MTP tensors for NVIDIA Blackwell (sm_100 / sm_120), and the full BF16 reference checkpoint.

Formats, VRAM targets and verification

BuildSizeTarget hardwareVerification & notes
GGUF IQ3_XXS + mmproj11.09 + 0.86 GB16 GB GPU (llama.cpp / LM Studio)imatrix; blk.64 MTP at Q8_0
GGUF Q3_K_M + mmproj13.00 + 0.86 GB16 GB GPU (llama.cpp / LM Studio)imatrix; blk.64 MTP at Q8_0
GGUF IQ4_XS + mmproj14.69 + 0.86 GB16–24 GB GPU / offloadimatrix; blk.64 MTP at Q8_0
GGUF Q4_K_M + mmproj15.84 + 0.86 GB24 GB GPU recommended defaultimatrix
AutoRound W4A16 INT418.21 GBIntel Arc Pro B70 32 GB / CUDA (vLLM)0/8 refused · 100.0% match (0.06364 mean TVD)
Mixed NVFP4 + FP822.99 GBNVIDIA Blackwell sm_100/sm_120 (vLLM + MTP)0/8 refused · 100.0% match (0.06123 mean TVD)
Base BF16Full precisionH100 / B200 / RTX PRO 60000/8 refused · 100.0% match (0.0192 mean TVD)

Sizes report weight files on disk (excluding KV cache). AutoRound INT4 and NVFP4 metrics come from their model cards (NVFP4 measured on calibrated QDQ weights prior to packing); GGUF inherits the base BF16 verification and has not been separately re-scored on the 30-probe decision suite.

Models

Caveats

  • Weight sizes exclude KV cache and runtime buffers; longer contexts on 16 GB cards need IQ3_XXS or Q3_K_M rather than Q4_K_M.
  • End-to-end scores on WebBrain’s frozen 100-case planner and vision benchmarks have not been published for this checkpoint yet.
  • Built on Alibaba’s Qwen3.8-27B, Perplexity’s pplx-decider-v1-27b and AveLabs’ Qwen3.8-27B-Perplexing-Pegasus.

Read more on the blog