A 27B AI Model Now Fits an iPhone - Apple Is Watching
PrismML compressed a 27B-parameter Qwen model from 54GB to under 4GB using 1-bit and ternary weights, and Apple is evaluating the technology for on-device Siri.
Articles Tagged "Quantization"
Bonsai 27B
Bonsai 27B compresses Alibaba's Qwen3.6-27B into 1-bit and ternary weights, shrinking a 54GB model to as little as 3.9GB so it runs on an iPhone.
Quantization's Hidden Tax, Cliff Tokens, Smarter Memory
Three new arXiv papers reveal hidden costs in quantized reasoning models, single-token failure triggers, and a new framework that cuts agent memory errors by up to 79%.
Google Gemma 4 QAT Fits Frontier AI in Under 1GB
Google DeepMind's new QAT checkpoints shrink the Gemma 4 E2B model to under 1GB, making serious on-device AI viable for phones and budget laptops.
Best Open-Source LLMs You Can Self-Host in 2026
Top open-weight models for self-hosting in 2026, with verified VRAM requirements, benchmark data, and tools to deploy them on consumer and server hardware.
Best Open-Weights AI Models 2026: Llama, DeepSeek, Qwen
The definitive guide to open-weights AI models in 2026 - top picks by size tier, use case, benchmark scores, and deployment hardware. From 400B+ MoE giants to 1B edge models.
LLM Quantization Impact Leaderboard 2026: INT4 vs FP16
How much quality do LLMs lose when quantized from BF16 to INT8, Q6, Q5, Q4, Q3, Q2? Per-model delta tables across MMLU, HumanEval, and perplexity, with VRAM and throughput data for every major quantization format.
Best Open-Source LLM Inference Servers 2026
A benchmark-driven comparison of the top open-source LLM inference servers - vLLM, SGLang, TGI, llama.cpp, TensorRT-LLM, LMDeploy, and more.
A $900 RTX 3090 Now Beats an M5 Max at LLM Inference
Two researchers fused all 24 layers of Qwen 3.5-0.8B into a single CUDA kernel launch, making a five-year-old RTX 3090 deliver 1.8x the throughput of an M5 Max at equal or better efficiency. The gap was software, not silicon.
Google's TurboQuant Cuts LLM Memory 6x With Zero Loss
Google Research's TurboQuant compresses LLM key-value cache by 6x and delivers 8x speedup on H100 GPUs with zero accuracy loss - no fine-tuning required.
NVIDIA Rubin CPX - Inference GPU With GDDR7
Full specs, benchmarks, and analysis of the NVIDIA Rubin CPX - a purpose-built inference GPU with 128GB GDDR7, 30 PFLOPS NVFP4, and 3x faster attention versus Blackwell, targeting million-token context workloads.
NVIDIA Rubin R200 - Next-Gen AI Superchip
Complete specs, benchmarks, and analysis of the NVIDIA Rubin R200 GPU - the post-Blackwell flagship with 288GB HBM4, 22 TB/s bandwidth, and 50 PFLOPS FP4.
