Gemini 3.5 Flash: Real Speed, Selective Benchmarks
Google's Gemini 3.5 Flash is genuinely fast at 289 tok/s and competitive on agentic tasks - but the benchmark portfolio has gaps worth knowing before you build on it.
Articles Tagged "Inference"
Chinese Models Claim 60% of OpenRouter Token Traffic
Chinese AI providers now handle over 60% of all tokens routed through OpenRouter, up from less than 2% just a year ago.
Gemini 3.5 Flash Review: When Flash Surpasses Pro
Gemini 3.5 Flash leads on agentic benchmarks, runs 4x faster than Claude and GPT-5.5, and undercuts both on price - but a hidden long-context weakness and a 3x price hike over its predecessor deserve scrutiny.
Gemini 3.5 Flash
Google DeepMind's fastest frontier model, hitting 76.2% on Terminal-Bench 2.1 and 289 tok/s, now powering AI Mode in Search for over 1 billion monthly users.
Thinking Machines Builds AI That Listens While Talking
Mira Murati's startup unveils TML-Interaction-Small, a 276B MoE model that hits 0.40-second response latency by listening and generating speech at the same time.
Lumai Iris Nova: Optical AI Inference Server
Lumai's optical AI inference server uses light-based computing to run billion-parameter LLMs with up to 90% less power than GPUs.
Skymizer HTX301: 700B LLMs at 240W on PCIe
Skymizer's HTX301 uses six 28nm chips and 384 GB LPDDR5 to run 700B-parameter LLMs on a single PCIe card at just 240W.
Meta MTIA 450: 18.4 TB/s Inference Accelerator
Meta's MTIA 450 doubles HBM bandwidth to 18.4 TB/s and adds FlashAttention hardware acceleration for GenAI inference in 2027.
Meta MTIA 400: GenAI Inference at Scale
Meta's second-gen ASIC delivers 6 PFLOPS FP8 and 288 GB HBM for GenAI and recommendation inference inside Meta's data centers.
SubQ Review: 52x Faster, but Show Your Work
Subquadratic's SubQ claims the first linear-scaling LLM with a 12M-token window - but private beta access, self-reported benchmarks, and a 17-point MRCR gap make independent verification the only test that matters.
SubQ
SubQ is the first LLM built on a fully subquadratic attention architecture, achieving a 12M-token research context and 52x faster inference than FlashAttention at 1M tokens.
ZAYA1-8B: Open Reasoning Model Rivals Claude on AMD GPUs
Zyphra's ZAYA1-8B matches Claude 4.5 Sonnet on HMMT 2025 math benchmarks at just 760M active parameters, trained entirely on AMD Instinct MI300X GPUs under Apache 2.0.
