NVIDIA Ships Nemotron 3 Ultra - 550B Open-Weight MoE
NVIDIA's 550B Nemotron 3 Ultra, released June 4, tops the US open-weight leaderboard with a hybrid Mamba-Transformer MoE architecture and 300-plus tokens per second throughput.
Articles Tagged "Inference"
NVIDIA Dynamo Snapshot Slashes Kubernetes AI Cold Starts
NVIDIA's Dynamo Snapshot uses CRIU and cuda-checkpoint to freeze and restore GPU inference containers in seconds, cutting Kubernetes cold-start times by up to 21x for large models.
Rapid-MLX Is 2.6x Faster Than Ollama on Apple Silicon
New open-source inference engine for Apple Silicon benchmarks up to 2.6x faster than Ollama, supports 66 model aliases, and drops in as an OpenAI-compatible server on any Mac.
Llama 3.3 70B Instruct
Meta's Llama 3.3 70B Instruct matches Llama 3.1 405B on instruction following and math while running at 4-5x lower cost, with the lowest hallucination rate of any open-weight model on the Vectara summarization leaderboard.
Intel Crescent Island GPU Skips HBM for 480GB LPDDR5X
Intel's Crescent Island inference GPU trades HBM bandwidth for 480GB of LPDDR5X, targeting customers locked out of NVIDIA's supply chain.
Open Source LLM Hosting Costs - June 2026
Verified June 2026: real cost per million tokens for self-hosting Llama 4 Scout, Maverick, Qwen3-235B, and DeepSeek V3.2 - GPU requirements, cost formulas, and when cheap APIs actually win.
Cut CoT Costs, Fix Agent Memory, Test Clinical AI
Three papers: smarter CoT trimming cuts reasoning length by 50%, a plug-in context manager rescues frozen agents on long tasks, and a 960K-item clinical benchmark exposes LLM gaps in hospitals.
AMD Instinct MI350P - CDNA 4 PCIe Inference Card
AMD Instinct MI350P brings CDNA 4 to a standard PCIe slot: 144GB HBM3E, 4 TB/s bandwidth, 2.3 PFLOPS MXFP8, and 600W passive cooling for air-cooled servers.
NVIDIA RTX Spark - ARM Blackwell Superchip for AI PCs
NVIDIA RTX Spark is a 20-core ARM + Blackwell GPU superchip delivering 1 petaFLOP FP4 and 128GB unified memory for AI-first Windows laptops and desktops.
Nvidia Cosmos 3 Is the First Open Physical AI Model
Nvidia's Cosmos 3 is the first fully open omnimodel for physical AI, trained on 20 trillion tokens to teach robots and autonomous vehicles how to reason and act in the real world.
XCENA Raises $135M Betting Memory Is AI's Real Bottleneck
XCENA raises $135M at a $570M valuation to build the MX1 - a CXL 3.2 chip with thousands of RISC-V cores that processes AI workloads where data lives, eliminating costly data transfers between GPU and RAM.
Groq Raises $650M to Pivot From Chip Maker to Cloud
After licensing its chip technology to Nvidia for $20 billion, Groq is raising $650M from existing investors to rebuild itself as an AI inference cloud provider.
