Throughput, latency, and queue depth for Gemma-4 31B served on vLLM under progressive load, from 12 to 24 concurrency The numbers that matter: 1.17k tok/s peak, ~0.7s median TTFT, and tail latency as the one thing to watch.
Open-source AI has crossed an important line. The question is no longer whether open models are good enough to power serious products. The question is how quickly teams can deploy them privately, rel
Gemma 4 31B is Google’s high-quality 31B-class instruction model — designed to deliver strong reasoning, coding, multilingual understanding, and reliable instruction-following while remaining lighter
