Qwen3.6-27B-FP8 on One RTX 6000 Ada: Fast TTFT, 668 tok/s Peak Throughput [Benchmark]
Qwen/Qwen3.6-27B-FP8 served through vLLM on a single RTX 6000 Ada is a strong and practical configuration for 8K-context chat serving.
Jul 5, 202613 min read171
![Qwen3.6-27B-FP8 on One RTX 6000 Ada: Fast TTFT, 668 tok/s Peak Throughput [Benchmark]](/_next/image?url=https%3A%2F%2Fcdn.hashnode.com%2Fuploads%2Fcovers%2F6a22b1a041d5b05f16273b50%2F8fd36dcb-515c-4f77-8071-9a1aedc1c2ed.png&w=3840&q=75)
Command Palette
Search for a command to run...
#llm
Articles tagged with #llm
Gemma-4 31B + vLLM on RTX 6000 PRO : 1.17k tokens/sec and still asking for more
Throughput, latency, and queue depth for Gemma-4 31B served on vLLM under progressive load, from 12 to 24 concurrency The numbers that matter: 1.17k tok/s peak, ~0.7s median TTFT, and tail latency as the one thing to watch.
Jun 29, 20265 min read116
Gemma-4 31B it, is Now Available on HexGrid.cloud
Open-source AI has crossed an important line. The question is no longer whether open models are good enough to power serious products. The question is how quickly teams can deploy them privately, rel
Jun 21, 20267 min read17
NVIDIA Nemotron 3 Nano 30B-A3B is Now Available on HexGrid.cloud
NVIDIA Nemotron 3 Nano 30B-A3B is now available for dedicated deployment on a GPU of your choice on HexGrid.cloud. Run in One-click and get an OpenAI-compatible endpoint. Nemotron 3 Nano is a 30B-cla
Jun 20, 20267 min read14
Deploy Gemma 4 31B-it in One Click: 4-bit, 8-bit, 16-bit Production API
Gemma 4 31B is Google’s high-quality 31B-class instruction model — designed to deliver strong reasoning, coding, multilingual understanding, and reliable instruction-following while remaining lighter
Jun 6, 20264 min read12
Deploy Llama-3.3 70B in One Click: 4-bit, 8-bit, 16-bit Production API
Llama 3.3 70B is Meta’s latest high-quality 70B-class instruction model — designed to deliver strong reasoning, coding, multilingual understanding, and tool-use performance while remaining much more c
Jun 6, 20264 min read11