Benchmarks
Real-world performance data from various hardware configurations running different LLMs. Use these benchmarks to understand what performance to expect from your setup and compare different hardware options before making upgrade decisions.
What You’ll Find Here
- Detailed performance tests across different hardware configurations
- Token generation speeds, memory usage, and loading times
- Comparison data for various model sizes and quantization levels
- Real-world performance metrics from community contributors
- Hardware recommendations based on actual testing results
Benchmark Categories
Consumer Hardware
- Gaming PCs with modern GPUs (RTX 30/40 series, RX 6000/7000 series)
- Workstations with professional graphics cards
- Laptops including gaming and professional models
- Mac Systems with Intel and Apple Silicon processors
Performance Metrics Tracked
- Model Load Time: How long it takes to load a model into memory
- First Token Latency: Time to generate the first response token
- Tokens per Second: Sustained generation speed during normal usage
- Memory Usage: RAM and VRAM consumption during operation
- Temperature and Power: Thermal performance and power consumption
How to Use These Benchmarks
- Find Similar Hardware: Look for systems with specs close to your current setup
- Compare Configurations: See how different GPUs, RAM amounts, and storage affect performance
- Plan Upgrades: Identify which hardware improvements provide the best performance gains
- Set Expectations: Understand realistic performance targets for your system
Contributing Your Results
We welcome community benchmark submissions! Each benchmark includes:
- Detailed hardware specifications
- Standardized test procedures
- Multiple model comparisons
- Real-world usage observations
Benchmark Methodology
All benchmarks follow consistent testing procedures:
- Clean System State: Tests run on systems with minimal background processes
- Multiple Runs: Results averaged across multiple test runs for accuracy
- Standard Prompts: Consistent prompt sets for comparable results
- Environmental Controls: Temperature and power settings documented
Table of contents
- Personal Computer Results
- Ryzen 7 5800X + RTX 4070 Super Performance Overview
- MacBook Pro M3 Max + 30-Core GPU Performance Overview
- Intel NUC9V7QNX + NVIDIA GeForce RTX 4060 Performance Overview
- ASUS ProArt P16 + RTX 4070 Performance Overview
- ASUS ROG Zephyrus G15 + RTX 3060 Performance Overview
- MacBook Air (2022) with Apple M2 + 8-Core GPU Performance Overview
- **MINISFORUM BD790i with AMD Ryzen 9 7945HX + NVIDIA GeForce RTX 4070 SUPER Performance Overview**
- **ASUS ROG Zephyrus G16 Intel Core Ultra 9 185H + NVIDIA GeForce RTX 4090 Performance Overview**