Benchmarks

Real-world performance data from various hardware configurations running different LLMs. Use these benchmarks to understand what performance to expect from your setup and compare different hardware options before making upgrade decisions.

What You’ll Find Here

  • Detailed performance tests across different hardware configurations
  • Token generation speeds, memory usage, and loading times
  • Comparison data for various model sizes and quantization levels
  • Real-world performance metrics from community contributors
  • Hardware recommendations based on actual testing results

Benchmark Categories

Consumer Hardware

  • Gaming PCs with modern GPUs (RTX 30/40 series, RX 6000/7000 series)
  • Workstations with professional graphics cards
  • Laptops including gaming and professional models
  • Mac Systems with Intel and Apple Silicon processors

Performance Metrics Tracked

  • Model Load Time: How long it takes to load a model into memory
  • First Token Latency: Time to generate the first response token
  • Tokens per Second: Sustained generation speed during normal usage
  • Memory Usage: RAM and VRAM consumption during operation
  • Temperature and Power: Thermal performance and power consumption

How to Use These Benchmarks

  1. Find Similar Hardware: Look for systems with specs close to your current setup
  2. Compare Configurations: See how different GPUs, RAM amounts, and storage affect performance
  3. Plan Upgrades: Identify which hardware improvements provide the best performance gains
  4. Set Expectations: Understand realistic performance targets for your system

Contributing Your Results

We welcome community benchmark submissions! Each benchmark includes:

  • Detailed hardware specifications
  • Standardized test procedures
  • Multiple model comparisons
  • Real-world usage observations

Benchmark Methodology

All benchmarks follow consistent testing procedures:

  • Clean System State: Tests run on systems with minimal background processes
  • Multiple Runs: Results averaged across multiple test runs for accuracy
  • Standard Prompts: Consistent prompt sets for comparable results
  • Environmental Controls: Temperature and power settings documented

Table of contents