Skip to main content
Link
Menu
Expand
(external link)
Document
Search
Copy
Copied
AI for Dev
How To Use This Guide
Why Run Large Language Models Locally?
Understanding Large Language Models
Overview of Large Language Models
Models Variations
Model Naming Conventions
Parameter Size
Quantization
Quantization Schemes
How Models Are Scaled
Hardware Requirements
Understanding Your System Capabilities
Memory Requirements by Model Size
Model Size and Memory
CPU vs. GPU and Inferencing
Offloading to the GPU
Models For Coding
Overview of Code Llama Variations
Other Notable Open Source Models for Coding
CodeLlama 3.1 Instruct Variations
The Models I Actually Use for TypeScript/JavaScript
Model Licensing and Commercial Usage Guide
Installation
Installing LM Studio
Configuring LM Studio on Apple Silicon
Optimizing LM Studio for Apple Silicon
Picking Models in LM Studio
Installing Ollama
Installing GPT4All
Installing LocalAI
Evaluation
Testing Each Model
Evaluating Models with a Standardized Test
Best Practices
Best Practices for Running Local LLMs
Prompts
Collection of Prompts
Creating a Shell Script
Advanced Usage
Advanced Model Tuning
Integrating LLMs into Workflows
Custom Built PCs
Benchmarks
Personal Computer Results
Ryzen 7 5800X + RTX 4070 Super Performance Overview
MacBook Pro M3 Max + 30-Core GPU Performance Overview
Intel NUC9V7QNX + NVIDIA GeForce RTX 4060 Performance Overview
ASUS ProArt P16 + RTX 4070 Performance Overview
ASUS ROG Zephyrus G15 + RTX 3060 Performance Overview
MacBook Air (2022) with Apple M2 + 8-Core GPU Performance Overview
**MINISFORUM BD790i with AMD Ryzen 9 7945HX + NVIDIA GeForce RTX 4070 SUPER Performance Overview**
**ASUS ROG Zephyrus G16 Intel Core Ultra 9 185H + NVIDIA GeForce RTX 4090 Performance Overview**
Additional Resources
Tools
Articles
AI for Dev on GitHub
Advanced Usage
Integrating LLMs into Workflows
Integrating LLMs into Workflows
This section discusses integrating LLMs into development workflows.