Skip to main content
Link
Menu
Expand
(external link)
Document
Search
Copy
Copied
AI for Dev
How To Use This Guide
Why Run Large Language Models Locally?
Understanding Large Language Models
Overview of Large Language Models
Models Variations
Model Naming Conventions
Parameter Size
Quantization
Quantization Schemes
How Models Are Scaled
Selecting Models
Selecting the Right Model
Pick the Right Model for Your Memory
Model Size and Memory
CPU vs. GPU and Inferencing
Offloading to the GPU
CodeLlama 3.1 Instruct Variations
Models For Coding
Overview of Code Llama Variations
Other Notable Open Source Models for Coding
Installation
Installing LM Studio
Configuring LM Studio on Apple Silicon
Optimizing LM Studio for Apple Silicon
Picking Models in LM Studio
Evaluation
Testing Each Model
Evaluating Models with a Standardized Test
Best Practices
Best Practices for Running Local LLMs
Prompts
Collection of Prompts
Creating a Shell Script
Advanced Usage
Advanced Model Tuning
Integrating LLMs into Workflows
Custom Built PCs
Benchmarks
Personal Computer Results
Ryzen 7 5800X
MacBook Pro M3 Max
Intel NUC9V7QNX
ASUS ProArt P16
ASUS Zephyrus G15
MacBook Air M2
MINISFORUM AMD Ryzen 9
ASUS Zephyrus G16
Additional Resources
Tools
Articles
In the News
AI for Dev on GitHub
Advanced Usage
Integrating LLMs into Workflows
Integrating LLMs into Workflows
This section discusses integrating LLMs into development workflows.