Skip to main content

How To Use This Guide
Why Run Large Language Models Locally?
Understanding Large Language Models
Selecting Models
Models For Coding
- Overview of Code Llama Variations
- Other Notable Open Source Models for Coding
Installation
Evaluation
- Testing Each Model
- Evaluating Models with a Standardized Test
Best Practices
- Best Practices for Running Local LLMs
Prompts
- Collection of Prompts
- Creating a Shell Script
Advanced Usage
Benchmarks
Additional Resources

AI for Dev was created by Jesse Freeman. Visit the GitHub repo to help contribute.

AI for Dev on GitHub

Selecting Models

This is the introduction to the chapter.

Table of contents

Selecting the Right Model
Pick the Right Model for Your Memory
Model Size and Memory
CPU vs. GPU and Inferencing
Offloading to the GPU
CodeLlama 3.1 Instruct Variations