Understanding Large Language Models

This section covers what I’ve learned about Large Language Models (LLMs) from my experience setting them up locally for development work. When I started this journey, I was confused by all the different models, naming conventions, and technical jargon. Here’s what I wish I had known from the beginning.

My Experience So Far

Getting into local LLMs felt overwhelming at first. There are so many models, each with cryptic names full of abbreviations, and everyone seems to assume you already know what Q4_K_M means. After months of trial and error, I’ve figured out the patterns and what actually matters for development work.

The key things I had to learn:

  • How these models actually work (spoiler: it’s simpler than it sounds)
  • What all those numbers and letters in model names mean
  • How to pick the right size model for my hardware
  • How to get decent performance without a server farm

Here’s how this section is organized based on my learning path:

Start here if you’re new: Overview of Large Language Models - I explain what these things actually are without the marketing hype.

If you’re confused by model names: Model Naming Conventions - Decode those cryptic model names once and for all.

If your computer is struggling: Parameter Size and Quantization - Learn how to pick models that won’t crash your system.

Table of contents