Before jumping into APIs, RAG, agents, and AI applications, I wanted to understand what actually happens inside an LLM.

I kept coming across terms like neural networks, deep learning, Transformers, attention, tokens, embeddings, BERT, GPT, and causal language modeling — but they all felt like disconnected pieces.

I could understand each concept individually, but I didn't have a clear picture of how they all connected.

So I decided to step back and build the mental model from the ground up.

This article is my attempt to connect those pieces.

1. Starting With Neural Networks

Before understanding LLMs, it helps to understand where they come from.

A neural network is a machine learning model that learns patterns from data.

For example, suppose we want to recognize handwritten digits.

We could give a neural network thousands of images of handwritten digits:

Images of handwritten digits
            ↓
      Neural Network
            ↓
     Learned patterns
            ↓
        Prediction

Instead of manually programming rules such as:

"If the image has this curve and this line, it must be a 3."

we allow the neural network to learn those patterns from examples.

The things the network learns are stored in its parameters, primarily weights and biases.

As we increase the number of layers in a neural network, we enter the world of deep learning.

Machine Learning
       ↓
Neural Networks
       ↓
Deep Learning

Different neural network architectures became useful for different types of problems.

For example:

• CNNs became popular for image-related tasks.
• RNNs and LSTMs were commonly used for sequential data such as language.
• Transformers later became extremely important for language and many other sequence-based tasks.

But language has a special challenge.

Context matters

Consider:

"The animal didn't cross the road because it was tired."

To understand what "it" refers to, we need to understand its relationship with the other words in the sentence.

This is where things like attention become important.

And this eventually leads us to Transformers.

2. Why Did Transformers Come Into the Picture?

Before Transformers, RNNs and LSTMs were commonly used for language-related tasks.

They processed sequences step by step.

For example:

I → love → machine → learning

This sequential processing created some challenges.

One major problem was that it made training difficult to parallelize efficiently.

It could also become difficult to capture relationships between tokens that were far apart in a long sequence.

Then, in 2017, researchers introduced the Transformer architecture in the paper:

"Attention Is All You Need"

The original Transformer was designed for machine translation.

The basic architecture looked like this:

English sentence
       ↓
    Encoder
       ↓
Contextual representation
       ↓
    Decoder
       ↓
French sentence

For example:

"I love cats"
      ↓
   Transformer
      ↓
"J'aime les chats"

The key idea that made Transformers different was attention.

Instead of processing a sequence strictly one step at a time, the Transformer could use attention to determine relationships between different tokens.

This made it much better suited to capturing context and also allowed much more parallel computation during training.

3. Attention: The Key Idea

At a high level, attention answers:

"When I'm processing this token, which other tokens should I pay attention to?"

Consider:

"The animal didn't cross the road because it was tired."

When processing "it", the model needs to determine which other tokens are relevant.

Conceptually:

The animal didn't cross the road because it was tired.
                                      ↑
                                     "it"

The model can assign different levels of importance to different tokens.

This is called self-attention, because tokens in the sequence attend to other tokens in the same sequence.

I initially thought attention was something separate from the Transformer.

It isn't.

Attention is one of the core mechanisms inside a Transformer.

For now, the mental model I use is:

Current token
     ↓
Look at other relevant tokens
     ↓
Combine useful information
     ↓
Create a contextual representation

The mathematical details — Query, Key, Value, attention scores, and matrix multiplication — deserve their own article.

4. The Original Transformer: Encoder + Decoder

The original Transformer architecture contained two major components:

Input
  ↓
Encoder
  ↓
Representation
  ↓
Decoder
  ↓
Output

A useful beginner mental model is:

Encoder → processes and represents the input

Decoder → generates the output

For translation:

English
   ↓
Encoder
   ↓
Representation
   ↓
Decoder
   ↓
French

This architecture was designed around a very natural problem:

Take one sequence and transform it into another sequence.

For example:

• English → French
• Article → Summary
• Question → Answer

But later, researchers realized that we don't always need both parts.

This led to different Transformer architectures.

5. Different Transformer Architectures

This was one of the things I initially found confusing.

If BERT, GPT, T5, and BART are all based on Transformers, why are they different?

The answer is:

Transformer is an architecture, not one specific model.

Different models can use different parts of the Transformer architecture.

A simple mental model is:

                         Transformer
                              │
              ┌───────────────┼───────────────┐
              ↓               ↓               ↓
        Encoder-only     Decoder-only    Encoder-decoder
              ↓               ↓               ↓
            BERT              GPT          T5 / BART

Let's look at each one.

5.1 Encoder-only Models

An encoder-only model uses the encoder part of the Transformer.

The encoder processes the input and creates contextual representations.

The most famous example is BERT.

BERT was trained using Masked Language Modeling.

For example:

The cat is [MASK] on the mat.

The model tries to predict the missing token.

Because the encoder can use context from both sides of the masked token, it can build a bidirectional representation.

This makes encoder-only models useful for tasks such as:

• Text classification
• Sentiment analysis
• Named Entity Recognition
• Search understanding
• Information extraction
• Some question-answering tasks

The simple mental model:

Encoder-only → understand/represent text

5.2 Decoder-only Models

A decoder-only model uses the decoder part of the Transformer.

The most famous example is GPT.

GPT stands for:

Generative Pre-trained Transformer

GPT is trained primarily using causal language modeling.

Its objective is:

Predict the next token using the previous tokens.

For example:

The cat is
       ↓
    sleeping

Then:

The cat is sleeping
                  ↓
              next token

The model keeps generating one token at a time.

This is why GPT is naturally suited to text generation.

The simple mental model:

Decoder-only → generate text

5.3 Encoder-decoder Models

Encoder-decoder models use both parts.

The encoder processes the input.

The decoder generates the output.

For example, in summarization:

Article
   ↓
Encoder
   ↓
Representation
   ↓
Decoder
   ↓
Summary

Or translation:

English
   ↓
Encoder
   ↓
Representation
   ↓
Decoder
   ↓
French

Examples include T5 and BART.

The simple mental model:

Encoder-decoder → transform one sequence into another

6. BERT vs GPT vs T5 vs BART

This is the simplest table I use to remember them:

The important thing to remember is:

They are different ways of using the Transformer architecture.

7. What Is an LLM?

Now we can finally talk about Large Language Models.

LLM stands for:

Large Language Model

Let's break down the name.

Large

"Large" generally refers to the enormous number of learned parameters.

Parameters are numerical values learned during training, primarily:

• Weights
• Biases

Modern language models can have billions of parameters.

Language

The model is trained on large amounts of language data and learns patterns and relationships in that data.

Model

It is ultimately a neural network that has learned these patterns through its parameters.

So:

LLM
│
├── Large
│     └── Many learned parameters
│
├── Language
│     └── Learns patterns from language
│
└── Model
      └── Neural network

8. GPT as an LLM

GPT-style models are generally large decoder-only Transformer models.

They are trained primarily using causal language modeling.

The basic objective is surprisingly simple:

Predict the next token based on the previous context.

Suppose the training data contains:

The sky is blue.

The model learns to predict:

The
 ↓
sky

Then:

The sky
 ↓
is

Then:

The sky is
 ↓
blue

This happens across enormous amounts of training data.

During training:

Input context
     ↓
Predict next token
     ↓
Compare with actual token
     ↓
Calculate error
     ↓
Update parameters
     ↓
Repeat

Over time, the model becomes better at predicting the next token.

9. How Can Next-token Prediction Do So Many Things?

This was one of the most interesting things for me to understand.

If GPT is fundamentally trained to predict the next token, how can it:

• Answer questions?
• Translate languages?
• Summarize articles?
• Write code?
• Write poems?
• Explain concepts?
• Have conversations?

The answer is largely context.

The model doesn't necessarily need a separate mechanism for every task.

The prompt provides context about what kind of continuation is expected.

For example:

Translate to French:

I love cats.

The context tells the model that the expected continuation is a translation.

Or:

Summarize this article:

[article]

Now the expected continuation is a summary.

Or:

Write a Python function that sorts a list.

Now the expected continuation is code.

The underlying mechanism is still:

Context
   ↓
Predict next token
   ↓
Add token to context
   ↓
Predict next token
   ↓
Add token to context
   ↓
Repeat

This is one of the most important mental models I have taken away:

A relatively simple training objective — next-token prediction — can result in a model capable of many different language tasks.

10. Where Do Tokens Come In?

At this point, another question naturally appears:

If the model predicts tokens, does it actually process words directly?

No.

The model needs to convert our text into numerical representations that a neural network can process.

This starts with tokenization.

For example:

"I love programming"

might become something conceptually like:

["I", " love", " programming"]

The exact tokens depend on the tokenizer.

A token can be:

• A complete word
• Part of a word
• Punctuation
• A space combined with text
• Other pieces of text

So:

A token is not necessarily a word.

The token is then mapped to a numerical ID:

Text
 ↓
Tokens
 ↓
Token IDs

11. Where Do Embeddings Come In?

The Transformer doesn't directly work with the token ID as a meaningful representation.

The token ID is essentially an index.

The model needs a richer numerical representation.

This is where embeddings come in.

Conceptually:

"cat"
  ↓
Token
  ↓
Token ID
  ↓
Embedding
  ↓
[0.21, -0.42, 0.17, ...]

An embedding is a vector containing many numerical values.

For example, if an embedding has 4,096 dimensions:

[0.21, -0.42, 0.17, ..., 0.31]

it simply means that the vector contains 4,096 numbers.

These numbers allow the Transformer to perform mathematical operations on the representation of the token.

At this point, our mental model becomes:

Text
 ↓
Tokenization
 ↓
Tokens
 ↓
Token IDs
 ↓
Embeddings
 ↓
Transformer

I'll go much deeper into tokenization and embeddings in the next article.

12. The Big Picture

After connecting all these concepts, this is the mental map I currently have:

                    Neural Networks
                           ↓
                     Deep Learning
                           ↓
                     Sequence Models
                           ↓
                      Transformers
                           ↓
                        Attention
                           ↓
              Transformer Architectures
                           ↓
          ┌────────────────┼────────────────┐
          ↓                ↓                ↓
     Encoder-only     Decoder-only    Encoder-decoder
          ↓                ↓                ↓
        BERT              GPT           T5 / BART
                           ↓
                          LLMs

And if I zoom into a GPT-style LLM:

User Prompt
     ↓
Tokenization
     ↓
Token IDs
     ↓
Embeddings
     ↓
Transformer Blocks
     ↓
Attention
     ↓
Contextual Representation
     ↓
Next-token prediction
     ↓
Generated token
     ↓
Repeat

This finally gave me the map I was missing.

I don't need to know every mathematical detail yet.

I first need to know where each concept belongs.

13. What Happens When I Actually Ask an LLM a Question?

Now comes the question I'm most interested in:

What actually happens inside an LLM when I type a prompt?

Suppose I ask:

What is the capital of India?

At a very high level, the process looks something like:

"What is the capital of India?"
             ↓
        Tokenization
             ↓
         Token IDs
             ↓
         Embeddings
             ↓
   Positional Information
             ↓
     Transformer Blocks
             ↓
         Attention
             ↓
 Contextual Representation
             ↓
        Output Scores
             ↓
          Softmax
             ↓
 Probability Distribution
             ↓
        Next Token
             ↓
           Repeat

And this is where I want to go next.

14. What's Next?

Now that I have the high-level map, I want to open up the black box.

In the next article, I'll start from the beginning of the inference process:

Prompt
  ↓
Tokenization
  ↓
Token IDs
  ↓
Embeddings
  ↓
Positional information
  ↓
Transformer
  ↓
Attention
  ↓
Hidden states
  ↓
Unembedding
  ↓
Logits
  ↓
Softmax
  ↓
Probability distribution
  ↓
Next token

I'll break down each step and answer questions like:

• What exactly is a token?
• What does an embedding represent?
• What does "4,096-dimensional embedding" actually mean?
• How does positional information work?
• How does self-attention calculate which tokens are relevant?
• What are Query, Key, and Value?
• What is a hidden state?
• What is the unembedding matrix?
• How does the model turn its final representation into scores for every possible next token?
• What exactly does softmax do?
• Where does temperature come into the picture?
• How does the model finally choose the next token?

The goal is not to start with mathematics.

The goal is to first build an intuitive mental model and then introduce the mathematics once the pieces make sense.

Final Takeaway

If I had to summarize everything I've learned so far in one sentence:

An LLM like GPT is a large neural network built using the Transformer architecture, trained on language to predict the next token, and capable of many language tasks because of the patterns and representations it learns from enormous amounts of data.

The biggest thing that changed for me was realizing that all these terms aren't isolated concepts.

They fit together:

Neural Networks
      ↓
Deep Learning
      ↓
Transformers
      ↓
Attention
      ↓
Encoder / Decoder Architectures
      ↓
GPT
      ↓
LLMs
      ↓
Tokens
      ↓
Embeddings
      ↓
Transformer Processing
      ↓
Next-token Prediction
      ↓
Generated Text

Now that I have the map, I'm ready to understand what's actually happening inside the box.