LLM

What is an LLM (Large Language Model)?

An LLM, or large language model, is an AI system trained on massive amounts of text so it can understand language and generate human-like responses. Give it a prompt, and it predicts, one token at a time, the most likely words to come next. That simple mechanism, repeated at huge scale, is what powers tools like ChatGPT, Gemini, and Claude.

“Large” refers to two things: the size of the training data (often billions of words) and the number of parameters the model adjusts during training, sometimes in the billions or trillions.

What is an LLM in AI, exactly?

In AI terms, an LLM is a type of language model, a system that estimates how likely a word (or part of a word, called a token) is to come next in a sequence. Older language models did this with limited context, just the last few words. LLMs use an architecture called a transformer, which lets them consider an entire passage at once, rather than word by word, so they capture context and meaning far better.

How large language models work

The working of large language models is mentioned below:

• Tokens: Text gets broken into tokens, small chunks like whole words, parts of words, or punctuation. “Unwatched” might split into “un,” “watch,” and “ed.”
• Prediction: Given a sequence of tokens, the model calculates the probability of what token comes next, then picks the most likely one (or samples from the top candidates).
• Self-attention: The model’s core mechanism. It weighs how relevant every other word in the input is to the word it’s currently processing, even words far apart in the sentence. This is what lets an LLM correctly connect “feeling” and “blue” in a sentence, even if other words sit between them.
• Parameters: The internal values the model adjusts during training to get better at this prediction task. More parameters generally means more capacity to capture complex patterns, though not always better real-world performance.

How LLMs are trained

Training happens in below stages:

1. Pretraining: The model is fed enormous amounts of text (books, websites, code) and learns to predict the next token, without anyone manually labeling the data. This stage is called self-supervised learning, and it’s where the model builds its general understanding of language.
2. Fine-tuning: The pretrained model is then trained further on a smaller, labeled dataset for a specific task or domain, like customer support or legal text, to sharpen its accuracy for that use case.
3. Alignment/RLHF (optional): Many modern LLMs go through an extra step using human feedback, so the model’s responses better match what people actually find helpful or safe.

Training a large model takes serious compute. GPT-4’s training run is estimated to have used around 50 gigawatt-hours of energy, which is one reason GPU infrastructure matters so much in this space.

Types of LLMs

Below are the types of LLM:

• Foundation models: General-purpose models trained on broad data (GPT-4, Gemini, Llama). Good at a wide range of tasks out of the box.
• Fine-tuned models: A foundation model further trained on domain-specific data, for example, a version tuned specifically for healthcare or legal text.
• Open-source vs proprietary: Open models (like Llama) can be downloaded and run on your own infrastructure. Proprietary models (like GPT-4) are accessed only through an API.
• Multimodal models: Extend beyond text to also understand or generate images, audio, or video. Sometimes called large multimodal models (LMMs).

Where LLMs fall short

There are few instances where LLMs falls short, such as:

• Hallucination: When a model doesn’t have enough reliable information, it can generate answers that sound confident but are simply wrong.
• Outdated knowledge: A model only knows what was in its training data, up to a certain cutoff date.
• Limited explainability: LLMs are good at producing an answer, but not always good at explaining exactly how they got there.

Techniques like retrieval-augmented generation (RAG) help with some of this by letting a model pull in current, specific data instead of relying only on what it learned during training.