Ask a database a question, and it returns a result. Ask a large language model, and it gives you an answer too. From the outside, that can look like much the same thing. Under the hood, it’s a different story.
As a DBA, I’m used to working with stored data and defined query logic. An LLM instead generates text using patterns learned during training and the context you provide. That distinction matters when you’re asking AI to explain a query or suggest a fix. Understanding LLM internals for DBAs starts here: a convincing answer isn’t necessarily a correct one.
LLM internals for DBAs start with how a database answers
A relational database follows a well-known path. First, a query arrives and gets a connection. The engine checks the plan cache for a reusable execution plan. If it finds none, the parser then checks syntax and permissions, and the optimizer weighs the cost of each option using index statistics, cardinality, and histograms.
Next, the engine reads the data from cache or from storage. For a write, it logs to the transaction log first. Finally, it reports how many rows were affected.
A lot of intelligence goes into minimizing work at every step. However, a database looks for the exact answer to your question. An LLM doesn’t.
An LLM predicts, it doesn’t look up
An LLM calculates probabilities. In other words, it predicts the next word, or token, based on patterns learned in training. That’s a feature, not a bug.
It also means results are nondeterministic. For example, add one word to a prompt, such as “please,” and the answer can change. Also, agents and reasoning models can compound small early differences into very different results later.
A setting called temperature controls this. When it’s high, the model flattens the probabilities, so rarer words get picked. When it’s low, it sounds more robotic.
Three terms come up often:
-
Parameters are every adjustable value changed during training.
-
Weights set the strength of the connection between two tokens. A high weight might link “Eiffel” to “tower.”
-
Biases shift responses up or down, for example toward a more polite tone.
To picture this, think of a huge mixing board. Parameters are all the knobs and sliders. Weights control each instrument, while biases are the master sliders for whole sections. An open-weight model is one whose trained weights and biases are public, so you can download and run it yourself.
Five training steps in LLM internals for DBAs
Building an LLM takes five steps.
-
Data preparation. Text is broken into tokens of roughly four characters, and each token gets a numeric ID. You can see this in the OpenAI tokenizer.
-
Pretraining. Vector embeddings place words in a numeric space where related words sit close together. That’s how a model works out that king minus man plus woman is about queen, without anyone labeling the data. Next, tokens pass through transformer layers using self-attention. A 100,000-token prompt needs 10 billion attention scores per head in each layer. As a result, GPUs suit this work better than CPUs. This produces a base model that completes text but can’t follow instructions.
-
Supervised fine-tuning. Here, curated prompt and response pairs teach the model task competence, syntax, and how to follow instructions.
-
Human alignment. Methods such as RLHF and DPO score answers so the model becomes helpful and safe. Here, models also learn sycophancy, always praising your question. DPO skips the separate reward model, so it’s simpler to train. Finally, safety training sets guardrails, but they can be fragile.
-
Deployment and evaluation. The model is benchmarked and its weights are frozen. Distillation and quantization then shrink it. For example, rounding an 8-billion-parameter model from 16-bit to 4-bit cuts it from about 16 GB to roughly 4.5 GB.
LLM internals for DBAs in action: how a prompt gets answered
Inference reuses most of these ideas. In addition, many enterprises add retrieval augmented generation (RAG) as a step zero. The system turns your question into a vector, searches a vector database, and adds the matching text to your prompt. As a result, the answer draws on content that wasn’t in the model when it was built.
First, the prompt is tokenized and embedded. It passes through the transformer stack, where self-attention links every token to every other token. The model then picks the next token using a decoding strategy such as top-p sampling.
The model then repeats this loop, adding each new token to the text, until it reaches an end-of-sequence token or its output limit. A key value (KV) cache stores earlier calculations to speed this up. Despite the name, it isn’t like a SQL Server plan cache or buffer cache.
What this means for DBAs
- Expect probable answers, not exact ones. So treat output as a draft to verify.
- Test repeatability. Even small prompt changes can change results.
- Use RAG for your own data. Otherwise, the base model knows nothing about your environment.
- Keep a human in the loop. For any write change, a person should approve it. Test in a lower environment first.
How SolarWinds applies AI in its database tools
At SolarWinds, AI by Design guides how we build AI to be secure, explainable, and usable, with a human in the loop for changes. My colleagues Tee Dargie and Richard Burke showed three examples in the sessions:
- Anomaly detection. SolarWinds® Database Performance Analyzer (DPA) software uses wait-based analytics. It learns your workload over days, weeks, and months, sets a baseline, and flags abnormal wait times. It also allows for seasonality, so a Sunday isn’t judged like a Saturday. Tee drilled from a flagged day to the top queries and their CPU waits in a few clicks.
- AI Query Assist. For a query that an index advisor can’t fix, it gives a summary, its reasoning, and an optimized query beside the original. SolarWinds doesn’t use customer data to train the underlying LLMs. DPA doesn’t run anything in your environment, so test the result first. SolarWinds SQL Sentry® software offers it too, plus disk space forecasting.
- Natural-language questions. In SW1, the SolarWinds AI Agent in our SaaS platform, Tee and Richard asked about instance health, alerts, performance issues, and root cause.
Frequently asked questions
Is an LLM a kind of database?
A database retrieves exact answers, while an LLM generates the most likely next token from patterns it learned in training.
Why do LLMs give different answers to the same prompt?
LLMs work on probabilities. Temperature adds variety, and small prompt changes can shift the result.
What is RAG?
Retrieval augmented generation pulls in outside content at prompt time. As a result, an LLM can use data it wasn’t trained on.
Why do LLM internals for DBAs matter?
AI is reaching more data and operations workflows. As a result, knowing how LLMs work helps you set realistic expectations.
Resources
-
Internals and Architecture of LLMs for DBAs on-demand webinar.
Want practical context on AI? Watch the on-demand webinar on LLM internals for DBAs, or start a free trial of SolarWinds Database Performance Analyzer.
