Question
Which of the following statements with regard to Large Language Models (LLMs) used in machine learning is/are correct?
1. LLMs assign probabilities to the next possible words and then pick the one with the highest probability.
2. LLMs process data through mathematical optimisation to minimise prediction errors.
3. LLMs produce unbiased outputs.
Explanation
Large Language Models (LLMs) are built around statistical prediction and mathematical optimisation. However, their outputs can also reflect patterns and biases present in the data on which they are trained.
Statement 1 — Correct: LLMs generate text by estimating the probability of possible next tokens based on the given context. The model then selects the next token, either by choosing the most probable one or through sampling methods such as temperature and top-k sampling.
Statement 2 — Correct: LLM training involves optimisation of model parameters using methods such as backpropagation and Stochastic Gradient Descent (SGD). The objective is to minimise a loss function, which measures the difference between the model’s predictions and the training data.
Statement 3 — Incorrect: LLMs are not inherently unbiased. Since they learn from large amounts of human-generated data, they can reproduce or amplify social, cultural and historical biases present in their training data.
Therefore, Statements 1 and 2 are correct.