What Large Language Models can and can't do

AI in Teaching and Learning series
Part 1 Part 2 Part 3

Image by Stable Diffusion (another tool for generative content)

What can and can't an LLM do?

Some things that ChatGPT can't do are pretty obvious from a legal standpoint, such as offering medical or legal 'advice'. It also can't comment on anything that's happened since about 2021 (ChatGPT3.5, as at May 2023) as all the data it was trained on pre-dates that. ChatGPT can't use the internet either, which may come as a surprise, though Microsoft's Bing Chat can. I won't go into the specifics of what each LLM can and can't do, but note that some capabilities differ between them, often due to training set, source or simply whether or not it can access the internet.

Importantly for academic settings, it can't cite references. Where it does, they tend to be entirely fabricated or unrelated to the content. This does make it simple to check if a student has used ChatGPT to completely write their assignment for them, but we will come back to that later. When it makes up references or includes incorrect data stated as fact, the phenomenon is called 'hallucinating', and occurs when the model doesn't have the contextual understanding required for the input provided. Asking it to generate a detailed description of a scientific process will usually lead to some very clearly wrong hallucinations where it reaches the limits of what it has been trained on. It also can't verify whether or not something is correct, and can give conflicting answers to a question if asked more than once, even if that data formed part of the training set.

Things that LLMs are very good at include summarising a body of text, such as creating a research abstract or summary; improving grammar and syntax for a provided piece of writing, including changing the tone for specific audiences; creating template responses for email; language translation; and data analysis. With the last two, the quality and accuracy of the results will depend on which particular LLM you use, of course. A model trained on translating language will far outperform one trained on a purely English-language database (ChatGPT has been trained on a range of languages, and as such will be better than many, but probably less accurate than say Google Translate or BLOOM).

As the name would suggest, one of the key elements of ChatGPT is offering a conversational interface. This makes interacting with the LLM far simpler for most people, as it doesn't have a learning curve that most new software presents, making the barrier to entry nearly zero. It's largely this accessibility of the technology that has driven the popularity of ChatGPT in the last six months.

Prompt crafting

Having the OpenAI window open on the side (as I have while writing this piece) allows some quick testing of prompt output and crafting, suggestions for topics, and testing some of the limits of what you're able to get from the output. As an example, here's some prompt crafting:

> As a scientist, tell me about the glucose pathway from glucose to pyruvate

The glucose pathway from glucose to pyruvate is called glycolysis, which is a central metabolic pathway that occurs in the cytoplasm of most organisms. The process involves the conversion of glucose, a six-carbon sugar, into two molecules of pyruvate, a three-carbon molecule. Glycolysis occurs in ten steps, which can be divided into two phases: the preparatory phase and the payoff phase. In the preparatory phase, two molecules of ATP are consumed in the initial steps of the pathway to convert glucose into fructose-1,6-bisphosphate, a six-carbon intermediate. This intermediate is then cleaved into two three-carbon molecules, glyceraldehyde-3-phosphate and dihydroxyacetone phosphate, which are isomers of each other. In the payoff phase, the two molecules of glyceraldehyde-3-phosphate are further metabolized to yield two molecules of pyruvate. This phase generates energy in the form of four molecules of ATP and two molecules of NADH (nicotinamide adenine dinucleotide), which can be used by the cell for various metabolic processes. Overall, glycolysis is a critical pathway in cellular metabolism, as it provides the cell with energy in the form of ATP and generates precursor molecules for the synthesis of other important biomolecules.

> As a primary school teacher, tell me about the glucose pathway from glucose to pyruvate

Sure! The glucose pathway from glucose to pyruvate is called glycolysis, and it's a process that occurs in the cells of living organisms. It's a series of chemical reactions that turn glucose, which is a type of sugar, into pyruvate, which is a type of molecule. During glycolysis, glucose is broken down into smaller molecules through a series of steps. This process releases energy that is used to produce ATP, which is the energy currency of the cell. The process of glycolysis can be broken down into two main stages: the energy-requiring stage and the energy-releasing stage. In the energy-requiring stage, glucose is converted into a molecule called fructose-1,6-bisphosphate. This step requires the use of two ATP molecules. In the energy-releasing stage, the fructose-1,6-bisphosphate is broken down into two smaller molecules called pyruvate. This step releases energy that is used to produce four ATP molecules. Pyruvate is then used by the cell in a number of different ways. It can be converted into other molecules that the cell needs for growth and repair, or it can be used to produce more energy through other metabolic pathways. Overall, glycolysis is an important process that allows cells to convert glucose into energy. It's a fundamental process that occurs in all living organisms, from tiny bacteria to large mammals like ourselves.

Hrmm. That didn't give me what I was after. Let's try specifying the target audience rather than the source:

> Tell me about the glucose pathway from glucose to pyruvate as you would explain it to a primary school student

Sure, I'd be happy to explain the glucose pathway from glucose to pyruvate in a way that's easy for primary school students to understand! So, our body needs energy to do everything from running and playing to thinking and breathing. One of the ways our body gets energy is by breaking down glucose, which is a type of sugar that we get from the food we eat. The process of breaking down glucose into energy is called glycolysis. During glycolysis, glucose is turned into a molecule called pyruvate, which the cell can then use to produce energy. Glycolysis happens in two stages. In the first stage, the cell uses a little bit of energy to get things started. It's kind of like starting a car - you need a little bit of energy to get the engine going. In this case, the cell uses two small energy molecules called ATP to get glycolysis started. In the second stage, the glucose is broken down into pyruvate. This releases energy, which the cell can use to make more ATP molecules. Think of it like a big energy factory inside the cell! The cell uses the energy from glycolysis to power all the different things it needs to do, like grow and repair itself, and even think! Overall, glycolysis is a really important process that helps us get the energy we need to live, grow, and play. Without glycolysis, our body wouldn't be able to function properly!

Great! That's what I was after here. This is similar to the type of crafting you might use for generating a lay summary for a grant, or generating an abstract for a paper. It's this tweaking of the input prompt that differentiates an okay response versus one that's actually going to be useful to you, regardless of what that prompt/response set may be addressing.

Let's now move on to how we can use this directly in our own teaching practice.

AI in Teaching and Learning series
Part 1 Part 2 Part 3
Written by Jairus Bowne, Academic Specialist in the School of Biomedical Science's Digital Learning Hub and ChatGPT.