In this article, we will learn how to split complex tasks into a series of simpler subtasks by chaining multiple prompts together. You might be wondering why would you split up a task into multiple prompts when you could achieve it with one prompt and Chain of Thought reasoning like we learned in the previous video. We’ve already shown that language models are very good at following complex instructions, especially the more advanced models like GPT-4.
So let me explain why we would do this with two analogies comparing Chain of Thought reasoning and chaining multiple prompts. The first analogy to compare the two is the difference between cooking a complex meal in one go versus cooking it in stages. Using one long complicated instruction can be like trying to cook a complex meal all at once, where you have to manage multiple ingredients, cooking techniques, and timings simultaneously. It can be challenging to keep track of everything and ensure that each component is cooked perfectly. Chaining prompts, on the other hand, is like cooking the meal in stages, where you focus on one component at a time, ensuring that each part is cooked correctly before moving on to the next. This approach breaks down the complexity of the task, making it easier to manage and reducing the likelihood of errors.
However, this approach might be unnecessary and overcomplicated for a very simple recipe. A slightly better analogy for the same thing is the difference between reading spaghetti code with everything in one long file and a simple modular program. The thing that can make spaghetti code bad and difficult to debug is ambiguity and complex dependencies between different parts of the logic. The same can be true of a complex single-step task submitted to a language model. Chaining prompts is a powerful strategy when you have a workflow where you can maintain the state of the system at any given point and take different actions depending on the current state. Each subtask contains only the instructions required for a single state of the task, which makes the system easier to manage, makes sure the model has all the information it needs to carry out a task, and reduces the likelihood of errors.
This approach can also reduce costs since longer prompts with more tokens cost more to run, and outlining all steps might be unnecessary in some cases. Another benefit of this approach is that it is also easier to test which steps might be failing more often or to have a human in the loop at a specific step. So, to summarize, instead of describing a whole complex workflow in dozens of bullet points or several paragraphs in one prompt like in the previous video, it might be better to keep track of the state externally and then inject relevant instructions as needed.
What makes a problem complex? In general, a problem is complex if there are many different instructions and potentially all of them could apply to any given situation. These are the cases where it could become hard for the model to reason about what to do. As you build with and interact with these models more, you’ll gain an intuition for when to use this strategy versus the previous one.
One additional benefit that I didn’t mention yet is that it also allows the model to use external tools at certain points of the workflow if necessary. For example, it might decide to look something up in a product catalog, call an API, or search a knowledge base. Something that could not be achieved with a single prompt.
So, with that, let’s dive into an example. We’re going to use the same example as in the previous video, where we want to answer a customer’s question about a specific product. But this time, we’ll use more products and also break the steps down into a number of different prompts. We’ll use the same delimiter that we’ve been using in the previous videos, and let’s read through our system message and user message into the messages array.
Now, we want to look up some information about the products that the user mentioned. So, we need to have some kind of product catalog to look up this information from. We have our product information, and for each product, we have a couple of different fields: name, category, brand, warranty, and so on. We can use helper functions to look up product information by product name or to get all of the products for a certain category.
Now, let’s generate the output string that contains the relevant product information. We can then add this string to the next instruction to the model, which is going to be the instruction where we ask it to answer the user question. We’ll use the generate output string helper function to do this.
Finally, we have our messages array, which contains the system message, user message, and additional assistant message. This is the input to the model. We can now get the final response and print it. As you can see, by breaking this up into a series of steps, we were able to load information relevant to the user query to give the model the relevant context it needed to answer the question effectively.
In conclusion, splitting complex tasks into simpler subtasks by chaining multiple prompts together is a powerful strategy for managing complex workflows and reducing the likelihood of errors. By selectively loading relevant information into the model’s context, we can provide the necessary information for the model to reason and perform useful tasks. This approach allows for more flexibility, reduces costs, and enables the model to use external tools when necessary. It’s important to consider the context limitations of language models and to dynamically load information as needed. By doing so, we can augment the capabilities of these models and create more interactive and effective user experiences.
In the next video, we will discuss how to evaluate the outputs from the language model.
