In the previous section, you learned about how Language Models (LLMs) can interact with external data sets. Now, let’s explore how they can interact with external applications.
To understand the types of problems and use cases that require this kind of augmentation of the LLM, let’s revisit the customer service bot example you saw earlier in the course.
During this walkthrough of a customer’s interaction with Shopbot, we’ll take a look at the integrations that are needed to allow the app to process a return request from end to end.
In this conversation, the customer expresses their desire to return some items they purchased. Shopbot responds by asking for the order number, which the customer provides. Shopbot then looks up the order number in the transaction database, possibly using a SQL query to a back-end order database.
Once Shopbot retrieves the customer’s order, the next step is to confirm the items that will be returned. The bot asks the customer if they want to return anything other than the purchased items. After the user states their answer, the bot initiates a request to the company’s shipping partner for a return label.
To request the label, the bot uses the shipper’s Python API. Shopbot asks the customer to confirm their email address, which is included in the API call to the shipper. Once the API request is completed, the bot informs the customer that the label has been sent by email, and the conversation comes to an end.
This example illustrates just one possible set of interactions that an LLM can handle to power an application. Connecting LLMs to external applications allows the model to interact with the broader world, extending their utility beyond language tasks.
LLMs can be used to trigger actions when given the ability to interact with APIs. They can also connect to other programming resources, such as a Python interpreter, enabling models to incorporate accurate calculations into their outputs.
It’s important to note that prompts and completions are at the heart of these workflows. The actions that the app will take in response to user requests are determined by the LLM, which serves as the application’s reasoning engine.
To trigger actions, the completions generated by the LLM must contain important information. First, the model needs to generate a set of instructions for the application to know what actions to take. These instructions need to be understandable and correspond to allowed actions.
Second, the completion needs to be formatted in a way that the broader application can understand. This could be as simple as a specific sentence structure or as complex as writing a script in Python or generating a SQL command.
Lastly, the model may need to collect information that allows it to validate an action. For example, in the Shopbot conversation, the application needed to verify the email address the customer used to make the original order. Any required information for validation needs to be obtained from the user and included in the completion.
Structuring the prompts correctly is crucial for all of these tasks and can make a huge difference in the quality of the generated plan or adherence to a desired output format specification.
