Now let’s talk about question answering. Can the model solve or answer a specific question based on the context that we provided, similar to information extraction but slightly different? Because the model, we can ask the model to infer something from the data that might not be there directly. So let’s try this out with an example.
I have already written the prompt here. The first thing I’m doing is asking the model to answer the question based on the context below. This is my context in Scorly brackets that I’ve provided. Keep the answer short, respond unsure about the answer if not sure about the answer.
Now, let’s analyze the context. A very different example, but I’m going to walk you through so you understand how the model interprets it. Typically, ZoomUp (excuse my pronunciation) traces its roots to a New Jersey drug company called Ortho Pharmaceutical. Their scientists generated an early version of an antibody dubbed OKT3, originally sourced from mice. The molecule was able to bind to the surface of T cells and limit their cell killing potential. In 1986, it was approved to help prevent organ rejection after kidney transplants, making it the first therapeutic antibody allowed for human use.
Now, the question is, what is OKT3 originally sourced from? The answer is mice. Now, let’s run the same prompt but change the answer. Where is OKT4 originally sourced from? This is something completely different that is not in the context. Let’s see how the model interprets a question that we haven’t provided any reference to.
Based on the question and the context, I’m not sure how to answer this question. The context only mentions that OKT3 was originally sourced from mice, but it does not mention anything about OKT4. It was also recently sourced from another organism. More information is needed to answer the question.
So, we did a pretty good job in answering the question when we provided the right question with the right context. It was able to figure it out. But when we asked the same question but with a different question that was not there in the data already, it did a remarkable job in understanding that this information is not in the data. At the same time, it inferred that it is a possibility that, just like OKT3, OKT4 is also coming from mice, but we’re not sure yet.
Now, let’s try the same two prompts in GPT-4. This is our first prompt where we talk about OKT3, originally sourcing. Very precise, to the point. It did the job exactly what we asked for. Now, let’s try the second prompt where we give it a hyper question that is not there in the context, which is OKT4. While the information we provided is for OKT3, unsure about the answer. You see how precise it was because it responded, ‘I’m not sure about the answer.’ So it took the instruction really well in terms of saying that okay, I don’t know the answer. And it was very precise. We gave the same prompt to Bart, but it tried to infer an answer even though it wasn’t there in the passage, and it provided that. So, a slightly different way the two models interacted, but they both do the job in terms of going through the context, looking for a question’s answer, and providing us the most relevant details that they can for that question.
