In this lesson, we will discuss how to compute words using language models. We will start by exploring the concept of word-to-word translation and how it can be achieved through mathematical operations on vectors.
To illustrate this concept, let’s consider the equation ‘King - Man + Woman = ?’. If we could apply math to words, we would expect the result to be ‘Queen’. However, since we cannot directly add or subtract words, we need to convert them into numbers.
To convert words into numbers, we can use a mapping called word to VEC, which assigns each word a vector representation. These vectors are collections of numbers that capture the meaning of the words. For example, the French word ‘ja’ maps to the vector [0.8, 1.1], and the English word ‘I’ maps to [0.1, 9].
Now, let’s focus on translating the word ‘je’ (French for ‘I’) into English. We subtract the French vector and add the English vector, resulting in a new vector that corresponds to the word ‘I’. This process of performing mathematical operations on vectors represents the computation on words.
To perform these computations, we use a multi-layer perceptron, which is a type of neural network. This network can handle various operations like addition, multiplication, and subtraction. It allows us to represent any word-to-word translation task.
Once we have the translated vector, we need to convert it back into a word. We find the closest vector that maps to a word, and that word becomes the output of our translation.
In summary, the pipeline for word-to-word translation involves converting words into vectors, performing computations on these vectors using a multi-layer perceptron, and then converting the resulting vector back into a word. This process allows us to translate one word into another. However, the ultimate goal is to translate multiple input words into multiple output words, which we will explore in the next lesson.
For more resources and slides related to this topic, please visit the course website. This concludes our discussion on computation with language models. In the next lesson, we will delve into the topic of translating multiple input words into multiple output words using large language models.
