Google just released a new feature for its barred chatbot that catapults it miles ahead of chatGPT. This new feature, quietly included in a list of additions to Bard release, is a total game changer. The ability to accept images in addition to text as a prompt makes Bard officially multimodal.
I have been testing the new system since its release this morning, and I am already amazed by some of the things it can do. For example, I gave Bard a simple picture of a leaf with aphids and asked it to describe what it saw. Bard correctly identified the problem as aphids chewing on the leaf, causing it to yellow.
Google’s lens platform has been able to do similar visual processing, but Bard takes it much further. It can derive information from simple photos, such as determining a person’s age or accurately identifying objects in a picture.
In my experiments, Bard accurately identified a yellow Jelly Bean in a close-up photo, even describing the bokeh blur used in the background. It also recommended the salmon dish for someone on the Mediterranean diet when given a photo of a menu.
Bard’s new visual abilities are a huge step forward for AI chatbots. With the ability to see the world, not just understand it via text, Bard’s visual understanding appears complex and nuanced. It is not simply deriving keywords or object tags from uploaded images, but truly multimodal.
Google has an advantage in terms of data and computer power, with decades worth of tagged photos and vast processing capabilities. This advantage allows Bard to leverage its visual abilities beautifully. As Bard continues to be trained and developed, its visual functions will only grow, making it an official AI chatbot that can see.
