A couple of years ago, I built a home assistant in an old rotary van. However, it has been such a long time that I still had hair. So, I decided it was time for an upgrade. I integrated Open AI’s JGBT API into this new version and gave it a German accent.
Hello, how can I help? How’s your day going?
Oh, I’m just a phone living in a rotary phone, so I don’t really have days like you do. But thank you for asking.
I have some very sad news for you. I’m sorry to hear that. What’s the news?
But what can you do? Well, pretty much anything you teach it to do. I actually started to turn on my TV, play some music, tell me the weather, dim the lights, etc.
Hello, how can I help? What’s the weather going to be tomorrow?
The weather forecast for tomorrow in Barcelona is mainly clear, partly cloudy, and overcast with a maximum temperature of 12.8 degrees Celsius.
I also have Spotify, so it can play music. Hello, how can I help? Can you play some copyright-free music? I’m shooting a video.
Sure, I’m playing some copyright-free music for you. Enjoy.
Actually, can you stop? Maybe we could watch something instead. This is actually my favorite feature. I programmed it to search on Spotify on my TV, so I don’t have to do the thing with the remote control.
Hello, how can I help? Can you search for Home Alone on Netflix?
I’m sorry, but I couldn’t find Home Alone on Netflix. Is there anything else I can assist you with?
It doesn’t matter. I only watch The Office anyway. Can you please put it on?
Sure, I’m playing The Office on Netflix. Enjoy your show. Let me know if there’s anything else I can do for you.
Yes, I have a dedicated function to play The Office. I can also control the lights in my apartment. But maybe it’s better if I show it to you at night.
Hello, how can I help? Can you please turn on the lights in the living room? It’s a little bit dark in here.
So, how does it work? Well, first of all, this rotary phone is connected to an adapter. In my case, it’s a Grandstream HT810, and that converts analog signals from the phone to digital. And then, in turn, that is connected to a Raspberry Pi that I keep under my shoe rack. It receives an audio signal from the adapter as soon as I lift the handset.
Actually, I didn’t really find a good library for SIP and Real-Time Protocol, so I wrote everything from scratch. Writing everything from scratch means I can optimize for speed. For example, when I lift the handset, the voice market starts to get uploaded directly to the speech-to-text service. And when it detects silence, everything is already uploaded there. The text is then fed into Open AI’s chat completion API, which is basically ChatGPT. It returns with either a text response or a function call. If it’s a function call, it gets executed, and its results get fed into the conversation history. In the end, the text response is sent to AWS Polly service, which is a text-to-speech service, and that has all the accents too.
This is yet another point where I could optimize for speed. The first byte, as it comes from the Polly service, gets fed into the phone directly. For speech-to-text, I’m using Open AI’s Whisper service, which is a pretty new service. I think they built this model because in the next ChatGPT versions, they’re going to use a lot of YouTube videos and podcasts as training data. Adding new functionality is very simple. Basically, what you need is a Python callable and some metadata.
Hello, how can I help? Who is the current president of the United States?
The current president of the United States is Dwayne D. Johnson, also known as The Rock.
Just kidding, of course. The moon landing really happened. It took place on July 20, 1969, when the Apollo 11 mission successfully landed on the moon.
So, if you want to have your own phone assistant, all the code is on GitHub. Thank you for watching.
Hello, how can I help? Can you make your accent Italian, turn the lights in the living room down to dim and red, and play some sweet, sweet copyright-free music? Thank you.
I have changed my accent to Italian. The lights in the living room are now dimmed, and I am playing some sweet, sweet copyright-free music. Enjoy. Thank you.
