Yeah, no basement dweller dev's are gonna be messing with that API until the costs drop by at least 100x, which I honestly only see as a near term incentive for Meta to get a Llama Voice model cookin'
I'll use it, but can't wait for an uncensored open source version. Text only is too boring. I lack the patience to use text only for too long for the tasks I want, like learning languages.
The Realtime API cost is high. I suggest that there is a cheaper way. 1.Using Google STT to get user's speech texts. 2.Send texts to GPT. 3. Get responses from GPT. 4.Send responses to Google TTS. 5.User gets AI responses in both texts and voices. The response time is longer and it costs lower.
In that case, you don't need to use realtime API. OpenAI chat completion API I think works just fine. I think the key point is that realtime API has the ability to not miss any information from your voice (tone, intonation or accent), which means it can feel you like a real person, as least it is trying to.
Happy to be the first to comment. Kris you are always up to date. Once again cool stuff from you. Spaghetti code... 🤣. Great that you did talk about the costs as well. I like your creative and often real funny ideas. Please keep up the great work! Regarding your phone call: saw a video from a guy in the US weeks ago (no Realtime API) - he did let his AI order a Pizza and it worked great. Latency even back then was good enough - should work perfectly. Maybe try it with an italian accent 😉. Thx from Tom!
Great video, thanks Kris! I'm interesting in the function calling and structured output from the voice websocket return. Can you use agents or agentic flows with constrained and structured outputs with the voice mode 🤔
I don't quite understand what realtime means here, especially in text version In voice version, yes, you can interact with it like really talking to a person, such as you can interrupt the conversation, or maybe openAI can understand extra information from your tone or intonation or accent. But in text version, I don't see any difference with just use OpenAI chat completion API
Could you achieve these results in an app just using the text to speech and speech to text with native ios features alongside openai NON realtime api's?
Would love the bankrupt myself with your code, i wont judge spaghetti, tried for 20 prompts with the new claude to get it up and running - no dice. Examples would be much apricated :)
Can't you just better prompt it to have a less talkative output so you don't have to break it's response that often? That would make a big difference and everything more seamless :)
No one is going to be even able to develop at these prices other than those with deep pockets. Just testing and figuring things out would be too expensive to even try.
Yeah, no basement dweller dev's are gonna be messing with that API until the costs drop by at least 100x, which I honestly only see as a near term incentive for Meta to get a Llama Voice model cookin'
I'll use it, but can't wait for an uncensored open source version. Text only is too boring. I lack the patience to use text only for too long for the tasks I want, like learning languages.
Well said, 3 tests today ~2mins each conversation. $1.5. Yikes!
can you share the url to the repo?
where can i find the code? pls help
Looking forward to seeing your alleged "spaghetti" code! (Right now 2 weeks ago is your latest repo)
Great work! You must have had a busy couple of days getting it working
you can really implement it in a few hours
I don't think this is the same model as advanced voice mode.
The Realtime API cost is high. I suggest that there is a cheaper way. 1.Using Google STT to get user's speech texts. 2.Send texts to GPT. 3. Get responses from GPT. 4.Send responses to Google TTS. 5.User gets AI responses in both texts and voices. The response time is longer and it costs lower.
In that case, you don't need to use realtime API. OpenAI chat completion API I think works just fine.
I think the key point is that realtime API has the ability to not miss any information from your voice (tone, intonation or accent), which means it can feel you like a real person, as least it is trying to.
Happy to be the first to comment. Kris you are always up to date. Once again cool stuff from you. Spaghetti code... 🤣. Great that you did talk about the costs as well. I like your creative and often real funny ideas. Please keep up the great work! Regarding your phone call: saw a video from a guy in the US weeks ago (no Realtime API) - he did let his AI order a Pizza and it worked great. Latency even back then was good enough - should work perfectly. Maybe try it with an italian accent 😉. Thx from Tom!
I just integrated it on Twilio, it changes everything, but it took me a bit of time.
Great video, thanks Kris! I'm interesting in the function calling and structured output from the voice websocket return. Can you use agents or agentic flows with constrained and structured outputs with the voice mode 🤔
Which function controls the interruption?
VAD
I don't quite understand what realtime means here, especially in text version
In voice version, yes, you can interact with it like really talking to a person, such as you can interrupt the conversation, or maybe openAI can understand extra information from your tone or intonation or accent.
But in text version, I don't see any difference with just use OpenAI chat completion API
Could you achieve these results in an app just using the text to speech and speech to text with native ios features alongside openai NON realtime api's?
Would love the bankrupt myself with your code, i wont judge spaghetti, tried for 20 prompts with the new claude to get it up and running - no dice. Examples would be much apricated :)
Where to find code?
can we have speech/voice as input to this app using websockets and get result as text as output?
This is mindblowing...
can It work for other languages such as urdu, hindi?
Why wouldn't you share the repo?
Does everyone have access to this beta? Anything we have to do?
where can i get the repo?
Can't you just better prompt it to have a less talkative output so you don't have to break it's response that often? That would make a big difference and everything more seamless :)
where is the code?
Thanks :)
can you provide a code ? please
I still don't have access to it :/
What is the code used?
It's not out yet
@@khalifarmili1256 How long will it take?
What is AVA?
Doesn't it have emotions?
No one is going to be even able to develop at these prices other than those with deep pockets. Just testing and figuring things out would be too expensive to even try.
i tested yesterday ,but
Error al conectar: 403
Acceso denegado. Verifica tu clave de API y los permisos para usar el API Realtime.
try checking your api key or just making a new one
@@elprox1290 again, thanks
By telling it it is playing a game with the user, it might be failing on purpose to let you win!
Im waiting to hear the Irish accent to be sure
How to get the repo ?
Can you share the repo link ?
Can you share the repo link ?