Wio Terminal TinyML - Speech Recognition
HTML-код
- Опубликовано: 21 сен 2024
- The Wio Terminal is a development platform by Seeed Studio (www.seeedstudi.... It combines a SAMD51 microcontroller with Bluetooth/WiFi connectivity, various sensors, an LED screen, a buzzer, and some buttons. While it makes for a fine IoT development kit, I wanted to see how well it would run embedded machine learning applications (TinyML). Specifically, I wanted to test its ability to handle speech recognition (or more specifically, keyword spotting).
Disclosure: Seeed Studio sent me a Wio Terminal to play with and create a video about.
I trained a convolutional neural network to identify 4 keywords: forward, left, right, stop. These words were chosen to demonstrate what you might find on something like a robot. However, any words could have been chosen (within some limits). It seems that the neural network gets a little too big or inaccurate after about 5 different keywords.
You can train your own keyword spotting system by following the steps I outlined in this GitHub repository: github.com/Sha...
The code for this demo can be found here: github.com/Sha...
Note that you will need to install the Edge Impulse Arduino library (.zip file) found here: github.com/Sha...
If you train your own model on Edge Impulse (www.edgeimpuls..., you can download and install the Arduino library containing your custom model. That will allow you to identify keywords of your choosing.
I recommend checking out my Coursera class on embedded machine learning if you'd like to learn more: www.coursera.o...
Welcome back Shawn :)
...
It would be nice if you create such demo projects with ESP32.
Good to know there's interest in doing TinyML on the ESP32 :)
This is a really cool demo, and even though I don’t have a Wio Terminal, it’s kinda giving me some ideas for an unused Adafruit PyPortal I have laying around. Thanks!
I just tried to recreate this at my end and works like a charm!
Good to hear! I have ordered two, am going to try it out as well.
This is great stuff! Well done
Shaun, Thanks for bringing this incredible device and mind-blowing online ML service to my attention.
The schematic you show at 2:24 is showing a pair of LM358 operational amplifiers in a non-inverting configuration amplifying the microphone signal. They have a combined voltage gain of 121 (=41dB) and pass the signal to the output pin via AC coupling cap. Getting sound should be possible based on this much at least, I guess if they take this to an ADC is another matter.
cheers and keep up the awesome videos, man.
Thanks for the help analyzing that circuit...op-amps were never my strong suit :) One of these days, I'll have to re-learn them. The output is tied to an ADC. When I tried capturing the ADC values and plotting them, it appeared to be some kind of peak detection of the sound signal (e.g. there was no discernable AC signal, even with a DC offset).
I actually just bought two of these and am going to attempt to recreate your project. This is so next-level.
Re: The audio circuit, the op amp output signal needs to be DC biased (and appropriately scaled) to be supply an ADC without causing clipping (distortion). Feeding an AC signal into the ADC has the effect of half wave rectifying it. With some tricky code it should be possible to still use this with reduced quality as a result.
However, If the signal is to large it will also have a flat top where the ADC maxs out. Depending on the extent of clipping, the any recovered signal could be pretty poor. Maybe a ML project in the making for this problem!
Nice. Thank you!
Hello! Thank you for your videos! I am hoping that soon you will upload PART 3 from Hadouken series!!! Thanks!!! It is awesome...
Can we implement a Translation system using this Kit by the help of googletrans Library?
Hello shawn I just completed your arduino course on udemy I loved it ,It taught me a lot .I am a freelance engineer who makes money by making projects demanded by clients I wanted to ask you can I use your content on youtube or your website to make projects?
I am formally asking for your permission
Glad to hear you enjoyed the course! You are welcome to make all the projects you see in my content. All code examples should be open source (likely MIT license--check the GitHub repository). You are welcome to link to my content (blog posts, RUclips videos), but please do not copy-and-paste the content to another site or video service.
@@ShawnHymel Thank you soo much
hi shawn..if I want to use esp32 with i2s INMP441, what should I change the code?or do you have example EI run on esp32?
Are you planning to make another course about TinyML (advanced). I already finished the introductory one on Coursera.
Yes, I'm working on one for vision right now :)
@@ShawnHymel Thanks very much
@Shawn Hymel
I come from
ruclips.net/video/0m387MkOyWw/видео.html this video
there is a error coming
Unable to fetch some
sudo apt -y install qt4-dev-tools libatlas-base-dev libhdf5-103