ESP32 SD Card Speedup With a Couple of Lines of Code
HTML-код
- Опубликовано: 9 окт 2023
- The ESP32-S3 can act as a USB Mass Storage Class device (USBMSC) - but how well does it work? Once again we're using my custom PCB from www.PCBWay.com/?from=atomic
It starts off pretty slow, but there's quite a few optimisations that we can make to get it quite a bit faster!
I'm using the USBMSC class to mount the SD Card connected to the ESP32 to my PC. It does work - but it's a bit slow - but we can make it faster!
The GitHub repo with my tests is here: github.com/atomic14/esp32-sdc...
---
If you'd like to help support the channel and help me buy more nonsense - I have a Patreon page: / atomic14
We've also got a shop now! shop.atomic14.com
And if you're shopping on AliExpress or Amazon then if you click on these links I will get a little bit of affiliate money:
Aliexpress - s.click.aliexpress.com/e/_Ac05mF
Amazon - www.amazon.com/?tag=atomic14-20
You can also join us on Discord here: / discord
There's also the Atomic14 newsletter here: atomic14.substack.com/
And don't forget about the MakerNews newsletter - makernews.substack.com/ 
Наука
If you'd like to help support the channel and help me buy more nonsense - I have a Patreon page: www.patreon.com/atomic14
We've also got a shop now! shop.atomic14.com
And if you're shopping on AliExpress or Amazon then if you click on these links I will get a little bit of affiliate money:
Aliexpress - s.click.aliexpress.com/e/_Ac05mF
Amazon - www.amazon.com/?tag=atomic14-20
You can also join us on Discord here: discord.gg/h2DDnDdjD2
There's also the Atomic14 newsletter here: atomic14.substack.com/
And don't forget about the MakerNews newsletter - makernews.substack.com/
😮 This was the hard one;
That’s me “😊😊😊” when I hear “we built a new pcb”
Thanks buddy
Alot of this is over my head but I think I’m gaining an appreciation for possibilities and limitations of these MCs
I see you found the rabbit-hole!
Yep, down we go...
I'm loving the project! The improvements made with a few changes on the code are tremendous, and that could also work but by adding a USB controller and a SD card slot the writting to the PCB the speeds will increase by significant ammounts. I'm really keen to see how this project goes
I’ve got the new schematic. Working on the PCB layout. It’s a lot simpler than the idea i Gad in the video as I’ve got an IC (thanks to a viewer comment!) that will put everything in Hugh impedance when’s it’s held in reset. But there’s now a lot more to layout and I’m trying to avoid going to a four layer board as the cost jumps up a lot..
Thanks for the helpful tips! Really appreciate the clear instructions, you made it so simple!
Glad it was helpful!
I'm still following with interest. The serial I/O bottlenecks are news to me, so this episode is really paying off. I follow with rapt attention. Continued Success!
Using USBMSC on ESP32-S3 I've had good results writing and reading to/from Winbond flash chips instead of SD notwithstanding the USB1.1 limitations. I like the idea of a dedicated chip instead of a removable medium. I've also tried FRAM which is very fast - almost as fast as SRAM - but capacities at the time of writing are small. IDF and Adafruit libs are available to support serial Flash / Fram. Wear leveling for Flash is even supported.
That's interesting, what kind of capacity do you get with the flash chips?
@@atomic14 128Mbit so 16MB. There are larger capacities but I believe the addressing scheme changes above 128. The chips are NOR rather than NAND flash so there are further advantages there. Look up Winbond W25Q128JV.
@@atomic14 There's also this chip: Adafruit SPI Flash SD Card - XTSD 512 MB
I bought one but there were no real advantages over an SD card and the OEM silicon was still pretty expensive.
@@atomic14 FRAM is definitely the way to go for fast lower capscity MCU storage. You don't read much about it but I think it has potential. Great endurance too so you don't have to worry about wear or implementing wear levelling. You just treat it as non-volatile SRAM. Reliably clocked at 80MHz SPI in my experiments.
What library did you use to simulate an USB Mass storage with a Winbond flash chip?
Thanks, this was very helpful
Glad it helped!
Whelp, that GL823K datasheet ain't great... but the status LED and GPIO might be ways to give you status and a way to do the switching after it's done. If you're doing a PCB to play with these things, I'd add in a couple of DNP resistor/jumpers to experiment with these, in addition to just keeping an eye on vbus. Curious to see how it works out with this IC.
werry interesting, was also looking for solution how to switch sd card from any device to usb, and of course best to usb3 :) thanks for sharing !!
Fingers crossed it actually works...
Neat!
Excellent!
Thanks for your support!
If ESP32 monitors the Presence of USB (5V + signal lines) then ESP32 can switch properly switch between computer reading/writing SD Card and ESP32 reading/writing SD card. It means finishing the ESP32 SC Card read/write cycle and enabling USB access to the SD Card. If ESP32 is not switched ON (by battery) then Logic is powered by USB 5V and access to SC Card via USB is enabled. If ESP32 is charged via charger (no USB signals are present) then Charging and reading/writing SC Card via ESP32 is enabled.
As well, USB switch IC is not needed. :-)
Excelente, magnifico trabajo.
Maybe instead of a USB mux, just use a hub ic and automatically mux the SD card to the GL823 card reader when you sense that USB is connected.
Interesting idea - I'll take a look at options.
I think the upload speed via USB1.1 is acceptable but the SPI read speed of SD is not. A background copy of SD data to PSRAM might help in some circumstances - like a picture frame or gif player. At least then your display will populate quickly on demand 'cos the data are buffered in a much faster storage.
Thank you for the very informative video. I have a problem with a software I'm developing: I read an Audio file from SD and play it via i2s. The reading is 3072 bytes at a time. It happens every now and then that the reading takes 500/600 ms instead of the 15ms it normally does. This behavior is not periodic but random. This ruins the audio streaming. Have you ever encountered this problem? I use esp32s3
I'm looking for code to read/write individual sectors on an SD card.
Can somebody point me to a download/code. They seem to be mentioned here but no reference
Gatchaman!!!!
How about using RTOS to run two threads, one reading data from the USB and buffering, and another to write from the buffer to the SD-card?
That’s what we end up doing really, the writes are pushed onto a queue and then written by a background task. While that’s happening more data is read from the USB.
Using the GL823 is an excellent solution. I have used a similar setup in the past with GL835. GL835 has a power-saving mode so you don't need the second switch (PI3A27518), just put the ESP32 pins into Hi-Z mode when connecting USB. GL835 can go into sleep/standby mode when USB is not connected. I'm not sure this will work with GL823 though.
Interesting! Getting rid of that switch would be ideal. When it's in power saving mode can it still be connected to the USB data lines? Ideally I'd like to be able to use the USB for programming and debugging when not in mass storage mode.
The GL835 seems to be very hard to find - I guess USB2.0 is old tech nowadays.
@@atomic14When the USB switch sends data to the ESP32, the lines are disconnected from the GL823 and you can use it for debug. I just don't know if GL823 will release the data lines to the SD card when the USB lines are disconnected.
Gotcha - if it can be used like that it would be ideal. Or even if holding it reset will keep the GL823 pins in high z.
@@jespernoer5521Looks like the USB224X will do what I need - it will definitely go into tristate when in reset mode. I think I might even be able to do away with the USB switch as when it's in reset mode the data pins are high Z as well.
Great !!!
Thanks - glad you liked it.
@@atomic14 thanks to you !!!
how can i upload a code in esp32 dev board using Arduino IDE.... NEED HELP
This for storage + the Linux "hack" on the ESP32-S3.. hmmmmmm
😮 This was the hard one;
That’s me “😊😊😊” when I hear “we built a new pcb”
Thanks buddy
No problem 👍