This is one of the best topics.. Electronics today not only tear down, or hardware stuff, software "firmware" becoming an integrated part of almost any H.W. I demand more of these stuff!!
I've just begun really getting into Arduino and embedded systems programming, so this is answers a TON of questions that hadn't quite formed in my mind. This is massively helpful in getting the big picture of what exactly is happening with the bootloader and why. Thank you!
14:47 Okay, that's an excellent tip. I've been going back to the this video periodically over the years to keep it in my head for future. Now I'm about to implement this on an ARM chip so I am able to do firmware updates over a CAN bus for an art installation which has a lot of devices that may need firmware updates.
Yes - build some stuff & put it online to show what you can do. Demonstrated ability to get things done is more useful to an employer than paper qualifications.
Very good talk. We mere mortals don't usually get to see what is done behind the scenes like this. I would like to see more like this (in addition to the teardowns for which you would not be off the hook for, of course).
I remember the good ol' days when a friend of mine was updating his firware for his PC BIOS and it failed. He almost cried as he thought he had bricked his motherboard but lucky him i had the same BIOS as him. The BIOS chips could be plucked out and i inserted my chip on his motherboard thus allowing him to boot. Then we switched to the previous bad chip and restarted the flash procedure and this time it succeeded. He was so happy!
you could have a simple crypto algo built into the bootloader, using a random one-time key stored in on-chip eeprom, so when the app calls the bootloader, it can decrypt it with the key from on-chip eeprom so the external eeprom is useless without the contents of the secure on-chip eeprom
I talk about IP protection; like some clever control or DSP algorithm that you want to keep private. Doing some ciphering in the app firmware, but then storing it deciphered on a FLASH chip sounds like a bad idea. Whenever data is in motion or stationary at a place that's accessible, it will be found and reversing goes from there. That's something you either want to severely slow down or stop completely.
If you were going to use an SD card for update, you could still use a traditional bootloader using the reasonably simple SPI interface of the cards. A filesystem like FAT could be a pain but you could always write a little PC app to distribute updates that just shoves the raw data on the card, sans filesystem.
One overcomplicated way of doing i went for is using a bootloader to load the firmware from FAT every time it boots, so a firmware update is simply replacing a file. It was a ARM that boots from external flash anyway and once the app code runs its set to use the memory that the bootloader was in as stack or something so that it simply overwrites as the app runs and uses it as ram.
Even if you did flash a corrupt firmware onto the micro. Having the known good copy in EEPROM would solve that also. As long as the "copy" function gets executed at reset it can detect a button-hold or some special signal to force a copy of the known good firmware.
Very neat idea Mike, thanks! I might have to use it in near future. When I can afford it, I will use half of the flash to store the new firmware. I might be wrong but just relocating the application code won't work well with absolute jumps and data segments.
Sounds like a cool idea, as long as there's some user-accessible way to use the PC failsafe interface to try again or even downgrade. In some cases though, the entire micro might have a simple program that's strictly read-only and only handles booting from an external flash ROM or something. A bit risky if you get a batch of devices out with a buggy bootloader though, heh.
It's a great idea my only concern is that it's one way only, if the app code is bad then you've bricked the device as you say. Why not enhance the solution by having two EE sections, one for current known good and one for the new code. The boot loader can then load/copy from either to fail back if required. I really enjoyed your video, excellent explanation.
If I get the gist of this you basically use your application as a second stage bootloader? I'm not sure if you noted it or not.. but it's worth noting that vendor bootloaders can be a complete pile of crap and that might be another reason you want to move some of it's functions into your own code (if the bootloader section isn't rewritable).
Cheers, Love your videos by the way. One other little thing, I'm currently in final year of EEE in England, it's job hunting time. Would you have any advice for someone who loves sitting down and building something from start to finish. Initial design, prototyping to case and PCB design. I much prefer this over the regular graduate jobs. Would you advise on moving away? Any help would be greatly appreciated =]
I have done this before and placed the "copy" function in RAM wish means that there is no separate boot loader, there is just one program. Of cause you can never bootload "broken/bad" code, that would brick the device.
Storing your whole program, even back-up images, sounds great if you add a tiny bit of start-up logic to check apps and restore backups, but doesn't it make it vulnerable to reverse engineering? Like say, you put code protection on, but if a flash chip contains the program, it has no use. Unless you use some kind of ciphering method that quickly can cipher and decipher the data, I guess it's fine though. Just a thing to watch out for.
If my main application takes up 1mbit of storage, won't this method require me to double that? Unless you have more than that available in RAM to potentially store the image, I can;t see a way around it...
Thanks for the video mike, Just a thought, when we have a EEPROM (SPI based).... Is the idea of Firmware Ver 1.0 (Flash) swapping with EEPROM (Firmware Version 1.1) by using small amount of swapping space on the Micro-controller Flash possible? This gives the advantage if there was a problem with Ver 1.1 to downgraded to 1.0....
True, but a lot of what Mike does is essentially a one off, the value add is that the customer comes to him for the update. Adding faux protection will only shoot himself in the foot down the line. Keeping is simple, short and easy works. 99% of the buyers would ask the dev team for updates, and likely either that is in the purchase price or in a support agreement anyway. If you copy it you are SOL and going up the creek where the stuff under the canoe aint water, and you gonna have to eat it.
Aside from processors like the PIC32 which can execute out of RAM, I fail to see the usefulness of that approach. If you're going to have to reserve space anyway to store your bootloader image, why not keep it there from the get go instead of storing it on an external device? On PIC24s for example you can also code protect that area.
And every time I encounter something like this it pisses me off. It's no fun dealing with an EOL product that's quite easy to fix - if only you'd have the clear code. In the long run you gain a lot more keeping it open-sourced . Or at least releasing the clear firmware dump once you've decide to end the support for that product.. But you don't necesarily need to store the entire image. You can just store a recovery image that contains the code needed for the update process. Saves space too lol
You're a lot more in depth and computer oriented than the EEVblog guy. Thanks for making useful videos.
This is one of the best topics.. Electronics today not only tear down, or hardware stuff, software "firmware" becoming an integrated part of almost any H.W. I demand more of these stuff!!
I've just begun really getting into Arduino and embedded systems programming, so this is answers a TON of questions that hadn't quite formed in my mind. This is massively helpful in getting the big picture of what exactly is happening with the bootloader and why.
Thank you!
14:47 Okay, that's an excellent tip. I've been going back to the this video periodically over the years to keep it in my head for future. Now I'm about to implement this on an ARM chip so I am able to do firmware updates over a CAN bus for an art installation which has a lot of devices that may need firmware updates.
Yes - build some stuff & put it online to show what you can do. Demonstrated ability to get things done is more useful to an employer than paper qualifications.
Very good talk. We mere mortals don't usually get to see what is done behind the scenes like this. I would like to see more like this (in addition to the teardowns for which you would not be off the hook for, of course).
I remember the good ol' days when a friend of mine was updating his firware for his PC BIOS and it failed.
He almost cried as he thought he had bricked his motherboard but lucky him i had the same BIOS as him. The BIOS chips could be plucked out and i inserted my chip on his motherboard thus allowing him to boot. Then we switched to the previous bad chip and restarted the flash procedure and this time it succeeded.
He was so happy!
I really connect with the way you explain things. Thanks for posting this video.
you could have a simple crypto algo built into the bootloader, using a random one-time key stored in on-chip eeprom, so when the app calls the bootloader, it can decrypt it with the key from on-chip eeprom
so the external eeprom is useless without the contents of the secure on-chip eeprom
Excellent stuff Mike, excellent! Keep it up, your tips are most welcomed. After all a good/clever design always means less MTBF!!
Very interesting stuff. I liked your diagrams.
I talk about IP protection; like some clever control or DSP algorithm that you want to keep private. Doing some ciphering in the app firmware, but then storing it deciphered on a FLASH chip sounds like a bad idea. Whenever data is in motion or stationary at a place that's accessible, it will be found and reversing goes from there. That's something you either want to severely slow down or stop completely.
It is applicable to _any_ micro that has self-programming capability, which nowadays is pretty much all of them. Even tiny ones like the PIC10F322!.
If you were going to use an SD card for update, you could still use a traditional bootloader using the reasonably simple SPI interface of the cards. A filesystem like FAT could be a pain but you could always write a little PC app to distribute updates that just shoves the raw data on the card, sans filesystem.
One overcomplicated way of doing i went for is using a bootloader to load the firmware from FAT every time it boots, so a firmware update is simply replacing a file. It was a ARM that boots from external flash anyway and once the app code runs its set to use the memory that the bootloader was in as stack or something so that it simply overwrites as the app runs and uses it as ram.
i like it when people share what they think about ...
Great video, really useful suggestions - i'm going to try some of them.
Even if you did flash a corrupt firmware onto the micro. Having the known good copy in EEPROM would solve that also. As long as the "copy" function gets executed at reset it can detect a button-hold or some special signal to force a copy of the known good firmware.
Very neat idea Mike, thanks! I might have to use it in near future. When I can afford it, I will use half of the flash to store the new firmware. I might be wrong but just relocating the application code won't work well with absolute jumps and data segments.
I have a sudden urge to order a load of eeprom chips.
Spot on :-) I left the comment before the end of the video as you can tell.
excelent tutorial mike, thank you!
Good stuff!
Ingenious! :)
Sounds like a cool idea, as long as there's some user-accessible way to use the PC failsafe interface to try again or even downgrade. In some cases though, the entire micro might have a simple program that's strictly read-only and only handles booting from an external flash ROM or something. A bit risky if you get a batch of devices out with a buggy bootloader though, heh.
It's a great idea my only concern is that it's one way only, if the app code is bad then you've bricked the device as you say. Why not enhance the solution by having two EE sections, one for current known good and one for the new code. The boot loader can then load/copy from either to fail back if required. I really enjoyed your video, excellent explanation.
Lol I was expecting satellite dishs after NFC and Bluetooth - "A little something like this for communicating with planetary rovers perhaps"
Thanks for this video. Brilliant :)
If I get the gist of this you basically use your application as a second stage bootloader? I'm not sure if you noted it or not.. but it's worth noting that vendor bootloaders can be a complete pile of crap and that might be another reason you want to move some of it's functions into your own code (if the bootloader section isn't rewritable).
Cheers, Love your videos by the way. One other little thing, I'm currently in final year of EEE in England, it's job hunting time. Would you have any advice for someone who loves sitting down and building something from start to finish. Initial design, prototyping to case and PCB design. I much prefer this over the regular graduate jobs. Would you advise on moving away? Any help would be greatly appreciated =]
I have done this before and placed the "copy" function in RAM wish means that there is no separate boot loader, there is just one program. Of cause you can never bootload "broken/bad" code, that would brick the device.
Never underestimate the ability of professional software developers to skip regression testing because "it's always worked before."
Storing your whole program, even back-up images, sounds great if you add a tiny bit of start-up logic to check apps and restore backups, but doesn't it make it vulnerable to reverse engineering? Like say, you put code protection on, but if a flash chip contains the program, it has no use.
Unless you use some kind of ciphering method that quickly can cipher and decipher the data, I guess it's fine though. Just a thing to watch out for.
as always a tad over my head but severely entertaining and educational nevertheless.
Also do you have an example of this code implementation that you can share ???
Could you please say something about check integrity and checksum of APP code. Thanks.
If my main application takes up 1mbit of storage, won't this method require me to double that? Unless you have more than that available in RAM to potentially store the image, I can;t see a way around it...
It probably works best on PIC32 since you can execute out of RAM. In this case you t need to reserve flash only for the copy routine.
NFC (Near Field Communication, similar to RFID)
Thanks for the video mike,
Just a thought, when we have a EEPROM (SPI based).... Is the idea of Firmware Ver 1.0 (Flash) swapping with EEPROM (Firmware Version 1.1) by using small amount of swapping space on the Micro-controller Flash possible? This gives the advantage if there was a problem with Ver 1.1 to downgraded to 1.0....
What was that after WiFi ? @04:07 "an FC perhaps"
NFC
True, but a lot of what Mike does is essentially a one off, the value add is that the customer comes to him for the update. Adding faux protection will only shoot himself in the foot down the line. Keeping is simple, short and easy works. 99% of the buyers would ask the dev team for updates, and likely either that is in the purchase price or in a support agreement anyway. If you copy it you are SOL and going up the creek where the stuff under the canoe aint water, and you gonna have to eat it.
Would this work on PIC32?
Aside from processors like the PIC32 which can execute out of RAM, I fail to see the usefulness of that approach. If you're going to have to reserve space anyway to store your bootloader image, why not keep it there from the get go instead of storing it on an external device?
On PIC24s for example you can also code protect that area.
LOL 12:03. You must stop "sugar coating" things Mike! Nice vid.
Thank you!
And every time I encounter something like this it pisses me off. It's no fun dealing with an EOL product that's quite easy to fix - if only you'd have the clear code.
In the long run you gain a lot more keeping it open-sourced . Or at least releasing the clear firmware dump once you've decide to end the support for that product..
But you don't necesarily need to store the entire image. You can just store a recovery image that contains the code needed for the update process. Saves space too lol
words of thought 2013 edition
15:49 "You dont have to stick you hand up the backside of a TCPIP stack"
you know what?...looks like doing fractals...divide by the same code every time....looks like it work everywhere...
Sounds like the evil nightmare of microcontrollers v's code.
:)
This reminds me of what an FPGA does.
That's why tablets & phones say they're done updating in 5 seconds, then take 20 minutes to reboot.
is it me or have you sped up the video.
AH THAT PEN!
you can spike mor fast plisssssssssssssssss
could you talk a little faster please?