Would be great if you guys could have your devices as a Hat for Single Board Computers so we could use it to read the data on professional scanning tools. I say that because we can install Windows OS on a SBC and most of the professional scanning tools require Microsoft Windows.
Is translation to Parquet on the CANedge devices themselves on the roadmap? So the edge device does the DBC decoding and then uploading of the Parquet data straight to S3... Thanks for sharing this webinar, I found it very useful.
Thanks for the question! We will most likely not change the way the device records the raw data on the SD card. DBC decoding the data to e.g. decoded MF4 or decoded Parquet may seem convenient and practical at face value - and for some use cases it definitely will be. However, there are many situations where this is not ideal. 1) For example, if you made a mistake in your DBC or used a DBC file that was of an incorrect version, this would result in basically all your data becoming invalid and useless - with no way of rectifying this. See our Case Studies section with the Koenigsegg case study where they describe exactly this factor as being a key reason why they select the CANedge. 2) For some use cases, you may want to store the "full data" in the raw compact MF4 format, while you then e.g. only DBC decode a small subset (maybe 10-20 signals, which could be e.g. just 10% of your dataset) to a Parquet data lake to avoid unnecessary data duplication. In such a situation, it is ideal to be able to distinguish between the 'original' data and more 'prepared' sub data sets. 3) Another important factor is the ability to 'compact' the data ex post. Some users may upload a new MF4 log file every 1 minute, for example. This results in a huge number of files in S3 and over time, the Parquet data lake also becomes inefficient to query if you're looking across long periods of many small files. Here, the MF4 decoders let you easily "re process" the entire raw data to create a compacted data lake, where the number of files can be reduced drastically through concatenation. 4) Further, many use cases deliberately require the ability to view the raw CAN/LIN frames for more direct analysis of the original communication. This is also relevant if you're e.g. looking to analyze CAN/LIN error frames.
Hey is it possible to read and store data without a higher protocoll like OBD2 or UDS. So in generell i mean raw CAN Data. Or do i need to have this higher protocoll. Thanks for this livestream it was really nice.
Yes, the CANedge lets you log any raw CAN bus or LIN bus data - it does not have to be from some specific higher layer protocol. In other words, if you were to simulate some arbitrary frames onto a CAN bus using e.g. a USB-to-CAN streaming tool or simulator, the CANedge would record those frames - without any change to the configuration required. In general, the device itself operates at the 'lower layer' CAN protocols and does not perform any form of processing or similar that assumes a specific protocol. The protocol specific aspects mainly come into play when you DBC decode the data via the various software tools - where your DBC might be specific to a J1939-protocol type application or e.g. OBD2, UDS, CANopen, ISOBUS, NMEA etc.
![CAN Bus Explained - A Simple Intro [v2.0 | 2021] 🌟](http://i.ytimg.com/vi/oYps7vT708E/mqdefault.jpg)
![CAN Bus Data Logging [Webinar | 2022]](http://i.ytimg.com/vi/AJQMSX6jVDA/mqdefault.jpg)
![BRASIL vs. PERÚ [4-0] | RESUMEN | ELIMINATORIAS SUDAMERICANAS | FECHA 10](http://i.ytimg.com/vi/tbyOJKXfQnI/mqdefault.jpg)
Awesome content - thank you
Would be great if you guys could have your devices as a Hat for Single Board Computers so we could use it to read the data on professional scanning tools. I say that because we can install Windows OS on a SBC and most of the professional scanning tools require Microsoft Windows.
Is translation to Parquet on the CANedge devices themselves on the roadmap? So the edge device does the DBC decoding and then uploading of the Parquet data straight to S3... Thanks for sharing this webinar, I found it very useful.
Thanks for the question! We will most likely not change the way the device records the raw data on the SD card. DBC decoding the data to e.g. decoded MF4 or decoded Parquet may seem convenient and practical at face value - and for some use cases it definitely will be. However, there are many situations where this is not ideal.
1) For example, if you made a mistake in your DBC or used a DBC file that was of an incorrect version, this would result in basically all your data becoming invalid and useless - with no way of rectifying this. See our Case Studies section with the Koenigsegg case study where they describe exactly this factor as being a key reason why they select the CANedge.
2) For some use cases, you may want to store the "full data" in the raw compact MF4 format, while you then e.g. only DBC decode a small subset (maybe 10-20 signals, which could be e.g. just 10% of your dataset) to a Parquet data lake to avoid unnecessary data duplication. In such a situation, it is ideal to be able to distinguish between the 'original' data and more 'prepared' sub data sets.
3) Another important factor is the ability to 'compact' the data ex post. Some users may upload a new MF4 log file every 1 minute, for example. This results in a huge number of files in S3 and over time, the Parquet data lake also becomes inefficient to query if you're looking across long periods of many small files. Here, the MF4 decoders let you easily "re process" the entire raw data to create a compacted data lake, where the number of files can be reduced drastically through concatenation.
4) Further, many use cases deliberately require the ability to view the raw CAN/LIN frames for more direct analysis of the original communication. This is also relevant if you're e.g. looking to analyze CAN/LIN error frames.
Hey is it possible to read and store data without a higher protocoll like OBD2 or UDS. So in generell i mean raw CAN Data. Or do i need to have this higher protocoll. Thanks for this livestream it was really nice.
Yes, the CANedge lets you log any raw CAN bus or LIN bus data - it does not have to be from some specific higher layer protocol. In other words, if you were to simulate some arbitrary frames onto a CAN bus using e.g. a USB-to-CAN streaming tool or simulator, the CANedge would record those frames - without any change to the configuration required. In general, the device itself operates at the 'lower layer' CAN protocols and does not perform any form of processing or similar that assumes a specific protocol. The protocol specific aspects mainly come into play when you DBC decode the data via the various software tools - where your DBC might be specific to a J1939-protocol type application or e.g. OBD2, UDS, CANopen, ISOBUS, NMEA etc.
Mighty Interesting. Thank you very much!