Thank you Tim Corey. Your videos here have been a real guide for me in learning C# and now I've gotten promoted at work because of how fast I've learned. Thanks a lot
This is going to be so useful for a future project of mine. So much quicker and I can already visualise the application! Thank you for showing this off!
Thank you for sharing, always love watching your tutorials! Btw, thanks to your paid courses (and here on yt also) I was able to build up enough experience and resumé that I am now hired full time as a full-stack developer 🎉
Another great video Tim! I just used CsvHelper for a small project where I needed to generate a csv with checks printed from our accounting software for positive pay system with the bank. It was super easy to do. I had to eliminate the header row and format datetime into date only, but it had all the options to do so in a clean way. I used your video as a general guide to make it happen.
I don't know how Tim's channel didn't reach millions of subs already. His content is top notch. I have used csvhelper but I did make that mistake of converting the records to a List. Thank you for showing me a better and efficient way to use this amazing library!
I am on Windows 11 , getting error after extracting zipfile. And trying to open csv file 0x80004005.. however any csv on my machine opens up no problem, I will try to use one of those, no big deal , just curious if anyone else had that
Never used it before but I have a feeling in my gut that in a matter of couple of months it will come in handy for one of my projects. Very fast and easy to use library. Thanks for sharing it. What interested me the most is this IEnumerable and ToList(). Does it also apply for getting the data from the database? Say for example I load 2000 records from the database for whatever reason say it would be a historical changes of some sort so it would be much more efficient to leave it as IEnumerable and create a virtualized table out of it (in blazor) rather then put it to the list and then create a virtualized table? Have you got any video that shows those differences? When to use what and so on? Also thanks for sharing github copilot chat. I think it will come in very handy.
The reason why I searched for this is because some of the fields that I am working with may be enclosed in quotes, and the content between the quotes can contain commas. I don't want these commas to be interpreted as field separators. Also, I didn't want the quotes to be included in parsed strings. I assume CSVHelper can handle these sorts of things. I'll give it a try. Thanks for the video.
I’m currently using a different library but this looks better. I’ll be looking to add a feature where I don’t know the columns or the order. Hopefully this library supports this. I’ll need to create the mapping at runtime from user input.
How can I deal with column name changes? If over time column names change and maybe their order. Or some are removed. Then I need some backwards compatibility between old reports in my app and new reports with the ever changing new format. When I download different formats that must map to the same class. 1. If the column header is dynamic in name then I can use regex. 2. If some columns are removed in the future I can use nullable properties.
Not using this tool. You would need to tap into the file system events to monitor for file changes, but my guess is that those events won't be as granular as you like. They will tell you the file changed, but not what changed. You will need to read the entire file to figure out what changed by comparing it to what you are storing in memory (or another file) to see what the changes are. If you want change tracking, you might want to use a database.
Thanks again for the fantastic tutorial. The program throws a System.IO.IOException if someone or a different service opens the file while it's in the for loop. It is possible to have concurrent read access from multiple systems?
I don't believe that you can. I believe that's a limitation of Windows mostly, but I am not positive if there is anything you can do to change how you lock the file.
Thank you .But ,Do we need to have any security concerns while using this kind of private NuGet packages as we don’t know what logic they have within their NuGet?
You should always evaluate your dependencies, but I think the open source movement has actually confused people when it comes to their security. If you take on an open source (or source available) dependency, do you do a thorough code review of it? Or do you just assume because it is open that it is safe? Closed source doesn't mean unsafe. You can still review its outputs and actions to validate that it is doing what you think it should. For instance, the .NET Framework is not open source (.NET is open source). Visual Studio still isn't open source (and has no plans to be). The key isn't so much the availability of the source but in how much effort you put into evaluating the solution and if is something you are comfortable depending on. Long story short, the security concerns don't change based upon the availability of the source.
Thank you Tim for an awesome walkthrough and guide. I've noticed on a few of your videos that when you type ie Console.WriteLine($"") at 14:40 you get a small grey box with a type (value)(text) inside it. What does that?
Thank you very much for this wonderful video. I have a wpf application with a textbox and a listbox. The source data is a CSV file with approx 1million rows. The goal is to filter the CSV for the data each time a user enters data in the textbox. Is there a performance cost to open, read and filter a file stream each time the user enters a text vs just having it in RAM? Is there a better way to solve this problem?
There is a major performance issue here. You would need to put all 1,000,000 records into memory in order to filter them. In theory, you could discard the non-matching records so you don't have to keep them in memory, but that's only true if you don't allow the user to backspace in the search field. If they did, you would need to re-process all 1,000,000 rows again. The better option here is to load that data into a database with indexes. It will take a while, but after that, you can easily query the data. That's why databases are so powerful compared to just text files.
So, I noticed briefly in this video you showed CoPilot producing Attributes. Do you have a video that explains them and how to use them manually? I have looked everywhere to understand how this work with no success.
Thanks for the reply! You have taught me so much you have NO idea. I have made an entire application that I sell now because of you. I can not thank you enough! Yes, attributes. Maybe one day you can make a 10-min short on them as I can not find anything on them. I know there is also a way to use them to import C++ libraries and I also have no clue how to do that.@@IAmTimCorey
I have a question. I would think a person is always supposed to use .ToList. I have found many cases where you had many memory leaks unless you used ToList. I would think libraries should always return a list and never ienumerable so if a person uses it in many ways, then it does not have to reprocess everything again.
Yield cannot work with List. To make it return List would be to destroy its efficiency. ToList would put the entire CSV in memory. IEnumerable and Yield allows the helper to only put in memory the row being returned.
Is there a PDFHelper package like this? I've used the CsvHelper package before and found it VERY helpful! I wish there was a package that was just as easy for PDF.
The pdf format is very complicated so they all have fairly major drawbacks: price, license, complexity, compatibility, support, quality of output. My opinion is html is a better format for most use cases, although usually people want pdf anyway because it feels familiar.
@Mike-qu6fp it definitely "feels" more familiar given that I've never worked with HTML and the company I work for uses PDF's like they're candy. If I am to print or visualize any files for the users, I'd need to use PDF. Maybe XPS, but that's its own can of worms 😅 I definitely get the issues with PDF, though, for sure.
@@IAmTimCoreyIt’s really weird if i remove the last line of code Console.Readline(); it will save the records to file, if I include this line it won’t 🤷♂️
@@IAmTimCorey Using (var csvOut = new CsvWriter(writer, CultureInfo.InvariantCulture)) { csvOut.WriteRecords(outputRecords); } Console.Readline(); Will write the records to file, but it’s not the same as your code so what gives?
You need to hit the Enter key and let the application complete before the file will be closed out properly with my code. With the above code, your using closes out before the application pauses to listen for user input.
CSV helper is missing tools for actually working on CSV data. It's an excellent mapper, but I miss having the data in a "datatable"-like structure as well, so I'm making it myself 😂
I don't think it is "missing" it. They purposefully chose not to include one because a CSV is not a database and cannot be treated like one. For instance, if you wanted to sort the data, you would need to load the whole CSV into memory. If you wanted to filter the data, you would need to load the whole CSV into memory. If you wanted to update a row, you would need to load the whole CSV into memory. That seems fine if you are loading 10,000 rows or less but it get messy when you try to load 1,000,000 rows. If you want database-like structure and capabilities, load the CSV into SQLite or another actual database. Then you can do whatever you want with the data without keeping the entire file in memory for the life of your application.
That wouldn't apply here. CSV injection is only relevant when you are opening a CSV file in Excel or other systems where it processes the fields. In C#, you would just see the formulas, etc. as text. You could load, update, and save a CSV file full of CSV injection junk and it wouldn't affect your system in the slightest.
Csv files are allways problematic. The main reason is that Excel outputs Csv, in non-english versions, as ;-separated ot tab-separated. Also date formats are usualy expected in mm/dd/yyy format, while Excel outputs dd-mm-yyyy. And similair for numerics, the decimal and thousands separators are usually swapped.
I find that Excel is more useful to view CSV files rather than creating them in the first place. Usually if you are creating something in Excel, it is an XLSX file.
Talking efficiency/fast while having a console writeline in a loop kind of defeats what you are trying to show. Does this CsvHelper handle embedded CRLF and all the other oddities which exist in the real world? Can it also support other delimiters, not just a comma? Guess I could check the doc. More importantly how do you or anyone trust nuget packages? Seems like a trojanHorseFactory.
1. That’s how demos work. 2. Yes, it can handle the oddities of CSV files. 3. Trust NuGet packages? That’s how C# works. Even Microsoft uses them for their core products. For open source products (like this one), you can inspect the source yourself. For closed source, you can evaluate the company that supports them. It is no different than trusting any other software or app.
Just a little feedback, but in my opinion it would be better if GitHub Copilot and other AI helper would be deactivated for these videos. It's a great tool, but I watch your videos to learn the details and understand them fundamentally - and you still do that, but when the AI is spitting out entire chunks of code I just think it hurts these kind of videos more than it helps. Just IMHO.
You can look at the code to see how it is doing it, but it could absolutely use reflection and still be incredibly performant. Don’t get so locked into a technology that you fail to consider the alternatives.
In this tutorial, I created a CSV file using the library. I also read a CSV file into memory. I'm not sure what else you were looking for from an intro tutorial.
Thank you Tim Corey. Your videos here have been a real guide for me in learning C# and now I've gotten promoted at work because of how fast I've learned. Thanks a lot
Awesome! Thanks for sharing and congratulations!
These tutorials on useful libraries are great, please do more of them.
Glad you like them!
Another banger Tim. Been using CSVHelper for years and it's excellent
Thank you!
Well... I wish I knew this some time before. That tool looks really good to work with.
Better late than never, I guess.
This is going to be so useful for a future project of mine. So much quicker and I can already visualise the application! Thank you for showing this off!
I am glad it was helpful.
the best man in USA . Thanks a lot.
You are welcome!
Thank you for sharing, always love watching your tutorials!
Btw, thanks to your paid courses (and here on yt also) I was able to build up enough experience and resumé that I am now hired full time as a full-stack developer 🎉
Fantastic! Thanks for sharing!
Another great video Tim! I just used CsvHelper for a small project where I needed to generate a csv with checks printed from our accounting software for positive pay system with the bank. It was super easy to do. I had to eliminate the header row and format datetime into date only, but it had all the options to do so in a clean way. I used your video as a general guide to make it happen.
I am glad it was helpful.
Awesome demo. I have an integration coming up, which is using CSV data... so glad I caught this video first!!!
I am glad it was helpful.
Master blessings, your content is the best, you have helped me a lot not to deviate from the path.
Glad to help.
I don't know how Tim's channel didn't reach millions of subs already. His content is top notch.
I have used csvhelper but I did make that mistake of converting the records to a List. Thank you for showing me a better and efficient way to use this amazing library!
You are welcome.
If you want the data for other usage, it is fine to use tolist.
Thanks for sharing such great concept about CSV file Library ,
Request you to please make series of all C# Library
I'm glad you enjoyed it.
Heavily used this library. Thank you for making a video to use this library.
You are welcome.
I am on Windows 11 , getting error after extracting zipfile. And trying to open csv file 0x80004005.. however any csv on my machine opens up no problem, I will try to use one of those, no big deal , just curious if anyone else had that
A great video as usual! Thanks Tim and your team.
You are welcome.
as always great video and really helpful, thanks for share your knowledge.
Glad it was helpful!
Thank you Mr Tim, for the great content
You are welcome.
Much appreciated. Like seeing how you use Copilot
Thanks!
Thank you very much, may the grace of God come with you!!!
Thank you!
Thankyou for your nice explanation.
You are welcome.
You are welcome.
Awesome! Use case we have is, person drops csv, load csv into sql server table incrementally.
Excellent!
muuito bom! obrigado professor!
You are welcome.
FileHelpers is another good library for processing CSV files, and unlike CsvHelper it's not under a copyleft license.
Thanks for sharing!
Never used it before but I have a feeling in my gut that in a matter of couple of months it will come in handy for one of my projects. Very fast and easy to use library. Thanks for sharing it. What interested me the most is this IEnumerable and ToList().
Does it also apply for getting the data from the database? Say for example I load 2000 records from the database for whatever reason say it would be a historical changes of some sort so it would be much more efficient to leave it as IEnumerable and create a virtualized table out of it (in blazor) rather then put it to the list and then create a virtualized table?
Have you got any video that shows those differences? When to use what and so on?
Also thanks for sharing github copilot chat. I think it will come in very handy.
finally.
Hope you get value out of it.
The reason why I searched for this is because some of the fields that I am working with may be enclosed in quotes, and the content between the quotes can contain commas. I don't want these commas to be interpreted as field separators. Also, I didn't want the quotes to be included in parsed strings. I assume CSVHelper can handle these sorts of things. I'll give it a try. Thanks for the video.
Yep, it can. CSV files can get quite complex, but this library handles it.
Thank you!
You are welcome.
I’m currently using a different library but this looks better. I’ll be looking to add a feature where I don’t know the columns or the order. Hopefully this library supports this. I’ll need to create the mapping at runtime from user input.
I am glad it was helpful.
I have not used CSV Helper yet but I have a project where it will come in handy. Thank you!
You are welcome.
How can I deal with column name changes? If over time column names change and maybe their order. Or some are removed. Then I need some backwards compatibility between old reports in my app and new reports with the ever changing new format. When I download different formats that must map to the same class. 1. If the column header is dynamic in name then I can use regex. 2. If some columns are removed in the future I can use nullable properties.
That's something you would need to write the logic for. It isn't something that the tool can do for you.
Thanks sir Tim. Is there some manner to update and prevent real time about it?
I am not sure what you mean by "update and prevent real time". Can you please explain?
@@IAmTimCorey i mean if is possible notify the changes on the csv somehow
Not using this tool. You would need to tap into the file system events to monitor for file changes, but my guess is that those events won't be as granular as you like. They will tell you the file changed, but not what changed. You will need to read the entire file to figure out what changed by comparing it to what you are storing in memory (or another file) to see what the changes are. If you want change tracking, you might want to use a database.
Great Video! Your videos are so perfect! Is there a way I can download the cvs file? I host my apps on Azure. Thanks for your help!
You would need to store the CSV file in a location where you can get it. Probably Azure Storage, since it is cheap and easy to use.
Thanks again for the fantastic tutorial. The program throws a System.IO.IOException if someone or a different service opens the file while it's in the for loop. It is possible to have concurrent read access from multiple systems?
I don't believe that you can. I believe that's a limitation of Windows mostly, but I am not positive if there is anything you can do to change how you lock the file.
Nice. I'm sure that I have mapped at least 20 times manually. Guess it does not work directly on excel files aswell?
Nope, but there are libraries for working with Excel files, though: ruclips.net/video/j3S3aI8nMeE/видео.html
@@IAmTimCorey okay would be nice if it just had the option to convert excel to csv
Thank you .But ,Do we need to have any security concerns while using this kind of private NuGet packages as we don’t know what logic they have within their NuGet?
You should always evaluate your dependencies, but I think the open source movement has actually confused people when it comes to their security. If you take on an open source (or source available) dependency, do you do a thorough code review of it? Or do you just assume because it is open that it is safe? Closed source doesn't mean unsafe. You can still review its outputs and actions to validate that it is doing what you think it should. For instance, the .NET Framework is not open source (.NET is open source). Visual Studio still isn't open source (and has no plans to be). The key isn't so much the availability of the source but in how much effort you put into evaluating the solution and if is something you are comfortable depending on.
Long story short, the security concerns don't change based upon the availability of the source.
Thank You!
You are welcome.
Thank you Tim for an awesome walkthrough and guide.
I've noticed on a few of your videos that when you type ie Console.WriteLine($"") at 14:40 you get a small grey box with a type (value)(text) inside it. What does that?
Here is a video showing you how to turn those on: ruclips.net/video/morBKqtqmso/видео.htmlsi=CLUYrvvEaqPpZE_v
Thank you very much for this wonderful video. I have a wpf application with a textbox and a listbox. The source data is a CSV file with approx 1million rows. The goal is to filter the CSV for the data each time a user enters data in the textbox. Is there a performance cost to open, read and filter a file stream each time the user enters a text vs just having it in RAM? Is there a better way to solve this problem?
There is a major performance issue here. You would need to put all 1,000,000 records into memory in order to filter them. In theory, you could discard the non-matching records so you don't have to keep them in memory, but that's only true if you don't allow the user to backspace in the search field. If they did, you would need to re-process all 1,000,000 rows again. The better option here is to load that data into a database with indexes. It will take a while, but after that, you can easily query the data. That's why databases are so powerful compared to just text files.
So, I noticed briefly in this video you showed CoPilot producing Attributes. Do you have a video that explains them and how to use them manually? I have looked everywhere to understand how this work with no success.
How to create attributes? No, I don't have a video on that. Copilot was just suggesting I use existing attributes, though.
Thanks for the reply! You have taught me so much you have NO idea. I have made an entire application that I sell now because of you. I can not thank you enough! Yes, attributes. Maybe one day you can make a 10-min short on them as I can not find anything on them. I know there is also a way to use them to import C++ libraries and I also have no clue how to do that.@@IAmTimCorey
Thanks for the suggestion. Please add it to the list on the suggestion site so others can vote on it as well: suggestions.iamtimcorey.com/
I have a question. I would think a person is always supposed to use .ToList. I have found many cases where you had many memory leaks unless you used ToList. I would think libraries should always return a list and never ienumerable so if a person uses it in many ways, then it does not have to reprocess everything again.
Yield cannot work with List. To make it return List would be to destroy its efficiency. ToList would put the entire CSV in memory. IEnumerable and Yield allows the helper to only put in memory the row being returned.
Is there a PDFHelper package like this? I've used the CsvHelper package before and found it VERY helpful! I wish there was a package that was just as easy for PDF.
The pdf format is very complicated so they all have fairly major drawbacks: price, license, complexity, compatibility, support, quality of output. My opinion is html is a better format for most use cases, although usually people want pdf anyway because it feels familiar.
@Mike-qu6fp it definitely "feels" more familiar given that I've never worked with HTML and the company I work for uses PDF's like they're candy. If I am to print or visualize any files for the users, I'd need to use PDF. Maybe XPS, but that's its own can of worms 😅
I definitely get the issues with PDF, though, for sure.
Hi i’ve encountered a problem the code creates the file but no records are in it 🤷♂️😤
What have i done wrong…HELP
Not sure, but it sounds like a good opportunity to debug.
@@IAmTimCoreyIt’s really weird if i remove the last line of code Console.Readline(); it will save the records to file, if I include this line it won’t 🤷♂️
@@IAmTimCorey
Using (var csvOut = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csvOut.WriteRecords(outputRecords);
}
Console.Readline();
Will write the records to file, but it’s not the same as your code so what gives?
You need to hit the Enter key and let the application complete before the file will be closed out properly with my code. With the above code, your using closes out before the application pauses to listen for user input.
CSV helper is missing tools for actually working on CSV data. It's an excellent mapper, but I miss having the data in a "datatable"-like structure as well, so I'm making it myself 😂
I don't think it is "missing" it. They purposefully chose not to include one because a CSV is not a database and cannot be treated like one. For instance, if you wanted to sort the data, you would need to load the whole CSV into memory. If you wanted to filter the data, you would need to load the whole CSV into memory. If you wanted to update a row, you would need to load the whole CSV into memory. That seems fine if you are loading 10,000 rows or less but it get messy when you try to load 1,000,000 rows. If you want database-like structure and capabilities, load the CSV into SQLite or another actual database. Then you can do whatever you want with the data without keeping the entire file in memory for the life of your application.
Is there a tsv equivalence to this library
You can use tab-separated format instead of comma-separated using this tool.
does it have protections against CSV Injection ?
That wouldn't apply here. CSV injection is only relevant when you are opening a CSV file in Excel or other systems where it processes the fields. In C#, you would just see the formulas, etc. as text. You could load, update, and save a CSV file full of CSV injection junk and it wouldn't affect your system in the slightest.
Csv files are allways problematic. The main reason is that Excel outputs Csv, in non-english versions, as ;-separated ot tab-separated. Also date formats are usualy expected in mm/dd/yyy format, while Excel outputs dd-mm-yyyy.
And similair for numerics, the decimal and thousands separators are usually swapped.
I find that Excel is more useful to view CSV files rather than creating them in the first place. Usually if you are creating something in Excel, it is an XLSX file.
Talking efficiency/fast while having a console writeline in a loop kind of defeats what you are trying to show. Does this CsvHelper handle embedded CRLF and all the other oddities which exist in the real world? Can it also support other delimiters, not just a comma? Guess I could check the doc. More importantly how do you or anyone trust nuget packages? Seems like a trojanHorseFactory.
1. That’s how demos work.
2. Yes, it can handle the oddities of CSV files.
3. Trust NuGet packages? That’s how C# works. Even Microsoft uses them for their core products. For open source products (like this one), you can inspect the source yourself. For closed source, you can evaluate the company that supports them. It is no different than trusting any other software or app.
Just a little feedback, but in my opinion it would be better if GitHub Copilot and other AI helper would be deactivated for these videos. It's a great tool, but I watch your videos to learn the details and understand them fundamentally - and you still do that, but when the AI is spitting out entire chunks of code I just think it hurts these kind of videos more than it helps. Just IMHO.
Thanks for sharing!
Fantastic library. DO NOT ROLL YOUR OWN CSV PROCESSING CODE. You are wasting your time.
Thanks for sharing!
Yeah, CSV is such a simple format that you might not even think of using a library at first, but that would be a mistake.
If this does not use source generators to find the classes, then it would be too slow because reflection is always too slow.
You can look at the code to see how it is doing it, but it could absolutely use reflection and still be incredibly performant. Don’t get so locked into a technology that you fail to consider the alternatives.
Tutorial did not rly show how the library really works. I hat this superficial, time wasting tutorials based on the documentation
In this tutorial, I created a CSV file using the library. I also read a CSV file into memory. I'm not sure what else you were looking for from an intro tutorial.