Testing Entity Framework Core Correctly in .NET
HTML-код
- Опубликовано: 3 фев 2025
- Use code TRANSIT20 and get 20% off the brand new "From Zero to Hero: Messaging in .NET with MassTransit" course on Dometrain: dometrain.com/...
Get the source code: mailchi.mp/dom...
Become a Patreon and get special perks: / nickchapsas
Hello, everybody. I'm Nick, and in this video, I will show you the biggest mistake .NET developers make when it comes to testing their database, especially when they are using Entity Framework Core, and that's replacing their actual data provider with the in-memory one. That approach leads to massive problems, and in this video, I will show you how you can deal with it.
Workshops: bit.ly/nickwor...
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: github.com/Elf...
Follow me on Twitter: / nickchapsas
Connect on LinkedIn: / nick-chapsas
Keep coding merch: keepcoding.shop
#csharp #dotnet
We used to use the SQLite in-memory DB for our tests. Once we started switching to using TestContainers (since our actual DB was Postgres), we learned that a lot of our tests were actually invalid because SQLiite (and the EF in-memory DB) don't enforce a lot of things, like string length limits and foreign keys.
The in-memory provider will not behave like your real database in many important ways. Some features cannot be tested with it at all (e.g. transactions, raw SQL..), while other features may behave differently than your production database (e.g. case-sensitivity in queries). While in-memory can work for simple, constrained query scenarios, it is highly limited and we discourage its use.
@@umutkayatuz9963 Yep... we learned that the hard way. But thanks to Nick and TestContainers we are going down the right path now!
That password joke was so smooth. I can't stop laughing
Been doing this for the last few years! Did a lot of discussion on it with EF maintainers and some other thinkers in the space. It works very well.
How do you handle large systems with setup and inserting setup data with 100s of tests? Do you use a transactional context that rolls back after a test or regular transactions? How do you do parallel testing if you are sharing the same instance?
In our testing, the answer is 'it depends'. It might make sense to spin up separate containers for tests running in parallel... or not. You may need to split out some tests from each other so you can run separate tests in separate containers in parallel. As for the test data creation, I suggest a simple script to run in the before of the test.
Containers start very fast, you can run tests in parallel and have a new container for every test
One strategy is to avoid tests interfering with eachother is to partition the data based on some scope like using different user, or tenant id's for different tests - if you can. Otherwise I use xunit collections to run tests in a sequence rather than concurrent. Final option is multiple instances of the DB but that is obviously a lot more resources intensive.
@@Crozz22 They are not so fast. We are talking seconds instead of miliseconds. When you have +1000 tests, running them all can take minutes and be very resource intensive (having several apps + databases running in parallel). The sqlite in-memory is much faster.
I love using the real db, but it's just not feasible to test everything at that level, you need to test edge cases more unitarily in large projects.
@@Crozz22 Doesn't matter if Containers start fast or not. The programs on the containers are important and not the containers themselves. A container for Postgres can be ready to be used immediately, however a container for SQL Server will take up to 10 seconds to be used and that's why the programs are important, not the containers themselves.
I always use a real database for my integration tests, but often I'll write higher-level behavioral unit tests for which I will use an in-memory database (being mindful of the limitations) to keep them quick and minimizing the need to mock.
I consider the database to be part of the application and part of the "unit" so I include it in my unit tests.
Ok, i have a question, if I set up containers for my tests but let's say that my integration tests require something to be configured in database like a stored procedure or something like that, how should I do it? EF core uses migrations to keep up the database state, is it necessary to run all migrations before the test in that start method or what would be the best solution here?
I understand the premise and agree that it's not testing the actual conversion of SQL, but you know what? I don't think it's that bad.
Surely it has a few drawbacks, but so does every approach. Some of the benefits I like are:
- It's portable, and runs straight from the IDE or pipeline without needing to set up a container runtime.
- The tests are usually quicker, as they can be easily parallelizable (Especially with SQLite provider), while actual DBs need to run mostly sequentially to avoid having the data of one test impacting the other.
- Easy to run + Quicker feedback loop = Better developer experience.
The TestContainer approach also has some drawbacks:
- Some teams don't have EF migrations, but the DB is versioned in SQL Scripts that run a separate pipeline, sometimes in a different repo. Creating a TestContainer with the correct schema may be more difficult.
- Now that Docker Desktop require licenses, some companies are using containerd under WSL2, which (AFAIK) would not run natively unless it's configured to have the socket exposed in a local port.
- Some CI/CD setups don't support containers.
You also mention that unless it's using the actual database, we don't have good integration tests. I'm not too sure I agree with this point, for a couple of reasons.
One of them is the difference between Unit and Integration tests. I'm aware of some heat in the community over the past years regarding the Chicago vs London school of thought (a.k.a., "Sociable Unit Tests") when it comes to the definition of unit test, but it's arguably that if Unit tests are supposed to test just one unit, then if you have more than one concrete (not mocked) service class instance being test you have an integration test.
The other reason is... does it really matter? Saying that you don't have good integration tests unless you use the same DB is like saying you don't have good integration tests if you don't use WebApplicationFactory and call your API in your tests because that's what your clients use. I'm not too keen on this mindset, as the core of our solution should be the domain, the business logic. That's the critical part, that's what needs a comprehensive test. Entrypoint layer (being it HTTP, gRPC, Service Bus events) and persistency (being it SQL, NoSQL or simple text files) are implementation details, coupling most of the test with those seems a bit odd. And making a test that is supposed to verify some rules under a chain of responsibility go all the way from API to the Persistency layer when just the raw classes would suffice seems like unnecessary overhead and hurts developer experience.
At the end of the day, there's no right or wrong. Each team uses what suits them best. I just don't think actual DBs on TestContainers are better than an in-memory provider, it just has different pros and cons.
Our production is Azure Sql, devs and tests all use localdb. Just a difference in connection string and all works great.
I also suggest specifying to test container builder a concrete image tag of the db that corresponds to your prod db version.
I usually use Sqlite provider with the "memory" connection string. One advantage of the in memory provider though: the error messages are actually better than the TRASH Sqlite provider's error messages. One that comes up most often for me is foreign key violations.
Testing with the actual provider you use in production has obvious advantages, but it's still relatively slow.
I write my Integration tests from scratch, without using any extra library. Excellent results so far.
link to source code not working.
IMO: In-memory database is for unit tests! Testcontainers are for integration tests. Question - are you able to run these tests in parallel? Is it one database per test, if so what is the overhead of doing that for 100-500 tests?
You can use a CollectionFixture to "group" tests together and have them all use a single db. You can then launch those groups in parallel
Testing out-of process dependencies with unit tests🤦♂️
@@darioferrai8691recipe for flaky tests
@lepingouindefeusometimes agree
@@dy0mber847no, specifically not in those cases
That's great. In memory db doesn't really convert the linq to real sql and therefore may hide potential issues when writing complex linq which may call on other methods in dotnet (which may or may not be valid for linq conversions). I will definitely start adopting this in my workplace.
It would be great if there was any option to do this without docker at all. Although, most of ci's supports it, the key word is 'most'. Moreover, I don't see a reason to use docker for testing purposes only if your app is not contenerized itself and you haven't considered it as a needed feature of the pipeline initially
TestContainers = mindblown, thank you for this!
I feel like a SQLIte database in memory mode (“DataSource=:memory:”) is what most devs think an in-memory database is.
But the in-memory provider for EF Core just holds objects in memory and acts upon those objects in memory when you call the various DbSet methods. I don’t think it even generates any SQL.
But that SQLite approach might work, if containers aren’t an option.
Hey Nick, I have used this TestContainers but when having many tests more than 100 lets say, its creating a lot of containers in docker, its taking 100% of cpu and memory running all of them and some of them will fail too. Do you know how can we optimize?
TestContainers can be used only when Docker is installed on the Build Server. Otherwise you have to ignore these tests on the Build Server and run them locally/manually which is not useful.
Also, if your Build Server is a Windows Server, then you can forget the whole thing, because there is no Docker for Windows Server and if you manage to install it with tricks on a Windows Server you can then only have Windows Containers and not Linux Containers.
tryied to replace my in-memory database with container but ended up with `Cannot resolve scoped service from root provider` even when i almost copied your code (important parts) ... :(
The only way to mock is to never mock
Feels like Deja Vu... Is this a re-upload?
Have you taken my integration testing course?
Nick has covered Test Containers in the channel before. There is a good video about using Bogard's Respawn
i think about this right after i saw sqlite in memory stuff
Same here 😅 liked twice onto the upload timestamp 😬
Definitely a re-upload
Okay but for me the bigger problem when testing EF Core is testing code that uses views. We use database-first + scaffold, but the test database has no information about the views. We have to mock the view db sets with lists, forwarding the IQueryable and IEnumerable. But you can't test when using views in a more complex query
Wow! I thought the database-first users were extinct. Thank God I'm not alone 😊
The sad reality is: the tools are not made for us anymore. If you're not developing microsservices + .NET 8 + EF + Code First, you're doomed. You're being forced into an architecture by the tools, and not the other way around, which is crazy if you think about it for a sec.
Just set up the views using the appropriate SQL commands when starting the container.
Populating the tables of the test database it is something you have to do no matter if you are using code first or database first. You can do it via executing an SQL script or by instantiating your DB context, adding the data to the corresponding DBSet and executing SaveChanges.
I do not think there is any place for any kind of replacement, mocking or anything for the integrations test. If one is testing the interaction between 2 components, how replacing one of them would make any sense? You are running MS SQL in prod, but using MySQL/SQLite in integration tests? How is that an integration test?
Quote from the in-memory db package page:
"This database provider allows Entity Framework Core to be used with an in-memory database. While some users use the in-memory database for testing, this is discouraged"
I used an InMemory db for some "integration" tests at work until I had introduced a temporal table (InMemory db doesn't support temporal tables). So I had to overwrite everything to use test containers. Now I don't use InMemory db even for unit tests
The mail chimp to get the source code doesn't work for me. I press "email me the source code" and nothing happens. At least give me an error, I mean, come on!
I don't understand. If code first is an option, shouldn't there be a reliable library that can do all the constraint checks also in memory or am I missing something? I already have a real database for integration testing for a legacy codebase but its a hell when it comes to speed. It literally takes around 3 to 5 minutes to execute all the integration tests.
How does this compare to SQLite Integration tests? We use SQL Server with TestContainers, but we aren't able to use it in our pipeline without changing it to a linux agent.
Same problem.
What IDE do you used?
That was JetBrains Rider in the video
I'd like to see how to use Podman instead of Docker Desktop.
Is this dependency implicitly creating and executing the migrations?
In which cases can we use the In-Memory Database? What is the correct usage of it?
Love the ease and speed of the InMemory variant though
Nice video but I really feel like it's missing the WHY in memory db tests are a bad idea (like I saw in some comment about constraints and other)
We tried InMemory db for unit tests for our EF6 db. The in memory db did not respect check constraints, default constraints or triggers; I guess this was obvious. We use the localdb which even works on azure devops.
It's exactly how we are handling integration tests in my company (in combination with Respawn) :)
The second solution has only one problem - docker desctop are denied for corporate users. So or buy license or run ubuntu with docker inside WSL2)))
You can use podman
Make simple... guys, make simple.. The customer/employer does not care how complex or well thought out the solution is. All they care about is 'Is it working? Are our customers satisfied?'.
I don’t see any problem with setting up real integration tests. The application should be ready to switch the connection and just works. You will end up also test migrations. Only benefits with this approach
Always test against the real thing! @nick I thought you were leading us down the wrong path, but then you switched it to testcontainers, nice!
Unit Testing is not really worth it in most modern applications where the domains are well separated and more easily testable. Integration tests and functional tests are FAR more cost effective, and can be done in parallel by different people, and are resilient to massive refactors. It makes your code better too, not to have to be worried about stupid things like TDD and unit test coverage.
I think of In-Memory as a fake for unit tests, instead of using mocks for repositories, you use a "real" implementation that behind the scenes calls the EF's In-Memory provider.
Ye, a big antipattern is using mocks (dynamic mocks like Moq, NSubstitute) for setting up repositories in unit tests. Using dynamic mocks couples your tests to the implementation of the SUT. You really feel that pain once you modify/refactor the implementation.
"flaky tests" means they pass/fail intermittently/randomly without changes to the code
Cool technology,
I had tried it before, and will give it another try buuut
I am struggling to understand what the real world usefulness of this is.
It’s not useful for unit tests, don’t need to hit databases for unit tests and far too slow.
Limited to Not useful for integration tests because the database is not likely in a testable state and would be super arduous to get it there.
Sure it is cool to have a database that can be scrapped after the tests, but using snapshots I can do that too, and the database can be in more useful states by using a real db image.
Can you give some more context or cool examples of how to actually make integration tests easily using this?? - nick I know you can- go on! 😊
Why would a db not be in a testable state? We make our own docker image that has a dacpac pre-applied, for example.
This is just a way to write more self-contained integration tests i guess, so as long as you do integration tests all is fine.
Just override connection string Environment.SetEnvironmentVariable(“connection_string", "…"); instead of reassigning dependency
Once you use EF.... you are EF'd
As not native speaker I like your videos that were made 2 years ago. You were talking not so fast and it was easier to undestand. Now I have to set speed to 0.75 (or even to 0.5) and you look drunk.
It’s because I am drunk
Or: get rid of EF entirely
Why exactly? I am pretty happy with EF provided you don't make the in memory provider mistake ofc.