The argument that "testing requires foresight, and my team doesn't have foresight" is wild to me. As tech lead on most of the projects I work on, I write unit tests *because* my team lacks the foresight.
50 dev, across 10 sevices, using 6 different languages, and hundreds of millions of lines of code; in an application that handles millions of dollars a day. Unit tests save my ass at least twice a week from breaking code that I haven't touched and sometimes don't even know exists.
50 devs absolutely are not even close to enough to maintain hundreds of millions loc. And what's even more silly is saying that 10 services is all that comes out of that 🤣
And it's usually tests that I didn't even seem necessary that can bring up the biggest bugs. Like I had 2 getters for 2 same type items, so I copy pasted it and changed the name. Unit tests told me I didn't return the correct item in the second getter, seems I forgot to change the field I returned. It's incredibly minor things like this that make it worth to even unit test mundane stuff, especially if it's public. I don't unit test everything either, I just make tests based on the requirements and some sanity tests and that exposes regressions and ensures features work as they're supposed to.
Theo has the take of someone that has never worked through large code migrations in big companies with compliance requirements. Tick any of those three criterias and you instantly thank unit tests every day of your existence.
Hey. You liked this right? YOU LIKED THIS RIGHT??? Much disappoint in you little algo inducer. BTW I have seen some mean responses about Theo. For me, say what you want, but be nice to the guests please. They are willing to come join us!
I hated Theo’s ideas and arguments. I don’t hate Theo, he seems to be nice. No unit tests? No rebase? I’m a backend engineer, most of the time I need to read code not written by me or anyone in my team, unit tests allow me to know if the behaviour is intended and gives me confidence that I’m not introducing a regression. Rebase helps to have a clean history, why this line that seems to be stupid was introduced (spoiler alert, there’s usually a reason to add that stupid line).
the best part about unit tests is that they force you to rethink your dependency graph if the test is a pain in the ass to write because of some object that you don't have in the test environment or because you need to set up a ton of global mutable data, those are huge hints that your code is turning into import spaghetti
This. I think the arguments Theo is presenting is mostly that it is easy to write bad unit tests. I think it's fair that being a new dev does not make you a bad dev, but new devs are inexperienced. The only way to learn how to write good tests is to spend time writing bad tests.
If that is what you are presenting as the best argument for unit testing then it is fundamentally incorrect way of addressing the purpose of why and who you are building your software to. Business solutions/goals and how the technology addresses those domains are things that are suppose to guide the testing process not the other way around. Identifying the minimal resources you need to make software and the kind of architecture you have to choose within domain constraints are non coding activities that always comes first before you ever dive in first to coding in the first place. The overarching purpose of building software is solving a problem that you value for a target audience(client). Unit tests are just one of many tools at trying address maintainability and consistency of behavior of modules. Unit tests are just a tool, methodology and a means to an end. Like other methodologies there are certain scenarios that unit testing just cannot do what it's suppose to do given the constraint of the software been built. A real life example would be in a banking client where you have to test interaction between your software and some other software and get the data of an obscure protocol of some obscure proprietary banking application that regulators require you to use for some compliance reasons. In this scenario, you cannot unit test this problem as that protocol and data you get are not open to developers in terms of the code but only the protocol and data received is what is needed to do integration tests. After all, why sabotage a banking regulatory contract for the company that do not want to be in the line of fire for regulatory violations. We have to treat software principles as nothing more but guidelines that can bend based on what we want to achieve. Otherwise it's dogmatic to prioritize methodologies and tools that even do not appropiately address the constraints of the domain for a certain audience.
It’s why DI and IoC are great in large applications, can be annoying to setup at the start of the application, boilerplate code, but down the line are a pleasure to work with
Damn that's a great point. I've suffered from making wrong choice of project structures in the beginning. Then I thought, let's write tests, and then gave up cause I was like, how the hell do I bring this dep into my test? And that personal project had a wonky structure
As a FE “engineer” using React I would say 90% of the tests I have seen in my career were already handled within React. Writing ACTUALLY useful unit tests is a skill that is rare at the level of businesses I’ve been working at.
But how do you know if something was handled by React? What if you update to "React 2.0"? It might not be handled anymore, you have to make sure it is.
@@QckSGaming right and I implied that if you update a dependency a whole integer version and the dependency is not backwards compatible with its previous version then you’re going to have to make changes beyond unit tests to even support that update
From Theos comments, I’m guessing he is completely okay with users finding bugs. In my line of work bugs generally come with Loss of life and lawsuits, so I rather have great test coverage.
exactly, ye it really is a question of, what level of testing can we afford to/not afford to NOT do? Instead of UNIT TESTING EEEEVERYTHING. (1000 broken unit tests later.......)
it's not only that, it's a running theme across frontend developers to have absolutely no clue about basic programming concepts yet still make bold statements about them
> DevHour #1 Hope this is a regular thing. Was wanting more after the last one about git merge vs rebase, went to twitch to find the full version but couldn't find it there, glad to see this pop up in my subscriptions.
This video came out at a perfect time, I am writing up some best practices at my company on what we should/shouldn't test and this helped shed some light on what is good for testing and what is pointless to test.
It's interesting that the argument is basically down to a fight between "fun work" for developers and maximizing feature release velocity, and actually learning good automated testing and making sure that the user gets well working stuff reasonably fast. I do like the idea that the first culture fits prototyping and getting things out fast for active user feedback, while the latter is suitable for maintenance or when you simply don't want to force your users to be paying beta testers 😉
I'm sad because I think Theo's self admitted extreme opinion hurts his arguments more than it forces a confrontations with his ideas. Looking at his arguments, ignoring the extreme stance can help people learn how to write better tests. Tools, like compliers/static type checking, are huge in being able to eliminate entire classes of bugs that you would have to test for, and that would make your code more rigid. I think this is why you see things like Typescript and MyPy do so well. If I was to boil the arguments down it is: 1. Unit tests aren't effective at prevention of bugs. 2. Unit tests are a strain on developers time. 3. Unit tests can cause code rigidity. For point 1, I'd say that's only somewhat true. Unit tests will not cover all situations. But they can demonstrate correctness, particularly for important use cases. 2. I just flat out think he's wrong about this. I think empirical studies have shown that testing improves the speed developers get things done in. Furthermore it seems the time "saved" by not unit testing is spent manual testing and in intensive code reviews. The only part I would agree on is that if you are spending a lot of time creating mocks and set up, that is wasting time. But then you might need to rethink your code under tests design. 3. This is true, I've seen a lot of tests that almost backpropogate themselves into dependencies. However this is often a misunderstand of how to write unit tests, not unit testing Great stuff and I'm glad Theo was willing to come on to argue his points.
On point #2, I'd argue that lead devs have to spend a lot of time on code review anyways. I've often been tricked by the fact that my teammates have written tests that pass and look fine without paying much attention to how they were written precisely. Point is : - no tests : we do have to review the code carefully - unit tests : we do have to review tests carefully, and also the code In some contexts we might better have no tests than s------ tests, cause discussing, teaching and bringing the testing culture requires a supplemental effort compared to teaching coding best practices alone.
@@ze2like that quality time should be spent in review regardless if there are tests or not is a fair point. I think that the effort in cultivating a culture and mentoring younger devs about writing tests is worth it and helps in developing cleaner code. Refactoring and adding documentation also take time away from directly developing, but they don't get the same vitriol as testing does.
The pay off for unit tests are when the complexity gets quite big. It's great if you are creating simple widgets. It sounds like a lot of manual testing. I honestly thought we moved past this argument but remember my php days "well it works when I ran it" Interesting to see we are still here
I'm a relatively new dev, but his argument of "I don't trust my developers to find edge cases more than I trust users to run into them" just sounds moronic. You trust the devs to find as many as possible so that the experience isn't completely broken when things are released and then you fix bugs users run into asap as well no? Also when Prime brings up legit use cases for them he's like "someone else will do this job and it will be fine because they're probably more pro unit tests". like what? how is that an argument?
In my experience devs not doing unit tests ends with QA having to find bugs late and then playing bug-ticket ping pong with the devs. It also usually happens way too close to release for comfort.
Can u please make a video about doing the "right" unit test? Like how to do the right mockup function, etc. It will be very helpful for beginners like me!
I feel like writing "tests" first is bad, but writing a "spec" first is good. Thinking about the requirements and the interface before you write the implementation works sometimes, I think.
I just finished the book on clean architecture and implemented it in two apps so far and I have changed my tune in unit tests. Everything I write now has an interface and tests for all public apis. I have been a lot less stressed adding all the tests too, and now I can build with more confidence. These projects also use typescript.
I love that Prime is replying to everyone. I have a job interview for MTA NYC subway as a SWE tomorrow and you have helped me a lot. I just wanted to thank tou
this was great. it was like the friends that you have from different groups meet each other and you are nervous lol I really enjoyed this. I hate writing tests but they are vital IMO.
The actual valuable tests are around enforcing business behavior correctness, not about technical concerns. Context matters a lot too. When the result of your error is some text blowing out of a div or janky scroll behavior, no big deal, roll forward. When your error produces tons of data that has to be reconciled while all the stake holders spaz out, you feel differently. Knowing its 'correct' and being able to confidently iterate on that 'correctness' is suddenly quite valuable.
Theo's position is only applicable for small teams working on CRUD-like applications IMHO. You simply can't have the energy to manually keep up with all the crap going on in a production codebase when dealing with most real life, entangled, non logicless software products.
@@ThePrimeagen 2 - 3 years, depends. First year is "go horse!! rush!! typescript is safe for type!! just make an crud! you don't need to validate this! you can use a library for that". After that any small change breaks everything. Nowadays a sprint(2 weeks) it's about 75% fixing bugs that return(bug -> fix -> bug the same thing again -> fix -> another bug...).
@@delsinrowe7328 Yep... I've recently inherited a code base that probably started out like that. It's gotten to the point where it's so bad our bug fixes break existing code or even worse reveal another layer of bugs which were simply hidden by the previous bug that was higher on the call stack... Testing unfortunately is not really viable in this state, since abstractions are almost completely lacking.
I mean, he doesn't work on a crud-like application, you can just check it for yourself. But I feel the important thing here is context. Delivering rapidly changing user experiences can and will be set back by unit tests, giving almost no value. This is true for almost any non-critical service. In the other hand, for systems that handles a lot of entangled business logic that has changed over decades, you have to have a tool like unit tests to assure you are not screwing up an obscure event handler written in 2014 that impacts revenue And giving it a second thought, unit test may very well have a place in a crud-like applications if, for example, you allow the creation of custom models. There is a lot of mapping hell in those kind of applications
I find unit testing to be like "show your work" math equivalent. At times tedious, but overall worthwhile to catch any simple mistakes that can cause bigger issues down the line.
@@ifstatementifstatement2704 if course you're thinking about them, but we all write bugs. Half the time, my bugs are a dumb typo when in my head, i knew exactly what I needed to do. If I'm writing a unit test, I'm pretty much garunteed to catch those little issues because it's very unlikely that I'll make the same dumb mistake twice.
@@ruyvieira104 yes it happened to me. In 8 years of testing I remember only one case. The test was asserting an expected exception as Exception, but a more specific one was actually expected. The code failed earlier than expected with another exception and we've found that by chance at least one year later.
So all I got from this is that Theo's approach is fundamentally reactive whereas doing unit tests is proactive. This is like not doing Chaos Engineering experiments to prepare for situations where it would be extremely catastrophic at scale if there were an outage. It seems Theo is over-indexing on efficiency/efficacy and doesn't realize that not investing in doing unit tests will inevitably result in some type of failure, especially as his team scales. Just like with anything in life, we should approach things pragmatically and I don't believe doing no unit tests at all is the right approach.
I think it also takes into consideration the possible impact a bug can make in production. If the worst bug can only impact UX or be easily rolled-back to fix then sure, be reactive like you said. If your product has legal consequences from bugs (things like Insurance, Health Care, etc) then being proactive to catch bugs is much more valuable. Just like any dev tool though, it comes down to how its used and what problems you are trying to solve (there is no silver bullet for testing).
@@ThewNell This is the correct take. If anything, when team scaling problems arise, like when the platform has 5x the amount of features, or they have different sub-products, unit test can be adopted. Until no critical business process has been affected, and the end user is happy with the response times, there is no reason allocate resources into adopting unit tests now, instead of working on having reliable response times, just because we think he may have scaling issues.
Yes, but he is actually saying that his approach is for version one of the product, and then we will hire somebody like theprimeagen to do to a more robust version with extensive testing
Had unit tests, helped me find that some dates were improperly converted so I could fix it. Now we get to avoid an angry client. So thanks again unit tests for helping me make a great product.
I loved this conversation. As a solo, senior, experienced (10+ years freelance, started in web around 2003) developer, I rarely get this kind of intellectual stimulation from other senior devs. I loved this. Also, regarding your channel, I LOVE finding another javascript backend realist-someone who hasn’t swallowed the javascript everywhere hype. 💯
yaya! Theo has a pretty based take on these things. He does love himself the jabascwipts, but he does recognize its weaknesses for what they are. big fan of that.
Tests as documentation is great. I like “functional tests” or tests that confirm the user level requirements. It’s hard to make enough functional test cases to cover the combinatorial complexity of every use case so we breakdown the responsibility into parts and create “unit” tests to fill in the gaps of the limited functional tests.
Having also built a video calling platform, I can see what Theo is saying, there is a bunch of stuff that is super hard to unit test, but, I feel like not unit-testing part of the code should be a decision not a default.
in most cases if this happens, it means the code design is one big mess, and luck of unit tests was one of the factors which could have prevented that.
When you mentioned a writing unit tests for a touch condition, I remembered back in the time I developed a multi touch gesture library. I had to write unit tests to develop the library, otherwise I would have to do gestures on the screen but wouldn't be able to control the exact input. By mocking the input, I managed to develop the library heaps faster. I have faced many cases like this, having the assumption of unit test will be a hinder for the developer is not always correct. Especially if you need to go through many steps to reproduce the issue, you will save time by encapsulating the logic and writing unit tests for it. 0 unit test argument is very hard to defend. IMO if the project has any critical process (i.e payment integration), or will have a long lifecycle (>1 years) you need to have a testing suite. You can choose to not to have any unit tests but you would loose the same amount of time if the lifetime of the project is long enough I know I turned this comment into an assay, so sorry for this. But it's good to remember that web development is just one branch of development and users of web are more comfortable using broken things than other areas. There are many other branches of development that you can't ship broken application or if a problem has been discovered and solved, the client won't be happy to see the same problem creeping up because of a new feature.
I almost always develop unit tests to speed up dev. There are just some things that are too complex and i need a way to keep me on track. I totally am right there with you.
Where I work, we're building mostly simpler, ecommerce/business focused websites. Unit tests rarely add value for us. If we have a particularly complex unit, we will unit test. But most of our code interacts with the database in complex fashions. Trying to strip out the database would remove most of the things we want to test. Most of our automated tests are at the service level, using nomenclature of our requirements. When we're integrating with 3rd party APIs, we really ramp up tests though, because that's where the sneaky stuff lies. If you start having problems when calling an API... did your code break, did theirs, or did they change their API? Our tests aim to answer these questions quickly and easily. Also, I can't imagine having to remember all the corner cases of my code. Put bluntly, I'm not that smart and my memory isn't that good. Having automated tests that tell me what I remembered to test is a good thing. I feel like this is much more difficult to convey to juniors though. It's much easier to just say "test everything!" Adding nuance, and value judgements, can make it much more difficult.
Awesome conversation. Very nice to see two points of view on some very important topic in our daily basis activities as developers/engineers. Hope to see that more often here!!
Unit testing is so misunderstood. Kent Beck did NOT mean to test every method of every class, which 99% of devs think. A unit was a feature unit, not a code unit. For me unit tests, functional tests, and even acceptance tests are basically the same thing. Hex/clean architecture really helped me here. I mostly just UNIT test behind my controllers (and only mock my DAOs/backend) If you do it this way, unit tests are not tedious nor hard to maintain. They don't break and major refactorings are easier. It also helps prevent the "build it and throw it away" approach, since refactoring is so much easier and the unit tests can still work (since they are really just testing an API). See Ian Cooper's TDD YT video for more info.
you are totally right. But most companies understood wrong what is unit testing. And forgot about other important safety nets including integration testing, linters, gradual deployments, etc. When your test takes 3 hours to run, are 3 times bigger than the code tested, whey they overload your CI server are after all the tests are run you still have problems in production... Even in a project I was there was a rule about at least 70% of coverage, and guess what? the developers started adding assertions like True == True just to bump the numbers
This was great to listen to. Both you guys are great programmers with nothing to prove. It was a pleasure lads. Loved the Powerrangers story and the insight into working at Netflix.
This guy is a bricklayer who solves bricklayer problems, but he'll never build a skyscraper. That's fine - sometimes you just need a small brick house. But I wouldn't get into an office building that he made.
There was someone way back when that broke up tests in a unique way that I haven't seen done else where but it made a ton of sense. 1 Unit Tests / TDD - These are tests written to aid in the software development process. They test only a single unit. They also describe what the software technically does. 2 Component-Level Tests - The above tests are white-box. These are black box tests. These tests describe what the software is intended to do. While the above tests are going to be appended/updated/removed on a refactor, these should stay almost the exact same on a refactor. 3 Integration Tests - These tests (as much as possible) verify the ability of your code to integrate with external systems, IE databases 4 End 2 End Tests - Used sparingly, these should simulate a complete end to end usage, which can end up relying on mulitple external systems, and exist outside of the program(s) being tested.
If it really took him that long he's not a senior. I appreciate that Prime is being encouraging of a new kid and seeing in him some of his own early immaturity and brashness etc. Theo has no mature takes at any point in this sequence, they're all things I've heard from brash young male devs who are bumbling around. None of them indicate any level of maturity or thought, just hormones.
the last company that I worked at had 0 tests. like literally just like 10 minutes of manual testing before deploying the new version. The amount of times that production went down or was unusable is insane. Funnily enough they still exist. I don't know how, but they do lol
Working in open source, I can't imagine not having some level of test coverage. The number of commits that get pushed by the community (and sometimes by us, even after manual QA) only to fail CI, help to prevent us from tagging release on a good number of bugs.
This. I'll keep my OSS projects at 100% coverage. However, I define 100% coverage as **"all the code we care about is covered"**. I'll slap some "ignore coverage" comments around stuff I don't care about, that way the coverage report being
@@JeffsterProductions Yea. I worked on a team where we weren't allowed to use any "ignore this for testing" type comments except for npm modules. Makes testing absolutely useless, when you have 90% coverage, and none of it is in the two actually complex pieces of logic in the entire UI, and most of it is just Jest snapshots testing if React can render a component...
I would love to hear their entire engineering flow. Having no unit tests is a death sentence for large scale software. It doesn’t matter if you can get to production in 5 minutes with a fix. The entire point is to develop software thoughtfully and prevent as many bugs as possible to deliver the best user experience. Having high quality tests has caught so many potential bugs or regressions very early in the process that not even a quality engineer let alone a user would have known.
Agree 100% The whole "we can fix it really fast" mentality is like refusing to wear PPE because you live really close to the hospital. Just spend a few minutes finding and putting the gloves on and don't get your fingers cut off in the first place.
For sure. Mostly doing RTL and pure unit tests at this point. Having RTL run off a mocked data layer via Swagger that’s maintained upstream. When we take a new version of a service the docs have example data that is consumed in the tests. Makes it very simple to write good tests that are fairly easy to maintain. We may regret this idea one day though. 😃
Theo makes some great points when it comes to unit tests. Now, I don't think you should get rid of them almost entirely like he is suggesting. But I agree that tests often have the tendency to weigh down bad code, and therefore the bad code persists. I also agree that good tools are better than good tests. If you have a million tests, none of that matters if your tools and your stack don't continuously steer you into the pit of success. Write fewer tests, and ruthlessly refine your tools, stack, and architecture.
I see where both y’all coming from. I like Theo’s quote of not necessarily trying to do it right, but wrong well-enough fast-enough. I’ll get bogged down trying to write the perfect thing and take too long at a foundation vs getting it functional and leveling up. I was brought up at school on literal assembly, c and python. My C teacher in particular drilled the hell out of tests so I do still use them. I think a hard part of tests is writing them well. You really need to understand what’s right, what’s wrong, and what’s the rest? I appreciate having both of y’all to learn from.
It is a matter of scaling, whenever you have a product that consists of multiple layers of complexity stacked upon each other which are being worked on by multiple teams, the testing is going to your best friend.
I'm realising how similar debates around testing can be to debates around git workflow. Like, CI/CD doesn't make sense for people doing feature branches because they just can't merge every day, and CI/CD people will think week-long branches are silly because they use feature flags that let them merge whenever they want. If you release every day, PRs make little sense. If you release twice a year, you need to have in-depth discussions on changes. I feel like a lot of debate around testing comes down to something similar. If you have a close relationship with your users, maybe testing isn't important and they see bugs as a fact of life. If you're contracting, they could see issues as broken promises that cost you a renewed contract. So, basically we're back to "it depends"
That was GOOD! Thanks to both of you for sharing this. Now I have to compute the answer to life, universe and sh-- out of this. Pretty sure over-testing inner parts of code brings a lot of inertia at my workplace too. Even though we don't unit test our frontend code. I'd recommend testing business domain services at a component level + having some unit tests on complex internal parts. Because as stated during this stream, the simple fact that unit tests exist, may scare junior devs and prevent them from engaging in heavy refactoring. That being said, if you have E2E and component-level tests above all of this, it's easier to engage in a rewrite/refactor.
Moving from heavy object oriented to more functional design + dependency injection moved me away from not finding value in unit tests (mainly because I was mocking so much, how could I test if I was mocking right lol), to finding value since it's so much easier to manage. Additionally, the further I moved away from the UI, the more unit testing felt valuable and less like a chore.
You are so close. Now go one step further and learn about generative testing. Let the coputer write gazillions of tests for you on the fly and find all the bugs you would never be able to find by writing your own unit tests.
@@Simon-xi8tb this is always my go to with web service APIs that I can either shove the design into swagger or something like protobufs, are there any other tools that I might be missing for other similar interfaces/approaches?
Man when Theo said about rxjs "I dont even want to consider it because it will infect my codebase and ruin everything" I was like hallelujiah! This perfectly sums up every rant I've ever went on about rxjs. It may be a fantastically powerful tool but it is not worth infecting your code with it.
Is there a reason to avoid infecting your code with it? Does it lead to bad code down the line, or is it just considered bad to have a single library touch a large part of your code base in general? I feel like if it doesn't make the code worse, then it just becomes an architectural decision on whether you want this thing in your code. For example, using the "async" keyword in languages like c# will cause half of your codebase to need to use that keyword, but i don't really think that's a bad thing.
I've found it amazing listening to John Romero one of the creators of the game franchise Doom (together with John Carmack) and discovering that they had absolutely no concept of "Unit testing" back in those days, they did every thing by hand, it's crazy how advanced for its time the game were.
I first heard TDD in 2012. I tried to convince my colleagues to use TDD on a new project, but the manager said no, because it would take more time. Also, we would have to learn how to do it properly. I did, however, manage to use TDD to fix a bug on legacy Java code. There is a RUclips video that taught me how to write tests in order to understand what the code did first. Then, I would refactor the code to make the functionality I wanted to fix testable. Finally, I would write tests and asserts for the correct functionality. I run them and see them fail, because the correct functionality is not implemented yet. I implement it, run the tests and repeat until they pass. Done.
If you have to ask your manager permission for doing your job in the best manner, your workplace is doing it wrong. I never asked anyone permission to make a test in my entire worklife; it is simply how we do the job most efficiently. Code the spec for the unit, implement it one bit at a time, refactoring and changing tests as you go along if the interface does not make sense
I've experienced a project where due to time issues no unit tests were written, refactoring caused regression so people didn't do it anymore, the code base got worse by the day. After a while the project got abandoned by our company. So I believe, even unit testing 100%, upon which I do not agree neither, is better than no unit testing at all. Non-testers might be more productive at the start, but the longer the project lives, the more regressions factors in and people get afraid of refactoring and keeping the code base tidy and well designed. Duplicate code will start piling up, and because of the time wasted on both solving bugs again and again due to regression and duplicate code, the productivity of a team that does unit test overtakes the team of non-testers. Also unit testing brings more than just preventing regression, it forces loose coupling and more desired aspects of code. P.s. not te be rude, but your guest makes silly conclusions a lot of the time...
Yeah, the refactoring case is a great reason to consider unit testing. To try to display Theo's argument best, he would suggest to rewrite it instead of unit test it. Its a valid argument, i just would still prefer unit tests :)
@@ThePrimeagen Yeah I should have written, conclusions that seem silly to me, he might have reasoning I did not grasp, but my comment was becoming rather lengthy already. But, rewriting instead of unit testing does not make sense to me. You would unit test and rewrite, but don't break as much doing so because you have the tests? And without the tests, developers get too anxious to touch something that works as intended, especially if altering it resolved in bugs earlier?
Great format Prime, I think you really shine when you go head-to-head with someone else with as much engineering knowledge. The banter is fun and you guys keep it entertaining while being educational at the same time. Waiting for Ep 2...N. A great part was when you proposed a class as a lightweight observable for the statemachine and Theo went into detail about the different state machiines interacting (1 clientside for audio levels etc, 1 server side for permissions about what can enter the chat room). Bits like that are really entertaining because you are tackling a real world problem and discussing solutions. Like a coding interview, but less formal , more fireside-chat style
TDD is not about defining architecture, it's more about defining your API and is 100% about user interaction with your system, because you are designing the API the way you would like your consumers to use it. Also what's most important thing - TDD leads to testable code. It's impossible to write untestable code with TDD. Anyway, TDD - just like every tool - is great, but not everywhere and not for everyone.
I think like all the buzzwords it got painted on everything. I’ve only found value in TDD outside of UI and it’s value was pretty limited in my experience. I always ended up focusing on the testing aspect and less about just molding the architecture. I don’t even worry about testing now until I build something that works.
I've never seen TDD done at a user interaction level (basically integration tests). All of the advocates I've seen for it use it for unit tests. And there's one very simple reason for that. If you write TDD tests for a brand new feature to test the way that a user will call your work, your pull request will likely be over 50 files long, often over 100 files, because you have to plumb every single piece of that feature to get the tests passing. No one can actually do a proper code review on PRs that big.
@@cyropox8235 ATDD does this on user interaction level. You write DSL (the what) and then poke your system from different sides as a user would (you can have the same tests test API and UI and whatever else, by using "drivers" for each channel - the how). Basically acceptance test first, then unit tests until the acceptance test passes and repeat. You can make multiple small PRs by story splitting - this way the reviewer has a bit of context what the PR adds/changes - and not one giant PR. Now... I think ATDD also says that business can write tests which is fucking bullshit, but if you ignore that part, you are left with a flow that helps you test what you need to test. Also, not everything needs to be tested. Also also, sometimes you cannot write tests first, because you have no idea about the system (this is particularly true for UI). Also also also, UI tests are not great.
I'm undecided on this after watching this. What this convinced me of is to conform to the company development policies (for lack of a better word) because the senior or lead software engineers probably have a good reason for why they do the things they do.
I think also don't just jump into a company and preach unit tests or not. Use what the company is doing now and then see if you can improve the process.
Before watching this, I was skeptical of the usefulness of testing. After hearing your points about how you dislike 100% code coverage and TDD but still value good, thought-out tests, persuaded me of their worth.
@@ThePrimeagen show us, teach us to write better tests and how to approach this. I’ve always struggled to find them helpful, especially with front end dev
I have this thing that I think Theo shares where once we figured out how powerful the type system in Typescript was, it become so obvious when bullshit code exist, code that just restricts functionality or constrains scope, those can be replaced with nifty types that themselves reveal novel fun ways of api interaction
I get both sides of the argument here. Experienced engineers know when to write unit tests and when not to. With junior, tests can often slow them down. If I need to familiarize myself with a new codebase and don't have the ability to sit down with the the creator, I'll take one with tests vs no tests any day of the week. Thinking about how I'm going to test something has definitely saved me from introducing unnecessary complexity to code I write, even if I end up writing no tests for it. It also depends on what you're building though. If you're building a library that a lot of people depend on and you can't immediately jump on a call with your users, it helps to have tests. If you're building a new startup and your users are ok with things breaking as long as they can get through to your devs, then that iteration speed provides more value, as your startup is burning through cash unless you're directly providing value to users. All products have churn and you'll hopefully have metrics in place to be able to understand how many users you're gaining vs losing. Thanks Prime & Theo for having us as a fly on the wall.
23:50 "Whenever I rewrite and burn down the whole city, I look at the unit tests as an example of all the things I have to remember." This, to me, is the most impactful line of this whole video and has me leaning more towards preferring unit testing. I don't know if anyone plays modded Bethesda Game Studios games like Elder Scrolls or Fallout, but to me, it seems like unit testing is to Vortex, what no unit testing is to Mod Organizer 2, at least as it relates to sorting your plugins. In MO2, it lets you drag and drop your plugins willy nilly, but in Vortex, it forces you to teach it the reason why you are reordering the plugins so that it can do it automatically in the future.
Man 45 minutes!! I thought it was going to be long but it finished so quick! Great discussion! Very interesting perspectives. Personally I am a major advocate of unit tests, but I agree with the idea of if you have only two extremes: 100% code coverage or NO unit tests, I would probably go with NO unit tests, though this is completely hypothetical.
And for the first quick version - you just do a POC and see if it works. If it does you do it with a proper architecture and tests as otherwise you'll get yourself into technical debt increasing with every new functionality built on it
I mean, I unit test to a degree like when things become a module. and you don't want to really maintain it anymore. it kept your sanity to be on the high level if your unit test is good enough
I will say, this conversation really boiled down to writing code that makes money, but knowing when money-making code needs to be rewritten and hardened with _good_ unit tests. And honestly, as someone not in favor of extensive unit testing, some things are just painful to manual test and I've written a test with 1000+ lines that was basically all mock, but allowed me to fully test the unit and all it's situations and I hated it, but I know it works.
This isn't really as subjective as you think it would be. It's simply poorly thought out from Theo's side; Theo's main point seems to be that he doesn't know which tests would be useful, that it slows down development, that he wants better tooling rather than creating a lot of unit tests that maybe wouldn't catch errors. In these days, with languages that have very poor type systems, if you have a project in one of these languages (i.e. JavaScript), then unit tests are simply a must for production software. You have no other way to ensure the results of your code, unit testing is absolutely necessary. And yes, there's linting, there's typescript, there's LSP, but they all are EXTREMELY shallow compared to unit tests. The thing is, with unit tests, the good tests to make are all of them. you have to make sure that every single function you create results in the output you expect. If you do this, you'll handle everything there is to be handled, if you don't miss any functions. The worst argument here is that it makes development slower, speed in this case does not matter, and for the usefulness unit tests bring is good enough to override any negative from slowness, if that even exists. But let's talk about actually good type systems, say... Rust. In these languages, you need WAY LESS tests for things because the language's type system stops a lot of errors, as well as thee error and borrow system. But you still need unit tests. No matter how good the language you use is, Haskell, OCaml, whatever, you *need* tests to make sure *your* program works as intended. Unit tests are objectively very useful for all production code, no matter who you are, and I bet my life savings that Theo had a couple issues that Unit Tests would have solved.
With languages like JavaScript you need tests for logic, types, etc, while with languages like Rust you only need tests for the logic. I have ecountered so many bugs in my logic that would have been imediatly catched, and easily fixed if I had used unit tests.
End to end/ functional tests - Ensure that both backend and frontend is working according to product owner's requirements. Unit/ Integration tests - Ensure code is logical, simplified and all branches are taken care of Manual tests - Ensure user interactions do not break code in edge cases and user flow makes sense. All tests have their own purpose. QAs take care of functional tests which help them with regular regression tests. Manual testers take care of all possible user interactions. I believe that at the very least, it is the responsibility of developers to do integration tests to prove their logic works on a code level and unit tests as a type of documentation to developers on what their code can do without having to read the code.
I like unit tests. I introduced unit tests to our product about a year ago and it's been mostly good. I don't like badly written unit tests that exist only because a reviewer dogmatically asked someone to write tests and they didn't know what to write because they didn't know what to test. Sometimes it isn't obvious what should be tested and you end up with tests that don't really test anything. 5:02 Absolutely agree. This is why a strong type system is a must and also why dynamically typed languages are terrible for large projects. I personally love Typescript's type system because you can do some powerful shit with it. You can accomplish with the type system what you might do with a package or framework in plain JS.
When he said new devs. I started in my current role and before my project got off the ground i was helping with a refactor. The first thing i did was write unittests so that i could make sure that when i changed the internals everything still worked in the same way. Ive seen some god awful code that would be really hard to unittest because its a rabbit hole of functions calling functions calling functions. Some of those with side effects. Inverting that control makes things easier to reason about and easier to test
Bring in an embedded developer to take the unit test requirement to the next level. The peak of this is writing VHDL or Verilog. If you write a page of code you will need to write 10 pages of tests.
In my eyes, Tests are mostly for other devs as documentation. Tests make small changes easier, but the moment the requirement changes (or was misunderstood, or if an architecture was chosen that doesn't work with the new requirements) it makes changing the code more difficult. (considering the time you needed to write the tests, to rewrite the code, to rewrite the tests, and so on) Even if you are the perfect developer, then you dont have to write tests as long as you expect nobody to change your code, but in the normal world this will happen. So I use unittests to ensure that changing a specific behavior is difficult. And if you have to change the unittests, the understanding of what the class / function should do is different and therefore should be discussed. Unittesting is simply another tool. Its amazing for mathematical problem. Input, Output is clear, no sozial discussion. But when you test your business logic it becomes more complicated. (the more mocks you have to do the "fishier" it gets) With business logic you build abstract concepts that would be build differently by different devs. And often its the design that the initial dev chooses. Lets say he writes a lot of tests and everything works as HE wants, but maybe his implementation of the abstract concept lacks... and other devs will be scared to change it, because it is so "well tested"
I think that is the stance of every 'never-tester'. They do not believe in the idea that evidence can be cumulative, instead of just a self contained mathematical proof.
I can see where Theo is coming from. He wants to ship faster and even if it results in breaking small parts of functionality then it does not matter, as long as the main parts of the application is functional. I find unit test can help you if you are writing algorithm based code or trying to solve a really complex business logic. Code coverage is a good negative indicator but not a good positive indicator. Most of software developers work on projects that do not need 100% code coverage. You will need 100% coverage if you are sending human to moon or Mars or your software drives a pace maker. If the most important parts are covered under unit tests then you are good. Business people or domain experts can decide, which parts are the most important. For a bank this may include deposit, withdraw, transfer and charging insufficient balance fees + more.
Excellent points: 9:45 "More often than not unit test break those assumptions with the weird way they mock shit" 29:00 "I think the era of TypeScript has changed where unit tests are necessary and bring value"
@@calder-ty Ya, it's impossible for code to not be legacy if there's no test. Eventually, it will grow large enough to make it impossible to test stuff locally & then people start to get afraid of touching anything & there's 0 chance of refactoring
@@fycode For frontend specifically, one of the most horrid legacy codebases I ever had to work on was tested - by an end to end jest-puppeteer like system. It caught a ton of bugs, yes, but it did literally nothing to help with refactoring.
I belong to the "unit tests are not great" camp, but in a very different way. Basically I do not trust unit tests to get the job done, and I find that there usually is some other kind of testing that covers your actual needs much better. One of the main problems with unit tests is how they look at problems in isolation, while in reality most bugs comes from complexities that comes when components are joined together - especially when we are talking about bugs that emerge later down the line, which is where automated testing really helps. If you are keeping a style where you break down the code into small units that are easily understood seperatedly, then you will see this issue far more. The 100% coverage also becomes more obvious as a lie when you remember how breaking code into smaller pieces does that, for instance you can take a loop with some branching options on each iteration, and take the inner loop to another function, and in that function you test that the operation of each type work, while in the outer you test that the loop works, but then you miss that in reality the combination either could break down on some loop iterations (like the first or last) or that it was maintaining some special properties such that it was only supposed to work in some cases, but now you do not know whether it contained those subcases and you have to deal with trying to get the subfunction to work even when it was not suppposed to. Some might think that I just do not work on code where you need the kind of reliability that unit testing can provide, but in truth it is the opposite. The code I work on an undiscovered bug in the logic would typically have a cost measured in millions of dollars, if we are lucky, so we have some much more stringet requirements for code to be production ready. If we have to change tests in a non-trivial way, we generally have to make a special build and sent it for certification, where someone will the conduct a scientific black box study on the effective logic to tell us whether the new thing we want to change it to is correctly working, and this can easily take weeks. So what do we use? Mainly a sweep of some combination of integration and functional test, plus a few unit tests used for examples and for some datastructures. Some of the most important tests are going through a series of full run calculations and the comparing the results to certified ground truths.
I`m not really amazed that people get by without proper testing.. but it's sad just to hear what are the work arounds of such a decision.. "code reviews" just sent chills down my spine. Its a useful thing to add to your quality strategy but that can't be a replacement for good tests.
I would say add unit test if you have conditional logic in your unit (function) along with typescript (without any type) and it will solve most of your problems 😀
I'll also say I have had some successes with writing tests first, as it has helped keep me on track of what I'm working on. But I don't exclusively write them first.
4:54 Would love to know the name of the TS rule Theo mentions here. Pretty tricky to google down only by this description though. Maybe it was an eslint plugin? 🤔
39:56 I don't have a lot of software engineering experience yet, but this statement resonates with me because, as a gamer, I'm a huge fan of highly moddable games. A lot of times, the community, working for free or based on donations, is much more motivated to squash all the tiny inconsequential bugs in a game than the game development company who has to weigh the pros and cons of paying expensive software engineers a full salary to spend time on fixing bugs that probably wouldn't affect the sales of the game in the first place.
I think Theo has a point in regards to good code quality being one of the best ways to remove bugs. Poorly chosen abstraction creates code full of pot-holes and bugs. I've been able to find and fix countless bugs by going through and spending time cleaning up existing code - many of these bugs are so nuanced that no reasonable unit test would have caught them. Unit tests tend to test individual features in isolation (as much as possible) - and if a bug only gets reproduced by combining different features in a certain way, you'd never be able to catch it with a unit test - your next best hope is good code quality. As another example - if my controllers are imperatively validating all input parameters using tedious hand-written code, then I would probably write a fair number of unit tests to cover that code and make sure I'm doing it right. On the other hand, if I use a very declarative validation library like Zod, I'm more likely to just trust that it'll do what I tell it to do, and will write at most one unit test related to validation, just to make sure it is wired up. The more code that can be written in a declarative fashion on top of trustworthy abstractions, the less unit tests are needed. This is all easier said than done. High quality code is a nice idea, but in our messy world that's easier said than done. So we have lots of unit tests behind all of our code.
What was that bit about dx? If i got it right, you don't think developer experience is an important thing? Isn't the whole vim / extensions / keymaps / tmux / scripting just dx at the end of the day?
This dude is very start-up-y - if the customer has specific needs, then it's worthwhile writing tests to ensure the customers' needs are meet. As changelogs grow, you need to make sure the core and growing functionality is retained
The article "Integrated Tests Are A Scam" from J.B. Rainsberger completely changed my mind about testing. I used to think tests suck, but it turned out that the way teams are writing tests is just plain garbage and there are actually better ways of doing it. My team is extremely fast in delivering features now, we ship at least 2-3 updates PER DAY. We have 92% coverage. And yeah we are using TDD. If tests are slowing you down rather than speeding you up you really need to question the quality of your code and tests. Good code is easy to test. Edit: this is from the point of view. of a backend developer. Frontend unit tests are bit less usefull.
I've also found that TDD forces me to write small and isolated things. If a test needs multiple DB entities and mocks I know something is wrong. I've been working on an online store for a few years and am going back to add more tests. It's difficult in many places because there are large transaction scripts that touch every part of the system at once. It's impossible to test all permutations of just the checkout page because it has a crap ton of businesses requirements and edge cases (around 10k sloc). I'll be clicking on buttons all day long to test everything just once lol
More junior dev here. My company produces an ecommerce site. They have no tech lead, no mentorship on my squad, and heavy testing, to the point where we are testing subfunction mocks => mock.setup().throws. assert throws(). And I must write my own unit tests. I am not happy, I am able to write maybe 5-10 lines of code each week. Between the extremely complicated structures, the unit tests, the integrated tests, and QA's testing suite, I spend more time waiting on feedback than I do thinking about and writing code.
The argument that "testing requires foresight, and my team doesn't have foresight" is wild to me.
As tech lead on most of the projects I work on, I write unit tests *because* my team lacks the foresight.
50 dev, across 10 sevices, using 6 different languages, and hundreds of millions of lines of code; in an application that handles millions of dollars a day. Unit tests save my ass at least twice a week from breaking code that I haven't touched and sometimes don't even know exists.
THIS IS SUCH FACTS
I think u might need a unit test to count your lines of code
50 devs absolutely are not even close to enough to maintain hundreds of millions loc. And what's even more silly is saying that 10 services is all that comes out of that 🤣
You are not supposed to count the lines of code in the OS your services run on.
And it's usually tests that I didn't even seem necessary that can bring up the biggest bugs.
Like I had 2 getters for 2 same type items, so I copy pasted it and changed the name. Unit tests told me I didn't return the correct item in the second getter, seems I forgot to change the field I returned. It's incredibly minor things like this that make it worth to even unit test mundane stuff, especially if it's public.
I don't unit test everything either, I just make tests based on the requirements and some sanity tests and that exposes regressions and ensures features work as they're supposed to.
Unit testing saved my marriage.
reasonable
Explain
Definitely not ruining mine 😂
did you mock your marriage?
@@rinkusamanta1065 yeah but it failed… awks
Theo has the take of someone that has never worked through large code migrations in big companies with compliance requirements. Tick any of those three criterias and you instantly thank unit tests every day of your existence.
Hey. You liked this right? YOU LIKED THIS RIGHT??? Much disappoint in you little algo inducer.
BTW I have seen some mean responses about Theo. For me, say what you want, but be nice to the guests please. They are willing to come join us!
I loved it!! Hearing senior devs argue (in a civilized way) about dev stuff is just amazing!
Whoa, bro.... Done! Sorry! I'll even unsubscribe so I can subscribe again.
It was a GOOD CHAT!!!
I hated Theo’s ideas and arguments. I don’t hate Theo, he seems to be nice. No unit tests? No rebase? I’m a backend engineer, most of the time I need to read code not written by me or anyone in my team, unit tests allow me to know if the behaviour is intended and gives me confidence that I’m not introducing a regression. Rebase helps to have a clean history, why this line that seems to be stupid was introduced (spoiler alert, there’s usually a reason to add that stupid line).
the best part about unit tests is that they force you to rethink your dependency graph
if the test is a pain in the ass to write because of some object that you don't have in the test environment or because you need to set up a ton of global mutable data, those are huge hints that your code is turning into import spaghetti
This. I think the arguments Theo is presenting is mostly that it is easy to write bad unit tests.
I think it's fair that being a new dev does not make you a bad dev, but new devs are inexperienced. The only way to learn how to write good tests is to spend time writing bad tests.
If that is what you are presenting as the best argument for unit testing then it is fundamentally incorrect way of addressing the purpose of why and who you are building your software to. Business solutions/goals and how the technology addresses those domains are things that are suppose to guide the testing process not the other way around. Identifying the minimal resources you need to make software and the kind of architecture you have to choose within domain constraints are non coding activities that always comes first before you ever dive in first to coding in the first place. The overarching purpose of building software is solving a problem that you value for a target audience(client). Unit tests are just one of many tools at trying address maintainability and consistency of behavior of modules.
Unit tests are just a tool, methodology and a means to an end. Like other methodologies there are certain scenarios that unit testing just cannot do what it's suppose to do given the constraint of the software been built. A real life example would be in a banking client where you have to test interaction between your software and some other software and get the data of an obscure protocol of some obscure proprietary banking application that regulators require you to use for some compliance reasons. In this scenario, you cannot unit test this problem as that protocol and data you get are not open to developers in terms of the code but only the protocol and data received is what is needed to do integration tests. After all, why sabotage a banking regulatory contract for the company that do not want to be in the line of fire for regulatory violations. We have to treat software principles as nothing more but guidelines that can bend based on what we want to achieve. Otherwise it's dogmatic to prioritize methodologies and tools that even do not appropiately address the constraints of the domain for a certain audience.
It’s why DI and IoC are great in large applications, can be annoying to setup at the start of the application, boilerplate code, but down the line are a pleasure to work with
Damn that's a great point. I've suffered from making wrong choice of project structures in the beginning. Then I thought, let's write tests, and then gave up cause I was like, how the hell do I bring this dep into my test?
And that personal project had a wonky structure
this is why DESIGN is an important paradigm of TDD
As a FE “engineer” using React I would say 90% of the tests I have seen in my career were already handled within React. Writing ACTUALLY useful unit tests is a skill that is rare at the level of businesses I’ve been working at.
its very hard to write proper unit tests when it comes to a heavily framework dependent code
But how do you know if something was handled by React? What if you update to "React 2.0"? It might not be handled anymore, you have to make sure it is.
@@QckSGaming if an integer update isn’t backwards compatible then there’s more to deal with than just unit tests.
@@hektik2200 what? I'm implying you update your dependencies
@@QckSGaming right and I implied that if you update a dependency a whole integer version and the dependency is not backwards compatible with its previous version then you’re going to have to make changes beyond unit tests to even support that update
From Theos comments, I’m guessing he is completely okay with users finding bugs.
In my line of work bugs generally come with Loss of life and lawsuits, so I rather have great test coverage.
yeah, its definitely exposing the thought process difference between front end / back end.
exactly, ye it really is a question of, what level of testing can we afford to/not afford to NOT do?
Instead of UNIT TESTING EEEEVERYTHING. (1000 broken unit tests later.......)
Totalmente! Por cierto... Hola Xavi xD.
@@ddomingo hey! Claramente las grandes mentes piensan igual! Jajaja
it's not only that, it's a running theme across frontend developers to have absolutely no clue about basic programming concepts yet still make bold statements about them
I had such a good time listening to this. I really hope it becomes a more regular thing. Love how opinionated you both are. Learning a lot!
Hey @ThePrimeagen thanks so much for this content! This kind of discussions are really valuable! Please keep then coming!
we trying :)
I've worked on so many projects where the lack of tests bit me and I can't imagine doing it any other way
same
Surely the public API is all that matters tho right? Doesnt matter if internals change. Devils advocate
> DevHour #1
Hope this is a regular thing. Was wanting more after the last one about git merge vs rebase, went to twitch to find the full version but couldn't find it there, glad to see this pop up in my subscriptions.
yayayaya!
Black box unit tests = good.
White box unit tests = good luck maintaining your tests every time you refactor something.
yes yes yes yes yes
Unit tests are so you can refactor without breaking anything. If you’re changing tests during a refactor you’re doing it wrong
whats the difference?
@@ness-ee What if refactoring code requires new expectations?
@@trappedcat3615 that’s not only a refactor then, it’s adding new requirements.
This video came out at a perfect time, I am writing up some best practices at my company on what we should/shouldn't test and this helped shed some light on what is good for testing and what is pointless to test.
yeah. there are TONS of pointless tests and I think Theo really did do a good job at addressing them.
It's interesting that the argument is basically down to a fight between "fun work" for developers and maximizing feature release velocity, and actually learning good automated testing and making sure that the user gets well working stuff reasonably fast. I do like the idea that the first culture fits prototyping and getting things out fast for active user feedback, while the latter is suitable for maintenance or when you simply don't want to force your users to be paying beta testers 😉
I'm sad because I think Theo's self admitted extreme opinion hurts his arguments more than it forces a confrontations with his ideas. Looking at his arguments, ignoring the extreme stance can help people learn how to write better tests.
Tools, like compliers/static type checking, are huge in being able to eliminate entire classes of bugs that you would have to test for, and that would make your code more rigid. I think this is why you see things like Typescript and MyPy do so well.
If I was to boil the arguments down it is:
1. Unit tests aren't effective at prevention of bugs.
2. Unit tests are a strain on developers time.
3. Unit tests can cause code rigidity.
For point 1, I'd say that's only somewhat true. Unit tests will not cover all situations. But they can demonstrate correctness, particularly for important use cases.
2. I just flat out think he's wrong about this. I think empirical studies have shown that testing improves the speed developers get things done in. Furthermore it seems the time "saved" by not unit testing is spent manual testing and in intensive code reviews. The only part I would agree on is that if you are spending a lot of time creating mocks and set up, that is wasting time. But then you might need to rethink your code under tests design.
3. This is true, I've seen a lot of tests that almost backpropogate themselves into dependencies. However this is often a misunderstand of how to write unit tests, not unit testing
Great stuff and I'm glad Theo was willing to come on to argue his points.
i would say that was a great breakdown riht here, gg
On point #2, I'd argue that lead devs have to spend a lot of time on code review anyways.
I've often been tricked by the fact that my teammates have written tests that pass and look fine without paying much attention to how they were written precisely.
Point is :
- no tests : we do have to review the code carefully
- unit tests : we do have to review tests carefully, and also the code
In some contexts we might better have no tests than s------ tests, cause discussing, teaching and bringing the testing culture requires a supplemental effort compared to teaching coding best practices alone.
@@ze2like that quality time should be spent in review regardless if there are tests or not is a fair point. I think that the effort in cultivating a culture and mentoring younger devs about writing tests is worth it and helps in developing cleaner code.
Refactoring and adding documentation also take time away from directly developing, but they don't get the same vitriol as testing does.
The pay off for unit tests are when the complexity gets quite big.
It's great if you are creating simple widgets.
It sounds like a lot of manual testing.
I honestly thought we moved past this argument but remember my php days "well it works when I ran it"
Interesting to see we are still here
I'm a relatively new dev, but his argument of "I don't trust my developers to find edge cases more than I trust users to run into them" just sounds moronic. You trust the devs to find as many as possible so that the experience isn't completely broken when things are released and then you fix bugs users run into asap as well no?
Also when Prime brings up legit use cases for them he's like "someone else will do this job and it will be fine because they're probably more pro unit tests". like what? how is that an argument?
In my experience devs not doing unit tests ends with QA having to find bugs late and then playing bug-ticket ping pong with the devs. It also usually happens way too close to release for comfort.
this convo is absolutely amazing cuz it shows the difference in approach between the back and front end nowadays
Lots of great tooling for FE unit tests, if people choose to use them.
Can u please make a video about doing the "right" unit test? Like how to do the right mockup function, etc. It will be very helpful for beginners like me!
I feel like writing "tests" first is bad, but writing a "spec" first is good. Thinking about the requirements and the interface before you write the implementation works sometimes, I think.
i am more on this side. I don't do tests first
I like doing this when I can and call it “write the interface you want to see in this world.”
I just finished the book on clean architecture and implemented it in two apps so far and I have changed my tune in unit tests.
Everything I write now has an interface and tests for all public apis.
I have been a lot less stressed adding all the tests too, and now I can build with more confidence.
These projects also use typescript.
Title and author(s)?
@@Average-Lizard probably clean architecture by Robert Martin (Uncle bob)
I love that Prime is replying to everyone. I have a job interview for MTA NYC subway as a SWE tomorrow and you have helped me a lot. I just wanted to thank tou
You are welcome! Also congrats on the yob interview, crush it!
this was great. it was like the friends that you have from different groups meet each other and you are nervous lol I really enjoyed this. I hate writing tests but they are vital IMO.
The actual valuable tests are around enforcing business behavior correctness, not about technical concerns.
Context matters a lot too. When the result of your error is some text blowing out of a div or janky scroll behavior, no big deal, roll forward. When your error produces tons of data that has to be reconciled while all the stake holders spaz out, you feel differently. Knowing its 'correct' and being able to confidently iterate on that 'correctness' is suddenly quite valuable.
Theo's position is only applicable for small teams working on CRUD-like applications IMHO. You simply can't have the energy to manually keep up with all the crap going on in a production codebase when dealing with most real life, entangled, non logicless software products.
i am curious how far this approach can go before it costs more. interesting concept though
@@ThePrimeagen 2 - 3 years, depends. First year is "go horse!! rush!! typescript is safe for type!! just make an crud! you don't need to validate this! you can use a library for that". After that any small change breaks everything. Nowadays a sprint(2 weeks) it's about 75% fixing bugs that return(bug -> fix -> bug the same thing again -> fix -> another bug...).
@@delsinrowe7328 Yep... I've recently inherited a code base that probably started out like that. It's gotten to the point where it's so bad our bug fixes break existing code or even worse reveal another layer of bugs which were simply hidden by the previous bug that was higher on the call stack... Testing unfortunately is not really viable in this state, since abstractions are almost completely lacking.
@@richardkerckhove 'Just media companies' which would pay lotta $$$$ if their systems'd be built to break repeatedly.
I mean, he doesn't work on a crud-like application, you can just check it for yourself. But I feel the important thing here is context. Delivering rapidly changing user experiences can and will be set back by unit tests, giving almost no value. This is true for almost any non-critical service. In the other hand, for systems that handles a lot of entangled business logic that has changed over decades, you have to have a tool like unit tests to assure you are not screwing up an obscure event handler written in 2014 that impacts revenue
And giving it a second thought, unit test may very well have a place in a crud-like applications if, for example, you allow the creation of custom models. There is a lot of mapping hell in those kind of applications
I find unit testing to be like "show your work" math equivalent. At times tedious, but overall worthwhile to catch any simple mistakes that can cause bigger issues down the line.
Don't you think about these possible issues as you program? Does it take a unit test to show this to you?
@@ifstatementifstatement2704 if course you're thinking about them, but we all write bugs. Half the time, my bugs are a dumb typo when in my head, i knew exactly what I needed to do. If I'm writing a unit test, I'm pretty much garunteed to catch those little issues because it's very unlikely that I'll make the same dumb mistake twice.
@@cyropox8235 what if you make a typo in your unit test and it doesn't test anything, or you just pass your test by sheer luck
@@ruyvieira104 yes it happened to me. In 8 years of testing I remember only one case. The test was asserting an expected exception as Exception, but a more specific one was actually expected. The code failed earlier than expected with another exception and we've found that by chance at least one year later.
@@bigstones84 then you should start unit testing the unit tests too
So all I got from this is that Theo's approach is fundamentally reactive whereas doing unit tests is proactive. This is like not doing Chaos Engineering experiments to prepare for situations where it would be extremely catastrophic at scale if there were an outage. It seems Theo is over-indexing on efficiency/efficacy and doesn't realize that not investing in doing unit tests will inevitably result in some type of failure, especially as his team scales.
Just like with anything in life, we should approach things pragmatically and I don't believe doing no unit tests at all is the right approach.
I think it also takes into consideration the possible impact a bug can make in production. If the worst bug can only impact UX or be easily rolled-back to fix then sure, be reactive like you said. If your product has legal consequences from bugs (things like Insurance, Health Care, etc) then being proactive to catch bugs is much more valuable. Just like any dev tool though, it comes down to how its used and what problems you are trying to solve (there is no silver bullet for testing).
@@ThewNell This is the correct take. If anything, when team scaling problems arise, like when the platform has 5x the amount of features, or they have different sub-products, unit test can be adopted.
Until no critical business process has been affected, and the end user is happy with the response times, there is no reason allocate resources into adopting unit tests now, instead of working on having reliable response times, just because we think he may have scaling issues.
Yes, but he is actually saying that his approach is for version one of the product, and then we will hire somebody like theprimeagen to do to a more robust version with extensive testing
Had unit tests, helped me find that some dates were improperly converted so I could fix it.
Now we get to avoid an angry client.
So thanks again unit tests for helping me make a great product.
I loved this conversation. As a solo, senior, experienced (10+ years freelance, started in web around 2003) developer, I rarely get this kind of intellectual stimulation from other senior devs. I loved this.
Also, regarding your channel, I LOVE finding another javascript backend realist-someone who hasn’t swallowed the javascript everywhere hype. 💯
yaya! Theo has a pretty based take on these things. He does love himself the jabascwipts, but he does recognize its weaknesses for what they are. big fan of that.
Tests as documentation is great. I like “functional tests” or tests that confirm the user level requirements. It’s hard to make enough functional test cases to cover the combinatorial complexity of every use case so we breakdown the responsibility into parts and create “unit” tests to fill in the gaps of the limited functional tests.
Having also built a video calling platform, I can see what Theo is saying, there is a bunch of stuff that is super hard to unit test, but, I feel like not unit-testing part of the code should be a decision not a default.
in most cases if this happens, it means the code design is one big mess, and luck of unit tests was one of the factors which could have prevented that.
this was awesome!!! please do more! this is much helpful for entry level engineers like us to understand how decisions as critical as this are took :)
I love this kind of conversation, especially coming from different views and two brilliant engineers. ❤
yayaya
When you mentioned a writing unit tests for a touch condition, I remembered back in the time I developed a multi touch gesture library. I had to write unit tests to develop the library, otherwise I would have to do gestures on the screen but wouldn't be able to control the exact input. By mocking the input, I managed to develop the library heaps faster.
I have faced many cases like this, having the assumption of unit test will be a hinder for the developer is not always correct. Especially if you need to go through many steps to reproduce the issue, you will save time by encapsulating the logic and writing unit tests for it.
0 unit test argument is very hard to defend. IMO if the project has any critical process (i.e payment integration), or will have a long lifecycle (>1 years) you need to have a testing suite. You can choose to not to have any unit tests but you would loose the same amount of time if the lifetime of the project is long enough
I know I turned this comment into an assay, so sorry for this. But it's good to remember that web development is just one branch of development and users of web are more comfortable using broken things than other areas. There are many other branches of development that you can't ship broken application or if a problem has been discovered and solved, the client won't be happy to see the same problem creeping up because of a new feature.
I almost always develop unit tests to speed up dev. There are just some things that are too complex and i need a way to keep me on track.
I totally am right there with you.
Where I work, we're building mostly simpler, ecommerce/business focused websites.
Unit tests rarely add value for us. If we have a particularly complex unit, we will unit test. But most of our code interacts with the database in complex fashions. Trying to strip out the database would remove most of the things we want to test.
Most of our automated tests are at the service level, using nomenclature of our requirements.
When we're integrating with 3rd party APIs, we really ramp up tests though, because that's where the sneaky stuff lies. If you start having problems when calling an API... did your code break, did theirs, or did they change their API? Our tests aim to answer these questions quickly and easily.
Also, I can't imagine having to remember all the corner cases of my code. Put bluntly, I'm not that smart and my memory isn't that good. Having automated tests that tell me what I remembered to test is a good thing.
I feel like this is much more difficult to convey to juniors though. It's much easier to just say "test everything!" Adding nuance, and value judgements, can make it much more difficult.
i think this is a reasonable take.
There are some things that just shouldn't be unit tested.
Awesome conversation. Very nice to see two points of view on some very important topic in our daily basis activities as developers/engineers. Hope to see that more often here!!
Unit testing is so misunderstood. Kent Beck did NOT mean to test every method of every class, which 99% of devs think. A unit was a feature unit, not a code unit. For me unit tests, functional tests, and even acceptance tests are basically the same thing. Hex/clean architecture really helped me here. I mostly just UNIT test behind my controllers (and only mock my DAOs/backend)
If you do it this way, unit tests are not tedious nor hard to maintain. They don't break and major refactorings are easier. It also helps prevent the "build it and throw it away" approach, since refactoring is so much easier and the unit tests can still work (since they are really just testing an API). See Ian Cooper's TDD YT video for more info.
you are totally right. But most companies understood wrong what is unit testing. And forgot about other important safety nets including integration testing, linters, gradual deployments, etc. When your test takes 3 hours to run, are 3 times bigger than the code tested, whey they overload your CI server are after all the tests are run you still have problems in production... Even in a project I was there was a rule about at least 70% of coverage, and guess what? the developers started adding assertions like True == True just to bump the numbers
This was great to listen to. Both you guys are great programmers with nothing to prove. It was a pleasure lads.
Loved the Powerrangers story and the insight into working at Netflix.
This guy is a bricklayer who solves bricklayer problems, but he'll never build a skyscraper. That's fine - sometimes you just need a small brick house. But I wouldn't get into an office building that he made.
One of the best ways to learn about anything else to watch to experts go at it in spirited conversation :-) thank you for that!
Theo's perspective seems to be limited to frontend.
perhaps more colored by TS than anything else.
@@ThePrimeagen 100% more colored by TS and serverless functions :)
Imagine fixing a bug in a system that actually matters..... from report in 7mins on average....
There was someone way back when that broke up tests in a unique way that I haven't seen done else where but it made a ton of sense.
1 Unit Tests / TDD - These are tests written to aid in the software development process. They test only a single unit. They also describe what the software technically does.
2 Component-Level Tests - The above tests are white-box. These are black box tests. These tests describe what the software is intended to do. While the above tests are going to be appended/updated/removed on a refactor, these should stay almost the exact same on a refactor.
3 Integration Tests - These tests (as much as possible) verify the ability of your code to integrate with external systems, IE databases
4 End 2 End Tests - Used sparingly, these should simulate a complete end to end usage, which can end up relying on mulitple external systems, and exist outside of the program(s) being tested.
you can see the exact moment Primeagen realizes Theo is a junior dev
Time stamp?
lol 9:07
This is a good example of what makes a senior dev senior
If it really took him that long he's not a senior.
I appreciate that Prime is being encouraging of a new kid and seeing in him some of his own early immaturity and brashness etc.
Theo has no mature takes at any point in this sequence, they're all things I've heard from brash young male devs who are bumbling around. None of them indicate any level of maturity or thought, just hormones.
@@fennecbesixdouze1794 female devs unit test?
the last company that I worked at had 0 tests. like literally just like 10 minutes of manual testing before deploying the new version. The amount of times that production went down or was unusable is insane. Funnily enough they still exist. I don't know how, but they do lol
Another great video of two super talented guys :) the flame wars are also fun :p
yeah Theo is awesome!
Working in open source, I can't imagine not having some level of test coverage. The number of commits that get pushed by the community (and sometimes by us, even after manual QA) only to fail CI, help to prevent us from tagging release on a good number of bugs.
yeah, that would be rough
This. I'll keep my OSS projects at 100% coverage. However, I define 100% coverage as **"all the code we care about is covered"**. I'll slap some "ignore coverage" comments around stuff I don't care about, that way the coverage report being
@@JeffsterProductions Yea. I worked on a team where we weren't allowed to use any "ignore this for testing" type comments except for npm modules. Makes testing absolutely useless, when you have 90% coverage, and none of it is in the two actually complex pieces of logic in the entire UI, and most of it is just Jest snapshots testing if React can render a component...
I would love to hear their entire engineering flow.
Having no unit tests is a death sentence for large scale software. It doesn’t matter if you can get to production in 5 minutes with a fix. The entire point is to develop software thoughtfully and prevent as many bugs as possible to deliver the best user experience.
Having high quality tests has caught so many potential bugs or regressions very early in the process that not even a quality engineer let alone a user would have known.
agreed, but its a fun perspective and hopefully the biggest thing to stick in your mind is stop testing type like requirements in TS
Agree 100% The whole "we can fix it really fast" mentality is like refusing to wear PPE because you live really close to the hospital. Just spend a few minutes finding and putting the gloves on and don't get your fingers cut off in the first place.
For sure.
Mostly doing RTL and pure unit tests at this point.
Having RTL run off a mocked data layer via Swagger that’s maintained upstream. When we take a new version of a service the docs have example data that is consumed in the tests. Makes it very simple to write good tests that are fairly easy to maintain.
We may regret this idea one day though. 😃
"If you say none, we're fist fighting" 🤣Loved this convo @ThePrimeagen.👏
Theo makes some great points when it comes to unit tests. Now, I don't think you should get rid of them almost entirely like he is suggesting. But I agree that tests often have the tendency to weigh down bad code, and therefore the bad code persists. I also agree that good tools are better than good tests. If you have a million tests, none of that matters if your tools and your stack don't continuously steer you into the pit of success.
Write fewer tests, and ruthlessly refine your tools, stack, and architecture.
I see where both y’all coming from.
I like Theo’s quote of not necessarily trying to do it right, but wrong well-enough fast-enough.
I’ll get bogged down trying to write the perfect thing and take too long at a foundation vs getting it functional and leveling up. I was brought up at school on literal assembly, c and python. My C teacher in particular drilled the hell out of tests so I do still use them.
I think a hard part of tests is writing them well.
You really need to understand what’s right, what’s wrong, and what’s the rest?
I appreciate having both of y’all to learn from.
It is a matter of scaling, whenever you have a product that consists of multiple layers of complexity stacked upon each other which are being worked on by multiple teams, the testing is going to your best friend.
Very nice debate! Thanks guys and great to hear two different points of view on testing from two very experienced developers
I'm realising how similar debates around testing can be to debates around git workflow. Like, CI/CD doesn't make sense for people doing feature branches because they just can't merge every day, and CI/CD people will think week-long branches are silly because they use feature flags that let them merge whenever they want. If you release every day, PRs make little sense. If you release twice a year, you need to have in-depth discussions on changes.
I feel like a lot of debate around testing comes down to something similar. If you have a close relationship with your users, maybe testing isn't important and they see bugs as a fact of life. If you're contracting, they could see issues as broken promises that cost you a renewed contract. So, basically we're back to "it depends"
That was GOOD! Thanks to both of you for sharing this. Now I have to compute the answer to life, universe and sh-- out of this. Pretty sure over-testing inner parts of code brings a lot of inertia at my workplace too. Even though we don't unit test our frontend code. I'd recommend testing business domain services at a component level + having some unit tests on complex internal parts.
Because as stated during this stream, the simple fact that unit tests exist, may scare junior devs and prevent them from engaging in heavy refactoring.
That being said, if you have E2E and component-level tests above all of this, it's easier to engage in a rewrite/refactor.
Moving from heavy object oriented to more functional design + dependency injection moved me away from not finding value in unit tests (mainly because I was mocking so much, how could I test if I was mocking right lol), to finding value since it's so much easier to manage. Additionally, the further I moved away from the UI, the more unit testing felt valuable and less like a chore.
You are so close. Now go one step further and learn about generative testing. Let the coputer write gazillions of tests for you on the fly and find all the bugs you would never be able to find by writing your own unit tests.
@@Simon-xi8tb this is always my go to with web service APIs that I can either shove the design into swagger or something like protobufs, are there any other tools that I might be missing for other similar interfaces/approaches?
Man when Theo said about rxjs "I dont even want to consider it because it will infect my codebase and ruin everything" I was like hallelujiah! This perfectly sums up every rant I've ever went on about rxjs. It may be a fantastically powerful tool but it is not worth infecting your code with it.
Yes, its a very leaky abstraction
Is there a reason to avoid infecting your code with it? Does it lead to bad code down the line, or is it just considered bad to have a single library touch a large part of your code base in general?
I feel like if it doesn't make the code worse, then it just becomes an architectural decision on whether you want this thing in your code. For example, using the "async" keyword in languages like c# will cause half of your codebase to need to use that keyword, but i don't really think that's a bad thing.
We need a whole episode for production bugs.
that would be fun
I've found it amazing listening to John Romero one of the creators of the game franchise Doom (together with John Carmack) and discovering that they had absolutely no concept of "Unit testing" back in those days, they did every thing by hand, it's crazy how advanced for its time the game were.
I first heard TDD in 2012. I tried to convince my colleagues to use TDD on a new project, but the manager said no, because it would take more time. Also, we would have to learn how to do it properly.
I did, however, manage to use TDD to fix a bug on legacy Java code. There is a RUclips video that taught me how to write tests in order to understand what the code did first. Then, I would refactor the code to make the functionality I wanted to fix testable. Finally, I would write tests and asserts for the correct functionality. I run them and see them fail, because the correct functionality is not implemented yet. I implement it, run the tests and repeat until they pass. Done.
If you have to ask your manager permission for doing your job in the best manner, your workplace is doing it wrong. I never asked anyone permission to make a test in my entire worklife; it is simply how we do the job most efficiently. Code the spec for the unit, implement it one bit at a time, refactoring and changing tests as you go along if the interface does not make sense
So, I really like these talks with a guest speaker. Dig the banter, and the challenge. You should do more like this if you can!
I've experienced a project where due to time issues no unit tests were written, refactoring caused regression so people didn't do it anymore, the code base got worse by the day. After a while the project got abandoned by our company. So I believe, even unit testing 100%, upon which I do not agree neither, is better than no unit testing at all. Non-testers might be more productive at the start, but the longer the project lives, the more regressions factors in and people get afraid of refactoring and keeping the code base tidy and well designed. Duplicate code will start piling up, and because of the time wasted on both solving bugs again and again due to regression and duplicate code, the productivity of a team that does unit test overtakes the team of non-testers.
Also unit testing brings more than just preventing regression, it forces loose coupling and more desired aspects of code.
P.s. not te be rude, but your guest makes silly conclusions a lot of the time...
Yeah, the refactoring case is a great reason to consider unit testing.
To try to display Theo's argument best, he would suggest to rewrite it instead of unit test it. Its a valid argument, i just would still prefer unit tests :)
@@ThePrimeagen Yeah I should have written, conclusions that seem silly to me, he might have reasoning I did not grasp, but my comment was becoming rather lengthy already.
But, rewriting instead of unit testing does not make sense to me. You would unit test and rewrite, but don't break as much doing so because you have the tests? And without the tests, developers get too anxious to touch something that works as intended, especially if altering it resolved in bugs earlier?
Great format Prime, I think you really shine when you go head-to-head with someone else with as much engineering knowledge. The banter is fun and you guys keep it entertaining while being educational at the same time. Waiting for Ep 2...N.
A great part was when you proposed a class as a lightweight observable for the statemachine and Theo went into detail about the different state machiines interacting (1 clientside for audio levels etc, 1 server side for permissions about what can enter the chat room). Bits like that are really entertaining because you are tackling a real world problem and discussing solutions. Like a coding interview, but less formal , more fireside-chat style
TDD is not about defining architecture, it's more about defining your API and is 100% about user interaction with your system, because you are designing the API the way you would like your consumers to use it. Also what's most important thing - TDD leads to testable code. It's impossible to write untestable code with TDD. Anyway, TDD - just like every tool - is great, but not everywhere and not for everyone.
I think like all the buzzwords it got painted on everything.
I’ve only found value in TDD outside of UI and it’s value was pretty limited in my experience. I always ended up focusing on the testing aspect and less about just molding the architecture.
I don’t even worry about testing now until I build something that works.
There is testable code and there's code that has been formally proven to be correct
I've never seen TDD done at a user interaction level (basically integration tests). All of the advocates I've seen for it use it for unit tests. And there's one very simple reason for that. If you write TDD tests for a brand new feature to test the way that a user will call your work, your pull request will likely be over 50 files long, often over 100 files, because you have to plumb every single piece of that feature to get the tests passing. No one can actually do a proper code review on PRs that big.
@@cyropox8235 ATDD does this on user interaction level. You write DSL (the what) and then poke your system from different sides as a user would (you can have the same tests test API and UI and whatever else, by using "drivers" for each channel - the how). Basically acceptance test first, then unit tests until the acceptance test passes and repeat. You can make multiple small PRs by story splitting - this way the reviewer has a bit of context what the PR adds/changes - and not one giant PR.
Now... I think ATDD also says that business can write tests which is fucking bullshit, but if you ignore that part, you are left with a flow that helps you test what you need to test. Also, not everything needs to be tested. Also also, sometimes you cannot write tests first, because you have no idea about the system (this is particularly true for UI). Also also also, UI tests are not great.
I'm undecided on this after watching this. What this convinced me of is to conform to the company development policies (for lack of a better word) because the senior or lead software engineers probably have a good reason for why they do the things they do.
I think also don't just jump into a company and preach unit tests or not. Use what the company is doing now and then see if you can improve the process.
Before watching this, I was skeptical of the usefulness of testing. After hearing your points about how you dislike 100% code coverage and TDD but still value good, thought-out tests, persuaded me of their worth.
i literally built a 200 test suite to ensure one of the new video players work at netflix :) too dangerous to code alone.
@@ThePrimeagen show us, teach us to write better tests and how to approach this. I’ve always struggled to find them helpful, especially with front end dev
if all it takes is 30 mins to erase your scepticism of something, then you don't do enough research.
I have this thing that I think Theo shares where once we figured out how powerful the type system in Typescript was, it become so obvious when bullshit code exist, code that just restricts functionality or constrains scope, those can be replaced with nifty types that themselves reveal novel fun ways of api interaction
I get both sides of the argument here. Experienced engineers know when to write unit tests and when not to. With junior, tests can often slow them down. If I need to familiarize myself with a new codebase and don't have the ability to sit down with the the creator, I'll take one with tests vs no tests any day of the week. Thinking about how I'm going to test something has definitely saved me from introducing unnecessary complexity to code I write, even if I end up writing no tests for it. It also depends on what you're building though. If you're building a library that a lot of people depend on and you can't immediately jump on a call with your users, it helps to have tests. If you're building a new startup and your users are ok with things breaking as long as they can get through to your devs, then that iteration speed provides more value, as your startup is burning through cash unless you're directly providing value to users. All products have churn and you'll hopefully have metrics in place to be able to understand how many users you're gaining vs losing.
Thanks Prime & Theo for having us as a fly on the wall.
yaya! thank you :)
23:50 "Whenever I rewrite and burn down the whole city, I look at the unit tests as an example of all the things I have to remember."
This, to me, is the most impactful line of this whole video and has me leaning more towards preferring unit testing. I don't know if anyone plays modded Bethesda Game Studios games like Elder Scrolls or Fallout, but to me, it seems like unit testing is to Vortex, what no unit testing is to Mod Organizer 2, at least as it relates to sorting your plugins. In MO2, it lets you drag and drop your plugins willy nilly, but in Vortex, it forces you to teach it the reason why you are reordering the plugins so that it can do it automatically in the future.
Man 45 minutes!! I thought it was going to be long but it finished so quick! Great discussion! Very interesting perspectives. Personally I am a major advocate of unit tests, but I agree with the idea of if you have only two extremes: 100% code coverage or NO unit tests, I would probably go with NO unit tests, though this is completely hypothetical.
yeah, the two extremes are both silly to me. Per usually, the world is grey :)
Great video, really interesting to hear your perspectives on unit testing.
And for the first quick version - you just do a POC and see if it works. If it does you do it with a proper architecture and tests as otherwise you'll get yourself into technical debt increasing with every new functionality built on it
this is effectively my dev style
This content is so valuable. Thank you, brethren! I just subscribed to Theo as well.
I also want to add, the comments here are also helpful. Maintaining the dialectic is the most important thing here.
thanks man! I appreciate that :)
I mean, I unit test to a degree
like when things become a module. and you don't want to really maintain it anymore. it kept your sanity to be on the high level
if your unit test is good enough
this is how i feels
I will say, this conversation really boiled down to writing code that makes money, but knowing when money-making code needs to be rewritten and hardened with _good_ unit tests.
And honestly, as someone not in favor of extensive unit testing, some things are just painful to manual test and I've written a test with 1000+ lines that was basically all mock, but allowed me to fully test the unit and all it's situations and I hated it, but I know it works.
This isn't really as subjective as you think it would be. It's simply poorly thought out from Theo's side; Theo's main point seems to be that he doesn't know which tests would be useful, that it slows down development, that he wants better tooling rather than creating a lot of unit tests that maybe wouldn't catch errors. In these days, with languages that have very poor type systems, if you have a project in one of these languages (i.e. JavaScript), then unit tests are simply a must for production software. You have no other way to ensure the results of your code, unit testing is absolutely necessary. And yes, there's linting, there's typescript, there's LSP, but they all are EXTREMELY shallow compared to unit tests. The thing is, with unit tests, the good tests to make are all of them. you have to make sure that every single function you create results in the output you expect. If you do this, you'll handle everything there is to be handled, if you don't miss any functions. The worst argument here is that it makes development slower, speed in this case does not matter, and for the usefulness unit tests bring is good enough to override any negative from slowness, if that even exists. But let's talk about actually good type systems, say... Rust. In these languages, you need WAY LESS tests for things because the language's type system stops a lot of errors, as well as thee error and borrow system. But you still need unit tests. No matter how good the language you use is, Haskell, OCaml, whatever, you *need* tests to make sure *your* program works as intended. Unit tests are objectively very useful for all production code, no matter who you are, and I bet my life savings that Theo had a couple issues that Unit Tests would have solved.
With languages like JavaScript you need tests for logic, types, etc, while with languages like Rust you only need tests for the logic. I have ecountered so many bugs in my logic that would have been imediatly catched, and easily fixed if I had used unit tests.
even in js i don't test for types. its a useless practice.
but should you use rust, yes
yes it seems a pretty lazy stand, i dont test because it takes time and I want tools that do that for me 😅
Maybe there's a tool that will auto write the tests... 🤔
that is like saying that maybe there it's a tool that writes code hehe
End to end/ functional tests - Ensure that both backend and frontend is working according to product owner's requirements.
Unit/ Integration tests - Ensure code is logical, simplified and all branches are taken care of
Manual tests - Ensure user interactions do not break code in edge cases and user flow makes sense.
All tests have their own purpose. QAs take care of functional tests which help them with regular regression tests. Manual testers take care of all possible user interactions. I believe that at the very least, it is the responsibility of developers to do integration tests to prove their logic works on a code level and unit tests as a type of documentation to developers on what their code can do without having to read the code.
I once worked on a company that did not unit test their react things.
The only features working properly were the ones shipped the latest week.
the true classic
Testing in React projects, even when done, is just "let's test if React can do what we all know React can do, and not our own code" 95% of the time.
@@botondhetyey159 that is because you are testing implementation details and not use cases.
Go take a basic RTL course somewhere.
I like unit tests. I introduced unit tests to our product about a year ago and it's been mostly good. I don't like badly written unit tests that exist only because a reviewer dogmatically asked someone to write tests and they didn't know what to write because they didn't know what to test. Sometimes it isn't obvious what should be tested and you end up with tests that don't really test anything.
5:02 Absolutely agree. This is why a strong type system is a must and also why dynamically typed languages are terrible for large projects. I personally love Typescript's type system because you can do some powerful shit with it. You can accomplish with the type system what you might do with a package or framework in plain JS.
Agree or not with Theo, I think he is really articulate about the philosophy he embraces and seems to fully understand the trade offs of his choice.
but some points he makes are just incorrect as well.
When he said new devs. I started in my current role and before my project got off the ground i was helping with a refactor. The first thing i did was write unittests so that i could make sure that when i changed the internals everything still worked in the same way.
Ive seen some god awful code that would be really hard to unittest because its a rabbit hole of functions calling functions calling functions. Some of those with side effects. Inverting that control makes things easier to reason about and easier to test
If only you've brought a backend developer for this debate. Of course front end dev's don't write unit tests...
Exactly. White hoodie means no unit tests
Prime is more back than front
@@oso_estudioso white hoodie is a pejorative for FED?
Bring in an embedded developer to take the unit test requirement to the next level.
The peak of this is writing VHDL or Verilog. If you write a page of code you will need to write 10 pages of tests.
In my eyes, Tests are mostly for other devs as documentation. Tests make small changes easier, but the moment the requirement changes (or was misunderstood, or if an architecture was chosen that doesn't work with the new requirements) it makes changing the code more difficult. (considering the time you needed to write the tests, to rewrite the code, to rewrite the tests, and so on)
Even if you are the perfect developer, then you dont have to write tests as long as you expect nobody to change your code, but in the normal world this will happen. So I use unittests to ensure that changing a specific behavior is difficult. And if you have to change the unittests, the understanding of what the class / function should do is different and therefore should be discussed.
Unittesting is simply another tool.
Its amazing for mathematical problem. Input, Output is clear, no sozial discussion.
But when you test your business logic it becomes more complicated. (the more mocks you have to do the "fishier" it gets) With business logic you build abstract concepts that would be build differently by different devs. And often its the design that the initial dev chooses. Lets say he writes a lot of tests and everything works as HE wants, but maybe his implementation of the abstract concept lacks... and other devs will be scared to change it, because it is so "well tested"
It sounds like Theo's stance is "There are things that unit tests can't catch so unit tests aren't worth it"
I think that is the stance of every 'never-tester'. They do not believe in the idea that evidence can be cumulative, instead of just a self contained mathematical proof.
I can see where Theo is coming from. He wants to ship faster and even if it results in breaking small parts of functionality then it does not matter, as long as the main parts of the application is functional. I find unit test can help you if you are writing algorithm based code or trying to solve a really complex business logic.
Code coverage is a good negative indicator but not a good positive indicator. Most of software developers work on projects that do not need 100% code coverage. You will need 100% coverage if you are sending human to moon or Mars or your software drives a pace maker.
If the most important parts are covered under unit tests then you are good. Business people or domain experts can decide, which parts are the most important. For a bank this may include deposit, withdraw, transfer and charging insufficient balance fees + more.
Two masterminds fighting! So lovely ❤
Excellent points:
9:45 "More often than not unit test break those assumptions with the weird way they mock shit"
29:00 "I think the era of TypeScript has changed where unit tests are necessary and bring value"
I'd follow Theo's approach if I ever wanted to write a legacy code lol
I know I've seen the definition of "legacy code" as "code that does not have tests"
@@calder-ty Ya, it's impossible for code to not be legacy if there's no test. Eventually, it will grow large enough to make it impossible to test stuff locally & then people start to get afraid of touching anything & there's 0 chance of refactoring
@@fycode For frontend specifically, one of the most horrid legacy codebases I ever had to work on was tested - by an end to end jest-puppeteer like system. It caught a ton of bugs, yes, but it did literally nothing to help with refactoring.
I belong to the "unit tests are not great" camp, but in a very different way. Basically I do not trust unit tests to get the job done, and I find that there usually is some other kind of testing that covers your actual needs much better. One of the main problems with unit tests is how they look at problems in isolation, while in reality most bugs comes from complexities that comes when components are joined together - especially when we are talking about bugs that emerge later down the line, which is where automated testing really helps. If you are keeping a style where you break down the code into small units that are easily understood seperatedly, then you will see this issue far more. The 100% coverage also becomes more obvious as a lie when you remember how breaking code into smaller pieces does that, for instance you can take a loop with some branching options on each iteration, and take the inner loop to another function, and in that function you test that the operation of each type work, while in the outer you test that the loop works, but then you miss that in reality the combination either could break down on some loop iterations (like the first or last) or that it was maintaining some special properties such that it was only supposed to work in some cases, but now you do not know whether it contained those subcases and you have to deal with trying to get the subfunction to work even when it was not suppposed to.
Some might think that I just do not work on code where you need the kind of reliability that unit testing can provide, but in truth it is the opposite. The code I work on an undiscovered bug in the logic would typically have a cost measured in millions of dollars, if we are lucky, so we have some much more stringet requirements for code to be production ready. If we have to change tests in a non-trivial way, we generally have to make a special build and sent it for certification, where someone will the conduct a scientific black box study on the effective logic to tell us whether the new thing we want to change it to is correctly working, and this can easily take weeks. So what do we use? Mainly a sweep of some combination of integration and functional test, plus a few unit tests used for examples and for some datastructures. Some of the most important tests are going through a series of full run calculations and the comparing the results to certified ground truths.
I`m not really amazed that people get by without proper testing.. but it's sad just to hear what are the work arounds of such a decision.. "code reviews" just sent chills down my spine. Its a useful thing to add to your quality strategy but that can't be a replacement for good tests.
16:58 really puts this entire conversation in context
I would say add unit test if you have conditional logic in your unit (function) along with typescript (without any type) and it will solve most of your problems 😀
interesting boundary.
I'll also say I have had some successes with writing tests first, as it has helped keep me on track of what I'm working on. But I don't exclusively write them first.
4:54 Would love to know the name of the TS rule Theo mentions here. Pretty tricky to google down only by this description though. Maybe it was an eslint plugin? 🤔
I believe Theo might be talking about noUncheckedIndexedAccess, introduced in TypeScript 4.1.
39:56 I don't have a lot of software engineering experience yet, but this statement resonates with me because, as a gamer, I'm a huge fan of highly moddable games. A lot of times, the community, working for free or based on donations, is much more motivated to squash all the tiny inconsequential bugs in a game than the game development company who has to weigh the pros and cons of paying expensive software engineers a full salary to spend time on fixing bugs that probably wouldn't affect the sales of the game in the first place.
loving DevHour
thanks waylon
I think Theo has a point in regards to good code quality being one of the best ways to remove bugs. Poorly chosen abstraction creates code full of pot-holes and bugs. I've been able to find and fix countless bugs by going through and spending time cleaning up existing code - many of these bugs are so nuanced that no reasonable unit test would have caught them. Unit tests tend to test individual features in isolation (as much as possible) - and if a bug only gets reproduced by combining different features in a certain way, you'd never be able to catch it with a unit test - your next best hope is good code quality.
As another example - if my controllers are imperatively validating all input parameters using tedious hand-written code, then I would probably write a fair number of unit tests to cover that code and make sure I'm doing it right. On the other hand, if I use a very declarative validation library like Zod, I'm more likely to just trust that it'll do what I tell it to do, and will write at most one unit test related to validation, just to make sure it is wired up. The more code that can be written in a declarative fashion on top of trustworthy abstractions, the less unit tests are needed.
This is all easier said than done. High quality code is a nice idea, but in our messy world that's easier said than done. So we have lots of unit tests behind all of our code.
What was that bit about dx?
If i got it right, you don't think developer experience is an important thing? Isn't the whole vim / extensions / keymaps / tmux / scripting just dx at the end of the day?
This dude is very start-up-y - if the customer has specific needs, then it's worthwhile writing tests to ensure the customers' needs are meet. As changelogs grow, you need to make
sure the core and growing functionality is retained
The article "Integrated Tests Are A Scam" from J.B. Rainsberger completely changed my mind about testing.
I used to think tests suck, but it turned out that the way teams are writing tests is just plain garbage and there are actually better ways of doing it.
My team is extremely fast in delivering features now, we ship at least 2-3 updates PER DAY. We have 92% coverage.
And yeah we are using TDD. If tests are slowing you down rather than speeding you up you really need to question the quality of your code and tests.
Good code is easy to test.
Edit: this is from the point of view. of a backend developer. Frontend unit tests are bit less usefull.
I've also found that TDD forces me to write small and isolated things. If a test needs multiple DB entities and mocks I know something is wrong.
I've been working on an online store for a few years and am going back to add more tests. It's difficult in many places because there are large transaction scripts that touch every part of the system at once.
It's impossible to test all permutations of just the checkout page because it has a crap ton of businesses requirements and edge cases (around 10k sloc). I'll be clicking on buttons all day long to test everything just once lol
More junior dev here. My company produces an ecommerce site. They have no tech lead, no mentorship on my squad, and heavy testing, to the point where we are testing subfunction mocks => mock.setup().throws. assert throws(). And I must write my own unit tests. I am not happy, I am able to write maybe 5-10 lines of code each week. Between the extremely complicated structures, the unit tests, the integrated tests, and QA's testing suite, I spend more time waiting on feedback than I do thinking about and writing code.