Using modules, huh? You're really brave. Any of the testing units out there work for C++23 as well, I don't know what benefit you'd gain from specifically putting a lower bound as strict as that. To my knowledge, Boost.UT is on C++20 (and doesn't have macros, which is really nice to see). I suppose that's the closest thing you'd want.
@21:00 wait so is the falsifiable hypothesis that "our code has no bugs" or "our code has bugs"? the ppt says one thing and he says another Is the falsifiable hypothesis a statement we're trying to prove false? or prove true?
@@IsaacC20 the falsifiable hypothesis is "our code has no bugs". That's something actually disprovable (by finding a bug). " our code has bugs" is extremely difficult/impossible to disprove, as you'd have to prove that your code is correct
44:34 - the test for "empty can be filled" is written poorly: The CHECK-Macro is looking at the state of a boolean variable, so a failing test could only report "Expected true, got false" or something equally meaningless. this runs counter to the earlier talk about using Test-frameworks and their capabilities of concise errorstate descriptions. (And yes we all do code-reviews - some just do it AFTER deployment as i had to learn recently)
Ha! Yes -- good point. I was stuck between "keep the examples simple enough for beginners to read quickly", and "Make the point I'm trying to make". At the time CHECK( cup.fill() == true ) maybe struck me as slightly too opaque for the topic? But, that was also probably at about midnight the night before (putting this talk together was very challenging) so we can also put this down to just late-night last-minute tweaks. I'll definitely fix this the next time I present this talk. Thanks !
@davesteffen2410 Really good talk! It helped me a lot to structure the knowlege. I have one question regarding unit testing, that's based on my real life cases. I vividly remember two opposite cases: tests passed, production software failed and tests failed, production software worked. In both cases tests were correct and code was correct! It was very difficult to find the source of the issue and we couldn't find a way to run the tests correctly (at least for the failing tests case). The cause for both issues was timing: some messages were arriving in the wrong order because they were too slow/too fast. The tests have to impact the timing of the execution and we couldn't find any way around that. So, my question is: what to do, when the very presence of tests impacts the results?
I mean in short fix your tests. You've got some sort of data race if running tests changes the outcome. That's the classic debug print line that can't be removed just in reverse
Unit-testing is "new" ? it is old than nearly everybody developing C++ - it is certainly older than C++ is. But a big problem is that encapsulation and unit-testing don't mix well if your software doesn't have a good design that makes it testable. It sadly often is the case that say a frontend-API has implicit assumptions and tight coupling to the backend. The function that receives a user-request also is making calls to the Database. This can serve as a good indicator: If your code is hard to unit-test (or worse - testing in general) then your design is hard to reason about, hard to understand, hard to maintain or change.
Well.... "new" -- no. That book by Glenford Myers is old, and it's still my favorite! But, unit testing in the current modern meaning, is "new" in that it became widely known in the community in say the early or mid 2000's and wasn't widely recognized as a best practice until sometime after that (depending on where you were in the software community). I took a job in 2013 at a place that had never heard of the idea. 🙂 So some places are still catching up. Also 'new' in that we're still learning about it, and figuring out what the basics are. Part of my motivation for this talk was the fact that every talk, book, and blog post has a different list of "the most important things" (except all the blog posts that just repeat stuff from Hyrum's and Titus' talk!) .... which must mean that we don't all agree on what the foundations are. I think that's the sign of a still-immature field.
Not a joke. A very-oversold idea that has become dogma in too many places? Probably. Still a very important technique that solves problems _in certain uses_. Now -- "A joke the way some people do it"? I might agree :-)
it's an overkill added step with no real value so long as I thoroughly test a function and all it's parts extensively before hand... I have never needed docs at all, no tests, no demos.. a simple example is as good as all that, you won't eliminate bugs, you won't prevent bad launch or patches, you won't eliminate hotfixes and maybe 3 people might ever read that shit while you can explain it lightspeeds faster and better. code documentations and testing are for noobs, simple as that, I built an entire game engine without it.
And then you slightly rework the function 3 months later, or optimize one of the data structures it accesses, etc. Unit testing means you can actually touch your code without worrying about regressions flying under the radar. It tells you almost immediately after making changes whether they are breaking changes. That is invaluable.
I'm not following -- what is "it" that you think is overkill? Unit testing in general? I'm afraid I have to disagree. You personally might be able to succeed without them, on your personal project that only you work on. When you've got teams, or multiple teams working for years on a project, I don't think so. The overwhelming opinion of the community is that unit testing (or something very much like it) is absolutely necessary. My personal experience matches this, but you don't have to believe me. Talk to people from Google or Bloomberg or any large-scale software project. Heck, even small-scale projects that have real engineering (in the sense of 'long-term maintainability is important') requirements.
@@davesteffen2410 I guess it's the team aspect and the graphical aspect, graphics programs have visual feedback where as other software development has no other form of feedback... so it's a graphics programmer thing to not use them, as well as autodidact.. but at least I finally understand what they are, and what they're for lol
That's a recipe for disaster. I've been developing software professionally for 27 years, and most of our regressions can be traced back to insufficient unit testing. It's just not possible to maintain, e.g., 100k lines of code reliably without automated testing.
What an absolutely great talk. Perfect introduction on unit tests without getting stuck in the nitty gritty. Thank you!
Excellent talk, lot of very useful information regarding software testing philosphies
good to know that "#define private public" is undefined behavior, I have used this a lot in the past without any issues 😀
One of these years they'll do it somewhere else and forget to take out Bjarne talking about the rockies.
What framework should I use for > C++23 and using modules?
Using modules, huh? You're really brave.
Any of the testing units out there work for C++23 as well, I don't know what benefit you'd gain from specifically putting a lower bound as strict as that.
To my knowledge, Boost.UT is on C++20 (and doesn't have macros, which is really nice to see). I suppose that's the closest thing you'd want.
@21:00 wait so is the falsifiable hypothesis that "our code has no bugs" or "our code has bugs"? the ppt says one thing and he says another
Is the falsifiable hypothesis a statement we're trying to prove false? or prove true?
@@IsaacC20 the falsifiable hypothesis is "our code has no bugs". That's something actually disprovable (by finding a bug). " our code has bugs" is extremely difficult/impossible to disprove, as you'd have to prove that your code is correct
Intro ends at 7:00 ish
44:34 - the test for "empty can be filled" is written poorly: The CHECK-Macro is looking at the state of a boolean variable, so a failing test could only report "Expected true, got false" or something equally meaningless. this runs counter to the earlier talk about using Test-frameworks and their capabilities of concise errorstate descriptions.
(And yes we all do code-reviews - some just do it AFTER deployment as i had to learn recently)
Ha! Yes -- good point. I was stuck between "keep the examples simple enough for beginners to read quickly", and "Make the point I'm trying to make". At the time CHECK( cup.fill() == true ) maybe struck me as slightly too opaque for the topic? But, that was also probably at about midnight the night before (putting this talk together was very challenging) so we can also put this down to just late-night last-minute tweaks.
I'll definitely fix this the next time I present this talk. Thanks !
@davesteffen2410
Really good talk! It helped me a lot to structure the knowlege. I have one question regarding unit testing, that's based on my real life cases. I vividly remember two opposite cases: tests passed, production software failed and tests failed, production software worked. In both cases tests were correct and code was correct! It was very difficult to find the source of the issue and we couldn't find a way to run the tests correctly (at least for the failing tests case). The cause for both issues was timing: some messages were arriving in the wrong order because they were too slow/too fast. The tests have to impact the timing of the execution and we couldn't find any way around that. So, my question is: what to do, when the very presence of tests impacts the results?
I mean in short fix your tests. You've got some sort of data race if running tests changes the outcome. That's the classic debug print line that can't be removed just in reverse
Unit-testing is "new" ? it is old than nearly everybody developing C++ - it is certainly older than C++ is.
But a big problem is that encapsulation and unit-testing don't mix well if your software doesn't have a good design that makes it testable. It sadly often is the case that say a frontend-API has implicit assumptions and tight coupling to the backend. The function that receives a user-request also is making calls to the Database.
This can serve as a good indicator: If your code is hard to unit-test (or worse - testing in general) then your design is hard to reason about, hard to understand, hard to maintain or change.
Well.... "new" -- no. That book by Glenford Myers is old, and it's still my favorite! But, unit testing in the current modern meaning, is "new" in that it became widely known in the community in say the early or mid 2000's and wasn't widely recognized as a best practice until sometime after that (depending on where you were in the software community). I took a job in 2013 at a place that had never heard of the idea. 🙂 So some places are still catching up.
Also 'new' in that we're still learning about it, and figuring out what the basics are. Part of my motivation for this talk was the fact that every talk, book, and blog post has a different list of "the most important things" (except all the blog posts that just repeat stuff from Hyrum's and Titus' talk!) .... which must mean that we don't all agree on what the foundations are. I think that's the sign of a still-immature field.
is OOP a joke?
Yes
Not a joke. A very-oversold idea that has become dogma in too many places? Probably. Still a very important technique that solves problems _in certain uses_.
Now -- "A joke the way some people do it"? I might agree :-)
@@davesteffen2410 Let me clarify. We encapsulate stuff in a class, so testing the stuff is instantly more complex.
@@possumcodeNot really.
@@possumcode how is it more complex exactly? Give concrete examples (ideally not using any LLMs in the process)
it's an overkill added step with no real value so long as I thoroughly test a function and all it's parts extensively before hand... I have never needed docs at all, no tests, no demos.. a simple example is as good as all that, you won't eliminate bugs, you won't prevent bad launch or patches, you won't eliminate hotfixes and maybe 3 people might ever read that shit while you can explain it lightspeeds faster and better. code documentations and testing are for noobs, simple as that, I built an entire game engine without it.
This will get the code working the first time, but do you have to retest everything every time you refactor?
And then you slightly rework the function 3 months later, or optimize one of the data structures it accesses, etc.
Unit testing means you can actually touch your code without worrying about regressions flying under the radar.
It tells you almost immediately after making changes whether they are breaking changes.
That is invaluable.
I'm not following -- what is "it" that you think is overkill? Unit testing in general? I'm afraid I have to disagree. You personally might be able to succeed without them, on your personal project that only you work on. When you've got teams, or multiple teams working for years on a project, I don't think so. The overwhelming opinion of the community is that unit testing (or something very much like it) is absolutely necessary.
My personal experience matches this, but you don't have to believe me. Talk to people from Google or Bloomberg or any large-scale software project. Heck, even small-scale projects that have real engineering (in the sense of 'long-term maintainability is important') requirements.
@@davesteffen2410 I guess it's the team aspect and the graphical aspect, graphics programs have visual feedback where as other software development has no other form of feedback... so it's a graphics programmer thing to not use them, as well as autodidact.. but at least I finally understand what they are, and what they're for lol
That's a recipe for disaster. I've been developing software professionally for 27 years, and most of our regressions can be traced back to insufficient unit testing. It's just not possible to maintain, e.g., 100k lines of code reliably without automated testing.