If in theory, the difference between theory and practice is less in theory than in practice; but in practice, the difference between theory and practice is greater in practice than in theory; then in theory, is the difference between theory and practice actually greater in theory than in practice?
Our precommit hooks include linting(we have developers on Windows, Linux, Mac, VsCode, PyCharm, with and without autosave, so to have the linting enforced and the same is essential), checks that you are not commiting to the main branch, checks that you are not editing commited migration files. I think it is only a benefit.
I know it’s not always possible but I would recommend merging on the same editor or at least same local formatter (so probably Prettier with a common config, so that users can write however they want but the result will be consistent)
We use husky to prettier and lint things. It just auto-formats things before you do your PR. It runs fast so you won't have to worry. For commiting to main branch would making it protected help? because it would stop you from pushing to it right?
Those tests should be run on a server managing the repository, not on the devs' local machines, especially because hooks can be skipped. especially committing to the main branch should be something that is handled by your repository's permissions, not a hook. Do you really want to stop devs from making a backup of their work on a feature branch at the end of their work day because of a linter noncompliance?
@@StefanErwinBaumer prettier wouldn’t ever block you from committing unless what you committed was not valid code/syntax, then it would probably be valid to stop the commit Doing so for linting is a separate issue
TISAX certification (a subset of ISO 27001 for automotive industry) requires that your development process has staging environment. If you told to the auditor that you only test in prod, he would close the notebook and say "See you in six months".
Re pre-commit hooks: Theos opinion is only valid for J/TS development on this one, because in python and Go, the relevant formatters are basically never blockers (and pre-commit always canceling the commit on files changed is better behavior than husky silently changing files)
0:40 "randomly click around" is not a bad idea (maybe just do it before prod). I've seen projects write hordes of tests just to produce a 500 on first use which they missed because they never used the thing they were building. You'll learn a lot about the thing you're building by using it.
One thing that wasnt covered is DATA. Often the application code depends on assumptions about the data structure beyond the raw db schema. Yes, you could use zod but what to do about legacy data that doesnt fit the current model?
I often told people at my university that production was the greatest test environment I was aware of. Rarely do you get a second chance to make a first impression, and testing in production solves none of these things, but boy is it fun!
About pre-commit hooks: I agree. However, pre-push hooks are perfectly fine. You don't push anywhere as often, and the only time where it matters what you've done with code is when you share it
Good points. Disagree that shouldn't have a staging server. It can catch a lot of deployment issues. I also believe in a simple smoke test that hits all the integration points (login, send email, charge CC, upload/download with S3, other external rest APIs). It just makes sure integrations aren't broken, it doesn't test logic. This further makes the staging environment important. It might just be a /health page. You can also run the smoke test on prod immediately after deployment.
I'm honestly not sure how their "Test on a server with a copy of the production data" _isn't_ just a staging or testing server. In fact, none of the things they list seem like 'testing in production' to me, but instead a combination of staging/testing servers with old copies of production data, combined with standard monitoring. I guess A/B testing, but that's less about testing code and more about testing if users like a change or not. In other words, it's not the code you're testing, it's the users.
Fun fact, there is a hidden setting on discord that the developers use to test new features, and variations on new features. If you find a way to activate it (they sometimes patch the ways people activate it) you can try out new features early. (It's called "experiments" btw)
re pre-commit hooks... it's sooo easy to make a git alias that appends `--no-verify` for a convenient opt out of pre-commit hooks when developing WIP. On the other hand, having that local check before waiting for a bunch of CI to start running can save hours over the course of a day. The important thing is that the pre-commit hooks should take less than 10 seconds - ideally 1 or 2 seconds Here's a bunch for anyone interested: cn = commit --no-verify cmn = commit -m --no-verify can = commit -a --no-verify camn = commit -am --no-verify cian = commit --amend -C HEAD --no-verify psn = push --no-verify pfn = push --force-with-lease --no-verify pffn = push --no-verify
What's wrong with staging just being a snapshot of production? All the benefits of this approach, none of the "oh fuck I've deleted a user's entire account" Edit: 2:38 I really wouldn't call that testing in prod though. But maybe that's bc it's the only thing I have experience with
_"Let me know in my Pull Request that my code is wrong, don't prevent me from committing because you don't like the way I did my code"_ --Theo 2024 I support this wholeheartedly, I've been championing this idea for years.
When it comes to pre commit hooks, what about something like biome that lints and formats in the less than a sec? Or using husky to do lint/format (w/ Biome or eslint+prettier) as a pre-push hook? This way you can run your linting/formatting locally which will in theory run faster than your CI before you push and you can commit as much as you want, without slowdown. Also this way you reduce needing to rewrite your local git history for a formatting/linting change and force push to remote.
A dancing crew should train a lot and make 'test runs' before giving the final performance. Testing - like repetition to become 'almost perfect' - is essential to give a good performance to the people. If you are creating stuff and do not test it, it will break and will freak out the users, so IRL you should become a Sir or Lady 'TestALot', so your users do not experience any incovenient bugs with your product.
Everyone is testing in production, some of us just have a good fortune to have a separate system that approximates some parameters of production in addition to production. Whoever imagines they're not testing in production either has more services than customers or is fooling themselves. Seriously: Software in production will invariably eventually fail, not accounting for it and prepared beforehand for it is asking for trouble.
I once had to test a UI update directly in prod... that was fun. (sarcasm) I was tasked with fixing a huge bug with the UI, and I could only test it in production (because of some data, etc, that was missing from the test environments... we could technically have done it in a test environment, but that would require setting up a lot of stuff that we didn't have time to set up), so I attempted to fix the bug, and I tested that the code should in work in theory, but I had to test the actual UI by pushing it to production. After debugging some bugs in the framework we were using, I finally got it working after a few tries.
Would people risk testing in production for a massive Internet Banking monolith, where a "small fail" costs a minimum 6 figure in fines by the government and various regulatory entities?
I cannot express in words how much I detest Husky. The first thing I do when opening a new project is to turn that crap off. It is literally the worst package in the history of programming.
Only fools test in production, everyone else mirrors into pre-prod/uat or startups. It's true however, that certain validations cannot be performed in lower env. But it's not the same as "testing". Production validation should only involve making sure components are deployed and configured.
It really depends on what prod does and what the impact of the change is. I've been a system administrator for companies that are manufacturing. You don't want to ever risk fucking up the assembly lines. But our BW process? Fuck that, they can wait a day or two if push comes to shove, test in prod.
@@gintoki_sakata__ Everyone all acts like they are the tech lead/CTO/founder of something who scales the architecture from hundred to million, or a team of several to hundred. Speaking from real anecdotes: 99% are just random dude joining a small/big team by luck where everything already had setup for them. All they did is just creating a new account, or adding cli commends (by copy/paste the existing example as guidance then twist around) for new features. But they all act like they setup the whole infra pipeline. Even for building a new team, most just don't have an idea how to manage people and possibly scale the number back after a year of money burning, and obviously big layoff. Larger companies just appears to have more free money to burn, longer time to figure out all the team problems.
![Blox Fruits Dragon Rework Update [Full Stream]](http://i.ytimg.com/vi/EqDAp8udhm0/mqdefault.jpg)
These thumbnails are way better than those fake tweet ones
Agreed, the fake tweets work but this feels way more legitimate
I skip the videos with fake tweet thumbnails
@@zyzlolme too
crybabies, here they come 👶
Theo is the reason I got an extension to change thumbnails on RUclips. He has good videos but the thumbnails were that bad
Just test in production so your users can be the QA team, save costs!
Works for Microsoft and others
works for many AAA games
Shift left.
To quote zack freedman: in practice, the difference between theory and practice is greater in practice than in theory
The difference between theory and practice is that theory does not work in practice
If in theory, the difference between theory and practice is less in theory than in practice; but in practice, the difference between theory and practice is greater in practice than in theory; then in theory, is the difference between theory and practice actually greater in theory than in practice?
They test in prod because of real datasets
I test un prod because I don't know what i'm doing
We're not the same
You guys are testing? I just hit deploy and grab a new ticket
True :D
Every app has a Test environment. Some of them also have a separate Production environment.
time to push to prod on save
Best strat ever!
Our precommit hooks include linting(we have developers on Windows, Linux, Mac, VsCode, PyCharm, with and without autosave, so to have the linting enforced and the same is essential), checks that you are not commiting to the main branch, checks that you are not editing commited migration files. I think it is only a benefit.
I know it’s not always possible but I would recommend merging on the same editor or at least same local formatter (so probably Prettier with a common config, so that users can write however they want but the result will be consistent)
We use husky to prettier and lint things. It just auto-formats things before you do your PR. It runs fast so you won't have to worry.
For commiting to main branch would making it protected help? because it would stop you from pushing to it right?
Those tests should be run on a server managing the repository, not on the devs' local machines, especially because hooks can be skipped. especially committing to the main branch should be something that is handled by your repository's permissions, not a hook.
Do you really want to stop devs from making a backup of their work on a feature branch at the end of their work day because of a linter noncompliance?
@@StefanErwinBaumer prettier wouldn’t ever block you from committing unless what you committed was not valid code/syntax, then it would probably be valid to stop the commit
Doing so for linting is a separate issue
@@StefanErwinBaumerJust look at the warnings sometimes. Don't force others to.
TISAX certification (a subset of ISO 27001 for automotive industry) requires that your development process has staging environment. If you told to the auditor that you only test in prod, he would close the notebook and say "See you in six months".
Re pre-commit hooks:
Theos opinion is only valid for J/TS development on this one, because in python and Go, the relevant formatters are basically never blockers (and pre-commit always canceling the commit on files changed is better behavior than husky silently changing files)
That quote about the difference between theory and practice is a paraphrase of Yogi Berra.
We have a test server just for this, which we push to before prod. Helped us avoid a nasty bug at least once.
0:40 "randomly click around" is not a bad idea (maybe just do it before prod). I've seen projects write hordes of tests just to produce a 500 on first use which they missed because they never used the thing they were building. You'll learn a lot about the thing you're building by using it.
Incremental failure is a beautiful thing in prod. Way better than catastrophic failure in prod which happens with overconfidence.
One thing that wasnt covered is DATA. Often the application code depends on assumptions about the data structure beyond the raw db schema. Yes, you could use zod but what to do about legacy data that doesnt fit the current model?
I often told people at my university that production was the greatest test environment I was aware of. Rarely do you get a second chance to make a first impression, and testing in production solves none of these things, but boy is it fun!
About pre-commit hooks: I agree. However, pre-push hooks are perfectly fine. You don't push anywhere as often, and the only time where it matters what you've done with code is when you share it
Good points. Disagree that shouldn't have a staging server. It can catch a lot of deployment issues.
I also believe in a simple smoke test that hits all the integration points (login, send email, charge CC, upload/download with S3, other external rest APIs). It just makes sure integrations aren't broken, it doesn't test logic. This further makes the staging environment important. It might just be a /health page. You can also run the smoke test on prod immediately after deployment.
I'm honestly not sure how their "Test on a server with a copy of the production data" _isn't_ just a staging or testing server. In fact, none of the things they list seem like 'testing in production' to me, but instead a combination of staging/testing servers with old copies of production data, combined with standard monitoring.
I guess A/B testing, but that's less about testing code and more about testing if users like a change or not. In other words, it's not the code you're testing, it's the users.
Fun fact, there is a hidden setting on discord that the developers use to test new features, and variations on new features.
If you find a way to activate it (they sometimes patch the ways people activate it) you can try out new features early.
(It's called "experiments" btw)
re pre-commit hooks... it's sooo easy to make a git alias that appends `--no-verify` for a convenient opt out of pre-commit hooks when developing WIP. On the other hand, having that local check before waiting for a bunch of CI to start running can save hours over the course of a day. The important thing is that the pre-commit hooks should take less than 10 seconds - ideally 1 or 2 seconds
Here's a bunch for anyone interested:
cn = commit --no-verify
cmn = commit -m --no-verify
can = commit -a --no-verify
camn = commit -am --no-verify
cian = commit --amend -C HEAD --no-verify
psn = push --no-verify
pfn = push --force-with-lease --no-verify
pffn = push --no-verify
How do you make sure that we’re not leaking feature in production and feature flag works as expected? 😊
Pushing to prod? Theo looks like he's pushing out the biggest log know to man in the thumbnail
What's wrong with staging just being a snapshot of production?
All the benefits of this approach, none of the "oh fuck I've deleted a user's entire account"
Edit: 2:38
I really wouldn't call that testing in prod though. But maybe that's bc it's the only thing I have experience with
14:40 I see apple lawyers finally forced ikea to drop the i.
_"Let me know in my Pull Request that my code is wrong, don't prevent me from committing because you don't like the way I did my code"_ --Theo 2024
I support this wholeheartedly, I've been championing this idea for years.
When it comes to pre commit hooks, what about something like biome that lints and formats in the less than a sec? Or using husky to do lint/format (w/ Biome or eslint+prettier) as a pre-push hook? This way you can run your linting/formatting locally which will in theory run faster than your CI before you push and you can commit as much as you want, without slowdown. Also this way you reduce needing to rewrite your local git history for a formatting/linting change and force push to remote.
A dancing crew should train a lot and make 'test runs' before giving the final performance.
Testing - like repetition to become 'almost perfect' - is essential to give a good performance to the people.
If you are creating stuff and do not test it, it will break and will freak out the users, so IRL you should become a Sir or Lady 'TestALot', so your users do not experience any incovenient bugs with your product.
Staging? What’s that? Real men test in prod
Crowdstrike Falcon engineering team likes this.
Hah. How about a bit of both? Side project only has a single prod database, but I have local dev and prod envs for the front end code... 😅
You didn't test the peace nerds bit in the end on your dev environment
Cool, except for the paste near the Titanic, but that's what waivers are for.
Where else would you test code other than your own machine?
Everyone is testing in production, some of us just have a good fortune to have a separate system that approximates some parameters of production in addition to production. Whoever imagines they're not testing in production either has more services than customers or is fooling themselves.
Seriously: Software in production will invariably eventually fail, not accounting for it and prepared beforehand for it is asking for trouble.
I once had to test a UI update directly in prod...
that was fun.
(sarcasm)
I was tasked with fixing a huge bug with the UI, and I could only test it in production (because of some data, etc, that was missing from the test environments... we could technically have done it in a test environment, but that would require setting up a lot of stuff that we didn't have time to set up), so I attempted to fix the bug, and I tested that the code should in work in theory, but I had to test the actual UI by pushing it to production. After debugging some bugs in the framework we were using, I finally got it working after a few tries.
Oh, no! It is zombies :) They broke the internet and Theo couldn't say 'Peace, Nerds' at the end of the video!
Would people risk testing in production for a massive Internet Banking monolith, where a "small fail" costs a minimum 6 figure in fines by the government and various regulatory entities?
As a php dev I edit the production code in notepad
Use the word "Validate" when it comes to "Producton." Then the CAB and CTO are onboard ;😉
Like the video before watching :)
Becaise od thumbnail! Like it!
As a joke goes 'The real men rest in production' :D
Theo to WWE, confirmed.
Size doesn't matter good take lmfao
We have to use mock data because of HIPPA compliancy.
Please please please learn about git commit --no-verify
I cannot express in words how much I detest Husky. The first thing I do when opening a new project is to turn that crap off.
It is literally the worst package in the history of programming.
Only fools test in production, everyone else mirrors into pre-prod/uat or startups. It's true however, that certain validations cannot be performed in lower env. But it's not the same as "testing". Production validation should only involve making sure components are deployed and configured.
Best case everything works.
Worst case you get fired 💀
post hog? okay
i fking h8 husky too
That thumbnail 😬
never wrote a test, my code is perfect from init
Sad cuz you didn’t say “peace nerds” :,(
for size of business maybe think binance
It really depends on what prod does and what the impact of the change is. I've been a system administrator for companies that are manufacturing. You don't want to ever risk fucking up the assembly lines. But our BW process? Fuck that, they can wait a day or two if push comes to shove, test in prod.
Hate precommit hooks as well! They‘re annoying. And most of them can be replicated in a better in CI
But then you get incremental lint/bug fixing commits, which might upset people who want a "clean" history.
I‘d rather have the CI just inform me about issues and abort the build. It‘s my duty to fix those issues
hell yea
Tell me you have a small team and a product no one cares about.
We all start somewhere
@@gintoki_sakata__ Everyone all acts like they are the tech lead/CTO/founder of something who scales the architecture from hundred to million, or a team of several to hundred.
Speaking from real anecdotes: 99% are just random dude joining a small/big team by luck where everything already had setup for them. All they did is just creating a new account, or adding cli commends (by copy/paste the existing example as guidance then twist around) for new features. But they all act like they setup the whole infra pipeline.
Even for building a new team, most just don't have an idea how to manage people and possibly scale the number back after a year of money burning, and obviously big layoff. Larger companies just appears to have more free money to burn, longer time to figure out all the team problems.
I'm too early for this 😅