Very informative video. Can you tell me something? -- In a data science project, for what phases I need to write test cases? Do I need to write test cases even during data collection, exploration, feature engineering, model training and hyper-parameter tuning phase or writing test cases just for the model deployment is sufficient?
I'm not a data scientist, but I would suggest writing tests when you want to be confident in your code, when your code will have a long life, or when you're working on the code with others.
@23:45 This isn't TDD. You still have a failing test case, you're not supposed to be talking about things like coupling or DIP (the specific technique you were referring to is actually called dependency *injection*, by the way) or refactoring right now. Once you get the test case to pass you get to refactor. The cycle is red-green-refactor. Your reward for getting the test to pass is being allowed to refactor. You do not get the reward until you have green tests.
For anyone unable to run pip install -e . at 14:00: You have to add py_modules=[] as a parameter in the setup.py file. Otherwise you get an error like "multiple top level packages discovered in a flat-layout". Hope it helps! And awesome video Wes
As much as I love VIM, I think you should be using an IDE. There are numerous typos throughout this video unrelated to text in strings, many of which would be automatically caught or autocompleted correctly if you used more modern tooling. Referring to variables or named parameters incorrectly, not closing the table body tag, etc. As well, at @1:08:00 if you actually ran the test this time, you would see the test give a false-positive, but for some reason you decided not to see a red test this time and just say you would? You only saw it red when you wrote the code and then realized the test was invalid? Why did you break the TDD approach mid-way through the video? Impatience? Difficult to be consistent with this approach? Thanks for the video overall, it was still a very informative look into TDD in practice.
Hi Wes! I've been looking for a more real world scenario example of TDD for a long time. This was extremely helpful for me. 😊 Looking forward to working through your other videos!
Hi Wes, I really like your explanations, great structure, relaxing tone of voice :) As a budding web developer, focussing mainly on Python/Django/React/JS/Bootstrap, I especially liked this video. Any chance of a follow-up, maybe testing Vanilla JS, React and/or Django? In any case, thanks for sharing, going through your channel currently, and looking forward to any future videos!
This is actually not the best way to learn TDD. If you want to learn TDD you should read Kent's Beck book titled "TDD by Example". That's the correct start.
Can I suggest you prefix variable with type i.e. obj = obj, int = integer, flo = float, str = string, lis, dic e.g. lis_dic_ents tells me ents is a list of dictionaries...
Hi Wes many thx for your video, just starting out with tdd and this helps a lot dont know much about nlp, but this is cool stuff now for whatever reason Madison is not recognized as a PERSON, it is a token though i have installed spacy 3.0.0 within 3.9.2 have an idea why they dont like Madison anymore? something is different with ents nevertheless i go with Madisona to follow along
Wow.. to not know what you do not know.. thanks mate. Watched thousands of YT tutorials in my lifetime and this is one of the top10 richest and clearest.
Amazing video. I am wondering though if displacy wasn't only added in to the running code, and so now 9/15 tests fail due to a missing "displacy" argument.
Hey First, thank you very much for this really good walking-through guide. One of the good ones I've seen. Secondly, a developer who knows his way around Vim is always appreciated
This is similar to my approach, and I'm not surprised you mention Clean Code. I like your vim setup, too. I would mention also the importance of clean TEST code. Copy-pasting (like you were doing in your tests) is always a good flag indicating necessary refactoring.
Hi Alan! Believe it or not, I don't entirely disagree with what you're getting at here. I probably don't need to tell you about the trade-offs. For some, TDD is "not worth it" in general. That said, "this video is over an hour long" because I'm trying to strike a balance between a useful pace of instruction and something approximating real world development. Secondly, an hour spent coding the tests along with this feature provide us with some assurance during the maintenance phase of the project. Speaking from my own experience, software engineers tend to vastly underestimate the length of time it takes to maintain production software (often even implicitly assuming it will be zero when estimating resource allocation for new feature development), so perhaps some of these tests will pay for themselves. Perhaps not, though. However, TDD or not, I'll take code with integration tests over code without integration tests any day when it comes to being assigned to the maintenance or extension of a codebase.
@ixternal9295 eh I've been a full time software engineer for a little over a year now, working on a codebase that's been around for close to a decade. I think starting from the test is helpful if you are adding a new feature and don't have a clear idea of what's needed to complete the feature. The test enforces what changes are necessary, but I usually save writing tests for last. If the test you write at the end of implementation is the same as the test you would begin with, then there's really no reason to write the test first.
At 28:40 minutes, spacy has parsed the sentence: "Madison is a city in Wisconsin" and labels Madison as a 'PERSON' even though the sentence identifies it as a city. So Natural Language Processing is not the same as natural language understanding.
Constantly changing your code .... plan ahead? Every 5 minutes I need 30 minutes to make something work. Course requirements: everything. This sucks. Know whose stuff I am not gonna watch on Udemy
What about if you will change your mind and not using spaCy but another nlp, then whole mock will be unusable, this cost can slow down prototyping especially if you not familiar with library that want to use.
Hi Mateusz, good point. We could create an intermediate object to wrap spaCy or any other base library, conforming the underlying implementations to a single interface, and use this in our NamedEntityClient.
@@WesDoyle I am struggling with unit test, I can understand that they are documenting business logic, but i still a lot of confusing for me, especially when going when it came to mocking something - this somehow have to be coupled with this what we trying to test. There is many points that can go wrong :( Maybe after your course I will change my mind ;)
![Tee Grizzley - Blow for Blow (feat. J. Cole) [Official Video]](http://i.ytimg.com/vi/wNLN5xsx5dc/mqdefault.jpg)
I'm too stupid to understand this lmao
It's so clear thanks a lot
Very informative video. Can you tell me something? -- In a data science project, for what phases I need to write test cases? Do I need to write test cases even during data collection, exploration, feature engineering, model training and hyper-parameter tuning phase or writing test cases just for the model deployment is sufficient?
I'm not a data scientist, but I would suggest writing tests when you want to be confident in your code, when your code will have a long life, or when you're working on the code with others.
@@grantcivyt Thanks
@23:45 This isn't TDD. You still have a failing test case, you're not supposed to be talking about things like coupling or DIP (the specific technique you were referring to is actually called dependency *injection*, by the way) or refactoring right now.
Once you get the test case to pass you get to refactor.
The cycle is red-green-refactor. Your reward for getting the test to pass is being allowed to refactor. You do not get the reward until you have green tests.
In the description it says you are using pytest which is what need to learn. Then I watch 25% of the video and see that you are using unittest?
For anyone unable to run pip install -e . at 14:00: You have to add py_modules=[] as a parameter in the setup.py file.
Otherwise you get an error like "multiple top level packages discovered in a flat-layout".
Hope it helps! And awesome video Wes
Thanks! This helped me.
thank you very much. This saved me a lot of time
thank you
Thanks, it helped and saved me time.
thanks a lot @Incertophile
As much as I love VIM, I think you should be using an IDE. There are numerous typos throughout this video unrelated to text in strings, many of which would be automatically caught or autocompleted correctly if you used more modern tooling. Referring to variables or named parameters incorrectly, not closing the table body tag, etc.
As well, at @1:08:00 if you actually ran the test this time, you would see the test give a false-positive, but for some reason you decided not to see a red test this time and just say you would? You only saw it red when you wrote the code and then realized the test was invalid? Why did you break the TDD approach mid-way through the video? Impatience? Difficult to be consistent with this approach?
Thanks for the video overall, it was still a very informative look into TDD in practice.
This course is amazing! thank you for putting this together
lol came here from your raodmap
this is probably one of the most valuable videos on the internet...
Wish this was dumbed down a little. Needed more explaining about all those defs and classes made.
thanks for the feedback, will try to make more intro level videos on python soon
Hi Wes! I've been looking for a more real world scenario example of TDD for a long time. This was extremely helpful for me. 😊 Looking forward to working through your other videos!
Glad it was helpful!
Hi Wes, I really like your explanations, great structure, relaxing tone of voice :)
As a budding web developer, focussing mainly on Python/Django/React/JS/Bootstrap, I especially liked this video. Any chance of a follow-up, maybe testing Vanilla JS, React and/or Django?
In any case, thanks for sharing, going through your channel currently, and looking forward to any future videos!
Anyone else also interested in his vim plugins and config? =) That's a full IDE!
dotfiles on my GitHub! Thanks for watching
Me tooo
Would you please let us know about that plug-in for VIM to execute pytest from within that you wanted to link in the description?
This is actually not the best way to learn TDD. If you want to learn TDD you should read Kent's Beck book titled "TDD by Example". That's the correct start.
Can I suggest you prefix variable with type i.e. obj = obj, int = integer, flo = float, str = string, lis, dic e.g. lis_dic_ents tells me ents is a list of dictionaries...
i don't want to sound negative but i think the goal of showing TDD is lost in the choice of overcomplicated scenario.
please what is the name of the editor you used in bulding the program in this tutorial?
Hi Wes many thx for your video, just starting out with tdd and this helps a lot
dont know much about nlp, but this is cool stuff
now for whatever reason Madison is not recognized as a PERSON, it is a token though
i have installed spacy 3.0.0 within 3.9.2
have an idea why they dont like Madison anymore?
something is different with ents
nevertheless i go with Madisona to follow along
same issue here
Anyone any clue why my flask tests take over a minute to simply assert the title of the app?
Ah, nice! Others have made videos on TDD but yeesh, they are boring (excluding Uncle Bob, of course...). Loved yours!
Interesting concept! I have not heard of it , looking forward to the series!
Thanks, David. Hope you enjoy it!
Why use virtual_env when venv comes with Python3?
An old habit I suppose! Good call 😄
@@WesDoyle Was just curious if you felt that something from the old was better than the new. Thanks for the response!
TBH venv is probably preferred, as it ships with Python, as you mentioned.
Very nice and clean code, thks keep them coming
Would love a deep-dive on your vim setup and workflow !
Wow.. to not know what you do not know.. thanks mate. Watched thousands of YT tutorials in my lifetime and this is one of the top10 richest and clearest.
What are the other 9, please
Really Nice Tutorial, Thank you!
RUclips algorithm = leetcode hard
It's awesome. Perfect explanation, voice and example. So glad that google suggest me this video.
Glad it was helpful! Appreciate it, Marek!
and subscribed. Thankyou for the content
Amazing video. I am wondering though if displacy wasn't only added in to the running code, and so now 9/15 tests fail due to a missing "displacy" argument.
The best video on tdd...
Great job on this video man - keep up the great work!
Hey
First, thank you very much for this really good walking-through guide. One of the good ones I've seen.
Secondly, a developer who knows his way around Vim is always appreciated
Thanks for this one bro, a good intro to TDD, it was my first time learning about this and I feel like I do understand everything you have covered.
can the dependencies like spacy, pytest and flask be installed using a python dependency management tool like poetry?
Very well done course, loved it. Thanks Wes for putting this together and sharing! Learned a ton following along.
Glad to hear it Tom, all the best
TDD is wonderful.
Awesome tutorial! Very helpful!
This is similar to my approach, and I'm not surprised you mention Clean Code. I like your vim setup, too. I would mention also the importance of clean TEST code. Copy-pasting (like you were doing in your tests) is always a good flag indicating necessary refactoring.
I need subtitles 😭
for the setup for windows it's : source env/Scripts/activate
The fact that this video is over an hour long and results in nothing like the app shown at the start is a testament to every argument against TDD
Hi Alan! Believe it or not, I don't entirely disagree with what you're getting at here. I probably don't need to tell you about the trade-offs. For some, TDD is "not worth it" in general. That said, "this video is over an hour long" because I'm trying to strike a balance between a useful pace of instruction and something approximating real world development. Secondly, an hour spent coding the tests along with this feature provide us with some assurance during the maintenance phase of the project. Speaking from my own experience, software engineers tend to vastly underestimate the length of time it takes to maintain production software (often even implicitly assuming it will be zero when estimating resource allocation for new feature development), so perhaps some of these tests will pay for themselves. Perhaps not, though. However, TDD or not, I'll take code with integration tests over code without integration tests any day when it comes to being assigned to the maintenance or extension of a codebase.
@@WesDoyle to be fair I ended up spending 2hrs to code along the video... just disappointed and annoyed at to find out the end product in the end
to be fair, TDD is aimed more at long-term maintainability and development speed in a mature codebase, not necessarily being faster in the short term
@ixternal9295 eh I've been a full time software engineer for a little over a year now, working on a codebase that's been around for close to a decade.
I think starting from the test is helpful if you are adding a new feature and don't have a clear idea of what's needed to complete the feature. The test enforces what changes are necessary, but I usually save writing tests for last.
If the test you write at the end of implementation is the same as the test you would begin with, then there's really no reason to write the test first.
thanks a lot
this is one of the great videos about TTD I've seen so far
def how_long_do_you_want_test_names_to_be():
return "Yes"
Good tutorial tho :)
Where's the vim plug in
tnx great job
Thanks for watching!
this guys build app with vim/neovim , love it
At 28:40 minutes, spacy has parsed the sentence: "Madison is a city in Wisconsin" and labels Madison as a 'PERSON' even though the sentence identifies it as a city. So Natural Language Processing is not the same as natural language understanding.
Constantly changing your code .... plan ahead? Every 5 minutes I need 30 minutes to make something work. Course requirements: everything. This sucks. Know whose stuff I am not gonna watch on Udemy
thanks for your feedback
What about if you will change your mind and not using spaCy but another nlp, then whole mock will be unusable, this cost can slow down prototyping especially if you not familiar with library that want to use.
Hi Mateusz, good point. We could create an intermediate object to wrap spaCy or any other base library, conforming the underlying implementations to a single interface, and use this in our NamedEntityClient.
@@WesDoyle I am struggling with unit test, I can understand that they are documenting business logic, but i still a lot of confusing for me, especially when going when it came to mocking something - this somehow have to be coupled with this what we trying to test. There is many points that can go wrong :( Maybe after your course I will change my mind ;)