I have being exposed to python for years but i feel i have never used it and learned it in a structured and coherent way. I consider myself beginner to intermediate user. which of your course do you suggest for me to start learning things and become a better developer not just being a coder?
I like the idea of making the code fairly self-documenting by making good choices of class/function/var names. And adding comments that explain WHY things are being done, not how. My merge requests are reviewed by someone else, so it's a fun challenge to make sure everything is clear enough that the reviewer won't ask me any annoying questions. And yes, I want to make sure then when I come back in a year or two, it's easy for me to remember what the heck I was doing in the code...
I want to add to these remarks that I feel grouping code that does work on the same aspect of your system is also very important for understanding and maintaing your code base. If you have a class that has many hundreds or even thousands of lines of code, identify all the different things it does, re-organize the code to group together functions that have the same context, and then refactor that out into different modules/classes. That makes your code base much easier to understand, search through and adapt.
Summary: - Code should be written in a way that promotes readability and is easy to understand for other developers. - Good documentation can take different forms such as comments, documentation sites, and API documentation. - Comments should provide meaningful explanations and not simply describe the syntax of the code. - Using type hints in Python code improves clarity and understanding. - Docstrings can be used to document functions, methods, classes, and modules, and tools like Auto docstring can assist in generating them.
Thank you Arjan this is pretty good tutorial I like you always do quick reasoning about motivation behind each documentation fragment. My general rule of thumb for `comments`. Comment has to explain "WHY" the code is here, not "WHAT" this part of code doing. If I have to write WHAT the code does, it means I messed up naming or code structure.
Very good points. Whenever I have just spent an hour figuring out how something works, I fervently wish that the original author had taken three minutes to document that instead. _Especially_ when that original author was me. 😏
Definitely leaning more towards self documenting myself after trying out full docstrings and then autodoc with sphinx. Navigating the code is an important part of readability
You know its going to be a good day when you open a module and the line counter stretches to 6 digits, adds a horizontal scrollbar on your widescreen, and there is only one docstring at the top. '''Abandon all hopes ye who enters here.'''
I like more verbose docstrings on functions and methods because they appear in the mouseover in the IDE. That allows me to more easily see the parameters etc. without having to open the file containing the method or function.
Great video as always. I might have added unit tests here as a form of documentation. Unit tests can also have clear names, and their docstrings can comment on how edge cases are handled. If I'm coming back to code I wrote six months ago, it's the first place I go.
I really, really appreciate the production values of your videos. Always love seeing your code on screen. It’s crisp, lol. Keep it up 💪🏾 Regarding the topic, I love it. I’m really trying to set some standards for documentation on our team, this is some good stuff
wow. you make VS look so useful. It seems that if you use the docstring helper, you will end up with soem really professional looking docstrings. It makes other people think that you really pay attention to details and make very beautiful looking code. And by using VS, you don't have to expend mental bandwidth trying to figure out what to format a docstring. Nice. You make a good case for VS. If I were a developer, I would def see that VS is a high on the list of preferred ide
Thanks for the video, it gives some great tips! I'd like to comment on some aspects (being a technical writer _and_ a developer) and would also like to share some of my experiences what I made from both roles: * Docstrings are awesome Yes, sometimes docstrings take more space than the actual documentation. If this is a public function or method, that shouldn't prevent us to write it. If the doc is autogenerated (like an API), the doc should be complete. * Focus on your target group(s). If your doc is autogenerated, it is probably more focused on developers. But depending on your project, developer docs are maybe not enough. Also take users and tutorials in mind. * Autogenerated API documentation isn't a replacement for user docs. It looks easy and it's tempting to get the impression, "I documented all functions and modules, so I have all documentation." This is the wrong approach. Depending on what you write in your docstring, a pure autogenerated API documentation listing only the functions and parameters is pretty useless. It doesn't tell you _how to use_ the project. We need flesh to the bones: some examples and context.
Man, I wish I knew you when I started programming. Currently I work in a Brazilian Company that provides R&D services on Robotics and Computer Vision globe wide. We're great researchers, but terrible on writing quality code. I'll force everyone in the company to watch you!
Regarding the docstrings I think it really depends on the use case, if you are writing a code for a project like an API for an app frontend or a data pipeline or some other type of product then it doesn't really make sense to write a docstrings because as you said the main goal should be for the code to be as clean and structured as possible and that should be enough, because adding docstrings basically just duplicates the stuff you already do in natural language and in case of bugs it can lead to even harder debugging because you could potentially inspect the docstring and move on. However the use case where it is absolutely necessary is for example writing libraries, the user of the library has to know what is going on with the function and all the arugments passed to it and it doesn't really make sense for him to go in the implementation rather he will read the method docstring and that's it.
Another suggestion for making your code more readable is adding units of measurement to the variable names. In this example, it would be nice if the parameter to the deposit and withdraw function would be called amount_in_cents or amount_in_euros etc.
when I try to hint that my output is generator, I get an error saying generator is undefined. I have to from typing import Generator as generator. Only then, can I hint def fx() -> generator:
I only make fully filled google style docstrings for my packages of the classes and functions that the package user is interacting with, and which I have documented using mkdocs. All other code will have a simple typehint and a short summary in the doc string.
I think that Sphinx could also be a great way to improve our documentation. I suggest to include this tool maybe in a part 2 or in an extension of this video. Thanks so much for making so interesting and complementary topics for the Python Community! 🐍 💻
The industry shift towards 'self-documenting code', where the naming and coding style itself explains the goal of the code, has been a good thing (IMHO). There have been times where some 10-15 year old code works one way, but the comments never got updated to agree. Some maintainer along the way just couldn't be bothered to 'fix the comments'. And not to sound TOO MUCH like some old 'boomer', but yes, there is a ton of code out there that has to be maintained that is probably older than half of all comp-sci degrees awarded. lol A few times when good comments really help is when you find yourself employing some 'trick'. Some coders like to keep some 'trick' all secretive (like maybe 'bit-twidling' some float to extract a fraction or shift lower-case ASCII to upper). They get a bit smug and make statements like, "Well if you were a good programmer, you wouldn't need a comment to understand this." But adding a tiny comment to explain some obscure detail is the mark of a really good coder.
A master coder -- heck, a master of _anything_ -- is not afraid to share knowledge with others because they know that to use that knowledge properly requires experience. Coders that scoff at others because others don't understand are not master coders; they are snobbish know-it-alls who are totally insecure.
About docstrings taking up too much space: It really depends on the style used, e.g. a section in numpy style takes at minimum 4 lines (title, -----, name, description) while in google style it's collapsed down to only two ("title:", "name: description")
I have two problems with typehints. First, imagine I have a class defined in a file and in another file I have a function that takes as a parameter an instance of that class. To use typehints I import the class in the function file and because I need the function in the class file I also import the function file in the class. This is an obvious problem because I do circular imports. They way of solving this is putting the function inside the class file but I dont know if this is a good practice or not (I mean it should be the good practice if you can't use typehints without an import, that's really my question, can you use type hints without an import?) The other problem is with typehints and numpy, I also like to have an strict type checker but with numpy I don't know how to use it, sometimes I have used the ArrayLike type but the type checker goes nuts.
I could be wrong, but I think if you surround the typehint in quotes ("), then it forward declares it and you don't need the import, e.g. -> "ObjectFromOtherFile". Also for type hinting numpy arrays you can use np.ndarray
If there's a function that your class needs, but the function needs an instance of your class, why not put them in the same module? They are already inseparably coupled. Putting them in the same module is just the most logical thing to do. ETA: If there's another module that needs them both, it saves an import as well.
Sometimes thise comments that explains simple code are helpful. Because they're plain English (or your language) you can quickly get the gist of what's happening as if your reading a book. I will then look at the code if I want further details.
@@edgeeffect No. Stop this sheepish nonsense. Perfect readable code doesn't exist. Go look at any of the Rust, Python, C++, C#, etc. source code. Sometimes your code is doing something complex and you dont need indentation, syntax, and error handling to get in the way. You just want to get to the meat and bones pretty quickly. Also you don't need to rewrite everything. Just been you don't understand it doesn't mean nobody understands it.
Sadly employees feel punished for writing documentation that is too good, because it becomes an employers guide for replacing you with a lower paid person.
If you do great work to a level that you are replaceable at any moment, you are irreplaceably valuable. Any project, any initiative, any codebase will want to have you
You are right, I do feel punished for having my PR fail because I didn't write a docstring on an init method with 2-3 vars that are fully annotated. Write simple, explanatory code and document what's needed, comments are going to get dated anyway and become more confusing to users. That's just reality.
Based on personal experience being the only one who knows the details of an "in house developed" software is no guarantee against layoffs, but it will make sure that you will have trouble going on a hokiday without being disturbed.
@OP If you are in an environment like that, and don‘t get fired for not enabling knowledge transfer, you are part of the problem and did not find a solution. Because you don’t get promoted or even fired since the next guy to you is doing the same thing. Try to set an example, if it doesn’t work get out of these places or be fine in being caught in a vicious cycle of despair reinforcing your own pain.
When giving a typehint for an argument of the same type as the self class, instead of Self one can use "": "BankAccount" in this case. This way you don't need to include anything.
I've tried to generate the documentation of my django project with mkdocstrings. After testing a few things and exploring further, I managed to get the yml file working with django. Then I installed with the python-legacy handler ... and It more or less works ... until ModelForms are discovered ... and then it crashes ! When I'm trying the same with the new python handler whcich does not execute the code ... I end up with empty documentation pages (official documentation say that the experimental new handler is not yet compatible with Django) ... Could anybody tell me how to generate docstring documentation on a Django app? ... or how could I bypass specific functions, classes or modules ?
I have the issue that certain shortcuts in vs code do not work (I assume the issue is that a shortcut is used multiple times by various extensions so vs code does not know what to do) Can you show how to handle this in a video? br
I've used to put docstrings on everything that was more than few lines long but then I've started applying clean code concepts and my code is self documented, I only ever use the single line # comments nowadays.
I started programming in the 80’s. I did a lot of code archeology. The running joke about documentation was if the code was hard to write it should be hard to read
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
I have being exposed to python for years but i feel i have never used it and learned it in a structured and coherent way. I consider myself beginner to intermediate user. which of your course do you suggest for me to start learning things and become a better developer not just being a coder?
I like the idea of making the code fairly self-documenting by making good choices of class/function/var names. And adding comments that explain WHY things are being done, not how. My merge requests are reviewed by someone else, so it's a fun challenge to make sure everything is clear enough that the reviewer won't ask me any annoying questions. And yes, I want to make sure then when I come back in a year or two, it's easy for me to remember what the heck I was doing in the code...
Yes the why is what matters. And when that explanation becomes too hard to understand it’s time to refactor :)
I want to add to these remarks that I feel grouping code that does work on the same aspect of your system is also very important for understanding and maintaing your code base. If you have a class that has many hundreds or even thousands of lines of code, identify all the different things it does, re-organize the code to group together functions that have the same context, and then refactor that out into different modules/classes.
That makes your code base much easier to understand, search through and adapt.
Your voice is so calming I'm basically using this video tutorial as a background music while coding and documenting. Great video and tutorial!
Awesome! Thank you!
Summary:
- Code should be written in a way that promotes readability and is easy to understand for other developers.
- Good documentation can take different forms such as comments, documentation sites, and API documentation.
- Comments should provide meaningful explanations and not simply describe the syntax of the code.
- Using type hints in Python code improves clarity and understanding.
- Docstrings can be used to document functions, methods, classes, and modules, and tools like Auto docstring can assist in generating them.
Thank you Arjan this is pretty good tutorial I like you always do quick reasoning about motivation behind each documentation fragment.
My general rule of thumb for `comments`. Comment has to explain "WHY" the code is here, not "WHAT" this part of code doing.
If I have to write WHAT the code does, it means I messed up naming or code structure.
Very good points.
Whenever I have just spent an hour figuring out how something works, I fervently wish that the original author had taken three minutes to document that instead.
_Especially_ when that original author was me. 😏
Definitely leaning more towards self documenting myself after trying out full docstrings and then autodoc with sphinx. Navigating the code is an important part of readability
Arjan, this video is great! You are the guy! Thank you!
Thank you, Mario!
You know its going to be a good day when you open a module and the line counter stretches to 6 digits, adds a horizontal scrollbar on your widescreen, and there is only one docstring at the top. '''Abandon all hopes ye who enters here.'''
I like more verbose docstrings on functions and methods because they appear in the mouseover in the IDE. That allows me to more easily see the parameters etc. without having to open the file containing the method or function.
Wow, whenever I watch your videos, I learn something. Better Comments, using now! Thanks
Thank you!
Great video as always. I might have added unit tests here as a form of documentation. Unit tests can also have clear names, and their docstrings can comment on how edge cases are handled. If I'm coming back to code I wrote six months ago, it's the first place I go.
I really, really appreciate the production values of your videos. Always love seeing your code on screen. It’s crisp, lol. Keep it up 💪🏾
Regarding the topic, I love it. I’m really trying to set some standards for documentation on our team, this is some good stuff
Thanks for sharing!
super vidéo très utile et très attendue sur RUclips
wow. you make VS look so useful. It seems that if you use the docstring helper, you will end up with soem really professional looking docstrings. It makes other people think that you really pay attention to details and make very beautiful looking code. And by using VS, you don't have to expend mental bandwidth trying to figure out what to format a docstring. Nice. You make a good case for VS. If I were a developer, I would def see that VS is a high on the list of preferred ide
Thank you for your precious advices.
I think I become a better developper with your videos.
Thanks for the video, it gives some great tips!
I'd like to comment on some aspects (being a technical writer _and_ a developer) and would also like to share some of my experiences what I made from both roles:
* Docstrings are awesome
Yes, sometimes docstrings take more space than the actual documentation. If this is a public function or method, that shouldn't prevent us to write it. If the doc is autogenerated (like an API), the doc should be complete.
* Focus on your target group(s).
If your doc is autogenerated, it is probably more focused on developers. But depending on your project, developer docs are maybe not enough. Also take users and tutorials in mind.
* Autogenerated API documentation isn't a replacement for user docs.
It looks easy and it's tempting to get the impression, "I documented all functions and modules, so I have all documentation." This is the wrong approach.
Depending on what you write in your docstring, a pure autogenerated API documentation listing only the functions and parameters is pretty useless. It doesn't tell you _how to use_ the project. We need flesh to the bones: some examples and context.
This is something I have been trying to get my team to do better. This is a very good into to what good code documentation should look like
Thanks David!
Came here from the Python Weekly :)
helpfull
i was anxety about find some easy way to document my project, and i found here ^^
Glad it helped!
Man, I wish I knew you when I started programming. Currently I work in a Brazilian Company that provides R&D services on Robotics and Computer Vision globe wide. We're great researchers, but terrible on writing quality code. I'll force everyone in the company to watch you!
Wow! Thanks for sharing that and for your support!
Regarding the docstrings I think it really depends on the use case, if you are writing a code for a project like an API for an app frontend or a data pipeline or some other type of product then it doesn't really make sense to write a docstrings because as you said the main goal should be for the code to be as clean and structured as possible and that should be enough, because adding docstrings basically just duplicates the stuff you already do in natural language and in case of bugs it can lead to even harder debugging because you could potentially inspect the docstring and move on.
However the use case where it is absolutely necessary is for example writing libraries, the user of the library has to know what is going on with the function and all the arugments passed to it and it doesn't really make sense for him to go in the implementation rather he will read the method docstring and that's it.
Another suggestion for making your code more readable is adding units of measurement to the variable names. In this example, it would be nice if the parameter to the deposit and withdraw function would be called amount_in_cents or amount_in_euros etc.
Outsourcing most of my documentation to LLMs
super helpful ❤❤
Wow!!...very good teacher, I hope it's enough motivation to learn Python 😊🐬
Glad to hear you liked the video!
when I try to hint that my output is generator, I get an error saying generator is undefined. I have to from typing import Generator as generator. Only then, can I hint def fx() -> generator:
Great tips! That would also be helpful if you make a video on how to write useful "git comments". Thanks!
Noted!
I only make fully filled google style docstrings for my packages of the classes and functions that the package user is interacting with, and which I have documented using mkdocs. All other code will have a simple typehint and a short summary in the doc string.
Just as I am writing docstrings and READMEs
Is there any reason why you picked the mypy format for docstrings? Is it the one that you use?
I think that Sphinx could also be a great way to improve our documentation. I suggest to include this tool maybe in a part 2 or in an extension of this video. Thanks so much for making so interesting and complementary topics for the Python Community! 🐍 💻
Thank you!
Best way to document your code is with tests. They define what the code does and they are always up-to-date
What is the shortcut used for auto imports ?
Is there a summary function for Better comments? Even if colored, they are still hidden within the code.
The industry shift towards 'self-documenting code', where the naming and coding style itself explains the goal of the code, has been a good thing (IMHO). There have been times where some 10-15 year old code works one way, but the comments never got updated to agree. Some maintainer along the way just couldn't be bothered to 'fix the comments'. And not to sound TOO MUCH like some old 'boomer', but yes, there is a ton of code out there that has to be maintained that is probably older than half of all comp-sci degrees awarded. lol
A few times when good comments really help is when you find yourself employing some 'trick'. Some coders like to keep some 'trick' all secretive (like maybe 'bit-twidling' some float to extract a fraction or shift lower-case ASCII to upper). They get a bit smug and make statements like, "Well if you were a good programmer, you wouldn't need a comment to understand this." But adding a tiny comment to explain some obscure detail is the mark of a really good coder.
A master coder -- heck, a master of _anything_ -- is not afraid to share knowledge with others because they know that to use that knowledge properly requires experience.
Coders that scoff at others because others don't understand are not master coders; they are snobbish know-it-alls who are totally insecure.
About docstrings taking up too much space: It really depends on the style used, e.g. a section in numpy style takes at minimum 4 lines (title, -----, name, description) while in google style it's collapsed down to only two ("title:", "name: description")
Re: Documentation taking up more space than code and making it harder to see. Is there not a way to auto fold all docs?
Depends on your IDE. PyCharm is able to 'fold' docstrings.
thank you
Glad you enjoyed it!
I have two problems with typehints. First, imagine I have a class defined in a file and in another file I have a function that takes as a parameter an instance of that class. To use typehints I import the class in the function file and because I need the function in the class file I also import the function file in the class. This is an obvious problem because I do circular imports. They way of solving this is putting the function inside the class file but I dont know if this is a good practice or not (I mean it should be the good practice if you can't use typehints without an import, that's really my question, can you use type hints without an import?)
The other problem is with typehints and numpy, I also like to have an strict type checker but with numpy I don't know how to use it, sometimes I have used the ArrayLike type but the type checker goes nuts.
I could be wrong, but I think if you surround the typehint in quotes ("), then it forward declares it and you don't need the import, e.g. -> "ObjectFromOtherFile".
Also for type hinting numpy arrays you can use np.ndarray
If there's a function that your class needs, but the function needs an instance of your class, why not put them in the same module?
They are already inseparably coupled. Putting them in the same module is just the most logical thing to do.
ETA: If there's another module that needs them both, it saves an import as well.
@@PanduPoluan Thaks, that was the answer all along
TNX, Nice
Thanks
I would love to build some visual documentation tool or something .
you ar best
Thank you so much!
Sometimes thise comments that explains simple code are helpful. Because they're plain English (or your language) you can quickly get the gist of what's happening as if your reading a book. I will then look at the code if I want further details.
If you need to explain your code in plain English, this *may* be an indication that the code is bad and needs rewriting.
@@edgeeffect No. Stop this sheepish nonsense. Perfect readable code doesn't exist. Go look at any of the Rust, Python, C++, C#, etc. source code. Sometimes your code is doing something complex and you dont need indentation, syntax, and error handling to get in the way. You just want to get to the meat and bones pretty quickly.
Also you don't need to rewrite everything. Just been you don't understand it doesn't mean nobody understands it.
I am struggling to include a table from an excel .xlsx file in my sphiinx documentation. If anyone has done this, I would appreciate some pointers.
Sadly employees feel punished for writing documentation that is too good, because it becomes an employers guide for replacing you with a lower paid person.
If you make yourself too irreplaceable then you make it harder to get promoted.
If you do great work to a level that you are replaceable at any moment, you are irreplaceably valuable. Any project, any initiative, any codebase will want to have you
You are right, I do feel punished for having my PR fail because I didn't write a docstring on an init method with 2-3 vars that are fully annotated.
Write simple, explanatory code and document what's needed, comments are going to get dated anyway and become more confusing to users. That's just reality.
Based on personal experience being the only one who knows the details of an "in house developed" software is no guarantee against layoffs, but it will make sure that you will have trouble going on a hokiday without being disturbed.
@OP If you are in an environment like that, and don‘t get fired for not enabling knowledge transfer, you are part of the problem and did not find a solution. Because you don’t get promoted or even fired since the next guy to you is doing the same thing. Try to set an example, if it doesn’t work get out of these places or be fine in being caught in a vicious cycle of despair reinforcing your own pain.
cool!
When giving a typehint for an argument of the same type as the self class, instead of Self one can use "": "BankAccount" in this case. This way you don't need to include anything.
I've tried to generate the documentation of my django project with mkdocstrings. After testing a few things and exploring further, I managed to get the yml file working with django. Then I installed with the python-legacy handler ... and It more or less works ... until ModelForms are discovered ... and then it crashes ! When I'm trying the same with the new python handler whcich does not execute the code ... I end up with empty documentation pages (official documentation say that the experimental new handler is not yet compatible with Django) ... Could anybody tell me how to generate docstring documentation on a Django app? ... or how could I bypass specific functions, classes or modules ?
The code tells you how, comments tell you why.
I have the issue that certain shortcuts in vs code do not work (I assume the issue is that a shortcut is used multiple times by various extensions so vs code does not know what to do)
Can you show how to handle this in a video?
br
Do a Mojo video 👍
I've used to put docstrings on everything that was more than few lines long but then I've started applying clean code concepts and my code is self documented, I only ever use the single line # comments nowadays.
the docstring part is a basic feature of pycharm what overall is a create ide for python imao
I started programming in the 80’s. I did a lot of code archeology. The running joke about documentation was if the code was hard to write it should be hard to read
Simply make the source description (NOT CODE, NO ONE IS TRYING TO ENCRYPT ANYTHING!) self-documenting! It's not rokit syance... (:'-)
I like Sphinx.