Yay, a code roast! XD There is one big thing that I think I would still do: turn this into a package. Currently, you just have a collection of modules, but you have a proper package. It’s super easy to do with poetry. Why? • Better separation between the library code (in src/my_package) and the main script (main.py lives outside). • But instead of a script, you can also create a simple CLI, and for that, having a package will greatly help too • Right now, running the main application and the tests ONLY WORKS because you run them from the root folder. But imports in the tests would fail if you were to run them from inside the test folder. And you cannot move the main.py elsewhere. Having a package removes those limitations, as you would import from the package in the tests and the scripts. The imports will always work because you have installed the package in your environment, so python knows where to look. • And as a bonus, this would FIX your red-lines issues in your IDE! I guess what happens is that Pylint is having trouble locations the modules you import. Usually imports work a lot better with packages. BTW, if you have a poetry environment, you have to tell VSCode to use it (cmd palette > Select Python Environment) I hope I convinced you 😊 Also, one more thing FYI: you have OUTPUT_TYPE = “visualizer” # “visualizer” | ”console”. Right now, the type of this variable would be inferred as string, but if you want to more precisely define it, you can do it like this: “OUTPUT_TYPE: Literal[“visualizer”] | Literal[“console”] = “visualizer”, and Literal imported from typing. However, I’m not sure I really recommend this approach, I wonder if a StrEnum wouldn’t be better… Okay, thanks for reading if you made it here 😊 And thank you again for the video 😊
I have been learning so much from you. You Not just python but how to gain job ready skills. I used to be afraid that I do not know how to use any of the python I learned. You bring this down to earth with clear succinct and byte sized lessons. THANK YOU! 🙏
Very useful video. I've been using old methods of building my Python packages, but banged on "deprecated" messages this morning... You video really helped me upgrading my vision of distributing Python code .
Great video! A good follow up would be good to configure the package with `setup.cfg` or the `.toml` file. The community it's trying to move away from `setup.py` because of security concerns about arbitrary code execution 👍
Absolutely! In fact, I just recorded a Poetry video that I’ll publish in a few weeks. I’m also going to move all the code examples I publish to use a toml file from now on.
Thanks you very much for your video, it's very hard to find a clear tutorial about building a python package. Now I'm ready to build a project for server-side transaction verification for mobile 😁
@11:00 I like to install the package like this: pip install -e . The e flag makes it "editable" so while you continue developing any changes you make will be picked up.
During the installation part is usefull to use pip install -e . so you don't need to reinstall you package for every modification you do in the project.
That video was superb. I was so intersting that I forgot to like it in the first place. full of useful information and pro-tips. I just returned to like and leave a comment. Thanks a million.
Excellent video! I think there is way too few videos on the topic of packaging python files. So I wouldn't be surprised if this video will generate quite some views...
With the test folder being on the same level as your src folder, does that mean you have to build and install your package every time you test? I can't get relative imports to work...
Excellent tutorial as always. Do you have any plans for a tutorial to distribute a python application for win/mac/linux (not just a single package)? Ideally through some kind of automated github pipeline.
Thank you for the assistance. I found it incredibly helpful. Suppose I have developed an application that is functioning perfectly on my machine. Now, I would like to obtain comprehensive information about the versions of all the requirements and ensure their accuracy. To achieve this, I used the "pip freeze" command to retrieve the versions of the libraries I utilized. I would appreciate your confirmation on whether this is considered a correct practice or if there are any alternative methods you would recommend.
The tendency nowadays is to avoid packaging as code. People get really imaginative when having the possibility to code and start adding logic in `setup.py`, making the building process not so strait forward. The first approach was to do all the same but in `setup.cfg`, an INI-like metadata file format that setuptools understands. Severing the possibility to write complex functions that run during the packaging process. Finally, the consensus is to use a standard build system, that may use setuptools as back-end as well, and define the building rules as metadata too, but in a `pyproject.toml` file. I think that last approach is the most accurate way to teach Python Packaging today. A follow up video perhaps, just to show the current recommended way to package? 😅😛
Any suggestions how to build sip-4.16.5 ? Should MS Visual Studio Express 2010 or mingw/gcc be used? For example, Nuitka reports: Python: 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)] Version C compiler: C:\Users\Temp\AppData\Local\Nuitka\Nuitka\Cache\downloads\gcc\x86\11.3.0-14.0.3-10.0.0-msvcrt-r3\mingw32\bin\gcc.exe (gcc). Which seems like both Microsoft Visual Studio 2010 and gcc... so I'd like to build using that gcc rather than microsoft studio since that may mess up Nuitka?
PEP 632 is about distutils-> setuptools migrations, which is probably interesting only to people who have been packaging python software for at least 10 years and do not need tutorials?
Hi Arjan, thanks for the video. After I run the build commands I'm not being able to import it. It says that the module was not found. Do you know whyt is it happening? Thanks!
Hmm, I didn't realize that import supported a tuple as the import targets. Not sure how I feel about that, stylistically! Great video though. I wish I'd seen it last year when I was detangling some package creation for a legacy project. Maybe I should try "import time_travel" and help Past-Me....
Hi Arjan, again a great video that covers a topic from begin to end. Although I have made quite some packages within the company where I work, I've never taken the threshold to actually publish a package (to a company internal Pypi server that is). I join suggestions from other commenters: could you also do a video on packaging with conda and one with poetry? And what the 'deeper thoughts' behind these alternatives are? So as to understand why they were created anyway? And, if possible, explain why you would favour one over the other? I realize, quite some questions but I sincerely think many struggle with this.
In 2023 you talk about the old setup? I thought it just got removed and we now have toml parser in core. Nevertheless: please more packaging videos. Poetry? Flit?
Thx for the video, but take into account that according to setuptools docs running python setup.py directly as a script on recent versions is considered deprecated
I believed that the "best practice" way to build a wheel is the "pip wheel" command and requirements should be in the usual `requirements.txt` file(s) and be loaded through `pkg_resources.parse_requirements`
One thing I struggle with python packaging is that most of the programs i develop require additional programs that are not python packages. How do we make it so that we can make thise requirements explicit and stop a package from instaling and declare the tools missing?
Sir i am having a problem i uploaded the file but when i load the file by pip install none of the function work it only imports nothing works idk why i imported all the modules in the __init__ file can you please help em
5:30 - `python setup.py bdist_wheel` apparently does nothing. Did you forget to show how to prepare setup.py? Even after going ahead and preparing `setup.py` and `__init__.py`, after installation, none of the projects, nor any python CLI can see anything inside the package. Can't import anything, can't use any classes, nothing.
Are you spying on me? Everytime you release a video it is relevant to what I am currently working on. xD But is there a reason why you do not use the folder structure they use on docs.python when they talk about file/folder structure? (A src folder with the package folder inside)
Next level: Try to compile a Python package written in C with manylinux tools and publish it via an automated GitHub workflow 😂😂😂 I mean the way you publish doesn't really require PyPI. You'd probably be better off with a Git submodule, tbh. Just clone the repo, cd into it and run "pip install .", basically does the same but without PyPI in your way 😉
Can you read my mind Arjan cos I was thinking of building a package that incorporates Twitter api with chat gpt. By the way I need frontend volunteers, comment below if you feel it’s a suitable project to be part of.
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
Man, you are taking Python learning to the next level. I am a big fan of your videos.
Thank you so much!
Yay, a code roast! XD
There is one big thing that I think I would still do: turn this into a package. Currently, you just have a collection of modules, but you have a proper package. It’s super easy to do with poetry. Why?
• Better separation between the library code (in src/my_package) and the main script (main.py lives outside).
• But instead of a script, you can also create a simple CLI, and for that, having a package will greatly help too
• Right now, running the main application and the tests ONLY WORKS because you run them from the root folder. But imports in the tests would fail if you were to run them from inside the test folder. And you cannot move the main.py elsewhere. Having a package removes those limitations, as you would import from the package in the tests and the scripts. The imports will always work because you have installed the package in your environment, so python knows where to look.
• And as a bonus, this would FIX your red-lines issues in your IDE! I guess what happens is that Pylint is having trouble locations the modules you import. Usually imports work a lot better with packages. BTW, if you have a poetry environment, you have to tell VSCode to use it (cmd palette > Select Python Environment)
I hope I convinced you 😊
Also, one more thing FYI: you have OUTPUT_TYPE = “visualizer” # “visualizer” | ”console”. Right now, the type of this variable would be inferred as string, but if you want to more precisely define it, you can do it like this: “OUTPUT_TYPE: Literal[“visualizer”] | Literal[“console”] = “visualizer”, and Literal imported from typing. However, I’m not sure I really recommend this approach, I wonder if a StrEnum wouldn’t be better…
Okay, thanks for reading if you made it here 😊 And thank you again for the video 😊
I have been learning so much from you. You Not just python but how to gain job ready skills. I used to be afraid that I do not know how to use any of the python I learned. You bring this down to earth with clear succinct and byte sized lessons. THANK YOU! 🙏
Thank you Arjan! It's so valuable to have all of this concentrated into a single video.
Glad it was helpful!
Finally, a simple and straightforward approach to do it. Thanks for sharing!
Glad you found it helpful!
Packaging is by far the most challenging aspect of Python for me. This helped quite a lot though, thanks
I personally love poetry :) Never learnt the other ways of code packing - so this is neat to know! Thanks.
Very useful video. I've been using old methods of building my Python packages, but banged on "deprecated" messages this morning... You video really helped me upgrading my vision of distributing Python code .
Glad it helped!
Great outline. I'm building some common modules at work and this will allow me to easily share with my colleagues
Great video! A good follow up would be good to configure the package with `setup.cfg` or the `.toml` file. The community it's trying to move away from `setup.py` because of security concerns about arbitrary code execution 👍
Sorry for the typos, RUclips won't let me edit the comment 😥
Absolutely! In fact, I just recorded a Poetry video that I’ll publish in a few weeks. I’m also going to move all the code examples I publish to use a toml file from now on.
@@ArjanCodes any plans for (Ana)Conda?
Thanks you very much for your video, it's very hard to find a clear tutorial about building a python package. Now I'm ready to build a project for server-side transaction verification for mobile 😁
@11:00 I like to install the package like this: pip install -e .
The e flag makes it "editable" so while you continue developing any changes you make will be picked up.
thanks so much Arjan! Stars aligned, I had put knowledge about Python packaging on my to-do list this week haha. All the best!
Wow, glad I could help!
Thanks for the quality video. Thanks to you I have become a much more structured dev.
Glad the videos have been helpful!
Rien a dire si ce n'est MERCI.Tres satisfaisant vos contenus.👍
Thanks for the complete guide Arjan!
Excellent video, Man. keep it up!
I'm eager to watch a video you explain about meta class and meta-programming!
Noted!
very helpful video, thank you!
Quite good. Thank you!
Glad you liked it, David!
great tutorial, thx
@arjancodes please make a video on poetry with typer. Making a CLI and packaging it please
During the installation part is usefull to use pip install -e . so you don't need to reinstall you package for every modification you do in the project.
That video was superb. I was so intersting that I forgot to like it in the first place. full of useful information and pro-tips. I just returned to like and leave a comment. Thanks a million.
Thank you and I'm so glad you enjoyed it!
This is amazing information. I believe this will be a great addition( having a course in details) to one of your amazing course library.
Wanna learn framework building process. Thanks for this awesome tutorial ❤️
Thanks. just what I needed. I am a relative newbie. I am curious about Wade Curry's questions
dude. you literally have a video for every python obstacle I encounter! lol
Haha, happy to be of service 😁
Excellent video!
I think there is way too few videos on the topic of packaging python files. So I wouldn't be surprised if this video will generate quite some views...
A M A Z I N G !!
:D
Amazing information 👍👍🙏
Thank you so much
You're most welcome
Great video, Arjan. Could you please make a video on Poetry?
Video on Poetry is coming soon! Stay tuned :)
@@ArjanCodes Eagerly waiting for it :)
Great video talking about the built in abilities of Python packaging! conda is a neat option too!
Thank you!
With the test folder being on the same level as your src folder, does that mean you have to build and install your package every time you test? I can't get relative imports to work...
Locally install 📦 - 10:56
Excellent tutorial as always. Do you have any plans for a tutorial to distribute a python application for win/mac/linux (not just a single package)? Ideally through some kind of automated github pipeline.
thanks man😊
Welcome 😊
Thank you for the assistance. I found it incredibly helpful. Suppose I have developed an application that is functioning perfectly on my machine. Now, I would like to obtain comprehensive information about the versions of all the requirements and ensure their accuracy. To achieve this, I used the "pip freeze" command to retrieve the versions of the libraries I utilized. I would appreciate your confirmation on whether this is considered a correct practice or if there are any alternative methods you would recommend.
Thanks Arjan! Have you ever used declarative metadata? What are their (dis)advantages compared to setuptools?
You simply goated
Thank you!
now we want a video about licenses!
Noted!
Woah! Holy crap you look like John Green! Great Videos btw.
Glad you liked the content!
Excellent tutorial, would you be interested in doing the same but with the poetry library?
The tendency nowadays is to avoid packaging as code. People get really imaginative when having the possibility to code and start adding logic in `setup.py`, making the building process not so strait forward.
The first approach was to do all the same but in `setup.cfg`, an INI-like metadata file format that setuptools understands. Severing the possibility to write complex functions that run during the packaging process.
Finally, the consensus is to use a standard build system, that may use setuptools as back-end as well, and define the building rules as metadata too, but in a `pyproject.toml` file.
I think that last approach is the most accurate way to teach Python Packaging today.
A follow up video perhaps, just to show the current recommended way to package? 😅😛
Coming soon! I'm also going to include a pyproject.toml file from now on in my code examples on GitHub :).
S-Teir content Thanks man.
With poetry, you can just do "poetry new your_project_name", "poetry build", "poetry publish" and you're done.
Wish you explain package folder structure in more detail.
Any suggestions how to build sip-4.16.5 ? Should MS Visual Studio Express 2010 or mingw/gcc be used?
For example, Nuitka reports:
Python: 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)]
Version C compiler: C:\Users\Temp\AppData\Local\Nuitka\Nuitka\Cache\downloads\gcc\x86\11.3.0-14.0.3-10.0.0-msvcrt-r3\mingw32\bin\gcc.exe (gcc).
Which seems like both Microsoft Visual Studio 2010 and gcc... so I'd like to build using that gcc rather than microsoft studio since that may mess up Nuitka?
I would like to verify if the setup.py can be replaced by a file.toml ? or am I wrong ?
Can we have an update (if applicable) with regards of python 3.12 PEP 632?
PEP 632 is about distutils-> setuptools migrations, which is probably interesting only to people who have been packaging python software for at least 10 years and do not need tutorials?
What pypi server do you use for private packages?
Hi Arjan, thanks for the video. After I run the build commands I'm not being able to import it. It says that the module was not found. Do you know whyt is it happening? Thanks!
Hmm, I didn't realize that import supported a tuple as the import targets. Not sure how I feel about that, stylistically! Great video though. I wish I'd seen it last year when I was detangling some package creation for a legacy project. Maybe I should try "import time_travel" and help Past-Me....
Hi Arjan, again a great video that covers a topic from begin to end.
Although I have made quite some packages within the company where I work, I've never taken the threshold to actually publish a package (to a company internal Pypi server that is).
I join suggestions from other commenters: could you also do a video on packaging with conda and one with poetry? And what the 'deeper thoughts' behind these alternatives are? So as to understand why they were created anyway? And, if possible, explain why you would favour one over the other? I realize, quite some questions but I sincerely think many struggle with this.
Thanks for sharing! Video on Poetry is coming soon! Stay tuned :)
In 2023 you talk about the old setup? I thought it just got removed and we now have toml parser in core. Nevertheless: please more packaging videos. Poetry? Flit?
Video on Poetry is coming soon! Stay tuned :)
amazing topic and video. want to see more on this.
Thank you! Will do!
what about using poetry?
Video on Poetry is coming soon! Stay tuned :)
Can you do build package with poetry too
Why have the app directory? is this the same thing as a src directory?
Thx for the video, but take into account that according to setuptools docs running python setup.py directly as a script on recent versions is considered deprecated
I believed that the "best practice" way to build a wheel is the "pip wheel" command and requirements should be in the usual `requirements.txt` file(s) and be loaded through `pkg_resources.parse_requirements`
One thing I struggle with python packaging is that most of the programs i develop require additional programs that are not python packages.
How do we make it so that we can make thise requirements explicit and stop a package from instaling and declare the tools missing?
where is the version of the package stored?
Sir i am having a problem i uploaded the file but when i load the file by pip install none of the function work it only imports nothing works idk why i imported all the modules in the __init__ file can you please help em
5:30 - `python setup.py bdist_wheel` apparently does nothing. Did you forget to show how to prepare setup.py?
Even after going ahead and preparing `setup.py` and `__init__.py`, after installation, none of the projects, nor any python CLI can see anything inside the package. Can't import anything, can't use any classes, nothing.
Not all punctuation are allowed, how select the desired ones
could not do on linkedin - will do on facebook though
can you give us a video to make a deployment file so that people can access it
from my perspective the community is moving away from setup tools. I'd probably go over a framework like poetry or flit these days
Are you spying on me?
Everytime you release a video it is relevant to what I am currently working on. xD
But is there a reason why you do not use the folder structure they use on docs.python when they talk about file/folder structure?
(A src folder with the package folder inside)
The same thing happens with me. One day I was searching for pydantic models in python and the same video came in 2 days.
I would like to know too.
How do you choose the name of the package?
have you published it yet
give me file which should work as same as the build file works in nodejs
Next level: Try to compile a Python package written in C with manylinux tools and publish it via an automated GitHub workflow 😂😂😂
I mean the way you publish doesn't really require PyPI. You'd probably be better off with a Git submodule, tbh. Just clone the repo, cd into it and run "pip install .", basically does the same but without PyPI in your way 😉
your file system here is confusing me, pls try showing the desktop. It will be way better
wow. it is overcomplicated.
I could watch another 12 movies on this topic. It’s the most confusing least intuitive part of python
Can you read my mind Arjan cos I was thinking of building a package that incorporates Twitter api with chat gpt. By the way I need frontend volunteers, comment below if you feel it’s a suitable project to be part of.
Not a frontend expert, but would like to be a part of this.
Awesome! Thanks for sharing this.
C'mmon where's poetry?
Coming soon!
Do you also cover Julia language?
Checkout pdm
Nope, the answer is poetry
I just recorded a video about Poetry, so I agree now :).
The actual answer is pdm! Another example of the „There are 14 competing standards“ xkcd^^ I like poetry but imo pdm is even better.
@@joelbeck1822 totally agree on PDM. poetry capped versions are far more troublesome than useful.
Pipenv is good enough!
I like pbr.
is he dutch? he looks dutch
He is as Dutch as they come.