How to Build a Complete Python Package Step-by-Step

Поделиться
HTML-код
  • Опубликовано: 24 дек 2024

Комментарии • 146

  • @ArjanCodes
    @ArjanCodes  Год назад +1

    👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis

  • @yahyamlaouhi9508
    @yahyamlaouhi9508 Год назад +60

    Man, you are taking Python learning to the next level. I am a big fan of your videos.

  • @loic1665
    @loic1665 11 месяцев назад +2

    Yay, a code roast! XD
    There is one big thing that I think I would still do: turn this into a package. Currently, you just have a collection of modules, but you have a proper package. It’s super easy to do with poetry. Why?
    • Better separation between the library code (in src/my_package) and the main script (main.py lives outside).
    • But instead of a script, you can also create a simple CLI, and for that, having a package will greatly help too
    • Right now, running the main application and the tests ONLY WORKS because you run them from the root folder. But imports in the tests would fail if you were to run them from inside the test folder. And you cannot move the main.py elsewhere. Having a package removes those limitations, as you would import from the package in the tests and the scripts. The imports will always work because you have installed the package in your environment, so python knows where to look.
    • And as a bonus, this would FIX your red-lines issues in your IDE! I guess what happens is that Pylint is having trouble locations the modules you import. Usually imports work a lot better with packages. BTW, if you have a poetry environment, you have to tell VSCode to use it (cmd palette > Select Python Environment)
    I hope I convinced you 😊
    Also, one more thing FYI: you have OUTPUT_TYPE = “visualizer” # “visualizer” | ”console”. Right now, the type of this variable would be inferred as string, but if you want to more precisely define it, you can do it like this: “OUTPUT_TYPE: Literal[“visualizer”] | Literal[“console”] = “visualizer”, and Literal imported from typing. However, I’m not sure I really recommend this approach, I wonder if a StrEnum wouldn’t be better…
    Okay, thanks for reading if you made it here 😊 And thank you again for the video 😊

  • @krimsonsun10
    @krimsonsun10 Год назад +5

    I have been learning so much from you. You Not just python but how to gain job ready skills. I used to be afraid that I do not know how to use any of the python I learned. You bring this down to earth with clear succinct and byte sized lessons. THANK YOU! 🙏

  • @Impatient_Ape
    @Impatient_Ape Год назад +7

    Thank you Arjan! It's so valuable to have all of this concentrated into a single video.

  • @KonstantinosAntonopoulos-pq2qm
    @KonstantinosAntonopoulos-pq2qm 8 месяцев назад

    Finally, a simple and straightforward approach to do it. Thanks for sharing!

    • @ArjanCodes
      @ArjanCodes  8 месяцев назад

      Glad you found it helpful!

  • @JensN113
    @JensN113 Год назад +2

    Packaging is by far the most challenging aspect of Python for me. This helped quite a lot though, thanks

  • @realplod
    @realplod Год назад +2

    I personally love poetry :) Never learnt the other ways of code packing - so this is neat to know! Thanks.

  • @DanielCordey
    @DanielCordey Месяц назад

    Very useful video. I've been using old methods of building my Python packages, but banged on "deprecated" messages this morning... You video really helped me upgrading my vision of distributing Python code .

  • @jhillyt
    @jhillyt 5 месяцев назад

    Great outline. I'm building some common modules at work and this will allow me to easily share with my colleagues

  • @virtualraider
    @virtualraider Год назад +27

    Great video! A good follow up would be good to configure the package with `setup.cfg` or the `.toml` file. The community it's trying to move away from `setup.py` because of security concerns about arbitrary code execution 👍

    • @virtualraider
      @virtualraider Год назад +2

      Sorry for the typos, RUclips won't let me edit the comment 😥

    • @ArjanCodes
      @ArjanCodes  Год назад +13

      Absolutely! In fact, I just recorded a Poetry video that I’ll publish in a few weeks. I’m also going to move all the code examples I publish to use a toml file from now on.

    • @alexanderminidis157
      @alexanderminidis157 Год назад +1

      @@ArjanCodes any plans for (Ana)Conda?

  • @JeromeDemyttenaere
    @JeromeDemyttenaere 8 месяцев назад

    Thanks you very much for your video, it's very hard to find a clear tutorial about building a python package. Now I'm ready to build a project for server-side transaction verification for mobile 😁

  • @qwertyuiopsdfgh
    @qwertyuiopsdfgh Год назад +1

    @11:00 I like to install the package like this: pip install -e .
    The e flag makes it "editable" so while you continue developing any changes you make will be picked up.

  • @pepecopter
    @pepecopter Год назад +1

    thanks so much Arjan! Stars aligned, I had put knowledge about Python packaging on my to-do list this week haha. All the best!

  • @brandonvolesky9867
    @brandonvolesky9867 6 месяцев назад

    Thanks for the quality video. Thanks to you I have become a much more structured dev.

    • @ArjanCodes
      @ArjanCodes  6 месяцев назад

      Glad the videos have been helpful!

  • @daoudakone8838
    @daoudakone8838 8 месяцев назад

    Rien a dire si ce n'est MERCI.Tres satisfaisant vos contenus.👍

  • @tugrulpinar16
    @tugrulpinar16 Год назад

    Thanks for the complete guide Arjan!

  • @PavithraSuLa
    @PavithraSuLa Год назад +2

    Excellent video, Man. keep it up!
    I'm eager to watch a video you explain about meta class and meta-programming!

  • @destroylevi
    @destroylevi Год назад

    very helpful video, thank you!

  • @daviddelaney363
    @daviddelaney363 9 месяцев назад

    Quite good. Thank you!

    • @ArjanCodes
      @ArjanCodes  9 месяцев назад

      Glad you liked it, David!

  • @kjuba2666
    @kjuba2666 3 месяца назад

    great tutorial, thx

  • @CruzanCaramel
    @CruzanCaramel Год назад +3

    @arjancodes please make a video on poetry with typer. Making a CLI and packaging it please

  • @adrianofernandosturaro142
    @adrianofernandosturaro142 Год назад

    During the installation part is usefull to use pip install -e . so you don't need to reinstall you package for every modification you do in the project.

  • @naderbazyari2
    @naderbazyari2 Год назад

    That video was superb. I was so intersting that I forgot to like it in the first place. full of useful information and pro-tips. I just returned to like and leave a comment. Thanks a million.

    • @ArjanCodes
      @ArjanCodes  Год назад

      Thank you and I'm so glad you enjoyed it!

  • @adityamathur2284
    @adityamathur2284 Год назад

    This is amazing information. I believe this will be a great addition( having a course in details) to one of your amazing course library.

  • @mahbub_bro
    @mahbub_bro Год назад

    Wanna learn framework building process. Thanks for this awesome tutorial ❤️

  • @michaelreekie2992
    @michaelreekie2992 Год назад

    Thanks. just what I needed. I am a relative newbie. I am curious about Wade Curry's questions

  • @DeKeL_BaYaZi
    @DeKeL_BaYaZi 7 месяцев назад

    dude. you literally have a video for every python obstacle I encounter! lol

    • @ArjanCodes
      @ArjanCodes  7 месяцев назад +1

      Haha, happy to be of service 😁

  • @ErikS-
    @ErikS- Год назад

    Excellent video!
    I think there is way too few videos on the topic of packaging python files. So I wouldn't be surprised if this video will generate quite some views...

  • @marcodigennarobari
    @marcodigennarobari Год назад

    A M A Z I N G !!
    :D

  • @beliputu4884
    @beliputu4884 Год назад

    Amazing information 👍👍🙏

  • @kamyagupta4030
    @kamyagupta4030 Год назад

    Thank you so much

  • @jithinsasikumar9576
    @jithinsasikumar9576 Год назад +2

    Great video, Arjan. Could you please make a video on Poetry?

  • @christensencode7538
    @christensencode7538 Год назад

    Great video talking about the built in abilities of Python packaging! conda is a neat option too!

  • @itsamemario6588
    @itsamemario6588 Год назад +1

    With the test folder being on the same level as your src folder, does that mean you have to build and install your package every time you test? I can't get relative imports to work...

  • @Relxm8
    @Relxm8 Месяц назад

    Locally install 📦 - 10:56

  • @UNgineering
    @UNgineering Год назад +6

    Excellent tutorial as always. Do you have any plans for a tutorial to distribute a python application for win/mac/linux (not just a single package)? Ideally through some kind of automated github pipeline.

  • @sayidinaahmadalqososyi9770
    @sayidinaahmadalqososyi9770 10 месяцев назад

    thanks man😊

  • @Didanihaaaa
    @Didanihaaaa Год назад

    Thank you for the assistance. I found it incredibly helpful. Suppose I have developed an application that is functioning perfectly on my machine. Now, I would like to obtain comprehensive information about the versions of all the requirements and ensure their accuracy. To achieve this, I used the "pip freeze" command to retrieve the versions of the libraries I utilized. I would appreciate your confirmation on whether this is considered a correct practice or if there are any alternative methods you would recommend.

  • @dimboump
    @dimboump Год назад +3

    Thanks Arjan! Have you ever used declarative metadata? What are their (dis)advantages compared to setuptools?

  • @RicardoSilva-l9q
    @RicardoSilva-l9q 9 месяцев назад

    You simply goated

  • @Max-zv9jh
    @Max-zv9jh Год назад +1

    now we want a video about licenses!

  • @HanktheDank167
    @HanktheDank167 8 месяцев назад

    Woah! Holy crap you look like John Green! Great Videos btw.

    • @ArjanCodes
      @ArjanCodes  8 месяцев назад

      Glad you liked the content!

  • @mikhailgoussarov4663
    @mikhailgoussarov4663 Год назад

    Excellent tutorial, would you be interested in doing the same but with the poetry library?

  • @_baco
    @_baco Год назад

    The tendency nowadays is to avoid packaging as code. People get really imaginative when having the possibility to code and start adding logic in `setup.py`, making the building process not so strait forward.
    The first approach was to do all the same but in `setup.cfg`, an INI-like metadata file format that setuptools understands. Severing the possibility to write complex functions that run during the packaging process.
    Finally, the consensus is to use a standard build system, that may use setuptools as back-end as well, and define the building rules as metadata too, but in a `pyproject.toml` file.
    I think that last approach is the most accurate way to teach Python Packaging today.
    A follow up video perhaps, just to show the current recommended way to package? 😅😛

    • @ArjanCodes
      @ArjanCodes  Год назад +1

      Coming soon! I'm also going to include a pyproject.toml file from now on in my code examples on GitHub :).

  • @AndrewMelnychuk0seen
    @AndrewMelnychuk0seen 3 месяца назад

    S-Teir content Thanks man.

  • @sea0920
    @sea0920 6 месяцев назад

    With poetry, you can just do "poetry new your_project_name", "poetry build", "poetry publish" and you're done.

  • @DatabaseAdministration
    @DatabaseAdministration 7 месяцев назад

    Wish you explain package folder structure in more detail.

  • @bennguyen1313
    @bennguyen1313 Год назад

    Any suggestions how to build sip-4.16.5 ? Should MS Visual Studio Express 2010 or mingw/gcc be used?
    For example, Nuitka reports:
    Python: 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)]
    Version C compiler: C:\Users\Temp\AppData\Local\Nuitka\Nuitka\Cache\downloads\gcc\x86\11.3.0-14.0.3-10.0.0-msvcrt-r3\mingw32\bin\gcc.exe (gcc).
    Which seems like both Microsoft Visual Studio 2010 and gcc... so I'd like to build using that gcc rather than microsoft studio since that may mess up Nuitka?

  • @3BDE3ALIAITTALB
    @3BDE3ALIAITTALB Месяц назад

    I would like to verify if the setup.py can be replaced by a file.toml ? or am I wrong ?

  • @madrag
    @madrag Год назад +1

    Can we have an update (if applicable) with regards of python 3.12 PEP 632?

    • @nickolay414
      @nickolay414 Год назад

      PEP 632 is about distutils-> setuptools migrations, which is probably interesting only to people who have been packaging python software for at least 10 years and do not need tutorials?

  • @dputra
    @dputra Год назад

    What pypi server do you use for private packages?

  • @numerofobia
    @numerofobia Год назад

    Hi Arjan, thanks for the video. After I run the build commands I'm not being able to import it. It says that the module was not found. Do you know whyt is it happening? Thanks!

  • @rrwoodyt
    @rrwoodyt Год назад

    Hmm, I didn't realize that import supported a tuple as the import targets. Not sure how I feel about that, stylistically! Great video though. I wish I'd seen it last year when I was detangling some package creation for a legacy project. Maybe I should try "import time_travel" and help Past-Me....

  • @jurgenrusch4041
    @jurgenrusch4041 Год назад +1

    Hi Arjan, again a great video that covers a topic from begin to end.
    Although I have made quite some packages within the company where I work, I've never taken the threshold to actually publish a package (to a company internal Pypi server that is).
    I join suggestions from other commenters: could you also do a video on packaging with conda and one with poetry? And what the 'deeper thoughts' behind these alternatives are? So as to understand why they were created anyway? And, if possible, explain why you would favour one over the other? I realize, quite some questions but I sincerely think many struggle with this.

    • @ArjanCodes
      @ArjanCodes  Год назад +1

      Thanks for sharing! Video on Poetry is coming soon! Stay tuned :)

  • @nuurnwui
    @nuurnwui Год назад +6

    In 2023 you talk about the old setup? I thought it just got removed and we now have toml parser in core. Nevertheless: please more packaging videos. Poetry? Flit?

    • @ArjanCodes
      @ArjanCodes  Год назад +1

      Video on Poetry is coming soon! Stay tuned :)

  • @djangodeveloper07
    @djangodeveloper07 Год назад

    amazing topic and video. want to see more on this.

  • @lukasbelck4514
    @lukasbelck4514 Год назад +3

    what about using poetry?

    • @ArjanCodes
      @ArjanCodes  Год назад

      Video on Poetry is coming soon! Stay tuned :)

  • @SakuraMizuno-k9t
    @SakuraMizuno-k9t 5 месяцев назад

    Can you do build package with poetry too

  • @branmuller
    @branmuller Год назад

    Why have the app directory? is this the same thing as a src directory?

  • @dmitryoshkalo789
    @dmitryoshkalo789 Год назад

    Thx for the video, but take into account that according to setuptools docs running python setup.py directly as a script on recent versions is considered deprecated

  • @AntonioDellElceUK
    @AntonioDellElceUK Год назад

    I believed that the "best practice" way to build a wheel is the "pip wheel" command and requirements should be in the usual `requirements.txt` file(s) and be loaded through `pkg_resources.parse_requirements`

  • @DuarteMolha
    @DuarteMolha Год назад

    One thing I struggle with python packaging is that most of the programs i develop require additional programs that are not python packages.
    How do we make it so that we can make thise requirements explicit and stop a package from instaling and declare the tools missing?

  • @cablegi
    @cablegi 6 месяцев назад

    where is the version of the package stored?

  • @FedxD
    @FedxD Год назад

    Sir i am having a problem i uploaded the file but when i load the file by pip install none of the function work it only imports nothing works idk why i imported all the modules in the __init__ file can you please help em

  • @thetaomegatheta
    @thetaomegatheta 5 месяцев назад

    5:30 - `python setup.py bdist_wheel` apparently does nothing. Did you forget to show how to prepare setup.py?
    Even after going ahead and preparing `setup.py` and `__init__.py`, after installation, none of the projects, nor any python CLI can see anything inside the package. Can't import anything, can't use any classes, nothing.

  • @SP-db6sh
    @SP-db6sh Год назад

    Not all punctuation are allowed, how select the desired ones

  • @vkit280
    @vkit280 Год назад

    could not do on linkedin - will do on facebook though

  • @pydisekhar
    @pydisekhar Год назад

    can you give us a video to make a deployment file so that people can access it

  • @kslader8
    @kslader8 Год назад +3

    from my perspective the community is moving away from setup tools. I'd probably go over a framework like poetry or flit these days

  • @tacozmacleo
    @tacozmacleo Год назад +3

    Are you spying on me?
    Everytime you release a video it is relevant to what I am currently working on. xD
    But is there a reason why you do not use the folder structure they use on docs.python when they talk about file/folder structure?
    (A src folder with the package folder inside)

    • @amitaniket1502
      @amitaniket1502 Год назад

      The same thing happens with me. One day I was searching for pydantic models in python and the same video came in 2 days.

    • @2005sty
      @2005sty 11 месяцев назад

      I would like to know too.

  • @abdullahyousef3596
    @abdullahyousef3596 4 месяца назад

    How do you choose the name of the package?

  • @1888jeremy
    @1888jeremy Год назад

    have you published it yet

  • @pydisekhar
    @pydisekhar Год назад

    give me file which should work as same as the build file works in nodejs

  • @marcotroster8247
    @marcotroster8247 Год назад

    Next level: Try to compile a Python package written in C with manylinux tools and publish it via an automated GitHub workflow 😂😂😂
    I mean the way you publish doesn't really require PyPI. You'd probably be better off with a Git submodule, tbh. Just clone the repo, cd into it and run "pip install .", basically does the same but without PyPI in your way 😉

  • @m_sitso
    @m_sitso 3 месяца назад

    your file system here is confusing me, pls try showing the desktop. It will be way better

  • @popovanatoliy4736
    @popovanatoliy4736 Год назад +3

    wow. it is overcomplicated.

  • @allo5668
    @allo5668 Год назад +4

    I could watch another 12 movies on this topic. It’s the most confusing least intuitive part of python

  • @bencipherx
    @bencipherx Год назад

    Can you read my mind Arjan cos I was thinking of building a package that incorporates Twitter api with chat gpt. By the way I need frontend volunteers, comment below if you feel it’s a suitable project to be part of.

    • @calvinmakokha6990
      @calvinmakokha6990 Год назад

      Not a frontend expert, but would like to be a part of this.

    • @ArjanCodes
      @ArjanCodes  Год назад

      Awesome! Thanks for sharing this.

  • @breno-jesus-fernandes
    @breno-jesus-fernandes Год назад +3

    C'mmon where's poetry?

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w Год назад

    Do you also cover Julia language?

  • @therealslimaddy
    @therealslimaddy Год назад

    Checkout pdm

  • @schedarr
    @schedarr Год назад +63

    Nope, the answer is poetry

    • @ArjanCodes
      @ArjanCodes  Год назад +34

      I just recorded a video about Poetry, so I agree now :).

    • @joelbeck1822
      @joelbeck1822 Год назад +13

      The actual answer is pdm! Another example of the „There are 14 competing standards“ xkcd^^ I like poetry but imo pdm is even better.

    • @__sassan__
      @__sassan__ Год назад +7

      @@joelbeck1822 totally agree on PDM. poetry capped versions are far more troublesome than useful.

    • @Jugge83
      @Jugge83 Год назад +1

      Pipenv is good enough!

    • @justinlapre2226
      @justinlapre2226 Год назад +1

      I like pbr.

  • @fillipeamg5877
    @fillipeamg5877 Год назад

    is he dutch? he looks dutch

    • @ArjanCodes
      @ArjanCodes  Год назад +1

      He is as Dutch as they come.