If this is from the perspective of a hobbyist or brand new Python dev, that’s a fair opinion to have, I suppose.
That said, if you’re using Python in a professional capacity, you really need to learn how to use the toolchain properly.
Python packaging and virtual environments are not difficult to understand, and I’d wager based on your comments elsewhere in this thread that your frustrations are born from not taking the time to understand why following the instructions from a fourteen-year-old blog post aren’t working.
99.99% of the time, the fault isn’t with pip, it’s with the maintainer of the broken package you’re trying to use.
Sometimes I wish there was a community for more advanced users, where the concept of deciding on the best build tool chain per project is not a major hurdle. Venvs? Nbd. Pipenv? Nbd. Conda/mamba/micromamba? Nbd. Pure pip? Oh boy, I hope it a simple one, but I’ll manage. Maven? Fml, but sure. Npm? Sure. “Complex” git workflows, no problem.
Idk, that’s just setting up the work environment, if your brains get squeezed by that I’m not sure if you will then be able to the actually code whatever its being asked of you. Some people…
But yeah, this is a newbie space so I guess that we have to ignore some noise.
Seriously, I usually use Poetry these days for most projects, shit just works, build well and lets me distribute my code from PiPy just fine.
Everything in one pyproject.yaml.
If we talk about solutions: python has plenty. Which might be overwhelming to the user.
I use Direnv to manage my python projects. I just have to add layout pyenv 3.12.0 on top and it will create the virtual environment for me. And it will set my shell up to use that virtual environment as I enter that directory. And reset back to default when I leave the directory.
But you could use pipenv, poetry, pdm, conda, mamba for your environment management. Pip and python do not care.
Most importantly: which tool should a beginner use? The PyPA has a few guides and tutorials, one is using pip + venv, another is using pipenv (why would you still do that?), and another tutorial that lets you pick between Hatchling (hatch’s build backend), setuptools, Flit, and PDM, without explaining the differences between them
Friend, while I appreciate the time and effort on the docs, it has a rather tiny section on one of the truly worst aspects of pip (and the only one that really guts usability): package conflicts.
Due to the nature of Python as an interpreted language, there is little that you can check in advance via automation around “can package A and package B coexist peacefully with the lowest common denominator of package X”? Will it work? Will it fail? Run your tool/code and hope for the best!
Pip is a nightmare with larger, spawling package solutions (i.e. a lot of the ML work out there). But even with the freshest of venv creations, things still go remarkably wrong rather quick in my experience. My favorite is when someone, somewhere in the dependency tree forgets to lock their version, which ends up blossoming into a ticking time bomb before it abruptly stops working.
Hopefully, your experiences have been far more pleasant than mine.
If you’re using a manually managed venv, you need to remember to activate it, or to use the appropriate Python.
That really doesn’t seem like a big ask.
I’ve been using python professionally for like 10 years and package management hasn’t really been a big problem.
If you’re doing professional work, you should probably be using docker or something anyway. Working on the host machine is just asking for “it works on my machine what do you mean it doesn’t work in production?” issues.
No, actually most devs don’t use docker like that. Not java devs, not JS devs, not rust devs. That is because maven, npm and cargo manage dependencies per project. You use it for python exactly because pip does it the wrong way and python has big compatibility issues.
The only time I ever interacted with python packaging was when packaging for nixos. And I can tell you that the whole ecosystem is nuts. You have like ten package managers each with thirty different ways to do things, none of which specify dependencies in a way that can be resolved without manual input because y’all have such glorious ideas as implementing the same interface in different packages and giving each the same name and such. Oh and don’t get me started on setup.py making http requests.
If this is from the perspective of a hobbyist or brand new Python dev, that’s a fair opinion to have, I suppose.
That said, if you’re using Python in a professional capacity, you really need to learn how to use the toolchain properly.
Python packaging and virtual environments are not difficult to understand, and I’d wager based on your comments elsewhere in this thread that your frustrations are born from not taking the time to understand why following the instructions from a fourteen-year-old blog post aren’t working.
99.99% of the time, the fault isn’t with pip, it’s with the maintainer of the broken package you’re trying to use.
This is programmer humor, 95% of the people here still get defeated by semicolons, have never used a debugger, and struggle to exit
vim.Sometimes I wish there was a community for more advanced users, where the concept of deciding on the best build tool chain per project is not a major hurdle. Venvs? Nbd. Pipenv? Nbd. Conda/mamba/micromamba? Nbd. Pure pip? Oh boy, I hope it a simple one, but I’ll manage. Maven? Fml, but sure. Npm? Sure. “Complex” git workflows, no problem.
Idk, that’s just setting up the work environment, if your brains get squeezed by that I’m not sure if you will then be able to the actually code whatever its being asked of you. Some people…
But yeah, this is a newbie space so I guess that we have to ignore some noise.
Seriously, I usually use Poetry these days for most projects, shit just works, build well and lets me distribute my code from PiPy just fine. Everything in one pyproject.yaml.
I like the pyproject.yaml, but checking dependencies with poetry takes 5 to 10 minutes for my projects.
Yeah, that’s rough. Is it any faster w PipEnv?
It actually is almost as instant as you would expect
Weird, Poetry gotta up their game then 😎
They’re not difficult by any means.
But they are tedious when compared to other solutions.
If we talk about solutions: python has plenty. Which might be overwhelming to the user.
I use Direnv to manage my python projects. I just have to add
layout pyenv 3.12.0on top and it will create the virtual environment for me. And it will set my shell up to use that virtual environment as I enter that directory. And reset back to default when I leave the directory.But you could use pipenv, poetry, pdm, conda, mamba for your environment management. Pip and python do not care.
This article someone linked is not 14 years old and it perfectly describes the mess python and pip are: https://chriswarrick.com/blog/2023/01/15/how-to-improve-python-packaging/
My favorite part is:
But yes, following old blog post is the issue.
Why not read the official python docs?
Hahaha!..
Oh shit, you’re serious.
They pretty simply describe how to handle a venv, pip, reqs, etc.
Friend, while I appreciate the time and effort on the docs, it has a rather tiny section on one of the truly worst aspects of pip (and the only one that really guts usability): package conflicts.
Due to the nature of Python as an interpreted language, there is little that you can check in advance via automation around “can package A and package B coexist peacefully with the lowest common denominator of package X”? Will it work? Will it fail? Run your tool/code and hope for the best!
Pip is a nightmare with larger, spawling package solutions (i.e. a lot of the ML work out there). But even with the freshest of venv creations, things still go remarkably wrong rather quick in my experience. My favorite is when someone, somewhere in the dependency tree forgets to lock their version, which ends up blossoming into a ticking time bomb before it abruptly stops working.
Hopefully, your experiences have been far more pleasant than mine.
That really doesn’t seem like a big ask.
I’ve been using python professionally for like 10 years and package management hasn’t really been a big problem.
If you’re doing professional work, you should probably be using docker or something anyway. Working on the host machine is just asking for “it works on my machine what do you mean it doesn’t work in production?” issues.
No, actually most devs don’t use docker like that. Not java devs, not JS devs, not rust devs. That is because maven, npm and cargo manage dependencies per project. You use it for python exactly because pip does it the wrong way and python has big compatibility issues.
I have to agree, I maintain and develop packages in fortrat/C/C++ that use Python as a user interface, and in my experience pip just works.
You only need to throw together a ≈30 line
setup.pyand a 5 line bash script and then you never have to think about it again.The only time I ever interacted with python packaging was when packaging for nixos. And I can tell you that the whole ecosystem is nuts. You have like ten package managers each with thirty different ways to do things, none of which specify dependencies in a way that can be resolved without manual input because y’all have such glorious ideas as implementing the same interface in different packages and giving each the same name and such. Oh and don’t get me started on
setup.pymaking http requests.