Make Python 3.12 install user packages without complaints

I have a habit of pip3 install --user and then expecting these packages under ~/.local/lib/ to be available for my Python scripts whenever I need them. However, with PEP 668 landing in Python 3.12, I now have to add --break-system-packages even for user packages. This is super annoying considered that I have multiple projects sharing the same set of common packages (e.g. mkdocs-material, a nice MkDocs theme). So time to tell pip to jerk off on that complaint.

Obviously, aliasing pip3 (as per my personal habit, I always prefer python3 and pip3 over python and pip) to pip3 --break-system-packages could work, with all the limitations that any other shell alias bear.

The key here is, by examining how virtual environments work, we can trick Python into thinking that ~/.local is one of them. This is already documented in the site package:

If a file named pyvenv.cfg exists one directory above sys.executable …

So here’s the solution, assuming ~/.local/bin is already in your $PATH:

1. Symlink /usr/bin/python3 to ~/.local/bin/python3
2. Copy /usr/bin/pip3 to ~/.local/bin/pip3, and change the shebang line to #!/home/example/.local/bin/python3 (you’ll have to use the absolute path here, though).

3. Create ~/.local/pyvenv.cfg with just one line of content:

    include-system-site-packages = true
   

   You can, of course, add other settings for venv, which is completely optional and up to you.

Now whenever you install something with pip3, it’ll happily install it under ~/.local/lib/python3.12/site-packages even without the need for --user.

If you prefer python or pip commands, you can just change the file names in the above steps accordingly.

Noteworthy points are:

1. This method certainly can work for the system Python installation (at /usr), which I wouldn’t recommend for obvious reasons. If you insist, you should at least do this under /usr/local instead.
2. This method (when applied to `~/.local`) is ineffective against scripts already shebanged with `#!/usr/bin/python3`. Consider developing the habit of running `python3 script.py` instead of `./script.py` like I do.

Corrigenda

• Scripts shebanged with #!/usr/bin/python3 will pick up packages under ~/.local/lib/ as this is the default user site directory, which comes in sys.path even before the system site directory.