Pip is Python’s package manager, providing essential core features for installing and maintaining Python packages and their dependencies. While many Python developers use Pip as a Dependency Manager, it was never intended to be used as one. Pip will not flag dependency conflicts. As a result, it will happily install multiple versions of a dependency into your project, which will likely result in errors.
One way to avoid dependency conflicts is to use an alternative Python package manager, like conda, poetry or ActiveState’s State Tool. While all three perform dependency resolution (unlike pip), only the State Tool is capable of resolving dependency conflicts, eliminating dependency hell. The easiest way to install the State Tool is to install Python 3.9 from ActiveState. Install Python 3.9.
If you choose to stick with pip, be aware that it has been integrated with higher-level tools, such as Pipenv which utilizes Pip’s command-line interface (CLI) to help perform enhanced dependency management in virtual environments. There are several different approaches to dealing with dependencies in Python, one of which is using Pip with Pipenv.
Another solution for better managing dependencies with Pip is to use it in conjunction with a requirements.txt file, which allows for the pinning of packages and their dependencies to specific versions (details below).
Dependency Management with Pip
Pip is installed by default with any latest version of Python greater than v3.4. To check which version of Pip you are running, enter the following command in a terminal or command window:
$ pip –version
You can also check to see if Pip is up-to-date:
$ pip install –upgrade pip
The pip command installs packages and their dependencies from PyPI (Python Package Index) by default:
$ pip install <packagename>
Pip relies on package authors to stipulate the dependencies for their code in order to successfully download and install the package plus all required dependencies from the Python Package Index (PyPI).
But if packages are installed one at a time, it may lead to dependency conflicts. A common scenario may involve an initially installed “Package 1” that requires v1.0 of “Dependency A,” while a subsequently installed “Package 2” requires v1.2 of “Dependency A,” which is incompatible with Package 1.
Instead, consider installing a pre-built runtime environment, such as Anaconda or Python by ActiveState, or else installing all required packages for your project from a requirements file.
Wheels and Dependencies
A wheel is an archived Python project consisting of one or more packages and their dependencies. Pip is able to install packages configured with their dependencies, because most authors package their code as wheels by default before submitting to PyPI.
It is not necessary to provide the full wheel name. The package name is all that is required to run pip when installing a package from PyPI:
$ pip install <packagename>
For example, the Twine package and its dependencies are provided in a Wheel named ‘twine-3.1.1-py3-none-any.whl’ located in https://pypi.org/project/twine.
Requirements.txt and Dependencies
Pip does not provide true dependency resolution, but this can be solved by using it in conjunction with a requirements.txt file. Requirements.txt files can be used to make pip resolve dependency conflicts between different packages.
The requirements.txt example below resolves a dependency conflict between two packages, where <package1> and <package2> require different versions of the same dependency.
If <package1> requires <package3==1.0> and <package2> requires <package3>=2.0, and if <package1> is resolved first, then Pip will use <package3>==1.0. In this scenario, a <package3> version that conflicts with <package2>’s dependency requirement will be installed. The solution is to place <package3>=2.0 in the requirements.txt file together with the other requirements.
Requirements.txt
<package1>
<package2>
<package3>=2.0
To install a requirements.txt file, cd into the directory where the file is be located, and enter:
$ pip install -r requirements.txt
-r, –requirement option installs multiple packages listed in a requirements.txt file.
Requirements.txt files can be used multiple times in a default single environment or in multiple virtual environments, as a method of version control to override dependencies with patches (different package versions).
Managing Dependencies when Updating Python Packages
Updating one or more packages in a project can result in dependency conflicts since newer package versions will often require newer versions of dependencies, as well.
To learn more about updating Python packages, refer to our Quick Read: How to Update all Python Packages.
A modern solution to Dependency management – Try ActiveState’s Platform
Dependency resolution is at the core of the ActiveState Platform. When you create a project and start adding requirements, the Platforms tell you what dependencies those requirements have.
The ActiveState Platform is a cloud-based build tool for Python. It provides build automation and vulnerability remediation for:
- Python language cores, including Python 2.7 and Python 3.5+
- Python packages and their dependencies, including:
- Transitive dependencies (ie., dependencies of dependencies)
- Linked C and Fortran libraries, so you can build data science packages
- Operating system-level dependencies for Windows, Linux, and macOS
- Shared dependencies (ie., OpenSSL)
- Find, fix and automatically rebuild a secure version of Python packages like Django and environments in minutes
The ActiveState Platform aims to handle every dependency for every language. That means handling libraries down to the C/C++ level, external tools, and all the conditional dependencies that exist. To take things even further, our ultimate goal is to support multi-language projects. That means that you can create a project using both Python and Perl packages, and we’ll make sure that both languages are using the same (up to date) OpenSSL version.
Dependency Management In Action
Get a hands-on appreciation for how the ActiveState Platform can help you manage your dependencies for Python environments. Just run the following command to install Python 3.9 and our package manager, the State Tool:
Windows
powershell -Command "& $([scriptblock]::Create((New-Object Net.WebClient).DownloadString('https://platform.www.activestate.com/dl/cli/install.ps1'))) -activate-default ActiveState-Labs/Python-3.9Beta"Linux
sh <(curl -q https://platform.www.activestate.com/dl/cli/install.sh) --activate-default ActiveState-Labs/Python-3.9Beta
Now you can run state install <packagename>. Learn more about how to use the State Tool to manage your Python environment.
Let us know your experience in the ActiveState Community forum.
You can try the ActiveState Platform by signing up for a free account using your email or GitHub credentials. Or sign up for a free demo and let us show you how you can better manage dependencies at your organization.
Watch this video to learn how to use the ActiveState Platform to create a Python 3.9 environment, and then use the Platform’s CLI (State Tool) to install and manage it.
