- Managing dependencies and resolving conflicts by hand
- Dealing with environment reproducibility and “works on my machine” issues
- Maintaining and updating multiple tools and build environments
We love pip, as well as the myriad of other Python package managers that have been introduced to tackle the problems listed above. It’s just that they were never designed for modern Python development at scale. In this post, I’ll walk you through how we’ve fixed these headaches with the ActiveState Platform, but the TL;DR is:
- Dependency management just got a lot easier – even for the hard cases
- No complex setup or installs – builds are done from source in the cloud
- Cross platform – same tools and commands on Linux and Windows (macOS soon)
- Vulnerability remediation – find and fix vulnerabilities, even for transitive dependencies
- Advanced features – virtual environments, versioning, rollbacks, branches, and more
You can try it out for yourself with our Python 3.9 offering while you’re reading by running the following command:
Why Use the ActiveState Platform for Python package management?
System-wide installations of Python are still widespread. However, best practices are moving away from global installs which are prone to corruption. Instead, the trend is toward virtual environments that isolate each instance of Python, allowing you to run multiple versions on your local machine without conflicts. This best practice is now an integral part of the ActiveState Platform, which provides you with:
- Python 3.9 that is automatically installed into a virtual environment
- Our command line interface (CLI) the State Tool, which allows you to “state install” all the packages your project requires
Rather than a revolution, Python 3.9 from ActiveState is an evolution of the Python ecosystem that offers you a number of features that can make your life as a developer less frustrating and far more productive. For example:
Resolve Dependency Conflicts
There are a few Python package management solutions, that will resolve dependencies for you, ensuring your environments contain only packages that will work together. However, there is currently no way to resolve dependency conflicts other than manually testing out various versions of packages to find a compatible solution. Sometimes the solution is fairly straightforward, but in other cases developers waste time and resources in dependency hell. The ActiveState Platform includes a dependency solver, that not only automatically resolves dependencies for you, but also flags conflicts AND is unique in providing you with a way to solve them. For example:
While the error message is a bit verbose (for good reason: we’ll soon support multi-language projects, which will need to distinguish conflicts by language), it’s still easy to follow and resolve. Simply following the instructions and changing the version of scipy to be >=0.19.1 (ie., by clicking on the dropdown and selecting a more recent version) would solve this conflict, eliminating dependency hell.
Automatically Build Python from Source Code
Building Python and third-party packages from source code increases security because it ensures:
- You won’t accidentally install malicious code inserted into the binary version of a package, as might happen with package typosquatting
- You know the origins of all the components in your open source code, helping secure your open source supply chain
But most developers prefer to install binary versions of packages instead, accepting the security risk because building from source is a pain, especially if you need to support multiple deployment platforms. That’s why the ActiveState Platform provides:
- A single, cloud-based build platform for both Windows and Linux
- Automated building of not only Python code, but also any linked C libraries, as well
So there’s no longer any excuse not to build from source. And best of all, you no longer need to maintain build environments on your local machine.
Python Environment Reproducibility
A requirements.txt or pipfile.lock file goes a long way to helping you consistently reproduce your environment on other systems, but they don’t address all issues, including:
- Deploying to a different operating system may miss OS-level dependencies
- Requirements.txt and pipfile.lock files can get out of sync as different developers on a team update their development environment for their own purposes
- DevOps is often left to resolve multiple conflicting code check-ins, leading to CI/CD environments that differ from development environments
All of which lead to developers wasting time rebuilding environments in order to debug issues. The ActiveState Platform takes a different approach to ensuring reproducibility by:
- Automatically building your Python environment for both Windows and Linux
- Providing a central “source of truth” for the Python runtime environment that both developers and DevOps can pull to build their dev and CI/CD environments
- Updates to the environment are made centrally, and can be updated locally with a single command.
Centrally managing and deploying your Python and Perl environments means that development and DevOps teams remain in sync, eliminating “works on my machine” issues.
Python Vulnerability Remediation
Developers spend far too much time investigating and remediating open source vulnerabilities to the point where the Mean Time To Resolution (MTTR) is typically measured in weeks. Obviously, everyone would prefer to be able to resolve critical vulnerabilities in a matter of days, if not hours, but our current tooling doesn’t help much beyond notifying you of a vulnerability and perhaps suggesting a solution. The ActiveState Platform goes a step further in providing you:
- Notifications
- Suggestions for upgrading or even downgrading a package to a version that is secure
- Showing you the ramifications of making the change on the rest of your environment BEFORE you commit to it
- Automatically rebuilding the environment
Spend far less time and effort resolving vulnerabilities, so you can get back to coding.
Python Package Management that Works Faster and Smarter
The ActiveState Platform also provides a number of other features that can help improve your productivity, including:
- All packages are built from source code in parallel, or pulled from cache, where possible, speeding up environment creation
- Automated installation of Python into virtual environments without requiring the previous creation of the virtual environment
- Cross-platform support: a single toolchain for working with Python on both Windows and Linux so you no longer have to manage multiple tools
- Ability to automatically revert to and restore previous versions of your environment, even if transitive dependencies are no longer publicly available
- Ability to create branches of your environment that can inherit changes from the parent. Now you can finally see and manage (for example) Development vs Test vs Production environments, all in one place:
Conclusions: The Future of Python Package Management
If you’re like most Python developers, you probably maintain multiple tools, build environments and other solutions in order to address the issues that a single tool, the ActiveState Platform can solve today. By adopting the ActiveState Platform, you can:
- Increase the security of your Python environments
- Improve the transparency of your open source supply chain
- Eliminate dependency hell
- Reduce “works on my machine” issues
Ultimately, developers that are willing to adopt the ActiveState Platform for Python package management will spend less time wrestling with tooling and more time focused on doing what they do best: coding. Ready to give it a try? You can install Python 3.9 and our CLI, the State Tool by running the following command:
