Python coding is an awesome experience. Python deployment? Not so much. It’s not quite Hell, and things do get moving eventually rather than get stuck in Limbo, so let’s just call it “deployment heck.”
Many of the pains associated with Python deployment are just growing pains: Python has suddenly become one of the most popular programming languages on the planet (primarily due to its data science and machine learning capabilities), and the cracks are beginning to show.
Python Peculiarities
For example, some of the Python deployment shortcomings we’ve seen our customers struggle with include:
- Package Naming – when DevOps receives a package from the dev team labeled “BS4” it’s unclear (without a knowledge transfer) that what the label is actually referring to is BeautifulSoup and not Boundlessgeo-Schema or some other package.
- Package Versioning – semantic versioning is considered a best practice, but it’s not enforced. The community is free to release a package versioned “0.9999999999Beta” and then follow it up with version 0.9, which appears (out of order) to be a minor release, but may actually make a breaking change – something that would be expected only in a major release.
- Environment Building – when DevOps tries to build their environments, they may need to compile in all of the packages used by multiple teams. If those teams are doing data science, many of the computational packages will have dependencies on C and C++ libraries, requiring the use of a C compiler which may have its own dependencies that need to be resolved. Resolving all these dependencies can be a real time suck. Even if it works out of the box, it requires longer to build than you might want at deployment time.
The net result is a lot of churn and rework time – time that DevOps teams rarely have the luxury of.
Docker: Panacea or Pandora?
To help mitigate these issues, enterprises have moved to encapsulating their Python distribution in a Docker container. This is a significant investment as the team must rework all their tooling and processes to handle deployments in this format. The upside is that this creates a standardized Python build that any team can just plug into their environment and start working with right away. But it doesn’t actually solve any of the fundamental problems. Rather, the pain is just pushed to the build team, who are now tasked not only with resolving all the Python issues listed above, but also Docker issues, as well.
For example, every Docker container requires an operating system. Two common choices are:
- A robust, popular OS like CentOS, which will result in a larger container, but also provide most of the OS libraries your code will need, or
- A slimmed down OS like Alpine, which will create a much smaller container, but you’ll need to manually resolve all the missing libraries.
Since most enterprises prefer smaller Docker containers, the net result is (once again) a lot of churn and rework, especially as teams request new third-party packages to be added to the standard build.
Size Matters
Deployment size can also be an issue for enterprises who are not using Docker. In this case, they want to ensure their production releases have the smallest footprint possible. This provides a host of benefits including smaller artifacts to transfer around, increasing speed of testing and deployments, and a reduced attack surface. On the other hand, OEM vendors want to ensure they ship their customers a solution with the smallest footprint possible, because it reduces the number of components they need to support. In these cases, the tradeoff is really between using a static version of a Python build versus a dynamic one:
- Static Python Build – provides consistency across environments by statically including all dependencies, but also results in a very large Python build
- Dynamic Python Build – results in a much smaller deployment, but you need to ensure that system libraries are available across all your environments or things will break
Summing Up
So where does that leave you? If you’re like most enterprises, you just suck it up and get on with things. But there is a better solution on the horizon, a solution that ActiveState is driving towards with your help. Come help us define the ActiveState Platform to ensure it’ll address your requirements, as well.
Create a free account today to try out the ActiveState Platform!
Learn more about how self-managed Python compares against a commercial offering or get the summary infographic.
