In the past couple of years we have seen some of the biggest security issues in open source, including Heartbleed, Shellshock, and POODLE. More recently, in March of this year, we faced the cross-protocol security bug DROWN. The impact of these security holes has been far-reaching–millions of users and sites were affected. Even a year and a half after the Heartbleed fix, over 200,000 devices were still at its mercy. Two guys named Steve can only do so much.
These events represented a tipping point: as enterprises increasingly rely on open source for running their businesses, it has become evident that they can’t rely on the security associated with it. We see enterprises that are still trying to wrap their heads around old security issues even as new ones continue to emerge.
The hope that “with enough eyeballs all bugs are shallow” or easy to find has proven false. For one thing, the most motivated eyeballs are the ones wearing black hats: a small number of dedicated hackers are more likely to find security holes than a large community of casual onlookers, which can’t provide the assurances enterprises need. While recently Mozilla has started the Secure Open Source Fund in order to help finance activities such as proper security audits for open source projects, the reality is that many issues will continue to catch us by surprise as zero-day exploits are discovered in the wild.
And this is where we come in at ActiveState.
28 Security Patches in 11 Years
At ActiveState, security is paramount. Since 2005 we have applied 28 security patches to ActivePerl–over half of them in the past few years as the war between security researchers and hackers has heated up. We have also made many more routine updates to OpenSSL and other security-critical libraries as provided by the community. In the past three years alone we have applied patches for the following 15 Common Vulnerabilities and Exposures:
CVE-2016-0800: Cross-protocol attack on TLS using SSLv2 (DROWN) (HIGH)
CVE-2016-0705: Double-free in DSA code (low)
CVE-2016-0798: Memory leak in SRP database lookups (low)
CVE-2016-0797: BN_hex2bn/BN_dec2bn NULL pointer deref/heap (low)
CVE-2016-0799: Fix memory issues in BIO_*printf functions (low)
CVE-2016-0702: Side channel attack on modular exponentiation (low)
CVE-2016-2381: Context-dependent attack to bypass taint protection (medium)
CVE-2015-3193: BN_mod_exp may produce incorrect results on (medium)
CVE-2015-3194: Certificate verify crash with missing PSS (HIGH)
CVE-2015-3195: X509_ATTRIBUTE memory leak (medium)
CVE-2015-3196: Race condition handling PSK identify hint (medium)
CVE-2015-1794: Anon DH ServerKeyExchange with 0 p parameter (medium)
CVE-2015-8608: out-of-bounds read and over-read vulnerabilities
CVE-2014-0160: Heartbleed (medium)
CVE-2014-0076: Cache Side-channel Attack (low)
In ActivePython there have been 20 CVE patches since 2007, along with routine updates of modules and OpenSSL. Right now we ship with the OpenSSL version as defined by the Python community. In ActiveTcl, we routinely update OpenSSL and the TLS module.
Facing the Security Challenge
As you can see from the list above, there is a big ramp-up in security after Heartbleed, which was also around the time Shellshock hit bash. The combined events raised everyone’s security awareness. The challenge enterprises and developers face is that they need to keep filling the security holes as they are discovered, which is one more thing on somebody’s already full plate. Fortunately for our enterprise customers, they don’t have to think about that–we handle it for them.
For example, when the HeartBleed bug hit, a number of enterprises were on an older version of ActivePerl, ActiveTcl and ActivePython. We were immediately able to deal with that very rapidly, in less than 24 hours, providing “peace of mind”. The open source Perl, Python, and Tcl communities also resolved the issue, but in a longer time frame.
At the same time, there is a balance to be struck between security and stability. One person’s security hole is another person’s feature, and almost any feature can be used in insecure ways, so there is a role for judgement regarding how important the insecure versus secure uses are. To take an extreme example, the C heap is clearly a major source of insecurity in the language: buffer overflows and the like are routinely at the root of C security issues. But a “security patch” that eliminated malloc from C wouldn’t be the right fix.
More realistically, there is currently an ongoing discussion in one of our language communities regarding a particular problem and fix where the risk of application breakage from the simplest fix is significant. We keep a careful eye on the debate around proposed fixes so we can make a well-informed judgement when the patches ship.
Our General Security Policy
Security is not a simple linear scale of severity because any security fix has both the potential to break existing functionality (some applications may depend on the behaviour that is being changed to fix the security issue) and the potential to introduce new bugs and security issues. Therefore ActiveState does not simply commit to shipping security updates based on CVE rating alone, but evaluate them based on three criteria:
- CVE/NVD CVSS value
- Usefulness (how likely it is that the problematic behaviour is important to Enterprise applications)
- Risk (how likely it is that the fix has introduced a new issue)
The latter two criteria are both somewhat subjective and focused on ActiveState’s enterprise customer base. We aim to fix CVE issues with CRITICAL ratings within 24 hours of their public announcement regardless of their usefulness or the risk of additional bugs. These fixes will be applied to the most recent versions of our Business Edition and Community Edition offerings, and all Enterprise builds. For vulnerabilities with ratings of HIGH or below we take a more measured approach: HIGH vulnerability issues will be fixed in the next regular release of our latest Business Edition offerings unless there have been reported issues with the fix in terms of usefulness or risk. Our Community Edition builds are always updated at the same time as our Business Edition builds. Older Business Edition builds may not be fixed, as patches are typically incompatible more than a few versions back. For our Enterprise Edition customers, HIGH vulnerability issues will usually be fixed more promptly unless there is a significant impact on usefulness or we deem the fix risky in other respects.
Security is ALWAYS an Issue
While security will always be a big issue in software, for enterprises it has a critical impact on their business and one that they can’t leave to chance. Taken together, Heartbleed, ShellShock, and other major security holes marked a turning point in open source software security. The new direction involves more formal inspection and testing, more aggressive patching, and more resources invested. ActiveState is continuing to enhance the security of our language distributions and looking for ways that we can contribute to the ongoing process of keeping the world’s open source infrastructure safe.
Contact us to learn more about how we can help you with your security issues or regarding our commercially supported language distributions for Perl, Python, and Tcl.
