DepSec: How DevOps Can Secure Open Source at Scale

DepSec Automation

Open source Dependency Security (DepSec) is key to the practice of ensuring the security of any application, but it rarely gets the focus, time and resources it requires. Instead, it’s lumped into the more well known AppSec practice, and treated as a second class citizen despite the fact that open source is typically >80% of any codebase. 

Securing the open source supply chain is a difficult task since it is both wide (e.g., tech stacks typically include multiple open source languages) and deep, because the open source supply chain extends from the import routine through the Continuous Integration (CI) build system to an artifact repository. 

As a result, every ISV is in the software supply chain business whether they know it or not. But only the very largest ISVs can afford the time and resources to implement a secure open source supply chain at scale. Everyone else is stuck cobbling together point solutions (SCA tools, repositories, build tools, binary scanners, and so on) just to be able to compete. 

But the results speak for themselves. Despite the fact that ISVs will spend ~$215B on software security in 2024:

  • Software supply chain attacks are up 633% YoY resulting in $46B in damages.
  • 81% of organizations ship applications with known vulnerabilities.
  • 91% of organizations experienced a software supply chain attack in 2023. 

Simply put, poor security practices when it comes to open source dependencies expose organizations to software supply chain threats, but doing DepSec at scale can be difficult because: 

  • Observability is Challenging – understanding where open source dependencies are deployed and who/which project is using them can be complex since no single tool provides a comprehensive view.
  • Ensuring Compliance is Complicated – verifying that open source dependencies are fit for use, as well as making sure they meet vulnerability risk thresholds can be hard to do at scale.
  • Making Updates is Complex – dependency hell is real! But more commonly, updating even a single dependency results in a cascade of changes throughout the dependency tree, threatening stability and resulting in breaking changes to your application.

The team that bears the brunt of all this is typically DevOps who have to figure out:

  • What open source software are we actually using? (as opposed to what’s been sitting in the artifact repository forever)
  • Where is it deployed? 
  • Is it secure?
  • Does it comply with company policy?

And so on. But DevOps already has a lot on their hands. Automation is the answer for any organization that wants to scale up their existing processes to include multiple open source ecosystems, remediate vulnerabilities at scale and tame open source complexities. 

How to Automate Dependency Security

The solution starts with Observability, which begins with a comprehensive view of all the open source software deployed in the organization. After all, you can’t get your arms around the problem if you don’t know what you have. 

Most organizations prefer to import prebuilt binary open source components, and therefore use binary scanners in order to create a manifest of open source software. Unfortunately, no two binary scanners are likely to generate the same manifest since they essentially use reverse engineering to generate an approximation. 

While some organizations have resorted to adopting Application Security Posture Management (ASPM) tools to help identify the differences between multiple inputs, do you really need yet another tool to resolve the issues your existing tools generate?  

Instead, ActiveState builds all open source dependencies from source code, including all transitive and OS-specific binaries (e.g., C libs). As a result, there is no ambiguity when it comes to understanding which versions of what open source are deployed in which projects across your organization. 

Ensuring Compliance needs to be a key component of any CI/CD process. While most organizations are already checking open source component vulnerability status against corporate policy, newer metadata can provide further security validation. For example:

  • Open source runtimes should be verified against a Software Bill Of Materials (SBOM) to ensure that no unexpected components have been injected into container builds.
  • The provenance and integrity of each open source component should be verified against a Provenance Attestation to ensure it has been sourced and built in accordance with corporate policy.

ActiveState not only securely builds open source dependencies from vetted source code, but also generates both SBOMs and Provenance Attestations to prove it. These metadata artifacts can be pulled directly from an API into the CI/CD process, and used to verify the security and integrity of each open source dependency. 

But at some point during the development process (or post deployment), one or more vulnerabilities that affect your application will inevitably be found. When that vulnerability poses an exceptional risk, Mean Time To Detection (MTTD) and Mean Time To Remediation (MTTR) are critical, especially if your application is used by government agencies or subject to regulatory SLAs. 

While most organizations already have tools deployed to help decrease MTTD when a vulnerability is found, the effectiveness of those tools is only as good as the limitations of their binary scanner to provide a comprehensive picture of what’s deployed. 

If the vulnerability needs to be addressed, it commonly means updating the runtime environment with a new version of the affected dependency, which is when Update Complexity can become a significant problem due to:

  • Dependency Hell, which arises because a single change to a dependency can result in multiple dependency/ transitive dependency updates, one or more of which may be mutually incompatible. 
  • Breaking the build, since newer versions of a dependency may introduce breaking changes to the way existing code has been implemented.
  • Redeployment of the updated code, which can be delayed by:
    • The need for extensive manual testing.
    • The need to wait for a deployment window in order to avoid disrupting existing customers/users.
    • The need for interdependent components, such as third-party APIs, services or applications to be updated prior to redeployment.

ActiveState can help minimize some of the work and much of the pain by providing the ability to:

  • Select a non-vulnerable version of the affected dependency from our continually updated catalog.
  • Automatically resolve dependencies/ transitive dependencies for you, and allow you to review the changes BEFORE you rebuild/redeploy so you can understand the ramifications.

Impact Report

  • Automatically rebuild the runtime environment in minutes for all target platforms so you can pull it directly into your CI/CD process for validation. 

Conclusions – DevOps Dependency Security Done Right

DevOps are all too often expected to wave their magic wand and deliver a build, despite a range of issues that they just “suck up and get done” including: 

  • Environment mismatch bugs between developer desktops and container runtimes. 
  • Urgency overload when critical vulnerabilities need to be remediated quickly, or broken build runs need to be fixed.
  • Automating everything, which takes time to do properly at scale.

But in “just getting it done” DevOps tends to address the symptoms, rather than the root cause. This often comes down to stitching together point solutions, which fail to scale when you need to support multiple operating systems, open source languages and environments, some of which may have extensive restrictions.

When open source is >80% of the codebase, all these issues become that much more difficult if you can’t:

  • Identify all the open source dependencies in use
  • Understand whether each dependency is fit for use
  • Get notified and fix vulnerabilities in a timely manner

ActiveState provides an open source management platform that is designed to integrate with typical software development processes in a matter of days, allowing you to automate key processes in order to:

  • Verify that all dependencies have been sourced and built securely
  • Identify where all dependencies are deployed, and their current security status
  • Decrease MTTD and MTTR for vulnerabilities

Next Steps

Automating DepSec is the key to allowing DevOps to manage the observability, compliance and remediation of open source at scale. Read more about the evolution of DepSec as a distinct practice within AppSec.

Recent Posts

Scroll to Top