GitHub Supply Chain Attacks Highlight the Urgency of Zero Trust SaaS Data Security

In early April, the tech industry witnessed a major GitHub security incident targeting GitHub organizations using Heroku and Travis CI. GitHub was made aware of this threat via an attack leveraging AWS API keys to GitHub’s own npm production infrastructure. As upstream security risks within SaaS platforms become more common, organizations that leverage these platforms are relying on tools like Nightfall to protect themselves. With that in mind, we wanted to take the opportunity to cover these security implications as well as discuss how Dividend Finance, a Nightfall customer, leveraged our platform to ensure security in the face of the uncertainty stemming from this supply chain attack.

What we know about the GitHub OAuth attack

This incident became public knowledge on April 15, when GitHub announced that an attacker was using stolen OAuth tokens issued to Heroku and Travis CI in order to download data from private repositories from a select list of organizations using these services. The hack was first spotted on April 12, when the AWS API keys connected to GitHub’s npm production were leveraged by an unknown unauthorized party. GitHub believes this API key was obtained by the attacker when they downloaded a set of private npm repositories leveraging either Heroku or Travis CI OAuth tokens.

GitHub immediately revoked these tokens when it discovered the intrusion, but as of now, it’s unclear what data, if any, had been accessed by the attacker(s). It’s unknown how long the threat actor(s) were in possession of these tokens, but with them, they would have had read and write access to any public or private repositories using Heroku or Travis CI.

GitHub’s “pattern of attacker activity” analysis highlights that:

1. The attacker gained access to the GitHub API using the stolen OAuth tokens issued to Heroku and Travis CI.

2. For most people who had the affected Heroku or Travis CI OAuth apps authorized in their GitHub accounts, the attacker listed all the user’s organizations.

3. The attacker then selectively chose targets based on the listed organizations.

4. The attacker listed the private repositories for user accounts of interest.

5. The attacker then proceeded to clone some of those private repositories.

GitHub has notified affected users directly, however, as is the case with supply chain attacks, anyone using a product or service that has recently been compromised must be vigilant for future abuse of their infrastructure and proactive in conducting their own investigations.

What are the risks surrounding this and similar attacks?

GitHub repositories can undoubtedly contain troves of sensitive information, be it proprietary code, customer data, and much more. A sample of GitHub security incidents from 2020 illustrates just how broad the contents of exposed repos can be. This is the core reason why, as we stated in our GitHub Security Webinar, threat actors have increasingly turned their attention to hijacking both public and private repos in order to leverage codebases as an attack vector. This generally involves using secrets contained within codebases to access even more sensitive systems.

‍

[youtube:f_vZHhVzhOY]

‍

How common are these types of attacks?

Much analysis has been done on attacks facing external repositories, especially those that developers might have accidentally made public through overly broad permissions. Threat actors have capitalized on this by developing easy-to-build and easy-to-deploy scanners that search public-facing GitHub for leaking PII, secrets, and credentials. 2016’s LinkedIn and Uber breaches provide a rather high-profile example of this tactic, which we illustrate in the video below. 

‍

[youtube:dmSk8X4pJgE]

‍

More recently, however, threat actors have moved beyond simply searching public repos for sensitive data and secrets. They are instead moving upstream and hijacking the platforms, tools, and services that integrate with repositories. Last year’s Codecov data exposure incident provides a sobering example of this.

The risk of supply chain attacks extends well beyond GitHub too, as illustrated by incidents like SolarWinds, Kaseya, and Okta.

How can organizations mitigate risks stemming from these types of attacks?

As an increasing number of organizations rely on cloud platforms and services, it’s important for security teams to develop ways to maintain visibility of their sensitive data, so they can properly enforce data security best practices in these environments. Because of the wide range of threats that can easily emerge from GitHub repository exposure, there are a number of development and security best practices to be aware of. The most important ones being: 

1. Train developers not to hardcode secrets or include business-critical data
2. Develop a formalized code review process
3. CI/CD process for review detection
4. Scan commits in real-time for secrets/sensitive data
5. Continuously review permissions of repositories
6. Review access logs to critical/sensitive repositories in your organization
7. Immediately rotate secrets that have been committed to your repos
8. Leverage cloud security tools that enhance visibility in cloud environments

Our CTO and Co-founder, Rohan Sathe, briefly goes over each of these points in the video segment below.

‍

[youtube:SmQwiKZG6A0]

‍

Practices like these form the core of zero trust data management and policy enforcement within SaaS environments like GitHub. Modern SaaS-first organizations like Dividend Finance leverage Nightfall to both enforce these best practices within environments like GitHub, and to monitor the integrity of the contents contained within these systems. We spoke to Marc Winner, CISO, at Dividend to better understand how Nightfall allowed him to determine what data was contained in his organization’s repositories, monitor for indicators of compromise, and respond effectively to the incident. Below is our abridged conversation with Marc.

How Dividend Finance leveraged Nightfall in response to the GitHub OAuth attack

Can you walk us through how you were impacted by this incident?

Marc: As a Heroku customer, we were part of the initial exposure that included the repositories of companies that use Heroku and GitHub to push changes to their Heroku environment. When incidents like these occur, one of the main concerns is determining whether any tokens or passwords were accessed by the attacker. Nightfall makes the answer to this question a definitive no, as we’re able to monitor our environments for embedded secrets that might be leaked or exfiltrated.

How did Nightfall help you address this incident?

Marc: The impacts and details of this incident were ambiguous. As is standard with breach notifications, Heroku suggested that someone may have accessed keys and secrets stored in our repositories and that we should monitor our logs for signs of data exfiltration, rotate any secrets or keys that we may be using, and disconnect Heroku from GitHub. 

This is good advice, but unfortunately we had no indicators of compromise, no indication of how long the threat actor(s) might have had access to our repositories, and really no sense of scope or scale of the type of threat we were dealing with. 

The Nightfall platform provided us with a history of who was utilizing our repositories and their commit history in terms of what types of content their commits normally contained. This helped provide a baseline of behavior that helped us determine if any anomalous activity was happening. Like, for example, if there were commits from new and unrecognized users, or if there were users suddenly making types of commits that they normally would not.

Would you recommend Nightfall to anyone facing similar threats? If so, why?

Marc: Absolutely. First, Nightfall helps ensure that we keep the proliferation of tokens and secrets to an absolute minimum, exactly for this reason. In this day and age, you simply shouldn’t assume that private cloud environments are inherently safe from snooping or exfiltration. So you need to know immediately if something that doesn’t belong in your environment has been added by an employee, in order to remove it as quickly as possible. This “housekeeping” is an essential part of preventing potential security incidents, like the Heroku breach, from becoming major disasters. 

Second. Nightfall provides detailed information about the contents of commits that have sensitive findings, as well as the users making these commits. This is useful for watching your security posture over time, in order to understand which employees are not following policy. But it was also useful for us during the Heroku incident, giving us peace of mind when it wasn’t clear if we’d personally been impacted by this breach.