Cloud computing puts enormous resources at your fingertips. With the stroke of a keyboard, you can tap into more storage and compute than a typical company could ever build in-house. This is incredibly useful, but cloud services can feel like black boxes. How do you know if your application performs as well as it can? And if it’s not, how can you find the bottlenecks? How do you know if your data is safe?
Datadog makes it simple to answer those questions with its cloud monitoring and security tools. Datadog gathers logs from servers, containers, databases, and other services to shed light on your entire infrastructure. You no longer have to guess how well your app is working: you can just check your Datadog dashboard.
The Datadog team likes to keep up-to-date with cutting-edge technologies and development practices. “It would be hard to help customers be more effective in their cloud platforms and DevSecOps practices if we’re not practicing that ourselves,” Chief Information Security Officer Emilio Escobar says. For example, the company uses Amazon Web Services, Google Cloud Platform, and Microsoft Azure for different parts of their operations. They use GitHub as an abstraction layer to make deploying to each cloud as simple and uniform as possible.
That’s just one way GitHub is central to Datadog’s work. Because Datadog’s software agent is open source, many of the company’s developers move seamlessly between public and private repositories throughout their day. “We’ve used GitHub from the inception of Datadog,” Escobar says. “It’s a high-quality product, and a lot of our engineers contribute to open source so there’s a sense of community there. GitHub is one of those products that it’s ingrained in the DNA of our engineering, it’s become part of the culture.”
Datadog is leveraging GitHub Advanced Security seamlessly into developers’ existing workflows, to ensure code is secure as applications are being built. Security is crucial to Datadog’s operations since they handle not only their own internal data, but sensitive data on behalf of customers. “The culture of the company has been built from communication and interactions where security is everyone’s responsibility,” Escobar says. “Whether you’re an engineer or you’re a product manager, you have to care about security, just like you have to care about the functionality and quality of the product.”
That said, there’s a strong need to avoid overburdening developers with too many tools or cumbersome security processes. “We prefer to have security that leverages what developers are already using rather than trying to force them to use some other tool,” Escobar says. “That feels interruptive and it always causes friction.”
We’ve used GitHub from the inception of Datadog. It’s a high-quality product, and a lot of our engineers contribute to open source so there’s a sense of community there. GitHub is one of those products that it’s ingrained in the DNA of our engineering, it’s become part of the culture.
Datadog also uses GitHub Actions to manage changes to “red zones”—particular parts of their products that have strict compliance requirements. Using Actions, Datadog’s security team can receive alerts when changes are made and track approvals for those changes. “Only about 2% of those changes require any sort of action, the other 98% are things that get closed because we either knew about them or there’s a good rationale for the change,” Escobar says. But this sort of individualized review doesn’t scale to all parts of the company’s codebase, so Datadog is increasingly turning to automated scanning for less sensitive repos.
False positives are a real worry for any scanning tools Datadog deploys, because they can reduce developers’ trust in the results. Developers might start ignoring a tool that produces too much noise. So Datadog is rolling out new scans gradually with tools like CodeQL to ensure quality results. So far, they’re seeing fewer false positives with CodeQL than with the other static analysis tools they’re evaluating.
Datadog is also exploring CodeQL as part of their provable security initiative. “We’d like to make facts about our security queryable through Datadog,” Escobar explains. “We might be able to put a query, such as the number of times a particular vulnerability has been introduced, into a dashboard.”
Datadog uses GitHub not just to improve their own internal security practices, but to help their customers as well. For example, Datadog partnered with GitHub to use Secret Scanning to find and notify customers who had accidentally published their Datadog API keys in public repos.
“We don’t automatically revoke keys because that could break things, but we can communicate with impacted customers and let them know they published something they probably didn’t mean to,” Escobar says. During the second quarter of 2021, Datadog detected over 250 leaked Datadog API keys. The service enabled Datadog to build an extra layer of trust with their customers by enabling them to remediate issues before they could lead to compromised data.
All of these efforts have enabled Datadog to use GitHub to “shift left” on security by weaving security efforts into the day-to-day work of developers, giving them immediate feedback that they can use to improve the quality and security of their code. It’s an approach that enables developers to fix problems up-front, instead of waiting weeks or months for a security review. “We want security to be collaborative,” Escobar says. “We want to work with engineers and not block productivity.”
Start collaborating with your team on GitHub
Want to use GitHub on your own? Check out our plans for individuals