The Adventure of the Compromised Keys

The recent Commvault incident reveals something troubling about how we approach cloud security. It’s not just another breach story; it’s a case study in how our assumptions about SaaS safety can backfire.

Here’s what actually happened: Attackers infiltrated Commvault’s systems and accessed client secrets for their Microsoft 365 backup service, Metallic. This gave them a pathway into some customers’ M365 environments, specifically those using the backup service with application secrets managed by Commvault. While not every Commvault customer was affected, those who were found themselves in an uncomfortable position: their security had been compromised through a service they trusted to protect them.

CISA believes this incident may be part of a broader campaign targeting SaaS providers. The logic is compelling from an attacker’s perspective—why target individual companies when you can compromise a service provider and potentially access multiple customer environments at once?

This should give pause to anyone using SaaS services with deep integrations into their core systems. Which, let’s face it, describes most modern businesses.

The Trust Paradox

The fundamental challenge here isn’t technical. It’s architectural and philosophical. When we integrate with SaaS providers, we’re making a bet that their security practices will hold up under pressure. Sometimes that bet pays off. Sometimes it doesn’t.

Consider what happened with Commvault’s M365 backup customers. They made a reasonable decision: outsource backup management to specialists who presumably know what they’re doing. They granted Commvault the necessary permissions to access their M365 environments. Everything worked smoothly until it didn’t.

This creates an interesting paradox. The more valuable a SaaS service becomes to your operations, the more access it typically needs. But the more access it has, the more damage a compromise can cause. We’re constantly balancing convenience against risk, often without fully understanding where that balance point lies.

The security certifications and compliance frameworks that companies rely on provide some comfort, but they’re snapshots in time. They tell you a company had good processes when they were audited, not whether those processes will withstand a determined attacker six months later.

Beyond Shared Responsibility

The shared responsibility model that governs most cloud relationships isn’t wrong, but it’s more nuanced than most people realize. Yes, you’re responsible for your data and access controls, while your cloud provider handles infrastructure security. But when you grant a third-party service elevated access to your cloud environment, you’re creating dependencies that blur these clean lines of responsibility.

In the Commvault case, customers were managing their side of the responsibility equation properly, but they couldn’t control how Commvault secured the credentials needed to provide the backup service. When that credential management failed, the shared responsibility model didn’t provide much consolation.

This doesn’t make the model useless, but it does highlight its limitations. When you’re evaluating SaaS providers, the standard questionnaires and compliance certifications only tell part of the story. They don’t reveal how a provider manages the specific credentials and access they need to serve you, which is often where the real vulnerabilities lie.

There are some lower-effort alternatives emerging. Security rating services can provide continuous, external assessments of vendors without requiring lengthy questionnaires or internal audits. These services monitor publicly observable security indicators like SSL certificate management, DNS configuration, and known vulnerabilities to generate ongoing risk scores. SaaS Security Posture Management (SSPM) tools can automatically monitor your existing SaaS integrations for misconfigurations, excessive permissions, and policy violations.

Some organizations are also exploring simplified assessment frameworks. CAIQ-Lite, for example, reduces the traditional Cloud Security Alliance questionnaire from 261 questions to a more manageable subset focused on critical controls, allowing for more frequent reviews with less effort.

But these alternatives have their own limitations. External security ratings can’t see inside a provider’s systems to assess credential management practices. SSPM tools only help after you’ve already integrated a service. And simplified questionnaires, while more efficient, provide even less insight into the security practices that matter most.

The Reactive Response

CISA’s response to the incident follows a familiar pattern: monitor your logs more carefully, restrict access where possible, implement conditional access policies, rotate credentials regularly. For on-premises systems, they recommend restricting management access and patching the known vulnerability.

These are sensible recommendations, but they’re inherently reactive. They help you detect problems faster and limit damage when things go wrong, but they don’t address the underlying structural issues that make these incidents possible in the first place.

The challenge is that most organizations approach SaaS security as a checklist exercise. Review the vendor’s security documentation, check their compliance certifications, maybe run a basic security assessment, then hope for the best. This process works reasonably well for identifying obvious red flags, but it’s less effective at predicting how a provider will handle sophisticated attacks.

Thinking Differently About Integration

What if we approached SaaS integration differently? Instead of asking “Is this provider secure enough?” we might ask “How can we limit the impact when something inevitably goes wrong?”

This shift in thinking leads to different decisions. You might choose service accounts with more restrictive permissions, even if it makes the integration slightly more complex. You might implement additional monitoring specifically for SaaS-related activities. You might develop procedures for rapidly rotating credentials when incidents occur.

Several established frameworks support this approach. NIST’s Cybersecurity Framework emphasizes the “principle of least privilege” for access controls, while Microsoft’s Cloud Adoption Framework provides specific guidance for managing service principal permissions in cloud environments. For credential rotation, OWASP’s Application Security Verification Standard recommends automated rotation schedules and secure secret storage practices.

Zero trust architectures and proper secrets management can help, but they require upfront investment that many organizations are reluctant to make. It’s easier to rely on your SaaS provider’s security assurances than to build additional safeguards into your own environment.

The honest truth is that there’s tension between convenience and security, and that tension doesn’t disappear just because we’d prefer it to. Sometimes the most convenient integration option isn’t the most secure one, and vice versa. Acknowledging this trade-off is the first step toward making better decisions.

The SolarWinds breach offers a perfect example of this tension. Security researcher Vinoth Kumar discovered that SolarWinds had exposed FTP server credentials in a public GitHub repository from June 2018 to November 2019, with the password being simply “solarwinds123”. SolarWinds had also been advising customers to disable antivirus tools before installing their software, prioritizing ease of installation over security.

These weren’t technical oversights. They were deliberate choices that favored convenience. Using simple passwords makes system administration easier. Telling customers to disable security software eliminates support calls about installation conflicts. But these “convenient” practices created the vulnerabilities that sophisticated attackers later exploited in what became one of the most damaging supply chain attacks in history.

The Reasoning Problem

Here’s what troubles me most about how we handle these decisions: we often work backward from the conclusion we want to reach. Someone in marketing discovers they absolutely must have a new social media scheduling tool that connects to seventeen different platforms and stores all your customer data in a server farm somewhere in Eastern Europe. The security team then performs what can only be described as policy yoga, bending and stretching their “zero trust” framework until it somehow accommodates giving a third party administrative access to the company’s entire digital presence.

By the time they’re done, the original policy resembles something you might find at the bottom of a lawyer’s briefcase after a particularly bumpy flight. The tools get approved, the policies get “updated for modern business needs,” and everyone pretends this was a thoughtful risk assessment rather than an elaborate justification for something they’d already decided to do.

This isn’t malicious. It’s human nature. We want the productivity benefits that SaaS tools provide, so we find ways to rationalize their adoption. But rationalization isn’t the same as reasoning, and the difference matters when you’re making decisions about risk.

Proper reasoning would start with security principles and work forward to conclusions about specific tools. If your principle is “minimize third-party access to core systems,” then certain SaaS integrations become harder to justify. But instead of questioning the integrations, we often question the principles.

Looking Forward

The Commvault incident won’t be the last of its kind. As business processes become more dependent on SaaS tools, and as the connections between different services become more complex, these supply chain attacks will likely become more common.

The organizations that adapt successfully will be those that develop better frameworks for thinking about SaaS risk. This means going beyond compliance checklists to understand the actual security implications of the integrations they’re considering. It means building systems that can gracefully handle provider compromises. And it means being honest about the trade-offs between convenience and security.

The technical solutions exist: better access controls, improved monitoring, more sophisticated secrets management. The challenge is building organizations that consistently apply these solutions, even when it means accepting some friction in exchange for better security.

That’s not just a technology problem. It’s a thinking problem. And thinking problems require us to be more deliberate about how we reason through risk, rather than just hoping someone else has figured it out for us.

The question isn’t whether your SaaS providers will ever be compromised. They will be. The question is whether you’ll be ready when it happens.

Consider what happened in just the past few years: LastPass was breached twice in 2022, leading to over $150 million in cryptocurrency thefts as attackers cracked stolen master passwords. Okta, the identity provider that millions of organizations trust for access control, was compromised in 2023 when attackers used session cookies from customer support files to impersonate users and bypass multi-factor authentication. Microsoft’s own cloud services have been breached multiple times, including a 2024 incident where Russian hackers accessed senior executive emails using password spraying against accounts without two-factor authentication.

These aren’t small companies with weak security practices. These are the providers that the rest of us rely on to keep our own systems secure. LastPass had a dedicated security team and regular audits. Okta’s entire business model depends on security credibility. Microsoft spends billions annually on cybersecurity research and employs some of the world’s top security experts.

Yet they all fell victim to relatively straightforward attacks: password spraying, credential theft, and social engineering. The attackers didn’t need to discover zero-day exploits or deploy sophisticated nation-state malware. They succeeded using techniques that any competent cybercriminal group could replicate.

If companies whose core business is security, with virtually unlimited resources and the best talent money can buy, can’t prevent these basic attacks, what makes you think your backup service or productivity suite will fare better? Your typical SaaS provider has a fraction of the security budget, fewer dedicated security personnel, and faces the same fundamental challenges around human error and system complexity that brought down the big players.

The uncomfortable truth is that if LastPass, Okta, and Microsoft can be compromised, everyone can be compromised. The question isn’t whether it will happen to your providers. It’s whether you’ll have designed your systems to survive when it does.