Skip to main content

How AI-Powered Attacks Led Salesforce to Reinvent Hyperscale DDoS Defense

Herman Kwong
Aug 28 - 6 min read
How AI-Powered Attacks Led Salesforce to Reinvent Hyperscale DDoS Defense featured image

In our Engineering Energizers Q&A series, we spotlight the engineering leaders driving innovation across Salesforce. Today, we meet Herman Kwong, EVP of Software Engineering, whose team developed DREAM (DDoS Response and Mitigation), an AI-powered DDoS mitigation platform protecting Salesforce’s global cloud infrastructure against machine-speed attacks. DREAM reduced time to first mitigation by 5x across a shared cloud that processes roughly 20 trillion transactions annually.

Explore how Salesforce reinvented hyperscale DDoS defense by combining AI-driven inference, durable execution, and AI-assisted software engineering to respond to machine-speed attacks in seconds while protecting millions of customers across a shared global cloud.

What mission did the DREAM team set out to accomplish, and why was it so important for Salesforce?

The mission was straightforward: protect Salesforce’s distributed cloud against hyperscale DDoS attacks and respond in seconds. At Salesforce’s scale, that means protecting more than 4.5 million domains and 3 million customer orgs while maintaining 99.99% availability.

What makes this challenge unique is Salesforce’s shared, multi-tenant architecture. Even when attackers target a single customer, they can consume shared infrastructure in ways that affect many others. This is why fast, intelligent mitigation matters: protecting Salesforce means protecting every customer sharing that environment.

One attack can cascade across shared tenants; DREAM contains it.

That challenge has grown more urgent, driven above all by the rise of application-layer attacks, which are harder to defend than the network and transport layers, alongside attackers’ growing use of AI. Sophisticated attack tooling that once required significant expertise can now be generated and launched at machine speed. Human-operated defenses cannot keep pace because if you cannot respond in seconds, you are doing forensics, not defense. That reality shaped everything. DREAM was built around three guiding principles: defend Salesforce against hyperscale attacks, respond in seconds, and use AI-driven inference to identify evolving attack patterns before they spread across shared infrastructure.

What engineering challenge forced Salesforce to reinvent hyperscale DDoS defense?

The biggest challenge was that attacks outpaced human-speed response. Today’s attackers use AI to generate sophisticated attacks that evolve rapidly and execute autonomously. Increasingly, these attacks target the application layer. Defending a global cloud against those attacks requires decisions measured in seconds. On a shared platform, an attack targeting one customer can quickly cascade across shared infrastructure resources, creating risk for every customer running alongside them.

Building DREAM was fundamentally a distributed systems problem. The solution required execution across regions, fault tolerance, concurrency controls, and complete visibility across a globally distributed platform. Extending the existing platform was considered, but it was not designed to orchestrate machine-speed, AI-assisted mitigation across a globally distributed environment. The gap between the sophistication of modern attacks and what the legacy platform could coordinate ultimately made the case for building DREAM from the ground up.

How did your team rebuild a hyperscale DDoS platform in just over three months?

The deadline was not negotiable. The first-generation platform was approaching end of life, leaving engineers just over three months to rebuild the entire orchestration layer before existing licenses expired. This was not a prototype. It was a rebuild of the system defending that entire shared cloud, which meant earning production trust before the deadline arrived.

Two decisions made that timeline achievable. First, the team chose not to rebuild distributed systems primitives from scratch. Instead of spending months implementing state management, retries, and failover, DREAM adopted a distributed orchestrator, allowing the team to focus on the defense logic that differentiates the platform. Second, the team embraced AI-assisted software engineering. Using AI coding assistants, engineers migrated roughly 100,000 lines of legacy code at approximately ten times the previous development velocity. More than 90% of the rewrite was AI-assisted, not AI-unsupervised. The team established coding standards, taught the models Temporal’s programming patterns, enforced human code reviews, and used AI to generate unit tests that pushed coverage beyond 95%. The result was a working MVP in three weeks and a production-ready platform delivered ahead of schedule.

When designing DREAM, what engineering challenges led your team to adopt a distributed orchestrator?

The hardest architectural challenge was not choosing a workflow engine, rather, it was building a globally distributed mitigation platform that could coordinate thousands of long-running operations without losing state, even when components failed. Every mitigation involved telemetry, AI inference, policy deployment, notifications, and retries across systems that could fail independently. Hand-building the orchestration primitives like state management, retry logic, distributed locks and safe concurrency just to name a few, would have consumed months before a single line of defense logic was written.

With that cost in mind, the team evaluated three options: build distributed systems primitives from scratch, stitch together multiple technologies, or use a distributed orchestrator. The team chose the Temporal distributed orchestrator because it let engineers focus on solving the compelling security problems instead of rebuilding the underlying primitives. Its workflow-and-activity model cleanly separated deterministic orchestration from external side effects such as AI inference, telemetry, policy deployment, and notifications, making DREAM easier to scale, maintain, and reason about as it evolved.

From attack to mitigation in seconds with the DREAM workflow.

For an engineering team working against an immovable deadline, that distinction mattered enormously. Instead of reinventing distributed orchestration, the team invested its engineering effort where it delivered the greatest customer value: building faster, more intelligent DDoS defenses to protect Salesforce’s global cloud.

What production lessons shaped DREAM’s distributed systems architecture?

Production has a way of teaching lessons no design document can anticipate. Two of ours came down to the same line, between the system that coordinates the work and the data that work operates on.

The first surfaced as the team passed increasingly large telemetry datasets directly through the orchestration layer, handing them from one step to the next. As those datasets grew, the payloads outgrew a hard message-size ceiling in the transport carrying them between steps. Meanwhile, the workflows carrying them didn’t degrade gracefully, they failed outright. The fix was to keep bulk data out of the orchestration layer entirely: store the large dataset externally and pass only a lightweight reference in its place.

The second was subtler and far harder to catch. A long-running workflow accumulates a record of everything it has done, and that record must be reprocessed to rebuild its state whenever the work resumes on a different node. As those records grew, recovery slowed with them, gradually enough to pass testing, then surfacing in production as sporadic, hard-to-reproduce timeouts. The fix was to keep that record bounded: periodically carry forward only the state the next step needs and let the accumulated record reset, so a workflow’s recovery footprint stays small no matter how long it runs.

Both problems reinforced the same principle: the orchestration layer is a coordinator, not a data store. It exists to sequence work reliably — what has run, what comes next, how to recover — not to hold the data that work operates on. Keeping application data out of the coordination path has become one of DREAM’s core architectural principles.

As AI-driven attacks continue evolving, what engineering challenges will shape the future of DDoS defense?

The barrier to launching sophisticated attacks has collapsed. AI allows adversaries to generate adaptive attacks in hours instead of weeks, meaning defenses must evolve toward autonomous, machine-speed response. Speed alone is not enough, however. The real engineering challenge is building systems that remain trustworthy while making more autonomous decisions.

The philosophy behind DREAM is straightforward. AI should analyze attacks, identify patterns, recommend mitigations, and automate routine response decisions, while humans remain accountable for high-impact operational decisions. Platforms should orchestrate those complex workflows reliably at global scale. The team distilled that philosophy into a guiding principle: Humans ideate. AI creates. Temporal operates.

As attackers evolve, the work is never finished. Every improvement raises the bar for both defenders and adversaries. The challenge is to ensure Salesforce continues learning and evolving just as quickly as the threats it defends against.

Learn More

Related Articles

View all