Load Testing Behind Cloudflare, a CDN, or a WAF

Introduction

Most websites people want to load test now sit behind a content delivery network (CDN), a web application firewall (WAF), or both. Cloudflare, AWS WAF with CloudFront, Fastly, Akamai, and similar platforms protect your site from floods of automated traffic.

Unfortunately, a load test is a flood of automated traffic. It’s planned and well-intentioned, but from the firewall’s point of view it can look a lot like an attack. So it’s fairly common for a load test against a protected site to get throttled, challenged, or blocked partway through.

This guide is about letting your own planned load test through without weakening the protection your real visitors rely on. Everything here assumes you’re testing a site you own or are authorized to test, as Loadster’s Acceptable Use Policy requires.

Why a Load Test Trips CDN and WAF Protection

Three kinds of protection commonly get in the way of a load test:

• Rate limiting counts requests over a time window, usually per client IP address, and blocks or challenges clients that exceed a threshold. A load test runs many bots from a relatively small number of cloud instances, so each instance’s IP address sends far more requests than any human visitor would.
• Bot detection looks for signs that a client isn’t a person using a normal browser. Protocol Bots send HTTP requests directly without running JavaScript, so they can’t pass a JavaScript challenge. Browser Bots run real headless browsers, but they can still be flagged as automated.
• DDoS protection watches for traffic surges and responds by challenging or blocking the clients behind them. A test that ramps up quickly can look like the start of an attack.

None of this means the protection is wrong. It’s doing its job. The goal is to make an exception for traffic you planned and can identify.

What a Blocked Load Test Looks Like in Loadster Results

A load test that’s being blocked by a CDN or WAF usually has a recognizable shape in the test results:

• A sudden jump in errors partway through the ramp-up. Response times look healthy and then, once bots cross some threshold, errors climb quickly while response times stay low. That’s a firewall rejecting requests cheaply, not a server struggling under load.
• 403, 429, or 503 status codes. Cloudflare rate limiting rules respond with 429 Too Many Requests by default, and AWS WAF’s Block action responds with 403 Forbidden by default. Both can be customized, so the codes in your test might be different.
• Challenge pages instead of your content. A Protocol Bot might get back a small HTML page where it expected JSON or your normal markup. Cloudflare marks its challenge responses with a cf-mitigated: challenge response header, which you can spot in the request details of a failed step.
• Browser Bots timing out while waiting for elements. A Browser Bot stuck on a challenge interstitial never sees the button or form field your script is waiting for, so the step eventually times out.

Validation rules that check for text that only appears on your real page help here too. If a block or challenge page ever comes back with a 200 status, checking the status code alone would count it as a success.

Testing Through the CDN vs. Against the Origin

Before setting up exceptions, decide what you want to measure, because a CDN changes what a load test tells you.

Testing through the CDN sends requests to the same hostname your users visit. Cacheable content (images, CSS, JavaScript, and sometimes whole pages) is served from the CDN’s edge, and only cache misses and uncacheable requests reach your servers. This measures the experience real users have, and it’s the right test for most launches and traffic events. It also tells you whether your caching actually works under load. A page you thought was cached but isn’t will show up as origin load.

Testing against the origin bypasses the CDN and sends every request straight to your servers. This measures raw origin capacity: how much uncached work your application and database can do before they slow down. It’s useful when you’re tuning the backend, or when you want a worst-case number for a cache outage or a burst of cache misses after a deploy.

Many teams run both, but testing through the CDN is usually the one to start with. Keep in mind that the origin is often locked down to accept traffic only from the CDN, so testing it directly might need its own firewall exception.

For Browser Bots, testing through the CDN is especially realistic. Each bot loads every page resource the way a real browser does, so the test exercises the CDN with the same mix of requests your visitors generate.

Identifying Load Test Traffic With a Secret Header

The simplest and most flexible way to let a load test through is to have every bot send a request header that your firewall rule recognizes. It doesn’t depend on IP addresses, so it works from any cloud region and keeps working as Loadster launches fresh instances for each test.

Add a code block as the first step of your script to set the header for every request that follows. In a Protocol Bot script:

http.setGlobalHeader('X-Load-Test-Key', 'a-long-random-value-only-you-know');

In a Browser Bot script:

browser.setGlobalHeader('X-Load-Test-Key', 'a-long-random-value-only-you-know');

In a Playwright script, Playwright’s own page.setExtraHTTPHeaders() does the same thing for requests the page makes after it’s called, so call it at the start of the test, before page.goto().

In Browser Bot scripts, custom headers can sometimes stop the browser from using its cache. Each bot might then fetch page resources again on later pages, which adds requests to the CDN compared with a real visitor’s browser. Keep that in mind when you read the request counts.

Use a long random value rather than something guessable like true. Anyone can send any header they like, so the value is the only thing that makes the exception yours. Treat it like a password, and change it after the test.

Global headers go to every host the bots talk to, including third-party domains your pages load. If you’d rather not send the secret to anyone else, Protocol Bots and Browser Bots also have a setHostHeader(host, name, value) method that limits a header to one domain.

Allowing a Load Test Through Cloudflare

In Cloudflare, the usual approach is a WAF custom rule that matches your secret header and uses the Skip action. A skip rule lets matching requests bypass security features that would otherwise block or challenge them, without changing anything for other traffic.

The skip options include All rate limiting rules, All managed rules, and All Super Bot Fight Mode rules, plus individual features such as Browser Integrity Check and Security Level. For most load tests, skipping rate limiting rules and Super Bot Fight Mode rules is the important part. Skip only what’s actually getting in the way of the test.

A few Cloudflare features need extra attention:

• Bot Fight Mode (the free plan’s bot protection) can’t be skipped. Cloudflare’s docs say you cannot bypass or skip Bot Fight Mode using WAF custom rules, because it runs in a separate pipeline. It doesn’t trigger when an IP Access rule matches the request first, but that only helps when you know the bots’ IP addresses. Otherwise, you might need to turn Bot Fight Mode off for the duration of the test.
• I’m Under Attack mode shows every visitor a JavaScript challenge interstitial. Protocol Bots can’t pass it. Leave it off during a planned test, unless rehearsing with it on is the point.
• IP Access rules can allowlist an IP address, and an allowed IP bypasses custom rules, rate limiting rules, and managed rules. Cloudflare recommends custom rules instead for most IP-based rules, and an IP rule is only practical if your load generators have fixed addresses.

Allowing a Load Test Through AWS WAF and CloudFront

AWS WAF evaluates the rules in a web ACL from the lowest numeric priority up until a rule with a terminating action matches. Allow is a terminating action, so the common approach is a rule that matches your secret header with the Allow action, placed above (at a lower priority number than) your rate-based rules and managed rule groups. Requests carrying the header are allowed before any rule that would block them gets a chance to run.

If you’d rather keep your other rules in force for test traffic and only exempt it from rate limiting, a rate-based rule can take a scope-down statement. The rule then counts and limits only requests that match it, so a scope-down statement that excludes requests carrying your header leaves the test traffic alone.

Rate-based rules can also aggregate by an IP address in a forwarded header such as X-Forwarded-For, not only the source IP. If your load balancer or another proxy sits between the bots and the web ACL, check which address the rule is actually counting.

AWS WAF can sit in front of CloudFront distributions, Application Load Balancers, and API Gateway stages, and the rules work the same way in each case.

Load Testing Behind Fastly, Akamai, and Other CDNs

Other CDN and WAF platforms have their own names and settings for the same ideas: rate limits, bot management, and rules that allow or exempt specific requests. Look in your provider’s documentation for a way to exempt requests that carry a specific header, and check which bot or DDoS protections sit outside those rules (as Bot Fight Mode does on Cloudflare).

If your provider can only allow traffic by IP address, see the next section.

Allowing Load Test Traffic by IP Address

A header is usually easier, but sometimes a firewall can only make exceptions by IP address. Loadster has a few options, all covered in more detail in the FAQ on whitelisting load test IP addresses:

• Self-hosted engines. Run your own Loadster Engine on machines with IP addresses you choose, and allow those addresses.
• Cloud provider IP ranges. Load tests in Loadster Cloud run on instances launched on demand, so their addresses aren’t known ahead of time. You can allow the published AWS or GCP IP ranges for the regions you test from, with the obvious drawback that those ranges are huge.
• Script players for playing and monitoring. Playing a script in the editor and running monitors use shared script players with fixed IP addresses, shown in the Script Player dropdown in your settings. A monitor might fall back to AWS Lambda if a script player is unavailable, as described in where script traffic originates.
• A fixed IP cluster. If nothing else works, Loadster can set up a cluster with a fixed IP address in a region of your choice. There’s typically a setup cost and some lead time.

Coordinating the Load Test With Your Firewall Owner

The person running the load test often isn’t the person who manages the CDN or firewall. A little coordination goes a long way:

• Agree on the window. Tell whoever owns the firewall, and whoever is on call, when the test starts and ends. A load test that nobody expected can easily be mistaken for an attack and trigger an incident response.
• Scope the exception. Limit the rule to the hostnames under test, and skip only the protections that block the test. Leave the rest in place.
• Test the exception first. Run a small test with a handful of bots and confirm the requests come back with your real content and status codes before you ramp up.
• Turn it off afterwards. Disable or delete the rule when the test is done, and rotate the header value before the next test. A forgotten bypass rule with a leaked secret is a hole in your protection.

Conclusion

CDNs and WAFs block load tests because load tests look like the traffic they’re designed to stop. Rather than turning protection off for everyone, have your bots send a secret header and make a narrow, temporary exception for it, or allow specific IP addresses when the firewall can only work that way.

Decide up front whether you’re testing through the CDN, which reflects what your users experience, or against the origin, which tells you the capacity of your own servers. Either way, a short test with a few bots confirms the exception works before the real test begins.

Next Steps: Ready to test your protected site? Try Load Testing with Loadster to run Protocol Bots or real browsers from cloud regions around the world, with a code block to send your firewall’s secret header.

Related Guides

• Website Load Testing — how to plan and run a load test against a website.
• Load Testing, Stress Testing, and DDoS — how a planned load test differs from an attack.
• Load Testing Best Practices — guidelines for getting useful results from any performance testing effort.
• Spike Testing — rehearsing a sudden surge, which is when rate limits are most likely to kick in.