Testing a WAF: Why a Block Page Is Not Enough
How client responses, gateway decisions and independent backend observations combine to verify WAF outcomes, including failures and inconclusive tests.
A test client receives a block page. That is useful evidence about the response it saw. To establish that a protected application did not receive the request, however, the test needs an observation of the backend path as well.
This distinction is particularly important when a security requirement concerns delivery, side effects or a failure transition. A client-facing status code and an application’s receipt of a request are different facts. A useful WAF test makes both observable and relates them to the same attempt.
Define the expected outcome first
Start with a test case that states the intended behaviour. For a permitted request, the expectation might include a successful client response and confirmed backend receipt. For a prohibited request, it might require a particular enforcement action and no delivery to the backend.
Keep routing and inspection requirements separate. A route can correctly forward a protocol without demonstrating inspection of every application message carried over that protocol. State which part of the interaction is under test: request headers, a payload, an upgraded connection or a particular message type.
The expected result should also identify the configuration being evaluated. Otherwise a suite can keep passing after its intended scope has changed.
Observe the backend independently
A controlled backend can record whether it received an identified request, along with a bounded set of relevant attributes. This is stronger than inferring backend behaviour from the WAF’s own response template.
Use a unique test identifier and keep the observation channel separate from the route being tested. The observation service needs an access boundary of its own: it should not become an Internet-accessible diagnostic endpoint or let arbitrary client data rewrite the evidence.
When a request is expected to be blocked, establish that the backend observer was available and its observation window was complete. The absence of a log entry can also result from a broken collector, a mismatched identifier or an incomplete search. Those cases need an inconclusive result rather than a security pass.
Distinguish failure from missing evidence
A test result needs enough states to describe what happened:
| Situation | Interpretation |
|---|---|
| Expected client outcome and complete matching backend evidence | The stated checks can be evaluated. |
| A prohibited request is recorded by the backend | The no-delivery requirement failed. |
| A permitted request never reaches an available backend | Investigate routing, policy or availability failure. |
| The backend observation service is unavailable | The backend assertion is unobserved or inconclusive. |
A concise report should show the expected result, the observations obtained and the assertion that failed or could not be evaluated. Turning every missing observation into “not received” creates false confidence.
Correlation is a tool for assembling evidence, not a substitute for its trust boundary. An identifier that matches across two records is useful only if the records actually describe the intended request and cannot be forged through the test path.
Include state and failure transitions
Normal requests exercise a small part of the platform’s behaviour. Add cases for expired decisions, unavailable state services, restart, recovery and conflicting inputs. For a one-use challenge pass, concurrent reuse and replay are separate tests from a successful first submission.
Check what reaches the backend during these transitions. A service may correctly deny after detecting an outage while briefly admitting traffic during startup. A test that begins only after readiness becomes healthy can miss that interval.
Be explicit about degraded modes. If an advisory source may be unavailable, test the policy intended for that condition. If an enforcement-critical source is unavailable, test the operation that must remain prohibited. “Fail closed” becomes assessable when those operations and observations are named.
Combine repeatable validation with exploration
Deterministic regression cases check whether defined inputs and conditions produce the expected outcomes for a known version. Exploratory security testing seeks weaknesses outside those predefined cases. Load testing examines behaviour under resource pressure. A separately scoped penetration test can challenge assumptions across the deployed system.
These activities complement one another. A large regression suite may omit an entire class of interactions; an exploratory finding may become a new regression case once its expected outcome is understood. Performance results also need their configuration and test conditions, rather than an isolated requests-per-second number.
NIST’s technical testing and assessment guide provides broader context on planning assessments and interpreting the benefits and limits of testing techniques.
Keep evidence attached to the deployed version
Record the relevant software, policy and configuration versions with the test run. Preserve the test definition, observation method and result. This makes a later change review more useful: the reviewer can see which property was previously exercised and whether the new change affects its assumptions.
This is part of the assurance work behind Sovereign Fortress and our SELinux Firewall project. A meaningful result explains what was required, what was observed and which conclusion the evidence supports.
References
- NIST SP 800-115: Technical Guide to Information Security Testing and Assessment.
- OpenTelemetry: context propagation, background on correlation across services.
- Sovereign Fortress: assurance through evidence.