Product · CloudNerve
RAG Assurance
Test that your AI assistant shows each user only what they may see, and forgets what you delete.
Four test suites for retrieval-augmented generation on Amazon Bedrock Knowledge Bases or any HTTP RAG endpoint. Run them nightly in your account or in your CI pipeline, with signed results.
Who it's for
Teams running an internal assistant or a customer-facing RAG feature on Amazon Bedrock, and teams moving off Amazon Kendra or Amazon Q Business.
When teams call us
A security review asking whether one department can read another's files through the assistant, a GDPR erasure request, a model or chunking change before release, or a Kendra migration.
What it does
Inside RAG Assurance
Access-control leak tests
Test identities and canary documents, probed with direct, paraphrased, prompt-injection and enumeration queries.
Silent failures caught
E-mail mismatches that hide documents, slow group changes, sources without access control, and requests without user context.
Erasure verification
Shows that a deleted document can no longer be retrieved, by exact and semantic probes, with timestamps.
Quality gate
Retrieval recall and ranking, faithfulness, citations, refusals and injection resistance, with thresholds that fail the build.
Finnish and Swedish sets
Golden question sets in English, Finnish and Swedish come with the product.
Migration parity
Compares a new Bedrock knowledge base with Amazon Kendra or Amazon Q Business on your real queries.
What you receive
On every run
- A self-contained HTML report
- Results (JSON) and findings (CSV)
- A signed manifest, stored under S3 Object Lock
- A check summary in GitHub when it runs in CI
How it works
Step by step
- 01
Describe the target
Your knowledge base, or an HTTP endpoint through a small adapter, plus test identities and canary documents (the tool can generate them).
- 02
Choose suites and thresholds
Access, erasure, quality and parity, each with its own pass mark.
- 03
Run
Nightly in your account, or as a GitHub Action on every change.
- 04
Act on findings
Leaks are marked critical; a failed threshold stops the release.
Built on AWS
The AWS services it uses
- Amazon Bedrock Knowledge Bases
- Amazon Bedrock
- AWS Step Functions
- AWS CodeBuild
- Amazon EventBridge Scheduler
- Amazon SNS
- Amazon S3 Object Lock
- AWS KMS
- Amazon Kendra
- Amazon Q Business
Your data
Runs in your account. Reports keep document text out, apart from short snippets, unless you turn it on. The judge model can be limited to the EU. Nothing is sent to CloudNerve.
Scope, stated plainly
What it does not do
- Certify your system or give legal advice; it produces assurance evidence
- Fix your access lists or delete documents for you; you delete and sync, it verifies
- Trace erasure in vector stores other than S3 Vectors, OpenSearch Serverless and Aurora pgvector
Questions
Common questions
- Does it need production data?
- No. Access and erasure tests use canary documents with known access lists, which the tool can generate.
- What does a run cost?
- Judge calls run on Bedrock in your account. Results are cached by content hash, and a cost estimate is printed before each run.
- Can it gate our releases?
- Yes. In CI it returns a failing exit code and a GitHub check summary when a threshold is missed.
See RAG Assurance on your own AWS data.
Book a free Intro Call with a CloudNerve architect, or ask us anything about how it would fit your accounts.