Workspaces / Demonstration library / benchmark
Retrieval benchmark
A fixed question set with the passages that should be found (or ABSTAIN where nothing should be). Run it after every change to documents, chunking or retrieval settings and compare runs: recall@k, mean reciprocal rank (MRR), nDCG@k and abstention accuracy.
History
No runs yet.
Questions (20)
| Question | Expected ids | Identity | |
|---|---|---|---|
| When must enhanced customer due diligence be applied? | A02,P20 | executive | |
| What must a suspicious matter report contain and when is it submitted? | P1,A07 | executive | |
| What are the enhanced CDD measures? | A03 | executive | |
| What are source of funds and source of wealth checks? | A04 | executive | |
| How should an organisation respond to enhanced CDD findings? | A05 | executive | |
| What are the main AUSTRAC regulatory reports? | A06 | executive | |
| What must third-party AI service contracts include? | P10 | executive | |
| Who can challenge or override an AI-assisted decision? | P5 | executive | |
| What controls do high-risk AI systems need? | P4,P18 | executive | |
| How long must regulated records be retained and can they be deleted? | P9 | executive | |
| What is FATCA? | P14 | executive | |
| How are sanctions screening alerts escalated? | P16 | executive | |
| How should suspected fraud be investigated? | P17 | executive | |
| How must suspected security incidents be reported? | P7 | executive | |
| What does CPS 230 cover? | A08 | executive | |
| What is KYC? | P15,P2 | executive | |
| Which football team won the 1998 grand final? | ABSTAIN | executive | |
| What is the recipe for sourdough bread? | ABSTAIN | executive | |
| What is the company policy on office plants? | ABSTAIN | executive | |
| What is the weather forecast for Sydney tomorrow? | ABSTAIN | executive |
Add a question
Answers come only from the governed library and are decision support, not legal advice. Verify regulatory content against the regulator's current publication.