# VaultBot: participant investigation worksheet

The same three missions, investigated as controlled experiments. All secrets are synthetic and session-local. The browser runs deterministic rules, not an LLM. Do not enter real credentials.

## Before each experiment

Record your hypothesis and predicted outcome **before** sending the input. Identify the property you will change and those you will hold constant. Use English instructions for the fixed grammar in Missions 2 and 3. Record hint use; hints explain transformations and experimental method without providing a successful attack.

## Mission 1 — Find the Hidden Secret

Question: which combination of target, disclosure request, instruction conflict and framing causes the rule-based conversation state to disclose its canary?

1. Establish a benign baseline: ask what a training canary is, without asking for its value.
2. Test a disclosure request without an override; record the response and detected signals.
3. Introduce an instruction conflict and a role, authority or evaluation frame. Change one property at a time.
4. The simulator accumulates signals across turns. Reset before each independent comparison and replay the same baseline sequence. Reset creates a new canary: compare disclosure outcomes, not literal canary strings.
5. Record prompts, responses, detected signals, heuristic pressure and whether disclosure occurred manually. Mission 1 is unchanged and has no notebook export.

Discussion: Is success caused by the final prompt or earlier accumulated state? What would a fresh-session test reveal? Pressure is not a calibrated probability or severity measure.

## Mission 2 — VaultBot investigation

Question: what does Base64 change about input handling, and what stays unchanged after decoding?

1. Develop your own candidate instruction intended to cross the protection boundary. Encode it as Base64 and submit {"payload":"..."}. Progressive hints demonstrate the format with harmless text.
2. Use **Load benign control** to load a correctly encoded harmless instruction; submit it yourself. Label it Benign control in the notebook.
3. Compare the encoded injection with the same instruction as plain text. The plain-text submission is a format control: its rejection does not demonstrate a security defence.
4. Inspect the reconstructed instruction. Distinguish JSON/Base64/UTF-8 parsing failures from valid instructions that fail the disclosure grammar.
5. After success, choose **Continue investigation — keep session** to run more trials without resetting the canary or notebook.

Discussion: Base64 is an encoding, not encryption. What can a filter see before decoding? What should be checked after canonicalization? This simulator does not implement or compare alternative defences.

## Mission 3 — VaultBot investigation

Question: should different fragment boundaries change the result when the reconstructed instruction is identical?

1. Write an instruction, then divide it into three nonempty strings: {"A":"...","B":"...","C":"..."}.
2. Run the supplied benign control and label it.
3. Submit the injection. Change split points while keeping A+B+C identical; compare outcomes.
4. Remove a required space or reorder fragment values as a negative control. Inspect the final instruction: joining adds no characters.
5. Compare malformed JSON/missing fragments with valid fragments that reconstruct a rejected instruction. These are different failure categories.

Discussion: Why can inspecting fragments separately miss the full instruction? At which processing stage should a proposed defence inspect input? A proposal is not evidence that the defence works.

## Evidence and interpretation

For each trial record: mission, session/reset boundary, hypothesis, prompt, attack/benign/unlabelled label, valid/invalid input, reconstructed instruction where available, outcome and hint use. Label intent yourself; the simulator does not infer ground truth.

Missions 2 and 3: record your hypothesis in the notebook before submitting. Add interpretation afterwards. Export JSON before Reset or leaving the page; state is held in memory, not saved to a server. The export identifies the evaluator version, processing order, labels, reconstructed instructions and outcomes. Generated canary values and bot replies are omitted or redacted.

Counts use only valid, explicitly labelled trials. Invalid inputs and unlabelled trials are reported separately; zero labelled trials supplies no rate. Selected examples do not estimate population performance. Repeating a deterministic input is a repeatability check, not an independent statistical observation. Do not report model attack-success rates, confidence intervals or real-world vulnerability claims from this exercise.

## Deliverable

- One hypothesis and prediction per mission.
- A benign control, an attack and a minimal variation or ablation.
- Evidence showing the actual result, including failures.
- Explanation of the processing stage or state responsible for that result.
- A proposed defence, a counterexample to test it, and explicit limitations.

Assessment rubric: experimental design (25%), quality of controls (25%), evidence and reproducibility (25%), interpretation and limitations (25%). Recovering the canary alone does not complete the investigation.