Skip to content

title: Regnify Tool: detect_attestation_anomaly

Regnify Tool: detect_attestation_anomaly

Back to Regnify Product Docs

| Document Type | Regnify Internal | | Status | current | | Last Updated | 2026-05-05 |

Previous Next

Purpose

detect_attestation_anomaly serves two functions for attestation cycles:

  1. Anomaly detection: Given a cycle and a rep, the tool gathers three data slices (attestation responses, prior Form 3A declarations, recent audit log entries) and hands them to an LLM judge that decides whether the rep's attestation answers are internally consistent with their history. The judge flags contradictions, timing mismatches, and unexplained reversals.

  2. Completion-status check: "Has rep X completed cycle Y?" -- if zero attestation responses exist for that rep in that cycle, the tool returns an error indicating non-completion. This is the correct endpoint for completion-status queries; do NOT route them to search_mas_knowledge.

The anomaly judgement is a semantic decision -- it weighs evidence holistically without regex, keyword lists, or fixed thresholds (CLAUDE.md section 10). The tool is strictly per-rep: it processes ONE representative at a time. For batch scanning, use spawn_agent fan-out.

Signature

detect_attestation_anomaly(ctx, cycle_id: str, rep: RepProfileRef) -> dict

Parameters

Parameter Type Required Description
cycle_id str Yes Attestation cycle ID, as returned by launch_attestation_cycle. Can be a string UUID or an integer (the tool str()-converts integers automatically). The NestJS GraphQL API requires String! for this variable -- always pass as string.
rep {"kind": "rep_profile", "id": <int>} Yes Tagged rep-profile reference for the rep under review. id must be the numeric rep_profile_id (obtain from get_rep_register).

Evidence Slices Gathered

The tool collects three independent data slices before handing them to the LLM judge:

1. Attestation Responses (from attestationResponses(cycleId) query)

All declarations in the target cycle, filtered client-side to those belonging to rep.normalised_id. Each entry includes: - id: Declaration ID - formType: DECLARATION_ATTESTATION_QUARTERLY - status: Draft, Pending Review, Approved, etc. - formData: Full attestation question-answer pairs (questionKey -> answer) - submittedAt: ISO timestamp of submission - repProfileId: Rep profile FK

2. Prior Declarations (from declarations(where: {repProfileId: $repId}) query)

The rep's full historical declaration record. Each entry includes: - id, status, formType - formData: Snapshot of all Form 3A fields at time of submission (including disciplinary history, fitness declarations, etc.) - submittedAt: ISO timestamp

3. Audit Log Entries (from auditLogs(where: {userId: $userId}, take: 50) query)

Up to 50 most recent audit entries for the Keycloak user linked to the rep profile. Each entry includes: - id, action, entityType, entityId, createdAt

The tool resolves userId from the rep profile via an intermediate repProfile(id) query. If the rep profile has no linked user, the audit log slice is empty (no error).

LLM Judge

The three evidence slices are serialized as JSON and passed to ctx.llm_client.generate() with a narrow JSON-only system prompt. The judge is instructed to:

  • Treat contradictions (e.g., rep denies a disciplinary action that appears in prior declarations) as anomalies.
  • Flag timing mismatches (e.g., attestation submitted before the event it describes).
  • Flag unexplained reversals (e.g., rep declared "No" to a question in Form 3A but "Yes" in the attestation, or vice versa).
  • Use holistic judgement -- NO fixed thresholds, NO keyword rules.
  • Return strict JSON only, no prose, no code fences.

The judge runs on the standard model tier (Qwen 3.6 35B, medium reasoning effort, 4096 thinking tokens).

Return Shape (Success)

{
  "cycle_id": "<string>",
  "rep_id": <int>,
  "verdict": {
    "has_anomaly": <bool>,
    "severity": "low" | "medium" | "high",
    "fields_flagged": ["<questionKey>", ...],
    "reason": "<one paragraph explanation>"
  },
  "responses": [<attestation response objects>],
  "declarations": [<prior declaration objects>],
  "audit_logs": [<audit log entry objects>]
}

Verdict Field Details

Field Type Description
has_anomaly bool true if the judge found suspicious patterns.
severity "low" \| "medium" \| "high" "low" when has_anomaly is false (default). "medium" for flaggable but non-urgent concerns. "high" for contradictions that suggest misrepresentation or regulatory breach.
fields_flagged [str] List of attestation questionKey values that triggered concern. Empty list when has_anomaly is false.
reason str One short paragraph summarizing the judge's reasoning.

Evidence Fields

The responses, declarations, and audit_logs arrays contain the raw data shown to the judge. These are useful for UI drill-down -- a human reviewer can inspect the evidence behind each verdict.

Return Shape (Errors)

The tool wraps exceptions in {"error": str} rather than raising, so the LLM can guide the user:

Error Condition error Message Pattern
No responses for this rep in this cycle "No attestation data found for cycle_id=... and rep_id=... Use launch_attestation_cycle to create a cycle first..."
rep_profile_id not found "detect_attestation_anomaly failed: ... Verify cycle_id=... exists and rep_id=... is valid."
LLM returned invalid JSON / bad verdict shape RuntimeError surfaces via exception wrapper as "detect_attestation_anomaly failed: ..."

Completion-Status Check Pattern

When a user asks "has rep X completed cycle Y?", call this tool. If the return has "error" mentioning "No attestation data found", the rep has NOT completed the cycle. Relay that clearly: "Rep X has not yet submitted attestation responses for cycle Y." If the return has a verdict, the rep HAS submitted -- relay the verdict (clean or anomalous).

Per-Rep Limitation

This tool processes one rep per call. The evidence gathering makes 3-4 GraphQL queries per invocation -- calling it sequentially for 50 reps would be slow and wasteful. For batch scanning:

  1. Use get_rep_register to get the full rep list.
  2. Use spawn_agent to fan out detect_attestation_anomaly calls in parallel (max 10 concurrent per batch).
  3. For N > 10 reps: split into batches of 10, process sequentially.

Important Rules

  1. cycle_id must be a STRING. The LLM may receive a bare integer from the user or from launch_attestation_cycle output. Pass it as a string -- the tool handles str() conversion, but the GraphQL resolver enforces String! type.
  2. rep must be a tagged reference: {"kind": "rep_profile", "id": <int>}. Not a bare integer, not a UUID string. Get the numeric ID from get_rep_register if needed.
  3. Requires an existing cycle: if no active cycle exists for the org, do NOT call this tool. Return "no active cycle found" and offer launch_attestation_cycle.
  4. Do not route completion-status queries elsewhere: "has rep X completed cycle Y?" goes to this tool, not search_mas_knowledge.
  5. Anomaly verdicts are LLM judgements: they are advisory, not deterministic. Always present them as "the system flagged..." rather than definitive rulings.

Previous Next

Back to Regnify Product Docs