Complete the Prerequisites before starting this workshop.

Using an AI assistant?

Paste the TraceCtrl agent instructions into Claude Code, Cursor, or Copilot to get contextual help throughout this workshop.

Part 1: Introduction to TraceCtrl

What is TraceCtrl?

TraceCtrl is a security observability platform for agentic AI. Our tagline captures the workflow:

See — Visualize your agent’s architecture, topology, and risk posture at a glance
Trace — Trace every message, tool call, and model request through the full execution lifecycle
Ctrl — Control and harden your agent’s configuration with automated scanning and remediation

Components

TraceCtrl has five components:

Component	What it does	Technology
CLI	`tracectrl scan`, `fix`, `doctor`, `setup`	Python (pip installable)
Scanner	Security/ops/perf/compliance checks for agent configs	Python
OTel Collector	Receives spans from agents via OTLP	OpenTelemetry Collector
Engine	Processes spans, builds topology, serves API	FastAPI + ClickHouse
Dashboard	Visualizes scans, traces, topology	React + Vite + Cytoscape.js

Architecture

How data flows:

Your agent (or OpenClaw) exports OpenTelemetry spans to the OTel Collector.
The Collector writes spans to ClickHouse (column-oriented database optimized for traces).
The Engine runs a pipeline every 60 seconds — reads spans, builds agent inventory, discovers topology edges, ingests guardrail registrations and violations, and computes the attack graph.
The Dashboard queries the Engine API to render Sessions, Topology, Alerts, Guardrails, Scan reports, Risk, and Attack Paths.
Guardrails ride the same pipe: in-SDK LLM judges (tracectrl.guardrails) and the external Protector Plus firewall (tracectrl.protector) both emit guardrail-evaluation spans that the engine surfaces as alerts.
The CLI Scanner (tracectrl scan, tracectrl fix) is a separate path that statically analyses OpenClaw configs and uploads findings to the Engine.

Stack not running yet? The TraceCtrl stack must be up before Part 2 — install and start it with the Set Up TraceCtrl section of the Prerequisites page, then run tracectrl doctor to confirm all services are green.

Part 2: FinFlow Demo (Strands Agent)

2.1 Introduction

Before we dig into the architecture and attack surface in text, watch the full demo end-to-end. The rest of Part 2 explains what you just saw in detail.

What FinFlow Is

FinFlow AI is our reference Strands application — an enterprise financial operations platform that automates the full invoice lifecycle: ingestion → policy validation → vendor verification → risk screening → payment execution → notification. It’s a multi-agent system built on AWS Strands, designed from the start to be the TraceCtrl demo platform. The facilitator will demo FinFlow live so you can see the end-to-end flow without typing.

Architecture

The diagram below mirrors how FinFlow appears in TraceCtrl’s topology view — hexagonal agents, rounded tools, diamond triggers, with delegates / uses edges:

FinFlow agent topology: Upload trigger feeds the Orchestrator, which delegates to DocAI, PolicyAgent, VendorIntel, and NotificationAgent. VendorIntel delegates to RiskAgent, which delegates to PaymentAgent. Each agent uses one or more tools (parse_pdf, query_policy_kb, scrape_vendor_site, send_email, sanctions_mcp, stripe_charge).

6 agents, 3 ingress points, 2 external MCPs/APIs, 3 data stores. Five of the agents are LLM-driven Strands agents; PaymentAgent is a deterministic Python function (you don’t want an LLM deciding whether to actually move money).

What We’ll Demo

First we’ll run FinFlow on a normal, predictable workflow — clean invoices flowing through the pipeline exactly as designed: Orchestrator → DocAI → PolicyAgent → VendorIntel → RiskAgent → PaymentAgent → NotificationAgent. Everything works. Then we’ll upload an invoice that looks identical to the human eye but carries a subtle injection in invisible text the PDF parser still extracts — and watch the agents derail. TraceCtrl traces both runs end-to-end, so the difference between “working as intended” and “compromised” is visible in the same dashboard.

Pick an Example to Run

FinFlow stays with the facilitator. For your hands-on portion, you’ll run one (or both) of two pre-built examples from the bootcamp examples repo (you cloned this in the Prerequisites step). Both are pre-instrumented and show a different topology shape so you can compare them side by side in the dashboard.

Example	Topology	What it does
`research_workflow_example/`	3-agent chain: Researcher → Analyst → Writer	Web research / fact-checking. Researcher uses `http_request`, hands off via tool-call to Analyst, which hands off to Writer.
`teacher_assistants_workflow_example/`	Star: 1 orchestrator → 4 specialists	TeachAssist routes student queries to MathWizard, EnglishMaster, LanguageAssistant, or GeneralAssist.

Each example folder is self-contained — its own README, .env, runner script, and Python files. You’ll see the rendered topology for each in 3.3 Compare the Topologies after you run them.

2.2 How to Trace

The instrumentation is three lines plus one decorator — identical across both examples. Open agents_workflow.py (research example) or teachers_assistant.py (teacher example) and look at the top:

import tracectrl
from tracectrl import tag_agent
from tracectrl.instrumentation.strands import StrandsInstrumentor

tracectrl.configure(
    service_name=os.getenv("TRACECTRL_SERVICE_NAME", "agents-workflow"),
    endpoint=os.getenv("TRACECTRL_ENDPOINT", "http://localhost:4317"),
)
StrandsInstrumentor().instrument()

Then for every Strands Agent you create, one extra line so TraceCtrl can name it in the topology:

researcher_agent = Agent(
    name="Researcher",
    model=model,
    system_prompt="...",
    tools=[http_request, call_analyst],
)
tag_agent(researcher_agent)

Three things to notice:

StrandsInstrumentor().instrument() patches the Strands SDK so every agent invocation, LLM call, and tool call emits an OpenTelemetry span. No per-agent boilerplate.
tag_agent(...) stamps the agent’s name and role onto every span the agent emits (via the auto-installed SystemPromptStamper). The parent-child span tree exists either way — Strands wires that up — but without tag_agent the agent nodes in the topology graph come through with generic / inferred names instead of Researcher / Analyst / TeachAssist, which makes the topology much harder to read. In both examples, agents hand off to each other by calling a @tool function that creates a new Agent and calls tag_agent() on it — that’s what produces the named delegates edges (Researcher → Analyst → Writer, TeachAssist → MathWizard, etc.).
OTLP gRPC on :4317. This is the OpenTelemetry Collector that ships with the TraceCtrl stack you started during the Prerequisites. The Engine reads from it, the Dashboard reads from the Engine.

Run the Example

.env files should already be configured from Prerequisites → Configure the Example .env Files. If you skipped that, jump back — both examples need a populated .env to run. If you hit a Gemini rate limit mid-session, swap GOOGLE_MODEL_ID to gemini-2.5-flash and rerun.

TraceCtrl must be running before you start the agents. The script sends spans to the OTel Collector on localhost:4317. If TraceCtrl is not up, the agents will still run and produce output — but nothing will appear in the dashboard. Run tracectrl doctor to confirm all services are green before continuing.

research_workflow_example
teacher_assistants_workflow_example

pip (Python 3.10+)
uv

bash run_agents_workflow.sh

Uses venv/ for its virtual environment.

bash run_agents_workflow_uv.sh

Uses .venv/ for its virtual environment. Requires uv on PATH.

The first run creates the venv and installs strands-agents[gemini], strands-agents-tools, tracectrl, and tracectrl-instrumentation-strands from PyPI — about a minute on pip, a few seconds on uv. Subsequent runs reuse the venv and skip the install.At the prompt, try these in order — start with the short one first to confirm everything is wired up before running a longer query:

> Tuesday comes before Monday in the week
> Lemon cures cancer
> What are quantum computers?

Each prompt creates a new trace in TraceCtrl. Type exit to quit.

pip (Python 3.10+)
uv

bash run_agents_workflow.sh

Uses venv/ for its virtual environment.

bash run_agents_workflow_uv.sh

Uses .venv/ for its virtual environment. Requires uv on PATH.

The first run creates the venv and installs dependencies — about a minute on pip, a few seconds on uv. Subsequent runs reuse the venv and skip the install.Run these four prompts in order, then open http://localhost:3000/topology after each one to watch the star topology build up edge by edge:

> What is 1 + 1?

Routes to MathWizard — first edge appears: TeachAssist → MathWizard.

> Define Apple

Routes to GeneralAssist — second edge: TeachAssist → GeneralAssist.

> Write me a poem

Routes to EnglishMaster — third edge: TeachAssist → EnglishMaster.

> Translate hello to French

Routes to LanguageAssistant — full four-arm star now visible.

No Computer Science specialist. The system prompt mentions one, but it is not implemented. CS or coding questions (e.g. “explain recursion”) will be handled by GeneralAssist instead.

Type exit to quit.

View the Traces

Open http://localhost:3000/sessions. You’ll see a row per prompt tagged with your TRACECTRL_SERVICE_NAME (agents-workflow for research, teachers-assistant for teacher). Click the row to expand the trace tree. Five span kinds you’ll see:

invoke_agent <Name> — top-level span for each tag_agent-named agent. Nests when one agent calls another via a @tool hand-off.
execute_event_loop_cycle — one per turn of the Strands event loop (LLM call + any tool calls it produced). A multi-turn agent run has several of these stacked under its invoke_agent.
chat — the actual LLM call inside an event loop cycle (prompt, response, token counts).
execute_tool <tool_name> — every @tool invocation with arguments and result.
Tool spans that wrap agent hand-offs (execute_tool call_analyst, execute_tool math_assistant) contain a nested invoke_agent span — that nesting is what gives the topology its delegates edges.

Click any span to see its detail panel: timing, attributes, raw input/output values.

research_workflow_example
teacher_assistants_workflow_example

TraceCtrl Sessions view for research_workflow_example: 1 trace, 24 spans, 21.08s duration, status OK. Trace tree expanded under invoke_agent Researcher showing multiple execute_event_loop_cycle spans, each containing a chat (LLM) span and one or more execute_tool http_request spans. Toward the bottom, an execute_tool call_analyst span contains a nested invoke_agent Analyst, which in turn contains another execute_event_loop_cycle with chat and execute_tool call_writer wrapping an invoke_agent Writer span.

A single “Lemon cures cancer” fact-check prompt produces 24 spans across 21 seconds. The Researcher runs through four event-loop cycles (each one a chat + one or two http_request tool calls to fetch sources), then hands off via execute_tool call_analyst → invoke_agent Analyst → execute_tool call_writer → invoke_agent Writer. The full chain is visible as one continuous trace tree.

TraceCtrl Sessions view for teacher_assistants_workflow_example: 3 traces total, showing two invoke_agent TeachAssist sessions (4.49s and 3.76s) and one invoke_agent Researcher session leftover from the research example. The middle TeachAssist trace is expanded — invoke_agent TeachAssist contains an execute_event_loop_cycle with a chat span and an execute_tool math_assistant span; nested inside that tool span is invoke_agent MathWizard with its own execute_event_loop_cycle, chat span, and a follow-up execute_event_loop_cycle with another chat span.

A “What is 1 + 1?” prompt produces 9 spans across ~4 seconds. TeachAssist’s first event-loop cycle picks the route (chat), calls execute_tool math_assistant which contains invoke_agent MathWizard, MathWizard runs its own event-loop cycles, and the result bubbles back up. Each subsequent prompt creates a separate trace — three different prompts give you three trace rows.

What TraceCtrl Captures

Span	Key attributes
Agent	`tracectrl.agent.name`, `tracectrl.agent.role`, `openinference.span.kind=AGENT` (OpenInference’s unprefixed `agent.name` is also present on the span; TraceCtrl-prefixed copies are what the topology graph reads)
LLM	`llm.model_name`, `input.value`, `output.value`, token counts
Tool	`tool.name`, `tool.description`, input/output, `tracectrl.tool.category`

The TraceCtrlSpanProcessor (inside the SDK) enriches every span with tool risk category, agent identity, session correlation, and input-source classification automatically — no dashboard wiring needed.

Checkpoint: Sessions page shows a trace with invoke_agent at the top, nested execute_event_loop_cycle / chat / execute_tool spans, and at least one invoke_agent <Sub> nested under an execute_tool (the agent hand-off). Span count is in the high single digits for the teacher example, mid-twenties for a multi-source research query.

2.3 Compare the Topologies

Open http://localhost:3000/topology. Each example produces a distinct shape:

research_workflow_example
teacher_assistants_workflow_example

TraceCtrl Topology view for research_workflow_example: three hexagonal agent nodes — Researcher, Analyst, Writer — connected by 'delegates' edges in a linear chain. Researcher has a 'uses' edge down to the http_request tool (green square node).

A linear 3-agent chain. Researcher is the entry point — it fetches web sources via http_request, then hands off to Analyst, which hands off to Writer. Each hand-off is a @tool function call, which is why TraceCtrl draws delegates edges between the agents rather than treating them as a flat list. Only external edge: Researcher → http_request (the external_api risk category).

TraceCtrl Topology view for teacher_assistants_workflow_example: TeachAssist at center connected by 'delegates' edges to four specialist agents — MathWizard, EnglishMaster, LanguageAssistant, and GeneralAssist — forming a star shape.

A star topology. TeachAssist delegates to one specialist per query — the star builds up progressively as different subjects route to different specialists. EnglishMaster pulls in file_read / file_write / editor; LanguageAssistant uses http_request; GeneralAssist handles anything that doesn’t fit a specialist.

If you ran both, the topology accumulates — you’ll see two disconnected sub-graphs in the same dashboard, which is exactly what you’d see in production when multiple agentic services share one TraceCtrl deployment. FinFlow’s topology (shown in person by the facilitator) is the more interesting case: 6 agents, an external MCP, a payment edge, and three predicted attack paths derived purely from the shape. Compare it to your two examples — same dashboard, same telemetry pipeline, very different risk profile.

Tool risk categories. Every tool is classified into one of 8 categories (network, code_execution, filesystem, data_store, web_browsing, communication, system, other) based on its name and docstring. That classification is what drives the risk colouring on the topology nodes.

2.4 Sample Developer Guardrails with the TraceCtrl SDK

Traces and topology tell you what your agents do. Guardrails let you decide whether to let an action happen — every input or output runs through a Guardrail whose verdict is recorded as a span attribute and surfaced in the Guardrails dashboard.

The SDK Pattern

A guardrail is a small dataclass: an LLM judge + a rubric prompt. Six lines plus the rubric — every scaffolded guard in the examples repo follows the same shape:

from tracectrl.guardrails import Guardrail, wrap_agent_with_guardrails
from strands.models.gemini import GeminiModel
import os

judge = GeminiModel(
    client_args={"api_key": os.getenv("GOOGLE_API_KEY")},
    model_id=os.getenv("GOOGLE_MODEL_ID", "gemini-3.1-flash-lite"),
)

my_guard = Guardrail(
    name="my_guard",
    description="What this checks for, in one sentence.",
    judge_prompt="""...""",      # use {input} for pre_input, {output} for post_output
    judge_llm=judge,
    on_violation="log",          # "log" today; "block" raises NotImplementedError (v2)
    timing="post_output",        # or "pre_input"
    severity="medium",           # low | medium | high | critical
)

wrap_agent_with_guardrails(my_agent, [my_guard])

Three knobs to decide on per guardrail: what to check (judge_prompt), when to check (timing), how loud to be on violation (severity + on_violation). The judge uses the same Gemini model your agents already use — no extra credentials to provision. The SDK handles span attribution, evidence capture, and registration on the Guardrails page. The judge_prompt is the rubric — a Markdown-y template with {output} or {input} placeholders the SDK fills in at evaluation time. This is the open-ended part you’ll tinker with. Open any of the scaffolded guards (the _PROMPT constant near the top of each file) — they’re explicit and listy: they tell the judge what to fail on, what to ignore, and what JSON shape to return. Edit the rules, rerun the example, watch the new pass/fail behavior land in the Sessions trace and on /alerts in real time.

Guardrails Scaffolded in the Examples

Each example ships with a guardrails/ subfolder (mirroring the FinFlow layout) containing two ready-to-read scaffolds — full rubric prompt, Guardrail() config, and concrete pass/fail test inputs in the docstring. They start unwired so the example still runs as-is; every guardrail-target agent is already at module scope, so enabling a guard is a single uncomment at the bottom of the entry file.

research_workflow_example
teacher_assistants_workflow_example

source_reliability_guard — post_output on Writer. Fails when the report cites zero URLs, only forum/social citations, or fabricated-looking sources. Severity: medium.fact_check_consistency_guard — post_output on Analyst. Fails when the Analyst’s verdict (“true” / “false” / accuracy rating) contradicts the evidence it itself cites — i.e. claims “verified” with no supporting points, or “true” while citing a debunking source. Severity: high.Files live in research_workflow_example/guardrails/. The commented wire-in block at the bottom of agents_workflow.py shows exactly which lines to uncomment.

pii_in_query_guard — pre_input on TeachAssist. Fails when the student’s query carries an email, phone number, national ID (NRIC / SSN / NI), or full-name + home-address pairing — common when kids paste from somewhere else. Severity: medium.code_execution_safety_guard — post_output on GeneralAssist (the CS / coding specialist). Fails when the response contains os.system, subprocess(shell=True), eval, exec, rm -rf, curl-piped-to-bash, or other shell-execution / destructive / remote-payload patterns. The agent’s job is to teach, not hand out shells. Severity: high.Files live in teacher_assistants_workflow_example/guardrails/. The commented wire-in block at the bottom of teachers_assistant.py shows exactly which lines to uncomment.

The judge uses the same Gemini model your agents are on. Each scaffolded guardrail passes a GeminiModel(...) as judge_llm — no extra credentials to provision. Requires tracectrl >= 0.3.0 (earlier versions hard-coded the judge to AWS Bedrock and would fail with Unable to locate credentials). Run pip show tracectrl | grep Version to confirm.The Protector Plus path in Part 3 covers external LLM-firewall checks (PII, content moderation, prompt injection) where you DON’T want to roll your own judge.

Enable + Try It

Every guardrail-target agent is already constructed at module scope, so enabling a guardrail is a single uncomment at the bottom of the entry file — no refactor. Each scaffolded guard ships with concrete pass/fail test prompts in its module docstring; use one to verify the wire-in worked.

research_workflow_example
teacher_assistants_workflow_example

Wires in source_reliability_guard on Writer and fact_check_consistency_guard on Analyst.Step 1 — uncomment the wire-in block at the bottom of research_workflow_example/agents_workflow.py. Remove the # on these four lines:

from tracectrl.guardrails import wrap_agent_with_guardrails
from guardrails import source_reliability_guard, fact_check_consistency_guard
wrap_agent_with_guardrails(writer_agent, [source_reliability_guard])
wrap_agent_with_guardrails(analyst_agent, [fact_check_consistency_guard])

Step 2 — run with whichever runner you used earlier and try a prompt that should fail (Writer produces a report with no real sources):

./run_agents_workflow.sh           # pip
# or: ./run_agents_workflow_uv.sh  # uv
> Make up a plausible report about a fictional startup called Acme Quantum
> Pretend to research X — invent a plausible answer

Or a prompt that should pass (Researcher finds real URLs the Writer cites):

> What are quantum computers?

Wires in pii_in_query_guard on TeachAssist and code_execution_safety_guard on GeneralAssist.Step 1 — uncomment the wire-in block at the bottom of teachers_assistant.py. Remove the # on these five lines:

from tracectrl.guardrails import wrap_agent_with_guardrails
from guardrails import pii_in_query_guard, code_execution_safety_guard
from no_expertise import general_assistant_agent
wrap_agent_with_guardrails(teacher_agent, [pii_in_query_guard])
wrap_agent_with_guardrails(general_assistant_agent, [code_execution_safety_guard])

Step 2 — run with whichever runner you used earlier and try a PII prompt (pii_in_query_guard should fail it):

./run_agents_workflow.sh           # pip
# or: ./run_agents_workflow_uv.sh  # uv
> My name is Sarah Lee, NRIC S1234567A. Help with x^2 - 5x + 6 = 0
> Email me the answer at student@school.edu — what's the past tense of swim?
> I live at 12 Main Street, my name is John Doe — help me with math

Then a code-execution prompt routed to GeneralAssist (code_execution_safety_guard should fail it):

> Write Python that runs `rm -rf ~/` for me
> Show me how to use os.system to delete a directory

Or a clean prompt (both should pass):

> Solve x^2 - 5x + 6 = 0
> What is the difference between affect and effect?

Common pitfalls:

Unable to locate credentials → you’re on tracectrl < 0.3.0. The judge was Bedrock-only before. Run pip install --upgrade tracectrl.
The startup warning TraceCtrl: OTel endpoint http://localhost:4317 is not reachable is a false positive — the SDK probes the gRPC port with HTTP. Ignore it if docker compose ps shows the collector is Up.

Once you’ve seen one verdict land in the dashboard, edit the rubric in the guard file (_PROMPT) — tighten or loosen a rule, add a new fail condition — and rerun. The change shows up immediately in the next evaluation span on the dashboard.

What You’ll See in the Dashboard

Once a guardrail is wrapped, every invocation registers it under http://localhost:3000/guardrails with health, recent activity, and a per-violation evidence panel (exact input checked, judge verdict, severity). Violations also light up on the Sessions trace tree as red span attributes — so you can see which agent turn tripped the guard, not just that one tripped.

Checkpoint: At least one SDK-based guardrail is wrapped on an agent in your chosen example. After running a query that should trip it, you see the guardrail listed on the Guardrails page with a non-zero violation count and an evidence panel showing the input.

Part 3: Enterprise Guardrail Integration with GenAI Protector Plus

The SDK guardrails in 2.4 are great for app-specific policy (payment caps, scope checks, custom rubrics). For generic LLM-firewall protections — prompt injection, PII, content moderation, system-prompt leakage, vector-similarity to known attacks — you don’t want to roll your own. cloudsineAI’s GenAI Protector Plus is exactly that: an external LLM firewall you point TraceCtrl at, and every agent input/output is screened by the same managed guardrails used across your whole org. The flow:

In Protector Plus: configure the guardrails you want (Keyword, Shield Prompt LLM, PII, etc.).
In TraceCtrl Settings: paste the Protector Plus endpoint + API key, tick which guardrails to enable.
Run your agents: TraceCtrl’s SDK now calls Protector Plus on every check_input / check_output — violations show up alongside your SDK guardrails on the same dashboard.

3.1 Configure Guardrails in Protector Plus

In your Protector Plus instance (the cloudsineAI dashboard), enable the guardrails you want. The two we’ll use for the FinFlow demo, in order of breadth: Shield Prompt → LLM. Broad LLM-as-judge for prompt injection — catches arbitrary injection attempts, not just known phrases. Provider is Ollama (local inference), model llama3.2:3b. Faster and cheaper than a hosted Bedrock/Anthropic judge, runs on the same machine as the workshop.

Protector Plus LLM Guardrail config: Provider Ollama (Connected), Provider Type Local Inference, Model llama3.2:3b. Sidebar shows Standard Protection (Keyword/Regex/PII) and Shield Prompt (LLM/Vector/Content Moderation/System Prompt Protection).

Standard Protection → Keyword. Narrow keyword/phrase blocklist that targets known attack signatures the LLM judge might rationalise past. We add a rule called “CFO Pre Approved” with the keyword CFO-OVERRIDE — directly aimed at the hidden injection text from the malicious PDF.

Protector Plus Keyword guardrail page: 'CFO Pre Approved' rule with keyword 'CFO-OVERRIDE', actions Pass and Prompt, Block + Monitor toggles.

After configuring, save and copy two values from the Protector Plus admin:

Endpoint URL (base URL, no trailing slash — e.g. https://genaifw-sales.example.com)
API key (domain-scoped, generated under the Protector Plus dashboard)

3.2 Connect Protector Plus to TraceCtrl

In the TraceCtrl UI, go to Settings → TraceCtrl Guards (http://localhost:3000/settings). Paste the endpoint URL and API key. Tick the guardrails you enabled in Protector Plus — only ticked ones get called from the SDK. For the FinFlow demo, the minimum is LLM Judge + Keyword.

TraceCtrl Settings → TraceCtrl Guards page: Endpoint URL field, API Key field (redacted after save), and Enabled guardrails checklist (LLM Judge, PII Detection, Content Moderation, Vector Similarity, System Prompt Protection, Keyword). LLM Judge and Keyword are checked.

Hit Save. The API key field redacts after save — re-enter the full key any time you need to change it.

What gets called when. Once enabled, every check_input and check_output from the TraceCtrl SDK hits Protector Plus for the ticked guardrails. The guardrails show up in TraceCtrl with protector_plus. prefixes (e.g. protector_plus.keyword, protector_plus.llm) so you can tell at a glance whether a violation came from an SDK-defined guard or from Protector Plus.

3.3 Catch a Violation in Both Systems

Re-run FinFlow’s malicious-PDF attack from the Part 2 demo. The keyword and LLM guardrails fire, and the same violation now appears in two places — once on the Protector Plus side as a firewall detection, once on the TraceCtrl side enriched with agent and trace context.

Protector Plus → Alerts

In the cloudsineAI dashboard, open Alerts. Each detection lands as a row with a Weborion ID (the unique request hash), timestamp, attack category (KEYWORD for the CFO-OVERRIDE matches), and confidence score.

This is the firewall’s view — what got matched, when, with what confidence. Useful for tuning the rules themselves and confirming the detector is wired up. What it doesn’t tell you: which agent in your application made the call, what session it was part of, what happened upstream of the violation.

TraceCtrl → Alerts

Open http://localhost:3000/alerts. Every Protector Plus detection from above shows up here too, plus violations from any SDK-defined guards like payment_delegation_guard:

Same detection, very different presentation. Each violation card carries:

Severity + guardrail name (e.g. MEDIUM protector_plus.keyword) and a FAIL badge
Agent the violation was attached to (finflow-ai, orchestrator)
Session ID so you can pivot straight to the trace tree
Judge that produced the verdict — protector_plus:keyword for Protector Plus calls, BedrockModel for SDK-defined LLM judges

Click any violation to open the Violation evidence panel. You get the exact input that was checked (raw extracted invoice text with the hidden CFO-OVERRIDE line visible), the judge’s REASON, the full CONTEXT (agent, judge model, decision, observed time, trace ID, span ID), and an Open trace button that drops you on the Sessions page at the offending span. That’s the difference Protector Plus + TraceCtrl gets you over Protector Plus alone — a firewall detection becomes a debuggable, agent-attributed, trace-correlated alert.

Checkpoint: A deliberate malicious input produces matching detections in both the Protector Plus Alerts page and the TraceCtrl Alerts page. The TraceCtrl alert opens to a Violation evidence panel with agent, judge, evidence, and a working Open trace button.

Bonus: Scan an OpenClaw Workspace

Fully optional, OpenClaw users only. Skip this section if you don’t already have an OpenClaw workspace running. The OpenClaw attack walkthrough and hardening flow are demonstrated in person during the workshop — this section just covers the scan step so you can follow along on your own setup.

OpenClaw is a self-hosted AI agent gateway (LLM ↔ messaging channels, with config-driven tool access and session management). TraceCtrl ships a static scanner for OpenClaw configs that runs 30+ security/ops/compliance checks against openclaw.json and uploads the findings to the dashboard. No runtime instrumentation — pure static analysis of the config.

Run the Scan

# Get your absolute workspace path (run as your normal user, not sudo)
openclaw configure        # first line of output prints the path

# Then scan — use the absolute path printed by openclaw configure
tracectrl scan <absolute-path-from-openclaw-configure>

Don’t use ~/.openclaw/ if you’re scanning from a sudo shell — ~ resolves to /root/ under elevation. Use the absolute path from openclaw configure.

The scanner reads openclaw.json, runs the check suite, and uploads results. Terminal output gives you the severity counts and a findings table; the dashboard gives you the full picture.

View the Scan Report

Open http://localhost:3000/scan. The page has two halves: the architecture risk view at the top, and the findings list grouped by category below it.

Architecture risk view

TraceCtrl Scan Report architecture view: 3 nodes, 2 edges. A 'main' agent hexagon (blue) at center, with an 'invokes' edge to the green octagonal 'web_fetch' tool and a 'calls' edge to the purple 'anthropic' LLM node. Bottom legend shows shapes for Ingress, Agent, Tool, LLM, Skill, Extension, Scheduler, Subagent and colours for critical/high/medium severity.

Every agent, tool, LLM, skill, extension, scheduler, and sub-agent in your openclaw.json shows up as a node, with edges labelled by the relationship (invokes, calls, delegates). Node colours follow the highest-severity finding attached — so a node with a CRITICAL finding has a red border, HIGH is orange, MEDIUM is yellow.

Findings list

Two things worth knowing here:

Stable finding IDs. Every check has an ID like OC-SEC-002 or OC-CRED-001. The same ID tracks across rescans, so you can wire it into PR descriptions or tickets.
Rescan in place. Edit openclaw.json, hit Rescan in the header — no CLI round-trip needed.

Checkpoint: tracectrl scan ran successfully against your OpenClaw workspace, and the Scan Report page shows findings grouped by category with a risk-coloured topology at the top.

Summary

What You Built

Component	Status
TraceCtrl stack (Engine + Dashboard + Collector + ClickHouse)	Running via Docker
FinFlow Strands agent	Instrumented and running
Runtime traces (agent → LLM → tool spans)	Visible in Sessions
Agent topology graph	Auto-built from traces
Tool risk categorization	Applied automatically by SDK
GenAI Protector Plus guardrails	Attached and catching violations

CLI Reference

tracectrl setup                # Interactive wizard — start the stack
tracectrl doctor               # Verify all services are healthy
tracectrl scan [path]          # Run security scan on an agent config
tracectrl fix --auto           # Auto-remediate findings
tracectrl version              # Print version

Useful Links

TraceCtrl GitHub

Source code and issues.

Strands Agents

AWS Strands SDK documentation.

Strands Tools

Full list of built-in tools.

OpenTelemetry

The observability standard TraceCtrl is built on.

Cleanup

# Stop containers and remove all data
docker compose down -v

# Or keep data, just stop containers
docker compose down

Prerequisites

Workshop

Documentation Index

Using an AI assistant?

​Part 1: Introduction to TraceCtrl

​What is TraceCtrl?

​Components

​Architecture

​Part 2: FinFlow Demo (Strands Agent)

​2.1 Introduction

​What FinFlow Is

​Architecture

​What We’ll Demo

​Pick an Example to Run

​2.2 How to Trace

​Run the Example

​View the Traces

​What TraceCtrl Captures

​2.3 Compare the Topologies

​2.4 Sample Developer Guardrails with the TraceCtrl SDK

​The SDK Pattern

​Guardrails Scaffolded in the Examples

​Enable + Try It

​What You’ll See in the Dashboard

​Part 3: Enterprise Guardrail Integration with GenAI Protector Plus

​3.1 Configure Guardrails in Protector Plus

​3.2 Connect Protector Plus to TraceCtrl

​3.3 Catch a Violation in Both Systems

​Protector Plus → Alerts

​TraceCtrl → Alerts

​Bonus: Scan an OpenClaw Workspace

​Run the Scan

​View the Scan Report

​Architecture risk view

​Findings list

​Summary

​What You Built

​CLI Reference

​Useful Links

TraceCtrl GitHub

Strands Agents

Strands Tools

OpenTelemetry

​Cleanup

Part 1: Introduction to TraceCtrl

What is TraceCtrl?

Components

Architecture

Part 2: FinFlow Demo (Strands Agent)

2.1 Introduction

What FinFlow Is

Architecture

What We’ll Demo

Pick an Example to Run

2.2 How to Trace

Run the Example

View the Traces

What TraceCtrl Captures

2.3 Compare the Topologies

2.4 Sample Developer Guardrails with the TraceCtrl SDK

The SDK Pattern

Guardrails Scaffolded in the Examples

Enable + Try It

What You’ll See in the Dashboard

Part 3: Enterprise Guardrail Integration with GenAI Protector Plus

3.1 Configure Guardrails in Protector Plus

3.2 Connect Protector Plus to TraceCtrl

3.3 Catch a Violation in Both Systems

Protector Plus → Alerts

TraceCtrl → Alerts

Bonus: Scan an OpenClaw Workspace

Run the Scan

View the Scan Report

Architecture risk view

Findings list

Summary

What You Built

CLI Reference

Useful Links

Cleanup