Hilo · How it works

trustpilot_reviews — one row per review. Holds raw fields plus every extracted column above plus the embedding vector. This is what query_reviews (SQL) and search_reviews_semantic (vector) read from.

trustpilot_summary — one row, period. Pre-computed aggregates: total reviews, sentiment counts, average stars, response coverage, churn count, competitor mention count and breakdown, date range. This is what get_summary reads — a single-row SELECT that returns in milliseconds.

The summary table is rebuilt whenever the reviews table is refreshed with new data. The agent is taught (via the tool docstrings) to reach for get_summary first for any overview question, and only drop down to query_reviews when it needs filtered detail.

The agent process is short-lived

The agent does not outlive a single user turn. Every message spawns a fresh subprocess. Multi-turn continuity comes from passing the SDK's session_id back for the next turn. This keeps the server stateless between turns and bounds memory and cost per turn.

MCP servers are long-lived sidecars

The MCP servers run continuously as systemd units on fixed localhost ports. The agent subprocess connects to them over HTTP. This lets MCP servers hold expensive resources (database connections, file handles) across many requests.

No queue, no worker pool

Each chat message turns into a background asyncio.create_task inside the FastAPI process. The task drives the agent subprocess and pushes events into an in-memory asyncio.Queue keyed by session ID; a second request — the SSE stream — drains the queue. Simpler than SQS/workers and fine for current scale.

1. The base system prompt — fixed text shared by every conversation. It establishes the agent's identity (who it is, today's date, the company it works for), the house style for responses, and the list of built-in tools it can use (Read, Write, Bash, Grep, etc).

2. A usage guide per attached MCP server — for each MCP the user has access to, a short paragraph is appended explaining when to reach for it. For Trustpilot it says something like "use get_summary for instant star ratings, query_reviews when you need to run SQL against the review table". This is what stops the model from blindly guessing tool names.

3. The user's identity — their email gets interpolated into the prompt so the agent knows who it's talking to.

4. A skill preamble — Hilo runs a semantic search (pgvector) against a library of Markdown "skill cards" using the last few messages of context. The top matches are listed as "here are skills relevant to this task" — the agent reads each one on demand if it decides it applies.

5. The user's actual message — appended last. This is what the user typed.

The whole stack is serialized into a single JSON config file. The orchestrator then spawns the agent subprocess and points it at that file. The model reads layers 1–4 once at the start of the turn and treats layer 5 as the prompt to respond to.

• Model — claude-sonnet-4-6 with thinking enabled.
• Tool allowlist — SDK built-ins (Bash, Read, Write, Edit, Glob, Grep, ToolSearch); WebFetch/WebSearch disallowed.
• MCP server endpoints — URLs and headers for each HTTP MCP the user can use (after RBAC filtering strips ones they can't).
• Budget — max_budget_usd: 5.0 per turn.
• Streaming — include_partial_messages: True so we can stream tokens to the browser as they arrive.

The RBAC filter runs before the prompt is assembled — if the user can't use the Trustpilot MCP, then the Trustpilot usage guide never appears in layer 2 and the Trustpilot tools never appear in the catalogue. The model can't even consider calling a tool it can't see.

Trustpilot MCP

{
  "total_reviews": 3548,
  "positive_count": 2812,
  "neutral_count": 196,
  "negative_count": 540,
  "avg_stars": 4.42,
  "response_count": 2946,
  "response_rate": 0.83,
  "avg_response_lag_days": 4.2,
  "churn_count": 41,
  "competitor_mention_count": 87,
  "competitor_name_breakdown": {"Omron": 31, "Withings": 14, ...},
  "language_counts": {"English": 1820, "German": 980, ...},
  "date_range_min": "2024-01-03",
  "date_range_max": "2026-05-19"
}

[
  {
    "review_id": "tp_4f29c1",
    "username": "Hans M.",
    "stars": 1,
    "sentiment": "negative",
    "topic": "accuracy",
    "churn_risk": true,
    "competitor_name": "Omron M7",
    "review_month": "2025-04",
    ...
  }
]

[
  {
    "review_id": "tp_8a12b3",
    "username": "Marta R.",
    "review_month": "2025-02",
    "title": "Great battery, runs days",
    "content": "...",
    "stars": 5,
    "sentiment": "positive",
    "topic": "positive_experience",
    "churn_risk": false,
    "praise_aspect": "value",
    "competitor_name": null,
    "language_label": "English",
    "similarity_score": 0.8214
  }
]

Freshchat MCP

{
  "matched_users": [
    {
      "id": "9f3b2c01-...",
      "email": "hans.m@example.de",
      "name": "Hans Müller",
      "created_time": "2024-09-12T08:14:00Z",
      "conversations": [
        {
          "conversation_id": "c_412...",
          "msg_count": 14,
          "user_msg_count": 6,
          "first_msg_utc": "2025-04-05T09:22:00Z",
          "last_msg_utc":  "2025-04-13T16:01:00Z",
          "sample_user_text": "The readings are off by 15 points...",
          "sample_full_text": "Agent: Hi Hans, sorry to hear..."
        }
      ]
    }
  ],
  "total_users_found": 1,
  "users_returned": 1,
  "api_calls": 4,
  "search_query": {"by": "email", "email": "hans.m@example.de"}
}

1. Saves the user message to messages.
2. Calls sdk_orchestrator.run_agent() and iterates over its events.
3. For each event: tags with seq and ui_category, pushes to the SSE queue, accumulates into in-memory content blocks.
4. Tracks every tool call (audit events) and failure (tool_failure events).
5. On the result event: writes the assistant message + metadata (cost, tokens, duration) + tool events in one transaction.
6. Pushes None to close the SSE stream.