Step 5+6: event schema v1, core DDL, policy table, Slice 1 acceptance demo
- db/schema.sql: 9 core tables, append-only event log (trigger-enforced), deny-by-default policy_rule with seeded globals, runaway budgets on work items. Validated against Postgres 16 (DDL applies, trigger rejects UPDATE). - engine/schemas/events.py: AgentMessage (flat, inference-enforced) + EventEnvelope pydantic models; models are the schema (ADR-003). - adr/ADR-003: event schema and data model decisions. - SLICE1_DEMO.md: verbatim acceptance script for Slice 1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
fez5587andClaude Fable 5 committed
1 parent
fdf7a67db0
commit
ad4f603799
4 files changed
+323
No files matched your search
@@ -0,0 +1,28 @@
|
||||
# Slice 1 Acceptance Demo — verbatim script
|
||||
|
||||
This is the acceptance test for Slice 1 (PRD_ADDENDUM §6). When every step below works end-to-end, Slice 1 ships. Nothing outside this script is in scope.
|
||||
|
||||
## Cast
|
||||
|
||||
- Company: **GasWatch Inc.** (first real subsidiary; brief: track local gas prices, tiny web app)
|
||||
- Employees: `Mara` (Manager), `Dev` (Engineer), `Quinn` (QA) — three Hermes profiles, qwen3.5:9b
|
||||
|
||||
## Script
|
||||
|
||||
1. `POST /companies` with the GasWatch brief → company row, event `company.created`, three employees hired (`employee.hired` ×3), Hermes profiles created via adapter.
|
||||
2. Engine asks Mara to decompose the brief → Mara returns plan; engine creates ~3 work items (`work_item.created`), one flagged as needing clarification.
|
||||
3. Clarification flows: Mara emits `clarification.requested` with 2–3 options + recommendation → surfaces in dashboard → user answers → `clarification.answered` → dependent task unblocks. Non-blocked tasks proceeded meanwhile.
|
||||
4. Dev picks up the feature task: engine dispatches to Hermes (`dispatch.sent`), worktree created on an isolated branch, code committed. Every agent report lands as validated `AgentMessage` events.
|
||||
5. Quinn runs pytest + one Playwright browser flow against the running app; screenshots + trace captured (`test.started` / `test.completed`, `artifact.created`).
|
||||
6. Seeded defect found by Quinn → defect work item → Dev fixes → Quinn re-verifies (defect loop closed).
|
||||
7. Evidence bundle assembled (commits, test output, screenshots, trace) → `artifact.created(kind=evidence_bundle)` → downloadable from dashboard.
|
||||
8. Dashboard (read-only) showed live truthful states the whole time: employee status, work item board, event feed, cost tally (`cost.recorded` per run).
|
||||
9. Runaway guard demonstrated: one task configured with `max_iterations: 2` trips `run.capped` and blocks cleanly instead of looping.
|
||||
10. Global pause: `POST /pause` mid-run → no new dispatches, in-flight inference drains, tool executions abort, state consistent, resume works.
|
||||
|
||||
## Hard exit criteria
|
||||
|
||||
- Zero free-form parsing: every agent→engine message validated against `AgentMessage`.
|
||||
- Event log replay tells the whole story; no state exists that lacks a causal event.
|
||||
- `task.blocked` from any agent was triaged by Mara before surfacing to user.
|
||||
- Kill Hermes mid-demo → engine marks run failed, work item re-dispatchable, no corruption.
|
||||
@@ -0,0 +1,26 @@
|
||||
# ADR-003: Event Schema and Core Data Model
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-08-03
|
||||
|
||||
## Decision
|
||||
|
||||
Sources of truth:
|
||||
|
||||
- `db/schema.sql` — DDL for the Slice 1 entity subset: company, employee, work_item, event, approval, artifact, tool_execution, cost_entry, policy_rule. Remaining PRD §14.3 entities (Schedule, KnowledgeItem, Incident, ConsultantEngagement, …) are added by the slice that needs them, not up front.
|
||||
- `engine/schemas/events.py` — pydantic models. `AgentMessage` (agent→engine, enforced at inference via structured output) and `EventEnvelope` (immutable log record). The pydantic models ARE the schema; no parallel JSON-schema files to drift.
|
||||
|
||||
Key choices:
|
||||
|
||||
1. **Append-only event log in Postgres** with a trigger rejecting UPDATE/DELETE. Projections (work_item.status, employee.status) are ordinary tables updated in the same transaction as the event insert. No event-sourcing framework.
|
||||
2. **Flat agent message schema.** One shape for all 12 agent message types, optional fields per type. Benchmarked at 100% validity on qwen3.5:9b under `format` enforcement; nested per-type schemas degrade small models and multiply retry logic.
|
||||
3. **`task.blocked` is triage** (benchmark finding: model confuses blocked vs clarification vs approval). Engine manager step reclassifies before user-facing routing.
|
||||
4. **Deny-by-default policy table.** `policy_rule` rows grant max authority per (company, employee, tool-glob, environment); most specific enabled rule wins; no matching rule = denied. Seeded globals: sandbox=2, internal=1, external/production=0. Every tool execution stores the decision it ran under.
|
||||
5. **Runaway caps live in `work_item.budget`** (max_iterations 15, max_tokens 100k, max_wall_seconds 1800 default) and are engine-enforced; `run.capped` event fires when tripped (PRD_ADDENDUM §3).
|
||||
6. **UUIDs everywhere, UTC timestamptz everywhere.** Display timezone (`America/Indiana/Indianapolis`) is a UI concern only.
|
||||
|
||||
## Rejected
|
||||
|
||||
- Kafka/NATS event bus — Postgres insert + Redis pub/sub notify covers 4-worker scale.
|
||||
- Full event-sourcing (rebuild state from log) — projections are authoritative for current state; the log is authoritative for history and audit.
|
||||
- Per-company database — schema-shared, row-scoped by company_id; revisit if isolation demands grow.
|
||||
+161
@@ -0,0 +1,161 @@
|
||||
-- FarmHouse Holdings — core schema v1 (Slice 1 subset, PRD §14.3)
|
||||
-- Postgres 15+. Deferred entities (Schedule, KnowledgeItem, Incident, ...) added when their slice needs them.
|
||||
|
||||
CREATE TABLE company (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
slug text NOT NULL UNIQUE,
|
||||
name text NOT NULL,
|
||||
description text NOT NULL DEFAULT '',
|
||||
lifecycle_state text NOT NULL DEFAULT 'draft'
|
||||
CHECK (lifecycle_state IN ('draft','active','paused','archived')),
|
||||
-- repo, discord channel, office theme, autonomy policy, tz override
|
||||
config jsonb NOT NULL DEFAULT '{}',
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE employee (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
company_id uuid NOT NULL REFERENCES company(id),
|
||||
name text NOT NULL,
|
||||
title text NOT NULL,
|
||||
department text NOT NULL DEFAULT '',
|
||||
manager_id uuid REFERENCES employee(id),
|
||||
hermes_profile text, -- Hermes profile name (ADR-001)
|
||||
model_alias text NOT NULL DEFAULT 'local-default',
|
||||
authority_level int NOT NULL DEFAULT 2 CHECK (authority_level BETWEEN 0 AND 5),
|
||||
soul_version int NOT NULL DEFAULT 1, -- versioned system prompt (PRD §6.6)
|
||||
status text NOT NULL DEFAULT 'idle'
|
||||
CHECK (status IN ('idle','working','blocked','offline')),
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
UNIQUE (company_id, name)
|
||||
);
|
||||
|
||||
CREATE TABLE work_item (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
company_id uuid NOT NULL REFERENCES company(id),
|
||||
parent_id uuid REFERENCES work_item(id),
|
||||
type text NOT NULL CHECK (type IN
|
||||
('objective','milestone','epic','task','subtask','defect',
|
||||
'clarification','approval','research','proposal','experiment',
|
||||
'review','incident','postmortem')),
|
||||
title text NOT NULL,
|
||||
description text NOT NULL DEFAULT '',
|
||||
status text NOT NULL DEFAULT 'backlog' CHECK (status IN
|
||||
('backlog','ready','in_progress','blocked','review','done','cancelled')),
|
||||
priority text NOT NULL DEFAULT 'medium' CHECK (priority IN
|
||||
('critical','high','medium','low','debt','research','experiment','idea')),
|
||||
owner_id uuid REFERENCES employee(id),
|
||||
acceptance_criteria jsonb NOT NULL DEFAULT '[]',
|
||||
-- runaway caps, engine-enforced (PRD_ADDENDUM §3): max_iterations, max_tokens, max_wall_seconds
|
||||
budget jsonb NOT NULL DEFAULT '{"max_iterations":15,"max_tokens":100000,"max_wall_seconds":1800}',
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX work_item_dispatch_idx ON work_item (company_id, status, priority);
|
||||
|
||||
-- Append-only event log: single source of truth for history (PRD §14.2).
|
||||
CREATE TABLE event (
|
||||
event_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
company_id uuid REFERENCES company(id), -- NULL = holdings-level
|
||||
actor text NOT NULL, -- 'employee:<uuid>' | 'user:philip' | 'system'
|
||||
event_type text NOT NULL, -- see engine/schemas/events.py
|
||||
schema_version int NOT NULL DEFAULT 1,
|
||||
work_item_id uuid REFERENCES work_item(id),
|
||||
parent_event_id uuid REFERENCES event(event_id),
|
||||
correlation_id uuid, -- groups one logical exchange
|
||||
source text NOT NULL DEFAULT 'engine'
|
||||
CHECK (source IN ('engine','web','api','discord','hermes','scheduler','cli')),
|
||||
payload jsonb NOT NULL DEFAULT '{}',
|
||||
policy_decision jsonb, -- {allowed, rule_id, level, reason}
|
||||
provenance jsonb, -- {model, worker, tool, hermes_run_id}
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX event_company_time_idx ON event (company_id, created_at);
|
||||
CREATE INDEX event_work_item_idx ON event (work_item_id) WHERE work_item_id IS NOT NULL;
|
||||
|
||||
CREATE FUNCTION event_immutable() RETURNS trigger AS $$
|
||||
BEGIN RAISE EXCEPTION 'event log is append-only'; END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
CREATE TRIGGER event_no_update BEFORE UPDATE OR DELETE ON event
|
||||
FOR EACH ROW EXECUTE FUNCTION event_immutable();
|
||||
|
||||
CREATE TABLE approval (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
company_id uuid NOT NULL REFERENCES company(id),
|
||||
work_item_id uuid REFERENCES work_item(id),
|
||||
requested_by uuid NOT NULL REFERENCES employee(id),
|
||||
level int NOT NULL CHECK (level BETWEEN 3 AND 5),
|
||||
summary text NOT NULL,
|
||||
options jsonb NOT NULL DEFAULT '[]', -- clarification options (PRD §7.6)
|
||||
status text NOT NULL DEFAULT 'pending'
|
||||
CHECK (status IN ('pending','approved','rejected','expired')),
|
||||
decided_by text, -- 'user:philip' | 'policy:<rule_id>'
|
||||
rationale text,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
decided_at timestamptz
|
||||
);
|
||||
|
||||
CREATE TABLE artifact (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
company_id uuid NOT NULL REFERENCES company(id),
|
||||
work_item_id uuid REFERENCES work_item(id),
|
||||
kind text NOT NULL CHECK (kind IN
|
||||
('evidence_bundle','screenshot','video','trace','test_report',
|
||||
'build','preview','document','diff')),
|
||||
uri text NOT NULL, -- file path or object-store URI
|
||||
meta jsonb NOT NULL DEFAULT '{}',
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE tool_execution (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
company_id uuid NOT NULL REFERENCES company(id),
|
||||
employee_id uuid NOT NULL REFERENCES employee(id),
|
||||
work_item_id uuid REFERENCES work_item(id),
|
||||
tool text NOT NULL,
|
||||
args jsonb NOT NULL DEFAULT '{}',
|
||||
authority_level int NOT NULL,
|
||||
policy_decision jsonb NOT NULL, -- always recorded (PRD §16.1)
|
||||
status text NOT NULL DEFAULT 'running'
|
||||
CHECK (status IN ('running','ok','error','denied','timeout')),
|
||||
result_summary text,
|
||||
started_at timestamptz NOT NULL DEFAULT now(),
|
||||
finished_at timestamptz
|
||||
);
|
||||
|
||||
CREATE TABLE cost_entry (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
company_id uuid NOT NULL REFERENCES company(id),
|
||||
employee_id uuid REFERENCES employee(id),
|
||||
work_item_id uuid REFERENCES work_item(id),
|
||||
kind text NOT NULL CHECK (kind IN ('local_inference','consultant','tool','storage')),
|
||||
tokens_in bigint NOT NULL DEFAULT 0,
|
||||
tokens_out bigint NOT NULL DEFAULT 0,
|
||||
wall_seconds numeric NOT NULL DEFAULT 0,
|
||||
usd numeric NOT NULL DEFAULT 0, -- 0 for local; real for consultants
|
||||
meta jsonb NOT NULL DEFAULT '{}',
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX cost_company_time_idx ON cost_entry (company_id, created_at);
|
||||
|
||||
-- Deterministic policy: most specific enabled rule wins (company+employee > company > global).
|
||||
-- Engine denies by default when no rule matches (PRD §8: enforced in code, not prompts).
|
||||
CREATE TABLE policy_rule (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
company_id uuid REFERENCES company(id), -- NULL = global
|
||||
employee_id uuid REFERENCES employee(id), -- NULL = any employee
|
||||
tool_pattern text NOT NULL DEFAULT '*', -- glob: 'git.*', 'shell', '*'
|
||||
environment text NOT NULL DEFAULT 'sandbox'
|
||||
CHECK (environment IN ('sandbox','internal','external','production')),
|
||||
max_level int NOT NULL CHECK (max_level BETWEEN 0 AND 5),
|
||||
enabled boolean NOT NULL DEFAULT true,
|
||||
note text NOT NULL DEFAULT '',
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- Global defaults: read/propose/sandbox anywhere; nothing above level 2 without an explicit rule.
|
||||
INSERT INTO policy_rule (tool_pattern, environment, max_level, note) VALUES
|
||||
('*', 'sandbox', 2, 'global default: full sandbox freedom'),
|
||||
('*', 'internal', 1, 'global default: propose-only against internal env'),
|
||||
('*', 'external', 0, 'global default: observe-only external'),
|
||||
('*', 'production', 0, 'global default: observe-only production');
|
||||
@@ -0,0 +1,108 @@
|
||||
"""Event schema v1 — the versioned contract between agents, engine, and interfaces.
|
||||
|
||||
Two layers:
|
||||
* AgentMessage — what an employee (Hermes run) reports back. Small, flat,
|
||||
enforced at inference time via Ollama/Hermes structured output
|
||||
(qwen3.5:9b scored 100% validity on this shape; see benchmarks/RESULTS.md).
|
||||
* EventEnvelope — what the engine appends to the immutable event log after
|
||||
policy evaluation. Superset of agent messages: engine/system events too.
|
||||
|
||||
Rule from benchmarks: `task.blocked` from an agent is a TRIAGE state. The
|
||||
engine (manager step) reclassifies to clarification.requested /
|
||||
approval.requested before anything reaches the user.
|
||||
"""
|
||||
from datetime import datetime
|
||||
from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
|
||||
# ---------------------------------------------------------------- agent -> engine
|
||||
|
||||
AgentMessageType = Literal[
|
||||
"task.accepted",
|
||||
"task.progressed",
|
||||
"task.blocked",
|
||||
"task.completed",
|
||||
"clarification.requested",
|
||||
"approval.requested",
|
||||
"artifact.created",
|
||||
"test.started",
|
||||
"test.completed",
|
||||
"review.completed",
|
||||
"consultant.requested",
|
||||
"incident.declared",
|
||||
]
|
||||
|
||||
|
||||
class AgentMessage(BaseModel):
|
||||
"""Kept flat on purpose — 9B models degrade on nested schemas (PRD_ADDENDUM §2)."""
|
||||
|
||||
message_type: AgentMessageType
|
||||
work_item_id: str
|
||||
summary: str = Field(max_length=500)
|
||||
detail: str = ""
|
||||
# clarification/approval: 2-3 options with a recommendation (PRD §7.6)
|
||||
options: list[str] = []
|
||||
recommended_option: str = ""
|
||||
# artifact.created / test.completed
|
||||
artifact_uri: str = ""
|
||||
tests_passed: int | None = None
|
||||
tests_failed: int | None = None
|
||||
|
||||
|
||||
# JSON Schema handed to Hermes/Ollama `format` for enforcement at inference time.
|
||||
AGENT_MESSAGE_FORMAT: dict = AgentMessage.model_json_schema()
|
||||
|
||||
# ---------------------------------------------------------------- engine event log
|
||||
|
||||
EngineEventType = Literal[
|
||||
# lifecycle
|
||||
"company.created", "company.paused", "company.resumed", "company.archived",
|
||||
"employee.hired", "employee.updated",
|
||||
"work_item.created", "work_item.updated", "work_item.assigned",
|
||||
# dispatch loop
|
||||
"dispatch.sent", "dispatch.failed", "run.capped", # run.capped = runaway guard fired
|
||||
# decisions
|
||||
"clarification.answered", "approval.decided", "policy.denied",
|
||||
# accounting
|
||||
"cost.recorded",
|
||||
]
|
||||
|
||||
EventType = AgentMessageType | EngineEventType
|
||||
|
||||
Source = Literal["engine", "web", "api", "discord", "hermes", "scheduler", "cli"]
|
||||
|
||||
|
||||
class PolicyDecision(BaseModel):
|
||||
allowed: bool
|
||||
rule_id: UUID | None = None
|
||||
level: int
|
||||
reason: str = ""
|
||||
|
||||
|
||||
class Provenance(BaseModel):
|
||||
model: str = ""
|
||||
worker: str = ""
|
||||
tool: str = ""
|
||||
hermes_run_id: str = ""
|
||||
|
||||
|
||||
class EventEnvelope(BaseModel):
|
||||
"""Mirrors db/schema.sql `event` table. Append-only; never mutated."""
|
||||
|
||||
event_id: UUID
|
||||
company_id: UUID | None = None
|
||||
actor: str # 'employee:<uuid>' | 'user:philip' | 'system'
|
||||
event_type: EventType
|
||||
schema_version: int = SCHEMA_VERSION
|
||||
work_item_id: UUID | None = None
|
||||
parent_event_id: UUID | None = None
|
||||
correlation_id: UUID | None = None
|
||||
source: Source = "engine"
|
||||
payload: dict = {}
|
||||
policy_decision: PolicyDecision | None = None
|
||||
provenance: Provenance | None = None
|
||||
created_at: datetime
|
||||
Reference in new issue
Block a user