# Hayva.ai — Complete Application Summary **Application:** Hayva.ai **Package version:** 0.1.0 **Summary date:** 2026-08-26 **Implementation baseline summarized:** current working tree on `codex/foundation-phase-1` **Current maturity:** Production-hardened application baseline; production release evidence blocked ## 1. Executive summary Hayva.ai is a security-first personal AI operating system designed to give a person or team one controlled workspace for AI agents, tasks, communications, files, calendars, contacts, automation, research, browser work, and voice. Its defining rule is that a model may propose work, but trusted application code must authorize, execute, verify, and audit it. The current repository is a working foundation rather than the completed operating system. It contains a premium authenticated dashboard, an Agent Studio, workspace-scoped identity and authorization, durable AI executions, model-routing profiles, a typed OpenAI planning boundary, two verified Core read tools, a first-class one-VM-per-agent control plane, a persistent Playwright/Chromium browser runtime, a Computer control room with lifecycle and human takeover, append-only audit evidence, migrations, hardened container definitions, and an automated test suite. External communication/business integrations are not represented as connected. Email, calendar, WhatsApp, CRM, contacts, files, memory, voice, automations, notifications, analytics, and invoice workflows remain planned. Browser-computer control is an implemented initial capability with its own explicit limitations. The interface labels unavailable modules as `Planned` or `Unconfigured` instead of displaying simulated success. ## 2. Product objective The target product is a private operating layer where a user can: - create specialized AI agents with explicit tools, permissions, models, schedules, hierarchy, autonomy, and budgets; - issue natural-language commands and observe a durable execution timeline; - review or approve risky actions before they happen; - connect communications, calendar, files, contacts, CRM, browser sessions, and voice providers; - automate recurring workflows while retaining pause, takeover, revocation, and emergency-stop controls; - inspect evidence showing what an agent proposed, what the application authorized, what actually ran, and whether the result was verified; and - keep every owned record and capability isolated by workspace. The target is deliberately broader than the current implementation. The status sections below are the authority for what works today. ## 3. Current status at a glance | Area | Status | Current capability | | --- | --- | --- | | First-run setup | Implemented | One-time, bootstrap-capability-gated owner and workspace creation | | Authentication | Implemented product baseline | Password login/reset, encrypted TOTP MFA/recovery, absolute/idle sessions, CSRF, rotation/revocation, throttling, Security Center | | Workspace isolation | Implemented foundation | Workspace-scoped records, authorization queries, composite ownership constraints | | RBAC | Implemented foundation | Six seeded roles, granular permissions, server-side resolution and denial | | Dashboard | Implemented | Authenticated overview, real agent/execution counts, health and integration states | | Agent Studio | Implemented initial product | Create, configure, activate, disable, archive, duplicate, version, grant tools/permissions | | Model profiles | Implemented initial product | Workspace-owned OpenAI routing metadata and honest provider state; no credential storage | | AI planning | Implemented initial runtime | Typed structured plans through an isolated OpenAI Responses adapter | | Tool execution | Limited | Two Core-owned, low-risk read tools only | | Durable executions | Implemented initial runtime | Create, list, inspect, cancel, run, checkpoint, verify, and replay completed reads | | Delegation | Partial | Audited request and permission intersection; no queued dispatch | | Approval engine | Partial | Policy kernel, immutable payload binding, expiry and one-time decision checks; no full UI/API lifecycle | | Audit | Implemented foundation | Per-workspace hash chain, serialized appends, append-only database triggers | | PostgreSQL/Redis topology | Defined | Compose topology, migrations, health checks, isolated networks; not locally container-verified | | Email/calendar | Planned | No live OAuth or provider synchronization | | WhatsApp | Planned | No WhatsApp Web runtime or QR connection | | Dedicated AI computer | Implemented control plane; provider unconfigured | Exactly one full-VM record per activatable agent, templates, quotas, lifecycle, snapshots, resize, provider verification, and deny-host policy; real hypervisor, golden images, in-guest daemon, and production isolation evidence remain blocked | | In-computer browser | Implemented initial runtime | Isolated persistent Chromium profile per assigned VM, DOM tools, live JPEG view, approvals, takeover, pause/stop, recovery, and emergency stop; current browser worker is not the VM provider | | Contacts/CRM/tasks | Planned | No product APIs or UI modules | | Files/knowledge/memory | Planned | No file workspace, embeddings, pgvector retrieval, or memory UI | | Automation engine | Planned | No triggers, conditions, visual builder, or scheduled workflow worker | | Voice/calls | Planned | No telephony or realtime voice pipeline | | Analytics/costs | Planned | Token counts exist; complete spend, latency, quality, and operational analytics do not | | Invoice workflow/page | Not present | No invoice page, billing module, email invoice extraction, or file-delivery workflow | | Production deployment | Blocked | TLS/DB-role/backup definitions exist; requires Docker/PostgreSQL/VPS execution, egress/storage controls, monitoring, drills, and review | ## 4. Implemented user experience ### 4.1 Secure first run The installation starts in a one-time setup state. An operator provides an out-of-band bootstrap capability of at least 32 characters. A successful setup transaction creates: - the first workspace; - the owner user and active membership; - the default role and permission catalogue; - the Owner role assignment; - a Personal Executive Assistant with read-only tools and approval-required autonomy; - the initial agent version and permission grants; - an unconfigured dedicated Ubuntu computer record, its private browser profile, and durable workspace control-state row; - a hashed device session and CSRF token; and - a chained audit event. The database stores a locked singleton installation record, so setup cannot run twice even if the bootstrap file remains mounted accidentally. ### 4.2 Authentication and session experience Initialized installations route users to sign-in. The current authentication flow includes: - Argon2 password hashing; - generic credential errors to reduce account enumeration; - dummy-hash timing equalization for unknown accounts; - Redis-backed source and invalid-account throttles; - correct-password protection from attacker-generated invalid-account counters; - hashed session and CSRF tokens in the database; - `HttpOnly`, `SameSite=Strict` session cookies and production `Secure` cookies; - active-user and active-membership checks on authenticated requests; - current-user and device-session history endpoints; - RFC 6238 TOTP with AES-256-GCM protected secrets, replay rejection, and recovery codes; - operator-assisted, short-lived, one-use password-reset capabilities; - absolute and idle expiry, session rotation, per-device revocation, and global invalidation; - CSRF-protected logout and logout-all; - a responsive Security Center for MFA, password, and device lifecycle; and - append-only login and authentication audit evidence. ### 4.3 Premium dashboard The responsive React dashboard displays only authenticated, server-backed state: - workspace identity and signed-in user; - resolved permission count; - active/configured agent counts; - persisted execution history and completed-run counts; - honest AI, email/calendar, and browser integration state; - a safe command surface connected to durable executions; and - navigation labels that distinguish working routes from planned modules. The command form is disabled when the AI provider is unconfigured or unavailable. No prompt is sent until provider readiness is reported. A browser QA pass verified the desktop experience and a 390-pixel mobile breakpoint without horizontal overflow or console errors. ### 4.4 Agent Studio Agent Studio is the primary implemented product module. An authorized user can: - view workspace agents and lifecycle state; - create a disabled agent with a name, purpose, description, instructions, and personality; - choose one of six autonomy levels; - select an enabled workspace model profile or the deployment default; - grant only registered tools and their matching permissions; - activate, disable, archive, or duplicate an agent; - observe the current configuration version; and - configure owner-defined model-routing profiles. Agent mutations are permission checked, CSRF protected, workspace scoped, audited, and versioned. Manager and escalation references cannot cross a workspace, and manager cycles are rejected. The initial UI does not yet expose full edit forms, schedules, hierarchy, delegation, version history, budgets, execution limits, or specialist templates, although most of those fields and backend operations already exist. ### 4.5 Computer control room The `/computer` route is a working premium control surface. An authorized user can: - inspect one dedicated full-VM record per agent, choose a built-in OS template and resources, and see honest compute-provider state; - request audited, idempotent VM provision/start/stop/restart/pause/resume/destroy, snapshot, restore, and resize operations when a real provider is configured; - use a workspace-owned persistent browser profile bound to that agent and VM; - start, list, and inspect durable browser sessions; - see session/control/service/emergency state and a polled live JPEG frame; - navigate, observe the DOM, type through an approval proposal, and capture screenshots; - inspect action, approval, artifact, and checkpoint evidence; - take exclusive human control, click the live frame, send keyboard keys/text, and explicitly return control to AI after a snapshot and fresh observation; - pause, resume, stop, or explicitly recover a failed/paused persistent session; and - emergency-stop the entire workspace, with verified versus unverified worker outcomes and no automatic restart on workspace resume. The current UI implements click and keyboard takeover controls. The worker contract also supports pointer movement and scrolling, but the UI does not yet expose them. Live view is polling JPEG, not WebRTC/WebSocket video. Downloads, uploads, clipboard, service workers, and WebSockets are disabled. On this workstation the compute provider is deliberately `unconfigured`, so the UI does not claim that a browser container is a full computer. Runtime agents remain disabled until a real VM is provider-verified. The immediate production blocker is deployment of the restricted hypervisor gateway, supported golden images, authenticated in-guest daemon, encrypted storage, and isolation test evidence. ## 5. Runtime agents and autonomy An agent is a durable workspace entity, not merely a prompt. Its current model supports: - identity, description, purpose, system instructions, and personality; - lifecycle state: `disabled`, `active`, or `archived`; - six autonomy levels; - preferred and fallback model-profile references; - explicit permission and tool grants; - manager and escalation relationships; - timezone-aware working schedules; - budget currency and limit fields; - execution-limit fields; - immutable configuration versions; and - audited delegation requests. ### Autonomy levels | Level | Name | Intended meaning | | --- | --- | --- | | 0 | Disabled | Agent cannot be activated or operate | | 1 | Observe | Read and inspect only | | 2 | Draft | Prepare proposed output without performing writes | | 3 | Approval | Writes require explicit approval | | 4 | Controlled | Selected lower-risk writes may run under deterministic policy | | 5 | Autonomous | Broadest allowed autonomy, still bounded by permissions and platform policy | Autonomy never grants a missing permission. The effective runtime authority is the intersection of the human actor's current permissions, the agent's current permission grants, the agent's enabled tools, and the platform tool registry. ## 6. AI planning and execution ### 6.1 Control principle > Models propose; the application authorizes and executes. Model output is treated as untrusted structured input. The isolated AI service can produce a typed plan, but it cannot access the Core database, grant permissions, invoke product tools directly, or mark an execution successful. ### 6.2 Current execution flow ```mermaid sequenceDiagram participant U as Authenticated user participant W as Web application participant C as Core API participant A as AI Agent service participant T as Core read-tool runtime participant D as PostgreSQL U->>W: Submit instruction W->>C: Create execution with CSRF token C->>C: Resolve workspace, actor, agent, tools, and permissions C->>D: Persist planning execution and audit event C->>A: Request typed plan with explicit allowlist A-->>C: Schema-constrained SupervisorPlan C->>C: Validate tools, arguments, dependencies, risk, and permission C->>D: Persist plan and ordered steps W->>C: Run execution C->>C: Reauthorize original actor and agent before each step C->>T: Execute allowed read tool T-->>C: Result and verification evidence C->>D: Persist checkpoint, status, token usage, and audit event C-->>W: Durable execution result ``` ### 6.3 Implemented tools Only these low-risk, read-only tools are registered: | Tool | Permission | Purpose | | --- | --- | --- | | `system.status.read` | `system.status.read` | Read trusted application status | | `workspace.profile.read` | `workspace.profile.read` | Read the authorized workspace profile | The runtime validates tool registration, argument schemas, dependency ordering, timeouts, cancellation, and current authority before every step. Completed read executions can be replayed without running the tools again. Side-effecting tools are intentionally absent. ### 6.4 Provider boundary The AI service currently has an OpenAI Responses adapter using Pydantic structured parsing. It: - requests a strict `SupervisorPlan`; - disables provider-side response storage; - sends a one-way safety identifier instead of raw user/workspace identifiers in provider input; - supports an owner-selected model identifier passed from Core; - reports provider state as `configured`, `unconfigured`, or `unavailable`; and - converts missing configuration into an explicit error rather than simulated output. Provider credentials are file-mounted deployment secrets. Model profiles contain routing metadata only: profile key, display name, provider, model identifier, enabled state, and declared capabilities. If planning cannot run, Core preserves a durable `blocked` execution with an error code and audit evidence. Fallback execution, streaming, final-answer synthesis, embeddings, vision workflows, realtime voice, measured costs, and capability-aware routing are incomplete. ## 7. System architecture Hayva uses a modular-monolith control plane plus isolated services for boundaries that require separate credentials, network access, or failure isolation. ```mermaid flowchart LR User[User browser] --> N[Nginx gateway] N --> Web[React web application] N --> Core[FastAPI Core API] Core --> DB[(PostgreSQL + pgvector)] Core --> Redis[(Redis)] Core --> AI[Isolated AI Agent] Core --> Computer[Isolated Computer Agent] AI --> Provider[OpenAI API] Computer --> WebSites[Public websites] subgraph Private application network Web Core AI Computer end subgraph Private data network DB Redis end subgraph Future isolated runtimes Voice[Voice Worker] end ``` ### Component ownership | Component | Current responsibility | Explicitly does not own | | --- | --- | --- | | Nginx | Local gateway, edge routing, proxy-header normalization, detailed-health blocking | Authorization or policy | | Web | Setup, sign-in, dashboard, Agent Studio, Computer control room, honest state presentation | Final authorization or provider/browser execution | | Core API | Identity, membership, RBAC, policy, approvals, agents, executions, computer control plane, leases, audit | Model inference or Chromium process ownership | | PostgreSQL | Authoritative durable state, constraints, migrations, audit chain, outbox | Ephemeral locks or queue delivery | | Redis | Login throttling and future queues/locks/cache | Authoritative business state | | AI Agent | Provider adapter, tool contracts, typed planning | Core database access, final authorization, or tool execution | | Computer Agent | Persistent Chromium, DOM actions, frames/artifacts, human input, request/fence enforcement | Policy decisions, database access, or public control endpoint | | Voice Worker | Planned telephony/realtime voice | Not implemented | ### Network boundaries The Compose topology defines five networks: - `edge` for the gateway; - `app` as a private application network; - `data` as a private PostgreSQL/Redis network; and - `egress` for AI provider access; and - `browser-egress` for Computer Agent public-site access. Only Nginx publishes a host port, currently `127.0.0.1:8080`. PostgreSQL, Redis, Core, Web, AI, and Computer Agent are not directly host-published. AI Agent and Computer Agent have no data-network attachment or database credential. They use separate service tokens and separate egress networks. ## 8. Technology stack ### Backend and AI - Python 3.12+ - FastAPI - Pydantic and Pydantic Settings - SQLAlchemy 2 async ORM - Alembic migrations - PostgreSQL 17 with pgvector - Redis 8 - HTTPX - OpenAI Python SDK and Responses API - Playwright 1.61 and Chromium persistent contexts - Argon2 through `pwdlib` - Uvicorn ### Frontend - React 19 - TypeScript - Vite - Tailwind Vite integration plus application CSS - Lucide icons - Vitest, Testing Library, and jsdom - Node.js 24 and pnpm 11 ### Infrastructure - Docker Compose - Nginx Alpine - non-root/read-only service containers where applicable - private application/data networks plus separate AI and browser egress - file-mounted bootstrap, internal-service, Redis, and provider secrets ## 9. Authorization model Authorization is deny-by-default and resolved server-side from an authenticated session and active workspace membership. A client-supplied workspace identifier does not establish access. Seeded roles are: - Owner - Administrator - Assistant Manager - Human Assistant - Viewer - Developer Permissions are granular and cover agents, executions, approvals, settings, audit, communications, calendar, tasks, CRM, memory, workspace profile, system status, and computer observation/control. Roles are collections of permissions, not authorization shortcuts. Owned resources include `workspace_id` and use workspace-led queries, uniqueness, and foreign-key constraints. Cross-workspace resources generally return an inaccessible/not-found response rather than disclosing existence. ## 10. Security and reliability controls ### Implemented controls - Atomic, permanently one-time owner bootstrap - Argon2 password hashing - Generic login failures and timing equalization - Redis login throttling without raw email/IP keys - Hashed session and CSRF tokens - SameSite cookies and production Secure cookies - Active-user and active-membership enforcement - CSRF protection for authenticated mutations - Workspace-scoped authorization - Permission-escalation and hierarchy-cycle rejection - Typed plan and tool schemas - Independent Core authorization of model-proposed tools - Reauthorization immediately before runtime steps - Bounded read-tool timeouts and cancellation checks - Canonical payload hashes for approvals and instructions - Durable execution checkpoints and explicit failure states - Idempotent replay for completed read executions - Per-workspace audit hash chains - Serialized audit appends through a locked workspace head - Database triggers rejecting audit/login update and deletion - Request correlation identifiers and redacted health failures - File-mounted provider and internal-service secrets - Non-root/read-only container settings and dropped capabilities where configured - Private PostgreSQL/Redis exposure and isolated AI data access - Workspace-scoped persistent computer profiles and session records - Exclusive expiring AI/human/system leases with monotonically increasing fencing tokens - Core-owned browser tool registry and actor/agent/tool authority intersection - Required browser-action idempotency keys and exact replay/conflict handling - Approval binding to the exact raw browser payload hash with redacted durable parameters - DOM-first action verification, durable checkpoints, and integrity-checked screenshot artifacts - Public-destination URL checks, redirect checks, downloads disabled, and blocked service workers/ WebSockets in the initial browser runtime - Human takeover/return-to-AI re-observation, explicit recovery, and durable workspace emergency stop - Non-root, capability-dropped, read-only Computer Agent container definition with resource/seccomp controls and no database network ### Required before production - Exercise authentication abuse/recovery and database-role controls in a production-like environment - Verify separate migration/runtime roles and append-only audit grants against live PostgreSQL - Encrypted sensitive execution/instruction storage and retention jobs - Enforcement of pause/emergency-stop across future queues and every future worker - Generalized external-write action intents beyond browser controls - Network-enforced browser egress and adversarial DNS-rebinding protection beyond application checks - Encrypted browser profile volumes, credential/session revocation, retention deletion, and restores - A revocation-aware realtime transport beyond the current signed, short-lived HTTP capabilities - Safe download/upload/clipboard handling with quarantine, scanning, and path controls - Durable browser-job dispatch and automatic worker crash detection beyond current lease/reconciliation maintenance - Webhook, upload, and file-path protections for future integrations - Credential lifecycle, OAuth scopes, rotation, revocation, and disconnect flows - Complete structured logging, metrics, traces, alerts, cost monitoring, and error reporting - Execute the defined container/IaC/dependency scans and inspect public ports - Execute the encrypted backup and isolated restore drill - TLS, firewall, SSH, VPS, and production-user hardening - Security review and complete production checklist evidence ## 11. Persistence model PostgreSQL is the authoritative store. Redis is limited to ephemeral coordination. ### Implemented aggregate groups | Group | Main records | | --- | --- | | Installation | `installation_state` | | Identity | `users`, `workspaces`, `workspace_memberships` | | Authorization | `roles`, `permissions`, `role_permissions`, `membership_roles` | | Authentication | `device_sessions`, `login_events`, reset-token foundation | | Policy/approval | policies, immutable approval requests/rules | | Agents | `agents`, `agent_versions`, `agent_permissions`, `agent_tools`, `agent_schedules` | | Models | `ai_model_profiles` | | Computers | `computer_profiles`, `workspace_control_states`, `computer_sessions`, `computer_control_leases`, `computer_actions`, `computer_artifacts`, `computer_checkpoints` | | Delegation | `agent_delegations` | | Executions | `executions`, `execution_steps` | | Reliability | idempotency and outbox foundation | | Audit | `audit_events` and per-workspace audit head/sequence | All current identifiers are application-generated UUIDv4 values. Time is stored in UTC with IANA timezones retained for presentation. Money fields use integer minor units and ISO currency codes. There are six explicit Alembic revisions covering the foundation, executions, runtime permissions, agents, model profiles, and the computer runtime. SQLite migration upgrade, drift check, downgrade, and re-upgrade are automated. PostgreSQL-specific migration and trigger CI exists but has not run on the current workstation. ## 12. Current public API ### Health - `GET /health` - `GET /health/database` - `GET /health/redis` - `GET /health/readiness` - `GET /health/ai` - `GET /health/browser` ### Setup and authentication - `GET /api/v1/setup/status` - `POST /api/v1/setup/owner` - `POST /api/v1/auth/login` - `POST /api/v1/auth/logout` - `POST /api/v1/auth/logout-all` - `GET /api/v1/auth/me` - `GET /api/v1/auth/sessions` ### Executions - `POST /api/v1/executions` - `GET /api/v1/executions` - `GET /api/v1/executions/{execution_id}` - `POST /api/v1/executions/{execution_id}/run` - `POST /api/v1/executions/{execution_id}/cancel` ### Agents - `GET /api/v1/agents` - `GET /api/v1/agents/{agent_id}` - `POST /api/v1/agents` - `PATCH /api/v1/agents/{agent_id}` - `POST /api/v1/agents/{agent_id}/status` - `POST /api/v1/agents/{agent_id}/duplicate` - `DELETE /api/v1/agents/{agent_id}` - `POST /api/v1/agents/{source_agent_id}/delegations` ### Model profiles - `GET /api/v1/model-profiles` - `POST /api/v1/model-profiles` - `PUT /api/v1/model-profiles/{profile_id}` - `DELETE /api/v1/model-profiles/{profile_id}` ### Computers - `GET /api/v1/computer/status` - `GET|POST /api/v1/computer/profiles` - `PUT /api/v1/computer/profiles/{profile_id}` and profile `revoke` - `GET|POST /api/v1/computer/sessions` - `GET /api/v1/computer/sessions/{session_id}` - session `actions`, `takeover`, `return-control`, `pause`, `resume`, `stop`, and `recover` mutations - session `human/pointer`, `human/keyboard`, and `frame` routes - `GET /api/v1/computer/approvals` and action `decision` - `GET /api/v1/computer/artifacts/{artifact_id}` - `POST /api/v1/computer/emergency-stop` and `/emergency-resume` The API carries or generates bounded request IDs. Application errors contain a stable code, safe message, and correlation ID. The target uniform success envelope and complete normalization of validation/unhandled errors are not finished. ### Internal AI API - `GET /health` - `GET /v1/tools` with service authentication - `POST /v1/plans` with service authentication ### Internal Computer API - `GET /health` - `POST /v1/sessions` - session `pause`, `resume`, `stop`, `snapshot`, `actions`, `human/pointer`, `human/keyboard`, and `frame` routes with service authentication and fencing tokens - `GET /v1/artifacts/{workspace_id}/{session_id}/{artifact_id}` with service authentication ## 13. Deployment model The target host is Ubuntu 24.04 LTS with Docker Engine and Compose. The declared services are: - `nginx` - `web` - `core-api` - `ai-agent` - `computer-agent` - `migrate` - `postgres` - `redis` The one-shot migration service must succeed before Core starts. Health checks exist for Web, Core, AI, Computer Agent, PostgreSQL, and Redis. Service filesystems are read-only where practical, temporary paths use `tmpfs`, and `no-new-privileges` is enabled. Required operator configuration includes: - application secret; - database and Redis credentials; - one-time owner bootstrap file; - Core-to-AI service-token file; - separate Core-to-Computer service-token file; - OpenAI API-key file and explicit deployment model when enabled; and - production URL, TLS, backup, monitoring, and host-security configuration. The current Compose definition is a development foundation, not permission to expose the application publicly. Docker is unavailable in the current workstation shell, so container build, live PostgreSQL/Redis integration, and Compose health behavior have not been locally verified. ## 14. Verification evidence The latest local release checks on 2026-08-25 produced: - Ruff: passed - Pytest: **63 passed** - Frontend TypeScript: passed - Frontend Vitest: **5 passed** - Frontend production bundle: passed - Python dependency integrity (`pip check`): passed - Alembic SQLite upgrade/drift/downgrade/re-upgrade: passed through tests - Real-browser desktop/mobile Agent Studio and Computer control-room flows: passed - Secret-pattern scan of repository sources: no matches - Git diff whitespace check: passed Browser verification covered setup, dashboard loading, model-profile creation, least-privilege agent creation, model assignment, activation/versioning, duplication, disabled provider commands, mobile drawer sizing, console output, and horizontal overflow. It found and led to a regression test for an asynchronous setup-form lifecycle defect. The Computer pass additionally covered real session start, public navigation, DOM observation, live frames, takeover, return-to-AI, session recovery, and a 390px layout. It found and fixed conflicting worker request IDs during return control and a mismatched action-timeline field. The OpenAI tests use provider-compatible fakes. No live model call was made. CI workflows exist, including a PostgreSQL probe, but have not run because this repository has no configured remote. ## 15. Repository structure ```text Hayva.ai/ ├── docs/ Engineering, security, operations, and product records ├── infra/ Nginx, Redis, Computer hardening, bootstrap, and secret templates ├── migrations/ Explicit Alembic revisions ├── services/ │ ├── core-api/ Identity, authorization, agents, executions, audit, read tools │ ├── ai-agent/ Typed planning service and OpenAI adapter │ └── computer-agent/ Isolated persistent Chromium runtime and DOM tools ├── tests/ Backend, migration, security, AI, and infrastructure tests ├── web/ React/TypeScript product interface ├── AGENTS.md Mandatory engineering rules ├── docker-compose.yml Development container topology ├── pyproject.toml Python package and tooling configuration └── .env.example Non-secret environment template ``` ## 16. Roadmap by master phase | Phase | Scope | Current state | | --- | --- | --- | | 1 | Foundation: repository, containers, DB, Redis, Nginx, auth, RBAC, dashboard | Substantial implementation; production evidence and auth/RBAC lifecycle incomplete | | 2 | AI abstraction, supervisor, runtime agents, tools, executions | Initial typed planner and verified read runtime implemented | | 3 | Agent Builder, permissions, models, autonomy, hierarchy, delegation | Main backend and UI implemented; templates, version-history UI, queues, budgets incomplete | | 4 | Short/long-term memory and pgvector UI | Planned | | 5 | Persistent Chromium computer and recovery | Initial product/runtime implemented; encryption, queue, crash automation, and production drills incomplete | | 6 | Live computer, streaming, takeover, pause, emergency stop | Initial polling-frame control room implemented; hardened realtime transport and richer human input incomplete | | 7 | WhatsApp Web | Planned | | 8 | Contacts and CRM | Planned | | 9 | Tasks, reminders, scheduling | Planned | | 10 | Google/Microsoft email and calendar | Planned | | 11 | Unified inbox | Planned | | 12 | Automation engine | Planned | | 13 | Voice calling | Planned | | 14 | Analytics, costs, monitoring | Planned | | 15 | Security hardening | Application/deployment controls implemented; environment evidence and independent review incomplete | | 16 | Complete end-to-end testing | Strong local coverage; production-like E2E/load/failure campaign incomplete | | 17 | Production deployment | TLS/DB-role/backup definitions implemented; target-host execution blocked | ## 17. Highest-priority next work 1. Verify the complete Compose stack and PostgreSQL-only invariants on a Docker-capable host. 2. Exercise the separate migration/runtime roles and audit restrictions on live PostgreSQL. 3. Encrypt browser profile storage, enforce host-level browser egress, and run container restart, emergency-stop, recovery, and backup/restore drills. 4. Add durable queued execution/delegation/browser dispatch and automatic worker crash detection; retain the implemented signed capabilities and unknown-outcome reconciliation boundary. 5. Finish Agent Studio templates, version history, schedule/hierarchy UI, and measured budget and concurrency enforcement. 6. Complete RBAC administration and production authentication abuse/recovery drills. 7. Add encrypted sensitive-payload storage, retention, deletion/export, backups, and restore tests. 8. Build integrations in roadmap order while preserving truthful `unconfigured` state until real owner credentials and authorization exist. 9. Complete observability, security testing, production networking, and the readiness checklist. ## 18. Honest limitations - The application code is production-hardened, but production release remains blocked on target-host evidence. - No external integration is currently connected. - Browser write tools are registered but approval-gated at the default agent autonomy; communication, financial, file, and other integration write tools are not implemented. - No invoice page or invoice workflow exists. - The product does not yet send email, messages, calendar invitations, WhatsApp content, files, or calls. - The persistent browser computer and live takeover experience are implemented initial capabilities, but are not production-ready: profile encryption, durable queueing/crash automation, controlled network egress, hardened streaming, downloads/uploads, and recovery drills remain. - Agent instructions and execution plans are currently plaintext in the database. - Budget and concurrency fields are stored but not enforced against measured consumption. - Fallback models are represented in configuration but automated failover is incomplete. - Delegation requests are durable and audited but not queued or executed. - The dashboard exposes Overview, Agent Studio, and Computer as working product routes; later modules are visibly planned. These limitations are deliberate release boundaries. Missing credentials or services produce `unconfigured`, `unavailable`, `blocked`, or failed states—never simulated success. ## 19. Engineering commands Install backend development dependencies: ```bash python -m pip install --upgrade pip python -m pip install -e ".[dev]" ``` Install the frontend lockfile: ```bash cd web pnpm install --frozen-lockfile ``` Run required verification from the repository root: ```bash ruff check . pytest cd web pnpm typecheck pnpm test pnpm build ``` For deployment preparation, copy `.env.example` to `.env`, create the protected secret files described in [DEPLOYMENT.md](DEPLOYMENT.md), validate the configuration, and run the migration service before Core. Do not expose the development Compose stack publicly. ## 20. Authoritative supporting documents - [Architecture](ARCHITECTURE.md) - [Security architecture](SECURITY.md) - [API contract](API.md) - [AI system](AI_SYSTEM.md) - [Database](DATABASE.md) - [Deployment](DEPLOYMENT.md) - [Operations](OPERATIONS.md) - [Testing](TESTING.md) - [Progress and blockers](PROGRESS.md) - [Production readiness report](FINAL_PRODUCTION_REPORT.md) - [Production checklist](PRODUCTION_CHECKLIST.md) - [Backlog](BACKLOG.md) This summary is a consolidated orientation document. When implementation changes, update this file together with the authoritative domain document and `PROGRESS.md`.