feat: add Atomizer HQ multi-agent cluster infrastructure
- 8-agent OpenClaw cluster (Manager, Tech-Lead, Secretary, Auditor, Optimizer, Study-Builder, NX-Expert, Webster) - Orchestration engine: orchestrate.py (sync delegation + handoffs) - Workflow engine: YAML-defined multi-step pipelines - Agent workspaces: SOUL.md, AGENTS.md, MEMORY.md per agent - Shared skills: delegate, orchestrate, atomizer-protocols - Capability registry (AGENTS_REGISTRY.json) - Cluster management: cluster.sh, systemd template - All secrets replaced with env var references
This commit is contained in:
116
hq/workspaces/shared/skills/orchestrate/README.md
Normal file
116
hq/workspaces/shared/skills/orchestrate/README.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# Orchestration Engine — Atomizer HQ
|
||||
|
||||
> Multi-instance synchronous delegation, workflow pipelines, and inter-agent coordination.
|
||||
|
||||
## Overview
|
||||
|
||||
The Orchestration Engine enables structured communication between 8 independent OpenClaw agent instances running on Discord. It replaces fire-and-forget delegation with synchronous handoffs, chaining, validation, and reusable YAML workflows.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ LAYER 3: WORKFLOWS │
|
||||
│ YAML multi-step pipelines │
|
||||
│ (workflow.py — parallel, sequential, gates) │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ LAYER 2: SMART ROUTING │
|
||||
│ Capability registry + channel context │
|
||||
│ (AGENTS_REGISTRY.json + fetch-channel-context) │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ LAYER 1: ORCHESTRATION CORE │
|
||||
│ Synchronous delegation + result return │
|
||||
│ (orchestrate.py — inotify + handoffs) │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ EXISTING INFRASTRUCTURE │
|
||||
│ 8 OpenClaw instances, hooks API, shared fs │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `orchestrate.py` | Core delegation engine — sends tasks, waits for handoff files via inotify |
|
||||
| `orchestrate.sh` | Thin bash wrapper for orchestrate.py |
|
||||
| `workflow.py` | YAML workflow engine — parses, resolves deps, executes pipelines |
|
||||
| `workflow.sh` | Thin bash wrapper for workflow.py |
|
||||
| `fetch-channel-context.sh` | Fetches Discord channel history as formatted context |
|
||||
| `metrics.py` | Analyzes handoff files and workflow runs for stats |
|
||||
| `metrics.sh` | Thin bash wrapper for metrics.py |
|
||||
|
||||
## Usage
|
||||
|
||||
### Single delegation
|
||||
```bash
|
||||
# Synchronous — blocks until agent responds
|
||||
python3 orchestrate.py webster "Find CTE of Zerodur" --caller manager --timeout 120
|
||||
|
||||
# With channel context
|
||||
python3 orchestrate.py tech-lead "Review thermal margins" --caller manager --channel-context technical --channel-messages 20
|
||||
|
||||
# With validation
|
||||
python3 orchestrate.py webster "Research ULE properties" --caller manager --validate --timeout 120
|
||||
```
|
||||
|
||||
### Workflow execution
|
||||
```bash
|
||||
# Dry-run (validate without executing)
|
||||
python3 workflow.py quick-research --input query="CTE of ULE" --caller manager --dry-run
|
||||
|
||||
# Live run
|
||||
python3 workflow.py quick-research --input query="CTE of ULE" --caller manager --non-interactive
|
||||
|
||||
# Material trade study (3-step pipeline)
|
||||
python3 workflow.py material-trade-study \
|
||||
--input materials="Zerodur, Clearceram-Z HS, ULE" \
|
||||
--input requirements="CTE < 0.01 ppm/K" \
|
||||
--caller manager --non-interactive
|
||||
```
|
||||
|
||||
### Metrics
|
||||
```bash
|
||||
python3 metrics.py text # Human-readable
|
||||
python3 metrics.py json # JSON output
|
||||
```
|
||||
|
||||
## Handoff Protocol
|
||||
|
||||
Agents write structured JSON to `/home/papa/atomizer/handoffs/{runId}.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"schemaVersion": "1.0",
|
||||
"runId": "orch-...",
|
||||
"agent": "webster",
|
||||
"status": "complete|partial|blocked|failed",
|
||||
"result": "...",
|
||||
"artifacts": [],
|
||||
"confidence": "high|medium|low",
|
||||
"notes": "...",
|
||||
"timestamp": "ISO-8601"
|
||||
}
|
||||
```
|
||||
|
||||
## ACL Matrix
|
||||
|
||||
| Caller | Can delegate to |
|
||||
|--------|----------------|
|
||||
| manager | All agents |
|
||||
| tech-lead | webster, nx-expert, study-builder, secretary |
|
||||
| optimizer | webster, study-builder, secretary |
|
||||
| Others | Cannot sub-delegate |
|
||||
|
||||
## Workflow Templates
|
||||
|
||||
- `quick-research.yaml` — 2 steps: Webster research → Tech-Lead validation
|
||||
- `material-trade-study.yaml` — 3 steps: Webster research → Tech-Lead evaluation → Auditor review
|
||||
- `design-review.yaml` — 3 steps: Tech-Lead + Optimizer (parallel) → Auditor consolidation
|
||||
|
||||
## Result Storage
|
||||
|
||||
- Individual handoffs: `/home/papa/atomizer/handoffs/orch-*.json`
|
||||
- Sub-delegations: `/home/papa/atomizer/handoffs/sub/`
|
||||
- Workflow runs: `/home/papa/atomizer/handoffs/workflows/{workflow-run-id}/`
|
||||
- Per-step: `{step-id}.json`
|
||||
- Summary: `summary.json`
|
||||
Reference in New Issue
Block a user