feat: one-click memory graduation button + host watcher
Closes the graduation UX loop: no more SSH required to populate the
entity graph from memories. Click button → host watcher picks up
→ graduation runs → entity candidates appear in the same triage UI.
New API endpoints (src/atocore/api/routes.py):
- POST /admin/graduation/request: takes {project, limit}, writes flag
to project_state. Host watcher picks up within 2 min.
- GET /admin/graduation/status: returns requested/running/last_result
fields for UI polling.
Triage UI (src/atocore/engineering/triage_ui.py):
- Graduation bar with:
- 🎓 Graduate memories button
- Project selector populated from registry (or "all projects")
- Limit number input (default 30, max 200)
- Status message area
- Poll every 10s until is_running=false, then auto-reload the page to
show new entity candidates in the Entity section below
- Graduation bar appears on both populated and empty triage page
states so you can kick off graduation from either
Host watcher (deploy/dalidou/graduation-watcher.sh):
- Mirrors auto-triage-watcher.sh pattern: poll, lock, clear flag,
run, record result, unlock
- Parses {project, limit} JSON from the flag payload
- Runs graduate_memories.py with those args
- Records graduation_running/started/finished/last_result in project
state for the UI to display
- Lock file prevents concurrent runs
Install on host (one-time, via cron):
*/2 * * * * /srv/storage/atocore/app/deploy/dalidou/graduation-watcher.sh \
>> /home/papa/atocore-logs/graduation-watcher.log 2>&1
This completes the Phase 5 self-service loop: queue triage happens
autonomously via the 3-tier escalation (shipped in 3ca1972); entity
graph population happens autonomously via a button click. No shell
required for daily use.
Tests: 366 passing (no new tests — UI + shell are integration-level).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
117
deploy/dalidou/graduation-watcher.sh
Normal file
117
deploy/dalidou/graduation-watcher.sh
Normal file
@@ -0,0 +1,117 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
#
|
||||||
|
# deploy/dalidou/graduation-watcher.sh
|
||||||
|
# ------------------------------------
|
||||||
|
# Host-side watcher for on-demand memory→entity graduation from the web UI.
|
||||||
|
#
|
||||||
|
# The /admin/triage page has a "🎓 Graduate memories" button that POSTs
|
||||||
|
# to /admin/graduation/request with {project, limit}. The container
|
||||||
|
# writes this to project_state (atocore/config/graduation_requested_at).
|
||||||
|
#
|
||||||
|
# This script runs on the Dalidou HOST (where claude CLI lives), polls
|
||||||
|
# for the flag, and runs graduate_memories.py when seen.
|
||||||
|
#
|
||||||
|
# Installed via cron every 2 minutes:
|
||||||
|
# */2 * * * * /srv/storage/atocore/app/deploy/dalidou/graduation-watcher.sh \
|
||||||
|
# >> /home/papa/atocore-logs/graduation-watcher.log 2>&1
|
||||||
|
#
|
||||||
|
# Safety:
|
||||||
|
# - Lock file prevents concurrent runs
|
||||||
|
# - Flag cleared before processing so duplicate clicks queue at most one re-run
|
||||||
|
# - Fail-open: any error logs but doesn't break the host
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ATOCORE_URL="${ATOCORE_URL:-http://127.0.0.1:8100}"
|
||||||
|
APP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
LOCK_FILE="/tmp/atocore-graduation.lock"
|
||||||
|
LOG_DIR="/home/papa/atocore-logs"
|
||||||
|
mkdir -p "$LOG_DIR"
|
||||||
|
|
||||||
|
TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||||
|
log() { printf '[%s] %s\n' "$TS" "$*"; }
|
||||||
|
|
||||||
|
# Fetch the flag via API
|
||||||
|
STATE_JSON=$(curl -sSf --max-time 5 "$ATOCORE_URL/project/state/atocore" 2>/dev/null || echo "{}")
|
||||||
|
REQUESTED=$(echo "$STATE_JSON" | python3 -c "
|
||||||
|
import sys, json
|
||||||
|
try:
|
||||||
|
d = json.load(sys.stdin)
|
||||||
|
for e in d.get('entries', d.get('state', [])):
|
||||||
|
if e.get('category') == 'config' and e.get('key') == 'graduation_requested_at':
|
||||||
|
print(e.get('value', ''))
|
||||||
|
break
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
" 2>/dev/null || echo "")
|
||||||
|
|
||||||
|
if [[ -z "$REQUESTED" ]]; then
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Parse JSON: {project, limit, requested_at}
|
||||||
|
PROJECT=$(echo "$REQUESTED" | python3 -c "import sys,json; d=json.load(sys.stdin) if '{' in sys.stdin.buffer.peek().decode(errors='ignore') else None; print((d or {}).get('project',''))" 2>/dev/null || echo "")
|
||||||
|
# Fallback: python inline above can be flaky; just re-parse
|
||||||
|
PROJECT=$(echo "$REQUESTED" | python3 -c "import sys,json; print(json.loads(sys.stdin.read() or '{}').get('project',''))" 2>/dev/null || echo "")
|
||||||
|
LIMIT=$(echo "$REQUESTED" | python3 -c "import sys,json; print(json.loads(sys.stdin.read() or '{}').get('limit',30))" 2>/dev/null || echo "30")
|
||||||
|
|
||||||
|
# Acquire lock
|
||||||
|
exec 9>"$LOCK_FILE" || exit 0
|
||||||
|
if ! flock -n 9; then
|
||||||
|
log "graduation already running, skipping"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Mark running
|
||||||
|
curl -sSf -X POST "$ATOCORE_URL/project/state" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d "{\"project\":\"atocore\",\"category\":\"status\",\"key\":\"graduation_running\",\"value\":\"1\",\"source\":\"host watcher\"}" \
|
||||||
|
>/dev/null 2>&1 || true
|
||||||
|
curl -sSf -X POST "$ATOCORE_URL/project/state" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d "{\"project\":\"atocore\",\"category\":\"status\",\"key\":\"graduation_last_started_at\",\"value\":\"$TS\",\"source\":\"host watcher\"}" \
|
||||||
|
>/dev/null 2>&1 || true
|
||||||
|
|
||||||
|
LOG_FILE="$LOG_DIR/graduation-ondemand-$(date -u +%Y%m%d-%H%M%S).log"
|
||||||
|
log "Starting graduation (project='$PROJECT' limit=$LIMIT, log: $LOG_FILE)"
|
||||||
|
|
||||||
|
# Clear the flag BEFORE running so duplicate clicks queue at most one
|
||||||
|
curl -sSf -X DELETE "$ATOCORE_URL/project/state" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d "{\"project\":\"atocore\",\"category\":\"config\",\"key\":\"graduation_requested_at\"}" \
|
||||||
|
>/dev/null 2>&1 || true
|
||||||
|
|
||||||
|
# Build script args
|
||||||
|
cd "$APP_DIR"
|
||||||
|
export PYTHONPATH="$APP_DIR/src:${PYTHONPATH:-}"
|
||||||
|
ARGS=(--base-url "$ATOCORE_URL" --limit "$LIMIT")
|
||||||
|
if [[ -n "$PROJECT" ]]; then
|
||||||
|
ARGS+=(--project "$PROJECT")
|
||||||
|
fi
|
||||||
|
|
||||||
|
if python3 scripts/graduate_memories.py "${ARGS[@]}" >> "$LOG_FILE" 2>&1; then
|
||||||
|
RESULT=$(tail -3 "$LOG_FILE" | grep "^total:" | tail -1 || tail -1 "$LOG_FILE")
|
||||||
|
RESULT="${RESULT:-completed}"
|
||||||
|
log "graduation finished: $RESULT"
|
||||||
|
else
|
||||||
|
RESULT="ERROR — see $LOG_FILE"
|
||||||
|
log "graduation FAILED"
|
||||||
|
fi
|
||||||
|
|
||||||
|
FINISH_TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||||
|
|
||||||
|
# Mark done
|
||||||
|
curl -sSf -X POST "$ATOCORE_URL/project/state" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d "{\"project\":\"atocore\",\"category\":\"status\",\"key\":\"graduation_running\",\"value\":\"0\",\"source\":\"host watcher\"}" \
|
||||||
|
>/dev/null 2>&1 || true
|
||||||
|
curl -sSf -X POST "$ATOCORE_URL/project/state" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d "{\"project\":\"atocore\",\"category\":\"status\",\"key\":\"graduation_last_finished_at\",\"value\":\"$FINISH_TS\",\"source\":\"host watcher\"}" \
|
||||||
|
>/dev/null 2>&1 || true
|
||||||
|
|
||||||
|
SAFE_RESULT=$(printf '%s' "$RESULT" | python3 -c "import sys,json; print(json.dumps(sys.stdin.read())[1:-1])")
|
||||||
|
curl -sSf -X POST "$ATOCORE_URL/project/state" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d "{\"project\":\"atocore\",\"category\":\"status\",\"key\":\"graduation_last_result\",\"value\":\"$SAFE_RESULT\",\"source\":\"host watcher\"}" \
|
||||||
|
>/dev/null 2>&1 || true
|
||||||
@@ -1358,6 +1358,77 @@ def api_resolve_conflict(conflict_id: str, req: ConflictResolveRequest) -> dict:
|
|||||||
return {"status": "resolved", "id": conflict_id, "action": req.action}
|
return {"status": "resolved", "id": conflict_id, "action": req.action}
|
||||||
|
|
||||||
|
|
||||||
|
class GraduationRequestBody(BaseModel):
|
||||||
|
project: str = ""
|
||||||
|
limit: int = 30
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/admin/graduation/request")
|
||||||
|
def api_request_graduation(req: GraduationRequestBody) -> dict:
|
||||||
|
"""Request a host-side memory-graduation run.
|
||||||
|
|
||||||
|
Writes a flag in project_state with project + limit. A host cron
|
||||||
|
watcher picks it up within ~2 min and runs graduate_memories.py.
|
||||||
|
Mirrors the /admin/triage/request-drain pattern (bridges container
|
||||||
|
→ host because claude CLI lives on host, not container).
|
||||||
|
"""
|
||||||
|
from datetime import datetime as _dt, timezone as _tz
|
||||||
|
from atocore.context.project_state import set_state
|
||||||
|
|
||||||
|
now = _dt.now(_tz.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
payload = json.dumps({
|
||||||
|
"project": (req.project or "").strip(),
|
||||||
|
"limit": max(1, min(req.limit, 500)),
|
||||||
|
"requested_at": now,
|
||||||
|
})
|
||||||
|
set_state(
|
||||||
|
project_name="atocore",
|
||||||
|
category="config",
|
||||||
|
key="graduation_requested_at",
|
||||||
|
value=payload,
|
||||||
|
source="admin ui",
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"requested_at": now,
|
||||||
|
"project": req.project,
|
||||||
|
"limit": req.limit,
|
||||||
|
"note": "Host watcher picks up within ~2 min. Poll /admin/graduation/status for progress.",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/admin/graduation/status")
|
||||||
|
def api_graduation_status() -> dict:
|
||||||
|
"""State of the graduation pipeline (UI polling)."""
|
||||||
|
from atocore.context.project_state import get_state
|
||||||
|
out = {
|
||||||
|
"requested": None,
|
||||||
|
"last_started_at": None,
|
||||||
|
"last_finished_at": None,
|
||||||
|
"last_result": None,
|
||||||
|
"is_running": False,
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
for e in get_state("atocore"):
|
||||||
|
if e.category != "config" and e.category != "status":
|
||||||
|
continue
|
||||||
|
if e.key == "graduation_requested_at":
|
||||||
|
try:
|
||||||
|
out["requested"] = json.loads(e.value)
|
||||||
|
except Exception:
|
||||||
|
out["requested"] = {"raw": e.value}
|
||||||
|
elif e.key == "graduation_last_started_at":
|
||||||
|
out["last_started_at"] = e.value
|
||||||
|
elif e.key == "graduation_last_finished_at":
|
||||||
|
out["last_finished_at"] = e.value
|
||||||
|
elif e.key == "graduation_last_result":
|
||||||
|
out["last_result"] = e.value
|
||||||
|
elif e.key == "graduation_running":
|
||||||
|
out["is_running"] = (e.value == "1")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
@router.get("/admin/graduation/stats")
|
@router.get("/admin/graduation/stats")
|
||||||
def api_graduation_stats() -> dict:
|
def api_graduation_stats() -> dict:
|
||||||
"""Phase 5F graduation stats for dashboard."""
|
"""Phase 5F graduation stats for dashboard."""
|
||||||
|
|||||||
@@ -377,6 +377,92 @@ _ENTITY_TRIAGE_CSS = """
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def _render_graduation_bar() -> str:
|
||||||
|
"""The 'Graduate memories → entity candidates' control bar."""
|
||||||
|
from atocore.projects.registry import load_project_registry
|
||||||
|
try:
|
||||||
|
projects = load_project_registry()
|
||||||
|
options = '<option value="">(all projects)</option>' + "".join(
|
||||||
|
f'<option value="{_escape(p.project_id)}">{_escape(p.project_id)}</option>'
|
||||||
|
for p in projects
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
options = '<option value="">(all projects)</option>'
|
||||||
|
|
||||||
|
return f"""
|
||||||
|
<div class="auto-triage-bar graduation-bar">
|
||||||
|
<button id="grad-btn" onclick="requestGraduation()" title="Run memory→entity graduation on Dalidou host">
|
||||||
|
🎓 Graduate memories
|
||||||
|
</button>
|
||||||
|
<label class="cand-field-label">Project:
|
||||||
|
<select id="grad-project" class="cand-type-select">{options}</select>
|
||||||
|
</label>
|
||||||
|
<label class="cand-field-label">Limit:
|
||||||
|
<input id="grad-limit" type="number" class="cand-tags-input" style="max-width:80px"
|
||||||
|
value="30" min="1" max="200" />
|
||||||
|
</label>
|
||||||
|
<span id="grad-status" class="auto-triage-msg">
|
||||||
|
Scans active memories, asks the LLM "does this describe a typed entity?",
|
||||||
|
and creates entity candidates. Review them in the Entity section below.
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
_GRADUATION_SCRIPT = """
|
||||||
|
<script>
|
||||||
|
async function requestGraduation() {
|
||||||
|
const btn = document.getElementById('grad-btn');
|
||||||
|
const status = document.getElementById('grad-status');
|
||||||
|
const project = document.getElementById('grad-project').value;
|
||||||
|
const limit = parseInt(document.getElementById('grad-limit').value || '30', 10);
|
||||||
|
btn.disabled = true;
|
||||||
|
btn.textContent = '⏳ Requesting...';
|
||||||
|
const r = await fetch('/admin/graduation/request', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {'Content-Type': 'application/json'},
|
||||||
|
body: JSON.stringify({project, limit}),
|
||||||
|
});
|
||||||
|
if (r.ok) {
|
||||||
|
const scope = project || 'all projects';
|
||||||
|
status.textContent = `✓ Queued graduation for ${scope} (limit ${limit}). Host watcher runs every 2 min; refresh this page in ~3 min to see candidates.`;
|
||||||
|
status.className = 'auto-triage-msg ok';
|
||||||
|
btn.textContent = '✓ Requested';
|
||||||
|
pollGraduationStatus();
|
||||||
|
} else {
|
||||||
|
status.textContent = '❌ Request failed: ' + r.status;
|
||||||
|
status.className = 'auto-triage-msg err';
|
||||||
|
btn.disabled = false;
|
||||||
|
btn.textContent = '🎓 Graduate memories';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function pollGraduationStatus() {
|
||||||
|
const status = document.getElementById('grad-status');
|
||||||
|
const btn = document.getElementById('grad-btn');
|
||||||
|
let polls = 0;
|
||||||
|
const timer = setInterval(async () => {
|
||||||
|
polls++;
|
||||||
|
const r = await fetch('/admin/graduation/status');
|
||||||
|
if (!r.ok) return;
|
||||||
|
const s = await r.json();
|
||||||
|
if (s.is_running) {
|
||||||
|
status.textContent = '⚙️ Graduation running... (started ' + (s.last_started_at || '?') + ')';
|
||||||
|
status.className = 'auto-triage-msg ok';
|
||||||
|
} else if (s.last_finished_at && !s.requested) {
|
||||||
|
status.textContent = '✅ Finished: ' + s.last_finished_at + ' → ' + (s.last_result || 'complete');
|
||||||
|
status.className = 'auto-triage-msg ok';
|
||||||
|
if (btn) { btn.disabled = false; btn.textContent = '🎓 Graduate memories'; }
|
||||||
|
clearInterval(timer);
|
||||||
|
setTimeout(() => window.location.reload(), 3000);
|
||||||
|
}
|
||||||
|
if (polls > 120) { clearInterval(timer); } // ~20 min cap
|
||||||
|
}, 10000);
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
def render_triage_page(limit: int = 100) -> str:
|
def render_triage_page(limit: int = 100) -> str:
|
||||||
"""Render the full triage page with pending memory + entity candidates."""
|
"""Render the full triage page with pending memory + entity candidates."""
|
||||||
from atocore.engineering.service import get_entities
|
from atocore.engineering.service import get_entities
|
||||||
@@ -393,17 +479,20 @@ def render_triage_page(limit: int = 100) -> str:
|
|||||||
entity_candidates = []
|
entity_candidates = []
|
||||||
|
|
||||||
total = len(mem_candidates) + len(entity_candidates)
|
total = len(mem_candidates) + len(entity_candidates)
|
||||||
|
graduation_bar = _render_graduation_bar()
|
||||||
|
|
||||||
if total == 0:
|
if total == 0:
|
||||||
body = _TRIAGE_CSS + _ENTITY_TRIAGE_CSS + """
|
body = _TRIAGE_CSS + _ENTITY_TRIAGE_CSS + f"""
|
||||||
<div class="triage-header">
|
<div class="triage-header">
|
||||||
<h1>Triage Queue</h1>
|
<h1>Triage Queue</h1>
|
||||||
</div>
|
</div>
|
||||||
|
{graduation_bar}
|
||||||
<div class="empty">
|
<div class="empty">
|
||||||
<p>🎉 No candidates to review.</p>
|
<p>🎉 No candidates to review.</p>
|
||||||
<p>The auto-triage pipeline keeps this queue empty unless something needs your judgment.</p>
|
<p>The auto-triage pipeline keeps this queue empty unless something needs your judgment.</p>
|
||||||
|
<p>Use the 🎓 Graduate memories button above to propose new entity candidates from existing memories.</p>
|
||||||
</div>
|
</div>
|
||||||
"""
|
""" + _GRADUATION_SCRIPT
|
||||||
return render_html("Triage — AtoCore", body, breadcrumbs=[("Wiki", "/wiki"), ("Triage", "")])
|
return render_html("Triage — AtoCore", body, breadcrumbs=[("Wiki", "/wiki"), ("Triage", "")])
|
||||||
|
|
||||||
# Memory cards
|
# Memory cards
|
||||||
@@ -442,14 +531,15 @@ def render_triage_page(limit: int = 100) -> str:
|
|||||||
🤖 Auto-process queue
|
🤖 Auto-process queue
|
||||||
</button>
|
</button>
|
||||||
<span id="auto-triage-status" class="auto-triage-msg">
|
<span id="auto-triage-status" class="auto-triage-msg">
|
||||||
Sends the full memory queue through LLM triage on the host. Entity candidates
|
Sends the full memory queue through 3-tier LLM triage on the host.
|
||||||
stay for manual review (types + relationships matter too much to auto-decide).
|
Sonnet → Opus → auto-discard. Only genuinely ambiguous items land here.
|
||||||
</span>
|
</span>
|
||||||
</div>
|
</div>
|
||||||
|
{graduation_bar}
|
||||||
<h2>📝 Memory Candidates ({len(mem_candidates)})</h2>
|
<h2>📝 Memory Candidates ({len(mem_candidates)})</h2>
|
||||||
{mem_cards}
|
{mem_cards}
|
||||||
{ent_cards_html}
|
{ent_cards_html}
|
||||||
""" + _TRIAGE_SCRIPT + _ENTITY_TRIAGE_SCRIPT
|
""" + _TRIAGE_SCRIPT + _ENTITY_TRIAGE_SCRIPT + _GRADUATION_SCRIPT
|
||||||
|
|
||||||
return render_html(
|
return render_html(
|
||||||
"Triage — AtoCore",
|
"Triage — AtoCore",
|
||||||
|
|||||||
Reference in New Issue
Block a user