Antoine/Atomizer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Implement ACE Context Engineering framework (SYS_17)

Browse Source 
Complete implementation of Agentic Context Engineering (ACE) framework:

Core modules (optimization_engine/context/):
- playbook.py: AtomizerPlaybook with helpful/harmful scoring
- reflector.py: AtomizerReflector for insight extraction
- session_state.py: Context isolation (exposed/isolated state)
- feedback_loop.py: Automated learning from trial results
- compaction.py: Long-session context management
- cache_monitor.py: KV-cache optimization tracking
- runner_integration.py: OptimizationRunner integration

Dashboard integration:
- context.py: 12 REST API endpoints for playbook management

Tests:
- test_context_engineering.py: 44 unit tests
- test_context_integration.py: 16 integration tests

Documentation:
- CONTEXT_ENGINEERING_REPORT.md: Comprehensive implementation report
- CONTEXT_ENGINEERING_API.md: Complete API reference
- SYS_17_CONTEXT_ENGINEERING.md: System protocol
- Updated cheatsheet with SYS_17 quick reference
- Enhanced bootstrap (00_BOOTSTRAP_V2.md)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Anto01
2025-12-29 20:21:20 -05:00
parent 0110d80401
commit 773f8ff8af
19 changed files with 8184 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1172
docs/CONTEXT_ENGINEERING_REPORT.md Normal file
View File

File diff suppressed because it is too large Load Diff

948
docs/api/CONTEXT_ENGINEERING_API.md Normal file
View File

@@ -0,0 +1,948 @@
# Context Engineering API Reference
**Version**: 1.0
**Updated**: 2025-12-29
**Module**: `optimization_engine.context`
This document provides complete API documentation for the Atomizer Context Engineering (ACE) framework.
---
## Table of Contents
1. [Module Overview](#module-overview)
2. [Core Classes](#core-classes)
- [AtomizerPlaybook](#atomizerplaybook)
- [PlaybookItem](#playbookitem)
- [InsightCategory](#insightcategory)
3. [Session Management](#session-management)
- [AtomizerSessionState](#atomizersessionstate)
- [ExposedState](#exposedstate)
- [IsolatedState](#isolatedstate)
- [TaskType](#tasktype)
4. [Analysis & Learning](#analysis--learning)
- [AtomizerReflector](#atomizerreflector)
- [FeedbackLoop](#feedbackloop)
5. [Optimization](#optimization)
- [CompactionManager](#compactionmanager)
- [ContextCacheOptimizer](#contextcacheoptimizer)
6. [Integration](#integration)
- [ContextEngineeringMixin](#contextengineeringmixin)
- [ContextAwareRunner](#contextawarerunner)
7. [REST API](#rest-api)
---
## Module Overview
### Import Patterns
```python
# Full import
from optimization_engine.context import (
# Core playbook
AtomizerPlaybook,
PlaybookItem,
InsightCategory,
# Session management
AtomizerSessionState,
ExposedState,
IsolatedState,
TaskType,
get_session,
# Analysis
AtomizerReflector,
OptimizationOutcome,
InsightCandidate,
# Learning
FeedbackLoop,
FeedbackLoopFactory,
# Optimization
CompactionManager,
ContextEvent,
EventType,
ContextBudgetManager,
ContextCacheOptimizer,
CacheStats,
StablePrefixBuilder,
# Integration
ContextEngineeringMixin,
ContextAwareRunner,
)
# Convenience imports
from optimization_engine.context import AtomizerPlaybook, get_session
```
---
## Core Classes
### AtomizerPlaybook
The central knowledge store for persistent learning across sessions.
#### Constructor
```python
AtomizerPlaybook(
items: Dict[str, PlaybookItem] = None,
version: int = 1,
created_at: str = None,
last_updated: str = None
)
```
#### Class Methods
##### `load(path: Path) -> AtomizerPlaybook`
Load playbook from JSON file.
```python
playbook = AtomizerPlaybook.load(Path("knowledge_base/playbook.json"))
```
**Parameters:**
- `path`: Path to JSON file
**Returns:** AtomizerPlaybook instance
**Raises:** FileNotFoundError if file doesn't exist (creates new if not found)
---
#### Instance Methods
##### `save(path: Path) -> None`
Save playbook to JSON file.
```python
playbook.save(Path("knowledge_base/playbook.json"))
```
---
##### `add_insight(category, content, source_trial=None, tags=None) -> PlaybookItem`
Add a new insight to the playbook.
```python
item = playbook.add_insight(
category=InsightCategory.STRATEGY,
content="CMA-ES converges faster on smooth surfaces",
source_trial=42,
tags=["sampler", "convergence", "mirror"]
)
```
**Parameters:**
- `category` (InsightCategory): Category of the insight
- `content` (str): The insight content
- `source_trial` (int, optional): Trial number that generated this insight
- `tags` (List[str], optional): Tags for filtering
**Returns:** The created PlaybookItem
---
##### `record_outcome(item_id: str, helpful: bool) -> None`
Record whether an insight was helpful or harmful.
```python
playbook.record_outcome("str_001", helpful=True)
playbook.record_outcome("mis_003", helpful=False)
```
**Parameters:**
- `item_id` (str): ID of the playbook item
- `helpful` (bool): True if helpful, False if harmful
---
##### `get_context_for_task(task_type, max_items=15, min_confidence=0.5) -> str`
Get formatted context string for LLM consumption.
```python
context = playbook.get_context_for_task(
task_type="optimization",
max_items=15,
min_confidence=0.5
)
```
**Parameters:**
- `task_type` (str): Type of task for filtering
- `max_items` (int): Maximum items to include
- `min_confidence` (float): Minimum confidence threshold (0.0-1.0)
**Returns:** Formatted string suitable for LLM context
---
##### `get_by_category(category, min_score=0) -> List[PlaybookItem]`
Get items filtered by category.
```python
mistakes = playbook.get_by_category(InsightCategory.MISTAKE, min_score=-2)
```
**Parameters:**
- `category` (InsightCategory): Category to filter by
- `min_score` (int): Minimum net score
**Returns:** List of matching PlaybookItems
---
##### `get_stats() -> Dict`
Get playbook statistics.
```python
stats = playbook.get_stats()
# Returns:
# {
# "total_items": 45,
# "by_category": {"STRATEGY": 12, "MISTAKE": 8, ...},
# "version": 3,
# "last_updated": "2025-12-29T10:30:00",
# "avg_score": 2.4,
# "max_score": 15,
# "min_score": -3
# }
```
---
##### `prune_harmful(threshold=-3) -> int`
Remove items with net score below threshold.
```python
removed_count = playbook.prune_harmful(threshold=-3)
```
**Parameters:**
- `threshold` (int): Items with net_score <= threshold are removed
**Returns:** Number of items removed
---
### PlaybookItem
Dataclass representing a single playbook entry.
```python
@dataclass
class PlaybookItem:
id: str # e.g., "str_001", "mis_003"
category: InsightCategory # Category enum
content: str # The insight text
helpful_count: int = 0 # Times marked helpful
harmful_count: int = 0 # Times marked harmful
tags: List[str] = field(default_factory=list)
source_trial: Optional[int] = None
created_at: str = "" # ISO timestamp
last_used: Optional[str] = None # ISO timestamp
```
#### Properties
```python
item.net_score # helpful_count - harmful_count
item.confidence # helpful / (helpful + harmful), or 0.5 if no feedback
```
#### Methods
```python
# Convert to context string for LLM
context_str = item.to_context_string()
# "[str_001] helpful=5 harmful=0 :: CMA-ES converges faster..."
```
---
### InsightCategory
Enum for categorizing insights.
```python
class InsightCategory(Enum):
STRATEGY = "str" # Optimization strategies that work
CALCULATION = "cal" # Formulas and calculations
MISTAKE = "mis" # Common mistakes to avoid
TOOL = "tool" # Tool usage patterns
DOMAIN = "dom" # Domain-specific knowledge (FEA, NX)
WORKFLOW = "wf" # Workflow patterns
```
**Usage:**
```python
# Create with enum
category = InsightCategory.STRATEGY
# Create from string
category = InsightCategory("str")
# Get string value
value = InsightCategory.STRATEGY.value # "str"
```
---
## Session Management
### AtomizerSessionState
Manages session context with exposed/isolated separation.
#### Constructor
```python
session = AtomizerSessionState(
session_id: str = None # Auto-generated UUID if not provided
)
```
#### Attributes
```python
session.session_id # Unique session identifier
session.exposed # ExposedState - always in LLM context
session.isolated # IsolatedState - on-demand access only
session.last_updated # ISO timestamp of last update
```
#### Methods
##### `get_llm_context() -> str`
Get exposed state formatted for LLM context.
```python
context = session.get_llm_context()
# Returns formatted string with task type, study info, progress, etc.
```
---
##### `add_action(action: str) -> None`
Record an action (keeps last 20).
```python
session.add_action("Started optimization with TPE sampler")
```
---
##### `add_error(error: str, error_type: str = None) -> None`
Record an error (keeps last 10).
```python
session.add_error("NX solver timeout after 600s", error_type="solver")
```
---
##### `to_dict() / from_dict(data) -> AtomizerSessionState`
Serialize/deserialize session state.
```python
# Save
data = session.to_dict()
# Restore
session = AtomizerSessionState.from_dict(data)
```
---
### ExposedState
State that's always included in LLM context.
```python
@dataclass
class ExposedState:
task_type: Optional[TaskType] = None
study_name: Optional[str] = None
study_status: str = "idle"
trials_completed: int = 0
trials_total: int = 0
best_value: Optional[float] = None
recent_actions: List[str] = field(default_factory=list) # Last 20
recent_errors: List[str] = field(default_factory=list) # Last 10
```
---
### IsolatedState
State available on-demand but not in default context.
```python
@dataclass
class IsolatedState:
full_trial_history: List[Dict] = field(default_factory=list)
detailed_errors: List[Dict] = field(default_factory=list)
performance_metrics: Dict = field(default_factory=dict)
debug_info: Dict = field(default_factory=dict)
```
---
### TaskType
Enum for session task classification.
```python
class TaskType(Enum):
CREATE_STUDY = "create_study"
RUN_OPTIMIZATION = "run_optimization"
MONITOR_PROGRESS = "monitor_progress"
ANALYZE_RESULTS = "analyze_results"
DEBUG_ERROR = "debug_error"
CONFIGURE_SETTINGS = "configure_settings"
NEURAL_ACCELERATION = "neural_acceleration"
```
---
### get_session()
Get or create the global session instance.
```python
from optimization_engine.context import get_session
session = get_session()
session.exposed.task_type = TaskType.RUN_OPTIMIZATION
```
---
## Analysis & Learning
### AtomizerReflector
Analyzes optimization outcomes and extracts insights.
#### Constructor
```python
reflector = AtomizerReflector(playbook: AtomizerPlaybook)
```
#### Methods
##### `analyze_outcome(outcome: OptimizationOutcome) -> List[InsightCandidate]`
Analyze an optimization outcome for insights.
```python
outcome = OptimizationOutcome(
study_name="bracket_v3",
trial_number=42,
params={'thickness': 10.5},
objectives={'mass': 5.2},
constraints_satisfied=True,
error_message=None,
solve_time=45.2
)
insights = reflector.analyze_outcome(outcome)
for insight in insights:
print(f"{insight.category}: {insight.content}")
```
---
##### `extract_error_insights(error_message: str) -> List[InsightCandidate]`
Extract insights from error messages.
```python
insights = reflector.extract_error_insights("Solution did not converge within tolerance")
# Returns insights about convergence failures
```
---
### OptimizationOutcome
Dataclass for optimization trial outcomes.
```python
@dataclass
class OptimizationOutcome:
study_name: str
trial_number: int
params: Dict[str, Any]
objectives: Dict[str, float]
constraints_satisfied: bool
error_message: Optional[str] = None
solve_time: Optional[float] = None
```
---
### FeedbackLoop
Automated learning from optimization execution.
#### Constructor
```python
feedback = FeedbackLoop(playbook_path: Path)
```
#### Methods
##### `process_trial_result(trial_number, params, objectives, is_feasible, error=None)`
Process a trial result for learning opportunities.
```python
feedback.process_trial_result(
trial_number=42,
params={'thickness': 10.5, 'width': 25.0},
objectives={'mass': 5.2, 'stress': 180.0},
is_feasible=True,
error=None
)
```
---
##### `finalize_study(study_summary: Dict) -> Dict`
Finalize learning at end of optimization study.
```python
result = feedback.finalize_study({
"name": "bracket_v3",
"total_trials": 100,
"best_value": 4.8,
"convergence_rate": 0.95
})
# Returns: {"insights_added": 3, "patterns_identified": ["fast_convergence"]}
```
---
## Optimization
### CompactionManager
Handles context compaction for long-running sessions.
#### Constructor
```python
compactor = CompactionManager(
max_events: int = 100,
preserve_errors: bool = True,
preserve_milestones: bool = True
)
```
#### Methods
##### `add_event(event: ContextEvent) -> None`
Add an event to the session history.
```python
from optimization_engine.context import ContextEvent, EventType
event = ContextEvent(
event_type=EventType.TRIAL_COMPLETE,
content="Trial 42 completed: mass=5.2kg",
timestamp=datetime.now().isoformat(),
is_error=False,
is_milestone=False
)
compactor.add_event(event)
```
---
##### `maybe_compact() -> Optional[str]`
Compact events if over threshold.
```python
summary = compactor.maybe_compact()
if summary:
print(f"Compacted: {summary}")
```
---
##### `get_context() -> str`
Get current context string.
```python
context = compactor.get_context()
```
---
### ContextCacheOptimizer
Monitors and optimizes KV-cache efficiency.
#### Constructor
```python
optimizer = ContextCacheOptimizer()
```
#### Methods
##### `track_request(prefix_tokens: int, total_tokens: int)`
Track a request for cache analysis.
```python
optimizer.track_request(prefix_tokens=5000, total_tokens=15000)
```
---
##### `track_completion(success: bool, response_tokens: int)`
Track completion for performance analysis.
```python
optimizer.track_completion(success=True, response_tokens=500)
```
---
##### `get_stats_dict() -> Dict`
Get cache statistics.
```python
stats = optimizer.get_stats_dict()
# Returns:
# {
# "total_requests": 150,
# "cache_hits": 120,
# "cache_hit_rate": 0.8,
# "avg_prefix_ratio": 0.33,
# ...
# }
```
---
##### `get_report() -> str`
Get human-readable report.
```python
report = optimizer.get_report()
print(report)
```
---
## Integration
### ContextEngineeringMixin
Mixin class for adding context engineering to optimization runners.
```python
class ContextEngineeringMixin:
def init_context_engineering(self, playbook_path: Path):
"""Initialize context engineering components."""
def record_trial_outcome(self, trial_number, params, objectives,
is_feasible, error=None):
"""Record trial outcome for learning."""
def get_context_for_llm(self) -> str:
"""Get combined context for LLM consumption."""
def finalize_context_engineering(self, study_summary: Dict):
"""Finalize learning at study completion."""
```
---
### ContextAwareRunner
Pre-built runner with context engineering enabled.
```python
from optimization_engine.context import ContextAwareRunner
runner = ContextAwareRunner(
config=config_dict,
playbook_path=Path("knowledge_base/playbook.json")
)
# Run optimization with automatic learning
runner.run()
```
---
## REST API
The Context Engineering module exposes REST endpoints via FastAPI.
### Base URL
```
http://localhost:5000/api/context
```
### Endpoints
#### GET `/playbook`
Get playbook summary statistics.
**Response:**
```json
{
"total_items": 45,
"by_category": {"STRATEGY": 12, "MISTAKE": 8},
"version": 3,
"last_updated": "2025-12-29T10:30:00",
"avg_score": 2.4,
"top_score": 15,
"lowest_score": -3
}
```
---
#### GET `/playbook/items`
List playbook items with optional filters.
**Query Parameters:**
- `category` (str): Filter by category (str, mis, tool, cal, dom, wf)
- `min_score` (int): Minimum net score (default: 0)
- `min_confidence` (float): Minimum confidence (default: 0.0)
- `limit` (int): Max items (default: 50)
- `offset` (int): Pagination offset (default: 0)
**Response:**
```json
[
{
"id": "str_001",
"category": "str",
"content": "CMA-ES converges faster on smooth surfaces",
"helpful_count": 5,
"harmful_count": 0,
"net_score": 5,
"confidence": 1.0,
"tags": ["sampler", "convergence"],
"created_at": "2025-12-29T10:00:00",
"last_used": "2025-12-29T10:30:00"
}
]
```
---
#### GET `/playbook/items/{item_id}`
Get a specific playbook item.
**Response:** Single PlaybookItemResponse object
---
#### POST `/playbook/feedback`
Record feedback on a playbook item.
**Request Body:**
```json
{
"item_id": "str_001",
"helpful": true
}
```
**Response:**
```json
{
"item_id": "str_001",
"new_score": 6,
"new_confidence": 1.0,
"helpful_count": 6,
"harmful_count": 0
}
```
---
#### POST `/playbook/insights`
Add a new insight.
**Request Body:**
```json
{
"category": "str",
"content": "New insight content",
"tags": ["tag1", "tag2"],
"source_trial": 42
}
```
**Response:**
```json
{
"item_id": "str_015",
"category": "str",
"content": "New insight content",
"message": "Insight added successfully"
}
```
---
#### DELETE `/playbook/items/{item_id}`
Delete a playbook item.
**Response:**
```json
{
"deleted": "str_001",
"content_preview": "CMA-ES converges faster..."
}
```
---
#### POST `/playbook/prune`
Remove harmful items.
**Query Parameters:**
- `threshold` (int): Net score threshold (default: -3)
**Response:**
```json
{
"items_pruned": 3,
"threshold_used": -3,
"remaining_items": 42
}
```
---
#### GET `/playbook/context`
Get playbook context for LLM consumption.
**Query Parameters:**
- `task_type` (str): Task type (default: "optimization")
- `max_items` (int): Maximum items (default: 15)
- `min_confidence` (float): Minimum confidence (default: 0.5)
**Response:**
```json
{
"context": "## Atomizer Knowledge Base\n...",
"items_included": 15,
"task_type": "optimization"
}
```
---
#### GET `/session`
Get current session state.
**Response:**
```json
{
"session_id": "abc123",
"task_type": "run_optimization",
"study_name": "bracket_v3",
"study_status": "running",
"trials_completed": 42,
"trials_total": 100,
"best_value": 5.2,
"recent_actions": ["Started optimization", "Trial 42 complete"],
"recent_errors": []
}
```
---
#### GET `/session/context`
Get session context for LLM consumption.
**Response:**
```json
{
"context": "## Current Session\nTask: run_optimization\n...",
"session_id": "abc123",
"last_updated": "2025-12-29T10:30:00"
}
```
---
#### GET `/cache/stats`
Get KV-cache statistics.
**Response:**
```json
{
"stats": {
"total_requests": 150,
"cache_hits": 120,
"cache_hit_rate": 0.8
},
"report": "Cache Performance Report\n..."
}
```
---
#### GET `/learning/report`
Get comprehensive learning report.
**Response:**
```json
{
"generated_at": "2025-12-29T10:30:00",
"playbook_stats": {...},
"top_performers": [
{"id": "str_001", "content": "...", "score": 15}
],
"worst_performers": [
{"id": "mis_003", "content": "...", "score": -2}
],
"recommendations": [
"Consider pruning 3 harmful items (net_score < -3)"
]
}
```
---
## Error Handling
All API endpoints return appropriate HTTP status codes:
| Code | Meaning |
|------|---------|
| 200 | Success |
| 400 | Bad request (invalid parameters) |
| 404 | Not found (item doesn't exist) |
| 500 | Server error (module not available) |
Error response format:
```json
{
"detail": "Error description"
}
```
---
## See Also
- [Context Engineering Report](../CONTEXT_ENGINEERING_REPORT.md) - Full implementation report
- [SYS_17 Protocol](../protocols/system/SYS_17_CONTEXT_ENGINEERING.md) - System protocol
- [Cheatsheet](../../.claude/skills/01_CHEATSHEET.md) - Quick reference

307
docs/protocols/system/SYS_17_CONTEXT_ENGINEERING.md Normal file
View File

@@ -0,0 +1,307 @@
---
protocol_id: SYS_17
version: 1.0
last_updated: 2025-12-29
status: active
owner: system
code_dependencies:
- optimization_engine.context.*
requires_protocols: []
---
# SYS_17: Context Engineering System
## Overview
The Context Engineering System implements the **Agentic Context Engineering (ACE)** framework, enabling Atomizer to learn from every optimization run and accumulate institutional knowledge over time.
## When to Load This Protocol
Load SYS_17 when:
- User asks about "learning", "playbook", or "context engineering"
- Debugging why certain knowledge isn't being applied
- Configuring context behavior
- Analyzing what the system has learned
## Core Concepts
### The ACE Framework
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Generator │────▶│ Reflector │────▶│ Curator │
│ (Opt Runs) │ │ (Analysis) │ │ (Playbook) │
└─────────────┘ └─────────────┘ └─────────────┘
│ │
└───────────── Feedback ───────────────┘
```
1. **Generator**: OptimizationRunner produces trial outcomes
2. **Reflector**: Analyzes outcomes, extracts patterns
3. **Curator**: Playbook stores and manages insights
4. **Feedback**: Success/failure updates insight scores
### Playbook Item Structure
```
[str-00001] helpful=8 harmful=0 :: "Use shell elements for thin walls"
│ │ │ │
│ │ │ └── Insight content
│ │ └── Times advice led to failure
│ └── Times advice led to success
└── Unique ID (category-number)
```
### Categories
| Code | Name | Description | Example |
|------|------|-------------|---------|
| `str` | STRATEGY | Optimization approaches | "Start with TPE, switch to CMA-ES" |
| `mis` | MISTAKE | Things to avoid | "Don't use coarse mesh for stress" |
| `tool` | TOOL | Tool usage tips | "Use GP sampler for few-shot" |
| `cal` | CALCULATION | Formulas | "Safety factor = yield/max_stress" |
| `dom` | DOMAIN | Domain knowledge | "Zernike coefficients for mirrors" |
| `wf` | WORKFLOW | Workflow patterns | "Load _i.prt before UpdateFemodel()" |
## Key Components
### 1. AtomizerPlaybook
Location: `optimization_engine/context/playbook.py`
The central knowledge store. Handles:
- Adding insights (with auto-deduplication)
- Recording helpful/harmful outcomes
- Generating filtered context for LLM
- Pruning consistently harmful items
- Persistence (JSON)
**Quick Usage:**
```python
from optimization_engine.context import get_playbook, save_playbook, InsightCategory
playbook = get_playbook()
playbook.add_insight(InsightCategory.STRATEGY, "Use shell elements for thin walls")
playbook.record_outcome("str-00001", helpful=True)
save_playbook()
```
### 2. AtomizerReflector
Location: `optimization_engine/context/reflector.py`
Analyzes optimization outcomes to extract insights:
- Classifies errors (convergence, mesh, singularity, etc.)
- Extracts success patterns
- Generates study-level insights
**Quick Usage:**
```python
from optimization_engine.context import AtomizerReflector, OptimizationOutcome
reflector = AtomizerReflector(playbook)
outcome = OptimizationOutcome(trial_number=42, success=True, ...)
insights = reflector.analyze_trial(outcome)
reflector.commit_insights()
```
### 3. FeedbackLoop
Location: `optimization_engine/context/feedback_loop.py`
Automated learning loop that:
- Processes trial results
- Updates playbook scores based on outcomes
- Tracks which items were active per trial
- Finalizes learning at study end
**Quick Usage:**
```python
from optimization_engine.context import FeedbackLoop
feedback = FeedbackLoop(playbook_path)
feedback.process_trial_result(trial_number=42, success=True, ...)
feedback.finalize_study({"name": "study", "total_trials": 100, ...})
```
### 4. SessionState
Location: `optimization_engine/context/session_state.py`
Manages context isolation:
- **Exposed**: Always in LLM context (task type, recent actions, errors)
- **Isolated**: On-demand access (full history, NX paths, F06 content)
**Quick Usage:**
```python
from optimization_engine.context import get_session, TaskType
session = get_session()
session.exposed.task_type = TaskType.RUN_OPTIMIZATION
session.add_action("Started trial 42")
context = session.get_llm_context()
```
### 5. CompactionManager
Location: `optimization_engine/context/compaction.py`
Handles long sessions:
- Triggers compaction at threshold (default 50 events)
- Summarizes old events into statistics
- Preserves errors and milestones
### 6. CacheOptimizer
Location: `optimization_engine/context/cache_monitor.py`
Optimizes for KV-cache:
- Three-tier context structure (stable/semi-stable/dynamic)
- Tracks cache hit rate
- Estimates cost savings
## Integration with OptimizationRunner
### Option 1: Mixin
```python
from optimization_engine.context.runner_integration import ContextEngineeringMixin
class MyRunner(ContextEngineeringMixin, OptimizationRunner):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.init_context_engineering()
```
### Option 2: Wrapper
```python
from optimization_engine.context.runner_integration import ContextAwareRunner
runner = OptimizationRunner(config_path=...)
context_runner = ContextAwareRunner(runner)
context_runner.run(n_trials=100)
```
## Dashboard API
Base URL: `/api/context`
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/playbook` | GET | Playbook summary |
| `/playbook/items` | GET | List items (with filters) |
| `/playbook/items/{id}` | GET | Get specific item |
| `/playbook/feedback` | POST | Record helpful/harmful |
| `/playbook/insights` | POST | Add new insight |
| `/playbook/prune` | POST | Prune harmful items |
| `/playbook/context` | GET | Get LLM context string |
| `/session` | GET | Session state |
| `/learning/report` | GET | Learning report |
## Best Practices
### 1. Record Immediately
Don't wait until session end:
```python
# RIGHT: Record immediately
playbook.add_insight(InsightCategory.MISTAKE, "Convergence failed with X")
playbook.save(path)
# WRONG: Wait until end
# (User might close session, learning lost)
```
### 2. Be Specific
```python
# GOOD: Specific and actionable
"For bracket optimization with >5 variables, TPE outperforms random search"
# BAD: Vague
"TPE is good"
```
### 3. Include Context
```python
playbook.add_insight(
InsightCategory.STRATEGY,
"Shell elements reduce solve time by 40% for thickness < 2mm",
tags=["mesh", "shell", "performance"]
)
```
### 4. Review Harmful Items
Periodically check items with negative scores:
```python
harmful = [i for i in playbook.items.values() if i.net_score < 0]
for item in harmful:
print(f"{item.id}: {item.content[:50]}... (score={item.net_score})")
```
## Troubleshooting
### Playbook Not Updating
1. Check playbook path:
```python
print(playbook_path) # Should be knowledge_base/playbook.json
```
2. Verify save is called:
```python
playbook.save(path) # Must be explicit
```
### Insights Not Appearing in Context
1. Check confidence threshold:
```python
# Default is 0.5 - new items start at 0.5
context = playbook.get_context_for_task("opt", min_confidence=0.3)
```
2. Check if items exist:
```python
print(f"Total items: {len(playbook.items)}")
```
### Learning Not Working
1. Verify FeedbackLoop is finalized:
```python
feedback.finalize_study(...) # MUST be called
```
2. Check context_items_used parameter:
```python
# Items must be explicitly tracked
feedback.process_trial_result(
...,
context_items_used=list(playbook.items.keys())[:10]
)
```
## Files Reference
| File | Purpose |
|------|---------|
| `optimization_engine/context/__init__.py` | Module exports |
| `optimization_engine/context/playbook.py` | Knowledge store |
| `optimization_engine/context/reflector.py` | Outcome analysis |
| `optimization_engine/context/session_state.py` | Context isolation |
| `optimization_engine/context/feedback_loop.py` | Learning loop |
| `optimization_engine/context/compaction.py` | Long session management |
| `optimization_engine/context/cache_monitor.py` | KV-cache optimization |
| `optimization_engine/context/runner_integration.py` | Runner integration |
| `knowledge_base/playbook.json` | Persistent storage |
## See Also
- `docs/CONTEXT_ENGINEERING_REPORT.md` - Full implementation report
- `.claude/skills/00_BOOTSTRAP_V2.md` - Enhanced bootstrap
- `tests/test_context_engineering.py` - Unit tests
- `tests/test_context_integration.py` - Integration tests