Refactor UI to Operator Control Plane and add operator documentation

Co-authored-by: Junie <junie@jetbrains.com>
This commit is contained in:
Oskar Kapala 2026-05-12 17:29:52 +02:00
parent 7c7024d66c
commit 3cff4db8f3
6 changed files with 595 additions and 495 deletions

View file

@ -0,0 +1,9 @@
# Incident Review Flow
When an incident occurs:
1. Identify the incident in the Dashboard or Events view.
2. Note the severity and the affected node/service.
3. Switch to the Events view to see the correlation chain.
4. Look for related events preceding the incident (e.g., latency spikes before a disconnect).
5. Check the Recommendations view for suggested fixes.
6. Once resolved, the incident will clear from the Active Incidents list.

View file

@ -0,0 +1,12 @@
# Reconcile Review Flow
The system continuously monitors for drift between desired and actual state.
1. If a service is in RECONCILING state, check the Services view.
2. Review the Recommendations view for automated or guarded actions.
3. For 'safe' actions with high confidence, the system may act autonomously if enabled.
4. For 'guarded' or 'dangerous' actions, an operator must manually approve the action.
5. Risk Levels:
- **Safe**: Minimal impact, high success rate.
- **Guarded**: Potential brief service interruption.
- **Dangerous**: Significant impact, potential data loss, or hardware interaction required.

View file

@ -0,0 +1,13 @@
# Operator Workflow Examples
## Daily Check
1. Open the Operator Control Plane.
2. Check the System Status in the sidebar (should be NOMINAL).
3. Review the Dashboard for any active incidents.
4. Switch to Nodes view to ensure all nodes are connected and healthy.
## New Service Deployment
1. Trigger deployment via CLI/API.
2. Monitor progress in the Deployments view.
3. If a stage fails, check the Diagnostics field in the deployment card.
4. Once stable, verify the service health in the Services view.

View file

@ -0,0 +1,70 @@
# Runtime Supervisor
The Runtime Supervisor is a reconciliation-oriented component responsible for ensuring the homelab infrastructure aligns with its desired state. It operates in a recommendation-only mode, detecting drifts and proposing actions without directly mutating the runtime.
## Reconciliation Loop
The supervisor performs a periodic (or on-demand) reconciliation loop:
1. **Load Desired State**: Reads configuration from the inventory model (`hosts/*/services.yaml`).
2. **Load Actual State**: Reads the current world state from the filesystem-first observer output (`/opt/homelab/world/`).
3. **Detect Drift**: Compares the two states to identify discrepancies.
4. **Recommend Actions**: Uses a recommendation engine to propose remediations.
5. **Emit Events**: Publishes reconciliation events to the platform's event system.
## Desired vs Actual State
- **Desired State**: Defined by the user in the inventory. It specifies which services should run on which nodes.
- **Actual State (World State)**: Produced by the observer runtime. It represents the ground truth of what is currently happening in the physical and virtual infrastructure.
The gap between these two states is known as **Drift**.
## Drift Conditions
The supervisor detects the following drift conditions:
- **Missing Service**: A service is defined in the inventory but not found in the world state.
- **Unhealthy Service**: A service is present but reporting a non-ok status.
- **Failed Deployment**: Recent deployment attempts for a service have failed.
- **Offline Node**: A node defined in the inventory is unreachable or inactive.
- **Unresolved Incidents**: Active incidents in the world state that require attention.
## Recommendation Engine
Based on the detected drift, the supervisor emits one of three reconciliation event types:
- `reconcile_required`: Immediate action recommended to restore service (e.g., redeploy unhealthy service).
- `reconcile_recommended`: Action suggested to improve stability or resolve minor issues (e.g., review unresolved incidents).
- `reconcile_blocked`: Action required but blocked by external factors or repeated failures (e.g., repeated deployment failures requiring manual diagnostics).
### Examples:
- **Unhealthy service** -> Recommend `redeploy`.
- **Repeated deployment failures** -> Recommend `diagnostics`.
- **Node offline** -> Recommend `failover review`.
- **Dependency unavailable** -> Recommend `delayed deployment`.
## Summary States
The supervisor extends the platform's runtime summary states:
- `nominal`: Desired and actual states are in sync.
- `degraded`: Non-critical drift detected; reconciliation required.
- `unstable`: Critical drift or blocked reconciliation detected.
- `reconciling`: (Future) Remediations are actively being applied.
## Future Autonomous Remediation Architecture
While the current stage is recommendation-only, the architecture is designed for future autonomy:
1. **Policy Engine**: A future component will ingest reconciliation recommendations and apply policies to decide whether to auto-remediate.
2. **Executor**: A mutation-capable runtime that will execute the proposed actions (restarts, redeploys, failovers).
3. **Closed-Loop Feedback**: The observer will immediately reflect the results of remediations, allowing the supervisor to verify success or escalate if the drift persists.
4. **Guardrails**: Implementation of rate-limiting, maintenance windows, and manual overrides to ensure autonomous actions remain safe.
## Filesystem-First Design
The supervisor adheres to the platform's filesystem-first philosophy:
- **Input**: Files in `/opt/homelab/world/` and `hosts/`.
- **Output**: Append-only logs in `/tmp/agent-events.log`.
- **Persistence**: Checkpoints stored in `/tmp/supervisor-checkpoint.json`.
- **Idempotency**: Every run is independent and produces the same recommendations for the same input state.

View file

@ -3,558 +3,495 @@
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Agent System</title>
<title>Operator Control Plane</title>
<style>
body {
margin: 0;
font-family: system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background: #101418;
color: #e7edf3;
:root {
--bg-color: #0a0c0e;
--sidebar-color: #14171a;
--card-color: #1c2024;
--border-color: #2a3540;
--text-color: #e7edf3;
--text-muted: #94a3b8;
--accent-color: #3eaf7c;
--nominal: #3eaf7c;
--degraded: #e7c000;
--unstable: #e67e22;
--reconciling: #3498db;
--error: #c0392b;
--safe: #3eaf7c;
--guarded: #e67e22;
--dangerous: #c0392b;
}
main {
max-width: 1100px;
margin: 0 auto;
body {
margin: 0;
font-family: 'Inter', system-ui, -apple-system, sans-serif;
background: var(--bg-color);
color: var(--text-color);
display: flex;
height: 100vh;
overflow: hidden;
}
/* Sidebar */
.sidebar {
width: 240px;
background: var(--sidebar-color);
border-right: 1px solid var(--border-color);
display: flex;
flex-direction: column;
flex-shrink: 0;
}
.sidebar-header {
padding: 24px;
font-weight: 800;
font-size: 14px;
letter-spacing: 0.1em;
color: var(--accent-color);
border-bottom: 1px solid var(--border-color);
}
.nav-list {
list-style: none;
padding: 12px 0;
margin: 0;
flex-grow: 1;
}
.nav-item {
padding: 12px 24px;
cursor: pointer;
font-size: 14px;
color: var(--text-muted);
transition: all 0.2s;
display: flex;
align-items: center;
gap: 12px;
}
.nav-item:hover {
background: rgba(255, 255, 255, 0.05);
color: var(--text-color);
}
.nav-item.active {
background: rgba(62, 175, 124, 0.1);
color: var(--accent-color);
border-left: 3px solid var(--accent-color);
}
.sidebar-footer {
padding: 16px;
border-top: 1px solid var(--border-color);
font-size: 12px;
}
/* Content Area */
.main-content {
flex-grow: 1;
display: flex;
flex-direction: column;
overflow: hidden;
}
header {
height: 64px;
border-bottom: 1px solid var(--border-color);
display: flex;
align-items: center;
padding: 0 24px;
justify-content: space-between;
background: var(--bg-color);
}
.view-title {
font-size: 18px;
font-weight: 600;
}
.content-scroll {
flex-grow: 1;
overflow-y: auto;
padding: 24px;
}
h1 {
margin: 0 0 16px;
font-size: 24px;
font-weight: 650;
/* Cards & Grids */
.grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(350px, 1fr));
gap: 20px;
}
#logs {
height: 70vh;
overflow-y: auto;
border: 1px solid #2a3540;
background: #0b0f13;
padding: 12px;
font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
font-size: 13px;
line-height: 1.45;
white-space: pre-wrap;
.card {
background: var(--card-color);
border: 1px solid var(--border-color);
padding: 20px;
border-radius: 4px;
position: relative;
}
#services {
border: 1px solid #2a3540;
background: #0b0f13;
padding: 12px;
margin-bottom: 12px;
font-size: 13px;
}
.panel-title {
font-weight: 700;
margin-bottom: 10px;
}
.service {
border-top: 1px solid #202a33;
padding: 10px 0;
}
.service:first-of-type {
border-top: 0;
padding-top: 0;
}
.service-header {
.card-header {
display: flex;
gap: 8px;
align-items: baseline;
justify-content: space-between;
margin-bottom: 6px;
}
.service-name {
font-weight: 700;
}
.service-history {
display: flex;
flex-wrap: wrap;
gap: 6px;
margin-top: 6px;
}
.history-entry {
border: 1px solid #2a3540;
padding: 3px 6px;
font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
}
.run {
border: 1px solid #333;
margin: 10px;
padding: 10px;
}
.run.stopped {
opacity: 0.45;
}
.header {
display: flex;
align-items: center;
gap: 8px;
color: #ffffff;
margin-bottom: 16px;
}
.card-title {
font-weight: 700;
margin-bottom: 8px;
font-size: 16px;
}
.run-title {
flex: 1;
min-width: 0;
/* Status Badges */
.badge {
padding: 4px 8px;
border-radius: 4px;
font-size: 11px;
font-weight: 700;
text-transform: uppercase;
}
.proposal {
border: 1px solid #4c5b2b;
background: #171d12;
color: #edf5d4;
margin: 8px 0;
padding: 10px;
}
.status-nominal { background: rgba(62, 175, 124, 0.1); color: var(--nominal); }
.status-degraded { background: rgba(231, 192, 0, 0.1); color: var(--degraded); }
.status-unstable { background: rgba(230, 126, 34, 0.1); color: var(--unstable); }
.status-reconciling { background: rgba(52, 152, 219, 0.1); color: var(--reconciling); }
.status-error { background: rgba(192, 57, 43, 0.1); color: var(--error); }
.proposal-actions {
/* Timeline */
.timeline {
display: flex;
gap: 8px;
margin-top: 8px;
flex-direction: column;
gap: 12px;
}
.proposal-divider {
border-top: 1px solid #3c4930;
margin: 10px 0;
}
.row {
display: grid;
grid-template-columns: 1fr auto auto;
gap: 8px;
margin-top: 12px;
}
.config-panel {
border: 1px solid #2a3540;
background: #111820;
margin-top: 12px;
.event {
padding: 12px;
border-left: 2px solid var(--border-color);
background: rgba(255, 255, 255, 0.02);
font-family: ui-monospace, monospace;
font-size: 13px;
}
.config-title {
font-weight: 700;
margin-bottom: 10px;
}
.event.high { border-left-color: var(--error); }
.event.medium { border-left-color: var(--unstable); }
.event.low { border-left-color: var(--nominal); }
.config-grid {
display: grid;
grid-template-columns: auto minmax(96px, 140px) auto;
align-items: center;
gap: 8px;
}
.toggle {
.event-header {
display: flex;
align-items: center;
gap: 6px;
border: 1px solid #354453;
background: #151c23;
color: #e7edf3;
padding: 10px 12px;
white-space: nowrap;
justify-content: space-between;
margin-bottom: 4px;
color: var(--text-muted);
}
.toggle input {
padding: 0;
/* Forms & Inputs */
.controls {
display: flex;
gap: 12px;
margin-top: 20px;
}
input, button {
font: inherit;
border: 1px solid #354453;
background: #151c23;
color: #e7edf3;
padding: 10px 12px;
background: var(--card-color);
border: 1px solid var(--border-color);
color: var(--text-color);
padding: 8px 16px;
font-size: 14px;
border-radius: 4px;
}
button {
cursor: pointer;
min-width: 88px;
font-weight: 600;
}
button:hover {
background: #1d2730;
}
button:hover { background: var(--border-color); }
.btn-primary { background: var(--accent-color); color: white; border: none; }
.btn-primary:hover { background: #359b6d; }
/* Utility */
.hidden { display: none !important; }
.mono { font-family: ui-monospace, monospace; }
.label { color: var(--text-muted); font-size: 12px; margin-bottom: 4px; }
.value { font-weight: 500; margin-bottom: 12px; }
.risk-safe { color: var(--safe); }
.risk-guarded { color: var(--guarded); }
.risk-dangerous { color: var(--dangerous); }
.error { color: #ff6b6b; }
.warning { color: #f4a261; }
.info { color: #5dade2; }
.action { color: #58d68d; }
.auto_action { color: #a7f3d0; }
.learning { color: #c4b5fd; }
.log, .result { color: #ccd6df; }
.status.running { color: yellow; }
.status.done { color: green; }
.status.error { color: red; }
.status.stopped { color: gray; }
.health-ok { color: #58d68d; }
.health-warning { color: #f4d03f; }
.health-error { color: #ff6b6b; }
.health-unknown { color: #95a5a6; }
</style>
</head>
<body>
<main>
<h1>Agent System</h1>
<section id="services">
<div class="panel-title">Services</div>
<div id="services-list"></div>
</section>
<div id="logs"></div>
<div class="row">
<input id="cmd" autocomplete="off" placeholder="Command">
<button onclick="send()">Send</button>
<button id="stop">STOP</button>
<aside class="sidebar">
<div class="sidebar-header">HOMELAB OPERATOR</div>
<ul class="nav-list">
<li class="nav-item active" onclick="showView('dashboard', this)">
<span>Dashboard</span>
</li>
<li class="nav-item" onclick="showView('nodes', this)">
<span>Nodes</span>
</li>
<li class="nav-item" onclick="showView('services', this)">
<span>Services</span>
</li>
<li class="nav-item" onclick="showView('deployments', this)">
<span>Deployments</span>
</li>
<li class="nav-item" onclick="showView('events', this)">
<span>Events</span>
</li>
<li class="nav-item" onclick="showView('recommendations', this)">
<span>Recommendations</span>
</li>
<li class="nav-item" onclick="showView('settings', this)">
<span>Settings</span>
</li>
</ul>
<div class="sidebar-footer">
<div id="summary-status">System Status: Loading...</div>
</div>
<section class="config-panel">
<div class="config-title">Auto Mode Config</div>
<div class="config-grid">
<label class="toggle">
<input id="config-auto-mode" type="checkbox">
auto_mode
</label>
<span></span>
<button id="save-config">Save</button>
</aside>
<label for="restart-threshold">restart_ha threshold</label>
<input id="restart-threshold" type="number" min="0" max="1" step="0.01">
<span></span>
<label for="network-threshold">check_network threshold</label>
<input id="network-threshold" type="number" min="0" max="1" step="0.01">
<span></span>
<main class="main-content">
<header>
<div class="view-title" id="current-view-title">Dashboard</div>
<div class="header-actions">
<button onclick="refreshData()">Refresh</button>
</div>
</header>
<div class="content-scroll">
<!-- Dashboard View -->
<div id="view-dashboard" class="view">
<div class="grid">
<div class="card">
<div class="card-title">System Overview</div>
<div id="dashboard-summary" style="margin-top:20px"></div>
</div>
<div class="card">
<div class="card-title">Active Incidents</div>
<div id="dashboard-incidents" style="margin-top:20px"></div>
</div>
</div>
</div>
<!-- Nodes View -->
<div id="view-nodes" class="view hidden">
<div class="grid" id="nodes-list"></div>
</div>
<!-- Services View -->
<div id="view-services" class="view hidden">
<div class="grid" id="services-list"></div>
</div>
<!-- Deployments View -->
<div id="view-deployments" class="view hidden">
<div class="grid" id="deployments-list"></div>
</div>
<!-- Events View -->
<div id="view-events" class="view hidden">
<div class="timeline" id="events-timeline"></div>
</div>
<!-- Recommendations View -->
<div id="view-recommendations" class="view hidden">
<div class="grid" id="recommendations-list"></div>
</div>
<!-- Settings View -->
<div id="view-settings" class="view hidden">
<div class="card">
<div class="card-title">Configuration</div>
<div id="settings-content" style="margin-top:20px"></div>
</div>
</div>
</div>
</section>
</main>
<script>
const logs = document.getElementById("logs");
const servicesList = document.getElementById("services-list");
let polling = true;
let currentConfig = {
auto_mode: true,
action_thresholds: {
restart_ha: 0.8,
check_network: 0.9
},
default_threshold: 0.9,
allowed_auto_actions: ["restart_ha"]
};
let currentView = 'dashboard';
const pollInterval = 5000;
function eventClass(event) {
const message = event.message || "";
if (event.type === "action") return "action";
if (message.startsWith("[error]")) return "error";
if (message.startsWith("[warning]")) return "warning";
if (message.startsWith("[info]")) return "info";
return event.type || "log";
function showView(viewId, el) {
document.querySelectorAll('.view').forEach(v => v.classList.add('hidden'));
document.getElementById('view-' + viewId).classList.remove('hidden');
document.querySelectorAll('.nav-item').forEach(i => i.classList.remove('active'));
if (el) el.classList.add('active');
currentView = viewId;
document.getElementById('current-view-title').textContent = viewId.charAt(0).toUpperCase() + viewId.slice(1);
refreshData();
}
function confidenceLabel(confidence) {
if (typeof confidence !== "number") return "";
return `${Math.round(confidence * 100)}%`;
}
function healthClass(status) {
const normalized = String(status || "unknown").toLowerCase();
if (["ok", "healthy", "up"].includes(normalized)) return "health-ok";
if (["warning", "degraded"].includes(normalized)) return "health-warning";
if (["error", "down", "failed"].includes(normalized)) return "health-error";
return "health-unknown";
}
function formatTime(timestamp) {
if (!timestamp) return "unknown";
return new Date(timestamp * 1000).toLocaleString();
}
async function fetchServices() {
const response = await fetch("/services", {cache: "no-store"});
const services = await response.json();
servicesList.innerHTML = "";
if (!Array.isArray(services) || services.length === 0) {
servicesList.textContent = "No service health data yet.";
return;
}
services.forEach((service) => {
const item = document.createElement("div");
item.className = "service";
const header = document.createElement("div");
header.className = "service-header";
const name = document.createElement("span");
name.className = "service-name";
name.textContent = service.name || "unknown";
header.appendChild(name);
const status = document.createElement("span");
status.className = healthClass(service.status);
status.textContent = String(service.status || "unknown").toUpperCase();
header.appendChild(status);
item.appendChild(header);
const lastCheck = document.createElement("div");
lastCheck.textContent = `Last check: ${formatTime(service.last_check)}`;
item.appendChild(lastCheck);
const history = document.createElement("div");
history.className = "service-history";
(service.history || []).slice(-5).forEach((entry) => {
const node = document.createElement("span");
node.className = `history-entry ${healthClass(entry.status)}`;
node.textContent = `${entry.status || "unknown"} ${formatTime(entry.timestamp)}`;
history.appendChild(node);
});
item.appendChild(history);
servicesList.appendChild(item);
});
}
async function fetchLogs() {
if (!polling) return;
const response = await fetch("/logs", {cache: "no-store"});
const text = await response.text();
logs.innerHTML = "";
const runs = {};
text.trim().split("\n").filter(Boolean).forEach((line) => {
let event;
async function fetchData(endpoint) {
try {
event = JSON.parse(line);
} catch {
event = {
type: "log",
message: line,
run_id: "legacy",
timestamp: Date.now() / 1000
};
const res = await fetch(endpoint, {cache: 'no-store'});
return await res.json();
} catch (e) {
console.error('Fetch error:', endpoint, e);
return null;
}
}
const runId = event.run_id || "no-run";
if (!runs[runId]) {
runs[runId] = {
events: [],
status: "running",
received: 0,
expected: 0
};
function formatTime(ts) {
if (!ts) return 'N/A';
return new Date(ts * 1000).toLocaleString();
}
const run = runs[runId];
run.events.push(event);
if (event.type === "run_status") {
run.status = event.status || run.status;
} else if (event.type === "run_progress") {
run.received = Number(event.received) || 0;
run.expected = Number(event.expected) || 0;
}
});
Object.entries(runs).forEach(([runId, runState]) => {
const run = document.createElement("div");
run.className = "run";
const status = runState.status;
if (status === "stopped") {
run.classList.add("stopped");
function getStatusClass(status) {
status = (status || '').toLowerCase();
if (['nominal', 'healthy', 'ok', 'up'].includes(status)) return 'status-nominal';
if (['degraded', 'warning'].includes(status)) return 'status-degraded';
if (['unstable'].includes(status)) return 'status-unstable';
if (['reconciling'].includes(status)) return 'status-reconciling';
if (['error', 'down', 'failed'].includes(status)) return 'status-error';
return '';
}
const header = document.createElement("div");
header.className = "header";
async function refreshData() {
// Refresh summary always
const summary = await fetchData('/summary');
if (summary) {
const statusEl = document.getElementById('summary-status');
statusEl.textContent = `System Status: ${summary.status.toUpperCase()}`;
statusEl.className = 'sidebar-footer ' + getStatusClass(summary.status);
const title = document.createElement("div");
title.className = "run-title";
title.appendChild(document.createTextNode(`RUN ${runId} [`));
const statusNode = document.createElement("span");
statusNode.className = `status ${status}`;
statusNode.textContent = status.toUpperCase();
title.appendChild(statusNode);
title.appendChild(
document.createTextNode(`] (${runState.received}/${runState.expected})`)
);
header.appendChild(title);
if (runId !== "no-run" && status === "running") {
const stopButton = document.createElement("button");
stopButton.textContent = "STOP";
stopButton.setAttribute("onclick", `stop(${JSON.stringify(runId)})`);
header.appendChild(stopButton);
if (currentView === 'dashboard') {
const dashSummary = document.getElementById('dashboard-summary');
dashSummary.innerHTML = `
<div class="label">Nodes</div><div class="value">${summary.node_count}</div>
<div class="label">Services</div><div class="value">${summary.service_count}</div>
<div class="label">Last Update</div><div class="value">${formatTime(summary.last_update)}</div>
`;
}
}
run.appendChild(header);
const eventList = document.createElement("div");
eventList.className = "events";
runState.events.forEach((event) => {
if (event.type === "proposal") {
const proposal = document.createElement("div");
proposal.className = "proposal";
const message = document.createElement("div");
message.textContent = event.message || "";
proposal.appendChild(message);
const confidence = document.createElement("div");
confidence.textContent = `Confidence: ${confidenceLabel(event.confidence)}`;
proposal.appendChild(confidence);
const actions = document.createElement("div");
actions.className = "proposal-actions";
const allOptions = event.options || event.actions || [];
const options = allOptions.filter((option) => !option.is_ignore).sort(
(a, b) => (b.confidence || 0) - (a.confidence || 0)
);
const ignoreOptions = allOptions.filter((option) => option.is_ignore);
options.forEach((option) => {
const button = document.createElement("button");
const optionConfidence = confidenceLabel(option.confidence);
button.textContent = optionConfidence
? `${option.label || option.command} (${optionConfidence})`
: option.label || option.command;
button.setAttribute(
"onclick",
`apply(${JSON.stringify(runId)}, ${JSON.stringify(option.command)})`
);
actions.appendChild(button);
});
proposal.appendChild(actions);
if (ignoreOptions.length) {
const divider = document.createElement("div");
divider.className = "proposal-divider";
proposal.appendChild(divider);
const ignoreActions = document.createElement("div");
ignoreActions.className = "proposal-actions";
ignoreOptions.forEach((option) => {
const button = document.createElement("button");
button.textContent = option.label || option.command;
button.setAttribute(
"onclick",
`apply(${JSON.stringify(runId)}, ${JSON.stringify(option.command)})`
);
ignoreActions.appendChild(button);
});
proposal.appendChild(ignoreActions);
if (currentView === 'dashboard' || currentView === 'events') {
const incidents = await fetchData('/incidents');
if (currentView === 'dashboard') {
const dashIncidents = document.getElementById('dashboard-incidents');
if (!incidents || incidents.length === 0) {
dashIncidents.textContent = 'No active incidents.';
} else {
dashIncidents.innerHTML = incidents.map(inc => `
<div class="event ${inc.severity}">
<strong>${inc.severity.toUpperCase()}:</strong> ${inc.message}<br>
<small>${formatTime(inc.timestamp)} | Node: ${inc.node}</small>
</div>
`).join('');
}
}
}
eventList.appendChild(proposal);
return;
if (currentView === 'nodes') {
const nodes = await fetchData('/nodes');
const list = document.getElementById('nodes-list');
list.innerHTML = nodes.map(node => `
<div class="card">
<div class="card-header">
<div class="card-title">${node.hostname}</div>
<span class="badge ${getStatusClass(node.health)}">${node.health}</span>
</div>
<div class="label">ID</div><div class="value mono">${node.id}</div>
<div class="label">Capabilities</div><div class="value">${node.capabilities.join(', ')}</div>
<div class="label">Connectivity</div><div class="value">${node.connectivity}</div>
<div class="label">Incidents (24h)</div><div class="value">${node.incidents}</div>
<div class="label">Last Seen</div><div class="value">${formatTime(node.last_seen)}</div>
<div class="label">Runtime Status</div><div class="value">${node.status}</div>
</div>
`).join('');
}
const div = document.createElement("div");
div.className = eventClass(event);
div.textContent = `${new Date(event.timestamp * 1000).toLocaleTimeString()} ${event.message}`;
eventList.appendChild(div);
});
run.appendChild(eventList);
logs.appendChild(run);
});
logs.scrollTop = logs.scrollHeight;
if (currentView === 'services') {
const services = await fetchData('/services');
const list = document.getElementById('services-list');
list.innerHTML = services.map(svc => `
<div class="card">
<div class="card-header">
<div class="card-title">${svc.name}</div>
<span class="badge ${getStatusClass(svc.health)}">${svc.health}</span>
</div>
<div class="label">State (Desired/Actual)</div><div class="value">${svc.desired_state} / ${svc.actual_state}</div>
<div class="label">Deployment</div><div class="value">${svc.deployment_state}</div>
<div class="label">Dependencies</div><div class="value">${svc.dependencies.join(', ') || 'None'}</div>
<div class="label">Recommendations</div><div class="value">${svc.recommendations.join(', ') || 'None'}</div>
</div>
`).join('');
}
async function send() {
const cmd = document.getElementById("cmd");
const value = cmd.value.trim();
if (!value) return;
await fetch("/command", {
method: "POST",
headers: {"Content-Type": "application/json"},
body: JSON.stringify({command: value})
});
cmd.value = "";
if (currentView === 'deployments') {
const deps = await fetchData('/deployments');
const list = document.getElementById('deployments-list');
list.innerHTML = deps.map(dep => `
<div class="card">
<div class="card-header">
<div class="card-title">${dep.service}</div>
<span class="badge ${dep.status === 'failed' ? 'status-error' : 'status-reconciling'}">${dep.status}</span>
</div>
<div class="label">ID</div><div class="value mono">${dep.id}</div>
<div class="label">Stage</div><div class="value">${dep.stage}</div>
<div class="label">Diagnostics</div><div class="value">${dep.diagnostics || 'No data'}</div>
<div class="label">Resumable</div><div class="value">${dep.resumable ? 'Yes' : 'No'}</div>
${dep.resumable ? '<button class="btn-primary">Resume</button>' : ''}
</div>
`).join('');
}
async function stop(run_id) {
await fetch("/stop", {
method: "POST",
headers: {"Content-Type": "application/json"},
body: JSON.stringify({run_id})
});
if (currentView === 'events') {
const events = await fetchData('/events');
const timeline = document.getElementById('events-timeline');
timeline.innerHTML = events.map(ev => `
<div class="event ${ev.severity}">
<div class="event-header">
<span>${ev.type.toUpperCase()}</span>
<span>${formatTime(ev.timestamp)}</span>
</div>
<div>${ev.message}</div>
<div class="label" style="margin-top:8px">Node: ${ev.node} ${ev.service ? '| Service: ' + ev.service : ''}</div>
</div>
`).join('');
}
async function apply(run_id, command) {
await fetch("/action", {
method: "POST",
headers: {"Content-Type": "application/json"},
body: JSON.stringify({run_id, command})
});
if (currentView === 'recommendations') {
const recs = await fetchData('/recommendations');
const list = document.getElementById('recommendations-list');
list.innerHTML = recs.map(rec => `
<div class="card">
<div class="card-header">
<div class="card-title">${rec.title}</div>
<span class="badge risk-${rec.risk_level}">${rec.risk_level}</span>
</div>
<p>${rec.description}</p>
<div class="label">Confidence</div><div class="value">${Math.round(rec.confidence * 100)}%</div>
<div class="label">Autonomous Eligible</div><div class="value">${rec.autonomous_eligible ? 'Yes' : 'No'}</div>
<div class="label">Blocked Actions</div><div class="value">${rec.blocked_actions.join(', ') || 'None'}</div>
<div class="controls">
<button class="btn-primary" ${rec.risk_level === 'dangerous' ? 'style="background:var(--dangerous)"' : ''}>Approve Action</button>
</div>
</div>
`).join('');
}
function populateConfig(config) {
currentConfig = config;
document.getElementById("config-auto-mode").checked = Boolean(config.auto_mode);
document.getElementById("restart-threshold").value =
config.action_thresholds?.restart_ha ?? "";
document.getElementById("network-threshold").value =
config.action_thresholds?.check_network ?? "";
if (currentView === 'settings') {
const config = await fetchData('/config');
const content = document.getElementById('settings-content');
content.innerHTML = `
<div class="label">Auto Mode</div>
<div class="value">${config.auto_mode ? 'Enabled' : 'Disabled'}</div>
<div class="label">Action Thresholds</div>
<div class="value mono">${JSON.stringify(config.action_thresholds, null, 2)}</div>
<button onclick="alert('Settings update not implemented in this demo')">Edit Configuration</button>
`;
}
}
async function loadConfig() {
const response = await fetch("/config", {cache: "no-store"});
populateConfig(await response.json());
}
// Initial load
refreshData();
// Poll for updates
setInterval(refreshData, pollInterval);
async function saveConfig() {
const restartThreshold = Number(document.getElementById("restart-threshold").value);
const networkThreshold = Number(document.getElementById("network-threshold").value);
if (!Number.isFinite(restartThreshold) || !Number.isFinite(networkThreshold)) {
return;
}
const config = {
auto_mode: document.getElementById("config-auto-mode").checked,
action_thresholds: {
...(currentConfig.action_thresholds || {}),
restart_ha: restartThreshold,
check_network: networkThreshold
},
default_threshold: currentConfig.default_threshold ?? 0.9,
allowed_auto_actions: currentConfig.allowed_auto_actions || ["restart_ha"]
};
await fetch("/config", {
method: "POST",
headers: {"Content-Type": "application/json"},
body: JSON.stringify(config)
});
populateConfig(config);
}
document.getElementById("stop").addEventListener("click", async () => {
await fetch("/command", {
method: "POST",
headers: {"Content-Type": "application/json"},
body: JSON.stringify({command: "stop"})
});
});
document.getElementById("save-config").addEventListener("click", saveConfig);
document.getElementById("cmd").addEventListener("keydown", (event) => {
if (event.key === "Enter") send();
});
loadConfig();
fetchServices();
fetchLogs();
setInterval(fetchLogs, 1000);
setInterval(fetchServices, 10000);
</script>
</body>
</html>

View file

@ -5,6 +5,9 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
STATE_DIR = Path("/opt/homelab/state")
EVENTS_DIR = Path("/opt/homelab/events")
WORLD_DIR = Path("/opt/homelab/world")
EVENT_LOG = Path("/tmp/agent-events.log")
STATIC_DIR = Path(__file__).parent
REDIS_HOST = os.getenv("REDIS_HOST", "redis")
@ -101,8 +104,18 @@ def send_event(event):
redis_command("LPUSH", "events", json.dumps(event))
def read_json_file(path, default=None):
if not path.exists():
return default if default is not None else []
try:
return json.loads(path.read_text())
except Exception:
return default if default is not None else []
def current_config():
config = json.loads(json.dumps(DEFAULT_CONFIG))
# We still keep reading from EVENT_LOG for auto_config if it's there
for line in tail_lines(EVENT_LOG, 500):
try:
event = json.loads(line)
@ -117,16 +130,38 @@ def current_config():
return config
def current_nodes():
return read_json_file(STATE_DIR / "nodes.json")
def current_services():
services = []
for line in tail_lines(EVENT_LOG, 1000):
try:
event = json.loads(line)
except json.JSONDecodeError:
continue
if event.get("type") == "services_state":
services = event.get("services", [])
return services
return read_json_file(STATE_DIR / "services.json")
def current_deployments():
return read_json_file(STATE_DIR / "deployments.json")
def current_incidents():
return read_json_file(STATE_DIR / "incidents.json")
def current_recommendations():
return read_json_file(STATE_DIR / "recommendations.json")
def current_summary():
return read_json_file(STATE_DIR / "runtime-summary.json", default={})
def current_events():
events = []
if EVENTS_DIR.exists():
for f in EVENTS_DIR.glob("*.json"):
data = read_json_file(f)
if data:
events.append(data)
return sorted(events, key=lambda x: x.get("timestamp", 0), reverse=True)
def send_json(status, payload, handler):
@ -144,10 +179,34 @@ class Handler(BaseHTTPRequestHandler):
send_json(200, current_config(), self)
return
if self.path == "/nodes":
send_json(200, current_nodes(), self)
return
if self.path == "/services":
send_json(200, current_services(), self)
return
if self.path == "/deployments":
send_json(200, current_deployments(), self)
return
if self.path == "/incidents":
send_json(200, current_incidents(), self)
return
if self.path == "/recommendations":
send_json(200, current_recommendations(), self)
return
if self.path == "/summary":
send_json(200, current_summary(), self)
return
if self.path == "/events":
send_json(200, current_events(), self)
return
if self.path == "/logs":
print("LOGS endpoint called", flush=True)
body = ("\n".join(tail_lines(EVENT_LOG, 200)) + "\n").encode("utf-8")