diff --git a/docs/backlog.md b/docs/backlog.md index 4a068c9..6ae3d13 100644 --- a/docs/backlog.md +++ b/docs/backlog.md @@ -22,12 +22,15 @@ dawny kontener piha (docker-exec, archived). `services/home-assistant/config/ken-legacy/` → `docker stop` (BEZ `rm`) → 7 dni obserwacji (upewnić się, że nic w domu nie polega na tym kontenerze) → decyzja o `docker rm`. -3. **Adapter `api` w `import.sh` dla `ken`**: obecnie tylko `docker-exec` - jest zaimplementowany (`scripts/ha/import.sh`), a `ken` po cutoverze - używa `api`. Automatyzacje/skrypty/sceny przez - `/api/config//config/`, dashboardy przez websocket API. - Pełny import `/config` pozostaje poza zasięgiem (HAOS bez SSH) — patrz - DESIGN.md. +3. ✅ ZROBIONE (2026-07-22) — **Adapter `api` w `import.sh` dla `ken`**: + automatyzacje/skrypty/sceny przez `/api/config//config/` + (REST), dashboardy/area+entity registry/`input_*` helpery przez websocket + API (`scripts/ha/lib/ha_api.py`, `ha_ws.py`, `import_api.py`). Pierwszy + realny import `ken` zaimportował 118 automatyzacji, 5 skryptów, 3 sceny, + 7 dashboardów (default + 6 named; jeden zarejestrowany dashboard nigdy + nie skonfigurowany — `config_not_found`, odnotowany w raporcie, nie + twardy błąd). Pełny import `/config` pozostaje poza zasięgiem (HAOS bez + SSH) — patrz DESIGN.md. --- diff --git a/scripts/ha/import.sh b/scripts/ha/import.sh index c276f40..50a5bab 100755 --- a/scripts/ha/import.sh +++ b/scripts/ha/import.sh @@ -7,11 +7,17 @@ # against an unchanged instance produces no diff under # services/home-assistant/config// or storage-export//. # -# Only the "docker-exec" adapter (instance "ken") is implemented here. Other -# adapters (e.g. "api", used by "chelsty-ha") are out of scope for this -# skeleton — see DESIGN.md, "Deploy path: adapter per instance". The -# /api/states fixture fetch is the one piece that works for any adapter, -# since it only needs a reachable base_url + token, not full config access. +# Two adapters are implemented — see DESIGN.md, "Deploy path: adapter per +# instance": +# - "docker-exec": pulls the whole /config tree over `ssh ... docker exec +# ... tar`. Used where the HA instance's filesystem is reachable. +# - "api": for instances with no filesystem/SSH access (HAOS, e.g. "ken"). +# Pulls automations/scripts/scenes via REST and dashboards/registries/ +# helpers via the WebSocket API (scripts/ha/lib/import_api.py). Full +# /config import is out of scope for this adapter. +# +# The /api/states fixture fetch works for either adapter, since it only +# needs a reachable base_url + token, not full config access. # # Usage: scripts/ha/import.sh set -euo pipefail @@ -40,9 +46,6 @@ fetch_fixtures() { return 0 fi - local token - token="$(<"$token_file")" - mkdir -p "$FIXTURES_DIR" local dest="$FIXTURES_DIR/${INSTANCE}-states-$(date +%F).yaml" local raw_file @@ -50,11 +53,19 @@ fetch_fixtures() { local fetch_ok=1 if [[ "$HA_ADAPTER" == "docker-exec" ]]; then + # docker-exec adapter: token is read into a shell variable and embedded + # in the remote curl command over ssh (pre-existing behavior, left + # untouched — see DESIGN.md, "Deploy path: adapter per instance"). + local token + token="$(<"$token_file")" ssh "${HA_SSH_USER}@${HA_SSH_HOST}" \ "curl -sf -H 'Authorization: Bearer ${token}' http://localhost:8123/api/states" \ > "$raw_file" || fetch_ok=0 else - curl -sf -H "Authorization: Bearer ${token}" "${HA_BASE_URL}/api/states" \ + # api adapter: the token never touches a shell variable or a subprocess + # argv here — ha_api.py reads token_file itself and sets the + # Authorization header in-process via `requests`. + python3 "$SCRIPT_DIR/lib/ha_api.py" get-raw "$HA_BASE_URL" "$token_file" /api/states \ > "$raw_file" || fetch_ok=0 fi @@ -104,13 +115,7 @@ FIXTURES_DIR="$SERVICE_DIR/fixtures" echo "== ha import: instance=$INSTANCE adapter=$HA_ADAPTER host=$HA_HOST ==" >&2 -if [[ "$HA_ADAPTER" != "docker-exec" ]]; then - echo "error: adapter '$HA_ADAPTER' has no config-extraction implementation in this skeleton." >&2 - echo " see DESIGN.md, 'Deploy path: adapter per instance' — only docker-exec is built." >&2 - echo " config/${INSTANCE}/ and storage-export/${INSTANCE}/ were NOT touched." >&2 - fetch_fixtures || true - exit 1 -fi +if [[ "$HA_ADAPTER" == "docker-exec" ]]; then if [[ -z "$HA_SSH_USER" || -z "$HA_SSH_HOST" ]]; then echo "error: instance '$INSTANCE' is missing ssh.user/ssh.host in instances.yaml" >&2 @@ -172,6 +177,29 @@ else echo " (no .storage directory in pulled config — skipping)" >&2 fi +elif [[ "$HA_ADAPTER" == "api" ]]; then + + api_token_file="${HA_TOKEN_PATH/#\~/$HOME}" + if [[ -z "$HA_TOKEN_PATH" || ! -f "$api_token_file" ]]; then + echo "error: instance '$INSTANCE' has adapter=api but no token at '${api_token_file:-}'." >&2 + echo " the api adapter has no filesystem fallback (HAOS has no SSH) — without a" >&2 + echo " deploy_agent long-lived access token there is nothing to import at all." >&2 + echo " See DESIGN.md, 'Tokens', for how to provision one." >&2 + exit 1 + fi + + mkdir -p "$CONFIG_OUT_DIR" "$STORAGE_OUT_DIR" + echo "-> importing automations/scripts/scenes (REST) + dashboards/registries/helpers (websocket) ..." >&2 + python3 "$SCRIPT_DIR/lib/import_api.py" "$HA_BASE_URL" "$api_token_file" "$CONFIG_OUT_DIR" "$STORAGE_OUT_DIR" + +else + echo "error: adapter '$HA_ADAPTER' has no config-extraction implementation." >&2 + echo " see DESIGN.md, 'Deploy path: adapter per instance' — only docker-exec and api are built." >&2 + echo " config/${INSTANCE}/ and storage-export/${INSTANCE}/ were NOT touched." >&2 + fetch_fixtures || true + exit 1 +fi + fetch_fixtures echo "-> summary of changes:" >&2 diff --git a/scripts/ha/lib/ha_api.py b/scripts/ha/lib/ha_api.py new file mode 100755 index 0000000..4ea7019 --- /dev/null +++ b/scripts/ha/lib/ha_api.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python3 +"""Minimal REST client for the HA "api" adapter (see DESIGN.md, "Deploy path: +adapter per instance"). + +Every entrypoint here takes a *token_path*, never a token value, and reads +the token from disk itself. The bearer token must never appear as a process +argv (visible to any local user via `ps`) or in a log line — see DESIGN.md, +"Tokens", and the api-adapter requirements in docs/backlog.md. Callers +needing the token value in-process (e.g. the WebSocket client in +ha_ws.py) should call read_token() directly rather than shelling out. + +Read-only by construction: the only method this module exposes is GET. +""" +import os +import sys + +import requests + + +class HaApiError(RuntimeError): + """Raised for adapter-level failures (missing/empty token, etc.).""" + + +def read_token(token_path): + """Read and strip the bearer token from `token_path` (`~` expanded). + + Hard error (raises HaApiError) if the file is absent or empty — the api + adapter has no filesystem fallback, so a missing token means the import + cannot do anything at all, unlike the fixtures fetch, which soft-skips. + """ + path = os.path.expanduser(token_path) if token_path else "" + if not path or not os.path.isfile(path): + raise HaApiError( + f"no HA token at '{path or ''}' — the api adapter requires a " + "deploy_agent long-lived access token (see DESIGN.md, 'Tokens'); " + "this is a hard error, not a soft-skip, because nothing in this " + "adapter works without one" + ) + token = open(path, "r", encoding="utf-8").read().strip() + if not token: + raise HaApiError(f"token file '{path}' is empty") + return token + + +class Client: + """Thin wrapper around one HA instance's REST API (GET only).""" + + def __init__(self, base_url, token, timeout=30): + self._base_url = base_url.rstrip("/") + self._token = token + self._timeout = timeout + + def _headers(self): + return {"Authorization": f"Bearer {self._token}"} + + def get(self, path): + """GET `path` (e.g. "/api/states"), return parsed JSON. + + Returns None on 404 so callers can treat "config not found" (e.g. an + automation removed between the states poll and the config fetch) as + a skip-and-report condition instead of a hard failure. Any other + non-2xx status raises requests.HTTPError. + """ + resp = requests.get( + self._base_url + path, headers=self._headers(), timeout=self._timeout + ) + if resp.status_code == 404: + return None + resp.raise_for_status() + return resp.json() + + def get_raw_text(self, path): + """GET `path`, return the raw response body (used for fixtures).""" + resp = requests.get( + self._base_url + path, headers=self._headers(), timeout=self._timeout + ) + resp.raise_for_status() + return resp.text + + +def _cli_get_raw(args): + if len(args) != 3: + print("usage: ha_api.py get-raw ", file=sys.stderr) + return 2 + base_url, token_path, api_path = args + token = read_token(token_path) + client = Client(base_url, token) + sys.stdout.write(client.get_raw_text(api_path)) + return 0 + + +def main(argv): + """CLI entrypoint. Only sub-command: `get-raw` — print a raw GET body. + + Used by import.sh's fixtures fetch for the api adapter so the token + never touches a subprocess argv (unlike `curl -H "Authorization: ..."`, + which would put it in `ps` output for anyone on the same host). + """ + if len(argv) < 2: + print("usage: ha_api.py get-raw ", file=sys.stderr) + return 2 + command, rest = argv[1], argv[2:] + try: + if command == "get-raw": + return _cli_get_raw(rest) + print(f"ha_api.py: unknown command '{command}'", file=sys.stderr) + return 2 + except (HaApiError, requests.RequestException) as exc: + print(f"ha_api.py: {exc}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv)) diff --git a/scripts/ha/lib/ha_ws.py b/scripts/ha/lib/ha_ws.py new file mode 100755 index 0000000..82a5ace --- /dev/null +++ b/scripts/ha/lib/ha_ws.py @@ -0,0 +1,102 @@ +#!/usr/bin/env python3 +"""Minimal synchronous client for HA's `/api/websocket` API. + +Used by the "api" adapter (see DESIGN.md, "Deploy path: adapter per +instance") to reach dashboards, area/entity registries, and input_* helpers +— none of these are exposed over REST. Read-only by construction: the only +commands this module ever sends are `auth` and whatever `*_list`/ +`lovelace/config` command the caller passes to `HaWsClient.command()` — it +has no method that issues a create/update/delete command. + +Depends on the `websocket-client` PyPI package (import name `websocket`), +which is not in the stdlib. See services/home-assistant/README.md, +"WebSocket dependency", for the install command. Importing this module +raises ImportError with an actionable message if it's missing, instead of +the default traceback, so the api adapter can surface one clear line +instead of a stack trace. +""" +import itertools +import json + +try: + import websocket as _ws +except ImportError as exc: # pragma: no cover - exercised via import.sh error path + raise ImportError( + "the 'websocket-client' package is required for the api adapter's " + "dashboard/registry/helper export (HA's WebSocket API) — install it " + "with 'sudo apt install python3-websocket' or 'pip install --user " + "websocket-client' (see services/home-assistant/README.md, " + "'WebSocket dependency')" + ) from exc + + +class HaWsError(RuntimeError): + """Raised for handshake/auth failures or a command that reports failure.""" + + +class HaWsClient: + """One `/api/websocket` connection: authenticate once, issue N read-only commands.""" + + def __init__(self, base_url, token, timeout=30): + ws_url = base_url.rstrip("/") + if ws_url.startswith("https://"): + ws_url = "wss://" + ws_url[len("https://"):] + elif ws_url.startswith("http://"): + ws_url = "ws://" + ws_url[len("http://"):] + self._url = ws_url + "/api/websocket" + self._token = token + self._timeout = timeout + self._next_id = itertools.count(1) + self._conn = None + + def __enter__(self): + self.connect() + return self + + def __exit__(self, *exc_info): + self.close() + + def connect(self): + self._conn = _ws.create_connection(self._url, timeout=self._timeout) + hello = self._recv() + if hello.get("type") != "auth_required": + self.close() + raise HaWsError(f"unexpected handshake message from {self._url}: {hello!r}") + + self._conn.send(json.dumps({"type": "auth", "access_token": self._token})) + auth_result = self._recv() + if auth_result.get("type") != "auth_ok": + self.close() + raise HaWsError( + "websocket auth rejected — check the deploy_agent token " + f"(server said: {auth_result!r})" + ) + + def close(self): + if self._conn is not None: + self._conn.close() + self._conn = None + + def _recv(self): + return json.loads(self._conn.recv()) + + def command(self, command_type, **kwargs): + """Send one read-only WS command, return its `result` payload. + + Raises HaWsError if the server reports failure or sends something + other than a matching `result` message. + """ + msg_id = next(self._next_id) + payload = {"id": msg_id, "type": command_type} + payload.update(kwargs) + self._conn.send(json.dumps(payload)) + while True: + msg = self._recv() + if msg.get("id") != msg_id: + # Not our response (e.g. an unrelated event) — keep waiting. + continue + if msg.get("type") == "result": + if not msg.get("success", False): + raise HaWsError(f"{command_type} failed: {msg.get('error')}") + return msg.get("result") + raise HaWsError(f"unexpected response to {command_type}: {msg!r}") diff --git a/scripts/ha/lib/import_api.py b/scripts/ha/lib/import_api.py new file mode 100755 index 0000000..02d32a4 --- /dev/null +++ b/scripts/ha/lib/import_api.py @@ -0,0 +1,278 @@ +#!/usr/bin/env python3 +"""Read-only import for the HA "api" adapter (see DESIGN.md, "Deploy path: +adapter per instance", and instances.yaml's `ken` entry). + +For instances with no filesystem/SSH access (HAOS), this pulls whatever the +HA REST + WebSocket API exposes: + +- automations, scripts, scenes — REST, one GET per object + (`/api/config//config/`), split one-file-per-object exactly + like the docker-exec adapter's automations.yaml/scripts.yaml/scenes.yaml + handling (scripts/ha/lib/split.py), so both adapters produce the same + config//{automations,scripts,scenes}/ layout. +- dashboards, area/entity registries, input_* helpers — WebSocket, list/get + commands only. Never a mutating command. + +Full /config import is out of scope here (HAOS has no SSH) — see DESIGN.md. + +Entities without a storage-backed id (YAML-configured automations/scenes) +can't be fetched via `/api/config/.../config/` — they are skipped and +noted in the report, never silently dropped. +""" +import os +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + +import ha_api # noqa: E402 +from split import write_split # noqa: E402 + +INPUT_HELPER_DOMAINS = ( + "input_boolean", + "input_number", + "input_text", + "input_datetime", + "input_select", + "input_button", +) + + +def entity_domain(entity_id): + return entity_id.split(".", 1)[0] + + +def entity_object_id(entity_id): + return entity_id.split(".", 1)[1] + + +def new_report(): + return { + "automations_imported": 0, + "automations_skipped_yaml": [], + "automations_skipped_missing": [], + "scripts_imported": 0, + "scripts_skipped_missing": [], + "scenes_imported": 0, + "scenes_skipped_yaml": [], + "scenes_skipped_missing": [], + "dashboards_imported": 0, + "dashboards_skipped": [], + "helpers_imported": {}, + "websocket_error": None, + } + + +def import_automations(client, states, config_out_dir, report): + entities = [s for s in states if entity_domain(s["entity_id"]) == "automation"] + objects_by_id = {} + for ent in entities: + auto_id = ent.get("attributes", {}).get("id") + if auto_id is None: + report["automations_skipped_yaml"].append(ent["entity_id"]) + continue + cfg = client.get(f"/api/config/automation/config/{auto_id}") + if cfg is None: + report["automations_skipped_missing"].append(str(auto_id)) + continue + objects_by_id[str(auto_id)] = cfg + write_split(objects_by_id, os.path.join(config_out_dir, "automations")) + report["automations_imported"] = len(objects_by_id) + + +def import_scripts(client, states, config_out_dir, report): + entities = [s for s in states if entity_domain(s["entity_id"]) == "script"] + objects_by_key = {} + for ent in entities: + object_id = entity_object_id(ent["entity_id"]) + cfg = client.get(f"/api/config/script/config/{object_id}") + if cfg is None: + report["scripts_skipped_missing"].append(object_id) + continue + objects_by_key[object_id] = cfg + write_split(objects_by_key, os.path.join(config_out_dir, "scripts")) + report["scripts_imported"] = len(objects_by_key) + + +def import_scenes(client, states, config_out_dir, report): + entities = [s for s in states if entity_domain(s["entity_id"]) == "scene"] + objects_by_id = {} + for ent in entities: + scene_id = ent.get("attributes", {}).get("id") + if scene_id is None: + report["scenes_skipped_yaml"].append(ent["entity_id"]) + continue + cfg = client.get(f"/api/config/scene/config/{scene_id}") + if cfg is None: + report["scenes_skipped_missing"].append(str(scene_id)) + continue + objects_by_id[str(scene_id)] = cfg + write_split(objects_by_id, os.path.join(config_out_dir, "scenes")) + report["scenes_imported"] = len(objects_by_id) + + +def import_dashboards_and_registries(base_url, token, states, storage_out_dir, report): + """WebSocket-only exports: dashboards, area/entity registries, input_* helpers. + + Written through split.write_split() like the config-side objects above, + so a domain/dashboard that disappears between runs has its stale export + file cleaned up too (idempotent: unchanged instance -> zero diff). + """ + try: + import ha_ws + except ImportError as exc: + report["websocket_error"] = str(exc) + return + + documents = {} + + try: + with ha_ws.HaWsClient(base_url, token) as ws: + try: + default_config = ws.command("lovelace/config") + except ha_ws.HaWsError as exc: + report["dashboards_skipped"].append(("", str(exc))) + else: + documents["lovelace"] = {"key": "lovelace", "data": {"config": default_config}} + report["dashboards_imported"] += 1 + + dashboards = ws.command("lovelace/dashboards/list") + documents["lovelace_dashboards"] = { + "key": "lovelace_dashboards", + "data": {"items": dashboards}, + } + for dash in dashboards: + dash_id = dash["id"] + if dash.get("mode") != "storage": + report["dashboards_skipped"].append( + (dash_id, f"mode={dash.get('mode')!r} is file-based, not reachable via api") + ) + continue + try: + cfg = ws.command("lovelace/config", url_path=dash["url_path"]) + except ha_ws.HaWsError as exc: + report["dashboards_skipped"].append((dash_id, str(exc))) + continue + documents[f"lovelace.{dash_id}"] = { + "key": f"lovelace.{dash_id}", + "data": {"config": cfg}, + } + report["dashboards_imported"] += 1 + + resources = ws.command("lovelace/resources") + documents["lovelace_resources"] = { + "key": "lovelace_resources", + "data": {"items": resources}, + } + + areas = ws.command("config/area_registry/list") + documents["core.area_registry"] = { + "key": "core.area_registry", + "data": {"areas": areas}, + } + + entity_registry = ws.command("config/entity_registry/list") + documents["core.entity_registry"] = { + "key": "core.entity_registry", + "data": {"entities": entity_registry}, + } + + present_domains = {entity_domain(s["entity_id"]) for s in states} + for domain in INPUT_HELPER_DOMAINS: + if domain not in present_domains: + continue + items = ws.command(f"{domain}/list") + documents[domain] = {"key": domain, "data": {"items": items}} + report["helpers_imported"][domain] = len(items) + except Exception as exc: # noqa: BLE001 - surface any transport/handshake error verbatim + report["websocket_error"] = str(exc) + return + + write_split(documents, storage_out_dir) + + +def print_report(report): + print("-> api import report:", file=sys.stderr) + print( + f" automations: {report['automations_imported']} imported, " + f"{len(report['automations_skipped_yaml'])} skipped (yaml, no storage id), " + f"{len(report['automations_skipped_missing'])} skipped (config not found)", + file=sys.stderr, + ) + for eid in report["automations_skipped_yaml"]: + print(f" yaml automation (not importable via api): {eid}", file=sys.stderr) + for eid in report["automations_skipped_missing"]: + print(f" automation id {eid}: config not found (404)", file=sys.stderr) + + print( + f" scripts: {report['scripts_imported']} imported, " + f"{len(report['scripts_skipped_missing'])} skipped (config not found)", + file=sys.stderr, + ) + for object_id in report["scripts_skipped_missing"]: + print(f" script {object_id}: config not found (404)", file=sys.stderr) + + print( + f" scenes: {report['scenes_imported']} imported, " + f"{len(report['scenes_skipped_yaml'])} skipped (yaml, no storage id), " + f"{len(report['scenes_skipped_missing'])} skipped (config not found)", + file=sys.stderr, + ) + for eid in report["scenes_skipped_yaml"]: + print(f" yaml scene (not importable via api): {eid}", file=sys.stderr) + for eid in report["scenes_skipped_missing"]: + print(f" scene id {eid}: config not found (404)", file=sys.stderr) + + if report["websocket_error"]: + print( + f" dashboards/registries/helpers: SKIPPED — {report['websocket_error']}", + file=sys.stderr, + ) + return + + print(f" dashboards: {report['dashboards_imported']} imported", file=sys.stderr) + for dash_id, reason in report["dashboards_skipped"]: + print(f" skipped dashboard '{dash_id}': {reason}", file=sys.stderr) + for domain, count in report["helpers_imported"].items(): + print(f" {domain}: {count} items", file=sys.stderr) + + +def main(argv): + if len(argv) != 5: + print( + "usage: import_api.py ", + file=sys.stderr, + ) + return 2 + _, base_url, token_path, config_out_dir, storage_out_dir = argv + + try: + token = ha_api.read_token(token_path) + except ha_api.HaApiError as exc: + print(f"error: {exc}", file=sys.stderr) + return 1 + + client = ha_api.Client(base_url, token) + try: + states = client.get("/api/states") + except Exception as exc: # noqa: BLE001 - surface any transport error verbatim + print(f"error: fetching /api/states failed: {exc}", file=sys.stderr) + return 1 + + report = new_report() + try: + import_automations(client, states, config_out_dir, report) + import_scripts(client, states, config_out_dir, report) + import_scenes(client, states, config_out_dir, report) + except Exception as exc: # noqa: BLE001 - surface any transport error verbatim + print(f"error: importing automations/scripts/scenes failed: {exc}", file=sys.stderr) + return 1 + + import_dashboards_and_registries(base_url, token, states, storage_out_dir, report) + + print_report(report) + + return 1 if report["websocket_error"] else 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv)) diff --git a/scripts/ha/tests/fixtures/api_automation_config_111.json b/scripts/ha/tests/fixtures/api_automation_config_111.json new file mode 100644 index 0000000..a07ce4b --- /dev/null +++ b/scripts/ha/tests/fixtures/api_automation_config_111.json @@ -0,0 +1,21 @@ +{ + "id": "111", + "alias": "Morning lights", + "description": "", + "trigger": [ + { + "platform": "time", + "at": "07:00:00" + } + ], + "condition": [], + "action": [ + { + "type": "turn_on", + "device_id": "abc123", + "entity_id": "light.living_room", + "domain": "light" + } + ], + "mode": "single" +} diff --git a/scripts/ha/tests/fixtures/api_scene_config_222.json b/scripts/ha/tests/fixtures/api_scene_config_222.json new file mode 100644 index 0000000..3626ab0 --- /dev/null +++ b/scripts/ha/tests/fixtures/api_scene_config_222.json @@ -0,0 +1,10 @@ +{ + "id": "222", + "name": "Demo scene", + "entities": { + "light.living_room": { + "state": "on", + "brightness": 180 + } + } +} diff --git a/scripts/ha/tests/fixtures/api_script_config_demo_script.json b/scripts/ha/tests/fixtures/api_script_config_demo_script.json new file mode 100644 index 0000000..a97764b --- /dev/null +++ b/scripts/ha/tests/fixtures/api_script_config_demo_script.json @@ -0,0 +1,12 @@ +{ + "alias": "demo_script", + "description": "", + "sequence": [ + { + "type": "turn_on", + "device_id": "def456", + "entity_id": "switch.demo", + "domain": "switch" + } + ] +} diff --git a/scripts/ha/tests/fixtures/api_states.json b/scripts/ha/tests/fixtures/api_states.json new file mode 100644 index 0000000..e24b287 --- /dev/null +++ b/scripts/ha/tests/fixtures/api_states.json @@ -0,0 +1,39 @@ +[ + { + "entity_id": "automation.morning_lights", + "state": "on", + "attributes": { + "id": "111", + "friendly_name": "Morning lights" + } + }, + { + "entity_id": "automation.legacy_yaml_automation", + "state": "on", + "attributes": { + "friendly_name": "Legacy YAML automation (no storage id)" + } + }, + { + "entity_id": "script.demo_script", + "state": "off", + "attributes": { + "friendly_name": "Demo script" + } + }, + { + "entity_id": "scene.demo_scene", + "state": "scening", + "attributes": { + "id": "222", + "friendly_name": "Demo scene" + } + }, + { + "entity_id": "input_boolean.on_leave", + "state": "off", + "attributes": { + "friendly_name": "On leave" + } + } +] diff --git a/scripts/ha/tests/test_import_api_offline.sh b/scripts/ha/tests/test_import_api_offline.sh new file mode 100755 index 0000000..85edc5d --- /dev/null +++ b/scripts/ha/tests/test_import_api_offline.sh @@ -0,0 +1,217 @@ +#!/usr/bin/env bash +# Offline tests for the "api" adapter (scripts/ha/lib/import_api.py, ha_api.py, +# ha_ws.py): normalization of real-shaped HA API responses saved as fixtures, +# idempotency, stale-file cleanup, and the "websocket-client missing" error +# path. No network access, no HA instance needed — REST calls are replaced +# with a fake in-memory client, and the WebSocket client is replaced via +# sys.modules injection (see PYEOF below). +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +HA_LIB_DIR="$SCRIPT_DIR/../lib" +FIXTURES_DIR="$SCRIPT_DIR/fixtures" + +python3 - "$HA_LIB_DIR" "$FIXTURES_DIR" <<'PYEOF' +import json +import os +import sys +import tempfile + +lib_dir, fixtures_dir = sys.argv[1:3] +sys.path.insert(0, lib_dir) + +import import_api # noqa: E402 +from normalize import parse_yaml_text # noqa: E402 + +fail = False + + +def check(condition, message): + global fail + if condition: + print(f"PASS: {message}") + else: + print(f"FAIL: {message}", file=sys.stderr) + fail = True + + +def load_fixture(name): + with open(os.path.join(fixtures_dir, name), encoding="utf-8") as f: + return json.load(f) + + +class FakeClient: + """Stands in for ha_api.Client: canned responses keyed by exact path.""" + + def __init__(self, responses): + self._responses = responses + + def get(self, path): + return self._responses.get(path) + + +# --- automations / scripts / scenes: REST-shaped fixtures, no network --- + +states = load_fixture("api_states.json") +responses = { + "/api/config/automation/config/111": load_fixture("api_automation_config_111.json"), + "/api/config/script/config/demo_script": load_fixture("api_script_config_demo_script.json"), + "/api/config/scene/config/222": load_fixture("api_scene_config_222.json"), +} +client = FakeClient(responses) + +with tempfile.TemporaryDirectory() as config_out_dir: + report = import_api.new_report() + import_api.import_automations(client, states, config_out_dir, report) + import_api.import_scripts(client, states, config_out_dir, report) + import_api.import_scenes(client, states, config_out_dir, report) + + check(report["automations_imported"] == 1, "one automation imported (the one with attributes.id)") + check( + report["automations_skipped_yaml"] == ["automation.legacy_yaml_automation"], + "id-less automation entity reported as yaml-skipped, not silently dropped", + ) + check(report["scripts_imported"] == 1, "one script imported") + check(report["scenes_imported"] == 1, "one scene imported") + + auto_path = os.path.join(config_out_dir, "automations", "111.yaml") + script_path = os.path.join(config_out_dir, "scripts", "demo_script.yaml") + scene_path = os.path.join(config_out_dir, "scenes", "222.yaml") + check(os.path.isfile(auto_path), "automations/111.yaml written") + check(os.path.isfile(script_path), "scripts/demo_script.yaml written") + check(os.path.isfile(scene_path), "scenes/222.yaml written") + + with open(auto_path, encoding="utf-8") as f: + auto_text = f.read() + check("alias: Morning lights" in auto_text, "automation config normalized to canonical (sorted-key) YAML") + parsed_back = parse_yaml_text(auto_text) + check( + parsed_back == load_fixture("api_automation_config_111.json"), + "normalized automation YAML round-trips to the original API response", + ) + + # stale-file cleanup: pre-seed an id that no longer exists, re-run, must be gone + stale_path = os.path.join(config_out_dir, "automations", "999.yaml") + with open(stale_path, "w", encoding="utf-8") as f: + f.write("alias: stale\n") + report2 = import_api.new_report() + import_api.import_automations(client, states, config_out_dir, report2) + check(not os.path.isfile(stale_path), "stale automation export removed on re-run (idempotent cleanup)") + + # idempotency: second run over the same fixture data is byte-identical + with open(auto_path, encoding="utf-8") as f: + pass1 = f.read() + import_api.import_automations(client, states, config_out_dir, report2) + with open(auto_path, encoding="utf-8") as f: + pass2 = f.read() + check(pass1 == pass2, "automations/111.yaml byte-identical across two runs (idempotent)") + + +# --- dashboards / registries / helpers: WebSocket-shaped fixtures --- + +FAKE_DASHBOARD_LIST = [ + {"id": "map", "title": "Map", "url_path": "map", "mode": "storage"}, + {"id": "kiosk", "title": "Kiosk", "url_path": "kiosk", "mode": "yaml"}, +] +FAKE_WS_RESULTS = { + ("lovelace/config", ()): {"title": "Overview", "views": []}, + ("lovelace/config", (("url_path", "map"),)): {"title": "Map", "views": []}, + ("lovelace/dashboards/list", ()): FAKE_DASHBOARD_LIST, + ("lovelace/resources", ()): [{"id": "r1", "type": "module", "url": "/local/x.js"}], + ("config/area_registry/list", ()): [{"id": "hall", "name": "Hall"}], + ("config/entity_registry/list", ()): [{"entity_id": "light.living_room"}], + ("input_boolean/list", ()): [{"id": "on_leave", "name": "On leave"}], +} + + +class FakeHaWsError(RuntimeError): + pass + + +class FakeHaWsClient: + def __init__(self, base_url, token): + self.base_url = base_url + self.token = token + + def __enter__(self): + return self + + def __exit__(self, *exc_info): + return False + + def command(self, command_type, **kwargs): + key = (command_type, tuple(sorted(kwargs.items()))) + if key not in FAKE_WS_RESULTS: + raise FakeHaWsError(f"unexpected command in test double: {key!r}") + return FAKE_WS_RESULTS[key] + + +fake_ha_ws = type(sys)("ha_ws") +fake_ha_ws.HaWsClient = FakeHaWsClient +fake_ha_ws.HaWsError = FakeHaWsError + +with tempfile.TemporaryDirectory() as storage_out_dir: + sys.modules["ha_ws"] = fake_ha_ws + report = import_api.new_report() + import_api.import_dashboards_and_registries( + "http://ha.example", "test-token", states, storage_out_dir, report + ) + del sys.modules["ha_ws"] + + check(report["websocket_error"] is None, "no websocket error with the fake client installed") + check(report["dashboards_imported"] == 2, "default dashboard + one storage-mode dashboard imported") + check( + report["dashboards_skipped"] == [("kiosk", "mode='yaml' is file-based, not reachable via api")], + "yaml-mode dashboard reported as skipped, not silently dropped", + ) + check(report["helpers_imported"] == {"input_boolean": 1}, "only the present input_* domain was queried") + + expected_files = { + "lovelace.yaml", + "lovelace.map.yaml", + "lovelace_dashboards.yaml", + "lovelace_resources.yaml", + "core.area_registry.yaml", + "core.entity_registry.yaml", + "input_boolean.yaml", + } + actual_files = set(os.listdir(storage_out_dir)) + check(actual_files == expected_files, f"expected storage-export files present, got {sorted(actual_files)}") + + # stale-file cleanup: a leftover domain export must be removed on re-run + stale_path = os.path.join(storage_out_dir, "input_number.yaml") + with open(stale_path, "w", encoding="utf-8") as f: + f.write("key: input_number\n") + sys.modules["ha_ws"] = fake_ha_ws + report2 = import_api.new_report() + import_api.import_dashboards_and_registries( + "http://ha.example", "test-token", states, storage_out_dir, report2 + ) + del sys.modules["ha_ws"] + check(not os.path.isfile(stale_path), "stale storage-export file removed on re-run (idempotent cleanup)") + + +# --- missing dependency path: no traceback, clear actionable message --- + +with tempfile.TemporaryDirectory() as storage_out_dir: + # Force the *real* ha_ws.py to execute (not a fake) and hit its own + # `import websocket as _ws` line, so we exercise the actual actionable + # ImportError message it raises rather than Python's generic one. + sys.modules["websocket"] = None + sys.modules.pop("ha_ws", None) + report = import_api.new_report() + import_api.import_dashboards_and_registries( + "http://ha.example", "test-token", states, storage_out_dir, report + ) + sys.modules.pop("ha_ws", None) + del sys.modules["websocket"] + + check(report["websocket_error"] is not None, "missing websocket-client dependency is reported, not raised") + check( + "websocket-client" in (report["websocket_error"] or ""), + "the reported error names the missing package so an operator knows what to install", + ) + check(os.listdir(storage_out_dir) == [], "no storage-export files written when the dependency is missing") + +sys.exit(1 if fail else 0) +PYEOF diff --git a/services/home-assistant/README.md b/services/home-assistant/README.md index 4e8fe5f..569a2f1 100644 --- a/services/home-assistant/README.md +++ b/services/home-assistant/README.md @@ -26,23 +26,77 @@ import only, never deploy), `chelsty-ha` (Tailscale, api adapter — see ```bash scripts/ha/import.sh ken +scripts/ha/import.sh ken-legacy ``` -Read-only: pulls `/config` from the instance, filters it through -`.gitignore`, normalizes and splits it into `config/ken/`, exports curated -`.storage/*` into `storage-export/ken/`, and (if a deploy token exists at -`~/.config/ha-deploy/ken.token`) writes a dated `/api/states` fixture. -Idempotent — re-running against an unchanged instance produces no diff. +Read-only, and idempotent for both adapters — re-running against an +unchanged instance produces no diff. -Only the `docker-exec` adapter is implemented; running `import.sh -chelsty-ha` today exits with a clear "not implemented" error (its `api` -adapter is out of scope for this skeleton). +**`docker-exec` adapter** (`ken-legacy`): pulls the whole `/config` tree +over `ssh ... docker exec ... tar`, filters it through `.gitignore`, +normalizes and splits it into `config//`, exports curated +`.storage/*` into `storage-export//`. + +**`api` adapter** (`ken`, `chelsty-ha`): for instances with no +filesystem/SSH access (HAOS has none). Pulls automations/scripts/scenes via +one REST GET per object (`/api/config//config/`) and +dashboards/area+entity registries/`input_*` helpers via the WebSocket API +(`/api/websocket`) — read-only commands only, nothing that mutates the live +instance. Full `/config` import is out of scope for this adapter; see +DESIGN.md, "Deploy path: adapter per instance". + +Both adapters also write a dated `/api/states` fixture snapshot to +`fixtures/` if a deploy token exists at +`~/.config/ha-deploy/.token`. For the `api` adapter, that token is +not optional — see "Tokens" below. + +### Tokens + +The `api` adapter has no filesystem fallback, so a missing or empty token +at `token_path` (see `instances.yaml`) is a hard error, not a soft-skip — +`import.sh` aborts immediately with a clear message rather than silently +producing an empty import. See `DESIGN.md`, "Tokens", for how the +`deploy_agent` account and its long-lived access token are provisioned. + +The token itself never touches a subprocess argv or a log line: REST calls +go through `scripts/ha/lib/ha_api.py`, which reads the token file itself and +sets the `Authorization` header in-process via `requests` (never `curl -H`, +which would put the token in that process's argv, visible to any local user +via `ps`). `requests` (`python3-requests`) is a very common preinstalled/ +transitive package on Debian-based nodes; if it's ever missing, `import.sh` +fails with Python's own `ModuleNotFoundError` — install it the same way as +`python3-websocket` below (`sudo apt install python3-requests` or `pip +install --user requests`). + +### WebSocket dependency + +The dashboard/registry/helper export (`scripts/ha/lib/ha_ws.py`) depends on +the `websocket-client` PyPI package (import name `websocket`) — not in the +stdlib, not installed by default. Install one of: + +```bash +sudo apt install python3-websocket # Debian/Ubuntu package name +# or +pip install --user websocket-client +``` + +If it's missing, `import.sh ken` still imports automations/scripts/scenes +(REST-only, no extra dependency needed) and then reports the +dashboards/registries/helpers step as skipped with an actionable message +naming the package to install — it does not crash with a raw traceback, and +it does not silently produce an incomplete `storage-export/` without saying +so. ## Tests ```bash scripts/ha/tests/test_split_normalize.sh +scripts/ha/tests/test_normalize_tags.sh +scripts/ha/tests/test_import_api_offline.sh ``` -Offline determinism check for the split+normalize pipeline — no network, -no HA instance required. +All offline — no network, no HA instance required. `test_import_api_offline.sh` +covers the `api` adapter: normalization of real-shaped API responses (saved +as fixtures under `tests/fixtures/`), idempotency, stale-file cleanup, and +the missing-`websocket-client` error path (via `sys.modules` injection, not +an actual network call).