2026-06-09 14:57:13 +02:00
|
|
|
# ipin-vr
|
|
|
|
|
|
|
|
|
|
Generator prostych, **gramatycznie poprawnych** polskich poleceń do terapii afazji
|
|
|
|
|
(np. *„Połóż jabłko na stole"*) wraz z wypowiadaniem ich przez Piper TTS.
|
|
|
|
|
|
|
|
|
|
Docelowo narzędzie ma działać w trybie mieszanej rzeczywistości na Meta Quest 3
|
|
|
|
|
(offline). **Etap 1** (ten kod) realizuje wyłącznie rdzeń: generowanie poleceń i ich
|
|
|
|
|
wypowiadanie na PC. Bez rozpoznawania mowy (STT), bez VR.
|
|
|
|
|
|
|
|
|
|
## Dokumentacja
|
|
|
|
|
- [`docs/SPEC.md`](docs/SPEC.md) — specyfikacja techniczna etapu 1 (cel buildu).
|
|
|
|
|
- [`docs/LEXICON.md`](docs/LEXICON.md) — zweryfikowany leksykon (formy gramatyczne).
|
|
|
|
|
- [`docs/ROADMAP.md`](docs/ROADMAP.md) — etapy i ograniczenia platformy docelowej.
|
Add export subcommand and Unity PoC-1 scaffold (ONDEVICE §4)
Python (ipin_vr/export.py):
- generate_bank(): levels 1–2 enumerated exhaustively (120/720 combos),
level 3 random with dedup; all deduplicated per-level
- run_export(): CLI --per-level / --out / --seed flags per ONDEVICE §4.1
- cli.py routes "export" subcommand before session args (backward-compat)
- 14 new tests: format, dedup for all 3 levels, space caps (L1=120, L2=720),
seed determinism, file output
Unity scaffold (ondevice/Scripts/):
- CommandBank.cs: loads commands.json from StreamingAssets (WebRequest on Android)
- TtsManager.cs: sherpa-onnx integration with [SHERPA] stubs, StreamingAssets→
persistentDataPath copy, defensive Speak() with onDone callback
- SessionController.cs: passthrough flow §6, level switching, busy guard on Next
- ondevice/README.md: full manual Editor steps §5 (Unity setup, Meta XR SDK,
sherpa-onnx install, scene wiring, font, build, metrics)
pytest: 37/37 passed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:11:53 +02:00
|
|
|
- [`docs/ONDEVICE.md`](docs/ONDEVICE.md) — PoC-1 na Meta Quest 3 (Unity + sherpa-onnx).
|
|
|
|
|
- [`ondevice/README.md`](ondevice/README.md) — ręczne kroki w Unity Editorze.
|
2026-06-09 14:57:13 +02:00
|
|
|
|
|
|
|
|
## Zasada naczelna
|
|
|
|
|
Poprawność fleksji jest nienegocjowalna. Wszystkie formy słów pochodzą wyłącznie
|
|
|
|
|
z ręcznie zweryfikowanego leksykonu; model językowy (gdy włączony) wybiera tylko
|
|
|
|
|
identyfikatory elementów, a nie generuje polskiego tekstu.
|
|
|
|
|
|
Implement Stage 1: command generator, CLI, TTS and LLM clients
- lexicon.py: exact forms from docs/LEXICON.md (Polish diacritics preserved)
- generator.py: Clause/Command dataclasses, render(), random_command(level 1-3)
- llm.py: OpenAI-compatible client via urllib, JSON-only contract, fallback to random
- tts.py: Piper via subprocess, defensive (prints [TTS off: reason] on any failure)
- cli.py: argparse interface per SPEC §9
- tests/test_generator.py: 23 tests covering all genders, locations, level-3 joining,
capitalisation; all pass
DoD verified: pytest 23/23, python -m ipin_vr --level 3 --count 6 --no-audio OK.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 15:03:50 +02:00
|
|
|
## Szybki start (bez audio)
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
python -m ipin_vr --level 3 --count 6 --no-audio
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Przykładowy wynik:
|
|
|
|
|
```
|
|
|
|
|
1. Połóż zieloną książkę na półce i umieść klucz na biurku.
|
|
|
|
|
2. Umieść żółtą piłkę na parapecie i połóż kubek na stole.
|
|
|
|
|
...
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Dostępne opcje CLI:
|
|
|
|
|
|
|
|
|
|
| Flaga | Domyślnie | Opis |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| `--level {1,2,3}` | 1 | poziom trudności |
|
|
|
|
|
| `--count N` | 5 | liczba poleceń |
|
|
|
|
|
| `--delay S` | 4.0 | pauza między poleceniami (s) |
|
|
|
|
|
| `--voice PATH` | — | ścieżka do modelu Piper `.onnx` |
|
|
|
|
|
| `--no-audio` | — | wyłącza TTS |
|
|
|
|
|
| `--llm` | — | użyj LLM do wyboru poleceń |
|
|
|
|
|
| `--llm-endpoint URL` | localhost:8080 | endpoint llama.cpp |
|
|
|
|
|
| `--llm-model NAME` | `local` | nazwa modelu |
|
|
|
|
|
| `--seed N` | — | deterministyczny generator |
|
|
|
|
|
|
|
|
|
|
## Audio (Piper TTS)
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
pip install piper-tts
|
|
|
|
|
# pobierz głos polski, np. ze https://huggingface.co/rhasspy/piper-voices
|
|
|
|
|
python -m ipin_vr --level 2 --count 5 --voice pl_PL-gosia-medium.onnx
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## LLM (llama.cpp, opcjonalny)
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
# uruchom lokalnie serwer llama.cpp na porcie 8080
|
|
|
|
|
python -m ipin_vr --level 3 --count 5 --no-audio --llm
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
LLM zwraca wyłącznie identyfikatory elementów; gramatykę buduje zawsze leksykon.
|
|
|
|
|
Przy błędzie (timeout, zły JSON, nieznany identyfikator) — fallback na generator losowy.
|
|
|
|
|
|
Add export subcommand and Unity PoC-1 scaffold (ONDEVICE §4)
Python (ipin_vr/export.py):
- generate_bank(): levels 1–2 enumerated exhaustively (120/720 combos),
level 3 random with dedup; all deduplicated per-level
- run_export(): CLI --per-level / --out / --seed flags per ONDEVICE §4.1
- cli.py routes "export" subcommand before session args (backward-compat)
- 14 new tests: format, dedup for all 3 levels, space caps (L1=120, L2=720),
seed determinism, file output
Unity scaffold (ondevice/Scripts/):
- CommandBank.cs: loads commands.json from StreamingAssets (WebRequest on Android)
- TtsManager.cs: sherpa-onnx integration with [SHERPA] stubs, StreamingAssets→
persistentDataPath copy, defensive Speak() with onDone callback
- SessionController.cs: passthrough flow §6, level switching, busy guard on Next
- ondevice/README.md: full manual Editor steps §5 (Unity setup, Meta XR SDK,
sherpa-onnx install, scene wiring, font, build, metrics)
pytest: 37/37 passed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:11:53 +02:00
|
|
|
## Eksport banku poleceń (na urządzenie)
|
|
|
|
|
|
|
|
|
|
Generuje gotowe, deduplikowane polecenia do JSON — używane przez aplikację Unity
|
|
|
|
|
na Quest 3 (PoC-1, `docs/ONDEVICE.md`):
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
python -m ipin_vr export --per-level 300 --out commands.json [--seed N]
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
| Flaga | Domyślnie | Opis |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| `--per-level N` | 300 | liczba unikatów na poziom |
|
|
|
|
|
| `--out PATH` | `-` (stdout) | ścieżka pliku wyjściowego |
|
|
|
|
|
| `--seed N` | — | powtarzalny wynik |
|
|
|
|
|
|
|
|
|
|
Format wyjściowy:
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"meta": {"version": "1", "per_level": 300},
|
|
|
|
|
"commands": [
|
|
|
|
|
{"level": 1, "text": "Połóż jabłko na stole."},
|
|
|
|
|
{"level": 2, "text": "Umieść niebieskie jabłko na stole."},
|
|
|
|
|
{"level": 3, "text": "Połóż zieloną książkę na półce i umieść klucz na biurku."}
|
|
|
|
|
]
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Przestrzeń każdego poziomu: L1 = 120 unikatów, L2 = 720, L3 ≈ 635 000.
|
|
|
|
|
Jeśli `--per-level` przekracza dostępną przestrzeń, zwracane jest tyle ile możliwe.
|
|
|
|
|
|
Implement Stage 1: command generator, CLI, TTS and LLM clients
- lexicon.py: exact forms from docs/LEXICON.md (Polish diacritics preserved)
- generator.py: Clause/Command dataclasses, render(), random_command(level 1-3)
- llm.py: OpenAI-compatible client via urllib, JSON-only contract, fallback to random
- tts.py: Piper via subprocess, defensive (prints [TTS off: reason] on any failure)
- cli.py: argparse interface per SPEC §9
- tests/test_generator.py: 23 tests covering all genders, locations, level-3 joining,
capitalisation; all pass
DoD verified: pytest 23/23, python -m ipin_vr --level 3 --count 6 --no-audio OK.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 15:03:50 +02:00
|
|
|
## Testy
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
pytest
|
|
|
|
|
```
|
|
|
|
|
|
2026-06-09 14:57:13 +02:00
|
|
|
## Status
|
Add export subcommand and Unity PoC-1 scaffold (ONDEVICE §4)
Python (ipin_vr/export.py):
- generate_bank(): levels 1–2 enumerated exhaustively (120/720 combos),
level 3 random with dedup; all deduplicated per-level
- run_export(): CLI --per-level / --out / --seed flags per ONDEVICE §4.1
- cli.py routes "export" subcommand before session args (backward-compat)
- 14 new tests: format, dedup for all 3 levels, space caps (L1=120, L2=720),
seed determinism, file output
Unity scaffold (ondevice/Scripts/):
- CommandBank.cs: loads commands.json from StreamingAssets (WebRequest on Android)
- TtsManager.cs: sherpa-onnx integration with [SHERPA] stubs, StreamingAssets→
persistentDataPath copy, defensive Speak() with onDone callback
- SessionController.cs: passthrough flow §6, level switching, busy guard on Next
- ondevice/README.md: full manual Editor steps §5 (Unity setup, Meta XR SDK,
sherpa-onnx install, scene wiring, font, build, metrics)
pytest: 37/37 passed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 16:11:53 +02:00
|
|
|
Etap 1 zrealizowany wg `docs/SPEC.md`. PoC-1 (Quest 3): skrypty C# w `ondevice/`.
|