4 Commits

Include renames

Author	SHA1	Message	Date
fam	8c7210abeb	doc: W1 audio smoke summary ... Walkthrough of the three W1 commits, the 4090 result (50 steps in ~1s, loss 5.55 → 0.17), and the limitations to keep in mind before reading into the loss-down (LM is also random + tiny vocab, so the drop is mostly memorisation, not Whisper-Projector alignment — W2 freezes the LM specifically to test that). Includes the W2 hand-off checklist.	2026-05-05 22:40:57 +01:00
fam	3c1cc3302f	omni: W1 audio align smoke — synthetic dataset + 50-step script ... End-to-end smoke proving the audio path: wav -> WhisperEncoder (frozen) -> Projector -> prepend to text embeddings -> tiny d6 GPT (random init) -> CE loss on text only Pass criterion is a plain "loss drops by at least 0.5". On a 4090 the run finishes in ~1 s and goes 5.55 -> 0.17 over 50 steps, so the threshold has plenty of headroom against false positives. Two design calls worth keeping in mind: 1. Synthetic sine clips, not LibriSpeech. W1 is forward-path proof, not alignment quality, and a deterministic offline dataset means no network on the smoke path. data/audio_smoke/manifest.jsonl is the only thing committed; wavs are regenerated by audio_smoke_data.py and gitignored. W2 swaps in real LibriSpeech. 2. Standalone byte-level tokenizer (UTF-8 bytes + a single BOS, vocab=257). Avoids depending on a trained nanochat BPE — the d6 GPT is random anyway, so vocab choice doesn't matter for "does the gradient flow" smoke. W2 onwards uses the real BPE on a real base. Caveat documented in doc/todo.md: because the LM is also random and being trained, the loss-down here mostly reflects the LM memorising 5 short strings, not Whisper-Projector alignment. That's fine for proving plumbing; W2 freezes the LM so projector-only gradient is the only path to lower loss.	2026-05-05 22:39:20 +01:00
mochi	62642b805b	doc: prefer ModelScope for Whisper encoder weights (closes #4) ... W1 todo 里 audio.py 的 WhisperEncoder 之前写的是从 HF mirror 拉权重， 国内拉 HF（哪怕走 hf-mirror）经常被卡。改成首选 ModelScope（例如 iic/Whisper-large-v3 / iic/Whisper-small），HF mirror 留作 fallback。 infra 决定那条也顺手把 mirror 列表对齐到 pip / 模型权重 / HF 数据集 三条线，写清楚 modelscope 是模型权重首选。	2026-05-05 22:25:38 +01:00
fam	b585e07dc2	omni: CI smoke + docs + README preamble ... - .gitea/workflows/smoke.yml: gitea CI on ailab gpu runner (manual git clone since actions/checkout@v4 mis-resolves subpath gitea); injects WANDB_API_KEY + CI_RUN_TAG=smoke-$run_number - scripts/smoke.sh: in-place smoke (uv sync + 1 shard + tokenizer + d=6 50-step base_train); idempotent cache at /data/nanochat-smoke/ - doc/research_feasibility.md: voice-first multimodal feasibility study (mochi) - doc/todo.md: phase-by-phase roadmap (W1 Whisper smoke → W4 MVP) - README.md: omni preamble pointing at upstream nanochat README - .gitignore: exclude .claude/ runtime files	2026-05-05 22:21:31 +01:00