wavtechyukky/pyshiro

The README gives good background. Now let me read the workflow scripts themselves (they have Japanese docstrings).

Now let me look at the notebooks and get a quick sense of the core package modules.

I have a complete picture now. Here’s the overview.

pyshiro — codebase overview

pyshiro is a Python reimplementation of SHIRO, a phoneme-to-speech forced alignment toolkit built on Hidden Semi-Markov Models (HSMM), specialized for Japanese singing voice. Given an audio file and its lyrics, it estimates where each phoneme begins and ends. It can also train .hsmm models from a labeled corpus and convert labels between formats.

The core library lives in pyshiro/ (~3k lines):

Module Role
model.py Load/save the .hsmm model format
features.py MFCC extraction (SHIRO/ciglet-compatible), 16kHz mono, 5ms frames
align.py Viterbi forced alignment (2-pass HMM→HSMM), plus realign_external_labels
labels.py Read/write .lab (HTK 100ns & seconds), Praat TextGrid, Audacity
phonemes.py Kana→phoneme conversion (hiragana lyrics → phoneme sequence)
train.py Corpus training (pyshiro-train), with HMM pretraining, DAEM, GMM splitting
untie.py Monophone → context-dependent triphone expansion

The pre-trained models sit in checkpoint/ and models/ (a git submodule).

The workflow/ directory

This directory is not library code — it’s the practical, user-facing annotation pipeline. It contains five numbered CLI scripts that form a linear stage-by-stage pipeline, plus two Jupyter notebooks that narrate and drive that pipeline. All documentation here is in Japanese.

The two pipelines

There are actually two distinct workflows bundled together:

A. Annotate-from-scratch (scripts 00→03, guided by annotation_guide.ipynb) — take raw audio + lyrics and produce phoneme labels.

B. Convert external labels (scripts 04, 05, guided by convert_labels.ipynb) — take labels made by another tool and re-align them to this model’s labeling conventions, then quality-check them.

The numbered scripts

00_resample.py — Converts WAV files to 16kHz mono (using scipy.signal.resample_poly with gcd-reduced up/down factors). pyshiro’s feature extraction assumes 16kHz mono, so this is the mandatory first step. Skippable if audio is already 16kHz mono.

01_cut.py — Splits each 16kHz song into phrase-sized segments. It detects silence via energy (RMS in dB per 10ms frame), then greedily builds segments that don’t cross long pauses and stay under a max duration (~15s). This matters because the README warns that long silent stretches make the HSMM search fail — you must feed it short phrases, not whole songs. It outputs:

  • segment .wav files
  • a {stem}_cuts.json recording each segment’s sample offset (used later by 03_merge)
  • optionally, empty .txt lyric templates named to match each segment

02_align.py — The core step. For each segment it converts the hand-written hiragana lyrics to phonemes, extracts MFCCs, builds a state sequence, and runs forced_align_2pass to get phoneme boundaries. Writes .lab/.TextGrid/.audacity. Runs segments in parallel via ProcessPoolExecutor. Between stages 02 and 03 the human manually corrects boundaries in Praat/Audacity/vLabeler.

03_merge.py — Merges the (hand-corrected) per-segment labels back into one full-song .lab, using the sample offsets in cuts.json. Offsets are applied as integer arithmetic in HTK 100ns units (10_000_000 // sample_rate per sample) so there’s no floating-point drift. It fills inter-segment gaps with pau and pads the tail out to the original song length.

04_convert_labels.py — The entry point for pipeline B (realign_external_labels). It re-labels external labels (e.g. from the Tohoku Kiritan DB) to match this model’s conventions — for instance, this model includes the closure/silence interval inside plosive/affricate consonant labels, while other DBs don’t. It finds reliable “anchors” (long vowels, long pauses), re-aligns the spans between anchors with the HSMM, and optionally energy-refines pau/consonant boundaries. --fix_transitions lets you fix only certain boundary types (e.g. vowel→consonant) while keeping the rest of the original labels. (Note: this file’s docstring header still says 05_convert_labels.py — a stale rename artifact.)

05_check_consonant_outliers.py — A QC tool. Within each consonant type it flags duration outliers (via IQR, with a larger multiplier for plosives/affricates since their closures make durations bimodal) and renders the flagged spots as a grid of waveform+label plots. Built on the heuristic that a too-short consonant is probably a mislabel; in practice you look at the short side (--side short).

The notebooks

  • annotation_guide.ipynb — Walks a user through pipeline A end to end (resample → cut → write lyrics by hand → align → correct by hand → merge), with a single config cell to edit and visualization cells. (Its steps are labeled 0,1,2,3,5 — the manual-correction step sits between 3 and 5.)
  • convert_labels.ipynb — Explains the labeling conventions of the bundled model (vowel ends set slightly before the waveform dies; plosive closures included in the consonant), the anchor-based re-alignment concept (with a diagram), and the recommended --fix_transitions vowel-consonant,silence-consonant,vowel-silence setting, ending with the outlier check.

The key data-flow contract to understand: 01_cut.py writes cuts.json, and 03_merge.py consumes it — that JSON is what lets per-segment alignment be stitched back to absolute song time without floating-point error.