QWEN × FISHDACRESEARCH NOTE / UPDATED 07 NOV 2026
APPROACH B · SINGLE-SPEAKER STUDY

A voice,
reconstructed.

One voice. Three model snapshots.
Compare a frozen Qwen decoder prefix with its unfrozen continuation, through the same FishDAC synthesis stack.

Enter the listening room ↗
PHIL BENSON / SPEAKER 6097Loading verified snapshot…
CHECKPOINT—Completed updates
TRAINING CORPUS29.95 h39,373 clean utterances · one speaker
OUTPUT SAMPLE RATE44.1 kHzOriginal amplitude · float WAV
LISTENING EXAMPLES6616 training + all 50 validation clips
THE SNAPSHOT

Loading measured results…

↓

NEW / COMPLETED 5,000-UPDATE ABLATION

Does unfreezing help?

Both branches started from the same full checkpoint at 73,179 and finished at 78,179. All 5,000 batches matched on every GPU. Only one branch unfroze the retained Qwen decoder pre-convolution, full 8-block transformer, and first ×2 upsampler + ConvNeXt (42.5M parameters; LR 1e-5). Encoder, codebooks and ASR teacher stayed frozen. Bridge, FishDAC and discriminator kept their existing learning rates and losses.

Held-out validation · all 50 clips · lower is better
MetricFrozen controlUnfrozen prefixUnfrozen vs frozen

The result is mixed: validation mel is nearly unchanged, while MR-STFT is slightly worse with unfreezing. This short single-seed test does not establish a perceptual improvement. Compare the audio below. The older 69,950 snapshot is preserved as a historical reference, not the matched control.

01 / LISTEN CLOSELY

The listening room

Matched utterances. No loudness normalization.
Switch signals or models at the same playback position.

HELD-OUT VALIDATION
SOURCE TRANSCRIPT
Loading…

Ground truth

A / ORIGINAL RECORDING↓ WAV
0:00 / 0:00

Reconstruction

B / MODEL OUTPUT↓ WAV
0:00 / 0:00

Reconstruction · 24 kHz

C / BAND-LIMITED CONTROL↓ WAV
0:00 / 0:00

What happens above 12 kHz?

The third player is the same reconstruction, anti-aliased and downsampled to 24 kHz—not another model. Compare all three mel spectrograms on identical time, frequency and color scales.

Ground truth, full-band reconstruction and 24-kHz reconstruction mel spectrograms, with shared scales and a dashed 12-kHz line
↓ Download spectrogramClick the image to inspect at full resolution ↗

Dashed line: 12-kHz Nyquist limit of the 24-kHz control. Color: fixed −100 to −20 dB; no per-panel normalization. High-band energy alone can be noise: fidelity requires agreement with the ground truth. The control is upsampled only for identical analysis axes; its audio download stays at 24 kHz.

Inspect the signal Waveforms, spectrum & source ↗

Welch power spectral density, 2,048-sample windows. Mint: ground truth; lavender: reconstruction; amber: 24-kHz control. All use the same axes; this is not a perceptual quality score.

Waveform envelopes share one peak scale across all three signals. No gain normalization or fitted time shift is applied.

Validation = official dev.clean, excluded from this fine-tuning run. Training examples are a fixed seeded panel, not the full training set. The 100 official test clips are not used here.

02 / FOLLOW THE FIT

A trajectory, not a verdict.

Unsmoothed, fixed-panel evaluation.
Lower is better for both reconstruction losses.

Train · fixed 16 Validation · all 50x = additional updates from the selected experiment’s parent

Mel-spectrogram loss

Multi-resolution STFT loss

Each point is a per-clip mean on the same 16 train / 50 validation utterances. These are evaluation losses—not minibatch training curves, MOS, or proof of perceptual equivalence. Use the audio to judge residual artifacts.

03 / WHAT YOU ARE HEARING

Frozen encoder.
Adaptable synthesis.

The full utterance passes through the frozen Qwen encoder and the retained beginning of its native decoder. That decoder prefix is frozen in the control and trainable in the new ablation. A trainable bridge then conditions FishDAC’s synthesis stack.

This is a single-speaker fitting study, not an unseen-speaker benchmark. Validation shares the speaker and source books with training. Upstream pretraining overlap has not been audited.
Retained Qwen path
Always-frozen encoder + RVQ → decoder pre-convolution → full 8-block pre-transformer → first ×2 upsampler + ConvNeXt.
The bridge
1,024 features @ 25 Hz → fixed rate conversion to ≈21.53 Hz → trainable adapter.
Trainable FishDAC
Both latent ×2 upsamplers + ConvNeXt, then the waveform decoder → 44.1 kHz. Fish post-transformer bypassed.
Fine-tuning
Warm-started from B step 10,950, including G/D optimizer state. Global batch 8; constant LR: bridge 2e-4, synthesis 5e-5, discriminator 2e-5. The original run was stopped at 73,179; both ablation branches then ran exactly 5,000 updates. Only the unfrozen branch adds Qwen decoder LR 1e-5.
Objective
Mel + MR-STFT + GAN + feature matching + frozen Qwen-ASR perceptual loss. Native gain, no data augmentation.
Evaluation
Fresh full-utterance inference from three fixed checkpoints. Each snapshot provides 66 matched original, full-band reconstruction and 24-kHz control triplets. Fresh losses verified against saved evaluation.

OPEN THE EVIDENCE

Reproducible by design.

Data: original Hi-Fi TTS / OpenSLR 109, NVIDIA et al., CC BY 4.0; speaker 6097, clean partition. Ground-truth WAVs preserve the original samples. Reconstructions are experimental model outputs. Original chapter recordings were MP3-sourced; 44.1 kHz does not imply pristine full-band originals.

↓ Metrics, samples & hashes JSON↓ Method & reproduction MD
Snapshot identity & limitations

Speaker selection used recording-quality proxies and development audio. No MOS study or blind listening study is claimed. This is a static report snapshot; both ablation branches are complete and training is stopped.