Differentiable Articulatory Copy-Synthesis
of Biphonic Singing

Sygyt is a Tuvan style of biphonic singing in which a low vocal drone is sustained while a high harmonic is selectively amplified in the 1–3 kHz region. Copy-synthesizing this effect remains challenging for articulatory models, since it requires fine control of narrowly focused resonances that standard low-dimensional tract parameterizations cannot easily reproduce.

We address this problem with a differentiable Kelly–Lochbaum waveguide augmented with a sublingual second source, cubic B-spline tract parameterization, and spatially varying learnable damping, optimized end-to-end by gradient descent from audio. On 20 segments from two independent sygyt datasets (5 singers, 10 pitches), the proposed model reduces log-spectral distance by 30–38% relative to an articulatory baseline, with the largest gains concentrated in the overtone region. Cepstral-envelope analysis further shows more accurate recovery of the merged formant structure characteristic of sygyt production. The model also outperforms a DDSP harmonic-plus-noise baseline with direct per-harmonic spectral control, suggesting that explicit acoustic structure is a useful inductive bias for overtone-singing copy-synthesis.

We compare three synthesis approaches across 20 segments from two datasets:

• Articulator chain — fixed glottis + waveguide (~19 DOFs/frame)
• DDSP baseline — harmonic-plus-noise neural synthesizer (~100k parameters)
• B-spline (ours) — differentiable waveguide with B-spline tract (86 DOFs/frame, 1100× fewer parameters than DDSP)

Video recording of a Tuvan singer demonstrating sygyt technique, with synchronized spectrogram overlay showing the characteristic amplified harmonic in the 1–3 kHz region.

Copy-synthesis results on 20 segments of Tuvan sygyt from two independent datasets. Objective metrics: mean ± std. PESQ (wideband MOS-LQO), CDPAM (learned perceptual distance, lower = closer), ViSQOL (audio-mode MOS-LQO, 1–5). Q1 (overall quality) and Q2 (harmonic similarity): subjective scores (mean ± 95% CI, 0–100, 20 listeners × 3 segments/dataset). Best in bold blue.

F3 — Scale (h8–h10)

G3 — Interval (h6→h7)

A3 — Arpeggio (h4–h6)

A3 — Interval (h5→h6)

A3 — Scale (h6–h8)

B♭3 — Interval (h6→h7)

B3 — Interval (h4→h5)

C4 — Arpeggio (h3–h5)

D4 — Glissando (h3→h8)

E♭4 — Arpeggio (h3–h5)

T1 — Take 3 Short

T2 — Take 1 Short A

T2 — Take 1 Short B

T2 — Take 1 Short C

T2 — Take 2 Short

T2 — Take 3 Short

T2 — Take 5 Short

T3 — Take 2 Short A

T3 — Take 2 Short B

T4 — Take 1 Short A

LSD (dB) across 10 HFA segments spanning pitches F3–E♭4. B-spline wins on every pitch.

Metrics for the 1–3 kHz overtone region across all 20 segments. SpCorr_OT: spectral correlation in 1–3 kHz; eR: Bergevin energy ratio (1–2 kHz / 0–8 kHz); S_ot: overtone salience (dB); HPR: harmonic prominence ratio. Best synthesis value (closest to target) in bold blue.

DDSP achieves the highest local spectral correlation (SpCorr_OT) because it fits each harmonic independently, but B-spline better preserves the overtone energy structure (eR, S_ot, HPR), which is closer to the target values and perceptually more important for the characteristic sygyt timbre.

2×2 factorial ablation (B-spline K_A=40, all 20 segments, 500 iterations). The sublingual source is the primary driver (ΔLSD ≈ 1.0 dB), with per-segment damping providing a secondary improvement (ΔLSD ≈ 0.14 dB). SpCorr_OT: spectral correlation in the 1–3 kHz overtone region.

Comparison with related approaches relevant to overtone singing synthesis. Phys. = physical vocal tract model; Diff. = differentiable (gradient-based); C-S = copy-synthesis from audio; Biph. = biphonic/overtone production. LSD on 10 HFA sygyt segments.

Protocol

We conducted a perceptual evaluation inspired by the MUSHRA methodology (ITU-R BS.1534-3). 20 listeners (3 expert, 2 trained, 15 novice; one monotone respondent excluded) evaluated 6 segments (3 per dataset) in a blind A/B/C comparison via a web interface. Each trial presented the reference recording followed by three unlabeled synthesized conditions. Listeners rated each condition on two 0–100 anchored scales:

• Q1 — Overall quality: How natural and similar to the reference does the synthesis sound overall?
• Q2 — Harmonic similarity: How closely does the whistled overtone match the reference in pitch, clarity, and loudness?

Scale anchors: Bad (0) – Poor (20) – Acceptable (40) – Good (60) – Very good (80) – Excellent (100). Presentation order and condition assignment were randomized per listener using a seeded PRNG. Two labeled practice trials preceded the test for familiarization.

Try the listening test yourself →

Example Trial (seg02 — HFA, F3 Scale)

Below is a reconstruction of one test trial with method labels revealed. In the actual test, conditions were labeled A/B/C in randomized order.

Aggregate Results (mean ± 95% CI)

B-spline and DDSP are both significantly preferred over the articulator chain (Friedman test, p < 10⁻⁴; pairwise Wilcoxon with Holm–Bonferroni: p < 0.001, Cliff’s δ > 0.7). The B-spline–DDSP difference is not statistically significant (p > 0.19). 20 listeners, 6 stimuli (3 per dataset), 0–100 anchored scales.

We apply the B-spline model to two sygyt segments extracted from real-time MRI recordings (MPILARS dataset), demonstrating generalization to a third independent data source. The fMRI data provides ground-truth vocal tract geometry for qualitative comparison with the learned B-spline profiles.

Vibrato Phonation — clip 21–23 (2.0 s, 126 frames)

MRI-Derived Vocal Tract Analysis

fMRI Mean. Mean vocal tract shape during vibrato sygyt production.

fMRI Variation. Temporal variation in tract shape across frames.

Spectrum Mean. Mean spectrum of the target vibrato recording.

Spectrum PCA. Principal components of spectral variation over time.

B-spline copy-synthesis results: spectrogram comparison and learned parameters.

Full 2D spectrogram of the vibrato segment. View full-resolution PDF (20 MB)

Stable Phonation — clip 24 (0.75 s, 47 frames)

MRI-Derived Vocal Tract Analysis

fMRI Mean. Mean vocal tract shape during stable sygyt production.

fMRI Variation. Temporal variation in tract shape across frames.

Spectrum Mean. Mean spectrum of the target stable recording.

Spectrum PCA. Principal components of spectral variation over time.

B-spline copy-synthesis results: spectrogram comparison and learned parameters.

Full 2D spectrogram of the stable segment. View full-resolution PDF (7.3 MB)