Study50 voices · 20 generations →
50 different AI voices, each one copied from its own last copy, over and over — a game of telephone played entirely by an AI. Here's what changed about how they sound, and why.
Listen to this page — narrated
One voice, twenty generations of copying
left = clone #1 · right = clone #20
This is zero-shot voice cloning: instead of training a new AI model on a voice, you hand an existing AI a short clip and say "talk like this." No training, just an example to imitate. Here's the twist we tested:
Start with a real voice
One of 50 existing AI voice models on Fish Audio.
Make it speak
Generate one short original clip — this is "clone #1."
Feed that clip back in
Instead of the real voice again, hand the AI clone #1's own audio as the new example to imitate.
Repeat, 20 times
Each clone is built only from the one before it. The real voice is only used once, at the very start.
Nothing here trains or changes the underlying AI voice model itself — every generation is a fresh, independent "sound like this clip" request. We're testing how far a copy-of-a-copy can drift, not whether the model is being retrained.
Scale. 50 voice models (the 50 most-used English models in Fish Audio's public catalogue, not curated for quality) × 20 generations × 3 formats (mp3, wav, raw pcm) = 3,000 generations, all on the free s2.1-pro-free backend at $0 spend. Every chart above the format-comparison section is the mp3 run.
Text protocol. Each run's script is freshly sampled from a pool of 35 filler sentences — not the same sentence repeated. This is deliberate: identical text run after run risks a cached or memoized response, and it lets us hold word count (not content) constant. The mp3, wav and pcm chains reuse the exact same per-run text as each other (verified against the manifest, 0 mismatches across all three), so the format comparison is apples-to-apples; within any single chain, generation 1's words and generation 20's words are different, matched only in count.
What's not validated. All 11 metrics behind the composite index are signal-processing proxies — spectral centroid, noise floor, zero-crossing rate, and so on — not perceptual ratings. Nobody has run a listener study (MOS-style) to confirm the composite tracks what a human would call "worse." A word-error-rate pass (transcribe every clip, diff against the requested script) would settle intelligibility directly; it hasn't been run yet, mainly because it costs API calls and this whole study was built around $0 spend.
Statistical caveats. Every metric gets its own hypothesis test (per-voice OLS, population mixed-effects, and a piecewise/AIC breakpoint search) with no multiple-comparisons correction across the 11 tests — most p-values are small enough it likely doesn't change the conclusions, but it isn't stated elsewhere on this page, so it's stated here. Each format is one chain per voice with no repeats, which is enough to show all three formats land within ~3.5% of each other on overall drift, but not enough to fully separate a real (small) format effect from ordinary chain-to-chain randomness — see the noise-floor discussion in the format section below for exactly where that bites.
Generalization. Only the free s2.1-pro-free backend was tested. Fish Audio's paid tiers may behave differently; nothing here speaks to that.
We compared every single clone back to its own original — combining brightness, background hiss, loudness, and more into one "drift score." The line below is how far the average voice has drifted, generation by generation.
Drift from the original voice
0 = sounds identical to the original · higher = sounds more different
By clone #20, the average voice has drifted noticeably — but almost half of that drift already happens in the first 5 clones. After that, it keeps sliding, just more gradually.
This isn't the steadiest voice in the test, and it isn't the worst — it's Adrian, the single closest voice to the exact median drift score (0.52 by clone #20). Five checkpoints from its chain, in order.
clone #1
clone #5
clone #10
clone #15
clone #20
Composite drift score, mp3 run · n = 50 voices × 20 generations = 1,000 clips analyzed. Positive = more degraded, in average standard deviations across 11 signal-level inputs; 0 at run 1 by construction.
| test | result |
|---|---|
| per-voice OLS | mean slope +0.0277/run (sd 0.021), t=9.30, p<0.0001, n=50 |
| mixed-effects (pop.) | run coefficient +0.0277/run, p<0.0001 |
| best-fit breakpoint | run 5 (kink p<0.0001) |
| piecewise vs. linear | AIC −426.7 vs. −360.8 — piecewise fits meaningfully better |
The composite equally weights 11 z-scored deltas (brightness, noise floor, silence ratio, clipping, RMS, peak, zero-crossing rate, spectral rolloff, duration, pace, and their signs). That weighting isn't validated against human listener ratings — nobody has run a MOS-style test to check whether this index tracks perceived quality, only whether the underlying acoustic measurements move. Treat "0.52 drift" as a summary of measurable change, not a perceptual score.
The drift score above is a blend. Two of its biggest ingredients behave completely differently from each other — which tells us something about what's really going on.
Background hiss, compared to the original clip
how much noisier each clone sounds, in decibels
This one doesn't level off. By clone #20, the typical voice's background noise has risen by about 21 decibels — roughly the jump from a quiet room to a noisy café. Every generation seems to add a little more hiss on top of the last.
The extreme case: Verity went from −94 dB of noise floor at clone #1 to −38 dB by clone #20 — a 56 dB jump, more than double the average. Clone #1 is essentially silent behind the voice; clone #20 has an audible hiss under every word.
clone #1
clone #20
| metric | per-voice OLS | mixed-effects | breakpoint |
|---|---|---|---|
| noise floor (dB) | +0.852/run (sd 0.745), t=8.08, p<0.0001 | p<0.0001 | run 4 · AIC 6191.3 vs. 6266.3 |
| brightness (Hz) | t=4.41, p=0.0001 | p=0.0001 | run 10 · AIC 13864.1 vs. 14197.1 |
Noise floor is the one metric with an early breakpoint (run 4) that still keeps climbing after it — it doesn't level off the way brightness does. All figures above are on the sign-flipped "positive = more degraded" convention this study uses throughout, so they read consistently in the same direction as the composite index; raw units and sign move opposite ways for brightness (rising score = falling Hz).
Unresolved confound: each generation's reference audio is the previous generation's compressed mp3, not the raw waveform, so codec re-encoding is baked into the same chain as any real model drift. The wav/pcm reruns below test whether the file format changes the outcome — they don't test whether repeated re-encoding of any format compounds differently than a single encode would.
Voices get steadily duller through about clone #12 — then this measurement swings back the other way.
We don't think that's a real recovery. Clips are also ballooning in length around this same point (see below), and longer clips get measured differently by this metric — it's very likely a side effect of the rambling, not the voice actually sounding better again.
Clip length tells a stranger story than either chart above. For the first ten or so clones, they actually got a little shorter and tighter. Then, without warning, a growing number of them didn't stop getting longer.
Average clip length, saying the same amount of text
shaded band shows how spread out voices are at each generation
It isn't just teto. Elon Musk (noise reduction) ran the exact same experiment — 34 seconds at clone #1, 2 minutes 23 seconds at clone #20 — from a completely different chain.
clone #1
clone #20
Every run's script is freshly sampled from a pool of 35 filler sentences, so no two generations say identical words — the "same sentence" language on this page (and in the narration) is loose shorthand for something more specific. What's actually held constant is the word budget: 9 sentences per run, every time. Across all 50 voices, mean word count is 108.7 at run 1 and 108.3 at run 20 (sd ≈4.5 both ends) — statistically flat. So this rules out "got a longer script by chance" as the explanation for runaway clips.
What isn't flat: seconds spent per word.
| run 1 | run 20 | |
|---|---|---|
| mean words | 108.7 | 108.3 |
| mean sec/word | 0.381 | 0.725 |
| median sec/word | 0.379 | 0.733 |
Seconds-per-word nearly doubles on an unchanged word count. That's the model itself slowing down and padding — not a longer prompt. Duration overall: per-voice OLS mean slope +2.128 s/run (sd 2.328), t=6.46, p<0.0001, n=50; mixed-effects population coefficient p<0.0001; best-fit breakpoint at run 10 (AIC 9030.7 piecewise vs. 9210.2 linear). Silence ratio: OLS mean slope −0.00433/run, t=−6.59, p<0.0001 — falls, not rises, which is the ambiguous case the paragraph above already flags.
While the clips get longer, the gaps inside them get scarcer. At clone #1, about 16% of an average clip is silence — the breaths and beats between phrases. By clone #20 that's down to 8%, and it fell for 44 of the 50 voices.
Less silence isn't automatically "worse" — and here it's genuinely ambiguous. Because the clips are simultaneously ballooning in length, the same handful of pauses gets diluted across far more speech, which would push this number down on its own. We can't separate "the voice stopped pausing" from "there's just more talking around the same pauses."
Two more measurements go somewhere the hiss-and-brightness charts don't. Neither has a "good" direction, so both are drawn as distance travelled from the original — drifting faster and drifting slower both count as drifting.
How far each clone's speaking pace has moved
words per second, distance from that voice's own clone #1 — either direction
The average voice starts out at about 2.7 words a second. By clone #20 it has moved 1.29 words a second away from wherever it started — roughly half its original pace, in one direction or the other. And it really is both directions: 33 of the 50 voices ended up slower, 17 ended up faster. There's no single "voices get draggy" story here; they just stop holding their own tempo.
Here's the "faster" direction: Slax went from 2.3 words/sec at clone #1 to 3.8 words/sec at clone #20 — a comparable-length script, said nearly 65% quicker, with the clip itself shrinking from 47 seconds to 26.
clone #1
clone #20
The chart above scores pace on distance travelled from clone #1, always ≥0 by construction: per-voice OLS mean slope +0.0631/run, t=16.54, p<0.0001, n=50; mixed-effects population coefficient p=0.15; best-fit breakpoint run 13 (AIC 1108.9 piecewise vs. 1162.1 linear).
A separate, stricter pass on this dataset — fitting signed words/sec (not distance) with a mixed model that lets each voice have its own random slope, not just its own random intercept — found something worth stating plainly: a naive per-voice t-test on 50 slopes gives p=0.0002 ("voices reliably slow down"), but the random-slope model gives p=0.64 for the exact same coefficient. The reason is the slope's own voice-to-voice variance (0.327) is large relative to the effect being claimed — Adrian speeds up +0.070 words/sec/run, teto slows −0.150, and there's no shared population direction underneath, just fifty voices disagreeing with each other. That's the same qualitative conclusion this page draws from "33 slower, 17 faster" — the rigorous version of it lives in the linked technical writeup below.
Loudness metrics (no principled "better" direction, scored as |delta| like pace): RMS mean slope +0.00236/run, t=4.98, p<0.0001; peak amplitude mean slope +0.0116/run, t=7.80, p<0.0001, breakpoint at run 8.
Almost everything on this page gets monotonically worse. This doesn't. Peak level drifts hard for the first dozen clones, tops out at 0.33 around clone #14 — a third of the entire 0–1 scale — and then stops, easing back to 0.27 by clone #20. It looks like the model settles into a preferred loudness and stays there, rather than wandering forever.
The drift score isn't the same for every voice — not even close. The steadiest voice in the test finished twenty times closer to its original than the shakiest one did.
| Voice | Drift | Relative |
|---|---|---|
| Held up best | ||
| Donald J. Trump (noise reduction) | 0.09 | |
| jjk narrator | 0.15 | |
| Super Smash Bros. announcer | 0.17 | |
| Fell apart worst | ||
| Mortal Kombat | 1.23 | |
| E-girl | 1.44 | |
| teto | 1.89 | |
Hear it for yourself — clone #1 against clone #20, for every voice in the table above.
clone #1
clone #20
clone #1
clone #20
clone #1
clone #20
clone #1
clone #20
clone #1
clone #20
clone #1
clone #20
Every number on this page is a measurement, and measurements can be argued with. So here is the raw audio for teto — the voice at the bottom of that table, the one that drifted further than any of the other 49.
It doesn't come apart gradually. Through fourteen generations teto actually gets tighter, from 36 seconds down to 25. Then, in a single step, generation 15 runs to 1 minute 44. Its drift score does exactly the same thing — 0.30 after fourteen generations, 1.84 after fifteen. That one step is five times everything the previous fourteen added up to.
teto's own clip length, generation by generation
this one voice — not the 50-voice average
mp3 · clone #1
mp3 · clone #5
mp3 · clone #14last one before the jump
mp3 · clone #15the cliff
mp3 · clone #20at the ceiling
pcm · clone #1
pcm · clone #20
wav · clone #1
wav · clone #20
This is not the same event as the turn on the clip-length chart further up. That line is the 50-voice average, and it bends gradually from about generation 11 because different voices break at different moments. teto doesn't bend — it holds, then snaps. The average is smooth because it's an average.
After generation 15 it stops growing, but not because the voice recovered. Runs 16 through 20 all land within a third of a second of 143.5s, and across all 3,000 clips in this study not one exceeds 144 seconds. That's a hard ceiling on how much audio the API will return — so teto almost certainly had more to say and simply ran out of room.
The two rows below are the same voice and the same script, run as two more separate chains — one wav, one raw pcm. All three land somewhere different: 1.89 drift on the mp3 chain, 0.99 on the pcm chain, 0.81 on the wav chain. That spread is not a format effect. Across all 50 voices the three runs finish at 0.62, 0.61 and 0.60 — close enough to call identical. What you're hearing is how far a single chain's luck can carry one voice, three separate times — the same point the noise-floor charts make below, only audible.
Every clip on this page, mp3 and wav alike, is the untouched original file from the study — nothing here is transcoded or re-compressed for playback.
Every clip so far was saved as a compressed mp3, and each new clone was built from the previous clone's own compressed file — not a fresh recording. A little information gets thrown away every time audio is saved as mp3, and feeding a compressed file back in as the source for the next compressed file, twenty times in a row, is a bit like photocopying a photocopy. So we reran the entire experiment twice more without compression — once as wav, once as raw pcm — to find out how much of the drift above is really the AI, and how much is the file format quietly piling up.
wav and pcm are the same uncompressed audio; wav just adds a small file header. For this test they aren't two different treatments — they're the same treatment run twice. That turns out to be the most useful thing in this section.
Drift score, all three formats
same 50 voices, same script, same everything — only the file format changes
All three lines track each other almost the whole way. By clone #20 they finish at 0.62 (mp3), 0.61 (pcm) and 0.60 (wav) — a spread of about 3.5% between the highest and the lowest. Compressed or not, the voices fall apart by the same amount. Almost none of what's on this page is a compression artifact. It's the AI itself.
Here the three runs don't agree, and it's worth being careful about why. mp3 ends at +20.7dB and pcm at +20.2dB — close together — while wav ends lower, at +18.2dB.
An earlier version of this page read that as compression: mp3 slowly adding hiss that uncompressed audio avoids. The pcm run makes that hard to sustain. Pcm is uncompressed too, and it lands within 0.6dB of mp3 — in fact, for 17 of the 20 generations it was noisier than the compressed mp3 run, not quieter.
The model samples randomly on every single call, so two uncompressed chains are two different rolls of the dice, never a repeat. A difference that size is exactly what chain-to-chain randomness looks like.
So: we can't tell you whether mp3 adds hiss over a long clone chain. This study has one chain per format and no repeats, which is enough to show that all three formats drift by roughly the same overall amount, and not enough to separate a small format effect on noise from ordinary run-to-run variation. Settling that would take several independent chains per format, so the spread between runs can be measured rather than guessed at. That's a real limit of a 50-voice, one-chain-per-format test — not a detail we can round off.
Lined up at clone #20, no format is an outlier on anything that describes how far the voice itself travelled.
mp3 vs. wav vs. pcm at clone #20
each row is on its own scale — compare formats within a row, not between rows
On drift score the three runs sit inside a 0.02 band. On runaway length they span 40% to 48% of voices — four voices out of fifty, and the two uncompressed runs are themselves two voices apart. And on background noise the two uncompressed runs disagree with each other nearly as much as either disagrees with mp3. Whatever the file format is doing to these voices, it's smaller than the difference between one run and the next.