Voicebox by Meta

Voicebox stands as a generative AI model specializing in speech, exhibiting adaptability to tasks beyond its explicit training scope while maintaining state-of-the-art performance. In contrast to conventional speech synthesizers, its training leverages diverse, unstructured data, eliminating the necessity for meticulous input labeling.

Voicebox adopts a novel methodology known as Flow Matching, representing Meta's latest advancement in non-autoregressive generative models, facilitating highly non-deterministic mapping between text and speech.

Voicebox excels in producing superior-quality audio segments across a spectrum of styles and synthesizing speech in six languages. Moreover, it offers capabilities such as noise reduction, content refinement, style transformation, and varied sample creation.

A key advantage of Voicebox lies in its capacity to modify any segment of a given sample, extending beyond the mere modification of the audio clip's conclusion. This characteristic enhances its adaptability and suitability for applications like in-context text-to-speech synthesis, cross-lingual style adaptation, speech denoising and modification, and diverse speech sampling.

Notably, Voicebox surpasses existing state-of-the-art speech models in word error rate and audio similarity metrics. While public availability is currently restricted due to potential misuse concerns, Meta has disseminated audio samples and a research paper detailing its methodology and outcomes.

This generative AI breakthrough in speech is promising, with potential in facilitating communication and voice customization for virtual assistants.