Meta unveils AudioCraft generative AI tool for music creation

Meta unveiled the AudioCraft AI tool that generates high-quality, realistic audio and music from text. AudioCraft consists of three models: MusicGen, AudioGen and EnCodec. MusicGen, which was trained with Meta-owned and specifically licensed music, generates music from text prompts, while AudioGen, which was trained on public sound effects, generates audio from text prompts. Meta has now released an improved version of EnCodec decoder, which allows higher quality music generation with fewer artifacts. 

The company is also releasing pre-trained AudioGen models, which let you generate environmental sounds and sound effects like a dog barking, cars honking, or footsteps on a wooden floor. And lastly, the company is sharing all of the AudioCraft model weights and code.

Meta is open-sourcing these models, giving researchers and practitioners access so they can train their own models with their own datasets for the first time, and help advance the field of AI-generated audio and music.

Generating high-fidelity audio of any kind requires modeling complex signals and patterns at varying scales. Music is arguably the most challenging type of audio to generate as it’s composed of local and long-range patterns, from a suite of notes to a global musical structure with multiple instruments. The AudioCraft family of models are capable of producing high-quality audio with long-term consistency, and they’re easy to use. 

AudioCraft works for music, sound, compression, and generation — all in the same place. Because it’s easy to build on and reuse, people who want to build better sound generators, compression algorithms, or music generators can do it all in the same code base and build on top of what others have done.