Meta, the company formerly known as Facebook, has pushed the boundaries of artificial intelligence in music creation with its latest release: MusicGen. This open-source AI tool has the unprecedented ability to create short pieces of music based on both text prompts and melodies. Using this technology, artists can generate unique compositions that align with specific style prompts and melodies.
A Closer Look at MusicGen’s Technology
MusicGen operates on a Transformer model, similar to most language models today. In the way a language model predicts subsequent characters in a sentence, MusicGen forecasts the next section in a musical piece.
This AI tool dissects audio data into smaller components utilizing Meta’s EnCodec audio tokenizer. As a single-stage model, MusicGen processes tokens in parallel, making it fast and efficient. For training, the team used 20,000 hours of licensed music, with an internal dataset of 10,000 high-quality tracks and music data from Shutterstock and Pond5.
Unique Feature: Handling Text and Music Prompts
MusicGen’s uniqueness stems from its ability to handle both text and music prompts. The text sets the basic style, aligning with the melody in the audio file. As a practical illustration, using the text prompt “a light and cheerful EDM track with syncopated drums, airy pads, and strong emotions, tempo: 130 BPM” along with the melody from Bach’s iconic “Toccata and Fugue in D Minor (BWV 565)” results in a novel composition.
However, one limitation is that the melody serves as a rough guideline and isn’t exactly mirrored in the output. The style orientation cannot be precisely controlled to hear a melody in different styles.
Comparing MusicGen and Other Models
When pitted against other music models like Riffusion, Mousai, MusicLM, and Noise2Music, MusicGen surpasses in both objective and subjective metrics testing how well the music matches the lyrics and the plausibility of the composition. Tests were run on three versions of MusicGen with different sizes: 300 million, 1.5 billion, and 3.3 billion parameters. Larger models yielded higher-quality audio. However, human raters rated the 1.5 billion parameter model best, while the 3.3 billion parameter model excelled in matching text input and audio output accurately.
Open Source Accessibility
In a move that further emphasizes its commitment to the open-source community, Meta has released the code and models for MusicGen on Github, permitting commercial use. A demo is also available on Huggingface.
Through the development of MusicGen, Meta has pushed the boundaries of AI and music. The technology holds immense potential for artists and producers, offering them a unique tool to aid in the creative process. It will be fascinating to see how MusicGen continues to evolve and influence the world of music creation in the years to come.
