Stability AI, the creators of the Stable Diffusion Model, have introduced another product called Stable Audio. This new product can produce audio from text descriptions. In this article, we will also discuss a new framework named Medusa that can speed up language model generation.
Stable Audio is an impressive advancement in AI music generation. Unlike traditional music generation techniques that use MIDI files, Stable Audio uses raw audio samples to create any sound, including musical instruments, human voices, sound effects, and background noises. This approach ensures high sound quality and captures the expressive qualities of music.
MIDI files, which are commonly used in music generation, only provide instructions for playing notes and lack the ability to capture the quality and character of the instruments’ sound. They also tend to be repetitive and struggle with aspects like chords, harmony, melody, rhythm, and structure.
Stable Audio overcomes these limitations by using raw audio samples and a method called contrastive language audio pre-training (CLAP). CLAP links language with audio by pairing words with their corresponding sounds. Stability AI trained Stable Audio using a vast dataset from the Audio Sparks Library, which includes detailed information about music tracks.
Medusa is a framework that speeds up language model generation, such as GPT-4 and CLAUDE-2, by using multiple decoding heads. This technique allows for parallel generation of text, reducing the time and resources required. Medusa also incorporates innovative features like tree attention and typical acceptance, which improve efficiency and adaptability.
Medusa has been shown to be up to four times faster than traditional decoding methods without sacrificing quality. The optimal configurations and thresholds for Medusa may vary depending on factors like model size, input length, sampling temperature, and hardware details.
If you are interested in trying Stable Audio or Medusa, you can visit their website or GitHub repository for more information and instructions. These advancements in AI music generation open up new possibilities for creating and discovering music and sounds using natural language.
In conclusion, Stable Audio and Medusa are exciting advancements in AI music generation. Stable Audio’s use of raw audio samples and CLAP allows for high-quality and expressive music generation. Medusa’s multiple decoding heads and innovative features improve the efficiency and performance of language model generation. These advancements empower users to explore their musical ideas and preferences using natural language.
