LyricJam, a real-time system that uses artificial intelligence to generate lyric lines for live instrumental music
Still image from Sound experiments with LyricJam and Resonant Garden
Music artists can find inspiration and new creative directions for their song writing with technology developed by Waterloo researchers.
LyricJam, a real-time system that uses artificial intelligence (AI) to generate lyric lines for live instrumental music, was created by members of the University’s Natural Language Processing Lab.
The lab, led by Olga Vechtomova, a Waterloo Engineering professor cross-appointed in Computer Science, has been researching creative applications of AI for several years.
The lab’s initial work led to the creation of a system that learns musical expressions of artists and generates lyrics in their style.
Recently, Vechtomova, along with Waterloo graduate students Gaurav Sahu and Dhruv Kumar, developed technology that relies on various aspects of music such as chord progressions, tempo and instrumentation to synthesize lyrics reflecting the mood and emotions expressed by live music.
As a musician or a band plays instrumental music, the system continuously receives the raw audio clips, which the neural network processes to generate new lyric lines. The artists can then use the lines to compose their own song lyrics.
“The purpose of the system is not to write a song for the artist,” Vechtomova explains. “Instead, we want to help artists realize their own creativity. The system generates poetic lines with new metaphors and expressions, potentially leading the artists in creative directions that they haven’t explored before.”
The neural network designed by the researchers learns what lyrical themes, words and stylistic devices are associated with different aspects of music captured in each audio clip.
For example, the researchers observed that lyrics generated for ambient music are very different than those for upbeat music.
The research team conducted a user study, inviting musicians to play live instruments while using the system.
Read the full article here
An article by the University of Waterloo