An important pioneer in AI music creation was François Pachet, who originally worked at the Université Pierre et Marie Curie in Paris before moving to the SONY Computer Science Laboratory to experiment with AI in music improvisation. Among other things, he created the first AI that could interact with jazz musicians on stage in real time, as well as the first AI-generated pop song, “Daddy’s Car”. François Pachet’s role as a pioneer in AI music creation is highlighted in this part of the series “AI in the Music Industry”.
AI in the Music Industry – Part 10: François Pachet: The Continuator, Flow Machines and “Daddy’s Car”
Let’s start with the weather forecast. Whether tomorrow will be rainy, stormy or sunny depends on today’s weather. Probabilities can be given for how the weather will develop. This stochastic process can be modelled using Markov chains. The Russian mathematician Andrei Andreyevich Markov realised at the end of the 19th century that future events could be predicted to some extent using probability theory and historical data. To prove this, he chose the verse novel ‘Eugene Onegin’ by Alexander Pushkin to show that the sequence of letters is not random but follows a stochastic process. He analysed the first 20,000 letters of the text according to the frequency of consonants and vowels, and counted 56.8 per cent consonants and 43.2 per cent vowels. Markow then calculated the probability of a consonant following a vowel and vice versa. He was thus able to show that the probability of a particular sequence of letters is not arbitrary, i.e. random, but that the result depends on the preceding letters. You might say that this is trivial, because Pushkin did not roll the dice for ‘Eugene Onegin’ with a cup full of letters, but there is an important insight in Markov’s work. Many events, such as the weather, are not random but depend on previous events that can be described by stochastic processes, i.e. Markov chains.[1]
Markow’s publication on this subject went unnoticed for a long time, but has experienced a renaissance, especially in the Internet age. Markov chains are used to detect SPAM emails, in automated online advertising, in financial mathematics to calculate price and interest rate movements and, last but not least, in music. In 1954, the Greek composer Iannis Xenakis composed stochastic pieces of music based on Markov chains.[2]
François Pachet also worked with Markov chains when he moved from the Université Pierre et Marie Curie to the SONY Computer Science Laboratory in Paris in 1997. It was here that the passionate musician began his research into AI and music. He will be jointly responsible for a total of 35 patents in this field, some of which can certainly be described as groundbreaking.[3] ‘The Continuator’ has probably attracted the most attention. It is an algorithm that can improvise with jazz musicians in a jam session. In 2010, Pachet described the Continuator in detail in a scientific article.[4] Pachet’s aim was to create an interactive musical system that could respond instantly to musical impulses and learn as the music progressed. Markov chains, which control a stochastic musical process that is not purely random, but reacts to given musical inputs such as rhythm, beat and harmony, but also recognises inaccuracies such as those that occur in live music-making, offered themselves for implementation. There have already been attempts to imitate jazz music using algorithms,[5] but programming a jazz algorithm that could play and improvise with human musicians in real time sounded more like science fiction. The experiment succeeded and jazz critics who were invited to a listening test were unable to tell the difference between the human and the AI interpreter.[6]
Another of Pachet’s projects was the ‘Flow Machine’, which, again based on the mathematical concept of Markov chains, uses learning processes to become familiar with an artist’s style, but then learns deviations that are intended to lead to new combinations by imposing restrictions on the programme. Pachet combined the bebop style of the great jazz saxophonist Charlie Parker with the serial compositions of Pierre Boulez to create new music on the border between jazz and serialism. The ‘Flow Machine’ can be used to combine all kinds of musical styles and create new works.[7] Sony continued the ‘Flow Machines’ project after Pachet’s departure in 2017. It is designed to help music creators compose new music or create new individual styles of music, as demonstrated on the company’s website with numerous YouTube videos.[8]
In 2016, with the help of the ‘Flow Machine’, Pachet composed what is probably the world’s first AI pop song, “Daddy’s Car”, in the style of the Beatles. The lyrics, however, were written by songwriter Benoît Carré. The end result, which can be heard in a YouTube video, sounds like a psychedelic mix of various Beatles hits.[9] Pachet went on to produce the first fully AI-produced pop album in music history, ‘Hello World’, with musician SKYGGE, financed with EU research funding. The ‘Hello World’ project website[10] explains that the title of the album refers to the initial assignment for the programming students, which was to write a programme that would output the text ‘Hello World’. A total of 15 tracks were generated by the AI, representing a variety of styles: “This album is the result of that story. 15 songs were created by artists using Flow Machines: composers, singers, musicians, producers, and sound engineers, in many musical genres (pop, electronic, ambient, and jazz). With this diversity of skills, we had a single objective: use these new technologies to create novel, interesting music, yet music that would please our ears. “[11] The album was released in 2018, following Pachet’s move to Spotify’s Creator Technology Research Lab (CTRL) in June 2017.
Endnotes
[1] A detailed biographical outline of Andrei Andreevich Markov and his work can be found in Spektrum der Wissenschaft, “Andrei Markov (1856-1922)”, June 1, 2016, accessed: 2024-04-07.
[2] Pascal Descroupet, 2005, “Iannis Xenakis: Metastaseis (1953/1955)”, in: Hanns-Werner Heister, Geschichte der Musik im 20. Jahrhundert, 1945-1975, Laaber: Laaber Verlag, pp 129-133.
[3] A short biography of Pachet can be found on his homepage: https://www.francoispachet.fr/biography-of-francois-pachet/, accessed: 2024-04-07.
[4] François Pachet, 2010, “The Continuator: Musical Interaction With Style”, Journal of New Music Research, vol 32(3), pp 333-341.
[5] See Geber Ramhalo & Jean-Gabriel Ganascia, 1993, “Simulating Creativity in Jazz Performance”, AAAI ’93 Workshop on Artificial Intelligence & Creativity, Melon Park and John A. Biles, 1998, “Interactive GenJam: Integrating Real-time Performance with a Genetic Algorithm”, International Computer Music Conference (ICMC) 1998 at Ann Arbor.
[6] Marcus du Sautoy, 2019, The Creativity Code. How AI is Learning to Write, Paint and Think, London: Fouth Estate, Kindle edition, pos 3543.
[7] Ibid. pos 3655.
[8] Sony CSL, “Flow Machines. Augmenting Creativity with AI”, n.d., accessed: 2024-04-07.
[9] YouTube, “Daddy’s Car: a song composed with Artificial Intelligence – in the style of the Beatles”, June 19, 2016, accessed: 2024-04-07.
[10] SKYGGE, “About Hello World”, n.d., accessed: 2024-04-07.
[11] Ibid.
Music Business Research 