The goal of metacreative artists is to endow computer programs with creative behaviors, to create computer algorithms that have the same ability to make artistic decisions within certain frameworks as human beings. To this end, metacreative artists employ the tools and techniques of artificial intelligence and machine learning, the same used in cognitive and life sciences. 1
Metacreative music composers create virtual performers who can generate new and unique compositions, often in real time, with no assistance from the human composer after the initial programming is complete. Metacreative composers do not just program a computer or machine to play a composition; they create artificial artists. Thus, this area of investigation threatens many traditional artists, and as metacreation develops, the role of the artist and artistic creativity in modern society must be reevaluated. Our methods of expression and evaluation must evolve.
Metacreative artists are arguably on the cutting edge of cultural production, seeking to unite the mechanical with the artistic. However, is it really possible to create a machine that is able to make autonomous artistic decisions? If so, could those artistic decisions ever be considered, not merely competent, but avant-garde? And what would that mean for our perception of human artists?
Algorithmic Composition in Academia
Although some popular artists have experimented with algorithmic composition techniques over the years, 2 much of the work in the metacreative field is conducted in universities. Simon Fraser University’s School of Contemporary Music produces some of today’s most innovative metacreative projects.
Associate Professor Dr. Arne Eigenfeldt has worked at SFU in various capacities since the mid-’90s, lecturing at the School for the Contemporary Art between the years 2000-2003. During this time, Eigenfeldt worked extensively with methods of sound manipulation and processing, such as granular synthesis 3 and Max/MSP. 4 By 2004, Dr. Eigenfeldt was developing computer software that would generate melodies, harmonies, and rhythms for ambient music, ultimately released through various labels under the name “Raemus”, this being his lone venture into the realm of popular music. After that, Eigenfeldt’s work became increasingly academic, more concerned with how to achieve art than the final result.
In 2005, Eigenfeldt began developing a software program he named “Kinetic Engine”. The program was originally conceived as rhythm generator that would play sound samples as installation art with patrons able to influence the density of its rhythms. In the absence of patron participation, the program would create its own independent rhythmic compositions ranging from one to fifteen minutes in length. (5) We’ll come back to that in a bit.
In 2008, Eigenfeldt met Assistant Professor Philippe Pasquier, a computer scientist with the School of Interactive Arts and Technology in SFU’s Faculty of Applied Sciences. Pasquier’s primary research discipline was in the field of artificial intelligence. Their ensuing collaboration pushed Eigenfeldt’s work from a more traditional “interactive” art approach to an exploration of metacreation and computational creativity, the cutting edge of computer programming where computers are endowed with the ability to make artistic decisions.
Together, Eigenfeldt and Pasquier founded an interdisciplinary group known as “MAMAS” (Metacreation Agent & Multi-Agent Systems), which is housed at SFU. With the group’s help, newer versions of the “Kinetic Engine” evolved to incorporate several new computer algorithms that pushed the piece from installation art to true metacreation. A harmonic component was added, as well as an evolutionary “Genetic” algorithm, which creates rhythm populations 6 based on the analysis of other music. The program was also integrated with Ajay Kapur’s “MahaDeviBot”, a solenoid-based drumming robot capable of simultaneously striking and shaking twelve physical percussive instruments. 7 Using the knowledge gained from “Kinetic Engine”, Eigenfeldt started developing “Coming Together”, which includes two sub-projects: “Freesound” and “Beauty and Truth”.
“Freesound” is a generative soundscape composition that features four autonomous artificial “agents”. It should be noted that there is a great deal of variation in the definition of an “agent” among metacreative artists, but the view held by Eigenfeldt and Ajay Kapur is, as he explains, “directly related to existing musical paradigms: the improvising musician. Such an agent must have a much higher level of knowledge, but, similar to other multi-agent systems, each agent has a ‘limited viewpoint’ of the artistic objective, and, as such, collaboration is required between agents to achieve (musical) success”. 8 Basically, an agent is a virtual performer, a singular piece of a larger program that possesses its own sense of agency.
In “Freesound”, agents are programmed to select and play sound files from a recording database in real time. This means that each “performance” of the piece is unique and unrepeatable. As agents select sound files, each spectrally analyzes 9 their sounds and compares them with those selected by the other agents. This allows the agents to avoid playing the same dominant frequencies of sounds so they do not play over each other. Agents also employ signal processing, whereby they manipulate their auto signals by filtering frequencies out, adding reverberation or processing delays. The piece culminates when the agents produce a series of granulated instrumental tones based on what they’ve “listened” to during the piece.
The “Beauty and Truth” sub-project is similar to the later versions of “Kinetic Engine” in that agents are used to create musical compositions. However, in “Beauty and Truth”, the agents are entirely autonomous in the sense that must “choose” the pitch, rhythm, volume and timbre. As the agents explore the terrain of their musical environment, they converge on related pitches, amplitudes, and timbres democratically, evaluating their collective successes and failures by listening to other agents and judging how well they form melodic phrases. On a macro level, the agents generate communal goals and execute plans to achieve the parameter of a “successful” performance, a successful evolution from random sound to synchronous production.
In the final moments of “Beauty and Truth”, if the agents successfully synchronize their artistic efforts, four additional agents called “quarks” are introduced to the piece. The quark agents improvise brief virtuosic flourishes on the emergent harmony and rhythm produced by the first set of agents. The piece will start over shortly after the quark agents appear, or the when the original agents suffer too many failed attempts at synchronicity.
Metacreation: Is It Avant-Garde?
When asked, Eigenfeldt’s definition of the avant-garde is typical among most practicing artists. For Eigenfeldt, avant-garde simply means to experiment, to push an art form beyond its current boundaries. Indeed, the French term ‘avant-garde’ literally translates to ‘advance guard’ or vanguard, and much of the popular conception of the term ends there. Enter into the dialogue Clement Greenberg, Peter Bürger, and Lev Manovich, three avant-garde theorists who represent different historical eras. Greenberg developed the discussion on avant-garde with his 1939 essay “Avant-Garde and Kitsch”, Bürger produced a canonical exploration of the term with his “Theory of the Avant-Garde” from 1975, and Manovich brought the discussion into the 21st century with his 2001 book The Language of New Media.
In his essay “Avant-Garde and Kitsch”, Clement Greenberg defined the avant-garde in opposition to a notion he referred to as “kitsch”. He saw kitsch as a product of consumer society, as the “easy stuff” that exacerbates the decline of aesthetic standards, and as a diversion for humanity’s increasing capacity for boredom. Both kitsch and academic art are described as calculated, mechanical, and formulaic.
Included his definition of kitsch is academic art, as he stated, “All kitsch is academic, and conversely, all that is academic is kitsch.” He saw academic art as a form unable to touch important, controversial issues, and in which the creative act is reduced from inspiration, introspection, and invocation to mere “virtuosity in the small details of form”. In other words, academia transforms artists into artisans, who are then absorbed and diluted by kitsch. As this recuperation –as founding Situationist theorist Guy Debord called it– takes place, the function of the avant-garde becomes a rolling wave of innovation, finding “a path along which it would be possible to keep culture moving in the midst of ideological confusion and violence.”
That said, a significant concept in Greenberg’s essay is the “umbilical cord of gold”. He argues that, while the masses have largely ignored the process of cultural development, the avant-garde was almost wholly supported by the elite ruling class. There are many examples of Greenberg’s umbilical cord of gold. In Beethoven’s time, wealthy patron aristocrats supported serious artists; today, institutions such as art galleries, publishing houses, and universities support artists and their often-extravagant works. Ironically, the avant-garde, armed with an awareness of historical and social criticism to position itself at odds with mainstream society, is almost always attached to it by this umbilical cord of gold.
In many ways, Greenberg’s definition of the avant-garde would place Eigenfeldt’s metacreative work firmly in the kitsch category. The mixture of computer science and the arts in Eigenfeldt’s work is not only academic, but could easily be seen as mechanical and, certainly on a mathematical level, formulaic. Yet, Eigenfeldt’s work could be seen as expressing the very tension articulated by Greenberg’s comparison of avant-garde and kitsch.
Eigenfeldt’s work is not purely artisanal craftsmanship, and obviously far from being the “easy stuff” that Greenberg despised. Though Eigenfeldt clearly uses scientific and mechanistic tools to develop his art, he conceives much of it with public performance in mind, as an art piece not as a computer program. While Eigenfeldt uses algorithms and digital code, technological processes that are clearly the product of consumer society, his works can be seen as questioning the very foundation of artistic inspiration and the human pursuit of art, asking whether a machine can truly be an artist. In this sense, Eigenfeldt’s works can be seen as recuperation in reverse, taking commodified, mainstream tools and using them to express a radical idea. Thus, Eigenfeldt may be seen to exemplify Greenberg’s definition of the avant-garde.
Peter Bürger notably enters the avant-garde discussion with his 1974 book, Theory of the Avant-Garde. The book weighs and measures much of the discourse produced to the time of its publication, from Adorno to Walter Benjamin. Like Greenberg, Bürger is highly critical of the institution of art, and central to his critique is the notion of recuperation, whereby the art establishment embraces socially critical objects and “neutralizes the political content of the individual work”.
Both Bürger and Greenberg refer to role of the institution and its effect on the avant-garde, with Greenberg focusing primarily on academia and Bürger discussing the art establishment more generally. The interaction between the institution and the artist highlights another tension in Eigenfeldt’s work, with an emphasis more on the artistic process than the production of actual art pieces.
Eigenfeldt works within the system of a university, and his work is very much affected by Greenberg’s umbilical cord of gold. Where artists in the public realm depend on populous support, creating art that brings an audience into concert venues or engaging them in other forms of consumption, Eigenfeldt’s projects are largely funded by grants. In order for him to attain grant funding, his work must appeal to juries consisting primarily of artists or academics.
When Machines Can Build Machines Endowed with Creativity
In this system, funding decisions are often dependent on the production of academic papers. As Eigenfeldt notes, “In order to get research money, you have to prove that you’re doing something that’s research based. They don’t necessarily take the final product into account. The types of things that are going to get funded are things look good on paper.”10 Thus, engagement with the public is not emphasized, and the academic environment becomes insular. Professors in the arts may become complacent, largely repeating works or recycling artistic concepts to produce projects that “look good on paper”.
While Eigenfeldt’s work is not political in the sense of being critical of academic or artistic institutions, he remains open to the influence of popular music, and cares a great deal about audience reaction. Practically all of his pieces have been performed at conferences, concerts, and festivals. While Eigenfeldt is dependent on the grant system for funding, thus realistically neutralizing much of the potential for politicization against the university, his refusal to become complacent within his own artistic process can be seen as a political act at least partially in line with Greenberg’s definition of the avant-garde. However, it is likely that Bürger would see Eigenfeldt’s financial dependency on the institution as “neutralizing the political content” of his work.
On the other hand, for Bürger, the avant-garde is was a more ephemeral concept. He notes that the attack by the historical avant-garde on the institution of art failed to unite the institution with the practice of life. It maintained its untouchable distance. Bürger sought to reunite the praxis of art with the praxis of life, which makes the contemporary avant-garde a process of constant evolution.
In this connection, Eigenfeldt’s work is generative, created in real time, making it arguably tied to the “praxis of life” as it happens. Many of Eigenfeldt’s works are directly influenced by the actions of the crowd and live performers, modulated by the living moment. In his Theses on the Cultural Revolution, Guy Debord proclaimed that the “goal is immediate participation in a passionate abundance of the variation of fleeting moments resolutely arranged.” Indeed, many of Eigenfeldt’s programs are designed to resolutely arrange various fleeting moments, truly in the moment.
Bürger also describes the relationship between “shock” and “newness”. These notions are not specific only to art, though. Any society, in general, can only survive if it sells the goods that it produces. 11 As such, consumers are constantly bombarded with shock, with appeals to the newness of products. 12 However, there is nothing that loses its effectiveness more quickly than shock, while newness merely means variation within the narrow, defined limits of genre.
Eigenfeldt also conceives of a future application for his works in the realm of video games, where pieces could be generated based on the actions of the player. This would bring metacreative music closer to the “praxis of life” in the 21st century, since video games are now a bigger market than film. Although, if and when that happens, that would also become part of the norm, an expectation of people hungry for distraction, and as the newness of generative music faded, so would their impressiveness and impact.
New Avant-Garde, New Media
Lev Manovich’s major contribution to the discussion was his 2001 book, The Language of New Media. Where Greenberg focuses his discussion on poetry and painting while largely ignoring film and music, and Bürger more or less follows suit while largely denying or ignoring entire movements such as Dadaism and Futurism, Manovich approached the theory of the avant-garde with an absorbing contemporary mindset, placing new media within the context of “the most suggestive and broad ranging media history since Marshall McLuhan”.13
In The Language of New Media, Manovich does not focus his analysis on the concept of the avant-garde but outlines the development of what he refers to as “new media objects”. Manovich prefers the term “objects” to “product” or “artwork”. According to Manovich’s expansive definition, a new media object may be a “digital image, a digitally composited film, a virtual 3D environment, a computer game, a self-contained hypermedia DVD, a hypermedia Web site, or the Web as a whole.”14 The realization and perception of new media objects are affected by or dependent on five general principals: numerical representation, modularity, automation, variability, and transcoding.
Numerical representation is a base level parameter, since all computer data can be reduced to ones and zeros, making it the genetic code through which all digital information is stored and communicated. Modularity refers to the “fractal structure of new media”, as the different elements of new media exist independently. A useful example Manovich gives for this is an HTML document. 15 An HTML document may use images, media clips, movies, and other media stored in a variety of locations, yet brought together through language commands to a computer into a perceivably singular form.
Through the use of the first two principles of numerical coding and modular structure emerge the distinct yet comparatively similar concepts of automation and variability. Automation is where new media objects are created and/or modified through templates and algorithms. Variability refers to the multiplicity and mutability of new media objects. Where traditional paintings are permanent once fixed to a canvas, and copies can only be made as identical albeit slightly weaker references to the original, new media objects “can exist in different, potentially infinite, versions.” For variability, Manovich again refers to the example of a Web page, where a computer, guided by human commands, assembles the final form of a page, and humans are free to purposefully alter the results through the HTML language.
Manovich’s final principal is transcoding.16 This is an observation of how the understanding and representation of our identities are influenced by the logic of the computer. Manovich outlines two layers of affluence: the media/cultural layer and the computer layer. His example to distinguish the two layers is that of a digital image, be it a GIF, JPG, BMP, etc. When a person views the image, they see a reference to human culture. The representation enters a dialogue, as the image relates to similar images the person has seen or to real world object events or objects.
Yet, the image simultaneously exists as a file, as ones and zeroes, on the computer, which presents the image by decoding the formatted language in which it was written, specifying its size and color. The image takes up space on the computers hard drive, and must then enter a dialogue with the rest of the information stored there. Superficially, one may notice “LOL” and “OMG”, abbreviations developed for ease of texting and forum chatting, now being a part of the everyday English language as an instance of transcoding, but the larger implications deal with the ways in which we process information and, in turn, program computers.
True to Manovich’s principals, Eigenfeldt is engaged in the production of new media objects. Modularity can be seen in the use of different agents “coming together” to create singular works, and the birth of quark agents in the instances of success. This can also be seen in the use of the Genetic Algorithm and “Kinetic Engine” across different applications.
In particular, Eigenfeldt’s objects employ automation and variability to great effect. He uses algorithms to achieve the overall goal of his objects: automation, while he constantly adapts his projects to work with human performers, adding variables to his equations. Furthermore, given his work with Philippe Pasquier and artificial intelligence, he is always playing with the notion of transcoding, teaching computers how to think like humans and, in order to do so, reorganizing his thought patterns to think more like a computer in order to figure out how to teach computers to learn.
The possibilities and ramifications of Eigenfeldt’s work are vast. There may very well be a day when human hands are removed from the situation, where machines can build other machines endowed with creativity. One has to wonder at what point the objects’ agents will truly become artists themselves, capable of avant-garde conceptualization. At what point are the individuals who program music metacreation avant-garde artists? Or does that determination depend on the artistic choices of the metacreative agents?
Some may see metacreative art as a threat to humanity’s pursuit of artistic expression. Indeed, as these systems continue to perfect, the role of the artist in modern society will be brought into question, particularly given that artistic creativity is one of the intangibles that defines our humanness. However, if human artists are deserving of their title, the result of this fear should be to push their works in more interesting directions, forcing them to be better and more creative to differentiate their abilities from that of machine learning.
There is no reason to fear the works of Dr. Arne Eigenfeldt. His art follows a natural progression in avant-garde music. Taken at face value, Eigenfeldt respects a rich tradition of experimentation with chance processes, from the roots of serial composition planted by Arnold Schoenberg, its expansion beyond the 12-tones by Pierre Boulez, to the organized ‘silence’ and philosophical indeterminacy that runs through the works of John Cage. While the works of those composers are universally recognized as major influences on the development of music, their work was supremely contentious when it premiered, and the work of Arne Eigenfeldt is no different. He creates the opportunity for new music, and lets the chips fall where they may.