Composing for Mobile Media

Peter Bussigel
Brown University
May 2010

Composing for Mobile Media

Abstract:
Composers have a long history of using new technologies to create music.  Mobile phones, equipped with powerful processors and capable network portals, are one of the new technologies making waves in the music community.  Composers and developers, recognizing the medium’s increasing potential for both creative and economic gain, have begun to create full albums of easily downloadable sonic snippets, interactive music generation and creation tools, and highly expressive software instrument applications.  Reports suggest that by 2014 smartphones will have captured nearly 40% of the cell phone market, bringing a dynamic and interactive listening, distribution and performance device into the hands of millions worldwide.  How are music creators responding to advancements in cell phone technology and what opportunities do these convergent devices provide?  How are these new devices changing traditional notions of the composer?  This essay will explore some of the opportunities, limitations, and potentials of mobile communication technology as a platform for new music creation. 

The past four years have seen an explosion of mobile audio experiments.  Bloom, an iPhone application created by Brian Eno, generates ambient soundscapes based on user input and, left idle, assumes the task of composition itself.  Ringtone, an album of free sound bytes by electronic composer Alva Noto, is sculpted specifically for tiny speakers.  Companies like Smule, creators of the popular Ocarina application for the iPhone, have created networked sonic experiences that use sound and cell phones as the building blocks for collaborative musical communities.  Mobile phone orchestras have even appeared on the scene. And all the while, new applications appear daily allowing smartphone users to become creators themselves, through their phones and for others’ phones.  Along with providing an overview of current work for mobile media, I speak with composers who are working with cell phones and mobile technology in an attempt to delve further into the future of mobile music.

 

WEB RESOURCES

  1. Kirn, Peter. Create Digital Music. Started in 2005.  Accessed April 9, 2010.  <http://www.createdigitalmusic.com>

Create Digital Music is an extensive blog, maintained by Peter Kern, a PhD candidate at the CUNY.  It provides a meeting place for digital artists interested in new technology.  Many of the examples and excerpts I will reference are likely to be found somewhere in the depths of its archives.  

  1. Levin, Golan. Dialtones (A telesymphony). 10 February 2004. Accessed on April 10, 2010. <http://www.flong.com/storage/experience/telesymphony/index.html>

Golan Levin’s Dialtones (A telesymphony) was one of the first concert works to use mobile phones as the primary sound producing agents.  The project website is an excellent resource for both documentation and writing about the work.

  1. Stanford Mobile Phone Orchestra (MoPhO), September 2007.  Accessed April 15, 2010.  <http://mopho.stanford.edu/>

The Stanford Mobile Phone Orchestra is an ensemble in which performer use soley mobile phones (sometimes through connected speakers) to produce textural music compositions and to perform covers of popular songs.  One of the first of its kind, MoPho, as it is called for short, is an poignant example of one of the ways mobile phones might be leveraged by composers to create new music. 

  1. C. Miller and M Helft. " From Pocket to Stage, Music in the Key of iPhone." The New York Times. December 4, 2009.  Accessed on April 11, 2010. <http://www.nytimes.com/2009/12/05/technology/05orchestra.html?_r=2&WT.mc_id=yt_nyt507&WT.mc_ev=click>.

This article from the New York Times provides a somewhat mainstream look at one side of the mobile phone music ‘revolution’.  It also highlights a few of the common criticisms of smartphone ‘instruments’. 

ARTICLES

  1. Jones, Steve. 2000. “Music and the Internet.” Popular Music 19(2);217-230.

Jones’s article on music and the internet deals predominant with “the movement of music from one place to another: from performer to listener and audience, from recording studio to pressing plant to retail outlet to consumer, from radio to any other by any variety of means.”  Mobile phone technologies provide a logical extension to some of his points, but more importantly possibly represent the return of musical movement to a physical form.

  1. Essl, G., Rohs, M. “Interactivity for Mobile Music Making”, Organised Sound 14:2 197-207, 2009

This essay clearly lays out some of the potentials in using mobile phones as instruments and platforms for music creation. 

  1. Wang, G., Essl. G., Penttinen, H., “Do Mobile Phones Dream of Electric Orchestras?,” In Proceedings of the International Computer Music Conference (ICMC), Belfast, 2008.

This text is the accompanying statement and theorization of the Mobile Phone orchestra  at Stanford.  I will be using MoPho as a case study in this paper.

  1. Misra, A., Essl, G., Rohs, M. "Microphone as Sensor in Mobile Phone Performance" In Proceedings of the 8th International Conference on New Interfaces for Musical Expression (NIME 2008), Genova, Italy, June 5-7, 2008

The microphones common on mobile phones (necessarily, as they are still phones) are the subject of this paper by Essl, Misra, and Rohs.  This is a technical paper that describes the process of transforming data from the microphone input and turning it into meaningful value streams that can be used to drive musical parameters.  The paper outlines the different ways these microphones can be used, which include blowing and striking.  The paper was globally important as it was integral to the creation of the Ocarina music application for the iphone, currently one of the top 20 downloaded iphone applications of all time. 

  1. Essl, G., Rohs, M., Kratz, S. "Squeezing the Sandwich: A Mobile Pressure-Sensitive Two-Sided Multi-Touch Prototype" 22nd Annual ACM Symposium on User Interface Software and Technology (UIST), Victoria, BC, Canada, October 4-7, 2009. Demo.

This paper deals with attaching external sensors to commercially available mobile phones, thereby extending the sensor capabilities of the phones, leveraging the software and processing capabilities while making use of DIY interfaces. 

  1. Essl, G. “SpeedDial: Rapid and On-The-Fly Mapping of Mobile Phone Instruments” In Proceedings of the International Conference on New Interfaces for Musical Expression, Pittsburgh, June 4-6, 2009.

Georg Essl’s paper on mapping and mobile phone instruments looks at some of the technical issues and affordances of mobile phone technology.  Essl describes some of the issues with mobile phones (including playback disruption, sensor reliability, and real-time error correction) and then proposes solutions to these difficult problems.  Essl also looks at the sensing capabilities of mobile phones and acknowledges some of the unavoidable technical difficulties with music performance on phones (small surface, floating point calculations become processor intensive).  This essay is a crucial study in the technical aspects of creating music for mobile phones and it will be referenced in relation to the study of MoPho and other projects that I will cover.

  1. Wang, G., G. Essl, and H. Pentinnen. 2010. “The Mobile Phone Orchestra”. Oxford Handbook of Mobile Music Studies. S. Gopinath and J. Stanyek Eds. Oxford University Press. (forthcoming; MoPhO's: stanford | umich)

A study and findings by the creators of mobile phone orchestras, outlining some of the potentials and possible pitfalls of the new medium.

  1. Wang, G., G. Essl, J. Smith, S. Salazar, P. Cook, R. Hamilton, R. Fiebrink, J. Berger, D. Zhu, M. Ljungstrom, A. Berry, J. Wu, T. Kirk, E. Berger, J. Segal. 2009. “Smule = Sonic Media: An Intersection of the Mobile, Musical, and Social”. In Proceedings of the International Computer Music Conference. Montreal.

A presentation that I actually witnessed live at the International Computer Music Conference in 2009 all about how Smule has been working with mobile technology and sound to create global communities.

  1. Benjamin, W. The Work of Art in the Age of Mechanical Reproduction. 1935.

Benjamin’s often cited text on loss of aura points to effects of reproduction as reducing the aura of artwork.  But what if the reproduced objects are instilled with their own creative agency, in a sense, not art themselves, but platforms for artistic creation.  I will argue that mobile phone music attempts to recapture the aura that Benjamin speaks of, or rather, shift the focus of aura from the art to the creator of the art. 

 

INTERVIEWS

  1. Wang, Ge.  Email Interview. April 18, 2010.

Stanford Professor, Ge Wang, is arguably the father of mobile music creation.  He has written multiple texts on the subject, founded a mobile phone orchestra, and has created multiple works for the mobile phone and mobile phone ensembles.  In addition to his creative output, Ge is a co-founder of the successful software company Smule, whose applications I will be covering in this essay.

  1. Biggs, B.  Personal Interview.  April 20, 2010.

Betsey Biggs is a sound artist who has done extensive work with mobile music technology (primarily with ipods).  She has also worked with GPS mapping technologies and sound.  In addition to her creative work, Betsey performed with the Princeton laptop ensemble, PLOrk, which can be argued was the precursor to the mobile phone orchestras created by Ge Wang (who was also a performing member of PLOrk)

  1. Milani, Matteo. Interview with Peter Chilvers. Unidentified Sound Object Project, September 19, 2009.  Accessed April 13, 2010. <http://usoproject.blogspot.com/2009/09/generative-music-interview-with-peter_19.html>

Matteo Milani interviews Peter Chilvers, who programmed Bloom, an iphone application designed by Brian Eno.  The interview covers some of the generative principles behind the application as well as the coding specifics. 

 

BOOKS

  1. Auslander, Philip. 1999. Liveness: Performance in a Mediatized Culture. New York: Routledge.

Auslander’s book, Liveness, is an in-depth study of live performance in today’s ultra-mediated world.  He discusses the evolution of our current notin of ‘liveness’ pointing to television’s relationship to theater. Using examples and studies form rock music, Auslander speaks about how the live has been devalued in contemporary culture.  Auslander also explores “live” in the framework of legal proceedings, arguing that liveness is still of great importance to the American judicial system.  Auslander’s text is relevant to this paper in that it describes a breakdown of the traditional value system, the ‘aura’ of the ‘live’.  Mobile phones, I might argue, could serve to partially reconstruct some of the aura surrounding ‘liveness’, even if they themselves are the pinnacle of current mediated experience. 

  1. Jenkins, Henry. 2006. Convergence Culture: Where Old and New Media Collide. New York: New York University Press.

Convergence Culture by Henry Jenkins attempts to define media convergence, and through definition, debunk many of the commonly held perceptions of digital utopia (ie: “the black box”).  Jenkins uses references from popular culture (ie: American Idol) to illustrate his points about convergence.  Jenkins argues that media convergence is actually a combination of multiple convergences, thus detaching the term from its most common technological associations.  Jenkins talks about economic convergence, social convergence and global convergence, thus complicating popular views.  Mobile phones are currently perhaps the most ‘converged’ technology around, especially given the growing popularity and advancements of smartphones.  Jenkins’ ideas on convergence will act as a launching pad for contextualizing the musical affordances of the mobile phone in today’s culture.  It will also provide a reference in comparing the mobile phone with traditional instruments where I will argue that phones represent new instruments but not replacements for traditional ones.

  1. Dixon, Steve. Digital Performance. Massachusetts: The MIT Press, 2007

Steve Dixon’s book, Digital Performance, is one of the most referenced works in studies of mediated performance practice.  Dixon provides a fairly comprehensive history of ‘digital’ performance in the 20th century, touching on futurism and dadaism among other trends and then moving to the current day (2007).  Dixon provides ample theoretical context interspersed with examples of work dealing with virtual bodies, robots, virtual reality, and online performance.  Most relevant to this paper is the final section of the book, dealing with interactivity.  Dixon discusses “performing” interactivity at some length, which is related directly to much of the mobile music work I will be analyzing. 

  1. Attali, J. 1985. Noise. The Political Economy of Music (Minneapolis)

Attali’s seminal text predicts, with arguable degrees of accuracy, many aspects of the current musical evolution.  Most notably the shift in creative focus from the composer to the consumer, a move that mobile phone technologies potentially reflect. 

 

RECORDINGS

  1. Alva Noto. Ringtone. Raster-Noton. Digital Download. 2007.  Accessed on April 11, 2010. <http://www.alvanoto.com/?a1=audio>

Alva Noto is an electronic composer and musician who is one of the leading composers in using new technology to create electronic music and multimedia work.  His free digital album, Ringtone, is crafted to be heard through the limited frequency speakers of modern mobile phones.  Ringtone is a good example of the potentials and limitations offered by mobile phones as public playback devices as well as marketing tools.