Research

Interactive Performance Systems

Wed, 23 Nov 2011 10:39:12 +1000

This research explores the relationship between musicians and interactive software systems (bots or agents).
While computer technology plays a role in most areas of music production and performance, the potential for a computer to act as a live performance partner is relatively under-explored. I hope this work will drive a new wave of music development based on interactive performance with digital systems.
Our research goal is to develop a music system that ‘listens’ to a performer and responds in real time, which is a significant step away from the passive systems which operate today.
There is capacity for digital systems to achieve greater sophistication and to demonstrate a sense of autonomy, which will open up new avenues of musical expression.
There is real potential for new genres of music to emerge as technology continues to expand its role and offer musicians an enhanced performance experience.
We see parallels between interactive music systems in the 21st century and previous performance technologies such as the saxophone that played an important role in shaping the 20th century musical genres of jazz and rock music.
“By measuring human gestures and interactions we’ll be able to transfer this knowledge to computer systems to ensure this technology can anticipate the actions and movements of human performers and intuitively respond.
This work is supported by a Discovery grant from the Australian Research Council for a project titled “Controlling Interactive Music Performance: Enhancing live performances by human musicians and algorithmic computer systems”
This project investigates how musicians can effectively and intuitively control algorithmic music systems during performance. It identifies and evaluates musical features that provide for expressive algorithmic variation during live performances between humans and generative computer music systems. The project tests intuitions and enhances theoretical clarity regarding the salience of musical features, providing a basis for future developments in the emerging fields of computational live arts and interactive digital media. The discoveries made by this project will have relevance to, more broadly, the phenomenology of human interactions with autonomous computer systems.
Collaborators on this project include:
• Dr Toby Gifford of the Queensland Conservatorium.
• Associate Professor Ian Whalley of the University of Waikato, New Zealand;
• Dr Michael Young of Goldsmiths, London;
• Associate Professor Francois Pachet of Sony Computer Science Laboratories, Paris

An interview of ABC radio about the project can be heard here.

Digital Creativity: Representation to Agency

Thu, 2 Dec 2010 07:47:48 +1000

The capturing and manual editing of media in digital form has been a revolution for creative artists and the industries that surround them. Look no further than music and video distribution and user-generated content for evidence of this. We are on the edge of a new revolution in digital creativity, this time driven by harnessing the computational power of computers to participate more integrally in the creative process through semi-automatic analysis and generation of digital works. This presentation will outline some of the ways this new revolution is unfolding, in particular looking at three aspects of computational arts; the support of human creativity, algorithmic aesthetics, and the modelling of creative processes.
I was invited to give a keynote address at the CreateWorld 2010 conference and presented this idea. A podcast interview after the keynote can be heard here.
I also presented this concept at a seminar in July 2011 as it relates to assisting people with limited musical ability to engage with music making with computational-agents as musical partners for well-being purposes. A video of that presentation is embedded below.

Generate in Context

Wed, 12 May 2010 10:59:30 +1000

With Toby Gifford I have devised and used the Generate In Context research method. In this process computational models of analytic music theories are built and used to generate music. This generated music is subject to aesthetic assessment as a way of interrogating the theory and model. In a sense GIC is used to reverse engineer a musicological or music perception theory. The ‘contextual’ aspect comes into play in the many aspects of music are required to generate listenable music, and this is often in contrast to the ‘scientific’ method often employed in musicology where attributes , say pitch organisation, are treated independently of other musical parameters or outside a use or cultural context.
We have employed the GIC method in the Smart Music research project that looks at theories of melodic construction that are both musicologically and psychologically valid.

This project is funded by a Discovery grant from the Australian Research Council, and includes an international team consisting of:
• Prof. Andrew Brown Griffith University
• Dr. Toby Gifford Griffith University
• Dr. Robert Davidson University of Queensland
• Prof. Eugene Narmour University of Pennsylvania
• Ass. Prof. David Temperley Eastman School of Music
• Prof. Geraint Wiggins University of London

A paper introducing and outlining the Generate in Context approach was published in 2009.

Education as Meaningful Engagement

Sun, 17 Feb 2008 21:27:27 +1000

The theory of meaningful engagement is a philosophy of education that draws together research by Steve Dillon and myself. It represents a significant shift in educational philosophy with its emphasis on education as primarily concerned with the development of a student’s awareness of their relationship and interactions with others and the world. That is, how they manage their engagements with the world to maximise the meaningfulness of their lives. This is contrasted to previous philosophies of education that focus on the acquisition, construction or management of knowledge. The theory of meaningful engagement acknowledges the 21st century reality that knowledge (at least information) is abundant, if not overwhelming, and that reorganisations of epistemological frameworks are no longer sufficient as educational reform. Rather the theory suggests that education is primarily an ontological endeavour in which students seek ways to live well, upon which decisions about relevant and timely knowledge and skills can be made and acted upon.
Modes of Engagement
• Attend: Listening carefully to music and analysing music representations.
• Evaluate: Making decisions about musical value and relationships
• Direct: Managing music making activities
• Explore: Searching through musical possibilities and assessing their value
• Embody: Engrossed in fluent music making
Types of Meaning
• Personal: The activity is intrinsically enjoyable.
• Social: Through activity the user develops relationships with others.
• Cultural: The activity, and therefore the person, is valued by the community.
There is a web-based MEM Scorecard that assists with the use of the MEM as an analytical framework. The Scorecard allows you to count, by clicking, the occurrences of different aspects of meaningful engagement in an activity or a document. Check out the MEM Scorecard here.

Software Development as Research (SoDaR)

Tue, 1 Jan 2008 14:37:26 +1000

SoDaR is an approach to research that involves software development as an active research method in the humanities. This theory describes how software development can be used as a method for research in the arts and design fields in particular. It explains how software development can be used to externalize emerging ideas, how such externalization can aid research by allowing reflection, how it stimulates acting upon theories, and provides evidence about the validity of theories in practice that involves new software-based experiences. There are parallels with interactive software development process and established research methods such as action research, case study, and activity theory.
The application of the SoDaR approach to music education research was outlined in the a paper published in the International Journal of Education & the Arts.

Collaborative Creativity

Fri, 3 Aug 2007 16:33:24 +1000

Computer supported, or assisted, creativity is a strong thread through many of my research projects.
My PhD thesis involved the case study examination of five professional music composers and an examination of the role the computer played in their practice. This research resulted in the modes of creative engagement theory.
I have been involved with a number of collaborative tools, such as Jam2jam (see picture above) where musical jamming is enabled across computer networks.
My work at the Australasian CRC for Interaction Design (ACID) focuses on technology mediated creative human interactions including the development of improved efficiency for workers in the creative industries, enhanced educational experiences in the creative arts, and community development through shared creative activities.
Relevant academic papers include:
Code Jamming (2007)
The role of generative arts in supporting cultural participation (2007)
Emergent Matters (2006)
How the computer assists composers (2001)

Computers in Music Education

Fri, 12 Jan 2007 08:08:54 +1000

This is my book published early in 2007 by Routledge on the applications of computers to music learning. From the back cover; “Computer technology has been rapidly changing, especially when it comes to the filed of music. From notation software to MIDI music creation, from downloading music for study and enjoyment to uploading personal files. Music teachers have traditionally been slow to embrace these technological advances, but that trend is changing. Computers in Music Education, written by a music educator and digital media specialist, cuts through the jargon to present a concise, easy to digest overview of the field, with practical suggestions throughout. This book will help educators keep abreast of new developments in the field and provide insight on the uses of a computer as a tool for efficient music making, a medium for sound and music manipulation, and an instrument for musical expression, amongst others. Because of the pace of technological change, the book focuses on the general principles of using computers in the music classroom rather than on the use of any particular software. Computers in Music education will be the standard text for the growing number of courses in this area.”

There are reviews of the book available online at Amazon.com, Computer Music Journal, Music Forum, Organised Sound, and on Jonanthan Savage’s blog.

Characteristics of Music Algorithms

Thu, 8 Jun 2006 06:59:52 +1000

The concepts and terms currently applied to discussions of algorithmic music are varied and, especially for new comers, often confusing. For example, consider the term “generative music”, which is used with different meanings by various scholars and famous practitioners. Myself, Rene Wooller, Joachim Dederich, Eduardo Miranda and Rodney Berry have developed framework designed to clarify discussion of algorithmic musical processes through cateogisation and clustering based on features. The framework is based on two principal attributes that enable intuitive comparison of algorithmic compositional processes, these are its type of function (analytic, transformational or generative) and its the breadth of its inclusion (encapsulation) of musical context.
Our framework for the comparison of processes in algorithmic music has several applications. It can help composers understand the processes and potential of algorithmic systems when designing or selecting algorithms that meet creative needs. It can assist musicians and researchers to differentiate between algorithmic processes and more clearly articulate the features of computer music systems. Unexplored applications of the framework could include tracking the evolution of algorithmic music systems and spot design trends. The framework, through extension, might also be applied to system evaluations. We look forward to seeing in what other contexts uses are found for applying this comparative framework.
For more information there is an academic paper on this that was presented at the Generative Arts Practice conference in 2005.

Computer Assisted Composition

Fri, 21 Oct 2005 19:25:51 +1000

Making Music with Java is a book that introduces the reader to music making through software development in the Java programming language using the jMusic library. It explains musical and programming concepts in a coordinated way. The book is written for the musician who wishes to learn about Java programming and computer music concepts, and for the programmer who is interested in music and sound design with Java. It assumes little musical or programming experience and introduces topics and issues as they arise. Sections on computer music and programming are interlaced throughout, but kept separate enough so that those with experience in either area can skip ahead as required.

Adaptive Music for Computer Games

Tue, 8 Mar 2005 06:45:04 +1000

My research at the Australasian CRC for Interaction Design has included working with Computer Game companies to develop adaptive music systems. Along with developing appropriate algorithms for music generation and variation, a significant part of understanding these systems is to have methods for describing the architecture of an adaptive system as it fits into a game scenario. To this end we developed the Interactive Music Architecture Description system, that showed the logical structure and organisation of adaptive systems. This has been used for both analysis of existing game music, development of new game music specifications, and as a compositional aid to the game music composer.

Digital Instrument Making

Mon, 8 Nov 2004 05:58:58 +1000

The methods of digital instrument building and the ways these enable creative expression are interests of mine. In 2003 with assistance of a grant from the Australia Council for the Arts and the efforts of some colleagues, Paul Doornbusch and Peter McIlwain in particular, I established a support site for digital instrument building, based on the jMusic library.
The site contains tutorials that cover cover topics including music software design, audio synthesis, digital signal processing, compositional systems, and computer programming concepts.
Tutorials draw upon the design and pedagogical skills of digital instrument makers from Australian Universities, with advice from experienced music software developers.
Visit the site here.

Modes of Creative Engagement

Sun, 5 Mar 2000 03:40:48 +1000

My research shows that creative activity is most importantly directed by how interested and involved a person is during the creative experience. It has also revealed, on closer examination, particular ways in which this experience can manifest itself as modes of creative engagement.
There is good reason to think that creative activities using a computer are inherently engaging, given the amount of time people spend at these activities, however, it is also possible for computer software to be boring or frustrating. Therefore, as designers or users we need to pay attention to the modes of creative engagement that different systems foster. When you are engaged in a creative activity, the theory suggests, you approach the task in one or more of the following modes.
• An appreciator – pays carefully attention to creative works and analyses associated ideas and theories.
• A director – manages creative activities
• An explorer – searches through possibilities and assesses their creative potential
• A participant – is involved in intuitive creative expression
• A selector – makes curatorial judgments about appropriateness or value of creative works or practices.

In a well-rounded creative life people can experiences different modes of creative engagement as they pursue their creative activities.
An outline of the modes of creative engagement applied to computer music composition was published in 2001 academic paper.