You have javascript disabled in your browser. We recommend turning it on for a better experience on this site.

Multimedia system provides new view of musical performance

Musicians can now use 3D computer analysis to radically improve their technique, thanks to the latest research in multimedia technology from the University of Leeds.

Dr Kia Ng of the University's Faculty of Engineering and School of Musicias devised a way to use motion capture to record a musician's posture and movement as they play and then map the results against ideal performance settings. The system is known as the i-Maestro 3D Augmented Mirror (AMIR) and is a powerful tool for music teachers, students and experienced or professional musicians to improve their technique.

"Learning to play an instrument is a physical activity," said Dr Ng. "If a student develops a bad posture early on, this can be potentially very damaging to their career and our system can help teachers to easily identify problems. Similarly, the system enables experienced musicians to make small changes in gesture and posture that can improve the sound they make.

"Many musicians already use video recordings of their performance to analyse technique, but this only provides a 2D image. The 3D image and analysis provided by AMIR will be of immense value to musicians and teachers alike."

The prototype has been designed for stringed instruments such as violin and cello but could be adapted for other instruments. Small markers are attached to key points on the instrument, the musician's body and the bow. As the musician plays, 12 cameras record the movement at very fast speed - 200 frames per second -  and map the instrument in 3D onto the screen. Bow speed, angle and position are all measured for real-time analysis and feedback, as is - for violinists - the pressure by which the instrument is held on the shoulder. Dr Ng has even incorporated a Wii Balance Board to include data on the musician's balance as they play.

The musician or teacher can then hear and see a video of the performance alongside an on-screen analysis of posture and bow technique, which if necessary they can work through frame by frame or bow stroke by bow stroke.

Dr Ng, himself a violinist, explains: "What makes a great sound is difficult to analyse, but with technique, some things come down to basic physics. If the bow is held perpendicular to the string and parallel to the bridge, the minimum effort will produce the maximum result. Our system can measure this and show musicians exactly when their technique becomes less effective."

A video of the system in action with cello and violin can be seen on the i-Maestro website at where the prototype software can also be downloaded free of charge.

Dr Ng hopes that AMIR will in the future be used widely by teachers and music colleges as a useful tool alongside more traditional teaching methods. However, at present the motion caption hardware needed to work the system can cost anywhere between £5,000 and - for the sophisticated set-up used at the University of Leeds - £100,000. Until hardware costs reduce to make such systems more widely affordable, he plans to offer musicians the opportunity to use the system at the University of Leeds Interdisciplinary Centre for Scientific Research in Music ( laboratories on a consultancy basis.

AMIR was developed as part of the European-funded i-Maestro project, coordinated by Dr Ng, which looked at ways to provide new pedagogical tools for music using multimedia and information technology.

For further information:

Please contact the University of Leeds Press Office on +44 (0)113 343 4031 or email

For academic/technical queries contact Dr Kia Ng on 0113 343 2572, mob 07921 527063, email

For potential collaborations or consultancy arrangements, contact Martin Hartley, Consulting Leeds, on 0113 343 0935, email

Notes for editors:

1. Screen grabs of the AMIR are available - see for examples.

2. The i-Maestro project was supported by the European Commission under the IST Sixth Framework Programme to develop interactive multimedia environment for technology enhanced music education. The €2.35 million project was coordinated by the University of Leeds and involved eight other partners: the University of Reading and City University London from the UK; the Institut de Recherche et de Coordination Acoustique/Musique (IRCAM) from France; the Dutch Federation of Libraries for the Blind (DEDICON); Extreme Information Technology (EXITECH), the Universita' Degli Studi di Firenze and the Academia Nazionale di Santa Cecilia - Fondazione from Italy; and the Fundación Albéniz from Spain. For more information see

3. The Faculty of Engineering at the University of Leeds is ranked 7th* in the UK for the quality of its research (2008 Research Assessment Exercise); an impressive 75% of the Faculty's research activity rated as internationally excellent or World leading. 
With 700 academic and research staff and 3,000 students the Faculty is a major player in the field with a track record of experience across the full spectrum of the engineering and computing disciplines.  The Faculty of Engineering is home to five schools: civil engineering; computing; electronic and electrical engineering; mechanical engineering; process, environmental and materials engineering.
Two thirds of students are undergraduates with the remaining third split evenly between taught masters and research degrees.  The Faculty attracts staff and students from all around the world; one third of students are from outside the UK and representing over 90 different nationalities.

4. The Faculty's School of Computing is ranked in the top 10 computing departments in the UK based on the quality of its research (2008 Research Assessment Exercise); an impressive 80% of research activity rated as internationally excellent or World leading.
The School has an established track record of high quality research in the foundations of computer science and in a variety of applied and multidisciplinary settings with particular expertise in the areas of Artificial Intelligence, Biological Systems and Computational & Systems Sciences.
With 70 academic and research staff and over 400 students the School is a major player in the field of computer science.

5. The University of Leeds is one of the largest higher education institutions in the UK with more than 30,000 students from 130 countries and a turnover of £450m. The University is a member of the Russell Group of research-intensive universities and the 2008 Research

Back to the top