LeapMotion Sound Design
Close

Project Description

This project was completed as my dissertation project for the MSc Sound Design course at the University of Edinburgh in 2022.

This project explored what might make Physical Sound Interactions 'joyful', and how one can tailor their design process around these principles.

Within the project, I created an Interface Software using Unity and C# which connected the LeapMotion camera data to local OSC ports. I then used this interface to design physical interactions with sound, using Max/MSP as my prototyping software to create the sonic behaviour.

You can read the full report from this project here.

Video Demos

  • Theremin

    Model of a theremin. One hand controls volume and vibrato, the other hand controls pitch through a combination of position, rotation, and hand shape.

  • FM Synth

    An interaction using FM Synthesis to create an interaction where each hand controls a distinct synthesiser through its movements.

  • Magic

    An interaction that combines sound systems from a few of my other interactions, and uses them to create a spell casting interaction, whereby the user charges and casts spells by closing and opening their fists.

  • Concatenative Synth

    An interaction that uses concatenative synthesis with a large corpus of sounds to create intense erratic sound effects. One hand controls volume by opening and closing, and a filter by rotating, whilst the other hands position determines the audio to playback.

  • Scrubbing Explorer

    An interaction that connects the playback rate of an audio file to the velocity of the users hand. The audio is 'frozen' in a particular point so that the user can choose which area of the audio file to 'explore' from. This interaction was designed with foley applications in mind.

  • Piano Clouds

    In this interaction, the users hand movements trigger piano samples in a wind chime-esque way. The opening and closing of the left hand determines which of 4 related chords these samples are generated from.

  • Drum Sampler

    An interaction where drum samples are triggered by threshold based velocity triggers. The direction of velocity is used to create a sense that these samples are placed in different positions.

  • Speech Synthesis

    This interaction uses the formant filtered square wave approach to speech synthesis to create an unpredictable interaction where the users movements produce speech-like sound.

  • Dial Music

    This interaction is designed to be a playable musical instrument, using the pinch of each hand to control the volume of two synthesisers. The pitch of each synthesiser is controlled by the rotation of the hand (hence 'dial'-music) and its position in the y axis.