MagnaQuest

Objective:

To develop a sound-controlled video game that inspires musical expression, teaches music concepts, and sparks creative collaboration among young learners.

Method:

I collaborated with a team of experts from different fields, including music education, gamification, music production, and 3D art to create innovative and engaging experiences using engineering tools.

I developed a real-time digital signal processing algorithm that enables music students to control games by playing their violin. It uses the sound's amplitude and frequency to trigger character movement and direction, resulting in intuitive and engaging human-computer interactions without the need for additional hardware or connections.

I created a metronome that provides didactic feedback to players based on their timing of notes relative to the beat of the background music to teach music concepts.

I visually represented the player's unique ability and personality through a tree that was distinctly shaped, sized, and colored based on their improvisation.The player's amplitude and frequency were sampled over time to alter the tree's growth pattern, with different patterns triggered by common techniques like arpeggio, crescendo, legato, staccato, and accents.

I created a multiplayer music practice environment using Unity3D networking protocols, where players could invite friends to their garden and share their previous improvisations and corresponding trees, play concerts, and react to friends' concerts through emoji messages. If all participants played music together, rain would fall for everyone.

I made the game accessible on various platforms by compiling it for the web, PC, Mac, Android, and a VR build with Google Cardboard.

Tools:

Sound-Controlled Videogame

Collaborative Music Practice Environment