I have always been fascinated with using nature as a medium for interaction and a source for new creation, especially in the area of music. And, for the final project, I want to focus on using water as a medium to create music and visualize the flow and beauty created with manipulating the dynamic features of water. Ripples and waves are often easily created by an external force, but I wanted to insert something more into those movements that can be appealing both visually and audibly.
In terms of technical side, the project entails the use of a projector, a kinect, a container to contain shallow level of water, and a stand that will hold the project and kinect in place above the container.
The ideal instance of a user interaction would be where users would play around with the water on the sides of the container, creating different waves and ripples on the surface. The kinect will capture the depth of the water, and color the points that are at higher level than the base level of the surface, which will be projected on the water through the projector.
Meanwhile, depending on the location of the ripples/waves, either a recorded or a set of notes will be played in a loop. The more waves created will lead the music either playing faster or at a higher pitch.
The area of projection and detection for the projector and kinect will be smaller than the area of the container (as seen in the figure below) in order to reduce the instances of where hands are detected as waves or ripples.
The expected size of the board would be 120cm x 120cm x 20cm. I would still need to calculate and see the level of the support for the projector and the kinect, but the area that the kinect and the projector will cover the area for 100cm x 100cm. I will mainly be using Processing for the program, in attention to the sound library and the library for connecting kinect with processing.
Four most difficult, complicated or frightening aspects:
- Detecting water waves/ripples using Kinect
- Playing sound depending on the location of particles/points on a certain dimension
- Changing the pitch of the sound depending on the number of points in a certain area
- Aligning the Kinect with the projector in terms of visuals
