Final project proposal:
For my final ICM project I plan to make 3-5 audio-visual collages that can be manipulated by the viewer. I would like the viewer to be able to change them visually as well as sonically.
I am trying to keep my expectations for this within reason so I can code them pretty much myself with some help but my goal is to have mastery over the basic elements of code involved here for these so that I could replicate more of them and potentially add onto this idea later and develop it.
I'm imagining a canvas for each, which may be a video going through the slit scan effect from the last assignment, if it's possible to put images on top of a video in the background(?), and some elements in a scene, probably abstract ones, that can be clicked on then dragged around to sort of "collage" or arrange them. Meanwhile a background song or audio layer would be playing, and when the viewer clicks on each element, a new layer of audio would be introduced or taken away, or a one time audio sample would be triggered. I would also like to have one element in each sketch that would add some subtle audio effect to the sound when it is dragged in an x-y axis. This interaction may be pretty simple but hopefully fun tool for interacting with a piece of music.
I got some feedback in class to check out Bjork's ReacTable-- here it is, to watch later!
Note:
For my Phys Comp final I am working with my partner in class to create a visualization of breathing exercises in a sort of computer program which we plan to make using P5, to be used for meditation, along with a sonification of the user's heartbeat which is triggered from a pulse sensor using an Arduino. We will also be creating a sound library of samples using P5 which will be triggered by the heartbeat data coming from the Arduino. So I will be working in P5 for that as well, but some of the math is beyond my capabilities in the code. However I think I will learn a lot and it will be fun to brainstorm the types of interactions or interface we can code using P5 as an engine.
Update on 11/13:
I wanted to add my plan for completing this project. I went back and forth between thinking that I should just focus on the P Comp project and my role helping with P5 for that instead of doing a separate final for ICM, but I think that I could create at least 3 of these musical sandbox ideas for the final, and would feel good about doing that. It would also be more based on code that we learned in this class. This is how I plan to do it, and I am scaling back what I will do based on the fact that I will not have all that much time:
For Nov 15:
Gather images and sounds for compiling/working with for 3 different compositions. Keep them simple. Some loops and some individual sounds (maybe this can be in an array so I can switch which one gets triggered?) Choose whether there will be video in the background or just one slit scanned image in the background that repeats and maybe changes how it repeats upon refreshing (or not)...
For Nov 22:
No class.
Meet with Aarón for advice. Maybe focus on the interactions between the sounds based on where they all are on the canvas. Get an image slit screening in the background and place images on top to drag around and trigger sounds with (ambient loops, and one hits in an array).
For Nov 29:
Come up with a pleasing interaction between the sounds. Play with different images and sounds. Figure out what is not working if anything and make a plan to debug it.
For Dec 6:
Do any debugging. Meet with a resident for help debugging if there are any issues. Tweak the visuals and sounds as needed (think of cooking like salt and pepper at this point). Think about the interaction. Is this a way to showcase music or to make music? Do a blog post.
For Dec 13:
Prepare final presentation. Demonstrate how it works and have people in class play with it maybe as part of the presentation if they would like?