Day 102 @ ITP: Phys Comp


“Great Heart” is a collaborative project created for Tom Igoe’s Introduction to Physical Computing class by Jim Schmitz and Camilla Padgitt-Coles. The project uses a pulse sensor to detect the user's heartbeat and translate it into sound. The user can hear their heart rate sonified and follow breathing visualizations which are designed to guide their breathing to help users achieve inner peace, release anxiety, and slow their heart rate down. The user puts on a wristband with the pulse sensor attached and sits as long as they choose with the sounds and visuals.

Using push buttons on the device’s enclosure, the user is invited by the audio guide on the Introduction screen to choose from different instruments and notes to represent their heartbeat. Their heart rate is also represented as a pulsing light on the enclosure and as a pulsing light next to the breathing exercises in P5. After the meditation has finished, the user is told what their beginning and ending heart rates were.

There are two options: A "challenge" mode where the program stops once your heart rate has slowed by 10%, and a "meditation" mode where the user can sit and breathe for a fixed or variable duration (for the winter show we chose a 2 minute duration).

The project is modular and allows the freedom to change the audio and visual aspects and various components of the program in P5 as needed. The Arduino sends the heart rate data and button-press messages for the instrument, note and sustain on/off buttons through serial communication to P5. The rest is coded in P5 using Javascript, audio samples, and images.

Our project on GitHub

Pulse Sensor Amped

MIDI Sprout

Heartmath Institute

Circuit diagram

Circuit diagram

System diagram

System diagram


We will be adding a product demo video using the school's cameras over the break in January. We will also tweak the code a little for the heartbeat detection and make one more enclosure. If possibilities come up to show it in a different way or different setting I will be blogging about that here as well!

Day 99 @ ITP: Fabrication

I'm honestly getting to this late, because the last week I was mostly consumed by my final for Physical Computing and ICM and working on that project right up until the deadline (which did involve some more fabricating and expanding on things I did earlier in Fabrication class, but not for this project). I was suggested by two people to play with something to do with light - which is something I have done in a live projection setting but have been meaning to play with in terms of building some things at ITP. However I think whatever happens today will have to be pared back a lot (!). A friend suggested making a Dream Machine, and I think with one of the things I have I will be able to make some kind of miniature version of it using a small servo motor.

Screen Shot 2017-12-13 at 6.06.07 PM.png

Update 12/14:

The "dream machine" cutout was too heavy for the platform and motor I was able to get working (a DC motor, in the end.) So I created a prototype of something else inspired by it--the "lightning machine" -- which may look better through video than in person, but possibly could be fun in a large darkened room with a flashlight. Will try that out today in class and see how that works.

DC motor
2 AA batteries
Hot glue
Piece of wood board
Gaffer's tape
Piece of mylar emergency blanket (gold and silver)

In progress 

In progress 

Update 12/16:

My friend ended up actually being inspired to use this contraption in a recording for an abstract dance piece. What I thought was a haphazard compromise ended up being a pretty cool musical instrument. It also actually looks beautiful and like an optical illusion sometimes when it spins without flopping on itself (unlike in the video below, where it is flopping, and I am also shining a flashlight on it while crouched in my closet). It is strangely relaxing to watch/listen to. Maybe I will document it more later as I never got a final photo or good video. I liked what my classmate said about how it reminded her of ASMR videos on YouTube. It's cool when things surprise you like that. In the future I would like to try making some motorized light and/or sound sculptures sculptures building off of ideas that came from this.

Day 98 @ ITP: Phys Comp + ICM

The time has come! Our final class for Physical Computing is tomorrow. Jim Schmitz and I have been working on this project since we were assigned to work together on the midterm project in October. We have spent a total of around 10 weeks of conceptualizing and prototyping, coding, fabricating, troubleshooting, rethinking, reworking, and ultimately finalizing this second phase of our idea. I think there are various directions this project could go in should we choose to do so, and manifestations it could take in general as an idea, and many ideas that will come of having worked on it potentially for other things I do after this (I can't speak for Jim here, but I hope it is the same case for him as well!) 

Today we met to finalize a few details which were keeping us from being 100% finished with the goals we had in mind for the final presentation. It was pretty down to the wire time-wise, but the last things we wanted to add though seemingly small are additions were ones that we felt would add infinitely to the user experience and conveyance and reception of the meaning of our project to someone who doesn't have us there to explain it to them. We updated the code to read the beats per minute, through registering the time it takes for a full heart beat sine wave to register, then repeating it (like a "stamp"). We decided to do this based on the feedback I got while presenting to my ICM class from Allison and my classmates about how it was distracting that the sensor would miss beats, and how it would not be unethical to fill in the missing beats through approaching it in a different way. With our new setup, once the sensor gets a new piece of information about a heartbeat it continues to play the sounds at that rate until it gets a new piece of information, which updates around every five seconds. This way we are able to provide the user with a consistent heart rate sound, which is ultimately more calming (some people were starting to worry if there was something wrong with their heart). With this method we are able to keep it steady without sacrificing the validity of the data that we are showing them/playing as a sound for them. As it is, it tracks heart rate and whether or not it is lowering, with a slight delay. The notes and flashing heartbeat colors in the visualization play at a rate set as 1 minute (or 6000 milliseconds) divided by the beats per minute, which consistently updates the rate at which the sounds are being played:

Screen Shot 2017-12-12 at 7.48.54 PM.png

We also added a "Challenge" version, which uses this new heartbeat bpm data to track when the user's heart rate has lowered by 10%. At the end of both the "Challenge" and "Duration" options the user sees their initial pulse rate next to their final pulse rate at the end of their session.

Here are some screenshots of our current visualizations:

(Jim sewed a heart onto the new wristband!)

(Jim sewed a heart onto the new wristband!)

Great Heart enclosure version #2


We also recorded Jim's voice for the audio guide. We were planning to record our classmate but something came up. Maybe we will record other versions in the future if needed! And ask her again. But I think Jim's voice was actually very calming, and appropriate since we've been working on it. We decided to use a male voice also because in general voices in technology are female, though I'm not sure about instructions... In any case, we now have an audio guide, which I will also attach here: 

Welcome to Great Heart.

Using your left wrist, carefully put on the wristband to see and hear your heartbeat, with the heart placed over your pulse. This is where the sensor is reading your heart rate.

In a moment you will hear notes in sync with your heartbeat. Feel free to adjust the sound using the blue buttons on the enclosure.

Lay your left arm on the table, palm facing up, and hand relaxed.

Take in a deep breath through your nose, if you are comfortable.

Press either button to start.

Choose a visualization on the screen, and breathe in and out deeply as the shapes gets bigger and smaller.

Breathe in: ...2...3...4 (Breathing sound in through nose)

And out: ...2...3...4 (Breathing sound out through nose)

Tomorrow we plan to project Great Heart through the classroom projector as well as plug into the classroom speakers. It should be fun to see and hear it big (as long as everything goes as planned)! We also found out that the project will be in the winter show which starts this weekend. There it will be displayed with headphones and a monitor. It's been very educational and inspiring to work on this up until this point. Further documentation to follow... 

The code for our project can be found on GitHub here

Day 96 @ ITP: Phys Comp

Yesterday I came to ITP and laser cut two more enclosures at a larger size (and failed to take photos of the process, but will take more later of the finished project). I made two because the wood was curved and I wasn't sure if they all would fit. It turned out that was true -- Ben Light was right, these boxes are tricky! However it still seemed like the best option for me at the moment to make sure it would stay together. 

In the afternoon Jim and I met up to make some new breathing visualizations. We added a "breathing rectangle" that grows up and down the screen, and a polygon that unfolds open with your breath cycle. We also beautified the "gui" or computer program pages he had started in P5. He also showed me how to "beautify" my code! Which is also helpful to find mistakes or places where you forgot a bracket or where it is broken.

Today we met again and wired up the new enclosure. Everything fit, however I think the enclosure could still be improved upon, maybe if I use flatter wood and also take my time more with glueing, as it was a bit of an ordeal trying to glue it last night, partially bc my plan was not thought out and I assumed it would be simpler than it turned out to be. I got some glue marks all over it which I think I could avoid in the future by removing excess glue with a dry cloth. 

It was helpful to wire up the new enclosure because it reminded us of how we organized the wiring on the Arduino/Breadboard, and also made it clearer how we would repeat it in the future if needed, and what we would change. 

We also attempted adding a heart LED onto the wristband which would also blink with the user's heartbeat, but for some reason that is not working yet, maybe because the LED is broken? We will troubleshoot later. 


Next up we will:

- Meet with Tom tomorrow afternoon for feedback before we continue.
- Troubleshoot the heart LED on wristband
- Update the heartbeat detection algorithm with Jim's reworked code to read the heart rate intelligently guess where missing heartbeats would go, to make the sound more consistent.
- Possibly add an option to view the heart rate.
- Have the dropdown menu show up when viewing the visualizations.
- Finish coding the "challenge" vs "duration" options and how those will work.
- Record audio guide

Day 96 @ ITP: ICM

I decided that my "musical sandboxes" project is something that maybe I will continue to work on over the whole course of ITP on my own or in classes, and I would like to refine different ways of making it in code or as a combo of software/code/physical objects. But for this semester since I am also working simultaneously on a project for Physical Computing that is using P5, it seemed to make most sense to focus on that and combine both finals and get the most out of that project, since it is also one that I am very interested in.

Recently we updated the code to add more breathing visualizations using P5 that I will screen capture and share here for this upcoming Wednesday. We also created a kind of "computer program" in P5 which I did not show at the final ICM presentation as it was not finished yet, which has an intro screen that leads you to the different visualization options, and a final screen that thanks you for participating before clicking back to the beginning. Using the basic infrastructure of this program, and the things I've learned this semester in ICM, I think I would be able to combine the code I worked on with Jim with various ideas and will definitely be referring back to the code we worked on together for future projects (and possibly for different iterations of our project).

Right now it is very helpful to have a modular system that is easily changeable, and to manage it in separate .js sketches so that we don't break our code. I'm learning about organization of code and proper coding habits, and the trick after this will be to combine it with all I learned is possible to do with coding in ICM, and make different kinds of projects and keep it up. It really does feel like learning a language and as with any language there is a hump that you have to get over before it starts to feel more like second nature.

Day 96 @ ITP: Animation

Today I am attempting to start my final animation in Unreal, after going through a few tutorials and playing a bit with the architecture of the program, I still a feel a bit lost but am excited by the possibilities. It also seems that not all the content packs are compatible with High Sierra, so most likely I think I will just be trying to get creative with the starter materials, of which there are a lot. I don't usually work with humans too much, I prefer working with abstract colors/shapes or animals if I am drawing. But it will be a good challenge! Right now my character looks a bit like one of the characters from Avatar the movie.

Will post updates soon...

Screen Shot 2017-12-13 at 6.29.43 PM.png

Update December 13th:

I spent a few hours today sitting with my classmate and trying to figure out Unreal on a PC laptop from the equipment room. We both had realized we had upgraded to High Sierra on our Mac laptops and weren't able to use the animating function of Unreal. We were trying to figure out importing a character on the PC out without too much success but we did start to build a world together with floating cubes that went up into the sky like a ladder (I forgot to take a screen shot!) and looked at various characters and tutorials on how to import them and navigate Unreal. It seems very complicated. But I think if I was walked through it once one on one, and maybe had time with it on my own laptop, it would make more sense. I ended up heading home because I thought I would be able to install it on my Desktop computer, but the operating system on that one is just one version too old... 

So at this point since I think I will be working on it at home for the rest of the night, I am thinking I will create a world in Unreal with a 3rd person character, and use a screen recording to follow it around as it moves. 

Update 12/14:
My final animation

Day 91 @ ITP: Phys Comp + ICM

Great Heart — Questions for User Testing

Screen shot of the breathing visualization #3 w/ blue ring (between heartbeats)  

Screen shot of the breathing visualization #3 w/ blue ring (between heartbeats)


Breathing visualization #3: blue ring + pink orb which triggers w/ heart beat

Breathing visualization #3:
blue ring + pink orb which triggers w/ heart beat

Today Jim and I got together and coded the visualization with some new images, and made a video of our project for applying to the 2017 winter show. For now there are 3 visualizations. I will also be showing this project as my final presentation in my ICM class to share the part using P5 and also get some I'm sure valuable feedback there on the far I have sat next to Jim and coded/typed while he explained what to do, which was extremely helpful for me-- and I hope also helpful for him to think about how it needed to be coded too? Not sure. He was very patient though. But it definitely helped with my muscle memory with coding to remember how to indent lines and add semicolons properly and put parentheses nested in the right way, etc. And also about using classes to organize code into sections, objects, and in general about simplifying code to the least amount of lines, and the syntax for that, etc. Jim's coding is very neat and simplified. I can't say that I would be able to recreate it or the math that is involved sometimes but it is starting to make sense to me, and it was nice of him to let me type. He explained how everything worked very well. At times I was able to fill in new parts on my own and other times I was pretty sloppy and left out brackets, etc, things I would not have caught for a while. So it was definitely very helpful to work with Jim on this (!) But I think we also worked well together, and had equal parts in conceptualizing it, and also both worked on fabricating it. Overall I am very proud of it and am interested to see what kind of reactions or suggestions people have! 


Day 89 @ ITP: Phys Comp

I uploaded new sounds here. I also swapped them out in the code and updated it on GitHub. 

Jim and I also met up today and wired together the Arduino in the new enclosure design:

Yay, the buttons light up and they are beautiful!

Yay, the buttons light up and they are beautiful!

Next we will:

- Implement Jim's new P5 library for more reliable heartbeat detection
- Work together on creating the heartbeat visualizations in P5
- Laser cut a new top at the correct size before Wednesday (and later in the week cut another box, or two if I have time)
- Try out using it with the wrist rest, possibly sew an LED in the wristband which could also potentially light up with the wearer's heart rate? Or, more simply light up all the time.

Day 89 @ ITP: Fabrication

For our next assignment with the two different materials I am thinking of making a chessboard with two different kinds of wood. I would cut the wood into strips. Maybe 12" x 12" is good.  Then I would glue the pieces together and then cut and glue again to make a checkerboard...also maybe I would order chess pieces for it and not make those! Or make rubber stamps like these... 



I would still like to maybe make the chess board later, or chess pieces for a marble board I have which has broken pieces. Instead this week partially due to lack of time/finals I ended up going with the stamp idea. I have always wanted to make a stamp of the Perfect Wave logo. At first I was going to carve it but realized I could engrave an inverted image onto rubber with the laser cutter. I think I could have rastered the raised part maybe one more time but I haven't tried it yet, so maybe it will work? I also cut 4 squares out of the 1/8" Alder wood I used for last week's project to glue together for the handle and rasterized the logo onto one of the pieces for the top of the "handle". This time I also used the 75 W Laser cutter, which did seem more powerful, you just have to manually focus each time.

Next I will glue the wooden pieces and the rubber to the wood. 

Update 12/7: 

I tried the stamp as it looked above and some of the flat parts around the image were still stamping, maybe because it wasn't cut deeply enough around it the image, so I decided to carve it a little more by hand. I also realized I should have flipped the image, as it is coming out backwards from how it normally is, but it's also OK in this case as I will still use it/sometimes I have flipped the logo around depending on the occasion to play with it...but I may redo it properly with a few more passes and less space around it so I don't waste as much laser cutting on just cutting a rectangle around the shape, and then cut off the excess with scissors. But now it makes a pretty decent stamp.

The final stamp and print it makes

The final stamp and print it makes

The rubber came from the Dick Blick art supplies store on Bond St. around the corner from NYU -- "soft rubber" for printing, where they are having a holiday sale. I also got carving tools and special adhesive from Amazon so now I will need to make more stamps either with the laser cutter or by hand to use it which will be fun...

Day 86 @ ITP: Animation

Assignment #2: Collaboration using After Effects w/ Dom C.:

Some notes: I definitely had fun working on this project. I think my skills in After Effects could still be much improved, and my animations here were a little shaky, and the character illustration maybe compromised because my tablet pen wasn't working for a moment while we were doing this project, so it was a little crude. But I understand more how it works now, and how to animate a character and  setting, and see how there is a world of possibilities in the applications of After Effects. I will definitely be using it in the future and am grateful to have an had an official introduction to it in this class.

Day 79 - 85 @ ITP: Phys Comp and Fabrication

Here is prototype design #1 for the enclosure, made in Illustrator...these measurements are not exact (I also made the back wall slightly lower, because our idea was to have the top plexiglas slide in and out of the enclosure...) I also made the sides slightly larger to accommodate walls inside which will support the plexiglas top so this box is around 2.5" high, 5.5" wide and 8.5" long. I'm also not sure if the rounded corners will work practically. Maybe some other technique would need to be used or learned to get those or at least thicker wood to begin with and then sanding. And we may need some other way to support it from the inside if there are no screws. But this is the general idea for the enclosure for our P Comp final (which I am combining with the next Fabrication class assignment, which is to make an enclosure...if I don't have all the materials in time I will make a cardboard version at least):

Screen Shot 2017-11-23 at 11.33.29 PM.png

And a second version incorporating Jim's feedback, this version has mostly right angle corners (except for the top plate) and a longer bottom plate to accommodate the longer irregular design we came up with (I had forgotten to make the bottom panel longer before, then fixed and edited it in again below) and added a slot in the back panel for the top plate...I'm still waiting on materials due to the shipping backup so I will be doing a cardboard mockup first then hopefully a first version of the actual enclosure this week.

Screen Shot 2017-11-24 at 3.04.25 PM.png

Update 11/24:

24093912_10100448303917194_1843448107_o (2).jpg

I printed out a tiny paper mockup of the design and found that it is not quite right. I will need to adjust the front panel and also the length of the sides. Need to get the measurements exactly right (also of the buttons) before cutting into the wood...

Also waiting on these light up buttons for "note" + "instrument" (momentary) and "sustain" (on/off) buttons, this switch for power off, and these LEDs for the heart rate.

Light up buttons 16mm


Toggle button 13mm


Product Dimensions: 32.9mm x 13.1mm x 12.0mm / 1.3" x 0.5" x 0.5"

Product Weight: 5.0g / 0.2o

Heart LED

These arrived today! So will be able to measure.

These arrived today! So will be able to measure.



Update 11/27: 
We met on Sunday and rethought the design together. It seemed that having the protruding front panel was a little advanced to be sure that it would hold up with the buttons on it being pressed constantly, and my designs were not really that well thought out to ensure that it wouldn't break. Before we met I made the above sketch going back to the original design. I actually thought it might help to go back to the slotted design to reinforce the box which would also require keeping it square (sorry Ben, I know you hate them!!) because it seemed to work okay for the cube I made for the last Fabrication assignment, and because it would add some stability considering I didn't have a great backup plan for if it needed reinforcements, especially in the area with the buttons which will be continuously pressed and would need to be sturdily placed. I basically would like to make this very simple and durable for use after the final is over as well (and make two of them!) for us both to use and test out meditating with over the break and beyond hopefully, so it shouldn't break. So simplifying the design seemed like a better way to make sure that happens at least for this project's timeframe. I have been thinking in general about other manifestations this project could take, also with some input from friends and classmates, like making it a standalone hardware/light/sound creating piece of hardware (like a phone app but without the phone and would not be an app, but sort of light and sound driven meditation device or object) or also different instruments that could be created this way, but that would be for later on. In my time at ITP it would be fun to work on at least one musical instrument project or something that could have various iterations with a similar vocabulary or way of being built. But I digress.

Jim and I came up with a design that is very similar to the one above, but with the buttons placed on the longer panel. We will add a ridge inside and top frame for the plexiglas plate to slide in and out, thereby making it easier to open up and to fix anything with the electronics inside.

I will start laser cutting tonight (I mistakenly left it this late because I did not realize ITP would be closed over Thanksgiving week, but maybe it's for the best as some small design changes happened, and hopefully now it will be easier to complete without wasting a bunch of materials). 

I'm starting in general to think about ways that various similar things could be made with the same parts. The only more expensive part at this point seems to be the Arduino, but maybe some of them wouldn't need an Arduino Uno but some smaller microcontroller in order to work. I will leave a bookmark here to come back to that idea later.

Update 11/28:

I booked two hours on the laser cutter today and brought the design that Jim and I adjusted together on Sunday. The wood and buttons had arrived so I was able to measure the holes for the buttons and adjust them. I had some issues with the power settings on the laser cutter resetting a few times while printing, and also misremembering the speed once, so one or two pieces took longer to cut before I realized that this had happened. Otherwise following the settings in the document on the Desktop of the laser cutter's computer worked fine and I am getting the hang of how it works. Still want to experiment more with the wood shop, but this seems good for doing finer work like cutting openings for enclosures like this. Though it took a while to cut through 1/8" pieces, and I wondered if actually a 1/4" board would have been sturdier, and if maybe I should have used the 75W cutter to do it that way? Otherwise, the slots do fit (we flipped some pieces around to get the text in the right places), and it's a little too big for the Arduino + breadboard but also won't be sure of that until we add the inside supports for the top and all the wires, so at least it's definitely not too small. I think it would be nicer if it was more compact but that's just a visual side note and could be adjusted later for the next one.

Most importantly I already snapped off the ledge above where the top plate is supposed to go in/out, so that is a bad sign and will need to be fixed somehow or rethought design wise. I think one side snapped when I took out the piece in the slot which had been laser cut out, and the other side snapped when I was seeing if the acrylic would fit through (it did fit, but it seemed a lot heavier compared to the 1/8" Alder wood hence it breaking). Another solution would be to have a wooden top, which is definitely an option, especially if that seems more long lasting. I got this 1/8" Alder wood because it was recommended for laser cutting and also seemed thin enough to do a full box almost in a short amount of time on the cutter, but I think in the next iteration of this enclosure I will be taking into mind all the problems that came up with this first one. I also got some 1/8" Cherry wood and am wondering if that would be sturdier also. Again having more time on the laser cutter would have helped of course but everyone else is trying to finish things too for finals and a 2 hour chunk at a time seems reasonable as you are watching it and standing there the whole time... once I get a real plan I think I will reserve some hours at an off time, like 9am Saturday morning or something just to finish it properly and have extra time if something goes wrong. I think as I do more of these kinds of things I will gain a better vocabulary of tools for putting things like this together and using the right materials and steps in the process etc. On a positive note, the heart LED fits! We used the "boolean union" option in Illustrator to combine the shapes of the two sides I traced, and it actually fits perfectly now. Next we will sand down the 3 larger button holes as they are still slightly too small, but thankfully Jim has some tools that will help with this, and wiring the Arduino with the buttons and the LED and the heart sensor (which will be attached to the Arduino through that small hole under the heart shape), and programming the buttons...

24201482_10100449732329644_1899316904_o (1).jpg

<~~~~~~~~~~~~the broken piece 

Also need to decide for sure where the Arduino will be placed to adjust where the openings will be accordingly.....

Update 11/29:

A little late on this update, but I did glue the enclosure together last night, which was a bit messy. I met up with Jim yesterday afternoon and he helped to file the holes for the 3 larger buttons so that they would fit properly, using a needle file (pictured to right!) to sand them larger it in a circular motion. It worked, but it was too labor intensive and in the future I will definitely just make those openings a little wider and keep in mind that I should size up a little bit... maybe a couple of mm?. Also maybe I should get some of these for fixing irregularities so things fit etc (I was using scissors on a few pieces, which was not ideal). These are the readings I got when measuring the buttons with the caliper: 


I did measure around the part with the threads, but it was still too small. The buttons also fit with the cardboard prototype at this size but not with the wood, I'm assuming because the wood has no give. 


The heart LED is staying in without any support at this point, which is nice! We will have to reinforce it with some glue somehow, which will close up the gaps. If it doesn't look right when lit up maybe we can also add the plastic behind as Ben suggested below, which could make it seem more solid I would imagine...

Other notes: I think the box could have been a bit smaller and less clunky feeling, also more solid feeling. And the top in acrylic doesn't look quite as integrated as I imagined so maybe design wise wood would be better for that as well. It would be great to try various iterations of enclosures that could be swapped out in different ways or used with different sounds and the same setup, in different sizes or something like this. Acrylic seems more durable but wood more fun to work with in general (except for maybe using acrylic to create see through parts/colors/having things light up/playing with colors or transparency ?) -- just need to find the most sturdy one to work with as this board snapped in a few places pretty easily after being cut, or maybe just going up to 1/4" or 3/8" (possibly from this link) would fix that. However I think I would need to upgrade to a higher powered laser cutter as it took 4-5 times to cut through 1/8" on the 50W laser cutter which ends up taking a lot of time.

Day 78 @ ITP: ICM

Three beginnings for the musical sandbox idea: (<-- this one not working for some reason though it was working in the editor...)

Currently they all have the same sounds and images but they will vary...

Next step is to make them drag around and pan the sound on x-y for one of them or have it somehow change the overall sound depending on where it is on the canvas, and meet with Aarón again to get some help with the code again next week...It was nice today how I was able to complete a bit more of the coding on my own after seeing how something is done or watching him code it and creating html objects.

Also not sure about keeping these shapes, they were just for testing, but for some reason they are growing on me. I was thinking of using a color from each photograph as the color of the shapes and keeping it simple.

I also like the idea of doing many different iterations of the same kind of combination of things.

Day 78 @ ITP: Phys Comp

It will look something like this...with a new logo design by Tristan Martineau. I'm not sure what it's called- "great heart"? Not sure yet if this is what the text will say but we will need some kind of instructions. How to have people know to meditate with it without scaring them off by mentioning meditation? This is a particularly cartoony envisioning of what we will do, but something like this:

Screen Shot 2017-11-22 at 10.49.55 PM.png

What is left to do: 

- Designing the enclosure for print using Illustrator

- Laser cutting the wood and plexiglas for the enclosure

- Building the enclosure with buttons and the heart LED

- Sewing the sensor into the wristband 

- Continuing to work on the code for heartbeat detection and doing user testing

- Finessing the visualizations and how the breathing exercises will work in P5

Day 69 @ ITP: Animation

Tonight I worked on a sloth with separate layers for the limbs and torso (which I guess includes the head for this), traced from this sloth that was also created for an animation. I have a Wacom monitor at home which I normally use with a pen, but it wasn't working so I just created the shapes in Photoshop clicking around with the mouse which is not it's a little rough. I guess I also could have drawn and scanned the pieces but this seemed like the easiest way to do it for this purpose, also as the point is mainly to play with animating it and it was easy to create layers for it like a marionette in Photoshop. I have yet to see how it would actually look when the limbs are moving. Tomorrow I will meet with Dom to figure out doing a sample animation with After Affects, also to decide what kind of scenes to add the character(s) into, and will also try to wrap my head around using 3D in After Affects with

Update on 11/16:

We made some short clips, and plan to make our animation with the storyline of the sloth leaving somewhere in Manhattan (near NYU area) to go to Brooklyn to get a pizza and come back. Not high art perhaps, but we are having fun! I'm still having trouble figuring out placing the character on the 3D axis, but maybe will be able to get that after today's class or go deeper into tutorials to figure out how to include using that for this animation before we finish it.

Day 69 @ ITP: ICM

Update on final project:

I would like to make three audio-visual sandboxes using these images as canvases, which are photos of 35mm prints I found at my parents' from Chile when we lived there circa 1995 or 1996...the last one is of our backyard in Vitacura. I just really like the colors and would like to see how they would look slit scanned in the backgrounds of these sandboxes. I also feel that for what I am imagining just one image repeating will be enough for these, except maybe would like to change exactly how it scans each time even if the source images do not change.


Day 68 @ ITP: Phys Comp

Timeline for working on our Final Project (Camilla and Jim)

WEEK 1: Prepare for playtesting.

Our diagram about how the new version (P5 Meditation - working title) will work! Before playtesting. (Click to enlarge)

Last Tuesday night before the playtesting we sat down and created a diagram of how we wanted our new version of the MIDI Meditation (working title: "P5 Meditation") would work. We ended up deciding to use P5 to store and trigger the sounds instead of using MIDI and software because we were having issues with using MIDI commands that would work cross platform for both OSX and Windows, and also this way the whole system would be more streamlined and also require less expensive software to use (and it will also be more modular and changeable based on what we end up wanting to do with the visualizations).

This way we can also just code the Arduino once to take in and send data to the sketch and make adjustments on the P5 end (and get better at using P5 and serial communication!) We decided that the user would have a choice of a few breathing exercises from a drop-down menu, and that we would use our feedback from the playtesting to figure out how that should work.

This way we can also just code the Arduino once to take in and send data to the sketch and make adjustments on the P5 end (and get better at using P5 and serial communication!) We decided that the user would have a choice of a few breathing exercises from a drop-down menu, and that we would use our feedback from the playtesting to figure out how that should work.

Our notes from playtesting... (Click to enlarge)

Our notes from playtesting part two... (Click to enlarge)

Using Processing to begin with we created a simple visualization of the breath that looked like a wave or water that filled up the screen and then subsided based on when you breathe in, hold your breath, or breathe out. People seemed to not respond to this so much. In fact we had a few people all have the same reaction, which is that they would prefer a visualization that would expand and contract rather than fill up the screen up and down. Here is our feedback written out to be able to see it more clearly:

- I told my wife about this project and she said that people could use this before getting a vaccine, or before therapy.
- Maybe have text that appears and disappears to prompt instructions (like breathe in, hold, breathe out...)
(Camilla Note: Will the text still be needed if we have detailed instructions written out in the program's dropdown (which we did not have yet?) Or maybe just have text appear at the startup of each exercise then it could go away eventually?)

- Would prefer to see circles that get bigger and smaller.

- Would prefer to see a physical light fading in and out or a dim object vs. a computer screen.
(Camilla Note: I totally agree! Though we chose to work with P5 because overall it will be the least buggy for our needs. It also requires the least amount of *stuff* and is the most versatile, so we would be able to tweak it easily up until the last moment, and also afterwards. However it would be so great to get the visualization off the screen and project it really big in a room, huge, if possible. Otherwise this is something we will take into consideration but probably only after getting all the coding to work, loading the sounds into P5 and being able to trigger then properly, and building the enclosure, rather than spending that time hardwiring LEDs which we cannot change or adjust as easily...)

Meng Zhen:
- Maybe have the heartbeat visualization slow down with your heartrate (or at least slow down).

- Would like to have a meditation timer or be able to pick the duration of the meditation. Need instructions for how to pick program change and then set it because it was not clear.

- Maybe some visualization of heartbeat vs. speed of breath. 
- Thinks wrist band would be good for sensor placement.
- Use images to explain breathing exercises so language is not necessarily needed. 
(Camilla Note: My first thought here was to have an animation of someone breathing in then breathing out, or some visual directions like that. Will have to think about this and what we can achieve in P5- maybe that can be some kind of intro animation if we can get around to thinking about and creating that?)

- Would like to see something that expands and contracts.
- Maybe background music as well as the heartbeat sound? Would like if we choose it, and not to have an option on the user end for the background music.
- Colors that come to mind for enclosure: Green, brown, wood, natural colors. 
(Camilla Note: I also brought up the idea of white, and he thought maybe this could work. I like the idea of incorporating natural wood as well, and maybe green buttons if I can find some that would work well?)

- Maybe create some goal or arc or game in order to help relax and know when to "stop". Helps to relax if you know how long it will be.

WEEK 2 (for Nov 15th)

  Update your system diagram and bill of materials based on what you learn from your playtest. By next week, you should know what devices or components you need, and should have ordered or obtained them.

  • Update Arduino code to work with sensor data (using MIDI Meditation prototype)
  • Write up feedback we got last week, blog change

WEEK 3 (for Nov 29th):

  Update your blog with your detailed system diagram, BOM, and interaction plan.

  • Sampling sounds, get those working in P5
  • Collect data using the NEW sensor (may work a little differently)

WEEK 4 (for Dec 6th):

   User-testable version of the project!

  • Build enclosure(s) - prototype and final design, sew sensors into wristbands(s).
  • Finalize the gui (something with a dropdown, nicer HTML)

WEEK 5 (for Dec 13th):

  Finish your final project
  Make sure your online documentation of the project is done as well
  Prepare and rehearse how you want to present your project

  • Documentation
  • Final testing (lots of user testing)


WINTER SHOW: December 17th-18th


- We got the communication working to send messages for each command from the Arduino to the P5 sketch.
- Our next step is making sound samples and triggering them in P5, and playing with the P5 interface. 

- For this next week: Think about enclosure. It will be pretty similar to the original design but with nicer more permanent materials (probably wood on the side panels with a white acrylic surface and small black buttons--which need to be higher this time to be above the acrylic surface). My next assignment for Intro to Fabrication is to make an enclosure so I plan to make the two enclosures for this combined with that assignment.

Day 67 @ ITP: Intro to Fab

Assignment #3:
Make something with the laser cutter.


Today I went into ITP to try out using the laser cutter for the first time ever. First I stopped by Canal Plastics and got a selection of acrylic to use hopefully for the full semester if not longer. I got large sheets of 1/8" transparent red, orange, yellow, green, light blue, and smaller sheets of gray smoke and 5 smaller sheets of opaque white for potential enclosure designs.

I originally reserved time on the 50 Watt cutter from 2-5:30pm, then ended up changing that earlier in the day to later from 5:30-9:00pm because I realized I hadn't left enough time for myself to prepare what I wanted to make and I ended up doing at home before coming in to Manhattan. I wasn't sure if I had messed up by changing my time on the day of, or how it works in the shop, so I went in not really knowing what to expect.

I got there at 5pm and took a moment to check over my files, and while I was doing that someone started using the 50 Watt machine. I told them a little before 5:30 came around that I had reserved time and they quickly finished what they were currently cutting but because of that I ended up starting closer to 6, which in itself wasn't a big deal. The person working in the shop was very nice about helping me set up my file for the first time, then I was able to do the rest and start printing/cutting jobs on my own... While I was on the machine a couple of people asked me when I would be done and obviously were very eager to use it right then, partially because the 60W cutter seemed to not be working properly. I thought that my project was going more quickly than I anticipated so I told them I would be done relatively soon, but I also felt some pressure to rush because they wanted to use it as well. I thought Saturday night would be a quiet time in there as nobody had reserved time on the calendar, but I found that nobody was checking the calendar anyway maybe? And if you do reserve time and two people are waiting for you to be done, what is the etiquette (especially if you still were planning to use it for at least an hour or two?) Do you just tell them when you will be done? How long is too long to reserve ahead of time? I learned some things I guess about how using the laser cutter is something that a lot of people are sometimes trying to use at once regardless of the calendar and am wondering how to handle that in the future. 

I also realized that I should have set up my file better beforehand, and set the vector lines to 0.001, and also taken the paper film off the acrylic before starting to cut, as it took up some time to do that. 

Anyway. For this first project, I decided to make a rainbow cube! I used this website to create a 4" cube with a top that would work with the 1/8" acrylic. I started with a test laser cut on cardboard then went on to cutting directly out of the sheets of acrylic, cutting out each side of the cube in a different color. I found that it took 4 runs of the laser cutter to cut all the way through the 1/8" acrylic following the settings in the chart on the cutter's computer desktop (note for future). I wonder if that means 1/4" thick acrylic would take 8 times to cut through?

I also got acrylic glue from Canal Plastic, and will glue these at home. I wanted to try making a baby block for my friend's new son, Leo, out of transparent yellow with a lion on one side and L, E, O on three of the other sides. I'm not sure if I will be able to do another session in the shop before class to do that before class...also wondered if the acrylic glue is baby safe? But I got to rasterize the letters "ITP" on some cardboard, though I didn't etch anything onto acrylic yet. I felt some pressure to get off the machine so other people could use it and wasn't sure if etching was necessary for this week's assignment. I now feel confident that I can do more projects with the laser cutter of different kinds though. Perhaps I will also try etching something onto the cutting boards from last week or individualize them for people. Just need to figure out the scheduling thing and how best to handle/bypass complications regarding that. 

First 5 pieces of my cube. I also got some acrylic glue at Canal Plastic and will glue these at home.

First 5 pieces of my cube. I also got some acrylic glue at Canal Plastic and will glue these at home.

All six sides of the cube (one will be the "top" of the box...) 

All six sides of the cube (one will be the "top" of the box...) 

Final cube - pre-glueing, but held in place: Cat for scale.

Final cube - pre-glueing, but held in place: Cat for scale.

Note: After glueing the first 5 pieces together, I realized I also could have laser cut some pieces or made a bent a piece and glued it on the top piece as a handle and also to indicate the top and how to open it. Because otherwise it is a little fiddly to open and close.

Day 65 @ ITP: Animation

Storyboard for After Effects Animation w/ Dom C.:

Keying out Dom's full-body green screen suit! 




Update on 11/11:

We will also be incorporating another character (possibly a skateboarding sloth) with limbs that will move around, following what we did in class this week. The green screening may or may not happen and we realized it wasn't really necessary for completing the assignment. However if we do get that footage we will incorporate it anyway. Either way something with a skateboarding character or two characters (either two animated characters, or one real/manipulated character and one animated) will most likely be part of this project... 

Day 64 @ ITP: ICM

Final project proposal:

For my final ICM project I plan to make 3-5 audio-visual collages that can be manipulated by the viewer. I would like the viewer to be able to change them visually as well as sonically. 

I am trying to keep my expectations for this within reason so I can code them pretty much myself with some help but my goal is to have mastery over the basic elements of code involved here for these so that I could replicate more of them and potentially add onto this idea later and develop it. 

I'm imagining a canvas for each, which may be a video going through the slit scan effect from the last assignment, if it's possible to put images on top of a video in the background(?), and some elements in a scene, probably abstract ones, that can be clicked on then dragged around to sort of "collage" or arrange them. Meanwhile a background song or audio layer would be playing, and when the viewer clicks on each element, a new layer of audio would be introduced or taken away, or a one time audio sample would be triggered. I would also like to have one element in each sketch that would add some subtle audio effect to the sound when it is dragged in an x-y axis. This interaction may be pretty simple but hopefully fun tool for interacting with a piece of music.

I got some feedback in class to check out Bjork's ReacTable-- here it is, to watch later!

Reactable - How it works (High Quality) - Bjork - ONE LOUDER! Reactable is a collaborative electronic musical instrument with a tangible interface based on a table, and inspired by modular synthesizers of the sixties. It was developed by the Music Technology Group at the University Pompeu Fabra in Barcelona.

For my Phys Comp final I am working with my partner in class to create a visualization of breathing exercises in a sort of computer program which we plan to make using P5, to be used for meditation, along with a sonification of the user's heartbeat which is triggered from a pulse sensor using an Arduino. We will also be creating a sound library of samples using P5 which will be triggered by the heartbeat data coming from the Arduino. So I will be working in P5 for that as well, but some of the math is beyond my capabilities in the code. However I think I will learn a lot and it will be fun to brainstorm the types of interactions or interface we can code using P5 as an engine.

Update on 11/13:

I wanted to add my plan for completing this project.  I went back and forth between thinking that I should just focus on the P Comp project and my role helping with P5 for that instead of doing a separate final for ICM, but I think that I could create at least 3 of these musical sandbox ideas for the final, and would feel good about doing that. It would also be more based on code that we learned in this class. This is how I plan to do it, and I am scaling back what I will do based on the fact that I will not have all that much time: 

For Nov 15: 

Gather images and sounds for compiling/working with for 3 different compositions. Keep them simple. Some loops and some individual sounds (maybe this can be in an array so I can switch which one gets triggered?) Choose whether there will be video in the background or just one slit scanned image in the background that repeats and maybe changes how it repeats upon refreshing (or not)... 

For Nov 22: 
No class. 
Meet with Aarón for advice. Maybe focus on the interactions between the sounds based on where they all are on the canvas. Get an image slit screening in the background and place images on top to drag around and trigger sounds with  (ambient loops, and one hits in an array).

For Nov 29:
Come up with a pleasing interaction between the sounds. Play with different images and sounds. Figure out what is not working if anything and make a plan to debug it.

For Dec 6: 
Do any debugging. Meet with a resident for help debugging if there are any issues. Tweak the visuals and sounds as needed (think of cooking like salt and pepper at this point). Think about the interaction. Is this a way to showcase music or to make music? Do a blog post.

For Dec 13:

Prepare final presentation. Demonstrate how it works and have people in class play with it maybe as part of the presentation if they would like?