Author: vim215

The ending process of this was much more complex than the beginning and middle. Which I totally called by the way. I fully remember being scared that my lack of issues in the beginning meant I would have issues later. I WAS RIGHT. 

Really, the only issue I had was very dumb on my part. I played a bunch of audio but didn’t tell it to stop playing, so I had a big error where it was all playing at once. All I had to do was tell it to STOP and then it worked. Yay! 

I also had a bit of an issue with mapping the volume. If I had more time, I should have put in like a little calibration mode for the human presence sensor. Because I didn’t do that, sometimes the mapping would start at 200, other times it would start at -800. It was really inconsistent and would have worked better if I calibrated at the beginning of the performance. 

Ultimately, I think this project was a bit lacking. I think I should have incorporated more of a sensor aspect to it and made it a bit longer. I think I approached it with, “Let’s do something simple so it’s good and I don’t struggle with it too much” but I think I went too far on the simple side and not enough on the intellectually stimulating side. After the performance, I heard that people were wondering how it worked with sensors. Yeahhh I think I should have made it more clear and expanded the idea a bit. I think it stayed at like 50-60% development but I wish I had developed it more. 

But alas, the performance went okay. There was a major glitch with the audio which I’m still not sure why it happened. I think maybe the tree moved a little and the speaker got a little jolted, which caused the radio static to turn on. I think it still worked, but it didn’t really work as planned. It was also hard to hear the mapping to the volume because my performer was a little bit off the sensor. Or maybe there was something in the code that could have made it more clear. 

Here’s the updated code!

I was unable to download the video due to lack space on my computer and it exceeding the monthly allowance for vimeo. But hey there’s a video! Check it out. I can share a link if anyone other than people in the class want it. 

 

I’m at an interesting point in my project where I’m not hitting much frustration. As great as that is, I feel like it’s going too well. I feel like I’ll hit a road block soon. I mean, I did a road block, but I’m confident that I’ll figure it out with Aaron’s help. 

I set out to do a few things for this deadline: 

1. make the speaker 
2. create a timer for the hyperventilation
3. make a key press for the “Pose/Superhero music” 
4. set up the human presence sensor and test the range I’d use for the stethoscope 
5. test how the serial communication is working 

One thing that I am a bit stressed about is the fact that I don’t have an actual tree yet. Prop Loan has been closed since the last class, so once they reopen I will go and check out what they have. Then most of my time will be spent on making that. If Prop Loan has nothing, I think I’ll ask the convenience store for some cardboard boxes that they aren’t using. I plan on going to prop loan tomorrow either during class (Wednesdays they are open 10am-12pm) or Thursday right after my class (5:20 it ends, Prop Loan closes at 6:00pm). 

Okay now onto what I’ve done for this deadline! I’m actually feeling okay about it. Could I have done more? Yes. But did I do a lot? Also yes. 

This was towards the beginning of my working. Here, you can see the Hyperventilation timer on for a few seconds and then turn off. Right now I’m just working with cout<< but this will then trigger the bird sounds to turn on at a set volume. The hyperventilationOff will then, in turn, shut off the bird sounds. 

You can also see the light turning on and off in response to the complete switch becoming incomplete. When I was first working with this, there was a bit of a delay. I’m not sure what was happening but I would remove the plate and then a few seconds later, the light would turn off. Later, I fixed this issue. I don’t know what I did but in the final version of the serial communication that I used, the light response was very fast! A miracle? Maybe!

To trigger this, I’m using a piece of cardboard that has copper tape in strips along the sides. In the middle is a small square that has some of the copper tape leading into it. When I put a little copper square in the middle of the two strips, it completes the circuit. I wasn’t sure how to get a valuable reading on this, but after trial and error with a few different sensors/inputs, I decided to go with a touchRead. So, I put one end of the circuit into pin 3 and then the other end into power. This gives me a giant reading of about 65000 when the circuit is complete. However, when the circuit is not complete, I get a reading of about 1000. This makes it very easy to distinguish between whether the circuit is completed or not. I’m using the little LED just as a way for me to easily see if the circuit is completed. This will be important come performance time!

Here is what I ended my work with. You can see the light response is much faster now! I’ve turned off the hyperventilation timer just so I can see the readings that I”m getting on the human presence sensor. 

One issue that I’m having is the serial communication with the human presence sensor. In arduino, I’m getting readings that range from -900 to 6000, however when I send that over to openFrameworks, I get very jittery readings that range from 0-120 and don’t seem to really react to any adjustment of the human presence sensor. That is to say I can’t find an easily map-able range that would control the volume of the bird noises. Why is this so jumpy? Do I need to add smoothing? Is this something I can fix? Why are the oF readings different from the arduino readings? 

The code is vaguely a mess right now with things being commented out to focus on other aspects, but here’s the link!

Okay so after a very sad email to Aaron and a few anxiety attacks, I got probably the funniest response from Aaron and got to work. He told me to specifically- 

1. Write a Script 
2. Watch Stranger Things Season Three
3. Break out into a crazy musical number 

So I did. 

Step one- write a script. Here are the highlights: 

The scene opens up with an actor, no gender preference, sitting at the base of a tree. They are reading a book.

They reach into their bag and pull out a tie. They put the tie on. They are now The Bird Detective. Their only mission is to investigate the birds in the palm trees. 

The actor poses, maybe a fun logo plays with music (like a superhero theme). They reach back into their backpack, ready to get started. We realize that this is not an academic backpack, but a bird backpack. 

They move the stethoscope around the tree, where it moves, different parts of the bird noises are amplified. A tweet, a chatter, A twiddle. Idk. Bird things. As they move the stethoscope up the trunk, towards the top, the sounds get louder.

They take a step back, take a deep and audible breath, and start to twirl the lasso.

They find a bird, reading a book. And that book is entitled, “How to make Students Less Stressed” 

Yes, there are silly elements, but doing this script really helped. I think it got my creativity flowing really nicely. I was able to work through what I wanted it to look like for the most part. I feel like maybe I should have done that, like, a week ago. 

Step 2 was watch Stranger Things and I actually did that while I was writing the script. Hell yeah multitasking. I didn’t draw any direct inspiration from it, but hey, I’ll keep watching and maybe it will help. 

Step 3…the musical number. I do not think this has a place in my performance but I did aggressively play guitar when I got frustrated. 

I think the ideas are shaping up pretty well. I’m concerned about the look and honestly how to do a lot of the things that I want to do with it? I think maybe using a camera to see the person and then having like…idk maybe different areas of the screen or different triggers. Like okay when the person hits this circle then the bird noises start or….when the person lassos the tree (how?? idk??? projection???) then the image just warps along a set curve so it looks like the tree is bending? Is that allowed? Is that possible? 

I still feel a bit unconfident with this, but I’m getting there. I did a very simple thing in open frameworks, where I just split the screen into four quarters and had each quarter trigger different things, whether it was a background change, a sound trigger, etc. Later on, I added a little circle that would make the triggers happen. And all of this would be triggered with the mouse. So maybe if I translate that to follow the person around? But do I need just a regular camera for that or do I need a depth camera? 

You are my firrreee my one desiiiiirreee (I’m singing about a good grade in this class). 

Sara and I used the pitch and roll to control the speed and volume of the song, “I Want It That Way” by the Backstreet Boys. We originally used heading too, but decided it made the movements more natural and more in line with what they affected if we used just pitch and roll. 

Here’s the code!! 

And here’s a video of it working. 

Here is my performance proposal!

SBM_Proposal

This exercise was extremely frustrating. If we hadn’t had Lateefa’s knowledge of the realsense2.lib and how to fix the absolutely ridiculous errors we get with it, I don’t know what we would have done. 

For the latest installment of this project, Mari, Lateefa, and I worked in two stages. The first, on Sunday night, was just getting the code working how it needed to. We got some good work done- changed the layout of the keyPresses to be more intuitive for us, worked more with the Projection mapping and video mapping libraries to keep smoothing the interaction, and ultimately got to a point where we could show it in class. 

However, that first night, we worked probably four hours on just these small things. Every time we would add something, we would have to combat new errors. The biggest issue was the realsense2 library. All three of us, constantly, would get an error about not being able to find this library. So we would have to go through about four steps of trouble shooting, involving manually linking it to the sketch through the properties tap. We had to do this not once, but every time we updated the sketch, which was pretty frequently once we started getting fancy with the addons. Any time we would add something new or change anything, we were always scared that we were going to get that same error. 

But we prevailed, we walked out of there with working code!! 

Tuesday was our day to install it. We got there and instantly there were issues- the biggest one was that one of our clamps was missing the screw that would hold it to the camera. So we whipped out the good old duct tape and made it work. Don’t worry, we exhibited safe ladder behavior. Installing everything was pretty straightforward and , luckily, realsense2 couldn’t mess up me screwing the clamp onto the bar. 

Once it was all up there, we got to work with finalizing it and maybe? adding some extra credit. We played with the threshold, adding a near threshold so it would see the projector as much, and worked with the mapping to get it lined up.  Then, we thought, “okay this looks great, but Aaron is going to expect us to do the extra credit”. So we proceeded to spend a bit of time trying to brainstorm and test ideas for that. Ultimately though, we were frustrated as how much more difficult this was for us as windows users and exhausted because we all had had incredibly busy days. We decided just to leave it as the lines and take the loss on the extra credit. It would have been nice, but there comes a point where you need to keep your sleep schedule and mental state in mind with these projects. 

This project is honestly the worst because the actual assignment- the code we needed to do to get it to work- took like…maybe an hour? But from troubleshooting all of these other issues, beyond the realsense2.lib. we spent probably over 8 hours on this project. 

I’m not feeling very creative right now. Doing this brainstorm was really challenging because I wasn’t getting as excited or as passionate as I usually would about this final project. Despite this, I tried to explore some different things and think about how this could be applied to a final project. 

My first step in the brainstorm was to look through the generative design book and mark the pages that interested me for this final project. I found that, when I did this, I was particularly interested in this sketch and this one. 

For the first one, with the text, I was drawn to the idea of academic pressure. I pictured possibly a dance performance going through the process of writing an essay, the highs and lows, showing the frustration and the stress and the satisfaction when it’s done. I’m not sure whether the sketch linked above or perhaps this sketch would be better for this idea. Personally, I love the look of the first one, showing the letters connecting and having the ability to scramble the words. With the second sketch, I worry about it becoming too scrambled. I don’t think the audience needs to be able to read the actual words, which I think is a good appeal of the first one, as it’s in a different language. I think the focus should be on the dancing and the text should be a complement to that. In terms of making these sketches unique, I think I’m leaning towards combining these two. Having the ability to draw the text, possibly using computer vision to replace the mouse function with a person.

The second sketch (the this one) would really be similar in the computer vision sense. I just don’t really have an idea as to the intention behind it. Right now, I’m exploring it because it looks cool. But I don’t have a higher purpose as to what statement I want to make with the performance. Which is a downside. And probably means I shouldn’t pursue it more. 

The issues I have with doing this type of performance is that like….man…I’m like 60% interested in it. I really want to work with lights in some way. I just don’t know how. So that’s when I went to processing. Because they have some cool light functions that I remember exploring last year. And I specifically remember playing with this sketch:

This is a really cool thing that plays with directional light. And honestly it just looks so cool. I have the same issue where I’m not sure what I would want to say about..the world? life?? something greater than this??? 

So this would be in a similar fashion as the others where there might be a dancer and the mouse follow is replaced with the computer vision. 

HOWEVER. All of these ideas. Seem wicked limited and, honestly, kinda lame on my part. What I really want to do is something like that video in the beginning of the year. You know..the one that fully made me cry when I watched it. 

I really feel like this is too much though. But something like this is what I want to do. Idk. Help. 

This was a very passive brainstorm. I think I’ll spend more time in the next few days changing values in sketches and playing around with different things. But for now, I think it’s good that I got thinking about what I want to do. 

Really, I want to do something with lights. That’s really where my passion lies and it’s what’s going to motivate me to work on this project more than anything. 

It’s finally done! Which is ironic, because I’m writing this when it isn’t done…in anticipation that I will eventually finish it. So if I don’t finish it…this is gonna be pretty awkward. But for now let’s focus on the positive- ITS FINALLY DONE!!!!

This idea started a good month ago now- back in September when I had no idea of the stress and sleeplessness it would cause. It was just an inkling of an idea that I scribbled down on a few post it notes to try and wrap my brain around something concrete that I could develop for the project. Having this initial brainstormed helped me find the root of what I wanted to explore in the process of creating this midterm project. It gave me direction on where I needed to expand my thinking as well as narrow down my focus.

Once I had the barebones of an idea, it was pretty straightforward to make a prototype. Get a book, add a sensor, and add audio. The most complicated part was trying to figure out whether I would play the audio through the prop shield or through serial communication with openFrameworks. I decided on the latter with help from Aaron. Once I had decided that, it was straightforward from there. Get one book done and repeat for the rest of the books. I got into a rhythm of working that made it very manageable to complete this project.

The challenges came a lot towards the end of the project. I have a war against the teensy3.2 and its horrible, fragile usb port. Yes that’s right, I broke another one. And promptly ordered one on amazon to replace it. I have been deemed Destroyer of Teensys. So no big deal, I switched to an LC but then…my project didn’t work with an LC???? I’m not sure why but at first, my project just did not work with an LC. And this was very frustrating. So I switched to a friend’s 3.2, then back to an LC. And bam, it worked. Who knows. Not me. My solution to this is for me to just stop being a dumbass and to be careful around the teensy ports.

The next challenge I had to overcome was making sure the tilt sensors were completely untriggered on the book at their resting state. A couple times, the sensor would shift slightly and the audio would start constantly playing. This is very easily fixed by just checking the tilt sensors and poking them back into place. I’ve added extra tape to make sure they don’t move too much even with people interacting with the books. However, it is very sensitive. I think in future iterations of this project, I would find a way to combat this sensitivity. I tried to add a trigger threshold, but then it wasn’t sensitive enough, even on the lowest possible value. Quite odd. Definitely something to revisit.

The reset button was also a challenge, and continues to haunt my dreams. While I was attempting to work on the project while I had a migraine, I could not for the life of me understand how to wire up a button. It was pretty embarrassing to be honest, but once my head cleared a little, I came back and reorganized some wires, giving me more space to see how the button wiring would go. The only downside to this is that I did not solder it into the breadboard, it is merely stuck into the unsolderable one, so I will frequently check to make sure nothing has come done. In future projects, making sure everything is soldered will be important for my sanity as well as the organization of the project.

It was actually really nice to work with the button because the code was very simple. ofSoundPlayer had a stop() function, which I then called upon all of my soundfiles if the button was pressed. Then, in order to make them stop until the next time they were triggered, instead of just stopping while the button was pressed, I reset all of the values to 0. With this, I am lucky to say that every part of the code makes sense to me, thankfully. Usually with my projects, I feel there is an aspect that I do not understand, but with this project, I really dug deep and understood every line.

This was especially important when I was transferring the sketch to the mac mini and xcode. It gave me a lot of problems due to contract issues and my ignorance of apple products. Throughout these issues, it was hard not to get demotivated and give up, especially with my migraines making things more difficult for me. However, I had one experience that really made it worth it.

After a busy work day, I went to work on the project and found that my friend had followed me. We started talking about the project and, once I plugged it in and turned it on, he freaked out! He loved it so much, he texted the group chat he was a part of telling everyone to come and check it out when they had the chance. It made me feel so good to see him enjoy it. He interacted with the project pretty much how I expected, softly touching the books and listening. However, since this was before the reset button existed, he quickly got overwhelmed by the overlapping audio. Waiting for it to end and then restarting the experience was fine, but I’m glad the reset button is there for people to cater their experience to.

Here was his immediate reaction to hearing the audio react to the touch:

On the complete opposite scale of human interaction was my boss, who was very interested in this project. He came by frequently to see how it was doing and, on one occasion, I got a video of him interacting with it. Now, in my blog post for Chp. 3 (A Critical Framework), I mentioned that I loved the idea of grown adults resorting back to childhood habits and behaviors. This was completely reflected in how Jon interacted with my installation. While Kevin was very soft with the books, almost as if he would break them, Jon came crashing in, slapping the books and triggering every audio he could to bask in the sound of chaos. Both interactions were fantastic and very motivating, but also completely different experiences. THIS IS WHAT I WANTED!!!

Overall, I’m extremely proud of this project. Getting it done during the busiest part of the semester, with chronic migraines, while still retaining some amount of healthy eating and sleeping habits is honestly a great achievement for me. It completely surpassed my expectations of how the installation would go. Honestly, not only am I proud of the work I did but I’m also very proud of how much I learned and took on independently with this project. A lot of times I feel like I rely on Kyle for some ideas and the physical building of installations. However, working alone was such a great challenge. I pushed myself to build it, code it, install it, all by myself. And yes, I had a lot of help from Aaron and Jack and Mari and Steven and…everyone. But I wasn’t truly dependent on someone else to share the work with me. I really think that this is the first project that I can say I made. (Although of course, the interactors are the ones who truly make a piece).

(this last image looks like it’s bad quality….basically it has the title, author, what I’m using it for, like plot or character, etc, and then it has the voice actor or recording)

I’m not going to lie, this past day was frustrating. I couldn’t get the white walls for m y location, I couldn’t get my voice actors to reply, and I was extremely busy. Working on my project would have to come in the middle of the night and I wasn’t looking forward to the frustration it would bring. 

Well, now here I am, true in some regard. It is the middle of the night that I finally have time to work on this, but I am a lot less frustrated than I expected. Because honestly, I know how to do this. It’s just a lot to do. I’m in good spirits and I need a break before I go and work out the serial communication so let’s see if I can document the first part of today. Because oh boy, a lot happened. 

I got an email that absolutely killed me. I couldn’t use the white walls. Now, I had absolutely no ide where this installation would go. I had no motivation to even think about serial communication or voice recording if I didn’t have a location, so I spent the afternoon in a frenzy around the art center, wondering about safety hazards and what I was willing to compromise in order to solidify a location. Kyle found me, frustrated, and we started talking and walking around the arts center together. What came out of that conversation was me, on the brink of tears, with Kyle’s words in my head, “I think you’re controlling the space too much”. He was right. I wanted to sync up the sounds, but that meant questioning whether the user needed to engage with a certain book first. While syncing the audio would have been cool, Kyle brought up a good point: I wanted to focus on the user’s interaction with the different books and any number of combinations of books. I didn’t want a set book order. 

Then, I slumped into the Majlis, still frustrated but now more stressed. I started recording the next day and now I wasn’t sure what my audio would be: would it be linear? Would it be jumbled? What kind of effect did I want? Did I even have a location? My boss, Jon, happened to be sitting next to me at this point. I had work in about twenty minutes and I knew I needed to get as much work done as possible on this project before my shift started. As I began to brainstorm more about location and audio, Jon noticed that I was significantly more stressed than I usually was. He inquired as to what was up. Oh, Jon, you don’t know what you’ve gotten yourself into. I told him about my location issue and he perked up. My boss had an idea: 

What if instead of having the books move in a line, we make a spiral staircase out of them using lighting boom arms? As I thought more about it, the more excited I was about the prospect of it. Once we had finished our work for the night, we went and got materials to make the structure. Together, we put it together, alternating between “designer” and “maker” as one of us stood on the ladder tightening the clamps and the other stood back looking at the aesthetic. It was a very strange feeling to make the executive decisions, as if I truly was the designer (which, I guess I am for this project). He would ask “what angle do you want this arm at?” and I would have to…..answer. Wild! The power I had was immense. And if I didn’t like something once we did it, we would just go back and do it again. It was a fantastic hour of complete motivation to work on this project. It was exactly what I needed to get excited about the idea again. 

So, now I have my structure, and it’s about 10pm. Now what? I have twelve hours to get this shit up and running. 

My first step was to just get the books up on the structure. And this proved to be a bit controversial for me. I did it in parts: the first half, I took measurements of where I wanted the book to hang, and cut the rope to fit that measurement. However, I found that once I put the books up, I used my aesthetic judgement to change the height of the book anyway. For the other half of the books, I did not take measurements, but I matched them and compared them to the other set of books. This was really interesting because it made me think about the usefulness of taking measurements and remaining flexible within those measurements. For example, knowing that I wanted the space between the top and bottom bar to be 69 inches was essential because then I could evenly space all five bars along the boom. However, it does not matter whether the book hangs 45 inches or 50 inches down from the arm. 

Okay, I’m ending my documentation here for now. My next step is to get serial communication between multiple tilt sensors up. Or….maybe I’ll just start with finding 10 tilt sensors. Wish me luck!!!

Okay I’m back. I found ten tilt sensors, but they’re the weird gold ones and not the black ones. I tested both the gold and the black with my books and I feel like the black is more receptive to touch? Maybe I just don’t understand how to wire up the gold ones. Also, I spent like an hour running tests on what would make the gold trigger, and then the freakin wire fell out. Soooo maybe my issue was just a loose wire? I’ll definitely be experimenting more with that, but if we have more black tilt sensors, that’d be ideal. 

Overall, I have it working with the serial communication so that one book corresponds to one sound. However, I’ve only done that with two books. I anticipate that a challenge will be scaling that up to include ten books. I would imagine it’s the same process of bit shifting, but perhaps a bit more complicated? I’ll definitely be asking for some assistance on that. 

I think I ended tonight in a good place. I have two tilt sensors (well…one and a half) hooked up, tested, and working the way I want them to. I also have all of my books set up on the fixture with sturdy rope. I also managed to find the soundtracks for my two underscores, a kitchen and an orchestra. I’m happy with both of them and spent some time playing with how the orchestra sounds with another sound on top of it. I’ll definitely need to tweak the volume. 

For the narration, I definitely want to play with overlaying some sounds. For example, one of the characters is said to enjoy certain Beatles songs. I’d love to have those playing underneath some of his narration. I also want to play with volume of the actors. How do they control their own voice to take attention from the users when there is more than one sound file playing at once? I’ll be keeping these thoughts in mind as I start recording. 

As for now, here are a bunch of pictures, videos, and other such documentation. 

Also- random problem that I ran into, the books are so light that they tend to just spin around wildly. I’ll put a bit of weight onto the string so it doesn’t spin around as much. That’ll also help root it in its place a bit 😀

 

here’s a diagram (oh no it looks really small) of the boom arm with its measurements. It’s 9.8 feet tall (sorry american here) and we put the top arm as high as we can and the bottom arm 69 inches below that. Then we evenly spread out the other arms from there. 

 

In reading this text, I found it very helpful to organize and reflect on projects using the 4 categories that Stern distinguishes.

1.Artist Inquiry
2.Artwork descriptions
3.Inter-activity
4.Relationality

Within the first one, I think it’s really glossed over when we view interactive art or even make it ourselves. But it’s so important to reflect on what the artist’s intention is in terms of incorporating the body and its relation to other elements of the work. After all, the artist’s intention truly shapes the piece from the beginning. Even if the concept changes through the process of making, the intention behind it- what the artist wants the user to get out of the interaction- will usually remain the same or similar.

One thing that has intrigued me a lot both in theater and interactive media is the idea of forcing the audience to return to a child-like state. For example, on page 96 when Stern talks about the weird noises that occur when people interact with his exhibits. I’m so interested in making work that forces the audience to let their hair down. Like in the installation with all of the ropes from last year where the kids could go in and play with it and get tangled in it. That kind of work where the interactor can just have fun without worrying about social norms or looking “intellectual” while they engage with a work is just so cool and so important to me. While this project I”m not focusing on that, I think my work in the future will be more geared in that direction.

Throughout the reading, what I understood as the main takeaway is to realize that with the addition of “embodiment”, you automatically must consider the relationship that “embodiment” has with every other element- society, language, etc. When we name them, we must actively fight against the total separation of the elements. Rather, realize that they are all interconnected and the effects of their collaboration must be considered. Similarly, when you involve other humans in your work, you involve society. You involve societal norms and all of the conversations that happen in modern day. You cannot avoid it because humans are ultimately what drives society. By including them in your work, you invite in their multiple perspectives and ways of engaging with your work. Most importantly, though, you invite in the different effects that your work could have on different people.