Making Interactive Art: Set the Stage, Then Shut up and Listen

Though the author of the piece is very much focused on interactive art, it seems that the article can be solidly placed in a discussion of both design and authorial intervention. Tigoe talks about authors providing basic context then letting the viewer/user/reader experience the work by themselves, and the way for the artist to do that is by using clear design choices to show how to interact or experience the art piece in question. In that respect, Tigoe seems focused on using clear design principles in interactive art so users/readers/viewers know how to approach the art and what meaning to derive from the interaction with it.

Apart from this, there is also a vein similar to Barthes’s Death of the Author, the idea that, once a piece of art is out in the world, it is up for viewers (or readers, considering the piece was originally written on literary work) to decide what meaning the piece holds through the piece itself, rather than through what the author/artist has to say about the piece. In a way, the authorial desire to say what the piece of art means takes away the agency of the reader/viewer/user. Instead, the piece should enable the reader/viewer/user to create meaning through the parts of the piece.

Physical Computing’s Greatest Hits (and misses)

In this article, Tigoe lists and explains a variety of physical computing projects and the multiple possibilities that a physical approach affords to both the user and the creator. All the possibilities seem, with my current skill level, manageable, though I wonder what, exactly, constitutes a “good” project. Put other way: what is a “hit” and what is a “miss?” Is a hit more complicated than a miss, or more simple? Does it matter? It seems that the baseline for a “hit” is how interesting it is and how many affordances of interaction it gives; in other words, something that is not entirely simplistic in how it responds to a user doing something. The hits seem to be those that mix a simple feature and add an interesting aspect to the mix, or alternatively something that takes inspiration in something simple and enhances it (like the drum glove).

Concept

My initial idea was to make a violin tuner, but that was too simple, so I decided to use a potentiometer to be able to use the buttons for multiple different notes with the purpose of extending utility beyond a violin tuner. Thus, with the potentiometer and six buttons, I made a tuner with 4 settings: violin, viola, cello, and guitar (where the extra 2 buttons actually do something). I originally wanted to include the notes for a double-bass, too, but they were too low for the piezo buzzer to play accurately (it was a low rumbling; it would be hard to tune an actual instrument with that).

So, in short, my project is a multi-instrument string tuner that should be redemptive for past mistakes with multiple notes on the buzzer.

Schematic

Midterm – Tuner Schematic

Code

You can access my code through GitHub by clicking here.

Behaviour

For the most part, the tuner works just as intended. There are, however, a couple of hiccups whose origin I can’t find. Every button controls a note that the buzzer plays, and changing the potentiometer’s position (indicated by the LEDs) changes the notes the buttons play. Setting the code up was relatively simple, and the breadboard prototype worked perfectly.

Building the cardboard structure took some time, but using hot glue and bending seemed to work fine for a basic structure. Since my project has 6 buttons, 1 potentiometer, 4 LEDs, and a buzzer, cable management and soldering all the pieces proved to be the most time-consuming activity. I gave up on the aesthetics of cables after connecting 2 of the buttons and just tried to keep everything neatly organised and easy to track. 

Though I did try to make the front look aesthetic, it kept falling forward due to the weight of the buttons and the knob on the cardboard, so I had to use cardboard bits to “box” it, in a way, but even then it didn’t hold as well, which is why there is now tape on the sides.

All in all, 3/4 of the instruments I added work as they should, and of the 1/4 that doesn’t work, 4/6 buttons work, but an LED flickers when it shouldn’t.

Problems

The setting for the guitar, as a whole, posed a problem for some reason. The circuitry and the code seem to be just fine, but the last two notes of the guitar setting don’t work as they should. There seems to be some interference with the other notes, or with other parts of the project. In the previous project, the problem seemed to be on the abundance of noTone commands; in this one, however, there is only a single noTone command, so that option is ruled out. The circuitry seems to work just fine, and so does the code. The buzzer plays all other notes correctly, it is just those two notes that don’t work.

Likewise, a red LED flickers when a button is pressed on the guitar setting, but not at any other point. I also haven’t been able to figure out why this is the case. The digitalWrite for the LEDs is entirely dependent on the knobs being in positions 1-4 (the knob was mapped to go from 1 to 4), and there is no reason in the code for why that LED would flicker, though it seems to be related to the specific note of the buzzer. Seems to be, to some degree, that the interference is electric, but why specifically the red LED?

Photographs

View post on imgur.com

View post on imgur.com

View post on imgur.com

View post on imgur.com

Video

Midterm – Instrument Tuner

Lessons Learned

Using noTone only once produces better results when producing multiple notes through a single buzzer, as opposed to using noTone every time I needed to silence the buzzer.

Soldering takes longer than I thought it would and was, easily, the most time-consuming part of the project, but now I can use it, even if it is not at the best level.

Stripped wire is usually bad, covered wire is usually good.

Sometimes connecting too many things to a single function or chain of functions can seem to create some sort of electric interference (as was the case with the LED), and sometimes things just don’t work and refuse to reveal the secret to why they don’t work.

Concept

Since both of us are video game fans and Undertale fans in particular, we decided to make a homage to Megalovania, a song from the game that has achieved fame as a meme.

Our project, then, was to make a keyboard whose notes only enable a playing of the first part of Megalovania (limited by the fact that we only had access to 8 buttons and 2 breadboards put together).  We planned on using the Servo as a percussion component, controlled by a potentiometer.

Schematic

“Megalovania” Keyboard Schematic

Program/Code

We have very long lines of comments and code, so the preformatted option ended up messing with the formatting. Instead, you can access the code on a GitHub Gist by clicking here.

Behaviour

“Megalovania” Failed Attempt (Click for Sound)

Every button is (supposed) to play a different note, and since it is an instrument, the Servo serves the role of a metronome or beat-keeper of sorts whose speed can be modified with the help of a potentiometer.

Even though the keyboard is weird in that its notes are B2, A3, C3, D3, F3, G3, GS3, and D4, its whole point is to be able to play the first three bars of the score of Megalovania and nothing else (of course, there may be another song that you can reliably play using these specific notes, but our purpose was to make it for that one song).

Problems

On a first test with the buttons, the keyboard worked as it should. As we added the Servo and a second speaker, the notes stopped working, and no matter what button we pressed, the speaker would always produce the same sound. The one exception to the rule was the last button, that sounded like static. We tried changing variables, using digitalRead instead of the integer we had set up, started defining notes using their frequency values instead of their NOTE_ identity.

Then we tried using two speakers, splitting 4 notes for each speaker. The result was the same, except now we had 2 static buttons instead of 1. As we kept messing around with the code, eventually we had 3 static buttons instead of 1. Interestingly enough, they were always the last buttons. So first it was the one on pin 9, then pin 8, and then pin 7, while pins 2 through 6 kept producing the same note.

Then we tried using a new file. Two buttons controlling two separate speakers and playing one note each worked just fine. As more were added, however, the same thing happened.

At first we thought it was the Servo messing the file up, but as we got it up and running we found that everything remained the same, even though it started out working oddly once we added Serial.print to see what it was doing. It seemed like the velocity of the Servo was directly controlled by the value used in Serial.begin(). Once we removed all Serial commands, everything was good with the Servo, but the notes were still broken.

Sparkfun’s online tutorial for it used a weird arrangement where every button was connected in such a way that there was no need for 5V. The pin cable connected to the switch, then the switch connected to ground; no resistance or electricity. Their code, likewise, had an if statement relying on buttonState == LOW instead of the seemingly logical buttonState == HIGH. We tried this, but it didn’t work. A tutorial on YouTube used while statements to get it to work, but we also needed the Servo to keep running in a manipulable state, so we had to stick to if statements.

It is not a problem with the speaker, as we had thought: we set a code for it to just play notes at the setup without us pressing any buttons and it worked just fine. The problem arises at the time of having it play multiple sounds from multiple switches, even though there was a single instance where it worked just fine.

I find Bret Victor’s article and follow up interesting. It is true that, as technology has moved forward, we have abandoned the “physical” and turned, instead, to the glass screen (as seen not just in tablet laptops, but on the fact that phones have shifted from having a physical keyboard to being just one big screen). Bret Victor suggests some sort of haptic approach that enables our hands to feel what’s happening in a more meaningful way than just touching glass. How does this work?

The first thing that comes to mind when I hear the word “haptic” is haptic feedback —sometimes called “HD Rumble” and, at times, mixed with accelerometers and gyroscopes— that video game consoles like the Nintendo Switch have implemented to enhance the user’s feeling of touch while playing. A particular example that comes to mind is in Nintendo’s game, 1-2 Switch, a party game meant to serve as a technology demonstration for the console and its controllers. One of the mini-games included in the collection shows a picture of a cylinder on-screen while asking how many balls are inside it. Then, the players hold one of the controllers and tilt it side to side, using the vibrations the controller produces to guess how many balls are sliding around inside the cylinder. To a player, it is just a game, but the technology behind sensitive vibrations, the positioning of the controller relative to previous positions, and the speed at which the player is moving it.

Although Bret Victor’s vision of the future goes way beyond haptic feedback, it may be the case that, as Pictures Under Glass technology may be a transition stage to something better, there may also be an awkward transitory stage of Pictures Under Glass But With High Quality Haptic Feedback that may, or may not, evolve into something more advanced that enables us to do more than just rely on haptic feedback.

The question of what this new technology will look like, however, is unknown. 8 years after the article was written, cell phones are more and more screen; their processors and systems are more potent that ever; their touch screens more precise now than they were just half a decade ago. And yet, are these upgrades? Nothing has actually “changed.” They are still computers with touch screens, just better now that they are more precise and powerful. Even now, development seems to have stalled, with companies like Huawei and Apple releasing different models of the same phone with a “Pro” addendum whose only add-ons are a better camera and more processing power. The giants of technology seem to be stuck in a war of who can release the phone with the better camera instead of who can innovate the most.

At the end of the day, it’s all about what sells the most.

In an attempt to map what is “interactive,” Crawford chooses to define it as one would imagine a conversation: two (or perhaps more?) actors that listen, think, and speak, be it literally or metaphorically. This approach allows him to better map what “interactivity” is. In the language of computers, one actor gives a command, then the computer reads it and delivers an output. In our case, we tell a computer what to do, press a button, and see an LED light up. It is interactive insofar as we told the computer not only what to output, but also how to think about our input. For programs we don’t make, the interactivity is clear, but the fact that the computer thinks like we make it think seems to pose some sort of a problem. Regardless, Crawford defines interactivity by degree, too, which gives interesting allowances. As a literature major, I can say books aren’t interactive. However, given the conversation idea, I wonder if there is anything “interactive” about referential work and intertextuality that could help perceive literature as more interactive than it seems.

Norman explores what goes into design choices as well as giving a survey of what, exactly, goes on in design. Although the ideal form of design where all interests collaborate with each other is surely a goal to look towards, Norman also acknowledges the multiple constraints that stop good design, be them economic (need for profit) or ignorance of design principles. There are parts, however, where the text strays away from what to do for good design. Thus, when mentioning the examples of sliding doors or what kind of feedback to use, the text goes on without saying what would be a good alternative and show acceptable design choices. However, since the text does give a very good explanation as to what makes for good design, it allows for a better conceptualization of projects that actively go against good design to satirize bad design.

Concept

A potentiometer controls the brightness of two LEDs, a red one and a green one. It turns the green one on after reaching the max brightness on the red one, turning the red one off after it. On the other side, a switch controls two LEDs of the same colour. When pressed, it turns on the LED currently being turned on by the potentiometer.

Schematic

Analog/Digital LED Schematic

Program

I haven’t found a way to embed it, but you can access it by clicking here.

Behaviour

Analog & Digital LED Circuit Behaviour

It works by showing the potentiometer’s control of brightness and the switch’s turning on and off the adequate LED. It works as expected in the latter stages of the project. It fulfills the assignment quota, since two LEDs are controlled by analog means and two other LEDs are controlled by digital means.

Problems

I originally wanted 4 LEDs for each, but the problem with arranging cables made the solution impractical and aesthetically unpleasing. On top of that, the problem with having 4 LEDs on an analog sensor is that the latter two LEDs kept turning on at the same time or behaved weirdly due to issues with the mapping on the input and the output.

Once it became two LEDs, the problem persisted where LEDs would reach max brightness then turn off and reach max brightness again before the other LED turned on. This was fixed, however, by programming the “analogWrite” to consider the “knobState” and divide it by 2 instead of using the “knobState” on its own.

As soon as the green LED turned on, however, even turning the knob all the way down would leave a very slight light on the green LED. This was fixed by adding an “else” to the “if” stating that the green LED should be set to 0 via “analogWrite”.

The button’s LEDs didn’t work with the bools at first; turned out the answer was to remove the “bool” inside the “if” statement and only add the name of the bool condition.

An unexpected behaviour once the switch and the potentiometer worked as expected was that, although the switch’s LEDs only turn on when the respective LED on the potentiometer part is on, if the switch is pressed as the potentiometer transitions from the red LED to the green LED or vice versa, it leaves the previous LED on while changing control to the other one. 

For my Arduino project with multiple inputs and outputs, I used the three tools we learned: a potentiometer, momentary switch, and a light sensor. I set each one to light up a different LED. For simplicity’s sake, I used red, green, and blue LEDs. The schematic looked as so:

Circuit Schematics

I used a mix of analog and digital commands to account for the fact that the light sensor and potentiometer allow for it, while the digital command was saved for the switch. To avoid over-cluttering on the breadboard, I used short cables to connect the + and – columns on each side of the breadboard and give me more options for cable management. Even with all that, however, the full setup ended up looking like a mess:

Multiple I/O LED setup

As the schematic shows, the button controls the blue light; the light sensor controls the green light; and the potentiometer controls the red light.

I didn’t set it all up at once, however. I started with the switch, then the light sensor, and then the potentiometer. I was having trouble with the program not reading the different names I gave to integers. Reading tutorials, however, I figured out that the problem was in the positioning. Putting the integers in the setup didn’t allow for the loop to read it, but putting it before worked.

Later, considering the fact that the light sensor and the potentiometer send an input from 0-1023 and the LEDs take values of 0-255, I tried using a map command on both the constant integer assigned to the sensor and the variable assigned to the sensor’s reading, but neither of them worked. So I wrote the Write command with the value and divided it by 4 to make up for the difference.

Multiple I/O LED Demonstration

I couldn’t figure out a way to embed the code, but you can access it by clicking here.

I am a frequent stair user. Since I live on the third floor, it makes sense that I’d favour stairs over the elevator to go up or down a single floor. Even when I was assigned to live on the seventh floor by a stroke of bad luck, I’d make a conscious effort to take the stairs down.

The stairs between floors on the residential buildings on campus, however, aren’t as neatly designed and pretty as the stairs between the ground floor and the Highline. Like many multi-floor residential buildings, they rely on practicality to squeeze as many rooms as possible with the space they have, so stairs end up becoming stairwells.

Stairwell

Though they may not be the peak of stair aesthetics, they have a charm of practicality around them. When walking through them alone, these stairwells never fail to remind me of Julio Cortázar’s short text Instructions on How to Climb a Staircase, where he describes the steps necessary to reach the top of a staircase. Namely, one takes one foot and places it on the first step, then takes the other foot —which, although different, shares the same name— and put it on the step above, repeating until you reach the top.

The stairwells, however, are hardly empty. With the to-and-fro of students going and coming to class, the stairwells can become quite busy. In such situations, visual communication is key to getting through. Architects, in their search for more room space, forgot to make the stairs wide enough to fit two people. This isn’t a problem for when multiple people go in the same direction at the same time. All you need to solve that is a line. The issue lies in the clash between someone going up and someone going down. Normally, one would walk down the middle of the stairs so as to avoid uncomfortably brushing against the wall or the railing. When met with someone going the other way, however, a quick adjustment is necessary. The rules of courtesy don’t apply here, since apparently everybody has a different idea of what “courtesy” means when it comes to stairs. Some people keep walking down the middle and expect the other party to change the orientation of their body and squish over to the side to make space for Royalty Passing Through. Others are more considerate and pick a side. Others are even more considerate and do a non-verbal agreement so both parties turn their shoulders 90 degrees and squish on the wall so both sides can go without any trouble. The more interesting interaction, however, is when both sides walk normally and pick sides. I always assume, given how much of our lives we have sacrificed to cars, that picking the side where cars drive (which is, in most cases, the right side) would be the logical solution. My logic, however, doesn’t seem to stretch beyond a few people. The majority of my encounters have always relied on some form of non-verbal negotiation where both parties have to awkwardly snake around until one side decides what side to pick and the other can adapt. Once a side is picked, that decision must be respected, as breaking the unspoken contract would result in a crash. Although not as catastrophic or life-dangering as a car crash, it is, nonetheless, annoying and pointless to go through it when it is so easily avoidable.

In conclusion, stairs are adequate tools for vertical movement when alone. When met with someone else, they breed interactions centred on spatial negotiation and unspoken contracts that must be upheld if all is to go smoothly.

For my handless switch, I decided to use my eyebrows. I took inspiration from the moustache switch we saw in class, but decided to use the fact that eyebrows get closer when furrowed to create a switch.

I took some longer pieces of cable than the ones that came in our packages (so they could reach all the way from the breadboard and the Arduino to my face), connected one end to the boards, and wrapped the other end in aluminium foil. I then proceeded to stick the rest of the foil using tape to the top of my brow, and the result is shown in the video below.

Eyebrow Switch

Although the result looks simple enough, I struggled to find a design for it. At first, I thought I’d just use cables with stripped ends instead of aluminium foil, but found that cables don’t tape to faces very well, particularly if they are connected elsewhere. Then, I had the idea to use a loose cable as the switch, so that whenever I moved my eyebrows, the stripped bits of the cables would touch and form a circuit. This idea, of course, also didn’t work. I then realized that the problem was that the area of contact between cables was quite small, so I turned to copper tape. However, since it was only conductive on one side, trying to find a contact point between two strips using just my eyebrows proved just as hard. It was then that I came upon aluminium foil, which proved easier to use, since all its area was conductive.

The circuit used is the same as seen in class, save for the fact that one of the connections (namely, the one between the Arduino’s 5V connector and the breadboard) was split to allow for the eyebrow switch.