Here’s what has happened since we last spoke:
Shortly after we first playtested our idea in class, I realized that the project had somehow evolved away from the original idea to the point where I wasn’t having fun with it anymore. We decided to return to the original idea of having a two way conversation and allowing the machine to do all the work in between instead of trying to force human interaction into the process. We had been playing with things like dictation, text to speech and language translation in p5 so it wasn’t difficult to change gears because the work we had already done until this point would still apply.
My original idea was to create a complex chain of communication technologies that would talk to each other and ultimately allow two people to converse over a distance through this chain. I liked the idea of different technologies talking to each other instead of being individual interfaces for humans. I also wanted to explore the idea that technology is supposed to make communication easier or more efficient but it also can make communication convoluted and distorted.
I took inspiration from Rube Goldberg machines and whisper walls, which both provide a very tangible experience and elicit a sense of wonder in their own way. The Rube Telephone makes communication more tangible, presents it in a fun way, and hopefully prompts people to think about how technology affects how we interact and communicate with others.
Here are a few things I’ve learned about in making this project:
- p5 web speech library
- Google Translate API
- OCR and the newocr.com API
- node and various modules (socket.io, serialport, python-shell, etc.)
- writing a morse code generator from scratch
- arduino serial handshaking
- rotary phone circuitry
My partner Aaron Montoya has been focusing more on the audio routing side of the project using Max and other audio software, and we have been working together on the circuitry related work.
Here is how it currently works:
- Someone speaks into one of the phones.
- p5 web speech converts what was said to text and then translates that to Morse code.
- The Morse code is tapped out on a Morse key by a solenoid and when the Morse key is pressed down, a piezo plays a tone.
- p5 translates the original English sentence to Spanish and this new sentence is spoken out of the speaker in the center.
- p5 translates the Spanish sentence back to English.
- The Arduino prints this English sentence on receipt paper.
- A webcam takes a photo of this printed text and the newOCR API pulls the text out of the image.
- The English text from the translation step is spoken out of the earpiece of the other phone.
- Reverse the flow (OCR > translation > Morse) when someone speaks into phone 2.
- Improve Morse and OCR steps and use the results from them.
- Improve behavior of rotary phones (ringing, hold music) and implement light bulb progress bar so users know what’s happening.
- Use Google search suggestions to make sure the Morse and OCR steps result in real words.
- Improve visuals/experience for language translation and OCR steps.
- Improve audio routing.
- Build an i/o box to connect each rotary phone to power and audio cabling.
Code to be posted soon!