My dream is to create a space where people can create audio/visual objects that communicate and talk to each other and for them to rapidly change perspectives in the midst of the creation process and viewing. I hope to create 360 scenes or fragments of scenes that would live in varied places and could contribute, change or disrupt the order of those specific places.
My vision is table, a chair, a sofa, a bench or any other space for that matter that an imaginary ecosystem live atop of. This ecosystem would be comprised of living like organisms that have life of their own. They move, make sound and interact with each other in symbiosis. viewers of this ecosystem would be able to contribute their own instances to the ecosystem by hand gestures. Viewers could also change their points of view of the ecosystem and by this would change their visuals and sounds accordingly.
My goal is to create an augmented reality multiplayer app for the creation and viewing of coral reef like ecosystems
that are both animative and generate sound. Each ecosystem will have its own distinct features created by hand gestures of the viewers along with random features dictated by the platform. these ecosystems would be like audio/visuals compositions in space that could be viewed and heard from numerous points of view.
Polynote3D
polynote is a multiplayer musical experience generated by random notation between multiple players on a webpage. Each player in the session is randomly assigned a different instrument as he begins the experience and is given a random set of instructions to build his part.
Inspiration:
https://www.are.na/itay-niv/development-studio
R&D:
First stage: To get myself familiar with the technical side of things I started with something simple. I knew I wanted this experiment to live in the internet so Javascript was my main tool. I built a really simple experiment made out of an html table that played sounds with a webAudio library called tone.js. The sounds played in a loop as soon as pressed and stopped when deactivated:
Second stage: After having a basic sound playing in the browser, I started building a mechanism that played sequenced noted through time – a basic sequencer.
Adding notation + polynote2D
- Adding a server that keeps everyone on the same page.
- Adding random notation for different players and restricting the interaction to a specific time frame.
Third stage:
Translating to 3D:
For this part I started refactoring my 2D code in ThreeJS. a lot of things in the architecture of the code have changed in the transition,
Stack:
Node.js
Express
Three.js
Tone.js
HTML / CSS