For this project, I wanted to implement user interaction in some way using multiple clients. Initially, I intended to create something of a sonic landscape using gravitational fields. Each client, when joined, would create a gravitational field that would morph the ground around them. This would create sound sort of tone that would then combine with the other clients to make a chord. Movement of the clients makes different tones, in turn making different chords.
I ended up actually doing something entirely different. Communication was still the goal but I sought to do it in a different way using the medium of text. At the start, I intended to make something that would resemble leaving different notes on a shared table. Users could come and leave a virtual note that would be represented in the canvas by a 3D object. That got me thinking: what if I rethought email? Three JS enables the ability to generate a 3D space, and so I thought about how to represent objects messages in a 3D way.
The final result of this project is a proof-of-concept email viewer that lets the user browse their emails in a 3D space using the mouse to click and navigate. The 3D objects in the space represent the sender’s profile image. It helps the recipient distinguish each mail from each other. When a user clicks on an object, a message viewer is displayed on the right side of the screen showing the contents of the message, the subject line, and the sender and receiver. Below the 3D inbox is a field where the user can type in their email and send it to a recipient. When sent, a new 3D object is generated on a random position on the sender’s screen, and the email would then appear in the recipient’s inbox in the same exact location.
The project uses Firebase Realtime Database to store messages in JSON format. Its extremely rudimentary and isn’t robust for a full scale mail system. Messages between users are copied between each user, so each user has their own copy of the conversation in a conversation.
I was going to use two canvases at one point. One for the inbox and one for the message viewing. I got the two canvas system to work, but the problem was that it took too many resources on my computer to run both of them simultaneously, especially when there was a post-processing pixel and bloom effect being applied every frame. The second canvas would be the message text rendered in a 3D space, which had dynamic lighting. I used two different methods to render the text. The first was using the default Three text library to render. The API for the library is not effective and the documentation for it isn’t useful, especially when trying to load in a custom font. I then tried the Troika 3D text library for Three and that worked alright. The text was rendered on a 2D flat plane and had no thickness, but the library was very easy to work with and you could load custom fonts with ease. The font and text rendering was an extreme hassle to work with, so I took it out entirely. The message viewing canvas code can be found in `src/lib/init.ts` under the function initSubThree.
I wrote the project in Typescript and React. I opted to use Typescript because having types for the various Three JS library components was going to be a massive help, especially since their documentation had references to examples that had no explanations. I used React to manage state, which was handy when I implemented a username input and displaying the messages on a different HTML element when the user selects a new object. There were no updated tutorials on how to integrate React, Typescript, and Three JS together into one project, so I had to figure it all out myself. Managing state with React was a challenge for me at least, because I’m not used to React and its idiosyncrasies.
In regard to code, I implemented a technique from functional programming. I created a builder function that returned another function, which makes creating a scene much easier. You can see this here:
I read the code for creating an animation loop from the examples given to us. I then just translated this idea into a builder function that would render an update function passed to the renderer object.
There’s a lot of places to improve this project, code-wise but also conceptually. I liked the feedback from the teachers. I’d like to implement more visceral animations for inbox functions, for example, when a message comes in, perhaps it doesn’t just pop in but rather animates in somehow. Or, there could be a way to “organize” the inbox in 3D space, placing things in certain areas, etc. to have typical inbox sorting abilities. I could also just make this 3D inbox concept an electron app and make some kind of email client with it.
https://drive.google.com/drive/folders/1_Wb36xEmQNKMaJE-WkzQ4X_l1q-eYzMc?usp=drive_link