WP2WS11: Navigation with Camera and Controls

– PROJECT GUIDELINE –

Develop a sketch with navigating interactive experience manipulating camera and/or controls in three.js.

• Develop a navigation method for your final project.
  • Feel free to use your midterm project as an environment that users navigate.
  • Or utilize random(), sin(), cos(), noise() and/or recursive functions to generate environmental structures. Don’t be too ambitious with creating the environmental generative structure.
• Focus on exploring the camera and controls.

Requirements:

• Manipulate the properties of the Controls, or manually update the camera position, rotation and lookAt position.
• Utilize dat.gui.js to manipulate key values of the camera.
• Optimize the performance (i.e. Attempt to keep the frame rate Ideally above 29fps.)

 

– MY SKETCH –

https://glitch.com/~project-update-control

 

– REFERENCE –

11 camera-third-pov from Prof. Moon.

 

– DEVELOPMENT –

Following my project idea of having a first-person perspective through camera while navigating the main character through ml5 KNN, I found Prof. Moon’s “camera_third_pov” sample very much inspiring — it allows keyboard interaction to control character movement while having the camera following all the time (despite certain dynamic lagging). Based on that, further development of the neviation system for my project are as follows:

• Interaction design

Given the nature of 3D space, the character should be able to move from all directions. Adding that what I intend to create is a relatively immersive experience with first-person perspective, the rules for user interaction should be designed properly. After careful consideration, I now determine to combine the hand (through keyboard) and head (through camera detection) interaction to guarantee movements naturally:

In brief, interactions with keys of “W”, “A”, “S”, “D”, “E”, “Z” would control the character to move forward, left, backward, right, upward, downward respectively while parallel to the according axis in the 3D space. Meanwhile, head interactions based on KNN models from ml5 are expected to allow the character to rotate left and right, up and down — whenever users move their heads in front of the camera.

I managed to realize the keyboard interaction by integrating and modifying the Character() class from the sample code. On one hand, the specific interactions on keys should be assigned as I designed. On the other hand, as my character is a sphere geometry with lighting effect (point light), the according movements and changes in position should be synchronized to the pointlight as well.

However, I failed to apply KNN and the head interactions to charatcer rotations so far, as I didn’t know how to intervene in the logic behind Three.Quaternion. This would be the first challenge that I have to overcome later.

• Camera following

Another indispensable element contributing to the first-person perspective and the immersiveness of my project lies in the camera following. Since the character is neviagted through user interaction, it will make more sense to present the view of the character through camera. Therefore, I referred to the ThirdPersonCamera() class and enabled the camera to follow the character all the time. By manipulating the parameters on GUI, I further determined to set the default parameters to (0,0,0) to make the camera position overlapping with the character. In this sense, the character won’t exist in front of the camera all the time, while users could still observe the light and shadow effect through the first-perspective camera to be indicated his/her character of lighting sphere in such space. Beyond my expectation, as I originally had reflective mirrors above the ruins, users could even see themselves, i.e. the lighting sphere, through camera. However they might not be able to directly realize the mirror effect, which makes users hard to tell the differences between real and reflection. This, therefore, opens more possibilities for me to manipulate with the reflection and even the conceptual development in a greater sense. 

One current problem, however, lies in the lagging effect: when moving backward through keyboard, the sphere will exist for a while because the speed of movement is greater than the one of the camera. I haven’t come up with an ideal solution to it, yet it might also highly related to the design of the appearance of this lighting ball.

 

– PROBLEMS TO SOLVE –

1. apply KNN and the head interactions to charatcer rotations through Three.Quaternion
2. solve the problem of the character existing for a while due to the difference in speed between the sphere and the camera
3. downsize the scale of parameters for camera rotation on GUI to make the camera view changing more naturally