Category: VR / AR Fundamentals

Posted on

Week 5 – VR/AR homework about the Oculus Connect 6 Developer’s conference

For the Oculus Connect 6 Developer’s conference, I choose to watch the 12-minute summary and two videos, Creating Spatialized Music for VR/AR and A New Architecture: Unity XR Platform. It is amazing for me to see that Oculus Quest is going to import hand tracking without using hand controllers, it is a great progress towards making VR experience more real, as how we use our heads in VR vision is not going to be controled by using a controller as we press button on it or swip our fingers on its touching pad. So by realizing this can imitate the detail movement of our hand, like how our joints has bent, how much strength we are using to realize more complicated movement and effects in VR applications.  For the Passthourgh+, I think it is also important and convenient as you do not need to take off the headset to see what is happening around, which makes using the VR headset much safer in case of any circumstances. Applying facebook society in Oculus is making it more interactive as users can share their experience and hold event in real time, and it is more conveient than in the reality to invite someone to play with you as everyone just need to wear the headset and join others online which feel like being together. The idea of facebook Horizon is also amazing as it is a highly interactive world which is described by Zuchberg that users can do anything they want in it. I believe this must be a great achievement as any open world, not only in VR, but also on other platform is extremely hard to realize, because there are always limits for the users to manipulate the open world, for example, if users want to create something, they want to cut something into a customized shape takes high demand to the phsical engine, not to mention the VR open world. To make it real but highly controllable is really amazing to me. The Ctrl-lab that is working on neural control system is the one that fascinates me the most, it is incredible to control user movements by just thinking with a wristband on, I have no idea how this works in detail, but instead of planting devices inside the body to make this work, just using a wristband is far beyond my expectation. Combining this with the hand tracking system, it is possible to foresee that we even don’t need a menu when using VR headset, as we can do the movements by hands and send commands by just thinking, these all make VR a second world to live in. For the machine perception part, how real the VR scene looks like and how precise the human face and body can be captured and displayed are both amazing, I  think it is a progress towards making the VR world more real, instead of only using cartoon figures to communicate.  

For the first video, the Unity one, has mentioned three points, Unity XR integration, API convergence and Universal Render Pipeline. The Unity XR integration has mentioned that many assets and scripts has been packed into pachages to make it easier for developers to download whatever they want by choosing the corresponding packages. But what interests me is that developers can use VR to do 3D modeling and rendering, instead of using mouse to drag objects on PC, this time, it is like being in the Unity to manipulate the objects, if it can also be applied with the hand tracking system, it will be amazing that we can shape the objects by hands to whatever shape we want and drag them to wherever we want. For the API convergence, it is about cross-platform APIs, that the API in VR headset can be applied to other applications even though they have different interfaces, it is very helpful to eliminate many problems when developers are switching from platforms to platforms. For the Universal Render Pipeline, I am quite confused how it works, but I thing I learn is that, it uses tile renderer which is quite expensive, so it is important to do the best rendering effect at the ultimate stage to save unnecessary computing cost.

For the second video, the spatialized music it talks about the preocess of making music spatialized in VR. Unlike traditional headlock stereo music which is very plane, Quad mixing sound can be heard from every direction and is world-relative, and there is always a compass to determine where the sound comes from. Ambisonic mixing sound follows users around and is also world-relative. The way how videos with spatial sound are created in a different sequence then usual, that music is created first then the visuals. Also, overall exeperience are made by unreal engine, while wise which is a game-oriented sound design tool is made to realized the sound effect, and music is placed inside the enviornment. It is totally new for me to learn about how music and visuals are combined in VR, because it is totally different than traditional graphical music on PC, take games for example, even though players can hear where the sound comes from and even how far they are but it is not in an immersive environment, not matter how real it feels, it cannot be compared with a VR one. But it is obvious that to realize spatial sounds is much harder on VR.

The following is the photo I took along the Huangpu River when I was riding along the river side, the reeds artfully cover the buildings behind them with the beautiful sunset.

Posted on

W3 assignment by Molly He

3 titles from the catalogue:

  1. Allumette–VR film

It is 3D. One can pierce into something else and even trigger some animation while the main plot is happening on the other side. (Thanks to the exquisite design, sometimes one would miss the plot bc of the distraction of attention…) The sound is spacial, making me feel like at the scene.

What I like about is the clay-featured character that really sets it apart from normal CGI animation. The story is somehow touching but could have worked on the emotional transition more. What I dislike is that the animation rate is not high enough to make me really immerse in to the film.

2. Knock out league–VR boxing game

It is 3D. When moving, I feel that I’m really inside the boxing ring. Also, since the sound is spacial, I can easily identify the cheers and applause coming from the audience seats—from all directions.

What I like is how boxing itself is simulated— I feel like I could even learn and practice boxing by playing this game. The device can even recognize 10 different punches! What I dislike is that it seems the opponent is not intelligent enough to recognize where the punches are and obey to the boxing rules.

3.Henry–VR interactive film

It is 3D. It seems that all corners and angles are considered by the producer so that if I try to look into any possible things, it would be of content.  The sound is spacial. When Henry is coming near, I could sense that just according to the sound getting from a low volume to a high one.

What I like is that Henry is often appearing very closely to me and it creates an intimate and immersive feeling of actually communicating with Henry. What I dislike is that the movie itself is kind of childish and lacks some spice.

Posted on

Week 3: Review of the VR Experiences – Kat Valachova

  1. Lucid Space Dreams – Oculus Quest

The Lucid Space Dreams is a close to 2 mins long video-based 360 degree VR experience, that brings the viewer into a very see-through, neon-like dark world, which in parts resembles the Avatar’s Pandora. This was my first experience with VR and I first felt slightly disappointed with the interaction this video provided, as it “only” allows the viewer to turn around his head and watch his “surroundings”. But after briefly experiencing other title on the VR Catalogue, that many times didn’t even offer the 360 view in the first place.

On the other hand, true to its name, this VR offers very authentic dream-like experience, where one is not in control of what is happening and can only watch as the space he finds himself in is constantly drifting. The result is a lucid-like feeling.

  1. Knockout League – Oculus Rift

Knockout League is an arcade-like VR boxing game. Situated in a boxing ring, one  can fight against various opponents. The game is rather colorful and creative when it comes to the opponents one can fight. Each one has his own very specific looks and it felt as fighting style as well, so the player can fight with figures such as Captain Hook, an octopus or a wolf girl.

In terms of the experience itself, I learned it is hard to move in a way so the imaginary hand hits the target – it is not based on muscle memory as is natural for normal body, which is especially vexing, when one has more experience with boxing, but cannot implement them. On the contrary, only after multiple failed tries, I realized that in order to hit the target, the movement coordination must be based solely on my sight, meaning I first have to watch my imaginary fist in the VR, calculate the distance to the opponent and finally attack. This VR requiring of the player to distance himself from his body feels somehow unnatural but I guess one can get used to it. Otherwise playing Knockout League was really exciting.

  1. The Night Cafe: A VR Tribute to Vincent Van Gogh – Oculus Rift

The Night Cafe is a rather boring 6 minutes long VR video experience, which lets one “have a peek” into what does it look like inside Van Gogh’s painting The Night Cafe and “behind the scene”, while being accompanied by slow piano music. This VR does its best to bring the Van Gogh’s 2D version into 3D, while leading the viewer around the room, further behind what would have been the painter’s back, showing what cannot be seen on the original painting. The VR version brings the painting into life by animating what Van Gogh tried to capture as motion, one can see the lamps radiate the light, the billiard player rigidly shifting weight from one leg to the other, then behind the painter’s back the pianist playing (it doesn’t feel as if he is the one who plays the background music, as they are not in sync). The room is filled with the inspirations and models of Van Gogh’s paintings, giving off the feeling of being in a place that Van Gogh frequented and from which he took inspiration. Then, the viewer can see Vincent Van Gogh himself sitting in a chair, getting up and gazing outside the window, seeing his famous starry night. The viewer is then led around the rest of the house, seeing people and places he cannot see just from the original painting, finally ending in an underground dark room/cellar, with a piece of paper in memory of Van Gogh.

Posted on

Week 3 – 3 VR Titles Review – Ryan Yuan

The three VR titles that I have experienced  are BBC Home, The Elder Scrolls V: Skyrim VR and Super Hot.

1. BBC Home on HTC Vive Pro

This VR experience is marvelous as I am acting like an astronaut to fix a space station. The audio effect is spatial as if I am really in an astronaut suit listening to a radio, and the sense of losing gravity really makes me feel like being in space. Even though the part that I need to press the button to catch on the bars does not make me feel real, but its physics engine that restore how it feels when grabbing something to push me forward still makes it perfect. Also the shadow and light in the graphics make the perfect enviornment of space, at the end of the experience when the station has exploede and the astronaut is floating away, it really makes me feel dizzy and helpless when I have failed to reach the station and ends up to death. Everything is real and amazing.

2.The Elder Scrolls V: Skyrim VR on PS4 VR

I fail to experience it on class, but I do have tried it else where. I have played the game on PC years ago, it is a game that I can customize a lot of things and almost do whatever I want, I can be a worrier who slays dragon and also I can use magic to fight against epic monsters. Everything are restored in the VR game which is impressive, even though the graphics is not that satisfying, but the audio part is amazing, the sound of monsters and environmental sounds, and the sound of hitting something or using a magic, these all make the game amazing. I really feel that I am being a wizard in a fantasy world, living with all kinds of species. I also need to dodge and defend in the battle, these movements also make it real as if I am really fighting with something in reality, even though I cannot feel the sense of death when my character is really killed.

3. Super Hot on Oculus

This is also a game that I have played on PC, the game is designed to kill all the enemies in slow-motion using everything you can grab within range. The way I control the character to move is not by my body movement but by the controller which make it less real, but the audio effect and the feeling of hitting something is brilliant. Even though the game is a bit violent, the experience is quite exciting. Also it is an amazing experience to move the body in a slow-motion.

Posted on

Week 3 (Catalogue Review) BLT

1.) Quill – Oculus Rift 
 
This is one of my favorite AR/VR experiences thus far! I thoroughly enjoyed using Quill (again) – I had the opportunity to use it for a short period both at a conference and in class. I love the concept of giving artists/creators tools to create more art and the ability to visualize/animate designs in realtime was an incredible feat! My favorite part of the application was the ability to visualize ideas instantly and how it gives the creator a better sense of space. However, similar to other applications like Quill (Adobe Photoshop, Illustrator, Etc.), there is a steep learning curve. It would take several tries to learn how to expedite the process and be able to use the product in a real-time/efficient manner. Otherwise, I would use Quill again and hopefully get better at it. In the near future, I hope to create a VR/AR application that will allow creators to “create” art or something new with my own VR/AR world. 
Image result for quill oculus
 
2.) Notes on Blindess – Samsung Gear VR
 
Notes on Blindness is also a very interesting and enjoyable experience. I’m very interested in film and creative storytelling, so the idea of a short film/narrative in a VR/AR format was very intriguing. The visuals were a stunning experience, I loved the visual imagery created by the lights and it really “played” with my senses. The audio was a huge part of the film and I thought that it was really well done. My one critique is that in some scenes Notes on Blindness was a bit dim (I get that its about blindness) but for such an ironic piece, I felt myself wanting more light in some situations just so that I could read the text or get an idea of the space they are trying to portray a bit better. Maybe that’s what the creators wanted us to feel??? Otherwise, I enjoyed this VR/AR experience as well, and I could also see myself doing something similar for a future project – creative/digital storytelling via a VR/AR format. 
Image result for notes on blindness
 
3.)  Star Chart – Oculus
 
The Star Chart on Oculus was beautifully done, but probably one of my least favorite experiences. Some key elements that I appreciated were the fact that the night sky alters based on the user’s location. So technically, had I tried the application in various locations – the night sky visuals would change as well. I think this feature is an ingenious one, but my biggest critique is that the progress for the user is quite finite. Granted there are countless stars in the sky, the application only shows first-tier and second-tier stars/galaxies. Once the user has “explored” enough the interface becomes a bit monotonous. Star Chart is a great resource for space and astrology fanatics, but in terms of user experience and development, it leaves the user with more to be desired. 
Image result for star chart oculus
 
Reflection: 
I genuinely enjoyed my first experiences with a variety of AR/VR applications – if I were to create my own, I would probably do something a bit more user and consumer-centric potentially through digital storytelling, a lifestyle based app, or even on that follows current trends. A prime example that I would do is an iteration of my least favorite experience, Star Chart. Currently, there is a tremendous millennial and Gen Z focus on horoscopes and astrology, and I would incorporate all the great facets of Star Chart and make it more user-friendly and story-driven – connecting them more with other “players” rather than a singular user experience. My iteration would give daily updates, horoscope predictions, and interactivity in a more dynamic sense.