Final Project DREAM Summaries – Yudi Ao (Amy)

Project Name: DREAM (Data Rules Everything Around Me)

Description: DREAM is one of the four “On Shanghai” VR projects in a NYU Shanghai Fall 2019 IMA course named VR/AR Fundamentals, completed by Ryan, John and Amy. It is a 360 video experience located at the center of Shanghai near the Pearl Tower in Lujiazui. What we wanted to express in this one minute experience is that everything in VR headset, or any technical equipments like computer, mobile or whatever nowadays, is composed of data and may vanish and collapse. We wanted to audience to get some futuristic and cyberpunk feelings from it.

Below is the 360 video:

Production: Firstly we started with getting used to a very “high-end” camera, Intsa 360 Pro 2. It was quite different with normal cameras but actually also very easy to use under the clear and in-order instructions. Then about the post-production, we went through the stitching them together, adding visual glitch effects pixel by pixel within certain areas and attempting to modify spatial audio. Stitching six files into a full 360 video was also easier than expected with present mature application, for the reason we have discussed in class how hard manual stitching was before the new camera like Insta 360 and application came out several years ago. About the effects, greatest thanks to my teammates’ hard working. Though I didn’t participate in this part much, I also explored a lot in post-production’s effects on a normal video by myself and got a taste about how powerful Premier Pro is to modify even just from pixel level. The VR toolset in Premier Pro can do more than we expected and gave us a great experience to try “movie-like” effects though ended with requiring more workloads, totally in expectation. The spatial audio in the end was not achieved because of time limits either. Anyway, the outcome is quite satisfying, especially with the demoing reactions in IMA Show. 

Demoing: The hardest part while demoing our project was in fact to introduce a zero-VR-experience user how to use a VR headset. They need to immediately know how to use the controller and how to chose and click in their own views. Fortunately all of them got used to it very quickly. It was a huge paid-off when you saw their exciting and “wow” reactions while watching the one-minute video. Some users even had questions after the experience, like “What course this project was made for?”, “How did you make this”, “Do you have more to show” and so on. We all believe VR experience will get more and more popular and if this short one-minute experience can trigger more people’s interests, we have succeeded.

Below is the documentation video intercut with the project and demoing reactions:

VR/AR News of the Week Selections – Yudi Ao

Top 4 accurate, prophetic, powerful:

   1). Sifting Reality From Hype: What 5G Does (and Doesn’t) Mean for VR & AR

5G has been on its rapid improving trend for a while. I’ve seen an introductory video indicating how 5G can’t help people a lot in the present time period of technology. Most of apps actually don’t have the software requirement or ability to reach that high speed and almost nobody will use mobile data to download movies or something. There’s a testing video about how different watching a movie on the mobile is between 4G and 5G; and the outcome is very slight that not influential at all. However, when it comes to the VR/AR games, movies or experiences, the difference will be more evident since the streaming speed requirement is more strict and definitely 4G can’t support them enough. I believe 5G will be an important tool for VR/AR in the future just as what 4G means for the streaming services now.

  2). Apple Is Apparently Working on AR Headphones 

It is not surprised to see Apple’s working on AR headphones while most of the others focusing on visually fooling the audience. Compared with fooling eyes, this is evidently much more easier to achieve and can be utilized to do a lot of things, such as not only in conference calls or remote workers, but also some public situations. To erase the clear differences between reality and digital world, AR headphones will be essential in the future market as a helpful weapon along with headset or glasses too. Therefore, this new trend is also fundamental and will have more marketing power with its importance for AR tools.

  3). Esqapes puts you in a VR experience while in a massage chair / VIRTUAL REALITY HEADSETS ARE INCREASINGLY BEING PUT TO USE AT THEME PARKS

I’m supposing the usage in VR massage chair is similar with the VR roller coaster and other sitting experiences. These are all the situations combining traditional services with VR technology to provide a better and more immersive experience. And in fact, they are already on the way for many years such as some riding facilities in theme amusement parks, adding VR headsets in to generate a virtual visualization within traditional roller coaster, which can definitely lead to better experiences. For instance, my personal deepest memory in USJ, besides Harry Potter theme riding, is the one I had with VR headset. The same it is for massage chair, and many other possible conditions. Combining VR can help improve them a lot for audience.

  4). AR and VR will make spatial journalism the future of reporting

This reminds me of a Chinese television programme named Legal Report, 今日说法. In every episode, a criminal case will be introduced and there’s one paragraph in which the host will be in a virtual spatial reality to help analyze the case. Though the effects of this form of spatial journalism were not that mature and that real to fool audience’ eyes, this attempt to try has shown how reporting can be more immersive if high-quality VR/AR technic being achieved.

As stated in the article, the spatial journalism has many limitations regarding to the high costs, technological obstacles, long-term production schedules. However, I believe in five years, at least for programmes like Legal Report, the improvements can be evident and bring audience more impactful memories and better empathy, as well as other news reporting situations.

Top 4 off-track, clueless, and ridiculous:

   1). Why The Music Industry Is Banking On The Growth Of VR Concerts

As a huge fan of a Japanese rock band, I can say 100 percent that I will prefer going to the concert by myself rather than VR concert even if in the future they provide it. It is so ridiculous for a fan to choose to watch a virtual concert through headset rather than be in that environment and feeling the real atmosphere plus to meet those singers in person maybe. However, there may be some people who would like to do so when they have the problem of physical location change in a certain time, or don’t have the time now; so they can experience the concert later in VR. I just feel the whole concert industry won’t be influenced a lot and the mainstream will still be the present forms. VR only adds a possibility but can’t change the essence of concerts, or the reason why fans want to go to a concert.

  2). Will virtual reality and AI help us to find love or make us lonelier?

This topic can be very completed and even a debate question. However, personally I doubt human feelings can be that easily influenced by VR or AI. There are many sci-fi works such as human in love with AI, love story starting within a VR world and so on. But these are still novelistic and real human love is much more complicated and hard to analyze or conclude. People may get company from AI and use VR to meet with friends simpler in the future, but I still believe realistic life is the main focus for us in the next decades.

  3). Russian dairy farmers gave cows VR goggles with hopes they would be happier and make better milk

I don’t see any points of doing this, even in the future it is affordable for agricultural industry to utilize huge amount of VR equipments. The cows’ production of milk has been improved and to the best situation after so many years and I doubt VR can make them “happier” therefore make better milk. Their living conditions are already the best for agricultural purposes.

  4). Can virtual reality make women’s lives better?

This one sounds the most offensive for me as a female. Sexism and racism problems should be taken more seriously. People who refuse to admit this real fact, ignore these problems, or unaware of what conditions we are in, will also not be influenced at all even if there are VR programmes like the ones mentioned in the article. The required efforts to decrease inequality should be focused on more human rights education and awareness raising. There have been lots of movements, slogans and actions used. I don’t think who have access to the VR will not have seen those, which are not less powerful than virtual reality experiences, if they have the empathy, or realize the importance. 

Week 10: Thoughts on high-end VR tools – Yudi Ao(Amy)

1. Mantra VR: This tool can apply 16 filters easily to modify the footages within flat, 360 monoscopic and stereoscopic formats through drag-and-drop effects. And the keyframe animations can vary from 0 – 100%. Through the application, user then can create surreal and fantastic VR videos in a simple way.

2. Cara VR: This one a plug-in toolset for Nuke. It includes advanced stitching function to automatically corrects the mono and stereo VR content. It can also provides easier insertion of 3D models. And the headset viewing inside Nuke allows the user to get the preview more easily.

3. Mocha VR: This software is armed with extremely advanced planar tracking which is even considered as “go-to tool” on Hollywood films and must-see TV. With this powerful foundation, Mocha Pro is then able to drive tracking, masking, object removal, stabilization and the unlimited tracking layers helps to handle reflections,  occlusions, noise and motion blur or low details. 

The highest priorities for these tools are actually to provide users more freedom and controllable editing in an easier and more steady way. And with these tools, VR video can be created not just by professionals but also anyone who is interested in. Therefore, these high-end softwares then also allow more producers to entry into the VR and AR market and increases the competition. This is especially when VR experiences are expanding among more people and have lots of profitable potentials. The new powerful tools can be utilized by anyone.

Week 6 : Relevant examples and spots

This is a short video made by myself three years ago.

The below is the re-photography example I found interesting in Google image searching. This work shows the building in different time periods. With the black effects in the left side one, it is obvious that it is the old one and right one is the newer.

And my proposed spot is central park, where I can sit for half day just relaxing.

Week 5 : Insights about image-based VR by Amy

The TED Talk was back in 2007, which is surprising but also reasonable that utilizing 2D images to make 3D models has been there. The used examples ,including Grassi Lakes and Notre Dame, were all the reconstruction with photos taken by tourists or software building teams. However this application had many limits at that time; for example, in the Notre Dame’s case, there was one image came from a poster and influenced the model more or less. The quality of images and the computing methods to reconstruct the model were both the focus areas for the development’s direction.

Then the evolvement of the challenges, according to the four publications of developers of Photosynth, are similarly related to machine leaning such as image classification, identification and also methods of image capturing and motion improvements. As time goes on, the enormous number of data and improving software computing logistics all contribute to the developments and it may take more time to reach a satisfactory level, especially when comparing this technology to other networking things. All these kinds will need a long term of researching and experiments to improve.

And about why some people quit and some stay. I’m guessing one reason may be the expanding areas of the image-based technology, from simply 3D model to VR, AR and more spaces. The expanding brings new blood and kicks out those who can’t follow up.