Interactive Media Arts @ NYUSH
VR/AR News of the Week has closed for the season with 92 entries, all of which we’ve seen at least briefly in class. Some of the stories will be looked back in 5 years and be considered accurate, prophetic, powerful. Others will be looked back in 5 years and be considered off-track, clueless, and ridiculous. PLEASE SELECT YOUR TOP 4 OF EACH. Post with a detailed description of why you’ve selected them in Documentation and be prepared to present.
Promising trends:
1. Facebook’s DeepFovea AI promises power-efficient VR foveated rendering.
This semester we witnessed Google announce the end of its phone-based VR grand trial, which essentially discard its efforts with the cardboard and Daydream projects. Two of the major reasons behind the low adoption rate, I believe, are the relatively low resolution and high power consumption. Thus, the research on foveated rendering becomes the focus of many VR firms, since it promises to provide high resolution for areas where the user pays attention to and achieve that with lower energy consumption and probably less latency. This technology, along with other power-saving improvements, is critical in building wearable VR/AR devices.
Source: DeepFovea
2. Apple Working on AR Headphones
While most of the AR companies out there compete in fooling the eyes, the world of spatial audio augmentation receives less attention. The patent Apple recently filed shows the firm’s ambition in building AR headphones that could spatialize sound in a video/voice conference setting. This use case could have great potential for business customers but also for public spaces like museums and schools. Also, compared to the complex optical requirements to create AR visuals, AR audio seems a low-hanging fruit that would hit the market in the coming five years.
Source: USPTO
3. Microsoft’s DreamWalker lets you see VR cities during real-world walks
This invention might seem weird at first glance: having the user see a VR city rendered by detecting the real-world city environment looks like a redundant step, especially considering the rough CGI models and blurry textures. However, I think this signals a potential use case which has a huge customer pool: commuter recreation. As cities expand, an increasing number of people need to take a long commute to work every day. People might read and watch videos on their smartphones, but a VR experience during the commute would definitely make the previously boring commute something to look forward to. A modified train cabin that allows travelers to safely experience VR during their commute would be very interesting.
Source: Microsoft Research
4. Create An Entire Home Gym With Oculus Quest
Another piece of news that I believe shows a promising commercial trend. It demonstrates how in less than $ 600 one can build his/her own VR gym at home. The vested weight and other accessories could be deliberately designed in the gym VR titles so that the experiences are better. Considering there is a growing demand in China for gym and fitness training, and that many physical gyms are run inefficiently and shut down soon after they open, people would probably show more interest in home gym options.
Not so promising trends:
1. Facebook’s Latest Purchase Gets Inside Users’ Heads—Literally
I used to believe that the brain-machine interface is an important step towards intelligence augmentation, IA (an idea that focuses on strengthening human intelligence instead of replacing it with machine intelligence). Facebook’s acquisition of CTRL-Labs gives me mixed feelings. Even though the tech giant claims the interface only enables users to give instructions to applications and collect no more information, the idea that one can share some photos or information by just “intending to” is unsettling. The ease of sharing encourages leakage of private information that users might not want to publish given a second chance to think about it. This brain-machine interface could be useful in many cases, but it should be heavily regulated in a social network application.
2. Russian dairy farmers gave cows VR goggles with hopes they would be happier and make better milk
This piece of news sounds like a joke but I think it suggests a possibility that VR headsets and related technology could be cheap enough one day to become a tool to make tedious work more tolerable. However, the value it brings to its users (cows in this case) might not be “real” and substantial. Though the cows see an ideal living environment in VR, their real living conditions are not any better. Calmed by the virtual reality, they may be confined to even smaller space physically. Such kind of virtual experience is more like a means of control, which should not be encouraged, no matter its audience is animals or humans.
Source: Ministry of Agriculture of the Moscow region
3. MediView XR raises $4.5 million to give surgeons X-ray vision with AR
The startup company is working towards a mixed reality solution for tumor removal surgeries. Despite the good intention and help it provides, I am not sure if it is reliable enough for real surgeries. Medical use cases are high stake scenarios and inaccurate AR projection could lead to legal mistakes. Even if we assume the AR imaging is correct, surgeons who get used to having AR assistance might find it hard to perform the operation when AR is not available due to technical reasons. A better use case might be medical personnel training, meanwhile, the technology should go through rounds of checks to confirm its accuracy.
Source: Mediview
4. Virtual Reality is the latest dinner party trick
Appealing as the news title sound, the dining experience described in the news piece is far from a casual and fun dinner party. The VR environment was somewhat avant-garde, featuring an abstract and artistic environment that don’t arouse my appetite. The unusual container of food and the bite-size limitations also makes the food more of a tool in an exotic VR experience, rather than a food itself. The news is reporting a trend that food served in such experiences is going high-end, with top chefs joining the team. However, I think a more popular use case might be creating VR representations of snacks for VR gamers or movie views who want to enjoy snacks while experiencing VR. It will be a natural extension of the VR environment instead of transporting the user to a random place and alienizing the food itself.
Best
What I found most interesting in this article is the new Fitness VR/AR experience called Mirror. I actually had the opportunity to see the product in person (whilst visiting a very boujie friend of mine) and let me tell you the experience was UNREAL. Personally, I like working out with others whether it be in a class, with a trainer, or just friends but having someone with you at the gym isn’t always a possibility. Mirror is a prime example of how AR/VR can be implemented in the world of fitness – a multi-billion dollar industry! I especially like this feature because it transforms a product so ingrained in our lives (a mirror) and gives it a fresh new take. My experience was quite enjoyable and I can see this product being used in the homes of high-end consumers, fitness addicts, and luxury gyms. The mirror experience was refreshing and it is exciting to see an iteration of AR/VR without the use of headsets or apps through mobile devices. Looking forward to what comes next! Maybe a VR dumbbell that looks like a slice of Pizza would be nice (lol).
My favorite AR/VR experiences are ones that incorporate both the digital and physical worlds. The article by Bloomberg showcases a top-tier restaurant that utilizes the functionality of AR/VR headsets giving fine dining a new meaning. Users would put on these headsets and enjoy a multi-course meal all while having a digital VR experience. I can see this methodology being implemented in other physical experiences (or at least I hope I see more). We’re already seen this done in amusement parks with rollercoasters that incorporate AR/VR headsets but seeing done with meals is SPECTACULAR!! I can see AR/VR implemented in other common human lifestyle experiences and I’d love to see companies utilize new tech to revolutionize the consumer lifestyle market or one that helps others. This article is reminiscent of an example I’ve seen where burn victims wear VR headsets to help alleviate the pain – this type of implementation is one that I’d love to see more of!
C’mon NOW!!! This was incredible. I probably spent the most time on this experience because the accuracy was none other. This is probably the most exciting part of this VR experience – how accurate the photogrammetry was. This feature revolutionizes the travel and leisure industry, but where I see it going is the ability for historians to use this as a platform for documenting areas that are constantly changing or endangered. Endangered in the sense that natural disasters or global warming have made a tremendous negative impact on the landscape. Some areas that come to mind are sinking countries or cities like Tuvalu and Venice. It can also be a tool that can document regions that are vastly unexplored and give insight into locations that are often misrepresented by media outlets. The tour factor is great, I’d love to see it done in museums like the Uffizi in Florence, but the underlying capabilities of documentation to shine a light on issues greater than itself is a huge “selling’ point for me.
We’ve talked about this so much and it never gets old! I totally agree that this is most likely fake~ The article doesn’t say much about the process of how the process is actually conducted. If anything, it looks like an article from The Onion (lol) and leaves a lot of people with questions to the implementation and humane/inhumane aspects of this experience.
For example:
“How do they supply enough VR headsets, batteries, wires, and etc. to all the cows?”
“Is it humane to give cows a VR experience when they could just be roaming free in a pasture instead of simulating one?”
“How do we know the milk is actually better?”
This VR experience is one that I have a difficult time grasping. I’m a firm believer that although VR/AR experiences can help in documenting real-world experiences – there are a few things that are sometimes best experienced in person. Sandbox VR is claiming that with backers in the music and entertainment industries they are able to recreate the feeling of concerts and other events. For me, at least, I like going to concerts and events because there is a feeling of community within that space. I’m able to meet new people who share the same passions and make memories with friends either new or old. I think it would be extremely difficult to recreate this sense of camaraderie within a VR space and I think this is where Sandbox VR would fall short unless they can somehow create thee experiences with a community in mind.
This is tinder or bumble on a whole new level. This article was interesting in the sense that it takes AI data to encourage new couples to proliferate. I think this a tricky subject/area. An online persona is sometimes very different from the “real-life” counterpart. I can see this implementation for suggestive purposes but definitely not in a ‘scientific’ or proven sense. Maybe I’m just an old soul, but I definitely prefer meeting people in person and dating in a face-to-face setting rather than an online one. I’m all for meeting someone in whichever platform is best – in fact, I’m a consumer of these apps now and I hope I can meet someone online and it better be the SAME interaction/appearance in person (lol).
This is an iteration of VR fitness that I don’t see taking off anytime soon. Mirror was a great example of how VR/AR can be implemented in the fitness market, but I think workout applications within the Oculus Quest is a bit more cumbersome. I definitely can’t imagine working out comfortably if I were wearing a headset and using two handheld controllers. What makes Mirror so much more successful in my eyes is the headset free interface and interaction. You are free and fully capable of using equipment you are used to and it almost feels like a personal trainer rather than an app or a game.
I think hand tracking will be the mainstreaming interactive way for VR/AR because I think that it makes sense from the perspective of historical trends. Decades ago we used our fingers to control mouse, keyboard, and controllers to interact with machines. Then we shifted to the era of touch screen, which allows us to use our own fingers to interact, more straightforward and easy. The same idea for VR/AR, as a new media, VR/AR is experiencing the same developing process that the previous media like phones and pads have gone through. Hand tracking tech can bring users a more immersive experience rather than controllers. Go into a new world, you need to use your “hand”.
I am really glad to see something like Microsoft Mat become reality because I think going into family life is an important sign showing that one kind of device becomes mainstream and there is a massive group of people who are using it. In the future, I believe that VR/AR will be a role in our daily life. At that time, people will need more relevant VR/AR devices at home for a better VR/AR experience. That is why I think this news is accurate and powerful.
5g means a greater bandwidth and lower latency. The advantage of 5G could promote the development of VR/AR video streaming and VR/AR cloud games, which are really important for the whole VR/AR industry’s development. Also, 5G tech can solve many current obstacles in the VR/AR industry, such as data transmission.
I agree that AR headphones will take place the current role of smartphones someday. Because in my opinion, people are trying to break the boundaries between the real world and the digital world. AR is the best tool for connecting.
It is ridiculous, no reason. I don’t think VR/AR could be used for improving agricultural production lol.
I think even though with the development of AI and VR/AR techs, the love relationship in the real world won’t be affected by human-like virtual avatars. AI and virtual reality, in my view, will not make us both feel lonelier or finding love. Because for the users who will use AI and VR to find love, they release their loneliness by this means. So they won’t feel lonelier; for the users whose love relation needs will not be affected AI and VR, they will not expect love and happiness brought by relationships in the virtual world, so they will not feel lonelier. In conclusion, VR and AI for love is a specific market for a group of people.
In my imagination, the VR music industry will take a position in the whole music industry in the future, but it cannot change the whole industry. Even though there are many artists holding concerts through VR platforms, I do not think that this way of giving concerts will be a long-last fashion. Because I think there is one thing which cannot be created in the VR world–vibe. In a live concert, You can feel the energy of performers; you can feel the passions of the audiences; you can feel the temperature of stages… These things cannot be provided if you wear a VR headset in your home with AC.
I remembered that many classmates expressed in class that they like the idea. I agree that this DreamWalker can bring a new walking experience to the users. But personally I do not think that it will be a company of everyone’s walking. Because firstly, I think it is still dangerous even if DreamWalker can one hundred percent shows the real street look. Besides, I think not everyone will wear headsets going out. So personally I think it is stupid to wear a VR headset walking along the street. Also, I think it is interesting for the first use of DreamWalker. I prefer real world beauty.
News of the week summary
Accurate, prophetic, powerful in 5 yrs
1.Apple Is Apparently Working on AR Headphones
To fool the ears of being in the space, spacial audio would be the very first trick to be accomplished. In the article, it describes a “pre-processor” that spits out a virtual, spatial representation of where each caller’s signal would be coming from, and a “head-tracking controller” that helps to mimic in-person meetings. Having AR Headphones plus glasses would probably be the future for video conferencing (or more), instead of having an actual space to cast hologram and install multiple speakers to mimic the scene. Or at least, the previous solution would be less money-consuming.
2. LookingGlass Factory Releases Monster 3D Display
This is a peek into what the future of hologram would look like. Though it’s just a screen that shows volumetric pics and videos, it fools the eyes into believing the object or human inside the screen is “real”. Maybe the screen is a breakthrough point to an authentic hologram.
3.AR Meditation App Helps Calm Veterans Suffering From Trauma
Although for now the data gathered for the app comes from Apple Watch or a compatible Muse brain-sensing headband that doesn’t reflect much of a person’s mental state, the idea to combine medical treatment with AR and VR is very promising. If household medical devices could do a better job at detecting one’s emotions and brainwaves, conveniently and cheaply, people suffering from mental illness would enjoy a better life (at least better treatments).
Off-track, clueless, and ridiculous in 5 yrs
The idea of creating an opera through AR is not ridiculous, but it is ridiculous in a way that it compromises the possibility of a more creative story-telling and the use of technology to “mobile phone” and “home”. What the video shows is a special place that furnished like home, and the opera is mapped onto the exact setting of the room. This is more like location-based VR, and is lamer than VR, for the background of the opera would only be one’s home! In 5 yrs time, I think location-based VR would thrive and become a hit in entertainment business. At that time, there’s no need for people to hold their phone to watch some child-like AR opera at home.
2.AR and VR will make spatial journalism the future of reporting
Since the news reporting have developed from static (pics and articles) to video-based, there could be a possibility that AR and VR would take it to the next level. However, what the article envisions is that one may be taken to the actual space of the news’ occurrence (more like VR tourism with a news feature), or be shown some vivid graphics spatially. The former would definitely help one to experience the news better, but given a choice to be in a battlefield or a place that has just struck by flood, I may not choose to experience it, for it could be horrifying or upsetting. For the latter, it’s not necessary at all. To conclude, I think articles are the quickest and the most convenient way to report and view a piece of news. Even if in the future everybody is wearing VR headsets, they will still choose to read news in their headset, just as people would read news articles instead of watch videos in their mobile phones.
3.DOJAGI – the world’s first spinning wheel simulation game for pottery making in virtual reality
I used to play a similar pottery simulation game on iPad a lot. The only manipulation method is to put a finger on the screen to push the mud. How I imagine it would be different in the VR game is that instead of one finger, user can use both of their hands to feel like they are actually touching the mud and squeezing it. However, the experience would not be complete without the haptic sensation. Even if one can feel it and create it virtually, he or she still needs to export the model and 3D print it to actual have a self-made pottery. Therefore, what’s purpose of doing pottery virtually?