Perceivers
ON THIS PAGE
GROUP INFORMATION
- Team Name: Perceivers
- Group Members: Dorcas, Christian, James, Yulan
PROJECT DESCRIPTION (DUE 10/13/21)
Description of Team Name
People who perceive…and achieve!
Main Point of Contact: Christian
Meeting Schedule for the Semester
- Meeting on Zoom per week or every another
- Asynchronous communication on WhatsApp
Timeline
10/7 – First Meeting with Dorcus and Steve
10/13 – Discussion about possible solutions and outside sources that can be helpful.
10/20 – Second Meeting with Dorcus and Steve
10/27 – Midterm presentation
11/3 – Meeting with Dorcas and Chancey
11/10 – Device development
11/17 – Device development
11/24 – Device prototyping/testing
12/1 – Device refining
12/08 – Project Presentation
12/15 – Final Documentation Due
The Problem
Even though a blind person has tools to walk such as a cane, guide dog, or familiarity with the area, there are times when they can become disoriented. Outside, disorientation can come because of distractions, or severe weather. Once disoriented the question becomes, “Where is the door or object I am going to?”
This problem is very similar to sighted people looking for their car in a large parking lot. “Where did I leave that car?” A push of the key fob button and the car identifies itself and you know which way to go.
Project Plan
Our plan is to create a device or set of devices to assist Dorcas when she navigates outside of her home. She and her husband already attempted a potential solution involving beacon-like technology, which only worked intermittently. Our device(s) may include a combination of beacon technology, GPS, and compass features to re-orient Dorcas to her surroundings.
Overview of Tools and Expertise
- What tools have the client already tried?
- A self-made audible-beacon at the door or object that needs to be found
- What tools/expertise will you leverage to complete the project?
- Familiarity with GPS, beacons, and navigational smartphone apps
- User experience research, particularly in open conversations with Dorcas
- Product Design
- What additional tools/expertise will you need to complete the project?
COMPETITIVE ANALYSIS AND LITERATURE REVIEW (DUE 10/27/21)
What is commercially available to solve the problem?
There is not one device in particular that solves all of Dorcas’s issues, but these are some that can help situationally.
- AirTag: “If your AirTag is nearby, your iPhone can lead you straight to it with Precision Finding”
- Pros
- Reliable and accurate
- Small, can be hidden
- Location of object/landmark can be given through iPhone audio, eliminating weather-induced audio issues of traditional beacons
- Cons
- People might take them
- Cost = $25 (4 for $$99)
- If Airtag is moved (or possibly doesn’t have to be(?), it will beep 8-24 hours after it’s away from iPhone.
- Pros
- Google Maps w/ VoiceOver
WEEKLY BUILD UPDATES (STARTING 11/3/21)
11/3
Upon Anita’s recommendation, we decided to schedule a meeting with Chancey Fleet, a blind librarian and technologist. It was very helpful because she had the opportunity to speak to Dorcas directly about her particular situations in which she gets disoriented, and discussed some of the new technology that is already out there that could possibly help. Her solutions were not too far off our own research ideas, which included the compass app, Google Maps, and Blind Square. Being in this field, Chancey has experience with all of these applications. In addition to those, she mentioned an app called Microsoft Soundscape, which can give you, through audio, the intersections in the direction you’re standing/holding the phone, as well as the ability to set beacons on specific locations/landmarks. It was great to hear about Soundscape because that is something our team did not know about but could be the most helpful solution for Dorcas.
Dorcas has little to no knowledge of these applications, let alone how to use them, so on 11/10, we are meeting for the first time in-person to introduce these apps to her, and hopefully, all learn and teach each other how to use them. Our focus as of now is the education of digital solutions for disorientation.
soundscape testing with Dorcas(11/10/21)
According to Chancey’s proposal, we discovered one app that could help – Soundscape. We decided to meet with Dorcas in person and solve the problem of using Soundscape based on her actual situation.
On Soundscape, users can set beacons for different locations and get 3D sound navigation. We introduced this feature to Dorcas and reached a consensus to set up some beacons in the places she frequently went. Dorcas successfully set up six beacons with our help.
- Beacon 1: Park entrance
- Beacon 2: Bench at Park
- Beacon 3: Parking lot entrance
- Beacon 4: Chase Bank
- Beacon 5: Back door of VA
- Beacon 6: VA hallway & Coffee shop near stairs
Pain points:
1. Soundscape’s 3D surround sound navigation relies on headphones. But Dorcas cannot use headphones because she needs to listen to the sound of the outside environment. Therefore, she can only use the iPhone external speaker during the test. Although it does not affect the results, Dorcas lost the 3D surround sound experience. We introduced bone conduction headphones to Dorcas. She said she had never used them but was very happy to try them. She explained that there are various products on the market, but there is no way to know which one is suitable.
2. Dorcas mentioned that she has to take the city bus on 43rd Street on workdays. Her biggest difficulty is: How to figure out which bus it is? There is a gap between her and the bus, a lack of information other than visual reminders.
Solution / Next step: Find the right bone conduction headphone for Dorcas. As Amy said: “Buy as many you can, you will never know which one is going to fit because everyone’s bones size is different. And keep an eye on 90 days return policy.”
FINAL DOCUMENTATION (DUE 12/15/21)
Transcrpit
James Lee: Hello everyone. We’re the Perceivers and our groups include Dorcas, Christian, me, and Yulan. This is the timeline for our presentation today, We’ll first talk about our recap, our first meeting with Chancey Fleet. Then we’ll talk more about this software Soundscape, our in-person meeting with Dorcas, and other solutions.
So what we did before midterm was we understand Dorcas’s problems. She’s looking for ways to help her navigate in the open spaces. And we also listed out our potential solutions which are AirTags, AI path detection, GPS and audio compasses. Acknowledging there might be many existing solutions to our problems, helped by Anita, we were connected to other assistive tech users and ask for their help. Next I’m going to talk about our meeting with Chancey Fleet. Chancey Fleet is an Assistive Technology Coordinator located here in Brooklyn, she is also a blind assistive tech user. Together we have a Zoom call along with Dorcas and Steve. After knowing about Dorcas’s problems, she asked about Dorcas’s mobility training. Knowing that Dorcas’s proficiency with self-orientation, and using guide dogs. She then gave us recommendations about some of the existing technologies. Here is an analysis about the software she recommended us. First is the compass app. It’s pretty simple and straightforward. But the problem is, is also too simple. Next is Be My Eyes, which is a visual interpreter. That advantage of this is it can provide many details since someone is looking the surrounding for you. But also it may have inconsistency because in the experience will be based on what volunteers you get. Next is the Google Map. Google Map is really well known and is really good for turn for turn guidance. But the visual map on Google Map is useless and also it contains the last 10 yards problems of GPS. And lastly we talk about Microsoft soundscape. The good thing about Microsoft soundscape is it specifically built for blind users. And also we can place audio beacons on it. But also it also have a problem with GPS.
Yulan Ren: And now I’m introduce soundscape. Soundscape is a Microsoft product that enables people to build a richer awareness of their surroundings. Thus, becoming more confident and empowered to get around. And this app is available for free. in ios in the US, in the UK and Australia, Unlike step-by-step navigation apps, soundscape uses 3D audio cues to enrich ambient awareness and provide a new way to related to the environment. It allows you to build a mental map and make personal route choices while being more comfortable within unfamiliar places.
And first, you should have your headset on. And then try the around me feature. It start by pressing around me and just listen to where the landmarks are. We can hear the directional 3D audio cues from the app. And here’s an example of me using the feature. I was located at the Dibner Library. In this video, you cannot hear the 3d audio as I did, because I had my headphone on. Or a condominial around 190 meters. School building around 35 meters. Metrotech around 25 meters. Other residents hall NYU Polytechnic School of Engineering around 90 meters. After, after than that. Or you can also set audio beacons and users can place the beacons an hear it. For example I set an Beacon at 370 Jay street And I will receive the beacon sound and intersection announcement school building beacon 290 meters, launch building. And you can make it as a marker.
This can be useful for anybody who’s in a new place. Even if you are fully sighted After we get familiar with this app, We met with Dorcas and began our first user testing on this app We met outside with Dorcas at her workplace in the stuytown area and immediately went to work. Dorcas show us her usual route to take Betty to her favorite trees. And we use soundscape to set audio beacons on Specific landmarks on the way. And James took a photo of us figuring out the app. And at the end of the day, Dorcas set-up totally six beacons near her working area, We started from the garden near her hospital. She usually have a walk with her dog Betty over there And we guide her to set up the first beacon at the entrance at the park And then we’ll move on to the chase bank and grocery store. And some places near her hospital. As you can see on the side the photo with pentagram marks are those set up by dorcas, independentlly Here a map a route map of our soundscape testing.
Christian Hadjigeorgiou: Here’s some examples of where we set beacons. So the first is a driveway where she passes through to enter her building and just a shortcut that she likes to use. (Unintelligible) That’s a lot of work. Tell me about it. Go straight, doggy. And the second is the first tree in the garden. where Betty (dog) likes to pee. It’s just a photo. And the third video is some ping-pong tables and a bench area. And it’s just adjacent to the tree at the top. (Unintelligible) ping-pong table and bench area is also adjacent to the sidewalk. So unfortunately, when we were walking on the sidewalk, after placing a beacon in that area, Soundscape told us that we had arrived to that area even though we were still on the sidewalk so that there’s just one issue we had with Soundscape. And the next slide we have Dorcas placing a beacon for the cafe in her building, (Unintelligible) Marker created. Woo! So Dorcas felt more secure now that Soundscape was confirming her current location. And not just the landmark she assigned beacons to, but also when she used the function to hear nearby places.
And for other solutions, we’ve thought about bone conduction headphones. So why does Dorcas need headphones in the first place? Well, one is to hear Soundscape better because if there’s lot of traffic or, or weather, it’ll be hard to hear the audio from her phone speakers. And the second is it involves another solution that I’ll get to later involving AirTags. But why bone conduction headphones? It’s because Dorcas needs your ears open for gaining information about her surroundings. She prefers not to use in-ear headphones or earbuds. Because if her ears are covered she’ll have less situational awareness for important things like traffic or for determining her distance from the side of a building, for example. So bone conduction [headphones] work because they won’t cover her ears. Instead they transmit sound through the bones just anterior to the ear. However, the issue is that the band that wraps around behind the head bothers her and made it difficult to wear hats which she needed when the weather is cold. So Dorcas and her husband Steve, tried one Aftershocks bone conduction headphones model, but it had that problem, It had the issue where where the band wrapped around the head and pretty much all bone conduction headphones had that band. So we looked into other solutions and we found some Bose frames instead, which are pretty much bone conduction headphones just in the shape of sunglasses. So Dorcas on these much more comfortable. And found the audio quality really good too. And that’s Dorcas rocking her Bose Frames.
And for other solutions, we have AirTags. So I want to talk about this picture first. So this Dorcas standing in the same place in both pictures at the courtyard of her apartment building. So as we know, GPS isn’t all that accurate. And sometimes we run into the issue where Soundscape thinks we’re at the beacon location when we’re actually just close by. And this had been a few times when we set an audio beacon on the ramp in Dorcas’s apartment building’s courtyard, which is the left picture. And next to the ramp are the stairs, the right picture. So like once or twice Soundscape guided or towards the stairs, which is obviously super dangerous. So we thought how can we be more specific? And one solution is using Apple AirTags, which are pretty much physical beacons that you attach to your personal items so that you can find them. And the good thing is that it gives us near pinpoint accuracy. And also the ability to play audio through Dorcas’s phone connected to her headphones. We haven’t had the time to try them out yet, but this weekend we will. And it comes with its own set of issues. Of course, like for example, people might discover them and take them. We can’t use them indoors, right? If it’s like a private building. And they can be a bit pricey. But overall, while we didn’t solve every issue, Dorcas has with her world. I think she’s gains more confidence and comfort navigating it, while having such an amazing attitude and being open to new ideas and technology. And we’re all so happy to have met and have worked with her. Thank you, From the Perceivers.