Manning Whitby
Age 17 | Toronto, ON
Canada-Wide Science Fair Platinum Award: Best Senior Project | Canada-Wide Science Fair Excellence Award: Gold Medal | Dalhousie University $5,000 Scholarship | University of British Columbia $4,000 Scholarship | University of New Brunswick Canada-Wide Science Fair Scholarship | University of Ottawa $4,000 Scholarship | Western University $4,000 Scholarship | Youth Can Innovate Award | Ted Rogers Innovation Award | European Union Contest for Young Scientists: Canadian Representative
DOI: 10.18192/csfj.v2i1.20192129
Independent travel and the ability to interact with the world are two of the greatest challenges facing blind/ visually impaired (BVI) persons (Jacobson, 1998). How to interact, map, navigate the environment, and move through it efficiently and collision-free are challenges faced when utilizing traditional tools, such as the white cane and guide dog (information collected from this research’s focus groups). These tools do not provide users with all the information they need to navigate effectively and gracefully (Wiener, Welsh, & Blasch, 1997; Shah, Bouzit, Youssef, & Vasquez, 2006). Because of these difficulties and limitations, many BVI persons find travel and exploration stressful, disorientating, and intimidating (Jacobson, 1998; Golledge, Marston, & Costanzo, 2001; Bruce, et al., 1991). As a result, many BVI individuals are significantly less willing to travel independently and tend to traverse only familiar routes (Golledge, Marston, & Costanzo, 2001). Globally, an estimated 285 million people have low-vision (World Health Organization, 2018); of these, 40-45 million are totally blind (World Health Organization, 2009). With an aging population, these numbers are expected to triple by 2050 to an estimated 115 million (Bourne, et al., 2017).
OBJECTIVE
My goal was to develop a more effective solution to the social and navigational challenges faced by the BVI community. See Figure 1 for the flow of investigation map. The research investigated three characteristics of a successful solution:
1. Collect rich spatial information from the dynamic environment around the user;
2. Communicate the collected depth and obstacle location information to the user effectively;
3. Provide a product/service that the target population wants to use and can access.
The work focused on designing a sensory substitution device (SSD), primarily concerned with obstacle avoidance, that aims to give users the freedom to explore the world independently with confidence and comfort. A sensor unit collects spatial information from the surroundings and communicates it to the user through vibro-tactile stimuli that corresponds to the depth and location of obstacles, ‘painting a picture’ of the environment for the user.
Figure 1. Research investigation map.
PROCEDURE
The steps used to inform each investigation were: 1) to review and analyze similar studies, work, and observations from those working in the BVI community; 2) prepare unique hypotheses informed by this research but tailored to meet the specific needs of wearable technology; 3) develop experimental procedures and tests in response to these hypotheses; and, 4) recruit unique sample groups from the BVI community to test the specific hypotheses as developed. Both visually impaired and sighted individuals were recruited for the investigations; however sighted individuals were only subject to those physical experiments that did not require knowledge of life as a member of the BVI community.
Collecting Data (Rich Environmental Information)
This sub-study investigated how best to collect environmental information. The most important characteristics of assistive technologies, or sensory substitution devices (SSD) that correspond the user’s needs, are defined as: Type of Analysis – how an SSD system needs to and can provide fast processing of information between the user and sensor; Coverage – how an SSD system can provide services both indoors and outdoors, and detect obstacles of varying heights and position (including moving objects); Time – an SSD system should perform equally under day and night conditions; Range – an SSD system should ideally detect objects at a minimum of 0.5m to more than 5.0m; and, Object Type – an SSD system should be able to detect both dynamic (e.g., a skateboarder) and static (e.g., a park bench) objects (Elmannai & Elleithy, 2017).
To address the requirements of Analysis, Coverage, Time, Range, and Object Type, the design includes a combination of ultrasonic sensors and Light Detection and Ranging (LIDAR). Ultra-sonic sensors use the speed of sound to measure obstacles, whereas LIDAR uses the speed of light. Neither sensor type is limited to indoor or outdoor use, nor time of day. Ultrasonic sensors (HCDOI:SR04) have a limited range of <4.0m, whereas LIDAR (Garmin Lidar Lite v2) has a range of up to 40m. Both sensor types were effective in measuring different object types and were easily enclosed in a small shell.
To determine the accuracy and most effective navigation walking speed, a comprehensive set of experiments was conducted. The experiments were designed to test each system in a variety of settings that would replicate the constantly changing world. The first experimental setting was an office space, which provided a variety of unique obstacles in both height and depth dimensions (Figure 2). The second test environment, a much simpler map, exposed the system to different ground heights – steps and stairs going up and down (Figure 3).
Figure 2. Office space test environment.
Figure 3. Stairwell test environment. Figure 4. Experiment #1 (E-1) – Fastest Muscle – Test Locations.
Communication of Complex Stimuli
In designing a sensory substitution device, it is critical to identify those senses that can substitute for sight. These senses require the ability to discern the complex stimuli that represent the rich environmental information collected by the sensors and to respond quickly under real-time conditions. This sub-study focused on understanding the feasibility and effectiveness of using the senses that remain for BVI people, which are olfaction (smell), gustation (taste), audition (hearing), and somatosensation (touch). This work explored the limitations and abilities of human sensitivity to unique vibration stimuli – the sense of touch proved to be the most effective. It also examined how to effectively alert users to a wide range of obstacle information while still being understandable together with those other senses.
Experiment #1 (E-1) – Fastest Muscles
Figure 4. Experiment #1 (E-1) – Fastest Muscle – Test Locations.
The first experiment investigated the questions: which muscles respond quickest to vibrations; and, which muscles does the target population prefer to be notified through? Essentially, which body area is most effective to use when delivering spatial information as vibrations. Figure 4 depicts those regions tested. In this experiment the location of the vibration motor was manipulated (IV) and the user’s reaction time to vibration-stimuli was measured (DV). Users responded to the same stimuli (CV) and were instructed to respond the same way (CV).
Experiment #2 (E-2) – Optimal Settings
The second sub-set exploration investigated the questions: what is the minimum distance apart that motors of different frequencies can be positioned and be highly discernible by users; and, what is the most effective frequency step to communicate varying distances of obstacles? The regions tested in E-2 drew from the most responsive muscle groups in E-1.
In this experiment two vibration motors were used, incrementally spaced from each other (IV). The user’s accuracy in differentiating between which of the two felt the strongest was measured using a trigger button (DV). The difference between these frequencies, known as the frequency step, was incrementally increased for each spacing (CV). Motors were adjusted along 1-2-4-6 cm spacings and triggered at 10-17-24-31-38-45 Hz for each spacing (CV).
Consumer (User Design)
It is important to design a device that performs precisely, safely and accurately in real-time, but is also effective for people the device is designed for. To achieve this, the designer must deeply understand the target population. Previous SSD systems have had limited market acceptance because they focused on designing a strong navigation system but not on the equally important aspects of dignity, comfort, fixability, daily maintenance, customized modularity, and ultimately, cost efficiency.
Designing a final marketable product was beyond the scope of this work. However, a second stream of research was conducted to better understand how the purpose, performance and cost of the few available solutions could be improved in the design of this device. Customer satisfaction and comfort leading to acceptance and use is as important as device accuracy.
Several in-depth interviews and focus groups were conducted to understand the thoughts, opinions, and interests of the BVI community with respect to available SSDs and other navigational tools. These interviews sought to understand: the current tools used by the community; the challenges faced using these tools; how the low vision community currently perceives electronic travel aids; the interest in electronic travel aids; the interest for specific features in these aids; and what individuals expect and are willing to pay. Market research was also conducted to evaluate those researched solutions and how they score against the characteristics defined in Sub-Study 1 – Collecting Data, adding to the data collected in Elmannai’s evaluation of devices designed before 2017 (Elmannai & Elleithy, 2017).
System Combination
Finally, the best results gathered from each system investigation, excluding user design, were paired and tested to determine how effectively these systems performed when worn by subjects. The experiment sought to answer the question: how efficient is the device compared to a white cane and guide dog? Currently marketed SSDs were excluded from the project due to budget limitations on obtaining SSDs. Users traversed the same paths as used in Sub-Study #1 – Collection (CV). Time and accuracy were measured (DV); the walking speed was varied, but constant for each trial (IV).
OBSERVATIONS AND RESULTS
Collecting Data (Rich Environmental Information)
Figure 5. First prototyped sensor shell.
Figure 6. Second prototyped sensor shell.
This sub study investigated an evolution of ideas through two prototype shells (See Figures 5 & 6). Initially the hypothesis was that all locations around the user are equally important, so sensors were positioned symmetrically, with a conical detection spread (Figure 7). However, through testing in complex environments it was determined that the most important areas to monitor were at and below shoulder level. These results led to the design of a second shell, that prioritized these zones relative to those above shoulder height; however, zones above the head were still monitored (Figure 8). The second shell was used to determine the relationship between user walking speed and Collection System accuracy; testing the system at walking speeds of 1m/s (casual), 1.5m/s (rushed), and 2m/s (very fast). The results showed that an increase in walking speed reduced the accuracy of the system’s outermost sensors. The greater the angle of the sensor from the centre, the less accurate the measured distances, as objects are moving away faster than those measured at a smaller angle.
Figure 7. First Shell - Sensor Location and Measuring Spread.
Figure 8. Second Shell - Sensor Location and Measuring Spread.
Communication of Complex Stimuli
From literature review it was determined that utilizing somatosensation, or the sense of touch, to communicate spatial information would be the most effective and affordable. Communicating through touch does not interfere with other sensorimotor functions or hinder the already limited navigational abilities of users. For example, communicating information auditorily would be ineffective, as it would overlay important information over a sense that is already used to understand reverberations from the environment. It would not only be ineffective; it would interfere with important auditory cues.
Experiment #1
Many muscle locations were eliminated as sites to receive spatial information, such as the quadriceps, due to their slow reaction time. This region, with its high density of muscle tissue, is not particularly sensitive. The range of reaction times among many muscle groups is very narrow, and it was also learned that sensitivity is not necessarily determined by muscle location.
The fastest reaction time occurred at the shoulders (mean = 0.299 ms); other locations that could be considered are the biceps (mean = 0.303 ms), triceps (mean = 0.303 ms), and external obliques (mean = 0.303 ms), (Figure 9). In general, the arms were the area most reactive to vibrations, which led to the design of the second experiment.
Figure 9. Data Results for Sub-Study #2 – Experiment #1.
Experiment #2
Forearms: The results indicate that it is optimal to space the motors at 2 cm and use step distance increments of 38Hz. Among both BVI and Sighted groups, another optimal setting would be 4 cm spacing at the same 38 Hz (Figure 10).
Figure 10. Experiment #2 – Forearm Data Results.
Biceps: Connected by joints to the forearms, the biceps results are similar to those for the forearms. The results indicate higher sensitivity to 38 Hz, and two optimal spacings at 1 cm and 4 cm. Unique to the biceps is the specific reactivity to 24Hz at 4cm (Figure 11).
Figure 11. Experiment #2 – Bicep Data Results.
Triceps: Unlike the forearm and biceps, data from the triceps show mixed results. The sighted (VP) group favours a closer spacing at 38 Hz, while the BVI (VI) group favours a closer spacing at the higher frequency of 45 Hz. The optimal setting appears to be a 1cm to 2cm spacing, at 38 Hz (Figure 12).
Figure 12. Experiment #2 – Triceps Data Results.
Deltoids: Unique to the deltoids are three peaks in optimal frequencies. For the visually impaired group the response was most accurate at 38Hz at 3cm spacing, while the sighted group showed peaks at 17Hz and 38 Hz. An additional optimal peak appears for the visually impaired group at 2 cm and 17Hz. Variability suggests that both groups cannot be compared equally (Figure 13).
Figure 13. Experiment #2 – Deltoids Data Results.
Consumer Design
Secondary research revealed that all SSDs available today are extremely expensive and only focus on preforming one task well. Based on my evaluation, devices reviewed did not meet the required criteria, (Figure 14). Responses from participants fell into one of three categories:
Figure 14 : Evaluation of Previously Researched SSD Systems.
Modularity (User-flexible): When designing an SSD for a population that is highly heterogeneous the solution cannot be restricted to a single design. Modularity gives users the flexibility to personalize a device to their skills and abilities.
Convenience (Multi-Purpose): It is difficult for individuals to manage and operate several single purpose devices simultaneously. The ideal SSD would integrate all the strong features into one device – a device that does it all, efficiently and conveniently.
Cost: Companies should consider reducing their prices as much as possible, and the government should increase subsidizes. Independent navigation should not be a luxury!
System Combination
Using the combination of sensor unit and vibration feedback was difficult. Similar to learning a new language, teaching those who were unfamiliar with the device was challenging. The office space test environment generated a positive response, at a basic level, for navigational abilities. Users were able to traverse the environment with high accuracy and reduced collisions. The device was as effective as the white cane; however, it underperformed when communicating the exact position of stairs, something a white cane is exceptionally good at. The relatively short testing period made it challenging for users to experience full device potential. A longitudinal study is currently being conducted to understand how users learn to use the device, identify strategies used to navigate with the device, and how best to reduce learning time.
CONCLUSION
The objective of this research was to investigate if a more effective solution could be designed to overcome the social and navigational challenges faced by the blind and visually impaired (BVI) community. The results from each sub-study showed a positive correlation to meeting this goal in a wearable, sensory substitution device (SSD). The final prototype collects significantly more spatial information than a white cane, which can be conveyed to the user as quickly as the white cane. Users were able to navigate smoothly and with confidence, once they learned the different cues. The final prototype was not subject to a longitudinal study to measure the effects of the design on social interactions; however, feedback from users indicated that the device would allow for closer and more personal interaction. The final prototype satisfied the basic design goal; however, it is not ready for real-world applications. The work is continuing, to gather data from users and to identify and understand strategies that reduce the time to learn how to navigate with this device.
FUTURE DIRECTION
The work presented at the Canada-Wide Science Fair was important to understand the foundational aspects of designing an advanced electronic travel aid primarily focused on obstacle avoidance. A device closer to commercial production could include an enhanced navigational service. For example, the device could determine location by incorporating GPS. The future direction of this device is commercialization. The marketable product would include functions like voice commands; Bluetooth connectivity to your smartphone, so you may control your in-phone applications; bone-conducting technology, to communicate auditory cues unobtrusively; advanced haptic fabric, highly comfortable and enclosing the haptic feedback system neatly; and menu functions to personalize preferences in-device. The goal for commercialization is optimistic at 2-3 years, adding another year for product launch.
ACKNOWLEDGEMENTS
All electronics, prototypes, and software – excluding manufactured components – were created in-house, without any guidance or mentorship. I would like to thank the CCB, CNIB, OBSA, and Variety Village for connecting me directly with the blind and low vision community, which was critical for arranging testing and conducting focus groups. Furthermore, I’d like to thank my school, Danforth C.T.I., for make numerous accommodations in support of my work; and those members of the Institutional Review Board (IRB) who volunteered time to evaluate the ethics of my work for testing on human participants. Those mentionable names are:
Family and Friends | Aaron Levisohn, Ajwad Kabhir, Brent Whitby, Cam Valcic, Evan Baxter, Gabriel Popovic, Laura Fyles, Ottley Whitby, Jack Evoy
Epsilon Learning Centre | Leslie Loh
Canadian Council for the Blind (CCB) | Ian White Canadian National Institute for the Blind (CNIB) | Shane Laurnitus
Danforth C.T.I. (Toronto) | Danijela Ninkovic, Deryk Jackson, Dora Liarakos, Ivonne Durairaj, Mohina Lal, Rob Mackinnon, Roberta Tevlin, Thomas Jackson Marquette University | Valay Shah
Ontario Blind Sports Association (OBSA) | Richard Amelard
Variety Village | Rob Hampson
Manning Whitby
I am highly passionate for the field of STEM and extremely process driven, applying the scientific process to many aspects of my life in order to improve efficiency. The research conducted here is just the tip of a much larger field I aim to pursue. With the opportunity from the CWSF, I had significant financial aid to study at the University of Ottawa, for a Major Computer Science with a Minor in Psychology. For young innovators looking for inspiration, I advise you to not look too hard. People of all populations face challenges at every level. Slow yourself and you will find one that interests you.
References
Bourne, R. R., Flaxman, S. R., Braithwaite, R., Cicinelli, M., Aditi, D., Jons, J. B., & al, e. (2017, September 1). Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. 5(9), PE888-E897. doi:https://doi. org/10.1016/S2214-109X(17)30293-0
Bruce, Ian, McKennel, Aubrey, Walker, & Errol. (1991). Blind and Partially Sighted Adults in Britain: the RNIB Survey (Vol. 1). London: Stationary Office Books.
Elmannai, W., & Elleithy, K. (2017, March 17). Sensor-Based Assistive Devices for Visually-Impaired People: Current Status, Challenges, and Future Directions. Sensors (Basel), 17(3), 565. doi:10.3390/s17030565
Golledge, R. G., Marston, J. R., & Costanzo, M. C. (2001). Attitudes of Visually Impaired Persons Toward the Use of Public Transportation. The University of California, Transportation Center, Berkeley. Retrieved from https://escholarship.org/uc/item/5pv2k256
Jacobson, R. D. (1998). Cognitive Mapping Without Sight: Four Preliminary Studies of Spatial Learning . Journal of Environmental Psychology , 18(1), 289305. Retrieved from https://people.ucalgary.ca/~rjacobso/web/publications/ JEB97.PDF
Shah, C., Bouzit, M., Youssef, M., & Vasquez, L. (2006). Evaluation of RUNetra tactile feedback navigation system for the visually-impaired. International Workshop on Virtual Rehabilitation, (pp. 72-77). New York. Retrieved February 10, 2019
Wiener, W. R., Welsh, R. L., & Blasch, B. B. (1997). Foundations of Orientation and Mobility (2 ed., Vol. 1). New York, New York, USA: AFB Press. Retrieved February 10, 2019
World Health Organization. (2009, May). Fact Sheet No. 282: Visual impairments and blindness. Retrieved February 10, 2019, from World Health Organization: https://www.ophed.com/system/files/2010/01/who-visual-impairment-and...2207-2207.pdf
World Health Organization. (2018, October 11). Blindness and vision impairment. Retrieved February 10, 2019, from World Health Organization: https://www. who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment
BIBLIOGRAPHY
Baldwin, D. (2003, October). Wayfinding technology: A road map to the future. 97(10), 612-620. Retrieved February 10, 2019, from https://www.researchgate.net/publication/283868418_Wayfinding_technology_A_road_map_to_ the_future
Baurman, M. K., & Mullen, S. (1965). Performance Tests of Intelligence for the Adult Blind: A Comparison of Three Measures. AWB Annual Report, pp. 76-85.
Beer, A. L., Batson, M. A., & Watanabe, T. (2010, November). Multisensory perceptual learning reshapes both fast and slow mechanisms of crossmodal processing. Cognitive Affective & Behavioral Neuroscience, 11(1), 1-12. doi:10.3758/s13415-010-0006-x
Yang, K., Wang, K., Bergasa L. M., Romera, E., Hu, W., Sun, D., Sun, J., Cheng, R., Chen, T., Lopez, E. (2018 May). Unifying Terrain Awareness for the Visually Impaied through Real-Time Semantic Segmentation. Sensors (Basel), 18(5), 1506. doi: 10.3390/s18051506
Marion, A. H., Michael, A. J. (2008). Assistive technology for Visually Impaired and Blind People. London : Springer. doi: 10.1007/978-1-84628-867-8
Jaime, S., Miguel E. (2007). Guidelines for Designing Mobility and Orientation Software for Blind Children. Proceedings from IFIP Conference on Human-Computer Interaction, 375-388. doi: 10.1007/978-3-540-74769-3_35
Slim, K., Marc, J. -M. M., Bernard, O., Chirstophe, J. (2011). Toward a Better Guidance in Wearable Electronic Orientation Aids. Proceedings from IFIP Conference on Human-Computer Interaction, 624-627. doi: 10.1007/9783-642-23768-3_98
Kupers, R., Ptito, M. (2014 April). Compensatory plasticity and cross-modal reorganization following early visual deprivation. Neuroscience & Biobehavioral Reviews, 41(1), 36-52. doi: 10.1016/j.neubiorev.2013.08.001
Yoshioka, T., Bensamia, S. J., Craig, J. C., Hsiao, S. S. (2009 February). Texture perception through direct and indirec touch: An analysis of perceptual space for tactile textures in two modes of exploration. Somatosensory & Motor Research, 24(1-2), 53-70. doi: 10.1080/08990220701318163