Image and Video Processing for Visually Handicapped People
EURASIP Journal on Image and Video Processing volume 2007, Article number: 025214 (2008)
This paper reviews the state of the art in the field of assistive devices for sight-handicapped people. It concentrates in particular on systems that use image and video processing for converting visual data into an alternate rendering modality that will be appropriate for a blind user. Such alternate modalities can be auditory, haptic, or a combination of both. There is thus the need for modality conversion, from the visual modality to another one; this is where image and video processing plays a crucial role. The possible alternate sensory channels are examined with the purpose of using them to present visual information to totally blind persons. Aids that are either already existing or still under development are then presented, where a distinction is made according to the final output channel. Haptic encoding is the most often used by means of either tactile or combined tactile/kinesthetic encoding of the visual data. Auditory encoding may lead to low-cost devices, but there is need to handle high information loss incurred when transforming visual data to auditory one. Despite a higher technical complexity, audio/haptic encoding has the advantage of making use of all available user's sensory channels.
World Health Organization : Magnitude and causes of visual impairment. Fact Sheet no. 282, November 2004, http://www.who.int/mediacentre/factsheets/fs282/en
Massof RW, Rickman DL: Obstacles encountered in the development of the low vision enhancement system. Optometry and Vision Science 1992,69(1):32-41. 10.1097/00006324-199201000-00005
Peli E, Arend LE, Timberlake GT: Computerized image enhancement for visually impaired people: new technology, new possibilities. Journal of Visual Impairment & Blindness 1986,80(7):849-854.
Peli E, Goldstein RB, Young GM, Trempe CL, Buzney SM: Image enhancement for the visually impaired: simulations and experimental results. Investigative Ophthalmology & Visual Science 1991,32(8):2337-2350.
Alonso M Jr., Barreto A, Gualberto Cremades J: Image pre-compensation to facilitate computer access for users with refractive errors. Proceedings of the 6th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '04), October 2004, Atlanta, Ga, USA 126-132.
Alonso M Jr., Barreto A, Jacko JA, Adjouadi M: A multi-domain approach for enhancing text with visual aberrations. Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '06), October 2006, Portland, Ore, USA 34-39.
Jefferson L, Harvey R: Accommodating color blind computer users. Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '06), October 2006, Portland, Ore, USA 40-47.
Brabyn JA: New developments in mobility and orientation aids for the blind. IEEE Transactions on Biomedical Engineering 1982,29(4):285-289.
Brabyn JA: Developments in electronic aids for the blind and visually impaired. IEEE Engineering in Medicine and Biology Magazine 1985, 4: 33-37.
Leventhal JD, Uslan MM, Schreier EM: A review of technology related publications. Journal of Visual Impairment & Blindness 1990, 84: 127-132.
Kosslyn SM: Image and Mind. Harvard University Press, Cambridge, Mass, USA; 1980.
Carrieras M, Codina B: Spatial cognition of blind and sighted: visual and amodal hypothesis. European Bulletin of Cognitive Psychology 1992,12(1):51-78.
Kennedy JM: Drawing and the Blind: Pictures to Touch. Yale University Press, New Haven, Conn, USA; 1993.
Arditi A, Holtzman JD, Kosslyn SM: Mental imagery and sensory experience in congenital blindness. Neuropsychologia 1988,26(1):1-12. 10.1016/0028-3932(88)90026-7
Hatwell Y: Images and non-visual spatial representations in the blind. In Non-Visual Human-Computer Interactions, Colloque. Volume 228. Edited by: Burger D, Sperandio J-C. INSERM/John Libbey Eurotext, Montrouge, France; 1993:13-35.
Passini R, Proulx G: Way finding without vision: an experiment with congenitally blind people. Environment and Behavior 1988,20(2):227-252. 10.1177/0013916588202006
Ungar S, Blades M, Spencer S: The construction of cognitive maps by children with visual impairments. In The Construction of Cognitive Maps. Edited by: Portugali J. Kluwer Academic Publishers, Dordrecht, The Netherlands; 1996:247-273.
Fritz J, Way T, Barner K: Haptic representation of scientific data for visually impaired or blind persons. Proceedings of the 11th Annual Technology and Persons with Disabilities Conference, March 1996, Los Angeles, Calif, USA
Hill E, Rieser J, Hill M, Halpin J, Halpin R: How persons with visual impairments explore novel spaces: strategies of good and poor performers. Journal of Visual Impairment & Blindness 1993,87(8):295-301.
Kamel HM, Landay JA: A study of blind drawing practice: creating graphical information without the visual channel. Proceedings of the 4th International ACM Conference on Assistive Technologies (ASSETS '00), November 2000, Arlington, Va, USA 34-41.
Kamel HM, Roth P, Sinha RR: Graphics and user's exploration via simple sonics (GUESS): providing interrelational representation of objects in a non-visual environment. Proceedings of the 7th International Conference on Auditory Display (ICAD '01), July-August 2001, Espoo, Finland 261-265.
Roth P: Représentation multimodale d'images digitales dans des systèmes informatiques multimédias pour utilisateurs non-voyants. Ph.D. thesis, Computer Science Department, University of Geneva, Geneva, Switzerland; 2002.
Loomis JM, Lederman SJ: Tactual perception. In Handbook of Perception and Human Performance: Cognitive Processes and Performance. Volume 2. Edited by: Boff KR, Kaufman L, Thomas JP. chapter 31, John Wiley & Sons, New York, NY, USA; 1986.
Millar S: Understanding and Representing Space: Theory and Evidence from Studies with Blind and Sighted Children. Oxford University Press, Oxford, UK; 1994.
Tzovaras D, Nikolakis G, Fergadis G, Malasiotis S, Stavrakis M: Design and implementation of haptic virtual environments for the training of the visually impaired. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2004,12(2):266-278. 10.1109/TNSRE.2004.828756
Blauert J: Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, Cambridge, Mass, USA; 1997.
Dobelle WH, Quest DO, Antunes JL, Roberts TS, Girvin JP: Artificial vision for the blind by electrical stimulation of the visual cortex. Neurosurgery 1979,5(4):521-527. 10.1227/00006123-197910000-00022
Schmidt EM, Bak MJ, Hambrecht FT, Kufta CV, O'Rourke DK, Vallabhanath P: Feasibility of a visual prosthesis for the blind based on intracortical microstimulation of the visual cortex. Brain 1996,119(2):507-522. 10.1093/brain/119.2.507
Srivastava NR, Troyk PR: A proposed intracortical visual prosthesis image processing system. Proceedings of the 27th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE-EMBS '05), September 2005, Shanghai, China 5264-5267.
Dagnelie G, Massof RW: Towards an artificial eye. IEEE Spectrum 1996,33(5):20-29. 10.1109/6.490053
Humayun MS, Freda R, Fine I, et al.: Implanted intraocular retinal prosthesis in six blind subjects. Proceedings of the Association for Research in Vision and Ophthalmology (ARVO '05), May 2005, Fort Lauderdale, Fla, USA
Espinosa M, Ochaita E: Using tactile maps to improve the practical spatial knowledge of adults who are blind. Journal of Visual Impairment & Blindness 1998,92(5):338-345.
Rieser JJ: Access to knowledge of spatial structure at novel points of observation. Journal of Experimental Psychology: Learning, Memory, and Cognition 1989,15(6):1157-1165.
Warren D, Strelow E: Electronic Spatial Sensing for the Blind. Martinus Nijhoff, Boston, Mass, USA; 1985.
Easton R, Bentzen B: The effect of extended acoustic training on spatial updating in adults who are congenitally blind. Journal of Visual Impairment & Blindness 1999,93(7):405-415.
Crandall W, Bentzen B, Myers L, Mitchell P: Transit accessibility improvement through talking signs remote infrared signage, a demonstration and evaluation. The Smith-Kettlewell Eye Research Institute, Rehabilitation Engineering Research Center, San Francisco, Calif, USA; 1995.
Golledge R, Klatzky R, Loomis J: Cognitive mapping and way finding by adults without vision. In The Construction of Cognitive Maps. Edited by: Portugali J. Kluwer Academic Publishers, Dordrecht, The Netherlands; 1996:215-246.
Burdea G, Coiffet P: Virtual Reality Technology. John Wiley & Sons, New York, NY, USA; 2003.
Standen PJ, Brown DJ, Cromby JJ: The effective use of virtual environments in the education and rehabilitation of students with intellectual disabilities. British Journal of Educational Technology 2001,32(3):289-299. 10.1111/1467-8535.00199
Schultheis M, Rizzo A: The application of virtual reality technology for rehabilitation. Rehabilitation Psychology 2001,46(3):296-311.
Giess C, Evers H, Meinzer H: Haptic volume rendering in different scenarios of surgical planning. In Proceedings of the 3rd Phantom Users Group Workshop (PUG '98), October 1998, Cambridge, Mass, USA. MIT; 19-22.
Gorman P, Lieser J, Murray W, Haluck R, Krummel T: Assessment and validation of force feedback virtual reality based surgical simulator. In Proceedings of the 3rd Phantom Users Group Workshop (PUG '98), October 1998, Cambridge, Mass, USA. MIT;
Jansson G, Fanger J, Konig H, Billberger K: Visually impaired persons' use of the phantom for information about texture and 3D form of virtual objects. In Proceedings of the 3rd Phantom Users Group Workshop, October 1998, Cambridge, Mass, USA. MIT;
Colwell C, Petrie H, Kornbrot D, Hardwick A, Furner S: Haptic virtual reality for blind computer users. Proceedings of the 3rd International ACM Conference on Assistive Technologies (ASSETS '98), April 1998, Marina del Rey, Calif, USA 92-99.
Sjöström C, Rassmus-Gröhn K: The sense of touch provides new computer interaction techniques for disabled people. Technology and Disability 1999,10(1):45-52.
Karshmer A, Bledsoe C: Access to mathematics by blind students: introduction to the special thematic session. Proceedings of the 8th International Conference on Computers Helping People with Special Needs (ICCHP '02), July 2002, Linz, Austria
Yu W, Ramloll R, Brewster SA: Haptic graphs for blind computer users. In Haptic Human-Computer Interaction. Edited by: Brewster S, Murray-Smith R. Springer, Berlin, Germany; 2001.
Parente P, Bishop G: BATS: the blind audio tactile mapping system. Proceedings of the 41st ACM Southeast Regional Conference (ACMSE '03), March 2003, Savannah, Ga, USA
Magnusson C, Rassmus-Gröhn K, Sjöström C, Danielsson H: Navigation and recognition in complex haptic virtual environments—reports from an extensive study with blind users. Proceedings of the Eurohaptics, July 2002, Edinburgh, UK
Lahav O, Mioduser D: Exploration of unknown spaces by people who are blind, using a multisensory virtual environment (MVE). Journal of Special Education Technology 2004,19(3):15-24.
Sánchez J, Lumbreras M: Virtual environment interaction through 3D audio by blind children. Cyberpsychology and Behavior 1999,2(2):101-111. 10.1089/cpb.1999.2.101
Semwal S, Evans-Kamp D: Virtual environments for visually impaired. Proceedings of the 2nd International Conference on Virtual Worlds (VW '00), July 2000, Paris, France 183: 270-285.
Grin C: Anoculoscope, appareil à faire voir les aveugles par le sens du toucher", "Description avec dessins photographiques, Paris, chez M. Grin, 6 rue Hippolyte-Lebas", of from Bernard et Cie, 1881, 48 pages. A somehow easier to obtain description of this work is: Gallois, "Anoculoscope: instrument pour faire voir les aveugles par le toucher. 1883.
Pun T: Tactile artificial sight: segmentation of images for scene simplification. IEEE Transactions on Biomedical Engineering 1982,29(4):293-299.
Way TP, Barner KE: Automatic visual to tactile translation. I. Human factors, access methods and image manipulation. II. Evaluation of the TACTile image creation system. IEEE Transactions on Rehabilitation Engineering 1997,5(1):81-105. 10.1109/86.559353
Hernandez SE, Barner KE: Tactile imaging using watershed-based image segmentation. Proceedings of the 4th International ACM Conference on Assistive Technologies, November 2000, Arlington, Va, USA 26-33.
Wall SA, Brewster S: Sensory substitution using tactile pin arrays: human factors, technology and applications. Signal Processing 2006,86(12):3674-3695. 10.1016/j.sigpro.2006.02.048
Palacz O, Kurcz E: The usefulness of modified electrophtalm EL-300 designed by Starkiewicz for the blind. Department of Pathopsychology of Vision, Medical Academy, Szczecin, Poland; 1977.
Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadden L: Vision substitution by tactile image projection. Nature 1969,221(5184):963-964. 10.1038/221963a0
Bach-y-Rita P: Visual information through the skin: a tactile vision substitution system (TVSS). Transactions of the American Academy of Opthalmology and Otolaryngology 1974, 78: 729-739.
Goldish LH, Harry E: The optacon: a valuable device for blind persons. New Outlook for the Blind 1974,68(2):49-56.
Stein DK: The Optacon: Past, Present, and Future. National Federation of the Blind (NFB), USA, 1998, http://www.nfb.org/Images/nfb/Publications/bm/bm98/bm980506.htm
Immersion Immersion Corp., 2006, http://www.immersion.com
Sensable Technology http://www.sensable.com
Ramstein C, Hayward V: The pantograph: a large workspace haptic device for a multi-modal human computer interaction. Proceedings of the Conference on Human Factors in Computing Systems (CHI '94), April, 1994, Boston, Mass, USA 57-58.
Fritz JP, Barner KE: Design of a haptic visualization system for people with visual impairments. IEEE Transactions on Rehabilitation Engineering 1999,7(3):372-384. 10.1109/86.788473
Nikolakis G, Moustakas K, Tzovaras D, Strintzis MG: Haptic representation of images for the blind and the visually impaired. Proceedings of the 11th International Conference on Human-Computer Interaction (HCI '05), July 2005, Las Vegas, Nev, USA
Fish R: An audio display for the blind. IEEE Transactions on Biomedical Engineering 1976,23(2):144-154.
Kay L: A sonar aid to enhance spatial perception of the blind: engineering design and evaluation. Radio and Electronic Engineer 1974,44(11):605-627. 10.1049/ree.1974.0148
Scadden LA: Blindness in the information age: equality or irony? Journal of Visual Impairment & Blindness 1984,78(9):394-400.
Kennel AR: Audiograf: a diagram-reader for the blind. Proceedings of the 2nd ACM Conference on Assistive Technologies (ASSETS '96), April 1996, Vancouver, BC, Canada 51-56.
Bennett DJ: Effects of navigation and position on task when presenting diagrams to blind people using sound. In Diagrammatic Representation and Inference, Springer Lecture Notes in Artificial Intelligence. Volume 2317. Springer, Berlin, Germany; 2002:161-175.
King A, Blenkhorn P, Crombie D, Dijkstra S, Evans G, Wood J: Presenting UML software engineering diagrams to blind people. In Proceedings of the 9th International Conference on Computers Helping People with Special Needs (ICCHP '04), July 2004, Paris, France, Lecture Notes in Computer Science. Volume 3118. Springer; 522-529.
Mikovec Z, Slavik P: Perception of pictures without graphical interface. Proceedings of the 5th ERCIM Workshop on User Interfaces for All (UI4ALL '99), November-December 1999, Dagstuhl, Germany
Meijer PBL: An experimental system for auditory image representations. IEEE Transactions on Biomedical Engineering 1992,39(2):112-121. 10.1109/10.121642
Capelle C, Trullemans C, Arno P, Veraart C: A real time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Transactions on Biomedical Engineering 1998,45(10):1279-1293. 10.1109/10.720206
Hollander AJ: An exploration of virtual auditory shape perception. M.S. thesis, University of Washington, Seattle, Wash, USA; 1994.
Gonzalez-Mora JL, Rodriguez-Hernandez A, Rodriguez-Ramos LF, Dfaz-Saco L, Sosa N: Development of a new space perception system for blind people, based on the creation of a virtual acoustic space. Proceedings of the International Work-Conference on Artificial and Natural Neural Networks (IWANN '99), June 1999, Alicante, Spain 2: 321-330.
Hedgpeth T, Rush PE M, Iyer V, Black J, Donderler M, Panchanathan S: iCare-reader: a truly portable reading device for the blind. Proceedings of the 9th Accessing Higher Grounds Conference Accessing Media, Web and Technology, November 2006, Boulder, Colo, USA
Krishna S, Little G, Black J, Panchanathan S: A wearable face recognition system for individuals with visual impairments. Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '05), October 2005, Baltimore, Md, USA 106-113.
Eddowes DM, Krahe JL: Pedestrian traffic lights recognition in a scene using a PDA. Proceedings of the 4th IASTED International Conference on Visualization, Imaging, and Image Processing (VIIP '04), September 2004, Marbella, Spain
Nagarajan R, Sainarayanan G, Yaacob S, Porle RR: Fuzzy-rule-based object identification methodology for NAVI system. EURASIP Journal on Applied Signal Processing 2005,2005(14):2260-2267. 10.1155/ASP.2005.2260
Uddin MS, Shioyama T: Detection of pedestrian crossing and measurement of crossing length—an image-based navigational aid for blind people. Proceedings of the 8th IEEE Conference on Intelligent Transportation Systems (ITSC '05), September 2005, Vienna, Austria 331-336.
Shioyama T: Computer vision based travel aid for the blind crossing roads. Proceedings of the 8th International Conference on Advanced Concepts for Intelligent Vision Systems (ACIVS '06), September 2006, Antwerp, Belgium, Lecture Notes in Computer Science 4179: 966-977.
Martinez-Alarcon JA, McKenna SJ: "Is it as I left it?"A computer vision aid for the blind. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC '04), October 2004, Hague, The Netherlands 7: 6439-6444.
Bologna G, Vinckenbosch M: Eye tracking in coloured image scenes represented by ambisonic fields of musical instrument sounds. Proceedings of the 1st International Work-Conference on the Interplay between Natural and Artificial Computation (IWINAC '05), June 2005, Las Palmas, Spain 327-333.
Bologna G, Deville B, Pun T, Vinckenbosch M: Transforming 3D coloured pixels into musical instrument notes for vision substitution applications. EURASIP Journal on Image and Video Processing 2007, 2007:-14.
Deville B, Bologna G, Vinckenbosch M, Pun T: Depth-based detection of salient moving objects in sonified videos for blind users. Proceedings of the 3rd International Conference on Computer Vision Theory and Applications (VISAPP '08), January 2008, Funchal, Portugal
Parkes D: "Nomad": an audio-tactile tool for the acquisition, use and management of spatially distributed information by visually impaired people. Proceedings of the 2nd International Symposium on Maps and Graphics for Visually Impaired People, April 1988, London, UK 24-29.
Parkes DN: Tactile audio tools for graphicay and mobility: "a circle is either a circle or it is not a circle". British Journal of Visual Impairment 1998,16(3):99-104. 10.1177/026461969801600304
Wall SA, Brewster S: Feeling what you hear: tactile feedback for navigation of audio graphs. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, April 2006, Montréal, Québec, Canada 1123-1132.
T3 Tactile tablet Royal National College for the Blind, UK, http://www.talktab.org
Kawai Y, Tomita F: Interactive tactile display system: a support system for the visually disabled to recognize 3D objects. Proceedings of the 2nd ACM Conference on Assistive Technologies (ASSETS '96), April 1996, Vancouver, BC, Canada 45-50.
Grabowski N, Barner KE: Data visualization methods for the blind using force feedback and sonification. Telemanipulator and Telepresence Technologies V, November 1998, Boston, Mass, USA, Proceedings of SPIE 3524: 131-139.
Moustakas K, Nikolakis G, Kostopoulos K, Tzovaras D, Strintzis MG: The force field haptic rendering method: application in haptic access to visual data for the training of the visually impaired. IEEE Multimedia Magazine 2007,14(1):62-72.
Roth P, Kamel H, Petrucci L, Pun T: A comparison of three nonvisual methods for presenting scientific graphs. Journal of Visual Impairment & Blindness 2002,96(6):420-428.
Roth P, Pun T: A multimodal system for the non-visual exploration of digital pictures. Proceedings of the 9th ICIP TC13 International Conference on Human-Computer Interaction (INTERACT '03), September 2003, Zürich, Switzerland
Sjöström C: The IT potential of haptics—touch access for people with disabilities. Licentiate thesis, Certec, Lund University, Lund, Sweden; 1999.
Saarinen R, Järvi J, Raisamo R, Salo J: Agent-based architecture for implementing multimodal learning environments for visually impaired children. Proceedings of the 7th International Conference on Multimodal Interfaces (ICMI '05), October 2005, Trento, Italy 309-316.
Coughlan J, Shen H: A fast algorithm for finding crosswalks using figure-ground segmentation. Proceedings of the 2nd Workshop on Applications of Computer Vision, in Conjunction with the European Conference on Computer Vision (ECCV '06), May 2006, Graz, Austria
Coughlan J, Manduchi R, Shen H: Computer vision-based terrain sensors for blind wheelchair users. Proceedings of the 10th International Conference on Computers Helping People with Special Needs (ICCHP '06), July 2006, Linz, Austria
About this article
Cite this article
Pun, T., Roth, P., Bologna, G. et al. Image and Video Processing for Visually Handicapped People. J Image Video Proc 2007, 025214 (2008). https://doi.org/10.1155/2007/25214