Seeing AI, launched in 2017 by Microsoft, is an intelligent camera application designed for people with vision impairment. With a combination of both artificial intelligence and computer vision, this application helps to narrate the world by describing text, objects and people using a mobile phone camera. Since its launch, it has guided people to complete five million tasks and has been downloaded by over 150,000 users [1].
The application is equipped with features known as channels [2], which help enhance the productivity of its users in their day to day activities. Users can turn the visual world around them into audible experience simply by pointing their phone’s camera at the object. It also helps to conveniently switch between channels depending on the requirements of the user.
Short Text Channel:
The short text channel helps users to read the short text written on envelopes, on a piece of paper, credit cards, vending machine etc. As soon as the text appears in front of the camera the voice over reads the text. Though sometimes slight adjustments to the camera need to be made to make this work but as soon as the camera is over the text and adjusted, voice over reading the text is instantaneous.
Document Channel:
The document channel enables the user to read the whole document. The edges of the document get detected with the voiceover guiding the user incase of adjustments needed in holding the phone. Once the edges are detected this channel takes a photograph automatically and then reads the document. It is helpful in scenarios like when a braille menu is not available in the restaurant and by using this channel a snapshot of the menu can be taken which can eventually make the process of ordering food easy.
Product Channel:
This channel identifies products based on their barcode. Audio cues in the form of beeps are given as feedback when it recognizes the barcode. Once the barcode is detected, it is scanned and the instructions are provided. Audio cues make it easy to find the barcode through feedback like increasing frequency of beeping as it sees the camera moving towards the barcode location on the product.
Recognizing People Channel:
This channel helps to recognize and describe the people around the user. The details of the person are given including an estimate of their age and also letting the user know their present emotion. The app guides the user to capture the photo through audio instructions and once the photo is taken the user hears the description. Users can tag the photo with a name and the application remembers it and helps the user to recognize the person when this channel is used again.
Currency Bill Channel:
The fifth channel of this application helps to identify currency bills. Identifying currency bills based only on shape and sizes is challenging. This application helps to mitigate this problem. The user can place the phone over the currency bill and the audio will denote the bill. This works even if the currency bill is crumbled or is not in a good shape. Currently, the application supports five different currencies: Euro, Indian Rupee, Canadian Dollar, US Dollar and British Pound.
This application touches both on the social model and functional model of disability. It is useful for people with low vision, dyslexia and print disablement. In the context of affordability, it is a free application. It is available in six different languages: English, Dutch, German, French, Japanese and Spanish making it accessible to people from different regions of the world. However, it is limited only to iPhone and iPad platforms and does not support Android devices. In conclusion, this application is good at addressing the various societal needs of people with visual challenges by successfully avoiding barriers to become independent learners through its various channels and helps them participate in society.
Bibliography:
- Seeing AI app can now identify Indian currency. (n.d.). Retrieved from https://news.microsoft.com/en-in/features/seeing-ai-indian-currency-recognition/
- Seeing AI App from Microsoft. (n.d.). Retrieved from https://www.microsoft.com/en-us/ai/seeing-ai