An AI Powered Glasses Attachment for the Visually Impaired

Abstract: This research presents the development and evaluation of a prototype AI-powered glasses attachment designed to enhance the daily mobility and independence of individuals with visual impairments. The work addresses the significant challenges faced by millions who experience limitations in navigating, recognising objects, and accessing information. It aims to contribute to the field of assistive technologies by creating a cost-effective and versatile solution. The project emphasises the importance of designing inclusive technology that is not only functional but also user-friendly and accessible to all. The system has several strengths:* Compact and portable design making it a discrete companion in daily life.* Versatile functionality combining object/text recognition with speech interaction.* Cost-effective approach making it accessible to a broader range of users.The core of the prototype integrates real-time object and text recognition, along with speech interaction capabilities. The system uses image processing algorithms, particularly convolutional neural networks (CNNs) to identify objects. Optical character recognition (OCR) is implemented to transform printed text into digital formats, enabling text-to-speech functionality. The prototype also incorporates a natural language processing (NLP) powered chatbot, facilitating spoken interaction and information retrieval. The project involved a thorough analysis of user requirements, focusing on the needs of individuals with varying degrees of visual impairment. This included understanding the challenges faced by people with different levels of sight loss, from moderate to complete blindness. The study also highlighted the importance of intuitive design and accessibility. The hardware design incorporates a compact ESP32-S3 microcontroller coupled with an OV5640 camera module, chosen for their balance of performance and power efficiency. The software is developed using Python for data processing and C++ for the microcontroller. Cloud-based AI services, including Google Cloud Text-to-Speech API and OpenAI's GPT API and Whisper API, are used for text-to-speech, object and text recognition, and speech recognition, respectively. The 3D-printed enclosure for the attachment was designed with several key considerations, including compactness, component integration, and user comfort. Key aspects of the hardware design are:* Component Layout: The enclosure was designed to house the ESP32 microcontroller, the OV5640 camera module, and a TTP223B touch sensor. The internal structure includes a separate compartment for the camera with a heatsink to manage its operating temperature.* Touch Sensor Integration: The touch sensor is located on the underside of the enclosure, designed with a concave depression for easy access and covered with a rubber membrane. This placement allows for user interaction and control of the system’s functions.* Size and Weight: The prototype has a weight of 23 grams. Its design aimed to be as small as possible whilst being able to accommodate the necessary hardware.The conducted functional tests focused on the accuracy of text and object recognition, and the efficacy of the chatbot. The text and object recognition performed well, however, issues with the camera quality, particularly in low-light conditions, limited the system's performance with smaller details and text. The speech interaction, while functional, encountered some difficulties with complex questions. The prototype is promising, future development should focus on addressing its limitations. Potential improvements include an automated configuration process, the integration of a navigation system and a more robust camera. Enhancing the user interface with a more tactile feedback system could improve accessibility.

Keywords: assistive technology, AI-Powered glasses, multimodal interaction

DOI: 10.54941/ahfe1006155

Cite this paper:

​