BlindSpot: An AI-Powered Intelligent Mirror Assisting with Facial Hygiene Analysis for Blind People

Abstract: Appearances are important for our everyday lives: it is how we present ourselves. Yet, for people who are blind or visually impaired, it can be challenging to notice or take care of their facial hygiene without sighted assistance. Furthermore, existing AI-powered visual assistive technologies such as Seeing AI do not provide such support.We first performed an online search to identify the needs of blind people and people who are visually impaired by checking their appearances. We found that checking appearances is a pertinent need shared by many people who are blind or visually impaired. For example, in the autoethnography by a blind girl [1], the author explains that growing up with visual impairments is physically and mentally strenuous on teenagers. In Li et al. [2], the authors find that people who are visually impaired “normally keep [their] appearance the same as before the loss of vision. This makes people focus less on [their] face and [their] visual impairments.” And that “people with visual impairments are attentive to their appearance,” by “applying makeup”, so that they “can control [their] appearance again.” Pradhan et al. [3] found that people with visual impairments care about their appearance as “they know people around them can see them,” they also “rely on sighted or partially sighted people to act as their mirrors.” Such prior work highlights the need to develop novel solutions and mechanisms to help blind people check on their appearances more independently. The advances in artificial intelligence, especially with regard to AI’s capabilities on analyzing images, brings tremendous potential to augment the vision of people who are blind or visually impaired. In this project, I propose the design of BlindSpot, which is an AI-powered intelligent mirror to help people who are blind and visually impaired check their appearances through uploading images to the platform. Our prior need-finding search revealed diverse needs by blind people on checking appearances, including checking food stains around mouth area, checking makeup quality, and checking cleanliness of their faces. To use BlindSpot, we imagine users can first upload a reference photo which displays their regular appearance, and then upload another photo that they wanted to have the intelligent mirror check for them. In cases where users do not have a reference photo, they can directly upload a photo of interest and ask BlindSpot to check their appearance for them. To evaluate the design concept of BlindSpot, we first constructed a repository composed of people’s headshot images with different appearance issues ranging from food stuck on teeth, to smudge of makeup. We then prompted GPT-4 to evaluate the headshot images and give users feedback for them to adjust their appearance. We found that in most of the scenarios, GPT-4 is able to generate accurate assessment and give helpful feedback. We report scenarios where GPT-4 makes mistakes and makes suggestions on how to further improve the design.

Keywords: Accessibility, Assistive Technology, Blind

DOI: 10.54941/ahfe1005486

Cite this paper:

​