Chaturchakshu
A Wearable Visual Guidance System for the Blind
The Problem
According to the World Health Organization, there are about 285 million people who are visually impaired worldwide, of whom 39 million are blind. These people face many challenges and difficulties in their daily lives, such as:
- Limited mobility and independence: They often rely on others or use assistive devices, such as canes or guide dogs, to move around and perform basic tasks. They also have difficulty accessing public transportation, education, employment, and health services.
- Reduced safety and security: They are more vulnerable to accidents, injuries, and crimes, such as falling, bumping, tripping, or being robbed or assaulted. They also have to deal with social stigma and discrimination.
- Poor experience and knowledge: They miss out on a lot of visual information that is essential for learning, communication, and enjoyment. They have difficulty recognizing faces, reading signs, watching movies, or appreciating art.
The Solution
This is a device that consists of a cap with Intel Real Sense Depth Camera, two TF Luna 1D Depth sensors, YD Lidar X2 a 2D Depth sensor, and a processing unit that processes the data from all the sensors. The camera captures the scene in front of the person and the processor uses TensorFlow API, an open-source machine learning library, to identify and classify multiple objects in the image. The processor then converts the text information to speech output and tells it to the user. The person can hear the narration of the scene and the objects in it, such as a cell phone, vase, person, couch, etc.
ChaturChakshu has many benefits for the blind and the visually impaired, such as:
The Technical Details
Smart Cap uses the following hardware and software components to achieve its functionality:
This is a device that consists of a cap with Intel Real Sense Depth Camera, two TF Luna 1D Depth sensors, YD Lidar X2 a 2D Depth sensor, and a processing unit that processes the data from all the sensors. The camera captures the scene in front of the person and the processor uses TensorFlow API, an open-source machine learning library, to identify and classify multiple objects in the image. The processor then converts the text information to speech output and tells it to the user. The person can hear the narration of the scene and the objects in it, such as a cell phone, vase, person, couch, etc.
ChaturChakshu has many benefits for the blind and the visually impaired, such as:
- It enhances their mobility and independence by allowing them to explore new places and environments without relying on others.
- It improves their safety and security by alerting them of potential obstacles and hazards in their path.
- It enriches their experience and knowledge by providing them with visual information that they otherwise miss out on.
- It boosts their confidence and self-esteem by giving them a sense of control and empowerment over their situation.
The Technical Details
Smart Cap uses the following hardware and software components to achieve its functionality:
- Hardware:
- Intel Real Sense camera: This is a camera module that can capture images in low-light conditions. With a field of view of 86 Degrees
- TF Luna: The TF-Luna is a small, lightweight LiDAR module designed for distance sensing in 1D, provides distance measurements with high precision.
- YD Lidar X2: The YD Lidar X2 is a compact LiDAR sensor module is desgined for distance sensing in 2D, which also provides distance measurements with high precision.
- Software:
- TensorFlow API: This is an open-source machine learning library that provides various tools and models for image recognition and classification. It uses deep neural networks to learn from large datasets and make predictions based on the input images. The device works as follows:
- Python: Used for integartion of all datas recieved from the sensors.
- The camera captures an image of the scene around the person and sends it to the Processing unit
- The Processing unit then uses the TensorFlow API to analyze the image and identify and classify the objects in it. It also assigns a confidence score to each object based on the probability of its recognition.
- The processing unit uses the pyttsx3 synthesizer to convert the text information to speech output. It also adds some contextual and directional cues to the speech output, such as “on your left”, “behind you”, or “in front of you”.
- The Processing unit sends the speech output to the earphones, which deliver it to the user’s ears.
The Student Team
From Left: (Back) Abhiroop Padate, Aaboli Shinde, Chinmay Parate
(Front) Avishka Joglekar, Arnav Padwal
(Front) Avishka Joglekar, Arnav Padwal