The combination of in-cabin cameras and artificial intelligence can provide a wealth of comfort and safety features to both drivers and passengers.
By Jean-Sébastien Landry
Today’s cars could be considered the dreams of science fiction not so long ago. Something as simple as electric windows- now considered a standard feature- were not always. Many drivers remember the hassle of having to wind their windows up and down by hand. The same can be said for wireless door locks, push-button start, and electric ignition.
The recent adoption of advanced tech for automotive is increasing at an exponential rate. In many respects, modern cars might be thought of as mobile computing, communication, and sensor platforms. A typical 2022 car might easily employ 50 or so microprocessors, while luxury models can boast 100 or more. Similarly, even relatively modest automotive offerings include a rear-facing camera to aid in reversing, while higher-end vehicles employ ten or more external cameras — along with radar, lidar, far infrared (FIR) and ultrasonic sensors — to power a wide range of advanced driver assistance systems (ADAS).
These sensors can be useful in their own right, but they really shine when combined with today’s state-of-the-art artificial intelligence (AI) systems. The fusion of data from multiple sensors coupled with advanced AI provides the driver with additional eyes focused on scanning the vehicle’s surroundings to identify potential problems (e.g., drifting out of lane without first indicating) and dangerous situations (e.g., the vehicle in front suddenly decelerating). In addition to providing appropriate warnings, these systems can go as far as take control of the car if necessary- maintaining its position in lane or applying the brakes in an emergency situation.
In the same way that cameras are now common outside the vehicle, there is a current trend for multiple in-cabin cameras to be present. In addition to one or more cameras mounted in the dashboard, on the steering column, or integrated into the rear-view mirror, it may not be long before it’s common practice to have a camera mounted near the light in the center of the ceiling to provide a bird’s eye view of the cabin’s interior.
The combination of in-cabin cameras and AI can provide a wealth of comfort and safety features to both drivers and passengers. In the case of a shared vehicle, for example, the car may recognize the user when the driver’s door is opened and automatically adjust comfort features like seating preferences, mirror positioning, and the default channel on the radio. While in motion, the Driver Monitoring System (DMS) may detect when the driver is distracted or starts to become drowsy. Furthermore, the car may also observe the driver looking for signs indicating health problems such as a heart attack, stroke, or epileptic fit.
In such a case, an emergency protocol could be deployed such as the hazard lights turning on and a call made to emergency services all while the car automatically brings itself to a safe stop.
The Occupant Monitoring System (OMS) could be used to identify the ages of passengers and prevent the doors from being opened by a child while the vehicle is in motion. On the other hand, an alarm could sound if the doors are prevented from being locked due to a child remaining inside the vehicle alone. In the case of a taxi or rideshare service, the vehicle could alert the driver if one of the passengers inadvertently leaves an object in the car (briefcase, package, smartphone, computer, etc.).
What will be the challenges for vision systems inside vehicles?
Automakers are committed to the safety of the occupants and their monitoring system must perform exceptionally well regardless of the light conditions inside the cabin. Also, ensuring complete coverage of the cabin can require multiple cameras, which comes at an additional price and impact the esthetic design of the cabin.
As the technology behind the camera lenses and sensors evolves rapidly, manufacturers will be able to provide wider in-cabin coverage and handle low-light sensitivity. It won’t be long until the DMS and OMS will be combined into a single camera module for the whole cabin.
This is an exciting time for everyone involved, from automotive designers to end users. What about you? Are you involved in creating next-generation automobiles, including their camera and AI systems? Reach out to our team to discuss your project and how Immervision can provide you with the best technology for your automotive projects.
About Jean-Sébastien Landry
Jean-Sébastien Landry has over twelve years of experience in the semiconductor and telecommunication industry working with OEMs and Tier1 companies. In his role of Director of Product Management at Immervision, he leads the development and commercialization strategy for Immervision’s automotive and drones/UAV/UAS product lines. Jean-Sébastien holds a degree in Electrical Engineering from University of Sherbrooke.