The Combination of In-cabin Cameras and Artificial Intelligence Can Provide a Wealth of Comfort and Safety Features to Both Drivers and Passengers.

By Sophie Gigliotti

Today’s cars could be considered the dreams of science fiction not so long ago. Something as simple as electric windows – now considered a standard feature – were not always a possibility. Many drivers remember the hassle of having to wind their windows up and down by hand. The same can be said for wireless door locks, push-button start, and electric ignition.

The adoption of advanced technology for automotive hasn’t slowed down, in fact, it’s increasing at an exponential rate. In many respects, modern cars are highly sophisticated mobile computing, communication, and sensor platforms. A typical 2022 car might easily employ 50 or so microprocessors, while luxury models can boast 100 or more. Similarly, even relatively modest automotive offerings include a rear-facing camera to aid in reversing, while higher-end vehicles employ ten or more external cameras — along with radar, lidar, far infrared (LWIR) and ultrasonic sensors — to power a wide range of advanced driver assistance systems (ADAS).

Having sensors is useful, but the fusion of data from multiple sensors coupled with advanced AI provides exponential opportunities. This combination offers drivers with additional eyes focused on scanning the vehicle’s surroundings to identify potential problems (e.g., drifting out of lane without first indicating) and dangerous situations (e.g., the vehicle in front suddenly decelerating). In addition to providing appropriate warnings, these systems can go as far as take control of the car if necessary – maintaining its position in lane or applying the brakes in an emergency.

In the same way that cameras are now found on the outside of most vehicles, they are now being found inside the cabin. In addition to one or more cameras mounted in the dashboard, on the steering column, or integrated into the rear-view mirror, it is becoming common practice to have a camera mounted near the light in the center of the ceiling to provide a bird’s eye view of the cabin’s interior.

The combination of in-cabin cameras and AI can provide a wealth of comfort and safety features to both drivers and passengers. In the case of a shared vehicle, for example, the car may recognize the user when the driver’s door is opened and automatically adjust comfort features like seating preferences, mirror positioning, and the default channel on the radio. While in motion, the Driver Monitoring System (DMS) may detect when the driver is distracted or starts to become drowsy. Furthermore, the car may also observe the driver looking for signs indicating health problems such as a heart attack, stroke, or epileptic fit.

In such a case, an emergency protocol could be deployed such as the hazard lights turning on and a call made to emergency services all while the car automatically brings itself to a safe stop.

The Occupant Monitoring System (OMS) could be used to identify the ages of passengers and prevent the doors from being opened by a child while the vehicle is in motion. On the other hand, an alarm could sound if the doors are prevented from being locked due to a child remaining inside the vehicle alone. In the case of a taxi or rideshare service, the vehicle could alert the driver if one of the passengers inadvertently leaves an object in the car (briefcase, package, smartphone, computer, etc.).

What Will be the Challenges for Vision Systems Inside Vehicles?

Automakers are committed to the safety of the occupants and their monitoring system must perform exceptionally well regardless of the light conditions inside the cabin. Also, ensuring complete coverage of the cabin can require multiple cameras, which comes at an additional price and impact the esthetic design of the cabin.

As the technology behind the camera lenses and sensors evolves rapidly, manufacturers will be able to provide wider in-cabin coverage and handle low-light sensitivity. It won’t be long until DMS, OMS and other functionality will be combined into a single camera module for the whole cabin, reducing cost, improving efficiency and bring more value to drivers and passengers.

This is an exciting time for everyone involved, from automotive OEMs and Tier 1s to end users. What about you? Are you involved in creating next-generation automobiles, including their vision and monitoring systems?

Learn about Immervision’s Automotive Solutions.