Recently our own Patrice Roulet, Vice President, Technology and Co-Founder at Immervision, presented at AutoSens Detroit, the virtual conference bringing together the world’s leading community for ADAS and autonomous vehicle technology development. In his talk, Patrice shared how cameras equipped with super wide-angle lenses and sensors can be specified, simulated and designed with the purpose of improving machine perception. 

Intelligent vision systems are complex. They are not easy to prototype and not easy to benchmark. One element that adds to the complexity is the numerous components with multiple parameters that interact together. When you examine the camera pipeline, starting with the lens, then sensors and image processing, there are multiple parameters that impact the quality of the pixels, quantity of pixels and how reliable they are. All of these factors combine to determine how good your final image is. 

machine perception

For over 20 years Immervision has been working on understanding all aspects of the vision system and camera technology. From designing wide-angle lens with smart pixel technology to delivering a total camera solution equipped for various application requirements, we have invested time and knowledge to become experts. At the beginning our focus was mainly on human vision needs. Recently we are seeing more requests for computer vision and those requirements are different, which adds in another layer of complexity that affects machine perception.  

How can technology improve machine perception and accuracy?  

See More 

Wide-angle optics with smart pixel management is inspired by human eyes and should act in a similar fashion to provide a wide field of view, while at the same time capturing resolution in the area of interest, the same as an eye would. Pixel management and distortion control are two crucial elements to capture a more ‘human view’ but you must also factor in Modulation Transfer Function (MTF), aberration and RI (relative illumination). New computer vision requirements result in an increase in pixel quality and quantity so the object in the area of field can be in focus and magnified, which can only be achieved with Immervision wide-angle lens technology.  

machine perception

See Smarter 

All technologies need to be ‘smarter’ to work more efficiently with artificial intelligence. But how can we maximize all the different parameters to optimize the efficiency of machine learning algorithms? The best way is to start with the appropriate lens, camera and image processing algorithms, designed on purpose [for Machine Learning, machine perception]. and you will see an impact on the Machine Learning (ML) algorithms. At Immervision, we compare different types of lenses simulating different parameters that we then pass through the neural network to measure differences in accuracy. We can then benchmark the efficiency of the network considering the complete imaging pipeline from glass to AI determining which processing is best to maximize the accuracy of the network.  

machine perception

Those image shows what the different lens produce as image on the image sensor (CMOS)
Smart pixel management bring more pixel where is required by the application. in this example on the periphery of the image.
Smart pixel management is done by the lens. Optical design creates this feature

See Better Quality 

Pre-process the image to maximize the machine perception accuracy and efficiency to provide customers with the perception quality they are requesting. We have developed different algorithms called ‘adaptive dewarping’ to remap the pixels and change their quality depending on the application. For example, if we are focusing on the straight line ahead of a vehicle, with adaptive dewarping we have the ability, in real time, to adapt/remap the pixels, so depending on the need, we can focus on the straight line or object proportion. This gives us the unique ability to deliver a finely tuned image depending on the application.  

dewarping

What do all these technologies contribute to improve machine perception and accuracy? 

As we can intentionally control the lens and camera key performance criteria such as distortion, we can improve the accuracy of machine perception such as single frame depth estimation, object classification and more. The simulation pipeline with machine learning in the loop can predict the outcome of the intelligent vision camera system before engaging in prototyping phase. 

For image processing, once you identify which processing can maximize the accuracy of the machine perception, you can apply that process to ‘dewarp’ and remap the pixels to maximize the efficiency of the neural network and achieve a 5% increase in accuracy and precision.  

Through these technologies you will also see a noticeable improvement under different conditions, including challenging lighting, objects on the edge of the field of view, as well as small and far object detection.  

Keep Learning 

We are subject matter experts in the field of optics, camera, image processing and computer vision. Since 2000 and through a collaborative approach between scientists, software engineers and optical designers, we have developed and patented a wide range of advanced image processing algorithms that address the specific challenges of wide-angle imaging in radically new ways and render superior image quality. 

Check out these additional articles and content to give you more ideas about what we can do for machine perception.