Mitsubishi Electric's
next-generation DMS,
which contributes to the safety and
comfort of all occupants, driving the future​

Developed the industry's first DMS with wide-angle camera​

NomiyamaEven before Mitsubishi Electric put the DMS on the market, there were already products that used a narrow-angle camera installed in the steering column to detect a driver's inattentiveness or drowsiness.
We were able to develop a DMS using similar specifications, but we intended to expand the imaging range not only to the driver's seat but also to the passenger seat with a single camera, and realize new functions that contribute to safety and comfort. Therefore, we developed the industry's first^※ DMS with a wide-angle camera installed in the dashboard center.​
However, using a wide-angle camera makes the occupant appear smaller, so the number of pixels that can be used for video analysis is reduced. Our strength is that we can accurately realize various functions even with a smaller number of pixels.​
Currently, we are working on the development of additional functions for all occupants using this wide-angle camera type DMS. ※For in cabin monitoring applications. Our research.

All passenger monitoring

Achieves smooth and speedy recognition that does not interfere with the movement of occupants​

YoneyamaThe personal recognition function of DMS helps improve usability and personalization by reproducing the seat position and air conditioning temperature in preset states when a registered person gets into the car. There are two main features of this personal recognition function.
The first is not to require the driver to have a specific pose when registering or recognizing an individual. We have made it so that registration and recognition can be performed in the natural movement of the ride without facing the camera.

The second is the speed of recognition. Since it uses a wide-angle camera, the system can get a face image from the time the user gets in the car, and it can complete the recognition before the vehicle even starts. However, since the wide-angle camera captures a wide range such as windows and rear seats, there is a lot of unnecessary information other than the driver. This made for development challenges, but we overcame many of these issues and realized mass production. We are proud of what we have achieved!​
Currently, we are working on more development to further improve recognition accuracy with the aim of utilizing our technology for applications that require high security such as e-commerce.

Quickly detects changes in the driver condition and helps to prevent accidents​​

ToramaWe are applying our various technologies to DMS in order to realize the functions described above. For example, we have a technology that detects an abnormal posture in which an unconscious driver collapses. By using this technology, DMS can judge that the posture is abnormal while observing the position of the head and the movement of falling down.
We also utilize technology to measure the driver's heart rate from camera images. The heart rate is estimated by capturing slight changes in the brightness of the skin that cannot be recognized by the human eye.​
Based on this technology, additional DMS functions are possible.

Judgment of abnormal condition​

Pulse detection

We utilize this DMS function in accident prevention efforts. You may have heard the news of an accident caused by the driver's loss of consciousness, but DMS should quickly detect changes in the driver and take measures such as urgently stopping the vehicle on the shoulder.
So, we hope to contribute to accident prevention.​​

Building a system that accurately recognizes in any situation

ToramaOne of the efforts to improve the recognition accuracy of DMS is robustness verification^※. In order to verify the robustness, we started by identifying various conditions that affect DMS on the desk.
For example, imagine a situation where sunlight hits your face, a situation where you shield your face while drinking a drink, a hat or mask, a hairstyle, a beard, the weather, and so on including different facial features.
Next, we needed to verify these conditions inside an actual vehicle, therefore we went to a location that met the conditions and evaluated them. For example, we evaluate it on snow in Hokkaido, and asked people of various locations to evaluate it overseas.
We couldn't visit these specific places many times, so if something went wrong, we repeated the needed fixes and revalidated it on the spot.
The work to improve the accuracy was really challenging.​
In these experiences, we are accumulating a lot of know-how. ※Resistance to external factors and changes

Case of Robustness verification
(collecting and evaluating various data of each environment)​​

YoneyamaCurrently, we are working on verification that combines this accumulated know-how with simulations using 3DCG technology.
For example, when trying to reproduce a situation where sunlight hits the face in an actual car, it is necessary to combine conditions such as weather, time, place and driver's seat position.
It is difficult to completely reproduce all this in the real world. But CG simulation can easily reproduce it. Furthermore, the driver's face will vary depending on different facial features, so that too can be verified very efficiently.

Examples of reproduction of various use cases by CG​

Utilizing past development and mass production know-how to deliver both function and design​

NomiyamaCustomers may request that the DMS be miniaturized or have an inconspicuous appearance in order to match the camera installation position and the interior design of the vehicle.
It is quite difficult to balance function and design, but we have refined our know-how from the development and mass production experience of many in-vehicle devices such as car navigation systems and displays.
For example, when we received a request to slim the product profile so it would not obstruct the front view, we were able, by revising the internal structure, to meet the request while maintaining the performance of the DMS.

Accelerating the further enhancement of next-generation DMS functions in the future​

ToramaIn the future, we will accelerate the further enhancement of next-generation DMS functions using wide-angle cameras, which is our strength.

Simulation of looking aside judgment by face direction and gaze detection.

By using a wide-angle camera, it is possible to detect the driver's inattentiveness and drowsiness, as well as determine the physique and behavior of the occupant, and detect the presence or absence of a child seat in the passenger seat.
This means that new safety and security functions that work with the control of airbags and seat belts will be possible.
By combining a wide-angle camera and a radar, it will also be possible to detect the abandonment of occupants and pets.​
In addition, by recognizing who is seated in the driver's seat or passenger seat, new functions that contribute to convenience and comfort in cooperation with music, video, air conditioning, etc. can be realized.
As another example, by capturing facial expressions, it will be possible to provide music that suits the occupant’s mood.
Furthermore, by using aperture recognition technology, it will be possible to determine which seat the person was in and when they spoke, and then perform voice operations according to the speaker.​

DMS is becoming an indispensable product for the automotive society of the future.
We want to play a part in the transformation of mobility represented by CASE and MaaS by engaging in product development that expands functions such as "comfort" and "convenience," as well as "safety" and "security," and by providing our vehicle interior monitoring products to vehicles around the globe.
Ultimately, we aspire to contribute to better quality of life for everyone.​​