Application of neural networks in occupational Safety and Health
Occupational health and safety, industrial safety and object video analytics, what do they have in common, and what potential intersection points do these completely different areas have? The answer is simple - the capabilities of video analytics in monitoring compliance with established rules, as well as automation of production and other processes of enterprises.
ComBox Technology is a system integrator in the field of information technology, video surveillance systems, video recording and recognition based on neural networks. We offer a full range of services for the development, creation, training and implementation of neural networks in industrial operation in the B2B / B2G segments. We have accumulated unique experience in working with video data and processing them. The company's portfolio includes 10 patents for hardware devices and software solutions in the field of object video analytics.
- ASUS/AAEON, smoking detection in car sharing services – https://www.aaeon.com/en/ai/success-story-vpc-3350s-aicx
- Russian Railways, detection of the state of railway traffic lights - https://habr.com/ru/post/515694/
- Intel IoT Solutions Alliance, counting passengers in transport - https://habr.com/ru/company/intel/blog/514618/
Currently, we produce a number of object video analytics solutions for such segments as: traffic control compliance, business process automation in retail and automotive, accounting and security control automation. Our software developments have been successfully proven in terms of recognition of vehicle types, brands, models, plates, as well as in the detection of people, poses and face recognition.
Video analytics system interface:
Figure 1. Displaying data from cameras in real time
Figure 2. Displaying and filtering the list of events by detection results
Possible use cases:
- Cloud service on our computing power. The advantage of this solution is the minimum implementation costs, but the data is transferred from the private circuit to the public cloud. Agreement is required regarding the transfer and storage of personal data and GDPR compliance (if applicable).
- Private circuit at the enterprise with the supply or use of existing server capacities. The advantage is scalability and the maximum number of processed streams in a closed circuit of the enterprise. The disadvantage is capital costs at the implementation stage.
- Edge solution with processing from 1 to 4 streams, located next to the camera. The advantages include fast scalability of the solution, but this option involves a large number of processing devices and a corresponding distribution of costs per point.
Service delivery method – a license agreement with the sale of fixed-term or perpetual inalienable licenses for the right to use selected detectors with SLA (Service Legal Agreement). The contract fixes stepwise criteria for the detection quality, according to which the customer pays the cost of video analytics from 100% reward for maximum quality to 0% for unacceptable (below a certain value).
Pose detection and recognition
One of the tasks related to occupational Safety is to detect the state of employees at work and notify internal services when incidents are recorded. One of the ways to detect incidents is pose recognition using object video analytics. Thus, it is possible to formalize a list of potentially dangerous poses in advance and generate information events when they are detected in a given zone (for a given period of time).
Figure 3. Pose detector
The following detectors can be used in parallel with pos detection:
- Head tilt vector detector
- View direction detector
- Detector of people in a given zone (including trajectory analysis and holding a person in a set of frames)
Figure 4. View direction detector and head tilt vector detector
The described detectors contribute to the achievement of the following results in production:
- Determination of the physical condition of employees
- Prompt incident response
Detection of PPE and other items of workwear or wearable equipment
Compliance with safety regulations and various requirements for occupational Safety may take into account the mandatory availability of personal protective equipment and special equipment for employees. For example, helmets, gowns, vests, respirators, masks, goggles and much more can be detected in industries.
In the face of the threat of coronavirus infection, it became necessary to wear medical masks. You can check how their detection works online, in the Telegram bot - https://t.me/imasks_bot. A photo of a person's face from a camera or from a smartphone gallery is sent to the bot, and as a result, it returns the probability of the presence or absence of a mask on the face.
The solutions described above are based on neural networks that process images from a camera or video stream in real time, detect and classify the found object with a certain probability.
Detection is performed from a variety of CCTV cameras, which can be installed both at checkpoints or entry / exit areas, and in the production work area. Reaction to an event - notification to the immediate supervisor, security service, sound or light alarm, depending on the purpose and place of installation.
Perimeter control tasks
The actual task may be marking security zones in the enterprise. As an example, consider monitoring the perimeter around active industrial equipment operating in semi-automatic mode. The presence of people in certain areas when the equipment is turned on is not allowed. It is possible to divide into 3 conditional zones around the equipment:
- Blue zone - alarm (light and / or sound) when a person is in the zone
- Gray zone - changing the parameters of the equipment and signaling to the control panel
- Red zone - emergency shutdown of equipment
- Detecting people on the railway track, at stations (virtual fence)
- Perimeter control for security services
- Counting the number of workers by zones
Demonstration of the process of marking and installation of detection zones:
Video 1. Demonstration of the process of marking and installing detection zones
ACS and checkpoint
Separately, it is worth highlighting the tasks of automating entry / exit zones for vehicles (parking, unloading zones, etc.).The standard solution is now a security post with entry passes and exit control.
The alternative is electronic passes and vehicle whitelisting. To do this, in the entrance zone to the protected area with a barrier or gate, several cameras are installed for entry and exit. As the vehicle approaches, the license plate is detected and recognized. Upon recognition, the presence of a pass in the system is checked and the opening of the barrier, if the pass is in the database.
Video 2. Demonstration of the license plate recognition system and automation of the vehicle entry zone
This allows you to strictly control the passage of vehicles at the checkpoint and accurately take into account the number of entered and deployed vehicles on the territory, which increases the throughput of entry and exit. This also allows for centralized management of multiple entry/exit zones with access to all logs. The very fact of minimizing direct contacts of employees of enterprises with drivers in checkpoint zones is also relevant.
Smoking detection in the workplace
Smoking not only causes great harm to the health of the smoker, but also negatively affects the health of the people around him. At the same time, the addiction can also affect the work process, since the time spent by the employee on smoke breaks adds up, as a result, to the hours lost from the work schedule.
Today, a variety of dust detectors and gas analyzers (eg CO2) are used to detect smoking. The general principle of operation of these devices is that the sensor detects changes in the environment, and the microcontroller creates a reaction event according to a predetermined algorithm. The obvious disadvantages are inapplicability in open spaces and low efficiency in rooms with high ceilings. Most sensors do not respond to electronic smoking devices.
Object video analytics using neural networks, where the input is a photo or video stream from a CCTV camera, and the output is the likelihood of the fact of smoking cigarettes, vapes or other smoking devices in the frame or set of frames, is a more effective alternative to gas analyzers and dust collection sensors.
At a low cost of hardware and software systems, many significant and security problems are solved, such as:
- Monitoring compliance with fire safety rules with high accuracy and photo-fixation of the fact of an offense (including time, date, place of the offense)
- Revealing the facts of offenses in hazardous industries and in companies whose activities are related to the use of flammable and fuel and lubricants
- Monitoring compliance with the internal regime at secure facilities
Figure 5. Smoking detector applications
- The ability to process data from multiple sources in one place
- The possibility of detecting the fact of smoking at a distance limited only by the focal length of the camera, a data source, for example, 5, 50 or 100 m (such values cannot be obtained with classical sensors and devices)
- The ability to detect smoking not only of classic cigarettes, but also of other smoking devices (electronic cigarettes)
- The ability to save the fact of the offense (photo and metadata of the event, such as date, time, location) when smoking in the wrong places
- Possibility of retrofitting existing cameras with the function of detecting the fact of smoking and reactions to the event
checkpoints, face authorization, and non-contact body temperature measurement
In the price segment up to $3k. a good solution is offered by UNIVIEW, namely, an access control terminal with face recognition and a digital temperature measurement module in a non-contact way, UNIVIEW OET-213H. The solution can be integrated within the existing access control system or installed separately from them:
Figure 6. UNIVIEW OET-213H
Temperature measurement method - contactless, on the wrist at a distance of 1-2.5 cm
Temperature measurement range - 30-45 degrees +/- 0.5 degree
Time required for recognition and measurement of temperature - 0.2 sec.
Face distance - 0.8-2.2 m, recognition accuracy 99% +
What the authorization process looks like:
Figure 7. Face identification with body temperature monitoring by wrist on UNIVIEW OET-213H
This example uses temperature detection, mask detection (warning in the 3rd photo), and face authorization. Left to right:
- Passage is allowed, the mask is there, the person is identified, the temperature is within normal limits
- Passage is prohibited, the temperature is higher than the maximum allowed value
- Pass is allowed, but a warning is displayed about the absence of a mask on the face
- Passage is prohibited, the temperature is higher than the maximum allowed value
Video analytics and neural networks are actively included in our lives and allow us to automate the processes of monitoring and analyzing data on photos/videos in real time. At the current stage of development, every day there are more and more new industry solutions, including potentially applicable in the field of occupational Safety. The review does not provide all the options for existing solutions, but only shows the direction in which the IT industry is developing today and the fact that in this direction there are already many potentially interesting and popular products that effectively solve current problems.Back to main page