demand analysis
Pingan City Construction plans to monitor and control video installations in cities across the country. It deployed more than 20 million cameras and played an active role in social management. However, the problems brought about by monitoring video data are increasingly prominent, mainly reflected in the effective use of massive video. After the “1.6†case took place in Nanjing in 2012, during the detection process of the case, the Nanjing police extracted a total of nearly 2000T of video data from more than 10,000 cameras in the city. In order to process these videos, more than 1,500 police officers were mobilized to search for search videos. It took more than a month. At present, video surveillance applications face the following important issues:
The video data is not well utilized, tasks such as human, vehicle and object search and identification, and video analysis, etc., are performed manually and cannot effectively process massive amounts of video data.
Due to the limitation of storage capacity, video data is deleted for one month and key information cannot be saved for a long time.
In response to the problems existing in the application of video facilities, the Department requires that: all levels of agencies should focus on the in-depth application of video surveillance technology, service, and actual combat to improve their performance. The requirements for monitoring and construction applications ranged from the first stage of hardware construction to the second stage of the main feature of intelligent applications, solving the problem of slow video retrieval and difficult analysis, and finding important information from massive videos. Value clues, quickly identify positioning targets, and tap their trajectories, thereby shortening the time to handle the case, reduce the intensity of the work of the investigators, and improve the joint operations efficiency of various departments. * Video surveillance, security, intelligence and other business departments should be equipped with practical tools and equipment with corresponding intelligent analysis capabilities. They can perform intelligent video analysis, information retrieval, feature recognition, image enhancement, alarming and monitoring of massive video.
For public safety applications, the Institute of Automation, Chinese Academy of Sciences, the China Internet of Things Research and Development Center (CIC) and China Science and Technology Co., Ltd. jointly conducted in-depth research on the video business and believed that the solution to these problems is to address massive video. Analytical analysis, research and development of intelligent video analysis, human-vehicle object recognition, target trajectory mining, mass data analysis and other cutting-edge high-tech, video surveillance and intelligence services for the * research and development of practical application-oriented professional applications, thereby improving the * business level and efficiency .
In response to these urgent needs, under the support of the National Science and Technology Support Program and the Special Projects of the Internet of Things of the Ministry of Industry and Information Technology, the Chinese Academy of Sciences and China Aosen have developed a massive video intelligent analysis system (VSearch). The system is oriented to the intelligent visual Internet of Things system for video surveillance, security, and intelligence applications. As shown in the figure below, it consists of two parts: (1) video clue analysis database (background), and (2) foreground video clue synthesis. Judgment (front end). Part (1) deals with the analysis of massive surveillance video data, extracts valuable video clues from it, and builds a massive video intelligence library. It describes multi-modal and omnidirectional descriptions of video objects, including space, time, appearance, and behavior. feature. Part (2) is a comprehensive business judgment platform, which provides multimodal video cues and information management, rapid target search, trajectory mining, case concatenation, and comprehensive judgment based on the massive video intelligence database. It can be seen that the system works in a similar way to Google's indexing and services and is a dedicated intelligent search system for China* business.
Video cues analysis and database
Massive video intelligent analysis is based on the intelligent visual Internet of Things-based visual tag system. The first step is to extract valuable video clues and build a massive video intelligence clue library. Video cues extraction includes target detection, target attribute extraction, target association, and relation mining. These clues form the basic elements of the video intelligence library and are stored in the cloud for client applications to access at any time.
In 2010, Dr. Li Ziqing of the Chinese Academy of Sciences proposed the Visual Internet of Things (VIOT). The core is the Internet of Things (IoT) visual labeling system. It analyzes the main items “people, vehicles, and objects†and their events in the environment and identifies the static attributes of the target labels. And dynamic behavior labels. The attribute tag is mainly the static attribute of the moving object in the video, such as the low-level features such as the color, shape, and texture of the moving object (such as clothes are leopard or plaid shirt), as well as the features of the headwear, glasses, hairstyles, and accessories. In addition, category attributes of sports goals (eg categories of pedestrians, bicycles, cars, buses, large trucks, etc., as well as pedestrians' gender, age, race, etc.), and specific attributes (such as pedestrian identity, and vehicle license plate number) are also included. Wait). The behavior tag is used to describe the motion information and behavior information of the moving object in the video data. For example, the moving direction and the moving speed of the moving object belong to the motion label. The public, deaf area, red light, public property, and graffiti are public Specific behavioral tags for the occasion.
The extraction of visual tags will constitute a multimodal, full-scale description of the moving object. This information will remain in the massive video intelligence database and constitute the key element of fast video retrieval. It is the core technology of VSearch.
Target attribute extraction
The target attribute (visual tag) extraction is the basic technology of VSearch. It requires the analysis of the main items “people, vehicles, objects†and their events in the environment, the identification of the target's static attribute tags, and the dynamic behavior tags. There are two steps: target detection and goal description.
Target detection detects and separates the target of the activity in the video and tracks its activity in a single scene. In actual monitoring scenarios, due to the different camera angles and changes in ambient lighting, higher requirements are placed on accurate pedestrian and vehicle detection and segmentation. As a core issue in intelligent video analytics, VSearch has industry-leading patented technology, changes in camera perspective, lighting, and dynamic background, as well as changes in the appearance of the target object (such as human movements and clothing, and the type and color of the car. Complex motion video caused by such as can achieve accurate pedestrian, vehicle detection, segmentation and tracking.
The purpose of VSearch is that there should be no omission in the target segmented from the video. This is the most concerned function in the business. Otherwise, valuable information may be lost. Followed by the control of false alarms. Too many false alarms not only affect the processing of subsequent modules, but also affect the efficiency of analysis and judgment.
The target description consists of a static attribute tag and a dynamic behavior tag. The attribute tag is mainly the static attribute of the moving object in the video, and the attribute tag extracted by VSearch includes:
Time attribute: time of movement target activity;
Spatial attribute: the spatial position of the moving target's activity;
Underlying feature attributes: such as the color, shape, and texture features of the moving object (if the garment is a leopard print or a plaid shirt);
Dressing attributes: such as hats, headwear, glasses, hairstyles, accessories and other features;
Category attributes: categories such as pedestrians, bicycles, cars, buses, big trucks, etc., and pedestrians' gender, age, race, etc.
Specific attributes: such as pedestrian identity, vehicle license plate number, etc.
The behavior tag is used to describe the motion information and behavior information of the moving object in the video data. The VSearch extracted behavior tag includes:
Motion labels: such as the moving direction, moving speed, and trajectory of a moving object;
Specific behavioral labels: such as defamation, imprisonment, red light, damage to public property, and scribbling are specific behavioral labels in public places.
The extraction of visual tags will constitute a multimodal, full-scale description of the moving object. This information extracted by VSearch will remain in the massive video intelligence database, which constitutes a key element of fast video retrieval.
Target association
Target correlation is the basis for achieving wide-range video-information retrieval across cameras. In practical applications, the monitoring range of a single camera is limited due to the physical distance and the spatial layout of the building. Usually, multiple cameras need to cooperate to jointly monitor a common area, such as airports, railway stations, subway stations, and shopping malls. Therefore, cross-camera target correlation becomes the key technology for VSearch system to collect large-scale video intelligence. With the cross-camera target correlation technology, VSearch will be able to draw out a wide range of activity trajectories for each moving target, which is very important to grasp the ins and outs of suspects when handling cases. VSearch technology overcomes many challenging problems that cause cross-camera target correlation difficulties, such as the complexity of the actual scene, lighting changes, distance changes, changes in the target attitude, target shielding, camera parameters and other difficult problems.
Due to the large population in the country, most public areas involve relatively dense crowds. Traditional color features in this case will not be sufficient to identify the target match across the camera. Massive video intelligent analysis and evaluation system has strengthened the research of high-definition camera video on the one hand, and enabled the local details of the target to be used as a more effective target matching method. On the other hand, VSearch technology also exploits multi-modal information such as time, space, and environment-related local constraints to achieve more reliable cross-camera target correlation.
The target discovery, association, and tag extraction described above all focus on the moving object itself. However, it is not enough to focus on the target itself in intelligence reconnaissance. Therefore, VSearch also pays attention to the interaction between a goal and its surrounding goals, such as affiliation (people who always move together) and contact (the two targets contact from time to time). At the same time, once the determination of the relationship is true, the trajectory of the wide range of activities of the associate or the contact person is also tapped at the same time, and all the visual tags associated with it and the derivative relationship of the associate or the contact person are extracted.
The deep relationship mining of VSearch technology can finally draw a map of associates or contacts, as well as a network of all gangs. These intelligences will have a complete grasp of the fugitives and a deep understanding of the activities of a gang. Important use value.
Cloud storage
The four processing steps described above will generate a huge amount of information. How to organize and store this information will also be a daunting task. The VSearch system aggregates large-scale monitoring videos step by step into distributed cloud servers, and guarantees the reliable security and stability of the transmission network. At the same time, the access of the storage side has set up step-by-step privilege control. In the entire surveillance video network of the private network, the Department has the highest authority, and it is down to the provincial office, the municipal office and local police stations.
The massive video intelligent analysis system supports the complete original video data stored in the case, the relevant video clips of the case and related information is stored in the city, and the case video cue information is stored in the provincial office. The large-scale case series video cue information national centralized storage Four-level storage management mechanism. Guarantee the integrity of the video information involved in the case and share and inquire. In order to formulate case storage standards for video information, a standardized, open storage platform for hierarchical management was established.
Comprehensive Judgment of Multi-modal Information by Video Clues
The surveillance video has changed the distribution and role model of case information under the background of traditional informationization, and has changed the way that case information is comprehensively judged. This is consistent with the development trend of the current "big information" information strategy. The video detection technology system involves the integrated use of information from the integration, to the competent authorities at the higher level, the coordination of various types of police and comprehensive use of various types of information, the construction of video trail comprehensive research and development technology platform and related standards, and ensuring sufficient and effective information acquisition to form the scale effect of information. Therefore, under the current background of informatization, how to use existing information resources to effectively integrate various types of video surveillance resources such as security monitoring, road monitoring, and social resource monitoring, and plan and implement it in a unified manner to meet the current frontline police video surveillance work. The battle mode, support for business operations of video-reconnaissance-related business units, support for case verification, and video sharing mechanisms for collusion cases have become the key to the comprehensive analysis of video clues.
The process of comprehensive analysis of video clues is a process that maximizes the use of video information, including the acquisition of known information, key information grooming, incomplete information inference, information expansion and correlation. The surveillance video not only reflects the static information of the crime scene, but also reflects the criminal behavior and the regional relevant dynamic information including the crime scene; it contains both local information of the time and space points, and includes large-scale information within the time period and space distance. . The information content is rich and the image has advantages that traditional information cannot match. At the same time, the surveillance video contains many redundant information associated with other types of information, which can be mutually verified with other information and further digging, which is conducive to the integration of intelligence information. For example, the appearance and appearance of the suspect in the surveillance video can be verified with the eyewitness description, and the vehicle information in the surveillance video can be verified with the vehicle library information.
In the construction of informationization, it currently contains public basic information. (The Golden Shield Project Information Base includes the National Population Basic Information Resource Base, the National Immigration Officer Resource Base, the National Motor Vehicle/Driver Information Base, and the National Police Basic Information Resource Base.) , National Information Database for At-risk Personnel, National Information Library for Criminal Offenders, National Information Database for Robbery, Information Library for National Security Key Units, etc.), Public Security Management Information System for the Hotel Industry, Guns, and Explosives Information database; site survey information (promotion site survey information system); criminal investigation application information system (five, including physical evidence information such as DNA database, fingerprint database, etc.); PGIS system and other professional application systems.
In the process of comprehensive analysis of video clues, police stations and investigators focused on video surveillance and collection. They mainly completed video extraction, suspected target retrieval, related information annotation, and uploading and uploading of video data; video investigation by the criminal investigation department (project team) of the district and county. Work as the main body, mainly completed the video summarization of the case involved, the clarity of processing, the information related to the judgment; the relevant departments of the prefecture and city with the investigation and technical support, mainly completed the regional video case information management, the complete technical system construction, the technical support and guidance for serious cases; The relevant departments of the hall take technical standards, planning and guidance as the main body, and mainly complete the technical system planning, information sharing platform construction, personnel training, and technical support for serious cases.
The "mass video intelligent analysis system" and "large-scale database face recognition system" were included in the "big video" system construction, juxtaposed with "big information". Implement unified management of video cues in cases, perform multi-modal and multi-dimensional description of images and video cues, conduct in-depth correlation analysis on major cases and crime cases, use video resources to capture law of occurrence, break through the direction, and improve video clue analysis. Use the "contribution rate" of the casework.
By constructing a massive video-information library, it is possible to achieve rapid query and analysis of specific targets through effective query means, thereby constructing a massive video intelligent analysis system (VSearch system) and its corresponding portable workstation equipment.
VSearch system
The massive video intelligent analysis and judgment system is oriented to the new video reconnaissance business and integrates intelligent video analysis technologies, including moving target extraction, visual tag retrieval, target trajectory mining, and target relationship mining, etc., to extract useful information in massive video and improve the monitoring video. Use efficiency to support * the intelligent video surveillance business platform.
For the characteristics of video reconnaissance services, VSearch is designed to be accessible to the video surveillance private network and connect to other video files. By means of manual semi-automatic video processing and manual processing, the two methods of automatic video processing and judgment are eliminated, and the case information clues contained in the video are saved to the case video library so that the police officers in front of the case can check the progress and clues at any time. , Or directly push the latest leads to police officers at the front office.
The main application businesses of VSearch* include:
Video thread browsing: This application provides * direct view of all the target images extracted, the advantage is that no goal will be missed, but the disadvantage is that the target number may be very large, it is difficult to quickly locate the suspect target.
Video Concise Quickview: Re-arrange all the extracted moving targets into a short video playback, similar to the video fast-forward function, but the regenerated video screen will not miss any moving target that has ever appeared. This application enables* to quickly browse all the objects in the video scene in a short time. At the same time, as long as you are interested in a target while watching the video, you can immediately locate the original video for detailed viewing.
Video target retrieval: Video target retrieval is the core business of VSearch. This service is similar to Google's search function and can search specific keywords that meet the requirements based on semantically clear keywords. All target attributes extracted by VSearch can be searched by single or combined search conditions. Such as: a red car; a red car on Zhongguancun East Road; a red car driving north on Zhongguancun East Road from 10:00 to 11:00 on September 16, 2012; a pedestrian wearing a white jacket; Exit A of Zhichun Road Subway Station Man in white shirt and black hat. All targets that meet the requirements will be listed for *policy review confirmation (just as in the Google search, confirming the retrieved web page entries). This greatly narrows the scope of investigations and speeds up the handling of cases. On the other hand, if there are screenshots or video clips of suspect targets, they can be used directly as input to VSearch to retrieve similar targets from the surveillance video. Image or video-based target re-identification technology can more accurately search for satisfying requirements than semantic retrieval, thereby realizing more convenient investigation.
Video image enhancement: For locked suspects, there may be ambiguities in the surveillance video, making key details unrecognizable (such as a person's face or vehicle's license plate number). VSearch provides video image enhancement technology and performs video image post-processing on the target of interest, so that the details of interest are significantly enhanced, providing more valuable clues for cracking cases. The current video image enhancement support functions include: denoising, deblurring, defogging, dark enhancement, and super-resolution based on dynamic sequences.
Target trajectory mining: After locking the suspect target, VSearch can further search for suspect targets in other related surveillance videos, and further trace the trajectory of the suspect target on the map, so as to grasp the ins and outs of the suspect target, obtain clues and make decisions. The Fleeing Plan provides a comprehensive reference.
Target relationship mining: After locking up suspect targets, VSearch can further mine suspected associates and contacts of the target, as well as their derived relationship networks. Deeper relational excavation can eventually draw out maps of associates or contacts, as well as a network of all gangs. These intelligences will have an overall grasp of the fugitives and a deep understanding of the activities of a gang. Use value.
VSearch device
The portable intelligent video workstation is a mobile extension of the VSearch system. The function of the VSearch system is the same as that of the aforementioned platform. It can be transferred to emergencies, major missions on the spot, can effectively enhance police force and business efficiency, and realize real-time policing. It is intelligent video intelligence. The three-dimensional extension of the reconnaissance platform is a golden weapon for massive video information processing and judgment.
From above, the massive video intelligent analysis system will build an information retrieval platform based on massive video and can respond quickly. The system works in a similar way to Google's indexing and services. It is a dedicated intelligent search system for China* business. The platform utilizes a massive video cloud intelligence database to retrieve real-time information on demand, such as target persons or vehicles that meet specific conditions (visual tags), the trajectory of the target in a wide range, the distribution of the target's associates, and their relationships. and many more. All of this information will be able to be presented in a three-dimensional manner and will be equipped with new and high-tech weapons.
Conclusion
The massive video intelligent analysis and evaluation system (VSearch) is based on the core technology of the Chinese Academy of Sciences. Its system products have been tested and drilled in actual combat, demonstrating the capability of intelligent analysis and judgment of video investigation, law and order and intelligence services. In the future, the key issues will be refined in practice and all efforts will be made to further strengthen core technology research and development so that it can handle accurate target tracking in dense populations, target correlation of long-distance cameras, and reliable inference of complex target relationship networks. VSearch continues to improve in the actual combat, gradually mature, will greatly enhance the police force and efficiency of handling cases. We look forward to the day!
About the author: Li Ziqing, MS, National Defense University, Ph.D., Surrey University, UK. Now he is director of the Biometrics and Safety Technology Research Center of the Institute of Automation, Chinese Academy of Sciences.
Pingan City Construction plans to monitor and control video installations in cities across the country. It deployed more than 20 million cameras and played an active role in social management. However, the problems brought about by monitoring video data are increasingly prominent, mainly reflected in the effective use of massive video. After the “1.6†case took place in Nanjing in 2012, during the detection process of the case, the Nanjing police extracted a total of nearly 2000T of video data from more than 10,000 cameras in the city. In order to process these videos, more than 1,500 police officers were mobilized to search for search videos. It took more than a month. At present, video surveillance applications face the following important issues:
The video data is not well utilized, tasks such as human, vehicle and object search and identification, and video analysis, etc., are performed manually and cannot effectively process massive amounts of video data.
Due to the limitation of storage capacity, video data is deleted for one month and key information cannot be saved for a long time.
In response to the problems existing in the application of video facilities, the Department requires that: all levels of agencies should focus on the in-depth application of video surveillance technology, service, and actual combat to improve their performance. The requirements for monitoring and construction applications ranged from the first stage of hardware construction to the second stage of the main feature of intelligent applications, solving the problem of slow video retrieval and difficult analysis, and finding important information from massive videos. Value clues, quickly identify positioning targets, and tap their trajectories, thereby shortening the time to handle the case, reduce the intensity of the work of the investigators, and improve the joint operations efficiency of various departments. * Video surveillance, security, intelligence and other business departments should be equipped with practical tools and equipment with corresponding intelligent analysis capabilities. They can perform intelligent video analysis, information retrieval, feature recognition, image enhancement, alarming and monitoring of massive video.
For public safety applications, the Institute of Automation, Chinese Academy of Sciences, the China Internet of Things Research and Development Center (CIC) and China Science and Technology Co., Ltd. jointly conducted in-depth research on the video business and believed that the solution to these problems is to address massive video. Analytical analysis, research and development of intelligent video analysis, human-vehicle object recognition, target trajectory mining, mass data analysis and other cutting-edge high-tech, video surveillance and intelligence services for the * research and development of practical application-oriented professional applications, thereby improving the * business level and efficiency .
In response to these urgent needs, under the support of the National Science and Technology Support Program and the Special Projects of the Internet of Things of the Ministry of Industry and Information Technology, the Chinese Academy of Sciences and China Aosen have developed a massive video intelligent analysis system (VSearch). The system is oriented to the intelligent visual Internet of Things system for video surveillance, security, and intelligence applications. As shown in the figure below, it consists of two parts: (1) video clue analysis database (background), and (2) foreground video clue synthesis. Judgment (front end). Part (1) deals with the analysis of massive surveillance video data, extracts valuable video clues from it, and builds a massive video intelligence library. It describes multi-modal and omnidirectional descriptions of video objects, including space, time, appearance, and behavior. feature. Part (2) is a comprehensive business judgment platform, which provides multimodal video cues and information management, rapid target search, trajectory mining, case concatenation, and comprehensive judgment based on the massive video intelligence database. It can be seen that the system works in a similar way to Google's indexing and services and is a dedicated intelligent search system for China* business.
Video cues analysis and database
Massive video intelligent analysis is based on the intelligent visual Internet of Things-based visual tag system. The first step is to extract valuable video clues and build a massive video intelligence clue library. Video cues extraction includes target detection, target attribute extraction, target association, and relation mining. These clues form the basic elements of the video intelligence library and are stored in the cloud for client applications to access at any time.
In 2010, Dr. Li Ziqing of the Chinese Academy of Sciences proposed the Visual Internet of Things (VIOT). The core is the Internet of Things (IoT) visual labeling system. It analyzes the main items “people, vehicles, and objects†and their events in the environment and identifies the static attributes of the target labels. And dynamic behavior labels. The attribute tag is mainly the static attribute of the moving object in the video, such as the low-level features such as the color, shape, and texture of the moving object (such as clothes are leopard or plaid shirt), as well as the features of the headwear, glasses, hairstyles, and accessories. In addition, category attributes of sports goals (eg categories of pedestrians, bicycles, cars, buses, large trucks, etc., as well as pedestrians' gender, age, race, etc.), and specific attributes (such as pedestrian identity, and vehicle license plate number) are also included. Wait). The behavior tag is used to describe the motion information and behavior information of the moving object in the video data. For example, the moving direction and the moving speed of the moving object belong to the motion label. The public, deaf area, red light, public property, and graffiti are public Specific behavioral tags for the occasion.
The extraction of visual tags will constitute a multimodal, full-scale description of the moving object. This information will remain in the massive video intelligence database and constitute the key element of fast video retrieval. It is the core technology of VSearch.
Target attribute extraction
The target attribute (visual tag) extraction is the basic technology of VSearch. It requires the analysis of the main items “people, vehicles, objects†and their events in the environment, the identification of the target's static attribute tags, and the dynamic behavior tags. There are two steps: target detection and goal description.
Target detection detects and separates the target of the activity in the video and tracks its activity in a single scene. In actual monitoring scenarios, due to the different camera angles and changes in ambient lighting, higher requirements are placed on accurate pedestrian and vehicle detection and segmentation. As a core issue in intelligent video analytics, VSearch has industry-leading patented technology, changes in camera perspective, lighting, and dynamic background, as well as changes in the appearance of the target object (such as human movements and clothing, and the type and color of the car. Complex motion video caused by such as can achieve accurate pedestrian, vehicle detection, segmentation and tracking.
The purpose of VSearch is that there should be no omission in the target segmented from the video. This is the most concerned function in the business. Otherwise, valuable information may be lost. Followed by the control of false alarms. Too many false alarms not only affect the processing of subsequent modules, but also affect the efficiency of analysis and judgment.
The target description consists of a static attribute tag and a dynamic behavior tag. The attribute tag is mainly the static attribute of the moving object in the video, and the attribute tag extracted by VSearch includes:
Time attribute: time of movement target activity;
Spatial attribute: the spatial position of the moving target's activity;
Underlying feature attributes: such as the color, shape, and texture features of the moving object (if the garment is a leopard print or a plaid shirt);
Dressing attributes: such as hats, headwear, glasses, hairstyles, accessories and other features;
Category attributes: categories such as pedestrians, bicycles, cars, buses, big trucks, etc., and pedestrians' gender, age, race, etc.
Specific attributes: such as pedestrian identity, vehicle license plate number, etc.
The behavior tag is used to describe the motion information and behavior information of the moving object in the video data. The VSearch extracted behavior tag includes:
Motion labels: such as the moving direction, moving speed, and trajectory of a moving object;
Specific behavioral labels: such as defamation, imprisonment, red light, damage to public property, and scribbling are specific behavioral labels in public places.
The extraction of visual tags will constitute a multimodal, full-scale description of the moving object. This information extracted by VSearch will remain in the massive video intelligence database, which constitutes a key element of fast video retrieval.
Target association
Target correlation is the basis for achieving wide-range video-information retrieval across cameras. In practical applications, the monitoring range of a single camera is limited due to the physical distance and the spatial layout of the building. Usually, multiple cameras need to cooperate to jointly monitor a common area, such as airports, railway stations, subway stations, and shopping malls. Therefore, cross-camera target correlation becomes the key technology for VSearch system to collect large-scale video intelligence. With the cross-camera target correlation technology, VSearch will be able to draw out a wide range of activity trajectories for each moving target, which is very important to grasp the ins and outs of suspects when handling cases. VSearch technology overcomes many challenging problems that cause cross-camera target correlation difficulties, such as the complexity of the actual scene, lighting changes, distance changes, changes in the target attitude, target shielding, camera parameters and other difficult problems.
Due to the large population in the country, most public areas involve relatively dense crowds. Traditional color features in this case will not be sufficient to identify the target match across the camera. Massive video intelligent analysis and evaluation system has strengthened the research of high-definition camera video on the one hand, and enabled the local details of the target to be used as a more effective target matching method. On the other hand, VSearch technology also exploits multi-modal information such as time, space, and environment-related local constraints to achieve more reliable cross-camera target correlation.
The target discovery, association, and tag extraction described above all focus on the moving object itself. However, it is not enough to focus on the target itself in intelligence reconnaissance. Therefore, VSearch also pays attention to the interaction between a goal and its surrounding goals, such as affiliation (people who always move together) and contact (the two targets contact from time to time). At the same time, once the determination of the relationship is true, the trajectory of the wide range of activities of the associate or the contact person is also tapped at the same time, and all the visual tags associated with it and the derivative relationship of the associate or the contact person are extracted.
The deep relationship mining of VSearch technology can finally draw a map of associates or contacts, as well as a network of all gangs. These intelligences will have a complete grasp of the fugitives and a deep understanding of the activities of a gang. Important use value.
Cloud storage
The four processing steps described above will generate a huge amount of information. How to organize and store this information will also be a daunting task. The VSearch system aggregates large-scale monitoring videos step by step into distributed cloud servers, and guarantees the reliable security and stability of the transmission network. At the same time, the access of the storage side has set up step-by-step privilege control. In the entire surveillance video network of the private network, the Department has the highest authority, and it is down to the provincial office, the municipal office and local police stations.
The massive video intelligent analysis system supports the complete original video data stored in the case, the relevant video clips of the case and related information is stored in the city, and the case video cue information is stored in the provincial office. The large-scale case series video cue information national centralized storage Four-level storage management mechanism. Guarantee the integrity of the video information involved in the case and share and inquire. In order to formulate case storage standards for video information, a standardized, open storage platform for hierarchical management was established.
Comprehensive Judgment of Multi-modal Information by Video Clues
The surveillance video has changed the distribution and role model of case information under the background of traditional informationization, and has changed the way that case information is comprehensively judged. This is consistent with the development trend of the current "big information" information strategy. The video detection technology system involves the integrated use of information from the integration, to the competent authorities at the higher level, the coordination of various types of police and comprehensive use of various types of information, the construction of video trail comprehensive research and development technology platform and related standards, and ensuring sufficient and effective information acquisition to form the scale effect of information. Therefore, under the current background of informatization, how to use existing information resources to effectively integrate various types of video surveillance resources such as security monitoring, road monitoring, and social resource monitoring, and plan and implement it in a unified manner to meet the current frontline police video surveillance work. The battle mode, support for business operations of video-reconnaissance-related business units, support for case verification, and video sharing mechanisms for collusion cases have become the key to the comprehensive analysis of video clues.
The process of comprehensive analysis of video clues is a process that maximizes the use of video information, including the acquisition of known information, key information grooming, incomplete information inference, information expansion and correlation. The surveillance video not only reflects the static information of the crime scene, but also reflects the criminal behavior and the regional relevant dynamic information including the crime scene; it contains both local information of the time and space points, and includes large-scale information within the time period and space distance. . The information content is rich and the image has advantages that traditional information cannot match. At the same time, the surveillance video contains many redundant information associated with other types of information, which can be mutually verified with other information and further digging, which is conducive to the integration of intelligence information. For example, the appearance and appearance of the suspect in the surveillance video can be verified with the eyewitness description, and the vehicle information in the surveillance video can be verified with the vehicle library information.
In the construction of informationization, it currently contains public basic information. (The Golden Shield Project Information Base includes the National Population Basic Information Resource Base, the National Immigration Officer Resource Base, the National Motor Vehicle/Driver Information Base, and the National Police Basic Information Resource Base.) , National Information Database for At-risk Personnel, National Information Library for Criminal Offenders, National Information Database for Robbery, Information Library for National Security Key Units, etc.), Public Security Management Information System for the Hotel Industry, Guns, and Explosives Information database; site survey information (promotion site survey information system); criminal investigation application information system (five, including physical evidence information such as DNA database, fingerprint database, etc.); PGIS system and other professional application systems.
In the process of comprehensive analysis of video clues, police stations and investigators focused on video surveillance and collection. They mainly completed video extraction, suspected target retrieval, related information annotation, and uploading and uploading of video data; video investigation by the criminal investigation department (project team) of the district and county. Work as the main body, mainly completed the video summarization of the case involved, the clarity of processing, the information related to the judgment; the relevant departments of the prefecture and city with the investigation and technical support, mainly completed the regional video case information management, the complete technical system construction, the technical support and guidance for serious cases; The relevant departments of the hall take technical standards, planning and guidance as the main body, and mainly complete the technical system planning, information sharing platform construction, personnel training, and technical support for serious cases.
The "mass video intelligent analysis system" and "large-scale database face recognition system" were included in the "big video" system construction, juxtaposed with "big information". Implement unified management of video cues in cases, perform multi-modal and multi-dimensional description of images and video cues, conduct in-depth correlation analysis on major cases and crime cases, use video resources to capture law of occurrence, break through the direction, and improve video clue analysis. Use the "contribution rate" of the casework.
By constructing a massive video-information library, it is possible to achieve rapid query and analysis of specific targets through effective query means, thereby constructing a massive video intelligent analysis system (VSearch system) and its corresponding portable workstation equipment.
VSearch system
The massive video intelligent analysis and judgment system is oriented to the new video reconnaissance business and integrates intelligent video analysis technologies, including moving target extraction, visual tag retrieval, target trajectory mining, and target relationship mining, etc., to extract useful information in massive video and improve the monitoring video. Use efficiency to support * the intelligent video surveillance business platform.
For the characteristics of video reconnaissance services, VSearch is designed to be accessible to the video surveillance private network and connect to other video files. By means of manual semi-automatic video processing and manual processing, the two methods of automatic video processing and judgment are eliminated, and the case information clues contained in the video are saved to the case video library so that the police officers in front of the case can check the progress and clues at any time. , Or directly push the latest leads to police officers at the front office.
The main application businesses of VSearch* include:
Video thread browsing: This application provides * direct view of all the target images extracted, the advantage is that no goal will be missed, but the disadvantage is that the target number may be very large, it is difficult to quickly locate the suspect target.
Video Concise Quickview: Re-arrange all the extracted moving targets into a short video playback, similar to the video fast-forward function, but the regenerated video screen will not miss any moving target that has ever appeared. This application enables* to quickly browse all the objects in the video scene in a short time. At the same time, as long as you are interested in a target while watching the video, you can immediately locate the original video for detailed viewing.
Video target retrieval: Video target retrieval is the core business of VSearch. This service is similar to Google's search function and can search specific keywords that meet the requirements based on semantically clear keywords. All target attributes extracted by VSearch can be searched by single or combined search conditions. Such as: a red car; a red car on Zhongguancun East Road; a red car driving north on Zhongguancun East Road from 10:00 to 11:00 on September 16, 2012; a pedestrian wearing a white jacket; Exit A of Zhichun Road Subway Station Man in white shirt and black hat. All targets that meet the requirements will be listed for *policy review confirmation (just as in the Google search, confirming the retrieved web page entries). This greatly narrows the scope of investigations and speeds up the handling of cases. On the other hand, if there are screenshots or video clips of suspect targets, they can be used directly as input to VSearch to retrieve similar targets from the surveillance video. Image or video-based target re-identification technology can more accurately search for satisfying requirements than semantic retrieval, thereby realizing more convenient investigation.
Video image enhancement: For locked suspects, there may be ambiguities in the surveillance video, making key details unrecognizable (such as a person's face or vehicle's license plate number). VSearch provides video image enhancement technology and performs video image post-processing on the target of interest, so that the details of interest are significantly enhanced, providing more valuable clues for cracking cases. The current video image enhancement support functions include: denoising, deblurring, defogging, dark enhancement, and super-resolution based on dynamic sequences.
Target trajectory mining: After locking the suspect target, VSearch can further search for suspect targets in other related surveillance videos, and further trace the trajectory of the suspect target on the map, so as to grasp the ins and outs of the suspect target, obtain clues and make decisions. The Fleeing Plan provides a comprehensive reference.
Target relationship mining: After locking up suspect targets, VSearch can further mine suspected associates and contacts of the target, as well as their derived relationship networks. Deeper relational excavation can eventually draw out maps of associates or contacts, as well as a network of all gangs. These intelligences will have an overall grasp of the fugitives and a deep understanding of the activities of a gang. Use value.
VSearch device
The portable intelligent video workstation is a mobile extension of the VSearch system. The function of the VSearch system is the same as that of the aforementioned platform. It can be transferred to emergencies, major missions on the spot, can effectively enhance police force and business efficiency, and realize real-time policing. It is intelligent video intelligence. The three-dimensional extension of the reconnaissance platform is a golden weapon for massive video information processing and judgment.
From above, the massive video intelligent analysis system will build an information retrieval platform based on massive video and can respond quickly. The system works in a similar way to Google's indexing and services. It is a dedicated intelligent search system for China* business. The platform utilizes a massive video cloud intelligence database to retrieve real-time information on demand, such as target persons or vehicles that meet specific conditions (visual tags), the trajectory of the target in a wide range, the distribution of the target's associates, and their relationships. and many more. All of this information will be able to be presented in a three-dimensional manner and will be equipped with new and high-tech weapons.
Conclusion
The massive video intelligent analysis and evaluation system (VSearch) is based on the core technology of the Chinese Academy of Sciences. Its system products have been tested and drilled in actual combat, demonstrating the capability of intelligent analysis and judgment of video investigation, law and order and intelligence services. In the future, the key issues will be refined in practice and all efforts will be made to further strengthen core technology research and development so that it can handle accurate target tracking in dense populations, target correlation of long-distance cameras, and reliable inference of complex target relationship networks. VSearch continues to improve in the actual combat, gradually mature, will greatly enhance the police force and efficiency of handling cases. We look forward to the day!
About the author: Li Ziqing, MS, National Defense University, Ph.D., Surrey University, UK. Now he is director of the Biometrics and Safety Technology Research Center of the Institute of Automation, Chinese Academy of Sciences.
It is intermediate to be used many chemicals.
General Auxiliary Agent
General Auxiliary Agent,Chemical Auxiliary Agent,Plastic Auxiliary Agents
NINGXIA PURU CHEMICAL CO., LTD , http://www.nxpuruchem.com