MATILDA: Multi-sensor Automatic Threat Identification and Localization with Deep-learning Architecture

Securing forward operating bases (FOBs) is of vital importance to the Air Force for the protection of aircraft, assets, personnel, and facilities. This project aims to develop advanced machine learning technologies that provide real time multi-modality sensor fusion for identification of a various types of threats.Intelligent Automation Inc. (IAI), along with Dr. Tianfu Wu of North Carolina State University, proposes to develop threat detection system called: Multi-sensor Automatic Threat Identification and Localization Deep-learning Architecture or MATILDA. Our solution consists of a hierarchical deep temporal model that will reason over individual people as well as groups of detected people. It uses a context aware fully convolutional networks to detect, localize, and identify multiple objects of interest in the scene. These will allow LSTMs to process features from multiple objects and allow the deep model to learn the relations between the people and their appearances that contribute to recognizing different types of threat activities.