CITRIS Research Exchange

As the amount of available data increases, the human ability to locate, process, and analyze it is strained and eventually overwhelmed. To address this challenge for nonproliferation analysts, we have been designing a large-scale multimodal retrieval system to help analysts triage and search open source science, technology, transaction, and news data for indicators of nuclear proliferation capabilities and activities. Our system relies on a set of deep neural networks (DNNs) trained to evaluate conceptual similarities across data modalities, e.g. text, image, video. These DNNs can be used to search and prioritize data, according to a nuclear fuel cycle (NFC) process template, that are conceptually closest to the seed query items regardless of data modality. We evaluate the system's ability to retrieve NFC related data that have been purposely hidden in collections of unrelated background data. Quantitative and qualitative results for text-to-image, image-to-image, and image-to-video retrievals are demonstrated.

Speaker: Yana Feldman, Lawrence Livermore National Lab