As a proud graduate of Carnegie Mellon University, I like to look in at some of the research that is ongoing. This post provides a little information about the Informedia project.

The overarching goal of the Informedia initiatives is to achieve machine understanding of video and film media, including all aspects of search, retrieval, visualization and summarization in both contemporaneous and archival content collections.

The base technology developed under Informedia-I combines speech, image and natural language understanding to automatically transcribe, segment and index linear video for intelligent search and image retrieval. Informedia-II seeks to improve the dynamic extraction, summarization, visualization, and presentation of distributed video, automatically producing “collages” and “auto-documentaries” that summarize documents from text, images, audio and video into one single abstraction.

This research has been ongoing since 1994. Some of the projects include:

• Informedia II Digital Video Library – established “new approaches for automated video and audio indexing, navigation, visualization, search and retrieval and embedded them in a system for use in education, information and entertainment environments.”
• Advanced Research Development Agency (ARDA) AQUAINT Program – The goal of Phase 1 was the “development of a system that provides analysts with answers from multi-media data streams, based on errorfully extracted information.” Phase 2 tools and techniques attempted to “automatically process and extract evidence from multimedia source content to understand questions, find answers, and to organize and present the answers as contextures supporting the analysts’ activities.”
• CareMedia: Automated Video and Sensor Analysis for Geriatric Care – “This research creates a meaningful, manageable information resource that enables more complete and accurate assessment, diagnosis, treatment, and evaluation of behavioral problems for the elderly. Through activity and environmental monitoring in a skilled nursing facility, a continuous, voluminous audio and video record is captured. Through work in information extraction, behavior analysis and synthesis, this record is transformed into an information asset whose efficient, secure presentation empowers geriatric care specialists with greater insights into problems, effectiveness of treatments, and determination of environmental and social influences on patient behavior.”
• Knowledge Discovery and Dissemination (KDD) – “Vast amounts of surveillance video, broadcast television and online multimedia overwhelm the human resources from the worldwide intelligence community who must watch and listen to it all. This research advances the ability to discover and track individual and group relationships from video sources. Emphasizing data mining link analysis and learning from extracted text and co-occurring imagery, the relationships amongst detected individuals and groups across locations and over time are extracted, visualized and summarized from continuously recorded and widely distributed video and other multimedia information sources. These include broadcast radio and television, field captured and surveillance video.”

Comments