This PDF file contains the front matter associated with SPIE Proceedings Volume 8389, including the Title Page, Copyright information, Table of Contents, and Conference Committee listing

Remote detection of harmful intent is necessary for eective and appropriate countermeasures and will reduce risks to life and property. Trained human observers and sensor systems typically use facial expressions, gaits, gestures, perspiration, and a number of other observable characteristics as possible indicators of harmful intent with mixed results. It is proposed that responses of human subjects to external stimuli can be used as additional indicators of harmful intent in surveillance contexts. A variety of alerting stimuli, possible responses to the stimuli, features to be sensed by sensors, and the utility of these sensed features as indicators of harmful intent are discussed in this paper. An ontology-based data-to-decision framework for assessing human intent, which would leverage the formal representations of the alerting stimuli, as well as the variety of possible responses, is proposed in the context of Semantic Web infrastructure.

We introduce an approach to representing intelligence, surveillance, and reconnaissance (ISR) tasks at a relatively high level in controlled natural language. We demonstrate that this facilitates both human interpretation and machine processing of tasks. More specically, it allows the automatic assignment of sensing assets to tasks, and the informed sharing of tasks between collaborating users in a coalition environment. To enable automatic matching of sensor types to tasks, we created a machine-processable knowledge representation based on the Military Missions and Means Framework (MMF), and implemented a semantic reasoner to match task types to sensor types. We combined this mechanism with a sensor-task assignment procedure based on a well-known distributed protocol for resource allocation. In this paper, we re-formulate the MMF ontology in Controlled English (CE), a type of controlled natural language designed to be readable by a native English speaker whilst representing information in a structured, unambiguous form to facilitate machine processing. We show how CE can be used to describe both ISR tasks (for example, detection, localization, or identication of particular kinds of object) and sensing assets (for example, acoustic, visual, or seismic sensors, mounted on motes or unmanned vehicles). We show how these representations enable an automatic sensor-task assignment process. Where a group of users are cooperating in a coalition, we show how CE task summaries give users in the eld a high-level picture of ISR coverage of an area of interest. This allows them to make ecient use of sensing resources by sharing tasks.

There is a need for effective and efficient dynamic management of scarce high-demand/low-density ISR collection assets. This paper presents a doctrinal analysis of ISR management which leads to the conclusion that a key role in the developing of dynamic ISR support solutions to dynamic ISR support requests is played by the ISTAR Manager within Collection Operations Management. It develops the hypothesis that visualisation, situational awareness and planning (aka decision support) tools could support the ISTAR Manager in the more effective and efficient development of ISR support solutions. Finally, it describes the experimental capability developed by Dstl, together with a number of industrial partners, to test this hypothesis.

To improve the effectiveness of network-centric decision making, we present a distributed network application and framework that provides users with actionable intelligence reports to support counter insurgency operations. ARL's Quality of Information (QoI) Intelligence Report Application uses QoI metrics like timeliness, accuracy, and precision combined with associated network performance data, such as throughput and latency, and mission-specific information requirements to deliver high quality data to users; that is data delivered in a manner which best supports the ability to make more informed decisions as it relates to the current mission. This application serves as a testing platform for integrated experimentation and validation of QoI processing techniques and methodologies. In this paper, we present the software-system framework and architecture, and show an example scenario that highlights how the framework aids in network integration and enables better data-to-decision.

In a military scenario, commanders need to determine what kinds of information will help them execute missions. The amount of information available to support each mission is constrained by the availability of information assets. For example, there may be limits on the numbers of sensors that can be deployed to cover a certain area, and limits on the bandwidth available to collect data from those sensors for processing. Therefore, options for satisfying information requirements should take into consideration constraints on the underlying information assets, which in certain cases could simultaneously support multiple missions. In this paper, we propose a system architecture for modeling missions and allocating information assets among them. We model a mission as a graph of tasks with temporal and probabilistic relations. Each task requires some information provided by the information assets. Our system suggests which information assets should be allocated among missions. Missions are compatible with each other if their needs do not exceed the limits of the information assets; otherwise, feedback is sent to the commander indicating information requirements need to be adjusted. The decision loop will eventually converge and the utilization of the resources is maximized.

The FY 10-15 Defense Intelligence Guidance (DIG) issued by the Under Secretary of Defense for Intelligence (USDI) tasked the Defense Intelligence Agency (DIA) to generate an Unattended Ground Sensors (UGS) roadmap. The roadmap was intended to encapsulate a broad understanding of ongoing and planned UGS development and deployment activities across the Department of Defense (DoD). This understanding supports an evolution to a cross-DoD and intelligence community (IC) integrated approach for acquiring and fielding UGS. To further this approach, a memorandum signed by the Deputy Under Secretary of Defense for Technical Collection & Analysis endorsed the creation of an UGS Standards Working Group (SWG). The UGS SWG is working across the defense community via its newly established Technology Focus Groups (TFGs) to address the specifics of the open architecture standards for unattended sensors (OASUS). Derived from the Terra Harvest-developed, open, integrated battlefield UGS architecture, OASUS separates the UGS system into fundamental components and standardizes internal and external interfaces to optimize interoperability. The TFGs are looking specifically at software component architecture, wired and wireless interfaces, and physical and logical user interfaces. This paper details the progress of these focus group efforts and invites participation in ongoing development of unattended sensor standards that advance the technology beyond the familiar pattern of controller, trigger, and camera.

The scope and success of Department of Defense (DoD), Intelligence Community (IC), and other United States Government (USG) Unattended Sensor efforts clearly requires a focused, expanded effort at integration and best practices development. Discussions with key stakeholders indicate strong support for the standup of an Unattended Sensors Community of Interest (USCOI) to improve visibility across programs and foster greater mission partnerships. The USCOI will advance understanding of legal and privacy issues, data sharing, Intelligence Surveillance and Reconnaissance (ISR) architecture integration standards, technical specifications, and other issues of common interest (power management, concealment, etc.); promoting opportunities for cost avoidance and improved effectiveness. Launching the USCOI concept provides a stakeholders' forum to identify, discuss, and facilitate Doctrine, Organization, Training, Material, Leadership, Personnel and Facilities (DOTMLPF) resolutions to issues related to the acquisition, development, and employment of unattended sensors across the DoD, IC, and broader USG, as appropriate. USCOI also provides a structured forum where appropriately cleared law enforcement can share operational and technical experiences and expertise on integration of unattended sensor data in existing/planned collection and analysis architectures. The USCOI concept can help facilitate extensive independent discussion across the community and enhance subject matter expert and leadership perspectives. Many senior leaders believe DoD and the IC have a great deal to learn from law enforcement use of unattended sensors and data integration. The USCOI will broaden awareness and open dialogue among users and developers across the interagency, law enforcement, and academia.

The War fighter lacks a unified Intelligence, Surveillance, and Reconnaissance (ISR) environment to conduct mission planning, command and control (C2), tasking, collection, exploitation, processing, and data discovery of disparate sensor data across the ISR Enterprise. Legacy sensors and applications are not standardized or integrated for assured, universal access. Existing tasking and collection capabilities are not unified across the enterprise, inhibiting robust C2 of ISR including near-real time, cross-cueing operations. To address these critical needs, the National Measurement and Signature Intelligence (MASINT) Office (NMO), and partnering Combatant Commands and Intelligence Agencies are developing SensorWeb, an architecture that harmonizes heterogeneous sensor data to a common standard for users to discover, access, observe, subscribe to and task sensors. The SensorWeb initiative long term goal is to establish an open commercial standards-based, service-oriented framework to facilitate plug and play sensors. The current development effort will produce non-proprietary deliverables, intended as a Government off the Shelf (GOTS) solution to address the U.S. and Coalition nations' inability to quickly and reliably detect, identify, map, track, and fully understand security threats and operational activities.

Multiple sensors, providing actionable intelligence to the war fighter, often have difficulty interoperating with each other. Northrop Grumman (NG) is dedicated to solving these problems and providing complete solutions for persistent surveillance. In August, 2011, NG was invited to participate in the Tactical Network Topology (TNT) Capabilities Based Experimentation at Camp Roberts, CA to demonstrate integrated system capabilities providing Forward Operating Base (FOB) protection. This experiment was an opportunity to leverage previous efforts from NG's Rotorcraft Avionics Innovation Laboratory (RAIL) to integrate five prime systems with widely different capabilities. The five systems included a Hostile Fire and Missile Warning Sensor System, SCORPION II Unattended Ground Sensor system, Smart Integrated Vehicle Area Network (SiVAN), STARLite Synthetic Aperture Radar (SAR)/Ground Moving Target Indications (GMTI) radar system, and a vehicle with Target Location Module (TLM) and Laser Designation Module (LDM). These systems were integrated with each other and a Tactical Operations Center (TOC) equipped with RaptorX and Falconview providing a Common Operational Picture (COP) via Cursor on Target (CoT) messages. This paper will discuss this exercise, and the lessons learned, by integrating these five prime systems for persistent surveillance and FOB protection.

This paper presents the results of a US-UK collaboration project sponsored by the Coalition Warfare Program (CWP) of the Office of the Secretary of Defense (OSD). The goal of this project was to integrate a single framework a set of sensor and policy tools and protocols to enable the rapid assembly and deployment of a disparate set of coalition assets combined into a sensor network with policy controlled sharing/dissemination of data and information. This project was based on two key technology components developed as part of the International Technology Alliance in Network and Information Science (ITA) - the Information Fabric and the Policy Management Toolkit, and technology components developed by the Institute for Collaborative Biotechnologies (ICB) - the Autonomous UAV Persistent Surveillance using Bio-Inspired Strategies.

The development of SOAs in the tactical domain has been hindered by a lack of interface standards suitable for the environment of unpredictable and low bandwidth communications, low powered computers and dynamic ad-hoc grouping of tactical participants. Existing commercial SOA standards have assumed reliable access to central servers and services and having relatively static participants. The proposal describes an open and published message-oriented interface created to support the aims of the upcoming MOD Generic Base Architecture¹ (GBA) Defence Standard and the associated Land Open Systems Architecture². The aims are; a) to support multiple open transport protocols, such as HTTP, SMTP, DDS and MQTT; b) to be suitable for integrating together low-level utility functions with their controlling systems (such as water, waste and power) and integrating together high-level mission-support functions (such as ISTAR and C2); c) reduce operator burden by using automated discovery and configuration where possible; d) dynamically integrate with MOD Generic Vehicle Architecture³ platforms to link base and vehicle mission and logistics systems over tactical radio links; e) extensible to support features such as security classification; f) to be lightweight in implementation and bandwidth and not dependent on central servers for operation. The paper will present the proposed interface and describe the features required for a military tactical rather than a commercial environment, and will report the outcome of a MOD-funded proof of concept that uses the proposed interface to interoperate several military systems.

The Defense Intelligence Agency's Terra Harvest program, launched in 2009, is developing an open, integrated battlefield unattended ground sensors (UGS) architecture that ensures interoperability among disparate UGS components and systems. Having successfully demonstrated a Terra Harvest-compliant UGS controller at Empire Challenge 2010, the program has since developed a Terra Harvest open architecture specification and developer's guide; created a web-based forum for architecture evolution and industry collaboration; and developed an array of asset plug-ins. These software plug-ins send data to or acquire it from an asset, and then convert it into a viable payload capable of supporting intelligence-driven operations. Building on this foundation, commercial vendors are developing field-ready implementations for Trident Spectre 2012. Once fully matured, Terra Harvest will streamline acquisition processes, reduce deployment timelines, and serve users as an intelligence, surveillance, and reconnaissance (ISR) force multiplier.

Under the Terra Harvest Program, the DIA has the objective of developing a universal Controller for the Unattended Ground Sensor (UGS) community. The mission is to define, implement, and thoroughly document an open architecture that universally supports UGS missions, integrating disparate systems, peripherals, etc. The Controller's inherent interoperability with numerous systems enables the integration of both legacy and future UGS System (UGSS) components, while the design's open architecture supports rapid third-party development to ensure operational readiness. The successful accomplishment of these objectives by the program's Phase 3b contractors is demonstrated via integration of the companies' respective plug-'n'-play contributions that include controllers, various peripherals, such as sensors, cameras, etc., and their associated software drivers. In order to independently validate the Terra Harvest architecture, L-3 Nova Engineering, along with its partner, the University of Dayton Research Institute, is developing the Terra Harvest Open Source Environment (THOSE), a Java Virtual Machine (JVM) running on an embedded Linux Operating System. The Use Cases on which the software is developed support the full range of UGS operational scenarios such as remote sensor triggering, image capture, and data exfiltration. The Team is additionally developing an ARM microprocessor-based evaluation platform that is both energy-efficient and operationally flexible. The paper describes the overall THOSE architecture, as well as the design decisions for some of the key software components. Development process for THOSE is discussed as well.

Terra Harvest is a DIA sponsored program that is developing a next generation architecture for Unattended Ground Sensor (UGS) interoperability. One key element of this program is a mission programming methodology that allows standardization of the work flow required to program an UGS system. This technology is a key enabler needed to standardize the user experience. This paper describes the requirements for the Terra Harvest mission system and summarizes the different approaches that were considered for the mission system. This paper also describes the implementation of the JavaScript-based approach for mission programming that was selected for use in the Terra Harvest system.

The DIA wants to create an UGS controller that is interoperable across all controller platforms, capable of easily adding new sensors, radios, and processes, etc., as well as backward compatibility with existing UGS systems. To achieve this, the defined Terra Harvest standard is based on the latest Java JRE 1.6 and an OSGI platform. OSGI is an extensible framework that provides a modularized environment that allows functionality to be deployed in "bundles". These bundles can publish, discover, and share services available from other bundles or bundles provided by the controller core. This paper will give you architectural overview and then show you how to use the THOSE SDK to develop and deploy a bundle for your asset, communications device, or algorithm using OSGI and the Terra Harvest Standard.

The Motion Imagery Standards Board (MISB) has previously established a metadata "micro-architecture" for standards-based tracking. The intent of this work is to facilitate both the collaborative development of competent tracking systems, and the potentially distributed and dispersed execution of tracker system components in real-world execution environments. The approach standardizes a set of five quasi-sequential modules in image-based tracking. However, in order to make the plug-and-play architecture truly useful we need metrics associated with each module (so that, for instance, a researcher who "plugs in" a new component can ascertain whether he/she did better or worse with the component). This paper proposes the choice of a new, unifying set of metrics based on an informationtheoretic approach to tracking, which the MISB is nominating as DoD/IC/NATO standards.

The notion of a proling sensor was rst realized by a near-IR, retro-re ective prototype consisting of a vertical column of sparse detectors. Alternative arrangements of detectors have been implemented in which a subset of the detectors have been oset from the vertical column and placed at arbitrary locations along the anticipated path of the objects of interest. All prior work with the near-IR, retro-re ective proling sensors has consisted of wired detectors. This paper advances this prior work by designing and implementing a wireless prototype version of a near-IR, retro-re ective proling sensor in which each detector is a wireless sensor node. In this novel architecture, a base station is responsible for collecting all data from the detector sensor nodes and coordinating all pre- processing of data collected from the sensor nodes, including data re-alignment, before subsequent classication algorithms are executed. Such a wireless detector conguration advances deployment options for near-IR, retro- re ective proling sensors.

During recent military conflicts, as well as for security interventions, the urban zone has taken a preponderant place. Studies have been initiated in national and in international programs to stimulate the technical innovations for these specific scenarios. For example joint field experiments have been organized by the NATO group SET-142 to evaluate the capability for the detection and localization of snipers, mortars or artillery guns using acoustic devices. Another important operational need corresponds to the protection of military sites or buildings. In this context, unattended acoustic and seismic sensors are envisaged to contribute to the survey of specific points by the detection of approaching enemy soldiers. This paper describes some measurements done in an anechoic chamber and in free field to characterize typical sounds generated by the soldier activities (walking, crawling, weapon handling, radio communication, clothing noises...). Footstep, speech and some specific impulsive sounds are detectable at various distances from the source. Such detection algorithms may be easily merged with the existing weapon firing detection algorithms to provide a more generic "battlefield acoustic" early warning system. Results obtained in various conditions (grassy terrain, gravel path, road, forest) will be presented. A method to extrapolate the distances of detection has been developed, based on an acoustic propagation model and applied to the laboratory measurements.

Information derived from sensor networks plays a crucial role in the success of many critical tasks such as surveillance, and border monitoring. In order to derive the correct information at the right time, sensor data must be captured at desired locations with respect to the operational tasks in concern. Therefore, it is important that at the planning stage of a mission, sensing resources are best placed in the field to capture the required data. For example, consider a mission goal identify snipers, in an operational area before troops are deployed - two acoustic arrays and a day-night video camera are needed to successfully achieve this goal. This is because, if the resources are placed in correct locations, two acoustic arrays could provide direction of the shooter and a possible location by triangulating acoustic data whereas the day-night camera could produce an affirmative image of the perpetrators. In order to deploy the sensing resources intelligently to support the user decisions, in this paper we propose a Semantic Web based knowledge layer to identify the required resources in a sensor network and deploy the needed resources through a sensor infrastructure. The knowledge layer captures crucial information such as resources configurations, their intended use (e.g., two acoustic arrays deployed in a particular formation with day-night camera are needed to identify perpetrators in a possible sniper attack). The underlying sensor infrastructure will assists the process by exposing the information about deployed resources, resources in theatre, and location information about tasks, resources and so on.

Moving vehicle detection and classification using multimodal data is a challenging task in data collection, audio-visual alignment, data labeling and feature selection under uncontrolled environments with occlusions, motion blurs, varying image resolutions and perspective distortions. In this work, we propose an effective multimodal temporal panorama approach for the task using a novel long-range audio-visual sensing system. A new audio-visual vehicle (AVV) dataset for moving vehicle detection and classification is created, which features automatic vehicle detection and audio-visual alignment, accurate vehicle extraction and reconstruction, and efficient data labeling. In particular, vehicles' visual images are reconstructed once detected in order to remove most of the occlusions, motion blurs, and variations of perspective views. Multimodal audio-visual features are extracted, including global geometric features (aspect ratios, profiles), local structure features (HOGs), as well various audio features (MFCCs, etc). Using radial-based SVMs, the effectiveness of the integration of these multimodal features is thoroughly and systemically studied. The concept of MTP may not be only limited to visual, motion and audio modalities; it could also be applicable to other sensing modalities that can obtain data in the temporal domain.

In this paper, we consider the agreement problem of estimating the locations of landmarks observed by multiple agents in the absence of a global frame of reference. Maximum likelihood estimators are derived from scenarios of independent and mutual landmark estimation. A triangle-based hypothesis test is proposed for the detection of common landmarks across the coordinate systems of the agents and is followed by a formulation of triangle matching as a linear assignment problem. A landmark agreement model is also proposed with consideration that the agents may observe both common and uncommon landmarks. Simulation examples are provided illustrating the proposed landmark estimation and triangle matching approaches.

In modern coalition operations, decision makers must be capable of obtaining and fusing data from diverse sources. The reliability of these sources can vary, and, in order to protect their interests, the data they provide can be obfuscated. The trustworthiness of fused data depends on both the reliability of these sources and their obfuscation strategy. Information consumers must determine how to evaluate trust in the presence of obfuscation, while information providers must determine the appropriate level of obfuscation in order to ensure both that they remain trusted, and do not reveal any private information. In this paper, through a coalition scenario, we discuss and formalise trust and obfuscation in these contexts and the complex relationships between them.

A canonical problem for autonomy is search and discovery. Often, searching needs to be unpredictable in order to be effective. In this paper, we investigate and compare the effectiveness of the traditional and predictable lawnmower search strategy to that of a random search. Specifically the family of searches with paths determined by heavy tailed distributions called Lévy stable searches is investigated. These searches are characterized by long flight paths, followed by a new random direction, with the flight path lengths determined by the distribution parameter α. Two basic search scenarios are considered in this study: stationary targets, and moving targets, both on planar surfaces. Monte-Carlo simulations demonstrate the advantages of Lévy over the lawnmower strategy especially for moving targets. Ultimately to corroborate the suitability of the Lévy strategy for UAVs, we implement and demonstrate the feasibility of the algorithm in the Multiple Unified Simulation Environment (MUSE), which includes vehicle's constraints and dynamics. The MUSE / Air Force Synthetic Environment for Reconnaissance and Surveillance (AFSERS) simulation system is the primary virtual ISR and UAV simulation within DOD for command and staff level training for the Joint Services.

Acoustic sensors are being employed on airborne platforms, such as Persistent Threat Detection System (PTDS) and Persistent Ground Surveillance System (PGSS), for source localization. Under certain atmospheric conditions, airborne sensors oer a distinct advantage over ground sensors. The performance of both ground and airborne sensors is aected by environmental factors, such as atmospheric turbulence and wind and temperature proles. For airborne sensors, the eects of refraction must be accounted for in order to determine the source coordinates. Such a method for ground-to-air applications has been developed and is further rened here. Ideally, knowledge of the exact atmospheric proles will allow for the most accurate mitigation of refractive eects. However, acoustic sensors deployed in theater are rarely supported by atmospheric sensing systems that retrieve real-time temperature and wind elds. Atmospheric conditions evolve through seasons, time of day, and are strongly location dependent. Therefore, the development of an atmospheric proles database based on a long time series climatological assessment will provide knowledge for use in physics-based bearing estimation algorithms, where otherwise no correction would have been performed. Long term atmospheric data sets from weather modeling systems are used for a climatological assessment of the refraction corrections and localization errors over selected sites.

Video surveillance applications are examples of complex distributed coalition tasks. Real-time capture and analysis of image sensor data is one of the most important tasks in a number of military critical decision making scenarios. In complex battlefield situations, there is a need to coordinate the operation of distributed image sensors and the analysis of their data as transmitted over a heterogeneous wireless network where bandwidth, power, and computational capabilities are constrained. There is also a need to automate decision making based on the results of the analysis of video data. Declarative Networking is a promising technology for controlling complex video surveillance applications in this sort of environment. This paper presents a flexible and extensible architecture for deploying distributed video surveillance applications using the declarative networking paradigm, which allows us to dynamically connect and manage distributed image sensors and deploy various modules for the analysis of video data to satisfy a variety of video surveillance requirements. With declarative computing, it becomes possible for us not only to express the program control structure in a declarative fashion, but also to simplify the management of distributed video surveillance applications.

There has been significant progress recognizing the value of Intelligence, Surveillance, and Reconnaissance (ISR) activities supporting Situational Awareness and Command and Control functions during the past several decades. We consider ISR operations to be proactive (discovering activities or areas of interest), active (activities performed for a particular task that flows down from a hierarchical process) or reactive (critical information gathering due to unexpected events). ISR synchronization includes the analysis and prioritization of information requirements, identification of intelligence gaps and the recommendation of available resources to gather information of interest, for all types of ISR operations. It has become critically important to perform synchronized ISR activities to maximize the efficient utilization of limited resources (both in quantity and capabilities) and, simultaneously, to increase the accuracy and timeliness of the information gain. A study evaluating the existing technologies and processes supporting ISR activities is performed suggesting a rigorous system optimization approach to the ISR synchronization process. Unfortunately, this approach is not used today. The study identifies existing gaps between the current ISR synchronization process and the proposed system optimization approach in the areas of communication and collaboration tools and advanced decision aids (analytics). Solutions are recommended that will help close this gap.

Providing high quality, low latency video from unmanned vehicles through bandwidth-limited communications channels remains a formidable challenge for modern vision system designers. SRI has developed a number of enabling technologies to address this, including the use of SWaP-optimized Systems-on-a-Chip which provide Multispectral Fusion and Contrast Enhancement as well as H.264 video compression. Further, the use of salience-based image prefiltering prior to image compression greatly reduces output video bandwidth by selectively blurring non-important scene regions. Combined with our customization of the VLC open source video viewer for low latency video decoding, SRI developed a prototype high performance, high quality vision system for UxV application in support of very demanding system latency requirements and user CONOPS.

The purpose of the NATO SET-153 field experiment was to provide an opportunity to demonstrate multiple sensor technologies in an urban environment and determine integration capabilities for future development. The Army Research Laboratory (ARL) experimental aerostat was used primarily as a persistent over watch capability as a substitute for a UAV. Continuous video was recorded on the aerostat and segments of video were captured of the scenarios on the ground that the camera was following manually. Some of the segments showing scenario activities will be presented. The captured pictures and video frames have telemetry in the headers that provides the UTM time and the Inertial Navigation System (INS) GPS location and the inertial roll, pitch, and yaw as well as the camera gimbal pan and tilt angles. The timing is useful to synchronize the images with the scenario events providing activity ground truth. The INS, GPS, and camera gimbal angle values can be used with the acoustic solution for the location of a sound source to determine the relative accuracy of the solution if the camera is pointed at the sound source. This method will be confirmed by the use of a propane cannon whose GPS location is logged. During the field experiment, other interesting acoustic events such as vehicle convoys, platoon level firefights with vehicles using blanks, and a UAV helicopter were recorded and will be presented in a quick analysis.

In this paper, we consider the problem of cooperative multitarget tracking and surveillance with multiple UAVs in a decentralized network architecture. In sensor fusion, we developed a decentralized particle ltering approach to maintain target location estimates. Sample sets are converted to Gaussian Mixtures using expectation maximiza- tion, and communicated between agents using a request/response strategy. Upon receiving Gaussian Mixtures from other agents, each UAV fuses them into its own estimates using generalized covariance intersection. In decision making, we decoupled the problem into two abstraction levels. At the higher level, leaders are selected for each subnetwork. Each leader, then, assigns operating regions to other agents within its subnetwork. At the lower level, individual agents plan their own motions to actively track targets within their assigned regions. Our approach is evaluated in a number of simulation runs. The results demonstrate the eectiveness of our decentralized approach.

In June 2011, the United States Army Research Laboratory (ARL) participated in Empire Challenge 2011 (EC-11). EC-11 was United States Joint Forces Command's (USJFCOM) annual live, joint and coalition intelligence, surveillance and reconnaissance (ISR) interoperability demonstration under the sponsorship of the Under Secretary of Defense for Intelligence (USD/I). EC-11 consisted of a series of ISR interoperability events, using a combination of modeling & simulation, laboratory and live-fly events. Wide-area Littoral Discreet Observation (WALDO) was ARL's maritime/littoral capability. WALDO met a USD(I) directive that EC-11 have a maritime component and WALDO was the primary player in the maritime scenario conducted at Camp Lejeune, North Carolina. The WALDO effort demonstrated the utility of a networked layered sensor array deployed in a maritime littoral environment, focusing on maritime surveillance targeting counter-drug, counter-piracy and suspect activity in a littoral or riverine environment. In addition to an embedded analytical capability, the sensor array and control infrastructure consisted of the Oriole acoustic sensor, iScout unattended ground sensor (UGS), OmniSense UGS, the Compact Radar and the Universal Distributed Management System (UDMS), which included the Proxy Skyraider, an optionally manned aircraft mounting both wide and narrow FOV EO/IR imaging sensors. The capability seeded a littoral area with riverine and unattended sensors in order to demonstrate the utility of a Wide Area Sensor (WAS) capability in a littoral environment focused on maritime surveillance activities. The sensors provided a cue for WAS placement/orbit. A narrow field of view sensor would be used to focus on more discreet activities within the WAS footprint. Additionally, the capability experimented with novel WAS orbits to determine if there are more optimal orbits for WAS collection in a littoral environment. The demonstration objectives for WALDO at EC-11 were: * Demonstrate a networked, layered, multi-modal sensor array deployed in a maritime littoral environment, focusing on maritime surveillance targeting counter-drug, counter-piracy and suspect activity * Assess the utility of a Wide Area Surveillance (WAS) sensor in a littoral environment focused on maritime surveillance activities * Demonstrate the effectiveness of using UGS sensors to cue WAS sensor tasking * Employ a narrow field of view full motion video (FMV) sensor package that is collocated with the WAS to conduct more discrete observation of potential items of interest when queued by near-real-time data from UGS or observers * Couple the ARL Oriole sensor with other modality UGS networks in a ground layer ISR capability, and incorporate data collected from aerial sensors with a GEOINT base layer to form a fused product * Swarm multiple aerial or naval platforms to prosecute single or multiple targets * Track fast moving surface vessels in littoral areas * Disseminate time sensitive, high value data to the users at the tactical edge In short we sought to answer the following question: how do you layer, control and display disparate sensors and sensor modalities in such a way as to facilitate appropriate sensor cross-cue, data integration, and analyst control to effectively monitor activity in a littoral (or novel) environment?

A team consisting of Teledyne Scientific Company, the University of California at Santa Barbara, the Army Research Laboratory, the Engineer Research and Development Center, and IBM UK is developing technologies in support of automated data exfiltration from heterogeneous battlefield sensor networks to enhance situational awareness for dismounts and command echelons. Unmanned aerial vehicles (UAV) provide an effective means to autonomously collect data from a sparse network of unattended ground sensors (UGSs) that cannot communicate with each other. UAVs are used to reduce the system reaction time by generating autonomous collection routes that are data-driven. Bioinspired techniques for autonomous search provide a novel strategy to detect, capture and fuse data from heterogeneous sensor networks. The bio-inspired algorithm is based on chemotaxis or the motion of bacteria seeking nutrients in their environment. Field tests of a bio-inspired system that routed UAVs were conducted in June 2011 at Camp Roberts, CA. The field test results showed that such a system can autonomously detect and locate the source of terrestrial events with very high accuracy and visually verify the event. In June 2011, field tests of the system were completed and include the use of multiple autonomously controlled UAVs, detection and disambiguation of multiple acoustic events occurring in short time frames, optimal sensor placement based on local phenomenology and the use of the International Technology Alliance (ITA) Sensor Network Fabric. The system demonstrated TRL 6 performance in the field at Camp Roberts.

Irregular asymmetric conflicts with non-traditional targets and shorter timelines are spawning new mission requirements for situation awareness. As a result, the increasing demands for timely decision-quality information across the battle space have presented tough challenges for stretched operational resources. The Layered Sensing construct developed by the Air Force Research Laboratory provides a unified vision for research to address the situational awareness challenges posed by the complex and dynamic environments seen in recent conflicts. Determining how to maximize available resources across the battle space with the best efficiency and effectiveness is a critical research thread in the Layered Sensing construct. Leveraging different capabilities across networked cooperative sensors to provide timely situational awareness is critical in accomplishing mission objectives with limited resources. This paper evaluates cooperative sensing using a constructive simulation environment by examining mission performance in various scenarios where Remotely Piloted Aircraft are seeking elusive mobile targets. Measured performance parameters include enemy targets killed and suppression of enemy missile attacks. Cooperative and non-cooperative conditions were simulated using variable factors: sensor capabilities, target densities and false target densities. Results demonstrate statistically significant mission performance improvements of over 200% for less capable, but cooperatively networked, sensors increasing their effectiveness to that of more capable sensors working alone. Key factors regarding when and how to leverage cooperation for improved mission performance are also identified and examined through sensitivity analysis.

Auction based methods are often used to perform distributed task allocation on multi-agent teams. Many existing approaches to auctions assume fully cooperative team members. On in-situ and dynamically formed teams, reciprocal collaboration may not always be a valid assumption. This paper presents an approach for dynamically selecting auction partners based on observed team member performance and shared reputation. In addition, we present the use of a shared reputation authority mechanism. Finally, experiments are performed in simulation on multiple UAV platforms to highlight situations in which it is better to enforce cooperation in auctions using this approach.

In order to provide actionable intelligence in a layered sensing paradigm, exploitation algorithms should produce a confidence estimate in addition to the inference variable. This article presents a methodology and results of one such algorithm for feature-aided tracking of vehicles in wide area motion imagery. To perform experiments a synthetic environment was developed, which provided explicit knowledge of ground truth, tracker prediction accuracy, and control of operating conditions. This synthetic environment leveraged physics-based modeling simulations to re-create both traffic flow, reflectance of vehicles, obscuration and shadowing. With the ability to control operating conditions as well as the availability of ground truth, several experiments were conducted to test both the tracker and expected performance. The results show that the performance model produces a meaningful estimate of the tracker performance over the subset of operating conditions.

We consider the problem of distributed sensing and detection using a network of sensor nodes, and the challenges that arise in fusing disparate data. Multiple sensors make local inferences on the state of nature (e.g., the presence of a signal), and those observations are then transmitted to a regional fusion center. The fusion center is tasked to make improved decisions. We develop methods to optimize those decisions. Interoperability between disparate sensor nodes can be addressed by combining similar types of parameters (e.g., direction of arrival and location estimates to better infer location), albeit with varying qualities. As an initial problem, we consider the case where each sensor makes a binary decision on the presence of a signal source and the fusion node combines these to make a more accurate decision. We consider a lossy medium in which signals undergo a range-dependent propagation loss. We determine local thresholds that optimize a performance metric, including both constrained global detection performance and asymptotic error performance. We study the eect sensor node density on detection performance under a network load constraint. The asymptotic performance metrics provide indicators of the amount of value that each sensor contributes to the fusion task.

Social network analysis is a powerful tool used to help analysts discover relationships amongst groups of people as well as individuals. It is the mathematics behind such social networks as Facebook and MySpace. These networks alone cause a huge amount of data to be generated and the issue is only compounded once one adds in other electronic media such as e-mails and twitter. In this paper we outline the basics of social network analysis and how it may be used in current and future Air Force applications.

Persistent surveillance provides decision makers with unprecedented access to multisource data collected from humans and sensor assets around the globe, yet these data exist in the physical world and provide few overt clues to meaning behind actions. In this paper we explore the recent growth in online social networking and ask the questions: 1) can these sites provide value-added information to compliment physical sensing and 2) what are the mechanisms by which these data could inform situational awareness and decision making? In seeking these answers we consider the range of options provided by Social Network Analysis (SNA), and focus especially on the dynamic nature of these networks. In our discussion we focus on the wave of reform experienced by the North African nations in early 2011 known as the Arab Spring. Demonstrators made widespread use of social networking applications to coordinate, document, and publish material to aid their cause. Unlike members of covert social networks who hide their activity and associations, these demonstrators openly posted multimedia information to coordinate activity and stimulate global support. In this paper we provide a review of SNA approaches and consider how one might track network adaptations by capturing temporal and conceptual trends. We identify opportunities and challenges for merging SNA with physical sensor output, and conclude by addressing future challenges in the persistent ISR domain with respect to SNA.

Ensuring the proper and effective ways to visualize network data is important for many areas of academia, applied sciences, the military, and the public. Fields such as social network analysis, genetics, biochemistry, intelligence, cybersecurity, neural network modeling, transit systems, communications, etc. often deal with large, complex network datasets that can be difficult to interact with, study, and use. There have been surprisingly few human factors performance studies on the relative effectiveness of different graph drawings or network diagram techniques to convey information to a viewer. This is particularly true for weighted networks which include the strength of connections between nodes, not just information about which nodes are linked to other nodes. We describe a human factors study in which participants performed four separate network analysis tasks (finding a direct link between given nodes, finding an interconnected node between given nodes, estimating link strengths, and estimating the most densely interconnected nodes) on two different network visualizations: an adjacency matrix with a heat-map versus a node-link diagram. The results should help shed light on effective methods of visualizing network data for some representative analysis tasks, with the ultimate goal of improving usability and performance for viewers of network data displays.

Identifying social network (SN) links within computer-mediated communication platforms without explicit relations among users poses challenges to researchers. Our research aims to extract SN links in internet chat with multiple users engaging in synchronous overlapping conversations all displayed in a single stream. We approached this problem using three methods which build on previous research. Response-time analysis builds on temporal proximity of chat messages; word context usage builds on keywords analysis and direct addressing which infers links by identifying the intended message recipient from the screen name (nickname) referenced in the message [1]. Our analysis of word usage within the chat stream also provides contexts for the extracted SN links. To test the capability of our methods, we used publicly available data from Internet Relay Chat (IRC), a real-time computer-mediated communication (CMC) tool used by millions of people around the world. The extraction performances of individual methods and their hybrids were assessed relative to a ground truth (determined a priori via manual scoring).

Information that propagates through social networks can carry a lot of false claims. For example, rumors on certain topics can propagate rapidly leading to a large number of nodes reporting the same (incorrect) observations. In this paper, we describe an approach for nding the rumor source and assessing the likelihood that a piece of information is in fact a rumor, in the absence of data provenance information. We model the social network as a directed graph, where vertices represent individuals and directed edges represent information ow (e.g., who follows whom on Twitter). A number of monitor nodes are injected into the network whose job is to report data they receive. Our algorithm identies rumors and their sources by observing which of the monitors received the given piece of information and which did not. We show that, with a sucient number of monitor nodes, it is possible to recognize most rumors and their sources with high accuracy.

In recent years, the United States Armed Services and various law enforcement agencies have shown increasing interest in evaluating the feasibility of using smartphones and hand-held devices as part of the standard gear for its personnel, who are actively engaged on battlefield or in crime-prone areas. The primary motive driving analysis efforts to employ smartphone-based technologies is the prospect of the increased "Situational Awareness" achievable thru a digitally connected network of armed personnel. Personnel would be equipped with customized smart applications that use the device's sensors (GPS, camera, compass, etc...) to sense the hostile environments as well as enabling them to perform collaborative tasks to effectively complete a given mission. In this vein, as part of the Summer At The Edge (SATE) program, a group of student interns under the guidance of mentors from Qbase and AFRL, have employed smartphones and built three smart applications to tackle three real-world scenarios: PinPoint, IStream, and Cooperative GPS. This paper provides implementation details for these prototype applications, along with the supporting visualization and sensor cloud platforms and discusses results obtained from field testing of the same. Further, the paper concludes by providing the implications of the present work and insights into future work.

In recent years the ubiquity and resources provided by smartphone devices have encouraged scientists to explore using these devices as remote sensing nodes. In addition, the United States Department of Defense has stated a mission of increasing persistent intelligence, surveillance, and reconnaissance capabilities or U.S. units. This paper presents a method of enabling large-scale, long-term smartphone-powered data collection. Key solutions discussed include the ability to directly allow domain experts to define and refine smartphone applications for data collection, technical advancements that allow rapid dissemination of a smartphone data collection application, and an algorithm for preserving the locational privacy of participating users.