Sensor Fusion Applications

Sensor Fusion is an umbrella term for applications that collect data from multiple sensors (cameras, analog to digital converters etc.) correlate and process it and then use the results to make decisions. In many cases this processing and decision making must be performed in real-time and could result in loss of life and or property damage if the correct decision is not made in time. Examples of sensor fusion include autonomous vehicles, big science, radar processing, satellite imaging, and 5G beamforming.

The multiple data sources are often heterogeneous in nature, requiring translation to a common format to achieve correlation and a composite result. This capability is critical to the success of terrestrial, airborne, and spaceborne autonomous platforms. Combining incoming information taken from multiple angles with some level of overlap can also provide a more complete picture to achieve situational awareness. But perfect time and special correlation is required to for this to be effective.

In general, the challenges involved in implementing sensor fusion are:

  • Scalability: Large numbers of sensors delivering high-bandwidth data.
  • Latency: Critical when actionable results are needed in a real time environment.
  • Determinism: Variability in latency is problematic where data streams need to be correlated.
  • Availability: Failure of these systems can often result loss of life or property.
  • Power: Particularly important in battery powered systems.

Currently RapidIO technology provides the optimum solutions for sensor fusion challenges. In contrast with competing protocols, RapidIO offers peer to peer connectivity, low deterministic latency and high availability features such as High Availability/Redundant System Hardware (HARSH) profiles. RapidIO also provides a rich range of packet semantics including non-coherent shared memory, cache coherent shared memory, streaming data, messaging and time synchronization.

Combining high performance processing using a combination of general-purpose processing, digital signal processing, and hard-wired algorithms in ASICs or FPGAs, the high rate, low latency incoming information provided via the RapidIO network can be translated into useful data that can be applied in real time to achieve autonomy. Sensor fusion reference platforms based on the use of the RapidIO protocol and high-performance computing engines are in development and an area of focus for RapidIO.