A Dynamic Parallel Data-Computing Environment for Cross-Sensor Satellite Data Merger and Scientific Analysis

Abstract

The Data Processing and Error Analysis System (DPEAS) is a dynamic parallel data processing system for cross-sensor satellite data merger and analysis. Using a peer-to-peer methodology, DPEAS is able to distribute and simplify the near-real-time merger of multiple datasets from several sensors and satellite platforms. The organization of the system and its practical benefits for research and operational use are discussed. Using several simplifying constructs, the technical requirements placed upon the end user are minimized, while maintaining high scalability and reliability of the system. Thus, the system encourages the flow of research into operations by making the system environment mobile, rather than placing the transferability burden on the research development effort. Several reliability enhancements, such as cluster failover and automatic file replication services, are incorporated into the system. The technical aspects of the various feature implementations are discussed in detail. Future adaptation of this processing approach to new Internet standards is also discussed.