The growing volume and quality of Earth observations offer improvements in data assimilation and weather forecasting that can only be fully realized by advances in AI. 3D atmospheric profiles, including temperature, humidity, and winds, are key missing observables in NASA's Earth Observation System (EOS) and presents large initial condition uncertainties in numerical weather prediction (NWP). Observations from radiosondes and microwave/infrared sounders as well as derived products like atmospheric motion vectors (AMVs) provide key inputs to data assimilation (DA). However, these observations are sparse and the spatio-temporal resolution of modern global DA systems have plateaued largely due to computational requirements. This stands in contrast with advances in artificial intelligence (AI) and computer vision (CV) that enable scalable and data efficient processing with the ability to synthetically generate variables not directly observed. Our work aims to fill EOS gaps and provide an alternative to traditional DA utilizing high-temporal resolution GOES-16/17 geostationary (GEO) satellites operated by NOAA/NASA and radiosonde observations with a probabilistic generative modeling approach. Variational autoencoders (VAEs) are used to independently compress GEO infrared bands and radiosonde profiles into latent representations. This enables us to learn a low-dimensional function between the latent representations to reconstruct temperature and humidity profiles on a pixel-wise basis. Our WindFlow model is applied to track the movement of humidity across sequences of frames to produce wind speed and direction. Lastly, a Neural Ordinary Difference Equation model is used to post-process the derived as a novel approach data-driven DA. The output of this proposal will include Zeus-Analysis, a novel 3D atmospheric dataset, with comprehensive evaluation and user access development through an application programming interface.
Applications of 3D atmospheric profiles are numerous throughout NASA Earth science research and development. Assimilation of our data with systems operated at the NASA's Global Modeling and Assimilation Office (GMAO) has the potential to improve analysis and forecast products, including short-term and sub-seasonal. Dense atmospheric winds will also have implications to wildfire monitoring and subsequent air quality issues.
Commercially the developed technology has applications to renewable energy, aviation, and finance/insurance. Renewable energy markets are largely powered by weather conditions and must be accurately estimated for stable operation of the power grid. Forecasts help prevent flight diversions and can cause dozens of downstream flight delays.