Machine Learning for Cloud Remote Sensing

Developing Passive Satellite Cloud Remote Sensing Algorithms Using Collocated Observations, Numerical Simulation and Deep Learning, 2020-2024, Funding Agency: NASA

Introduction

Clouds cover about two thirds of Earth’s surface and play a critical role in our climate system, with fundamental influence on the energy, water, and biological cycles. Currently, satellite-based remote sensing is the only way to observe clouds on a global scale. For these reasons, cloud observations have always been a major task of NASA’s Earth Science endeavor.

In the latest NASA Decadal Survey, cloud observations have been given top priority for NASA’s missions. Numerous satellite sensors have been developed to observe and retrieve cloud properties. They can be largely divided into two groups: active sensors such as those used by the Cloud Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) and CloudSat missions, and passive sensors such as the Moderate Resolution Imaging Spectroradiometer (MODIS), the Visible Infrared Imaging Radiometer Suite (VIIRS), and Advanced Baseline Imager (ABI).

The advantages of active sensors include their capability of resolving the vertical location of cloud layer and better performance during nighttime and over polar regions. On the other hand, passive sensors have a much better spatial sampling rate.

Machine learning (ML)-based algorithms have brought revolutionary changes to almost every aspect of our lives. ML is also increasingly used in NASA’s satellite remote sensing algorithms. For most machine learning, especially deep learning (DL)-based algorithms, high-quality training datasets are critical.

The overarching goal of our proposed project is to develop an extensible platform that combines collocated satellite observations, numerical simulations, and deep learning methods to generate a highly accurate cloud property training dataset for NASA, NOAA, and the broad science community to develop and benchmark algorithms for passive satellite cloud remote sensing.

This project will deliver:

  • Novel deep-learning-based domain-adaptation algorithms to retrieve passive satellite remote sensing cloud bulk properties (e.g., cloud mask and thermodynamic phase) by leveraging one or more available active sensing data.
  • A novel hybrid approach combining advanced 3-D radiative transfer simulations based on collocated global satellite observations and deep learning based multi-pixel cloud microphysical and optical property retrieval.
  • Scalable data processing and analytics services in a public cloud computing environment (i.e., Amazon Web Service) for the above components/capabilities.
  • Comprehensive data quality evaluation of the training datasets (where retrieved cloud properties are data labels) to be delivered from multiple aspects including statistics, climatology, ground observation, and ad hoc case studies.

In particular, we will generate four-year (2017–2020) labeled cloud property training data from the joint NASA/NOAA Suomi National Polar-orbiting Partnership (Suomi NPP) satellite and the geostationary GOES-16 satellite.

The overview of the proposed platform is shown in the figure below where T1-T5 are major tasks and D1-D3 are data deliverables.The outputs of this project will greatly help NASA scientists and the broader community to independently or collectively develop machine-learning based cloud remote sensing algorithms, compare and evaluate the cloud retrieval products.

Team

  • Principal Investigator: Jianwu Wang, Department of Information Systems, University of Maryland, Baltimore County
  • Co-Investigator: Sanjay Purushotham, Department of Information Systems, University of Maryland, Baltimore County
  • Co-Investigator: Zhibo Zhang, Department of Physics, University of Maryland, Baltimore County
  • Co-Investigator: Chenxi Wang, Joint Center for Earth Systems Technology (JCET) at UMBC and NASA Goddard Space Flight Center (GSFC)
  • Co-Investigator: Kerry Meyer, Climate and Radiation Branch of the Laboratory for Atmospheres, NASA Goddard Space Flight Center (GSFC)
  • Researcher: Benjamin Marchant, Climate and Radiation Branch of the Laboratory for Atmospheres, NASA Goddard Space Flight Center (GSFC)
  • PhD students: Xingyan Li, Xin Huang, Zahid Hassan Tushar, Xiangyang Meng, Xin Wang, Seraj Mostafa, Department of Information Systems, University of Maryland, Baltimore County
  • PhD students: Adeleke Segun Ademakinwa, Physics Department, University of Maryland, Baltimore County

GitHub Repositories for our Open Source Codes

Our repositories are located at https://github.com/AI-4-atmosphere-remote-sensing. One repository is now publicly available (see below). We will make others public soon.

  1. Satellite Collocation: This repository provides a toolkit that collocates data from two or more satellites.

Repositories for Public Available Data Deliverables

Please check below for the current data deliverables. We are working with potential data repositories to make more data deliverables available. We will update the information once the data are available for public.

  1. Post-processed LES fields utilized as inputs for 3D radiative transfer (cloud microphysics input, Radiance and Broadband flux radiative transfer simulation results and cloud property retrieval ) for the 27 June 2015 and 18 August 2016 LES cases. [DOI:10.5281/zenodo.10511732].

Publications

  1. Seraj Al Mahmud Mostafa, Jinbo Wang, Benjamin Holt, Jianwu Wang. YOLO based Ocean Eddy Localization with AWS SageMaker. In Proceedings of the Seventh IEEE International Workshop on Benchmarking, Performance Tuning and Optimization for Big Data Applications (BPOD 2024) at 2024 IEEE International Conference on Big Data (IEEE Big Data 2024). [arXiv Pre-Print], [10.1109/BigData62323.2024.10825286], [Open Source Code]
  2. Zahid Hassan Tushar, Adeleke S. Ademakinwa, Jianwu Wang, Zhibo Zhang, Sanjay Purushotham. CloudUNet: Adapting UNet for Retrieving Cloud Properties. Accepted by the 2024 IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2024), IEEE, 2024.
  3. Adeleke S. Ademakinwa, Zahid H. Tushar, Jianyu Zheng, Chenxi Wang, Sanjay Purushotham, Jianwu Wang, Kerry G. Meyer, Tamas Várnai, and Zhibo Zhang. Influence of Cloud Retrieval Errors Due to Three Dimensional Radiative Effects on Calculations of Broadband Cloud Radiative Effect, Atmospheric Chemistry and Physics, 24, pages 3093–3114, DOI:10.5194/acp-24-3093-2024, 2024.
  4. Seraj Al Mahmud Mostafa, Jinbo Wang, Benjamin Holt, Sanjay Purushotham, Jianwu Wang. CNN based Ocean Eddy Detection using Cloud Services. Accepted in IEEE IGARSS 2023. [Open Access Paper, Open Source Code].
  5. Xin Wang, Pei Guo, Xingyan Li, Aryya Gangopadhyay, Carl E. Busart, Jade Freeman and Jianwu Wang. Reproducible and Portable Big Data Analytics in the Cloud. DOI:10.1109/TCC.2023.3245081, Accepted by the IEEE Transactions on Cloud Computing, 2023. [Open Access Paper, Open Source Code].
  6. Xin Huang, Chenxi Wang, Sanjay Purushotham, Jianwu Wang. VDAM: VAE based Domain Adaptation for Cloud Property Retrieval from Multi-satellite Data. In Proceedings of The thirteenth International Conference on Advances in Geographic Information Systems 2022 (ACM SIGSPATIAL 2022). Article No.: 107, pages 1–10, DOI:10.1145/3557915.3561044 [Paper Pre-Print, Open Source Code]
  7. Ziheng Sun, Laura Sandoval, Robert Crystal-Ornelas, S. Mostafa Mousavi, Jinbo Wang, Cindy Lin, Nicoleta Cristea, Daniel Tong, Wendy Hawley Carande, Xiaogang Ma, Yuhan Rao, James A. Bednar, Amanda Tan, Jianwu Wang, Sanjay Purushotham, Thomas E. Gill, Julien Chastang, Daniel Howard, Benjamin Holt, Chandana Gangodagamage, Peisheng Zhao, Pablo Rivas, Zachary Chester, Javier Orduz, Aji John. A Review of Earth Artificial Intelligence, Computers & Geosciences, volume 159, 105034, DOI:10.1016/j.cageo.2022.105034, Elsevier, 2022. [Open Access Paper].
  8. Xin Huang, Sahara Ali, Chenxi Wang, Zeyu Ning, Sanjay Purushotham, Jianwu Wang, Zhibo Zhang. Deep Domain Adaptation based Cloud Type Detection using Active and Passive Satellite Data. In Proceedings of the 2020 IEEE International Conference on Big Data (BigData 2020), pages 1330-1337, IEEE, 2020

Presentations

  1. Zhibo Zhang et al. Radiative Closure Studies of How Cloud Property Retrieval Errors due to 3-D Effects Influence Our Understanding of Broadband Cloud Radiative Effect, The AeroCenter-Cloud Precipitation Center (AeroCenter-CPC), March, 2023
  2. Zahid Hassan Tushar, et al. Deep Learning Transformers for Retrieval of Cloud Optical Properties. Poster, 2023 ESDSWG Meeting, March, 2023
  3. Xin Huang, et al. VDAM : VAE based Domain Adaptation for Cloud Property Retrieval from Multi-Satellite Data, Poster, 2023 ESDSWG Meeting, March, 2023
  4. Chenxi Wang, et al. A Convenient and Flexible Satellite Data Spatial-temporal Collocation System, Poster, 2023 ESDSWG Meeting, March, 2023
  5. Jianwu Wang, et al. Developing Passive Satellite Cloud Remote Sensing Algorithms Using Collocated Observations, Numerical Simulation and Deep Learning. Talk, 2023 ESDSWG Meeting, March, 2023
  6. Nga Chung, Thomas Huang, Vardis M. Tsontos, Stepheny Perez, Wai Phyo, Joshua Rodriguez, Riley Kuttruff, Shawn R. Smith, Jordan Gethers, Thomas Cram, Zaihua Ji, Kimberly Sparling, Jianwu Wang. Cloud-based Data Match-Up Service (CDMS) and AI/ML, Talk. ESIP Summer 2022 Meeting, 2022
  7. Xin Huang, et al., Multi-Sensor Deep Domain Adaptation for Cloud Bulk Property Retrieval, ESDSWG 2022 Poster Session, April 20, 2022
  8. Xiangyang Meng, Adeleke Ademakinwa, Jianwu Wang, Zhibo Zhang, Sanjay Purushotham, Deep Learning Transformers for Retrieval of Cloud Optical Properties. ESIP. Poster. https://doi.org/10.6084/m9.figshare.19594171.v1 (ESDSWG 2022 Poster Session, April 20, 2022)
  9. Chenxi Wang, et al., A Convenient and Flexible Satellite Data Spatial-temporal Collocation System, ESDSWG 2022 Poster Session, April 20, 2022
  10. Jianwu Wang, et al., Improving FAIRness of AI/ML in Earth science via Reproducible Big Data Analytics in the Cloud, Invited Talk, ESIP Spring 2022 Meeting, Jan. 19, 2022
  11. Jianwu Wang, Big Data Analytics in the Cloud. Information Science and Systems Department, Morgan State University, 2021/11
  12. Jianwu Wang. Developing Passive Satellite Cloud Remote Sensing Algorithms using Collocated Observations, Numerical Simulation and Deep Learning, Invited Talk, NASA GSFC Climate & Radiation Laboratory (613) Seminar, 2021/09
  13. Sanjay Purushotham, et. al., Deep Multi-Sensor Domain Adaptation on Active and Passive Satellite Remote Sensing Data, 2nd NOAA Workshop on Leveraging AI in Environmental Sciences, Session 25, AI/ML for Data Fusion/Assimilation, 2021/01
  14. Jianwu Wang, et. al., Deep Multi-Sensor Domain Adaptation on Active and Passive Satellite Remote Sensing Data, AGU Fall Meeting 2020

Acknowledgement

The work is funded by an NASA ACCESS project: Developing Passive Satellite Cloud Remote Sensing Algorithms Using Collocated Observations, Numerical Simulation and Deep Learning, grant ID: 80NSSC21M0027.