TEMPO UVAI vs DSCOVR (spatial)

Summary

This notebook illustrates a comparison of TEMPO ultra-violet aerosol index (UVAI) against DSCOVR EPIC UVAI. TEMPO_O3TOT_L2_V03 and DSCOVR_EPIC_L2_AER_03 are the data collections used as sources of UVAI.

TEMPO and DSCOVR granules are downloaded on-the-fly with earthaccess library, which may need to be installed first.

Dataset Information

“DSCOVR_EPIC_L2_AER_03 is the Deep Space Climate Observatory (DSCOVR) Enhanced Polychromatic Imaging Camera (EPIC) Level 2 UV Aerosol Version 3 data product. Observations for this data product are at 340 and 388 nm and are used to derive near UV (ultraviolet) aerosol properties. The EPIC aerosol retrieval algorithm (EPICAERUV) uses a set of aerosol models to account for the presence of carbonaceous aerosols from biomass burning and wildfires (BIO), desert dust (DST), and sulfate-based (SLF) aerosols. These aerosol models are identical to those assumed in the OMI (Ozone Monitoring Instrument) algorithm (Torres et al., 2007; Jethva and Torres, 2011).” (Source)

Total ozone Level 2 files provide ozone information at Tropospheric Emissions: Monitoring of Pollution (TEMPO)’s native spatial resolution, ~10 km^2 at the center of the Field of Regard (FOR), for individual granules. Each granule covers the entire North-South TEMPO FOR but only a portion of the East-West FOR.

Table of Contents

1. Setup
2. Define utility functions for DSCOVR and TEMPO data
3. Establish access to Earthdata
4. Select timeframe of interest
5. Retrieving DSCOVR EPIC granules
6. For every DSCOVR EPIC granule, find simultaneous TEMPO granules and re-map DSCOVR EPIC data to geolocations of TEMPO

Notebook’s general code outline:

• Timeframe of interest is selected by a user.
• Searches for DSCOVR EPIC granules withing the TEMPO field of regard (FOR) and within user’s timeframe by means of earthaccess library.
• After downloading DSCOVR EPIC granules, a loop by these granules DSCOVR L2 AER data searches for TEMPO granules simultaneous with DSCOVR EPIC one.
• If such TEMPO granules exist, DSCOVR EPIC UVAI retroevals are interpolated to the positions of the TEMPO pixels. The interpolated values are ritten into a netCDF file along with TEMPO geolocations.
• Finally original UVAI from DSCOVR EPIC and TEMPO are plotted along with interpolated DSCOVR EPIC values in the same plot. Output images are written into PNG files.

1. Setup

import earthaccess # needed to discover and download TEMPO data
import netCDF4 as nc # needed to read TEMPO data

import os
import sys

import platform
from subprocess import Popen
import shutil

from shapely.geometry import Point, Polygon # needed to search a point within a polygon
from scipy.interpolate import griddata # needed to interpolate TEMPO data to the point of interest
from scipy import stats # needed for linear regression analysis

import requests # needed to search for and download Pandora data
import codecs # needed to read Pandora data
import numpy as np

import h5py # needed to read DSCOVR_EPIC_L2_TO3 files
import matplotlib.pyplot as plt # needed to plot the resulting time series
from urllib.request import urlopen, Request # needed to search for and download Pandora data
from pathlib import Path # needed to check whether a needed data file is already downloaded
from datetime import datetime, timedelta # needed to work with time in plotting time series

import cartopy.crs as ccrs
from cartopy.mpl.gridliner import LONGITUDE_FORMATTER, LATITUDE_FORMATTER

2. Define utility functions for DSCOVR and TEMPO data

2.1 Function to read DSCOVR AER data files

function read_epic_l2_AER reads DSCOVR_EPIC_L2_AER product file given by its fname and returns arrays of 2D latitudes, longitudes, UVAI, and AOD along with wavelength along with their fill values and time.

def read_epic_l2_AER(fname):

  aod_name = '/HDFEOS/SWATHS/Aerosol NearUV Swath/Data Fields/FinalAerosolOpticalDepth'
  uvai_name = '/HDFEOS/SWATHS/Aerosol NearUV Swath/Data Fields/UVAerosolIndex'
  lat_name = '/HDFEOS/SWATHS/Aerosol NearUV Swath/Geolocation Fields/Latitude'
  lon_name = '/HDFEOS/SWATHS/Aerosol NearUV Swath/Geolocation Fields/Longitude'
  wl_name = '/HDFEOS/SWATHS/Aerosol NearUV Swath/Data Fields/Wavelength'

  try:
    f = h5py.File(fname, "r" )

    item = f[aod_name]
    aod2D = np.array(item[:])
    fv_aod = item.fillvalue

    item = f[uvai_name]
    uvai2D = np.array(item[:])
    fv_uvai = item.fillvalue

    item = f[lat_name]
    lat2D = np.array(item[:])
    fv_lat = item.fillvalue

    item = f[lon_name]
    lon2D = np.array(item[:])
    fv_lon = item.fillvalue

    item = f[wl_name]
    wl = np.array(item[:])
    fv_wl = item.fillvalue

    f.close()

# getting time from the granule's filename
    fname_split = fname.split('_')
    timestamp = fname_split[-2]
    yyyy= int(timestamp[0 : 4])
    mm = int(timestamp[4 : 6])
    dd = int(timestamp[6 : 8])
    hh = int(timestamp[8 : 10])
    mn = int(timestamp[10 : 12])
    ss = int(timestamp[12 : 14])

  except:
    print("Unable to find or read hdf5 input granule file ", fname)
    aod2D  = 0.
    fv_aod  = 0.
    uvai2D  = 0.
    fv_uvai  = 0.
    lat2D  = 0.
    fv_lat  = 0.
    lon2D  = 0.
    fv_lon  = 0.
    wl  = 0.
    fv_wl  = 0.
    yyyy  = 0.
    mm  = 0.
    dd  = 0.
    hh  = 0.
    mn  = 0.
    ss  = 0.

  return aod2D, fv_aod, uvai2D, fv_uvai, lat2D, fv_lat, lon2D, fv_lon\
, wl, fv_wl, yyyy, mm, dd, hh, mn, ss

2.2 Function to read UV Aerosol Index from TEMPO O3TOT data file

function read_TEMPO_O3TOT_L2_UVAI reads the following arrays from the TEMPO L2 O3TOT product TEMPO_O3TOT_L2_V01(2): vertical_column; vertical_column_uncertainty; and returns respective fields along with coordinates of the pixels.

If one requested variables cannot be read, all returned variables are zeroed

def read_TEMPO_O3TOT_L2_UVAI(fn):

  var_name = 'uv_aerosol_index'
  var_QF_name = 'quality_flag'

  try:
    ds = nc.Dataset(fn)

    prod = ds.groups['product'] # this opens group product, /product, as prod

    var = prod.variables[var_name] # this reads variable column_amount_o3 from prod (group product, /product)
    uvai = np.array(var)
    uvai_fv = var.getncattr('_FillValue')

    var_QF = prod.variables[var_QF_name] # this reads variable column_amount_o3 from prod (group product, /product)
    uvai_QF = np.array(var_QF)
# there is no fill value for the quality flag.
# Once it is available in the next version of the product,
# un-comment the line below and add fv_QF to the return line.
#    fv_QF = var_QF.getncattr('_FillValue')

    geo = ds.groups['geolocation'] # this opens group geolocation, /geolocation, as geo

    lat = np.array(geo.variables['latitude']) # this reads variable latitude from geo (geolocation group, /geolocation) into a numpy array
    lon = np.array(geo.variables['longitude']) # this reads variable longitude from geo (geolocation group, /geolocation) into a numpy array
    fv_geo = geo.variables['latitude'].getncattr('_FillValue')
# it appeared that garbage values of latitudes and longitudes in the L2 files
# are 9.969209968386869E36 while fill value is -1.2676506E30
# (after deeper search it was found that actual value in the file is -1.2676506002282294E30).
# For this reason, fv_geo is set to 9.96921E36 to make the code working.
# Once the problem is resolved and garbage values of latitudes and longitudes
# equal to their fill value, the line below must be removed.
    fv_geo = 9.969209968386869E36

    time = np.array(geo.variables['time'] )# this reads variable longitude from geo (geolocation group, /geolocation) into a numpy array

    ds.close()

  except:
    print('variable '+var_name+' cannot be read in file '+fn)
    lat = 0.
    lon = 0.
    time = 0.
    fv_geo = 0.
    uvai = 0.
    uvai_QF = 0.
    fv_uvai = 0.
#    fv_QF = -999
    prod_unit = ''

  return lat, lon, fv_geo, time, uvai, uvai_QF, uvai_fv

2.3 Function creating TEMPO O3 granule polygon

def TEMPO_L2_polygon(lat, lon, fv_geo):
  nx = lon.shape[0]
  ny = lon.shape[1]
  print('granule has %3d scanlines by %4d pixels' %(nx, ny))

  dpos = np.empty([0,2])

  x_ind = np.empty([nx, ny], dtype = int) # creating array in x indices
  for ix in range(nx): x_ind[ix, :] = ix # populating array in x indices
  y_ind = np.empty([nx, ny], dtype = int)
  for iy in range(ny): y_ind[:, iy] = iy # populating array in x indices

  mask = (lon[ix, iy] != fv_geo)&(lat[ix, iy] != fv_geo)
  if len(lon[mask]) == 0:
    print('the granule is empty - no meaningful positions')
    return dpos

# right boundary
  r_m = min(x_ind[mask].flatten())
  local_mask = (lon[r_m, :] != fv_geo)&(lat[r_m, :] != fv_geo)
  r_b = np.stack((lon[r_m, local_mask], lat[r_m, local_mask])).T

# left boundary
  l_m = max(x_ind[mask].flatten())
  local_mask = (lon[l_m, :] != fv_geo)&(lat[l_m, :] != fv_geo)
  l_b = np.stack((lon[l_m, local_mask], lat[l_m, local_mask])).T

#top and bottom boundaries
  t_b = np.empty([0,2])
  b_b = np.empty([0,2])
  for ix in range(r_m + 1, l_m):
    local_mask = (lon[ix, :] != fv_geo)&(lat[ix, :] != fv_geo)
    local_y_ind = y_ind[ix, local_mask]
    y_ind_top = min(local_y_ind)
    y_ind_bottom = max(local_y_ind)
    t_b = np.append(t_b, [[lon[ix, y_ind_top], lat[ix, y_ind_top]]], axis=0)
    b_b = np.append(b_b, [[lon[ix, y_ind_bottom], lat[ix, y_ind_bottom]]], axis=0)

# combining right, top, left, and bottom boundaries together, going along the combined boundary counterclockwise
  dpos = np.append(dpos, r_b[ : :-1, :], axis=0) # this adds right boundary, counterclockwise
  dpos = np.append(dpos, t_b, axis=0) # this adds top boundary, counterclockwise
  dpos = np.append(dpos, l_b, axis=0) # this adds left boundary, counterclockwise
  dpos = np.append(dpos, b_b[ : :-1, :], axis=0) # this adds bottom boundary, counterclockwise

  print('polygon shape: ',dpos.shape)

  return dpos

2.4 Function writing DSCOVR EPIC UV Aerosol Index re-mapped to TEMPO granule locations

def write_DSCOVR_TEMPO_UVAI(fname, lat2D, lon2D, uvai2D):
#
# variables:
#   fname    - TEMPO file name, will be used to create output file name
#   lat2D    - 2D array of TEMPO latitudes
#   lon2D    - 2D array of TEMPO longitudes
#   uvai2D   - 2D array of DSCOVR EPIC UVAI re-mapped to TEMPO locations
# arrays above shoud be of the same shape

  try:

    (nx, ny) = lat2D.shape
    ncf = nc.Dataset('DSCOVR_UVAI_'+fname, mode='w', format='NETCDF4_CLASSIC')
    x_dim = ncf.createDimension('mirror_step', nx) # number of scanlines
    y_dim = ncf.createDimension('xtrack', ny) # number of pixels in a scanline

    lat = ncf.createVariable('lat', np.float32, ('mirror_step', 'xtrack'))
    lat.units = 'degrees_north'
    lat.long_name = 'latitude'
    lat[:,:] = lat2D

    lon = ncf.createVariable('lon', np.float32, ('mirror_step', 'xtrack'))
    lon.units = 'degrees_east'
    lon.long_name = 'longitude'
    lon[:,:] = lon2D

    uv_aerosol_index = ncf.createVariable('uv_aerosol_index', np.float32, ('mirror_step', 'xtrack'))
    uv_aerosol_index[:,:] = uvai2D

    ncf.close()

    success = True

  except: success = False

  return success

3. Establish access to EarthData

3.1. Log in

User needs to create an account at https://www.earthdata.nasa.gov/ Function earthaccess.login prompts for EarthData login and password.

auth = earthaccess.login(strategy="interactive", persist=True)

3.2. Create local directory

homeDir = os.path.expanduser("~") + os.sep

with open(homeDir + '.dodsrc', 'w') as file:
    file.write('HTTP.COOKIEJAR={}.urs_cookies\n'.format(homeDir))
    file.write('HTTP.NETRC={}.netrc'.format(homeDir))
    file.close()

print('Saved .dodsrc to:', homeDir)

# Set appropriate permissions for Linux/macOS
if platform.system() != "Windows":
    Popen('chmod og-rw ~/.netrc', shell=True)
else:
    # Copy dodsrc to working directory in Windows
    shutil.copy2(homeDir + '.dodsrc', os.getcwd())
    print('Copied .dodsrc to:', os.getcwd())

Saved .dodsrc to: /home/jovyan/

4. Select timeframe of interest

DSCOVR EPIC granules will be searched within this timeframe

print('enter period of interest, start and end dates, in the form YYYYMMDD')
datestamp_ini = input('enter start date of interest ')
datestamp_fin = input('enter end date of interest ')

start_date = int(datestamp_ini)
end_date = int(datestamp_fin)

yyyy_ini = start_date//10000
mm_ini = (start_date//100 - yyyy_ini*100)
dd_ini = (start_date - yyyy_ini*10000 - mm_ini*100)

yyyy_fin = end_date//10000
mm_fin = (end_date//100 - yyyy_fin*100)
dd_fin = (end_date - yyyy_fin*10000 - mm_fin*100)
print(yyyy_ini, mm_ini, dd_ini, yyyy_fin, mm_fin, dd_fin)

date_start = str('%4.4i-%2.2i-%2.2i 00:00:00' %(yyyy_ini, mm_ini, dd_ini))
date_end = str('%4.4i-%2.2i-%2.2i 23:59:59' %(yyyy_fin, mm_fin, dd_fin))

enter period of interest, start and end dates, in the form YYYYMMDD
enter start date of interest  20230805
enter end date of interest  20230805
2023 8 5 2023 8 5

5. Retrieving DSCOVR EPIC granules

in the time of interest falling into TEMPO polygon

short_name = 'DSCOVR_EPIC_L2_AER' # collection name to search for in the EarthData

# polygon below is taken from MMT description of TEMPO_O3TOT_L2,
# see https://mmt.earthdata.nasa.gov/collections/C2842849465-LARC_CLOUD
# Polygon: (10.0°, -170.0°), (10.0°, -10.0°), (80.0°, -10.0°), (80.0°, -170.0°), (10.0°, -170.0°)

bbox = (-170., 10., -10., 80.)

FOR_results_EPIC = earthaccess.search_data(short_name = short_name\
                                         , temporal = (date_start, date_end)\
                                         , bounding_box = bbox)

n_EPIC = len(FOR_results_EPIC)

print('total number of DSCOVR EPIC L2_AER granules found for TEMPO FOR'\
    , '\nwithin period of interes between', date_start, 'and', date_end, 'is', n_EPIC)

Granules found: 21
total number of DSCOVR EPIC L2_AER granules found for TEMPO FOR 
within period of interes between 2023-08-05 00:00:00 and 2023-08-05 23:59:59 is 21

5.1. ensuring all discovered granules have download links

without this step, those granules crash the call of earthaccess.download()

granule_links_EPIC = []
FOR_results_EPIC_bad = []
for result in FOR_results_EPIC:
  try:
    granule_links_EPIC.append(result['umm']['RelatedUrls'][0]['URL'])
  except:
    FOR_results_EPIC_bad.append(result)

for granule_link in sorted(granule_links_EPIC): print(granule_link)
for result in FOR_results_EPIC_bad: FOR_results_EPIC.remove(result)

https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805004554_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805015122_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805025649_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805040216_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805050743_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805071838_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805082405_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805092932_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805103500_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805114028_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805124555_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805135123_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805145650_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805160217_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805170745_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805181312_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805191839_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805202406_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805212934_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805223501_03.he5
https://asdc.larc.nasa.gov/data/DSCOVR/EPIC/L2_AER_03/2023/08/DSCOVR_EPIC_L2_AER_03_20230805234028_03.he5

5.2. Download DSCOVR EPIC granules

and ensuring all granules have been downloaded

downloaded_files = earthaccess.download(FOR_results_EPIC, local_path='.',)

# Checking whether all DSCOVR EPIC data files have been downloaded
for granule_link in granule_links_EPIC:
  EPIC_fname = granule_link.split('/')[-1]
# check if file exists in the local directory
  if not os.path.exists(EPIC_fname):
    print(EPIC_fname, 'does not exist in local directory')
# repeat attempt to download
    downloaded_files = earthaccess.download(granule_link,
                                            local_path='.')
# if file still does not exist in the directory, remove its link from the list of links
    if not os.path.exists(EPIC_fname): granule_links_EPIC.remove(granule_link)

 Getting 21 granules, approx download size: 0.0 GB

File DSCOVR_EPIC_L2_AER_03_20230805135123_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805004554_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805114028_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805223501_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805170745_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805191839_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805015122_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805145650_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805181312_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805071838_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805212934_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805092932_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805025649_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805082405_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805202406_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805103500_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805050743_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805040216_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805124555_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805234028_03.he5 already downloaded
File DSCOVR_EPIC_L2_AER_03_20230805160217_03.he5 already downloaded

6. For every DSCOVR EPIC granule, find simultaneous TEMPO granules and re-map DSCOVR EPIC data to geolocations of TEMPO

write re-mapped DSCOVR EPIC UVAI to a netCDF file and plot the original DSCOVR EPIC and TEMPO along with re-mapped DSCOVR EPIC UVAI

# Setting TEMPO name constants
short_name = 'TEMPO_O3TOT_L2' # collection name to search for in the EarthData
version = 'V03' # this is the latest available version as of August 02, 2024

# cycle by found DSCOVR EPIC granules
for granule_link in sorted(granule_links_EPIC):

  last_slash_ind = granule_link.rfind('/')
  Dfname = granule_link[last_slash_ind+1 : ]
  print(Dfname)

  aod2D, fv_aod, uvai2D, fv_uvai, lat2D, fv_lat, lon2D, fv_lon\
, wl, fv_wl, yyyy, mm, dd, hh, mn, ss = read_epic_l2_AER(Dfname)

  if isinstance(lat2D, float): continue

  timestamp = datetime(yyyy, mm, dd, hh, mn, ss)
# it was discovered that actual timespan of an EPIC granule begins 289 s before
# the granule timestamp and ends 107 s after it.
# This timeframe will be used for search of TEMPO granules
  timestamp1 = timestamp + timedelta(seconds = -289)
  timestamp2 = timestamp + timedelta(seconds = 107)
  print(timestamp, timestamp1, timestamp2)

  for attempt in range(2):
    try:
      results = earthaccess.search_data(short_name = short_name\
                                      , version = version\
                                      , temporal = (timestamp1, timestamp2))
      break
    except: continue

  try: n_gr = len(results)
  except: n_gr = 0

  print('total number of TEMPO version ', version,' granules found', \
        '\nwithin period of interes between', timestamp1, 'and', timestamp2,\
        ' is', n_gr)

  if n_gr == 0: continue # if no TEMPO granules found within the DSCOVR EPIC timeframe, go to the next EPIC granule

# masking out DSCOVR fillvalues
  mask = (lat2D != fv_lat)&(lon2D != fv_lon)&(uvai2D != fv_uvai)
  points = np.column_stack((lon2D[mask], lat2D[mask]))
  ff = uvai2D[mask]

  downloaded_files = earthaccess.download(results, local_path='.',)

  for r in results:
    granule_links = r.data_links()
    last_slash_ind = granule_links[0].rfind('/')
    Tfname = granule_links[0][last_slash_ind+1 : ]

    lat, lon, fv_geo, time, uvai, uvai_QF, uvai_fv\
 = read_TEMPO_O3TOT_L2_UVAI(Tfname)

    polygon = TEMPO_L2_polygon(lat, lon, fv_geo)
    coords_poly = list(polygon)
    poly = Polygon(coords_poly)

# create arrays in indices to restore 2D array after re-mapping
    (nx, ny) = lat.shape
    y_ind = np.tile(np.linspace(0,ny,ny, endpoint = False, dtype = int), (nx,1))
    x_ind = np.tile(np.linspace(0,nx,nx, endpoint = False, dtype = int), (ny,1))\
.transpose()

# masking out fill values of TEMPO lat/lon positions
    mask_TEMPO = (lat != fv_geo)&(lon != fv_geo)&(uvai != uvai_fv)
    lon1D = lon[mask_TEMPO]
    lat1D = lat[mask_TEMPO]
    pp = np.column_stack((lon1D, lat1D))

    x_ind_m = x_ind[mask_TEMPO]
    y_ind_m = y_ind[mask_TEMPO]

# masking out DSCOVR UVAI to the ranges of TEMPO granule
    min_TEMPO_lon = min(lon1D)
    max_TEMPO_lon = max(lon1D)
    min_TEMPO_lat = min(lat1D)
    max_TEMPO_lat = max(lat1D)

    mask_DSCOVR = (uvai2D != fv_uvai)\
                 &(lat2D > min_TEMPO_lat)&(lat2D < max_TEMPO_lat)\
                 &(lon2D > min_TEMPO_lon)&(lon2D < max_TEMPO_lon)
    lon1D_DSCOVR = lon2D[mask_DSCOVR]
    lat1D_DSCOVR = lat2D[mask_DSCOVR]
    uvai1D_DSCOVR = uvai2D[mask_DSCOVR]
# number of DSCOVR pixels falling into ranges min_TEMPO_lat < lat2D < max_TEMPO_lat, min_TEMPO_lon < lat2D < max_TEMPO_lon
    n_DSCOVR_TEMPO = len(uvai1D_DSCOVR)
    if n_DSCOVR_TEMPO == 0:
      print('no original DSCOVR pixels within TEMPO granule')
      continue

    mask_DSCOVR_TEMPO = np.empty(n_DSCOVR_TEMPO, dtype = np.bool_)
    for i in range(n_DSCOVR_TEMPO):
      pp_DSCOVR = np.array([lon1D_DSCOVR[i], lat1D_DSCOVR[i]])
      p = Point(pp_DSCOVR)
      mask_DSCOVR_TEMPO[i] = p.within(poly)
      
    lon1D_DSCOVR_TEMPO = lon1D_DSCOVR[mask_DSCOVR_TEMPO]
    lat1D_DSCOVR_TEMPO = lat1D_DSCOVR[mask_DSCOVR_TEMPO]
    uvai1D_DSCOVR_TEMPO = uvai1D_DSCOVR[mask_DSCOVR_TEMPO]

# line below performs re-mapping
    DSCOVR_TEMPO_uvai = griddata(points, ff, pp, method='linear'\
, fill_value=-999., rescale=False)
# check whether there are any values within valid range
    valid_mask = (DSCOVR_TEMPO_uvai>-30)&(DSCOVR_TEMPO_uvai<30)
    if len(DSCOVR_TEMPO_uvai[valid_mask]) == 0:
      print('no re-mapped DSCOVR pixels within TEMPO granule')
      continue

# create and fill 2D arrays to be restored
    lat2D_TEMPO = np.empty([nx, ny])
    lat2D_TEMPO[:, :] = -999.
    lon2D_TEMPO = np.empty([nx, ny])
    lon2D_TEMPO[:, :] = -999.
    uvai2D_TEMPO = np.empty([nx, ny])
    uvai2D_TEMPO[:, :] = -999.

# restore 2D arrays
    for ix, iy, lon1, lat1, uvai1 in\
 zip(x_ind_m, y_ind_m, lon1D, lat1D, DSCOVR_TEMPO_uvai):
      lat2D_TEMPO[ix, iy] = lat1
      lon2D_TEMPO[ix, iy] = lon1
      uvai2D_TEMPO[ix, iy] = uvai1

# write restored 2D arrays to a netCDF file
    output_success = write_DSCOVR_TEMPO_UVAI(Tfname, lat2D_TEMPO, lon2D_TEMPO, uvai2D_TEMPO)
    if not output_success: print('failed to write DSCOVR UVAI re-mapped to TEMPO granule into the output file')

# plotting the output comparing TEMPO and DSCOVR EPIC UVAI
    fig = plt.figure(figsize=(20, 9), dpi=300, facecolor = None)

    proj = ccrs.LambertConformal(central_longitude=(min_TEMPO_lon + max_TEMPO_lon)*.5 # -96.0
                               , central_latitude=39.0
                               , false_easting=0.0
                               , false_northing=0.0
                               , standard_parallels=(33, 45)
                               , globe=None
                               , cutoff=10)
    transform=ccrs.PlateCarree()

    mask_TEMPO = (lat != fv_geo)&(lon != fv_geo)&(uvai != uvai_fv)
    lon1D = lon[mask_TEMPO]
    lat1D = lat[mask_TEMPO]
    uvai1D = uvai[mask_TEMPO]

    ax1 = fig.add_subplot(132, projection=proj)
    ax1.set_extent([min_TEMPO_lon, max_TEMPO_lon, min_TEMPO_lat, max_TEMPO_lat], crs=transform)
    im1 = ax1.scatter(lon1D, lat1D, c=uvai1D, s=1, cmap=plt.cm.jet\
                    , vmin=-4., vmax=4., transform=transform)
    ax1.coastlines(resolution='50m', color='black', linewidth=1)
    gl = ax1.gridlines(draw_labels=True, dms=True)
    gl.xformatter = LONGITUDE_FORMATTER
    gl.yformatter = LATITUDE_FORMATTER
    cb1 = plt.colorbar(im1, ticks=[-4, -2, 0, 2, 4], fraction=0.022, pad=0.01)
    cb1.set_label('UVAI', fontsize=10)
    ax1.set_title('UVAI '+Tfname, size = 10)

    ax2 = fig.add_subplot(133, projection=proj)
    ax2.set_extent([min_TEMPO_lon, max_TEMPO_lon, min_TEMPO_lat, max_TEMPO_lat], crs=transform)
    im2 = ax2.scatter(pp[valid_mask, 0], pp[valid_mask, 1]\
                    , c=DSCOVR_TEMPO_uvai[valid_mask], s=1, cmap=plt.cm.jet\
                    , vmin=-4., vmax=4., transform=transform)
    ax2.coastlines(resolution='50m', color='black', linewidth=1)
    gl = ax2.gridlines(draw_labels=True, dms=True)
    gl.xformatter = LONGITUDE_FORMATTER
    gl.yformatter = LATITUDE_FORMATTER
    cb2 = plt.colorbar(im2, ticks=[-4, -2, 0, 2, 4], fraction=0.022, pad=0.01)
    cb2.set_label('UVAI', fontsize=10)
    ax2.set_title('DSCOVR EPIC UVAI re-mapped', size = 10)

    ax3 = fig.add_subplot(131, projection=proj)
    ax3.set_extent([min_TEMPO_lon, max_TEMPO_lon, min_TEMPO_lat, max_TEMPO_lat], crs=transform)
    im3 = ax3.scatter(lon1D_DSCOVR_TEMPO, lat1D_DSCOVR_TEMPO, c=uvai1D_DSCOVR_TEMPO, s=1, cmap=plt.cm.jet\
                    , vmin=-4., vmax=4., transform=transform)
    ax3.coastlines(resolution='50m', color='black', linewidth=1)
    gl = ax3.gridlines(draw_labels=True, dms=True)
    gl.xformatter = LONGITUDE_FORMATTER
    gl.yformatter = LATITUDE_FORMATTER
    cb3 = plt.colorbar(im3, ticks=[-4, -2, 0, 2, 4], fraction=0.022, pad=0.01)
    cb3.set_label('UVAI', fontsize=10)
    ax3.set_title('UVAI '+Dfname, size = 10)

    plt.savefig('UVAI_'+Tfname+'.png', dpi=300)
    plt.close()

DSCOVR_EPIC_L2_AER_03_20230805004554_03.he5
2023-08-05 00:45:54 2023-08-05 00:41:05 2023-08-05 00:47:41
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 00:41:05 and 2023-08-05 00:47:41  is 0
DSCOVR_EPIC_L2_AER_03_20230805015122_03.he5
2023-08-05 01:51:22 2023-08-05 01:46:33 2023-08-05 01:53:09
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 01:46:33 and 2023-08-05 01:53:09  is 0
DSCOVR_EPIC_L2_AER_03_20230805025649_03.he5
2023-08-05 02:56:49 2023-08-05 02:52:00 2023-08-05 02:58:36
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 02:52:00 and 2023-08-05 02:58:36  is 0
DSCOVR_EPIC_L2_AER_03_20230805040216_03.he5
2023-08-05 04:02:16 2023-08-05 03:57:27 2023-08-05 04:04:03
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 03:57:27 and 2023-08-05 04:04:03  is 0
DSCOVR_EPIC_L2_AER_03_20230805050743_03.he5
2023-08-05 05:07:43 2023-08-05 05:02:54 2023-08-05 05:09:30
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 05:02:54 and 2023-08-05 05:09:30  is 0
DSCOVR_EPIC_L2_AER_03_20230805071838_03.he5
2023-08-05 07:18:38 2023-08-05 07:13:49 2023-08-05 07:20:25
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 07:13:49 and 2023-08-05 07:20:25  is 0
DSCOVR_EPIC_L2_AER_03_20230805082405_03.he5
2023-08-05 08:24:05 2023-08-05 08:19:16 2023-08-05 08:25:52
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 08:19:16 and 2023-08-05 08:25:52  is 0
DSCOVR_EPIC_L2_AER_03_20230805092932_03.he5
2023-08-05 09:29:32 2023-08-05 09:24:43 2023-08-05 09:31:19
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 09:24:43 and 2023-08-05 09:31:19  is 0
DSCOVR_EPIC_L2_AER_03_20230805103500_03.he5
2023-08-05 10:35:00 2023-08-05 10:30:11 2023-08-05 10:36:47
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 10:30:11 and 2023-08-05 10:36:47  is 0
DSCOVR_EPIC_L2_AER_03_20230805114028_03.he5
2023-08-05 11:40:28 2023-08-05 11:35:39 2023-08-05 11:42:15
Granules found: 0
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 11:35:39 and 2023-08-05 11:42:15  is 0
DSCOVR_EPIC_L2_AER_03_20230805124555_03.he5
2023-08-05 12:45:55 2023-08-05 12:41:06 2023-08-05 12:47:42
Granules found: 2
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 12:41:06 and 2023-08-05 12:47:42  is 2
 Getting 2 granules, approx download size: 0.21 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T123711Z_S001G05.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T124324Z_S001G06.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805135123_03.he5
2023-08-05 13:51:23 2023-08-05 13:46:34 2023-08-05 13:53:10
Granules found: 2
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 13:46:34 and 2023-08-05 13:53:10  is 2
 Getting 2 granules, approx download size: 0.2 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T134555Z_S002G06.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T135208Z_S002G07.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805145650_03.he5
2023-08-05 14:56:50 2023-08-05 14:52:01 2023-08-05 14:58:37
Granules found: 2
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 14:52:01 and 2023-08-05 14:58:37  is 2
 Getting 2 granules, approx download size: 0.2 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T144826Z_S003G06.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T145439Z_S003G07.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805160217_03.he5
2023-08-05 16:02:17 2023-08-05 15:57:28 2023-08-05 16:04:04
Granules found: 2
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 15:57:28 and 2023-08-05 16:04:04  is 2
 Getting 2 granules, approx download size: 0.2 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T155710Z_S004G07.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T160323Z_S004G08.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805170745_03.he5
2023-08-05 17:07:45 2023-08-05 17:02:56 2023-08-05 17:09:32
Granules found: 2
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 17:02:56 and 2023-08-05 17:09:32  is 2
 Getting 2 granules, approx download size: 0.21 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T165941Z_S005G07.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T170554Z_S005G08.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805181312_03.he5
2023-08-05 18:13:12 2023-08-05 18:08:23 2023-08-05 18:14:59
Granules found: 3
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 18:08:23 and 2023-08-05 18:14:59  is 3
 Getting 3 granules, approx download size: 0.31 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T180212Z_S006G07.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T180825Z_S006G08.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T181438Z_S006G09.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805191839_03.he5
2023-08-05 19:18:39 2023-08-05 19:13:50 2023-08-05 19:20:26
Granules found: 2
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 19:13:50 and 2023-08-05 19:20:26  is 2
 Getting 2 granules, approx download size: 0.21 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T191056Z_S007G08.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T191709Z_S007G09.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805202406_03.he5
2023-08-05 20:24:06 2023-08-05 20:19:17 2023-08-05 20:25:53
Granules found: 3
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 20:19:17 and 2023-08-05 20:25:53  is 3
 Getting 3 granules, approx download size: 0.31 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T201327Z_S008G08.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T201940Z_S008G09.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T202553Z_S008G10.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805212934_03.he5
2023-08-05 21:29:34 2023-08-05 21:24:45 2023-08-05 21:31:21
Granules found: 2
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 21:24:45 and 2023-08-05 21:31:21  is 2
 Getting 2 granules, approx download size: 0.21 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T212211Z_S009G09.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T212824Z_S009G10.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805223501_03.he5
2023-08-05 22:35:01 2023-08-05 22:30:12 2023-08-05 22:36:48
Granules found: 2
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 22:30:12 and 2023-08-05 22:36:48  is 2
 Getting 2 granules, approx download size: 0.21 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T222442Z_S010G09.nc
Downloaded: TEMPO_O3TOT_L2_V03_20230805T223055Z_S010G10.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)
DSCOVR_EPIC_L2_AER_03_20230805234028_03.he5
2023-08-05 23:40:28 2023-08-05 23:35:39 2023-08-05 23:42:15
Granules found: 1
total number of TEMPO version  V03  granules found 
within period of interes between 2023-08-05 23:35:39 and 2023-08-05 23:42:15  is 1
 Getting 1 granules, approx download size: 0.1 GB
Accessing cloud dataset using dataset endpoint credentials: https://data.asdc.earthdata.nasa.gov/s3credentials
Downloaded: TEMPO_O3TOT_L2_V03_20230805T233326Z_S011G10.nc
granule has 123 scanlines by 2048 pixels
polygon shape:  (4338, 2)