Generating PREFIRE browse images using HyBIG

Overview

This notebook demonstrates how to generate browse images, i.e., tiled images, by sending a request to the NASA Harmony API, which will run the HyBIG service and return the results.

Table of Contents

  1. Setup
  2. Make request to Harmony
  3. View the results

Dataset Information

PREFIRE is Polar Radiant Energy in the Far InfraRed Experiment. PREFIRE data has projection EPSG:4087 with a resolution of 500ths of a degree (~2200 m) over the full earth.

The Global Imagery Browse Services (GIBS) desires EPSG:4326 for its global images, and the Harmony Browse Imagery Generator (HyBIG) will default to that projection.

By agreement with GIBS, HyBIG’s maxumum untiled image size is limited to 8192 * 8192 (67,108,864) cells. Since PREFIRE TIFs exceed that (18000 x 9000 or 162,000,000 cells), HyBIG will tile the output to 4096 x 4096 cells.

Background

We began this work when Hazem reached out to Owen over slack and told us that when he was testing PREFIRE data all of his images were black.

Turns out I handled missing data incorrectly when we turned on scaling and masking of the input data. So I implemented a quick fix to normalize the data correctly (DAS-2108) and now we handle PREFIRE data.

1. Setup

We will use the harmony-py library.

from harmony import Collection, Environment, Client, Request

# creates an output directory for the downloaded files
from pathlib import Path

output_dir = Path('./prefire-output')
Path.mkdir(output_dir, exist_ok=True)

Create a Harmony Client object to interact with the UAT environment with the PREFIRE collection and selected granule.

prefire_collection = Collection(id='C1263096190-EEDTEST')
prefire_granule = 'G1263096196-EEDTEST'

harmony_client = Client(env=Environment.UAT)

2. Make the request to Harmony

# Specify a request to create a browse image from the PREFIRE granule:
prefire_request = Request(
    collection=prefire_collection, granule_id=prefire_granule, format='image/png'
)

# Submit the request and wait for it to complete:
prefire_job_id = harmony_client.submit(prefire_request)
harmony_client.wait_for_processing(prefire_job_id, show_progress=True)

# Download all of the generated files:
Path.mkdir(output_dir, exist_ok=True)
downloaded_outputs = [
    file_future.result()
    for file_future in harmony_client.download_all(
        prefire_job_id, overwrite=True, directory=output_dir
    )
]
 [ Processing: 100% ] |###################################################| [|]
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.txt
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c00.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c00.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c00.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c01.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c01.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c01.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c02.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c02.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c02.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c03.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c03.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c03.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c04.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c04.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c04.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c00.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c00.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c00.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c01.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c01.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c01.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c02.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c02.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c02.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c03.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c03.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c03.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c04.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c04.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r01c04.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c00.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c00.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c00.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c01.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c01.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c01.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c02.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c02.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c02.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c03.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c03.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c03.png.aux.xml
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c04.png
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c04.pgw
prefire-output/4149276_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r02c04.png.aux.xml

3. View the result

We see HyBIG has created 15 output tiles (3 rows of 5)

!ls prefire-output/*.png | wc
      15      15    1350

Each browse image tile is comprised of three files:

A PNG image file: 4145523_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c00.png - PNG files are the paletted raster representation of the data in a standard image format.

An ESRI world file: 4145523_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c00.pgw - World files establish an image-to-world transformation that converts the image coordinates to real-world coordinates. This is the metadata file to submit to GIBS. A drawback to these files is that they do not contain the Spatial Reference System (SRS) for the image, but only the GeoTransoform.

An ARCGIS auxiliary file: 4145523_PREFIRE_SAT2_2B-FLX_S07_R00_20210721044449_03042.nc.G00.r00c00.png.aux.xml - An auxiliary file (.aux.xml) accompanies the raster in the same location and stores any additional information that cannot be stored in the raster file itself. In this case it contains the Spatial Reference System (SRS) and the GeoTransform describing the pixel location. This file is used when importing the image into a GIS workspace.

Open all of the png images in QGIS

!open -a qgis prefire-output/*.png

Questions?

Please reach out in #analysis-browse-image in slack.