LIS/OTD 2.5 Degree Low Resolution Diurnal Climatology (LRDC)

Table of Contents

Name

LIS/OTD 2.5 Degree Low Resolution Diurnal Climatology (LRDC)

Version

v2.2

Short Description

The product is a 2.5 deg x 2.5 deg gridded composite of climatological total (IC+CG) lightning bulk production as a function of local hour, expressed as a flash rate density (fl/km^2/hr). Climatologies from the 5-yr OTD (4/95-3/00) and 8-yr LIS (1/98-12/05) missions are included, as well as a combined OTD+LIS climatology and supporting base data (flash counts and viewing times). Best-available detection efficiency corrections and instrument cross-normalizations, as of the product generation date (1/28/06), have been applied.

Format

The product is distributed in HDF (Hierarchical Data Format). HDF is accessible from C or Fortran using interfaces provided by NCSA, or through various commercial software packages, such as IDL or Noesys (http://www.rsinc.com). All data in this product are stored in HDF Scientific Data Sets (SDS).

Citation and Acknowledgements

In presentations or publications, users are to acknowledge the dataset as follows:

The v2.2 gridded satellite lightning data were produced by the NASA LIS/OTD Science Team (Principal Investigator, Dr. Hugh J. Christian, NASA / Marshall Space Flight Center) and are available from the Global Hydrology Resource Center (http://ghrc.msfc.nasa.gov).

Long Description

NOTE: Version 2.2 release included new documentation in PDF format, which can be viewed and downloaded here.

Approach

Bulk lightning production, expressed in fl/km²/yr, is calculated as a "counting experiment", i.e., all flashes observed by the sensors in a given grid location are summed for the full period considered (OTD, 5 yr; LIS, 8 yr), then divided by the actual instrument viewing time of the grid location. Flashes are weighted during the summation by the best instantaneous estimate of each instrument’s flash detection efficiency (DE); e.g., if the DE estimate is 50%, a value of 2.0 is added to the flash summation grid. Instrument viewing dropouts due to temporary platform or instrument down-time are considered. Partial grazing of a grid location by the instrument FOV is considered in the LIS estimates, but not considered in the OTD estimates. Best available DE corrections are applied as a function of instrument, local hour, and for the OTD, spatial location (see below). The combined flash rate climatology merges the corrected OTD and LIS climatologies to reduce variance in the tropical estimates.

Contents

The HDF file contains 12 Scientific Data Set (SDS) grids. These include:

LRDC_COM_FR     Combined Flash Rate [24x144x72, float, fl/km²/yr]

LRDC_OTD_RF     OTD Raw Flashes [24x 144x72, float, flashes]

LRDC_OTD_SF     OTD Scaled Flashes [24x 144x72, float, flashes]

LRDC_OTD_VT      OTD Viewtime [24x 144x72, float, km²-sec]

LRDC_OTD_FR      OTD Flash Rate [24x 144x72, float, fl/km²/hr]

LRDC_LIS_RF         LIS Raw Flashes [24x 144x72, float, flashes]

LRDC_LIS_SF         LIS Scaled Flashes [24x 144x72, float, flashes]

LRDC_LIS_VT         LIS Viewtime [24x 144x72, float, km²-sec]

LRDC_LIS_FR         LIS Flash Rate [24x 144x72, float, fl/km²/hr]

LRDC_OTD_DE      OTD Detection Eff. [144x72x24x3, float, %]

LRDC_LIS_DE         LIS Detection Eff. [144x72x24, float, %]

LRDC_AREA            Grid cell areas [144x72, km²]

Of these, LRDC_COM_FR is the primary data product; the combined LIS/OTD flash rate (FR) climatology. It may derived as: LRDC_COM_FR = (LRDC_OTD_SF + LRDC_LIS_SF) / (LRDC_OTD_VT + LRDC_LIS_VT) (for nonzero total viewing time grid cells). The supporting base grids (raw flash counts RF, detection efficiency-scaled flash counts _SF, and total viewing _VT) are provided only for reference or rebinning. Users are encouraged to view the viewing grids to understand the nature of the Low Earth Orbit satellite sampling, which is a component of the uncertainty at each grid location. The diurnally-applied detection efficiency corrections are also included (_DE; see below). A grid cell area (_AREA) product is also included, to simplify net production calculations (e.g., global annual production in fl/sec = sum(LRDC_COM_FR * LRDC_AREA)).

The dimension scales accompanying the SDS grids represent the center coordinate of each 2.5 degree grid cell (e.g., -178.75, -176.25 � 176.25, 178.75 deg lon, etc.) and local hour (e.g., 0.5 � 23.5).

Calibration

Best-available calibrations for instrument detection efficiency (as of 1/28/06) have been applied in the v2.2 product. These consider variations in instrument sensitivity across the angular field of view, variability in the background scene radiance (the instruments are designed to reduce sensitivity as background radiance increases in order to preserve signal-to-noise ratio), actual cross-sensor ground validation, changing sensor thresholds and cross-normalization of climatological results between the two instruments. The procedure, documented in Boccippio et al [2002] and Christian et al [2003] is roughly as follows:

The diurnal distribution of cloud background radiance (upon which sensitivity depends) is estimated from LIS background scenes concurrent with lightning events [Boccippio et al, JAOT, 2002].

Laboratory calibration data [Koshak et al, 2000] are used with (1) to estimate the diurnal distributions of minimum detectable pixel radiance.

A pre-launch Optical Pulse Sensor (OPS) data base of pulse radiance distributions acquired from high altitude (U2) airplane measurements [Christian and Goodman, 1987] are used with (2) to predict the diurnal distribution of maximum possible LIS and OTD flash detection efficiency (following and expanding upon [Koshak et al, 2000]). The OTD prediction is computed for threshold settings used from 10/23/96-4/1/00.

The LIS nighttime flash detection efficiency is estimated to be 88%, based on ground validation by [Thomas et al, 2000] and [Koshak et al, AGU Fall Meeting, 2000]. The LIS maximum possible flash detection efficiency diurnal distribution is scaled by this value.

The results of (3) and the uncorrected climatological data yield a LIS:OTD flash detection efficiency ratio of 1.67 [Boccippio et al, JAM, 2000; Boccippio et al, JAOT, 2002]. The OTD maximum possible nighttime flash detection efficiency is set to (1/1.67)*0.88, and the OTD predicted diurnal flash detection efficiency cycle from (3) is scaled accordingly.

The OTD diurnal flash detection efficiency cycle during other threshold setting windows (04/01/95-06/09/95, 06/09/95-07/20/95, 07/20/95-10/23/96, 10/23/96-04/01/00) is adjusted slightly by relative differences determined from ground validation [Boccippio et al, JAOT, 2000]. Three grids are reported in the file; these date windows use grids (0,2,1,0) respectively.

The OTD flash detection efficiency over South America is further scaled using the climatological ratio of LIS to OTD detections. OTD flash detection efficiency in this region drops within the South Atlantic Anomaly as adaptive software noise filters are more active to remove radiation noise events in the CCD pixel array. The peak reduction in DE from this effect is estimated to be an additional 50%, centered over Sao Paulo, Brazil.

Detection efficiency adjustments are thus prescribed as a function of (sensor, longitude, latitude, local hour, date in mission [OTD only]). These are included, for reference, in the HRFC_OTD_DE and HRFC_LIS_DE Scientific Data Sets. The bulk effect of the instantaneous corrections can be estimated as: BULK_OTD_DE = sum(HRFC_OTD_RF) / sum(HRFC_OTD_SF) (~ 47%) or BULK_LIS_DE = sum(HRFC_LIS_RF) / sum(BULK_LIS_SF) (~ 82%).

Users wishing to cite this calibration procedure may use or modify the following:

Observations in the LIS/OTD v2.2 reanalysis have been corrected by the LIS Science Team by estimated flash detection efficiency, applied as a function of sensor, local hour, date of mission, and (for the OTD) geographic location. For the entire dataset, these corrections correspond to average flash detection efficiencies of 47% (OTD) and 82% (LIS). The adjustments derive from a combination of laboratory calibration, ground-validation, and cross-normalization between the two instruments. Uncertainty in these corrections is estimated as +/- 10%. The calibration procedure is described in the dataset documentation.

Uncertainty

Consistency between modeled and ground-validated detection efficiency suggests that the applied values are known within about +/- 10%. This is thus the minimum uncertainty in the gridded data, arising from possible bias in the correction. Local variance arises from undersampling of a given grid location, and its relative impact increases as the actual local climatological flash rate decreases. This effect is likely dominant in the LRDC, but less important in the LRDC. During the third through fifth years of the OTD mission, problems with the host platform caused observations near local noon and midnight to be preferentially lost. In the combined diurnal climatology, early morning and mid-afternoon hours are thus sampled slightly (~15%) more than local midnight and noon. The effect in the diurnal decomposition is thus nonuniform variance in the hourly estimates, not a bias.

The primary error sources (measurement uncertainty, sampling-related variance) are currently being quantified.

Quality Assurance

All OTD and LIS orbits undergo both automated and manual quality assurance. For the preliminary reanalysis, the most stringent orbit rejection criterion was applied: any orbit which was assigned a manual Q/A "warning" flag has been rejected from the reanalysis. In subsequent versions, these rejected orbits may be examined individually and, if acceptable, added into the climatology to improve sampling.

Each OTD flash is further assigned an automated quality index (the ‘Thunderstorm Area Count’, or ‘Density Index’), indicating its likelihood of being lightning rather than optical or radiation noise. For this reanalysis, only flashes with values of the metric >= 140 have been included. This is the same cutoff value used in all validation and science analysis published by the LIS science team to date. This filter removes most radiation noise from the climatology; a slight residual ‘ring’ artefact of very low spurious flash rates remains at the periphery of the South Atlantic Anomaly (southeast Pacific and southeast Atlantic).

Recommended Usage

There are no restrictions on the use of these data. While best-available instrument calibrations have been applied, there are several known performance and algorithm issues still being addressed, and the applied calibrations have not yet been fully peer-reviewed and documented in refereed journals (see section 6). The data are best used for examination of relative diurnal and spatial variability. Further bias adjustments to the applied detection efficiencies will likely occur in the near future, and result in product version updates (see section 8).

Sampling in the Low Resolution diurnal climatology should be adequate to resolve broad regional trends in the phase and amplitude of the diurnal cycle. However, the diurnal cycle estimates for individual grid cells are fairly noisy and undersampled, so aggregation of data from several grid cells is recommended. As with the high resolution flash rate climatology, it is recommended that the individual scaled flashes (_SF) and viewtime (_VT) grids be aggregated separately, then divided, to yield lower resolution flash rate estimates; this is preferable over simple averaging of adjacent 2.5 degree flash rate (_FR) grid cells.

Versioning and Updates

All subsequent versions of this dataset (both minor and major revisions) will maintain the same product definition, resolution and file format. Additional years of data from the LIS mission may be added into the climatology for future minor versions, but will not alter file format. Any actual definitional changes would be reflected by the creation of a new product.

Subsequent major version updates will only occur if/when the dataset is recomputed using new major version OTD or LIS orbit data. E.g:

Status OTD Orbit Version LIS Orbit Version LRAC Version:

Preliminary                    1.1 4.0 0.x
Peer-reviewed               1.1 4.0 1.x
Subsequent 2 or higher
OR 5 or higher               2.x, 3.x, …

Current version is at v2.2, and covers years 1995-2005.

Version History

v0.1: 8/1/01

Preliminary Release; 5 yr. OTD (v1.1), 3 yr LIS (v4.0) orbit data

v1.0: 4/1/03   

First Official Release; 5 yr. OTD (v1.1), 5 yr LIS (v4.0) orbit data

v2.2: 9/1/06   

Second Official Release; 5 yr. OTD (v1.1), 8 yr LIS (v4.0) orbit data

Minor OTD calibration errors during 1995-1996 fixed.

Calibration procedure now peer-reviewed.

References

Instrument and Calibration/Validation:

Goodman,SJ; Christian,HJ; Rust,WD (1988): A comparison of the optical pulse characteristics of intracloud and cloud-to-ground lightning as observed above clouds. J. Appl. Met. 27, 1369-1381.

Christian,HJ; Blakeslee,RJ; Goodman,SJ (1989): The Detection of Lightning from Geostationary Orbit. J. Geophys. Res. 94, 13329-13337.

Christian,HJ; Blakeslee,RJ; Goodman,SJ (1992): Lightning imaging sensor for the Earth Observing System. (TM-4350.) NASA. 44 pages. Available from Center for Aerospace Information, P.O. Box 8757, Baltimore Washington International Airport, Baltimore, MD 21240.

Christian,HJ; Driscoll,KT; Goodman,SJ; Blakeslee,RJ; Mach,DA; Buechler,DE (1996): The Optical Transient Detector (OTD). Proc. 10th International Conference on Atmospheric Electricity, Osaka, Japan.

Kummerow, C; Barnes, W; Kozu, T; Shiue, J; Simpson, J (1998): The Tropical Rainfall Measuring Mission (TRMM) sensor package. J. Atmos. Oc. Tech. 15, 809-817.

Christian,HJ; Blakeslee,RJ; Goodman,SJ; Mach,DA; Stewart,MF; Buechler,DE; Koshak,WJ; Hall,JM; Boeck,WL; Driscoll,KT; Boccippio,DJ (1999): The Lightning Imaging Sensor. Proc. 11th Intl. Conf. on Atmospheric Electricity (NASA), Guntersville, AL, 7-11 June. 746-749.

Ushio,T; Driscoll,KT; Heckman,S; Boccippio,DJ; Koshak,WJ; Christian,HJ (1999): Initial comparison of the Lightning Imaging Sensor (LIS) with Lightning Detection and Ranging (LDAR). Proc. 11th Intl. Conf. on Atmospheric Electricity (ICAE), Guntersville, AL, 6-11 June. 738-741.

Christian,HJ; Blakeslee,RJ; Goodman,SJ; Mach,DM (eds.) (2000): Algorithm Theoretical Basis Document (ATBD) for the Lightning Imaging Sensor (LIS). Posted: 1 Feb 2000. (NASA / Marshall Space Flight Center, AL 35812)

Koshak,WJ; Bergstrom,JW; Stewart,MF; Christian,HJ; Hall,JM; Solakiewicz,RJ (2000): Laboratory calibration of the Optical Transient Detector and Lightning Imaging Sensor. J. Atmos. Oc. Tech. 17, 905-915.

Boccippio,DJ; Driscoll,KT; Koshak,WJ; Blakeslee,RJ; Boeck,WL; Mach,DA; Buechler,DE; Christian,HJ; Goodman,SJ (2000): The Optical Transient Detector (OTD): Instrument characteristics and cross-sensor validation. J. Atmos. Oc. Tech. 17, 441-458.

Thomas,RJ; Krehbiel,PR; Rison,W; Hamlin,T; Boccippio,DJ; Goodman,SJ; Christian,HJ (2000): Comparison of ground-based 3-dimensional lightning mapping observations with satellite-based LIS observations in Oklahoma. Geophys. Res. Lett. 27, 1703-1706.

Boccippio, DJ; Koshak, WJ; Blakeslee, RJ (2002): Performance assessment of the Optical Transient Detector and Lightning Imaging Sensor: I. Predicted diurnal variability. J. Atmos. Oc. Tech. 19, 1318-1332.

LIS/OTD-Enabled Science & Applications:

Christian, HJ et al (2003): Global frequency and distribution of lightning as observed by the Optical Transient Detector. J. Geophys. Res, 108 4005, doi: 10.1029/2002JD002347.

Boccippio,DJ; Wong,C; Williams,ER; Boldi,R; Christian,HJ; Goodman,SJ (1998): Global validation of single-station Schumann resonance lightning location. J. Atmos. Sp. Terr. Phys. 60, 701-712.

Christian,HJ; Blakeslee,RJ; Boccippio,DJ; Boeck,WL; Buechler,DE; Driscoll,KT; Goodman,SJ; Hall,JM; Koshak,WJ; Mach,DM; Stewart,MF (1999): Global frequency and distribution of lightning as observed by the Optical Transient Detector (OTD). Proc. 11th Intl. Conf. on Atmospheric Electricity (ICAE), Guntersville, AL, 7-11 June. 726-729.

Driscoll,KT (1999): A comparison between lightning activity and passive microwave measurements. 11th International Conf. on Atmospheric Electricity (ICAE), Guntersville, AL, 7-11 June 1999. 523-526.

Cecil, DJ; Zipser, EJ (2000): Relationships between tropical cyclone intensity and satellite-based indicators of inner core convection: 85 GHz ice-scattering signature and lightning. Mon. Wea. Rev., 111, 979-996.

Buechler,DE; Driscoll,KT; Goodman,SJ; Christian,HJ (2000): Lightning activity within a tornadic thunderstorm observed by the Optical Transient Detector (OTD). Geophys. Res. Lett ., 27, 2253-2256.

Goodman,SJ; Buechler,DE; Knupp,K; Driscoll,KT; McCaul,EW (2000): The 1997-98 El Nino event and related wintertime lightning variations in the Southeastern United States. Geophys. Res. Lett. 27, 541-544.

Jeker,DP; Pfister,L; Thompson,AM; Brunner,D; Boccippio,DJ; Pickering,KE; Wernli,H; Kondo,Y; Staehelin,J (2000): Measurements of nitrogen oxides at the tropopause: Attribution to convection and correlation with lightning. J. Geophys. Res., D 105, 3679-3700.

Boccippio,DJ; Goodman,SJ; Heckman,S (2000): Regional differences in tropical lightning distributions. J. Appl. Met. 39, 2231-2248.

Williams,ER; Rothkin,K; Stevenson,D; Boccippio,DJ (2000): Global lightning variations caused by changes in thunderstorm flash rate and by changes in the number of thunderstorms. J. Appl. Met. 39, 2223-2230.

Nesbitt,SW; Zipser,EJ; Cecil,DJ (2000): A census of precipitation features in the tropics using TRMM: Radar, ice scattering and lightning observations. J. Clim. 13, 4087-4106.

Rodgers, E; Olson, W; Halverson, J; Simpson, J; Pierce, H (2000): Environmental forcing of supertyphoon Paka’s (1997) latent heat structure. J. Appl. Met., 39, 1983-2006.

Boccippio,DJ; Cummins,KL; Christian,HJ; Goodman,SJ (2001): Combined satellite and surface-based estimation of the intracloud / cloud-to-ground lightning ratio over the continental United States. Mon. Wea. Rev. 129, 108-122.

Toracinta, ER; Zipser, EJ (2001): Lightning and SSM/I ice-scattering mesoscale convective systems in the global tropics. J. Appl. Met. 40, 983-1002.

Boccippio, DJ; Heckman, S; Goodman, SJ (2001): A diagnostic analysis of the Kennedy Space Center LDAR network. 2: Cross-sensor studies. J. Geophys. Res. 106, 4787-4796.

Chang, DE;Weinman, JA; Morales, CA; Olson, WS (2001): The effect of spaceborne microwave and ground-based continuous lightning measurements on forecasts of the 1998 Groundhog Day storm. Mon. Wea. Rev., 129, 1809-1833.

Ushio,T; Heckman,S; Boccippio,DJ; Christian,HJ (2001): A survey of thunderstorm flash rates compared to cloud top height using TRMM satellite data. J. Geophys. Res., D, 106, 24089-24095.

Boccippio,DJ (2002): Lightning scaling laws revisited. J. Atmos. Sci. 59, 1086-1104.

Bond, DW; Zhang, R; Tie, X.; Brasseur, G.; Huffines, G; Orville, R.E; Boccippio, DJ (2001): NO_x production by lightning over the continental United States. J. Geophys. Res, 106, 27701-27710.

Koike, M.; Kondo, Y.; Akutagawa, D.; Kita, K.; Nishi, N.; Liu, S.C.; Blake, D.; Kawakami, S.; Takegawa, N.; Ko, M.; Zhao, Y.; Ogawa, T. (2003) Reactive nitrogen over the tropical Western Pacific: Influence from lightning and biomass burning. J. Geophys. Res, 108, 8403, doi:10.1029/2001JD00823.

Sample Code

A set of Interactive Data Language (IDL) routines to extract and process the LIS/OTD High Resolution Full Climatology (HRFC), Low Resolution Full Climatology (LRFC), Low Resolution Diurnal Climatology (LRDC) and Low Resolution Annual Climatology (LRAC) are distributed with the data. The IDL syntax is roughly similar to C or FORTRAN and, with the exception of animation code, porting of these routines should be fairly straightforward. Full documentation can be found in the program files themselves; a summary is below.

NOTE: These routines are being provided as a courtesy to the user community. The GHRC and LIS Science Team cannot guarantee technical support or compatibility with IDL version updates or platform-specific implementations.

GETGRID.PRO:

GETGRID,HDF_NAME,SDS_NAME,GRID,[DIMS,DIMNAMES,DIM0 …]

Retrieves a scientific data set (and optionally, its dimensions) from one of the HDF climatology files.

LONLAT_TO_XY.PRO:

Result = LONLAT_TO_XY([LON,LAT],RESOLUTION)

Converts a [lon,lat] pair to a grid [x,y] index, for a given grid resolution (0.5 or 2.5 deg)

Result = XY_TO_LONLAT([X,Y],RESOLUTION)

Converts an [x,y] index pair to a grid cell center [lon,lat], for a given grid resolution (0.5 or 2.5 deg)

SMOOTHING_ROUTINES.PRO:

Result = HRFC_XY_SMOOTH(N_GRIDCELLS,SENSOR_NAME)

Performs a spatial moving average on the 0.5 deg High Resolution Full  Climatology with size N_GRIDCELLS. This, e.g., can help smooth the values while preserving high-contrast information (as compared with  the Low Resolution Full Climatology). The routine loads the needed grids directly from the HDF file.

Result = LRAC_TIME_SMOOTH(N_DAYS,SENSOR_NAME)

Performs a temporal moving average with an N_DAYS window on the Low Resolution Annual Climatology. Strongly recommended for all use  of this climatology; with N_DAYS prescribed in multiples of 55 days.  The routine loads the needed grids directly from the HDF file.

Result = LRAC_XY_SMOOTH(OLDGRID,N_GRIDCELLS)

Performs a spatial moving average on the 2.5 deg results of LRAC_TIME_SMOOTH, to improve samplin

MAKE_JGR.PRO:

Recomputes the statistics and graphics used in H.J. Christian et al, "Global frequency and distribution of lightning as observed by the Optical Transient Detector", submitted to J. Geophys. Res., to illustrate statistical processing of the climatological datasets.

MORE_EXAMPLES.PRO:

THREE_CLIMS

Renders three versions of the Full Climatology: the 0.5 degree HRFC, the HRFC smoothed with a 2.5 deg moving average, and the 2.5 degree LRFC.

ANIMATE_ANNUAL,XSIZE,YSIZE

Uses the IDL routine XINTERANIMATE to animate the combined climatology annual cycle, and optionally export the movie to MPEG. The movie will have pixel dimensions XSIZE and YSIZE.

ANIMATE_DIURNAL,XSIZE,YSIZE

Uses the IDL routine XINTERANIMATE to animate the combined climatology local hour diurnal cycle, and optionally export the movie to MPEG. The movie will have pixel dimensions XSIZE and YSIZE

Contact Information

Questions regarding dataset ordering, media issues, file handling or HDF file access (input/output) should be directed to the Global Hydrology Resource Center (ghrc@msfc.nasa.gov). Questions regarding the science data processing, including gridding, viewing, calibration and variance should be addressed to Dennis.Boccippio@nasa.gov. Questions regarding the OTD or LIS missions themselves should be addressed to the OTD/LIS Principal Investigator, Hugh.Christian@nasa.gov.

To order these data or for further information, please contact:

Global Hydrology Resource Center
User Services
320 Sparkman Drive
Huntsville, AL 35805
Phone: 256-961-7932
E-mail: support-ghrc@earthdata.nasa.gov
Web: http://ghrc.nsstc.nasa.gov/