The satelight molecular disassembler sees the real and the artifical , I.S.I.S.
eye in the sky
www.osti.gov/bridge/purl.cover.jsp;jsessionid=F5A2D1938944CFD68BF0766EF6186E95?purl=/1262-3OvErv/webviewable/ISIS; AN -NFORMATIOK’-EFFICIENT SPECTRAL IMAGING SYSTEM
WC Sweatt, CA Boye, SM Gentry, MR Descour*,
BR Stallard, and CL Grotbeck
Sandia National Laboratories, Albuquerque, NM
“University of Arizona, Tucson AZ
wsweatt@sandia.gov
A specialized hyperspectral imager has been developed that preprocesses the spectra
from an image before the light reaches the detectors. This “optical computer” does not
allow the flexibility of digital post-processing. However, the processing is done in real
time and the system can examine = 2 x lo6 scene pixels/sec. Therefore, outdoors it
could search for pollutants, vegetation types, minerals, or man-made objects. On a highspeed
production line it could identify defects in sheet products like plastic wrap or film,
or on painted or plastic parts.
ISIS is a line scan imager. A spectrally dispersed slit image is projected on a Spatial
-Li ght Modulator. The SLM is programmed to take the inner product of the spectral
intensity vector and a spectral basis vector. The SLM directs the positive and negative
parts of the inner product to different linear detector arrays so the signal difference equals
the inner product. We envision a system with one telescope and =4 SLMs.
INTRODUCTION
Different classes of materials can be separated from one-another because they have
different reflectance spectra in the 0.5-pm to 2.4-pm spectral region. Figure 1 a is an
image of a scene containing soil, trees, and a tank. Figure 1 b shows their spectra in the
visiblehear-IR’ (0.45 pm Ih I1. 05 pm). A set of basis vectors, shown in Figure IC,c an
be calculated for these spectra using spectral linear discriminant analysis2. These vectors,
reminiscent of eigenvectors, would emphasize the differences between the spectra of the
different material classes.
One can search for specific materials in an area by scanning it with a hyperspectral
imager, processing the data, and comparing the results with known results. The data
processing consists of taking the spectra from each image pixel and taking the inner
product (dot product) of the pixel’s spectra and each of the spectral basis vectors. The
constants computed from the inner products can then be compared with known results.
As an example, consider searching a scene (Fig. 1 a) using two basis vectors. For each
pixel of the scene, the two inner products have been taken and the results have been
plotted, producing the scatter plot shown in Figure Id. Note that the background and the
material sought tend to cluster into three different areas, one of which is “interesting”.
Most hyperspectral instruments use a line-scan-imager architecture. These instruments
move an image of the scene across a slit. After passing through the slit, the spectra are
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the
United States Department of Energy under Contract DE-AC04-94AL8.5000.
DISCLAIMER
This report was prepared as an account of work sponsored
by an agency of the United States Government. Neither the
United States Government nor any agency thereof, nor any
of their employees, make any warranty, express or implied,
or assumes any legal liability or responsibility for the
accuracy, completeness, or usefulness of any information,
apparatus, product, or process disclosed, or represents that
its use would not infringe privately owned rights. Reference
herein to any specific commercial product, process, or
service by trade name, trademark, manufacturer, or
otherwise does not necessarily constitute or imply its
endorsement, recommendation, or favoring by the United
States Government or any agency thereof. The views and
opinions of authors expressed herein do not necessarily
state or reflect those of the United States Government or
any agency thereof.
DISCLAIMER
Portions of this document may be illegible
in electronic image products. Images are
produced from the best available original
document.
dispersed perpendicular to the slit, and then the slit is reimaged on a two-dimensional
detector array (e.g. a CCD). The coordinates on the detector array are x, measured
along the length of the slit, and h, the wavelength which varies in the perpendicular
direction. Time multiplied by the velocity of the scan produces y, the other spatial
coordinate.
Two typical airborne hyperspectral systems now in operation are HYDICE33a4n d
AVIRIS5.6. These systems collect 128 and 256 spectral bands respectively. They both
record the data during each flight and then process it afterwards. This architecture is
quite appropriate for aircraft-mounted systems.
However, this architecture is not nearly as logical for a satellite-borne hyperspectral
imager if one needs to search large areas with high resolution. For example; if one
wished to search a 1 00-km wide swath of land with a 10-m resolution, this implies that 7
million pixels must be processed per second. If there are 128 spectral channels, then
there must be 9 0 0 ~ 1 A0/~D c onversions per second. If 12-bit accuracy is required, the
system must store 10" bits/sec. If the desired swath length were 2000 km, then the
system would have to store 3 . 2 ~ 1 0b' i~ts of data, and process it or send it to earth.. . This
all sounds rather prohibitive.
We propose the ISIS architecture for this application because it reads about twenty-fold
fewer spectral channels. Thus there are twenty-fold fewer detectors and A D operations
per second. Furthermore, the collected data can be process real-time, since only 3
arithmetic operations per pixel are required (per spectral basis vector). Therefore,
uninteresting pixels can be ignored and only the useful data needs to be stored and/or sent
to earth. In many applications, only a small percentage (2%?) of the data is interesting,
so the required storage capacity is reduced by three orders of magnitude. The downlink
capacity will also be reduced a thousand-fold. Remember that there is a price for this
efficiency. One can only search for one (or a few) materials with this architecture. So if
you know what you are looking for, the ISIS scheme will allow you to either reduce the
amount of data handled, or to increase the number of pixels searched per orbit. If you
don't know what will be found in the scene, a standard hyperspectral imager may be
better.
ISIS ARCHITECTURES
ISIS is an imaging system containing a programmable spectral filter that can accentuate
the spectral differences between the target and the background. It could be described as
an optical computer that multiplies the spectra from the scene by a spectral basis vector
before the photons reach the detectors. Figure 2 is a schematic showing the ISIS
architecture. A telescope images a moving scene onto a slit. Light passing through the
slit is dispersed perpendicular to the slit and then the slit is reimaged on an amplitude
mask. The transmission of this mask is set equal to the spectral basis vector. Thus the
transmission varies only in the spectral direction so each pixel along the slit length is
treated the same. This mask passes the desired colors (appropriately attenuated) and
rejects the rest. Following the amplitude mask, the spectra are recombined and an image
of the slit is created on a linear array of detectors.
The amplitude mask shown in Figure 2 can be programmed once which might be useful
for some industrial applications where the selection of targets and the light level never
change. However, for the remote sensing application a spatial light modulator (SLM)
that could be reprogrammed would be far more useful. One possibility is shown in
Figure 3, where the amplitude mask has been replaced with a liquid crystal SLM and a
pair of polarizers. In this design, the first polarizer linearly polarizes the light fiom the
scene. The liquid-crystal SLM must have a linear array of long, narrow windows that are
individually addressable. Each liquid-crystal window rotates the polarization of the light
in the narrow bandwidth impinging on it. A specified fraction of the light in each
waveband can thereby pass the second polarizer (the analyzer) and will eventually reach
the detector array.
Figure l b shows two basis vectors cpz(h) and cp3(h) in a set, (cpl(h), cpz(h), cp@),
cpd(h). . . >. The first vector, cpl(h), is usually similar to the solar spectrum and always
positive. Hence it is not near as useful as the others are (<pl(h)c,p 3(h),c p4(h).. .) for
discriminating between materials in the scene because it has relatively little structure.
However, the other basis vectors are negative over parts of the spectrum, so they can't be
accommodated with the simple ISIS architectures shown in Figures 2 and 3. However,
using the double system shown in Figures 4a and 4b, basis vectors with negative parts
can be accommodated. This is achieved as follows.
First the basis vector is separated into its positive and negative parts as shown in Figure
5.
vi(h) = Pi(h) - Ni(h), for i = 2 & 3 . (1)
The two halves of <pi(h) are coded into the two SLMs in Fig. 4a, which then take the two
dot products of the basis vectors with the scene spectra;
Afi(x,t) = S(h,x,t) Pi(h)
A-i(x,t) = S(h,x,t) Ni(h)
Therefore the dot product of the whole basis vector and the scene spectrum is the
difference between the two parts,
and they can be subtracted digitally. Note: The dot product can be normalized if the
difference in (3) is divided by the sum of the two parts, A+i(x,t) + A-i(x:t). This
modification would reduce the effect of illumination brightness.
This led us to the (Fig. 4a) architecture that uses two liquid crystal SLMs. The two SLMs
take the two halves of the dot product described by equations 2a and 2b. The system
shown in Figure 4b was built from off-the shelf components, including a silicon-based
detector array, a liquid crystal SLM, and lenses and prisms made for visible light
operation. The optical train has very good image quality over a modest field of view, and
for wavelengths between 0.45um and 1.05 um.
EXPERIMENTAL RESULTS
The prototype ISIS system (Figs. 4a & 4b) has been used to distinguish artificial from
real ivy. Figure 6a is a photograph of the scene. Figure 6b shows the spectra and a
spectral basis vector to find the artificial ivy. Figure IC is the scene (Fig. 6a) as seen by
the ISIS system. Figure 6d is a histogram of the pixels, where the small hump is the
artificial ivy, and Figure 6e is the scene as seen by ISIS (Fig. 6c), and then thresholded.
Admittedly this is a contrived example, but it does demonstrate the utility of the ISIS
approach. S tallard and Gentry’ will discuss these experiments in detail.
CONCLUSIONS AND FUTURE WORK
A hyperspectral system architecture has been proposed that allow large areas to be
searched with high resolution; rates of 27 x 1 O6 pixel& are anticipated. Furthermore,
the computations can all be done essentially in real time. Therefore, only “interesting”
data needs to be stored. ISIS could also be used to direct the attention of a standard
hyperspectral system to the ‘interesting” regions, and it could collect a full complement
of spectral data.
A proof-of-principle ISIS system has been built using visible-light components. In a first
set of tests, the ISIS system has been used to distinguish between plastic and natural
vegetation. More realistic tests are planned, most under natural lighting conditions. If
the near-term testing is encouraging, an ISIS will be constructed for a bandwidth that
covers the visible and the short-wave IR ( 0.5 pm to 2.2 pm ). We also anticipate the
development of ISIS architectures that can take two or more dot products simultaneously,