Satellite Image Compositing

In the past, most detailed remotely-sensed views of the earth have been acquired from airplanes. But commercial satellite sensors have been increasingly more relevant lately, with the best having sub-1m accuracy. Although far from cutting-edge, moderate resolution Landsat data sets have multiple spectral bands, good availability, and the advantage of being able to see larger areas without too much compositing. For this case study, we will be using Landsat satellite data to produce a color image of a 500km by 200km area around Mexico City (comparison for U.S. folks: Think the size of Illinois). Even though Landsat satellite scenes cover a fairly large area, the total area of interest is large enough that it will require adjacent scenes to be composited together. Each of the following pages takes a look at some challenges encountered during this project.

A good first consideration is final image size. Modern Landsat images have a resolution of 30m by 30m per pixel. This means that an image representing a 500km by 200km area would have a pixel size of 16,666x6,667. On most systems, this would be 10 to 15 times as wide as the computer monitor. Some uses may require this precision, but often it would just be overkill. Rescaling is the way to get around this. For the Mexico example, downsampling by a factor of two results in reasonable image sizes (around 8000x3500) while still preserving details. An example of the resampling's effect on a portion of the image is shown below.

Next we must do data acquisition for the image. Landsat data sets are divided into scenes which cover different parts of the earth, with each scene being designated by a path, row, and date. The satellite passes over any given spot on the earth every 16 days. Due mostly to cloud cover, good and clear images are usually only available for selected dates. For the Mexico image, we will need data from seven different scenes. Each scene has an image for up to seven panchromatic spectral bands, which can be combined for color images. A resampled part of Band 3 from Path 27, Row 47 is shown below. These are huge images, around 8000x8000 pixels each.

Actually putting the images together will come later, but for now we need to create uniform color images for each scene and deal with any image quality problems that might come up. Bands 3, 2, and 1 in Landsat represent the red, green, and blue parts of the spectrum. Because of equipment quirks and the hundreds of kilometers of atmosphere the satellite has to look through, a simple combination of these three bands does not produce a very realistic image. As you can see in the sample combination below, the image is murky, tinted blue, and has low contrast.

To correct this, each image requires some color manipulation, often using the fourth (infrared) band to make a realistic color combination. Every scene also requires a slightly different technique to get looking right, but good results can be achieved.

Clouds: Part 1

After image colorization, the next challenge in compositing shows itself clearly in the image below.

Most scenes are remarkably clear of clouds, but a few remain. The white spots on the right side of this image are all clouds. While they do not completely spoil the image, clouds are an annoyance that somewhat lessen the image's impact. For some uses like overlay onto an elevation model, clouds would really ruin the effect. But there are some ways to deal with cloud cover. Note that, in the image above, the southernmost clouds are close to the edge of this scene. Overlap from one scene to the next is considerable, and in this case the next scene does indeed have the desired area, cloud-free.

A combination of these images, replacing the original clouded areas with data from the bottom scene, achieves the desired effect of erasing the clouds. You may notice that the colors of theses scenes are different. This is due to the images being acquired at different times of the year, an issue that will be discussed later. The eventual cloud-free final image is shown below.

Clouds: Part 2

The previous case of cloud cover was fairly easily corrected because we had redundant data, from the adjacent scene. For the northernmost clouds in the scene, however, that option is not available. These clouds are in the center of the scene, with no chance of overlap from other scenes.

Notice that the vegetation patterns in this area are fairly uniform. The forest appears largely undeveloped. One technique to remove clouds from this location is cloning. Available in most image processing software, cloning is a fairly simple technique of replacing certain parts of the image with other parts. If done correctly, areas with clouds can be replaced by forest pixels to effectively remove the clouds. The resulting image is shown below.

Clouds: Part 3

Just southeast of Mexico City, we encounter another cloud problem. These mountains are right in the center of the scene. Cloning is not a reasonable option here, since mountains contain intricate details that are not easily replicated. Worse, these volcanoes are some of the most famous in Mexico, so showing them obscured by clouds is not an option. But there are reasonably cloud-free, but older, images available. They were acquired over 10 years earlier, using a different satellite sensor, with undoubtedly different settings.

Not surprisingly, colors from this older image are considerably different than on the newer one, even with extensive color work. Fortunately, only a small portion of the images, right around the volcanoes, needs to be used. Smoothly merging the images together gives an excellent view of the volcanoes without cloud cover.

Seasonal Change and Final Compositing

One potential issue, noted early in the data acquisition phase, was that most scenes had different acquisition dates. All were in the year 2000, but several were taken in different seasons. Most of the images were acquired in March, September, or December. The problem comes when you try to line up two adjacent scenes, taken at completely different times of year. This image illustrates the problem.

Here we have two adjacent scenes, acquired at two different times, September and December. The September image obviously has much more vegetation than the December one. Altering either image is really not feasible, so perhaps the best solution to this is to simply do a smooth fade between the scenes (using the scene overlap) to create a nice transition. It's not perfect, but in this case it works well since the northernmost scene is at a lower elevation and naturally greener anyway.

After clouds and colors have been systematically dealt with in each of the scenes, it's time to put them all together. Since each scene has been dealt with individually, compositing itself is fairly easy. A greatly resampled version of the final image is shown below.

Of course, one application of having this photorealistic image is using it as an overlay onto an elevation model. The image below was created by using a small part of the final composite to show the volcano Orizaba, highest point in Mexico. A larger version can be found in the Orizaba Image Gallery.

Questions or comments? Send to info @ topoimagery.com.