Orthophotos from multiple images
The Orthophoto option spawns external programmes, some of which do not tolerate blank spaces in file names or folder names together with an image name longer than 255 characters.
An option will be offered to copy to a new directory path if this is detectted.
If you don't want to do this and click the No button, please rename all image files or folders which do not conform to these rules or error messages will be displayed.
Click here on the main toolbar in AirphotoSE to begin:
The main window for making orthophotos from multiple images appears:
Click Select to load images from a directory. Right click on the name area and choose icons (thumbnail images) if you prefer. Select images in a desired sequence with Ctrl-Left mouse or Shift-Left mouse. The order of selected images affects the success of including the images in the set. Choose the image which shows the largest area first so that other images can be joined to it. It sometimes helps to rotate images so that their orientation approximately corresponds to that of the first image chosen. This can be done with the normal tools in AirPhotoSE after displaying a file. Be sure to save the rotated image under a new name so that you can identify it easily.
Right click on the selection window, select View and choose "Icons" of a convenient size first so that you can see thumbnail images. When you click on one of these, it will also be shown in the preview at the upper right. Most RAW image file types do not display in the icons, but they can be seen in the preview in the upper right corner when clicked on.
When the icons for the files have been loaded, a right click on an icon temporarily disables its use in the computations but does not delete it. The image name background changes from green to red, and if you want to re-enable it, a second right click changes it back to green.
If you have disabled a number of images whose icons are not visible because you have scrolled the display, you can re-enable all of them by holding down the Alt key and right clicking on any icon.
Never select images in which the horizon is visible. The image in the upper left corner of the selection window shown below has a bit of the horizon visible at the top. The horizon is at "infinity", and this will cause calculations to fail. Crop the image first using the cropping tool on the main toolbar in AirPhotoSE to remove this part of the picture before opening it in the orthophoto window. If the image is very oblique, crop the top of the image showing the most distant area. If computation of the orthophoto tile from this image is too wide at the top, an error message will be displayed. When this happens, close the orthophoto window, crop those showing too wide an area in the distance and recompute everything.
When you have finished selecting images and close the selection window, thumbnails showing your selection along with their names, sizes and image depth in bits will be drawn on the thumbnails. If there are no blanks in file or directory names, then the processing order will be the order in which you selected the images, otherwise, the order will be alphabetical by image name with blanks removed.
When you close the selection window, begin computation by clicking on the Run button
A log of all processing steps is shown in the left window;
What is happening:
Keypoints are salient image points like road crossings, field boundary edges, houses and other geometrically pronounced features. These are detected automatically by the programme. Keypoint detection, correspondences between the keypoints in the selected images and bundle adjustment are then used to compute orientations, image locations and other needed data which will be run as soon as the selection window is closed.
If Details to show computation steps is selected (the text on the button changes from red to green and the window caption changes too, these are displayed in pop-up windows. If one of the many steps involved fails, it is helpful to see what has happened by looking at these pop-up windows and read the error message. If the matching of keypoints fail or if the adjustment of the re-projection errors between images fails, then an image may be excluded from use in further computation. In this case, the word "excluded" appears next to it's name. This can happen if an image is chosen which does not overlap any of the other images or if it's orientation, colour and content differs greatly from them.
When Bundle Adjustment finishes, a Stanford PLY file with the name of the first image selected and the extension PLY will automatically be written to the image directory. This file contains height information computed from the multiple images and may be used in other programmes or in AirPhoto3D which can be downloaded from the AirPhotoSE websites to make 3D displays of multiple image data.
Click Reload if you wish to load a new set of images. Click Run to run the steps above again if you want to see the computation details and forgot to click the Details
A number of computation options are available. Click These may be changed either when starting or after running if the results are unsatisfactory.
Computing the orthophotos and displaying them with CloudCompare:
If you have processed a large number of images, CloudCompare has problems with the limited memory administration in the OpenGL graphic library code it uses. Read the content of this link for a very effective way to work around this limitation:
Using Cloud Compare with Microsoft ICE.
to start CloudCompare. CloudCompare is an external programme spawned by AirPhotoSE.
An options window appears:
The result will now be computed by CloudCompare by rectifying the selected images and superimposing them on a Digital Terrain Model made from the point cloud. The DTM lies behind the images and can be viewed by dragging with the mouse. Unchecking the boxes next to the names of the images will selectively remove each image. Select a value for the maximum width/height of the result which will contain all the orthophotos. Clicking on an image name in the list will show a rectangle drawn around the extents of an image to show it's location relative to the others in case it is entirely hidden by a larger image.
If you have not cropped extreme oblique images which show the distant background, you may receive an "out of memory" error message from the operating system or an error message from CloudCompare stating that the image is too wide to display. If this happens, close CloudCompare, close the orthophoto window, return to the main AirphotoSE window, load the extreme obliques one at a time and crop the tops with the crop option. For details about the error messages from CloudCompare, click the link above.
If images used overlap so much that one image is hidden behind all the others, you may de-activate it by unchecking the box with it's name:
If there are very oblique images included, some of them may project out far beyond the central area of interest. You can prevent this by cropping the offending images as described above,
Making output images in CloudCompare:
You can save the output to a Windows file in various formats by clicking on the Display item of the CloudCompare menu. CloudCompare has limited memory capabilities for this option. If a very large number of images have been processed, e.g. from a UAV, the option described in the link above should be used or the suggestion below followed.
The first time you use the CloudCompare output option, click the Display Settings option.
A dialogue box will open showing default settings.
Select a desired background colour. White or Black must be selected if you want to calibrate the image for georeferencing with the transformation option of AirPhotoSE so that the surrounding area will not overwrite a georeferenced map, Google Earth image, satellite image or orthophoto chosen in the GeoPortal option. Four or more identical points in the CloudCompare or ICE output and in the reference image must be identifiable, see Transformation
Uncheck the Display gradient background if you have chosen a white or black background for this calibration method, and uncheck Show middle screen cross boxes. If you plan to calibrate an image manually, any background or gradient may be selected.
Microsoft's Image Compositing Editor as described in the link above always makes a black background. No other choice is available.
Click Apply and then OK. The settings will be recorded for future use.
Then click Render to file.
A dialogue box will pop up in which you can select the type of file to be written to disk and the degree of enlargement needed.
The maximum output size is limited by the display card hardware and driver. If you choose a degree of enlargement which the display card can not process, the result may be a completely black image or you may see an error message. The first time this option is used, it is advisable to start with a modest amount (4x screen size) of enlargement. Look at the result shown at the bottom of CloudCompare. If the enlargement is too great, a red error message appears. Repeat the save operation with increased or decreased enlargement until no error message appears. The value must be chosen afresh each time the feature is used.
In hilly terrain, a result may look like:
Processing large numbers of images:
If you have a large number of images, for example from an unmanned aerial vehicle (UAV), then it may be helpful to use options which automate most operations. If you are only interested in making a large orthophoto from many oblique images at high resolution, turn off the "Use CC Gui" option in the orthophoto options dialogue. When the bundle adjustment phase of AirPhotoSE has completed, clicking on the Apply button will create a partial orthophoto from each image while displaying a small progress bar window:
When this completes,
you will be shown an option window offering to display the results. If Microsoft ICE has been installed, it will look like:
and if not, it will look like:
Selecting the container display option if ICE is installed, or simply the stitch and show option if not, will load all partial orthophotos into a single image very quickly without the manual intervention as is necessary in CloudCompare described above.
This can be saved to nearly all of the AirphotoSE supported output file formats. The container is a Windows memory mapped file which can be zoomed with a wheel mouse or left and right mouse clicks and dragged by holding down the left mouse button. There are no limits on the size of the container other than available disk space, so the limited memory resources available when saving results when working directly with CloudCompare do not cause problems.
The appearance of the result will be the same when using the container method or rendering it in CloudCompare. When Microsoft ICE is used, differences in colour, brightness, alignment and seams will be equalised and the result may be more satisfactory as you can see by comparing the image below with the image above.
However, Microsoft ICE also has memory limits which may cause failure if there are more images than it can stitch and write to a file, whereas the container method will usually be able to deal with a much larger number.
Georeferencing (calibrating) the Orthophoto from the CloudCompare output:
CloudCompare stitches the images together and superimposes them with use of the collinearity equations of photogrammetry on a Digital Terrain Model (DTM) computed from the images themselves. However, the DTM has neither a defined North nor is it horizontal with respect to the Earth's surface when computed from the images alone. One needs a map or a vertical image with defined scale which can be obtained either from one of the GeoPortal sites in the working area or with Google Earth or Bing 3D if their resolutions in the area of interest are roughly similar to that of the rectified oblique images from orthophoto.
Here are step by step instructions on what to do next for example with Google Earth. You can also use a digital map of the area of interest.
Move the GE image so that the point of interest at the centre of one of the photos stitched with Cloud compare, usually a photo showing the archaeological features is roughly centred in GE. Save the GE image and if need be stitch it to show a larger area at higher resolution as described in the GeoPortal calibration chapter, having previously set the grid to whatever is desired, e.g. WGS84-UTM, EPSG etc. A GIS world file for the output may also be selected.
Save the orthophoto result from CloudCompare with a white or black background set in CloudCompare's display settings and with gradients turned off.
Following the instructions in the transformation section, enter four or more control points at road crossings, field boundaries or other clearly observable points in the CC image and at the corresponding four or more points in the GE image.
You can also select a white or a black background for the result.
In AirphotoSE, open Options
and set the output to Insert Black . Source-Target if you have chosen a black background, or choose Insert White Bgd. Source-Target if you have chosen a white background. Do not choose other colours. If the image file format is PNG, these must be reversed. All other formats behave normally.
This choice will prevent the white or black background of the CloudCompare image from appearing after transformation, leaving only the orthophoto from CC superimposed on the Google Earth image.
If the GE image has been calibrated, the lower status bar will show the mouse's position in the currently chosen grid, and if also chosen, a World file may be written for further use in a GIS programme.
It may be helpful to select the area of interest area in the CC image with the selection tool from the main toolbar before transformation so that the CC output is framed exactly. The rectification accuracy is not good in those areas which are in the distance on an oblique image, so it is advisable to hide these using the selection tool before transformation.
If the GE window has the focus, the scale can be set in meters per pixel or in percentage of the size of the CC result if the CC image has the focus. The resulting image will be an orthophoto with north at the top and the image plane horizontal with respect to the Earth's surface. When moving the mouse cursor, the ground coordinates will be shown correctly on the status bar.
With both the GE and the CC windows click on the camera symbol on the main toolbar with a white lens to transform from the CC image to the GE image. This automatically then makes a black or white clone of the GE image, with the calibration and with the control points copied from the GE original. Choose the White or Black background option for this.
If you want to use the georeferenced output with a GIS programme, check the appropriate kind in the Setup-Options dialogue before you start.
A "point cloud" graphically displays heights computed from all the images coloured according to the image colours at that point. Some other programmes use point clouds as their primary display technique.
It is not necessary to compute or display a point cloud when orthophotos or coloured digital terrain models are wanted. Click simply on Apply when the calculations have finished. If you have elected to display the point cloud, clicking on Apply will display it in greater detail and offer all the advanced graphic options available in CloudCompare instead of the options described above.
The point cloud option is included for comparison with other programmes which do not have the more sophisticated display options available in AirPhotoSE for orthophotos and coloured digital terrain models. Select it only if you want to make this kind of comparison.
On termination of the Run option, click the PointCloud button to see either a preview of the point cloud or all details using CloudCompare.
The protocol of creation and display of the point cloud will be shown.
A large number of images will require very long computation times during which the entire processing power of the machine will be devoted to this task and slow or prevent execution of anything else. The machine may appear to freeze if you switch to another task. When the PointCloud button has been chosen, the Apply button will only display the point cloud in full screen mode and offer the graphic processing options of CloudCompare only.
Point cloud computation can be accelerated by choosing the smallest possible sample window in the orthophoto options dialogue.
The point cloud calculation is done by a separate spawned programme plus several preliminary data preparation programmes and takes roughly a minute or more per image depending on resolution on a machine with a 32 bit operating system. Calculation is somewhat faster on a machine with a 64 bit operating system.
If there are lot of large images, it will require a considerable amount of time. The more overlapping images there are, the better the appearance of the point cloud will be. However, unless a very large number of images overlap, there will be visible ugly black holes in the displayed image where height data has not been computed. Click on F6 to turn off the artificial lighting and appearance may improve.
You can also expand the Point Size by clicking on the + sign in the upper left corner to fill in some of the holes in the cloud if you wish, but the image sharpness suffers if the point size is too large.
You can drag the point cloud with the left mouse button to any position and orientation. You can enlarge or reduce it with the mouse wheel, and if you press F6 you can turn the artificial light source on and off. F11 expands the image to full screen and pressing it again returns to the original size.
When there is considerable structure in the data, the illuminated point cloud is sometimes instructive.
You can use a pre-computed point cloud for alternate keypoints in Cloud Compare instead of those passed by the bundle adjustment phase of computation. There is usually more surface detail in this set, but anomalous points are also introduced which usually have to be removed by editing using the "scissors" or CC options in Cloud Compare before using it to make a surface reconstruction.