Reconstructing 3D Bug Shape and Colour
This page presents a preliminary result from research by Mitchell Buckley under the supervision of Changming Sun at the CSIRO. This research is being funded by CMIS (CSIRO Mathematics, Informatics and Statistics) under the Graduate Fellow program and CSIRO's Transformational Biology Platform. This is part of a broader effort in accelerating phenomics within CSIRO.
The fundamental problem we are aiming to solve is "3D shape reconstruction from multiple images". What we require is a robust algorithm that can take a series of images of an object or scene and develop a 3D representation of the scene from the 2D image input. This problem does not have many easy solutions and is still a very active area of research. In the long term we aim to apply our research to automated insect taxonomy and passive plant phenotype measurement.
The images displayed below were acquired by Paul Jackway (supported by the CSIRO's Transformational Biology Platform) in May 2010. The subject is an insect that Paul collected and prepared himself. It appears to be of the class "Hemiptera", the "true bugs". In total, Paul captured 360 images. They are evenly spaced at one degree intervals in a ring around the bug. Each image came in 24-bit colour with resolution 2452 x 2052. Paul has imaged a number of other insects that are not displayed here.
To obtain the 3D model below we have used a simple "shape-from-silhouette" technique. Shape-from-silhouette is among the most robust and efficient techniques for obtaining basic shape information. Unfortunately, it is severely limited in the detail it can produce and is highly sensitive to a prior image segmentation step. Note that in this case the basic shape has been well recovered, but small errors are also apparent.
The model displayed below is about 1MB, please be patient if it takes a little while to load. A high resolution model can be found here (about 14MB, you will need Meshlab or similar to view this file).
Note that we are mainly concerned with the shape of the model and not the colour. This is because the colouring problem is relatively straight forward and relies heavily on obtaining a good shape.
It should also be noted that this result, while pleasing to the eye, is just a taste of what is possible. State-of-the-art methods produce results that are more detailed and more accurate.