Light Field Mapping - New Appearance For 3D Scanning?

On 14 August 2001 the Intel Corporation announced the development of a new software technology called Light Field Mapping (LFM), which helps to create more life-like 3D images for interactive applications by correctly modelling light reflection properties of 3D objects. Among its applications, Intel believes that LFM has great potential for improving the 3D scanning industry. CAD SPAGHETTI asked Radek Grzeszczuk, Senior Research Scientist with Intel's Microprocessor Research Labs, to elaborate.

The growing availability of inexpensive yet powerful graphics hardware and new advances in digital imaging technology are enabling novel methods for realistic modeling of the appearance of physical objects. Especially popular are image-based rendering and modeling techniques that attempt to represent the radiance properties of objects directly in the image-based format. There are however problems to overcome before these image-based approaches become practical. Most notably, the amount of data that needs to be acquired, processed and displayed is enormous. Also, the quality of images generated from image-based representations is often overlooked. Because of these limitations, image-based techniques, despite their appealing characteristics, have failed to produce compelling computer graphics applications.

Introducing LFM

At Intel we are showing that it's possible to overcome these problems. We believe that light field mapping (LFM) provides a method for the compact representation and interactive visualization of highly realistic 3D models. An LFM-based model describes simply and accurately, not only the 3D shape of an object, but also its precise appearance from all possible viewing angles. Even though the information represented by each LFM-based model is vast, our data format is very compact and allows for interactive visualization of these models on the personal computer. This method is particularly relevant to modeling reflectance properties of objects scanned into the computer using 3D photography techniques because it preserves the natural look of these objects, as shown in Figure 1.

Figure 1: a combination of synthetic and physical objects rendered directly from their light field map approximations. Complex, physically realistic reflectance properties of these objects are correctly represented and visualized.

However, LFM can also be applied to visualization, not of individual objects, but of complete environments that are either scanned or generated synthetically. Figure 2 shows snapshots of a complex synthetic environment we were able to render in real time using LFM.

Figure 2: snap shots of a complex synthetic environment visualized interactively using light field mapping.

Note that all objects shown in these figures change their appearance very rapidly from viewpoint to viewpoint because their surface reflects light differently in different directions. Objects that exhibit this property are generally difficult to scan using traditional 3D scanners, and are difficult to visualize using traditional computer graphics techniques.

Applications

Let's assume for now that we have a 3D scanner that can capture both the 3D shape and the view - dependent appearance of an object. (We will describe later a working prototype of such a scanner that we have built in our lab.) The potential applications of this device look very interesting. For example, one of the persistent problems in 3D graphics today is how to automatically generate photorealistic 3D content that can be put on the web or used in movies, broadcast media, and computer games. Most of the existing 3D content providers sell 3D models that only describe the object's shape, because the object's appearance is too difficult to acquire, process, and display. The LFM technology allows for simple acquisition, description, and fast visualization of the photorealistic appearance of objects.

Almost any industry that needs to convey the natural look of an object, or a scene, stands to benefit from LFM. For example, if we can scan both the shape and the view-dependent appearance of complete environments-such as rooms, houses, and factories - using LFM we can have virtual, interactive tours of these environments that are stunningly real. Industries such as architectural design, virtual reality and 3D computer games would certainly like to have this capability. Electronic educational material providers would be able to enhance their products, such as e-books or electronic encyclopaedias, with realistic, interactive 3D illustrations that better explain new concepts and ideas. E-commerce generally could take on a new look; companies would be able to make their online presence more appealing by placing natural looking 3D models of their products on their websites.

'Almost any industry that needs to convey the natural look of an object, or a scene, stands to benefit from LFM.'

Current Limitations

3D scanners available today can acquire the shape of a scanned object quite accurately. Rarely, however, do these devices come equipped with the capability of capturing the appearance of the scanned object too. In the best case, the scanner will produce a single image describing the appearance of the object from one viewing angle. This image can then be pasted on the scanned geometry of the object using the technique called texture mapping. Although a texture mapped model

looks more realistic, it still does not look natural, because it has the same appearance from every viewing angle. In reality, objects change their appearance, often dramatically, from viewpoint to viewpoint. For example, objects that are specular, or made of a velvet-like material, have this property. One image is therefore not enough to represent the accurate appearance of many physical objects. Ideally, we would like to have a different image for each view of the object, but this is clearly impractical. LFM offers a smarter and more compact method of encoding the view dependent radiance information.

'In reality, objects change their appearance, often dramatically, from viewpoint to viewpoint.'

How LFM Works

LFM representation can be thought of as a special type of texture map that changes its appearance with the viewing angle. Because it is compact and allows for hardware-accelerated rendering, this representation is ideal for accurate modeling of the appearance of many physical objects with complex surface reflectance properties. That's why we think that LFM has a great potential for pushing the envelope of 3D scanning technology.

To build the LFM representation of a physical object, first we need to acquire the information about its 3D shape and its view-dependent appearance. To this end, we have built a prototype 3D scanner that scans the shape of the object using a structured-light approach and then collects samples of the appearance of the object in the form of 200-400 photographs acquired using a digital camera from different viewpoints around the object. The raw images coming from the scanner are then organized, resampled, and

processed using statistical data analysis tools. This data processing pipeline reduces the size of raw data coming from the scanner from 2-3GB to less than 1MB. The final LFM representation consists of a small set of texture maps and the 3D geometry. Unfortunately, a specialized viewer is required to visualize the 3D objects from the LFM representation but the viewer is very simple and should be able to run on almost any PC. Intel is currently working on standardizing LFM and so, in the near future, the majority of 3D visualization tools should support this data format.

'Intel is currently working on standardizing LFM and so, in the near future, the majority of 3D visualization tools should support this data format.'

Although our scanner is not completely automatic, it works as a proof of concept. It enables us to construct the LFM representation of diverse physical objects with complex reflectance properties. For example, the metallic bust shown in Figure 1 is highly specular and its surface is covered with chisel marks that are difficult to model geometrically. We were able to reproduce the appearance of this object very accurately and we were able to model its surface details using images only. The glass star shown in Figure 1 is even more difficult to model using traditional computer graphics techniques, since it looks semi-transparent from certain viewing angles and anisotropically reflective from others. Although difficult to model analytically, these complex reflectance properties of the star are easily represented using LFM.

More Information about Light Field Mapping

For more information on LFM and for example videos see the web site: http://www.intel.com/research/mrl/research/lfm/
Intel's press release on LFM can be found at:
http://www.intel.com/pressroom/archive/releases/20010814tech.htm

Many thanks to Radek Grzeszczuk for producing this article for CAD SPAGHETTI. Please note that the original US spelling style has been preserved in this article.

If you need to research the features required for the launch of a successful new product or service, and assess its likely demand, our Market Research Services can provide the solution. If you want to generate sales leads in domestic or international markets, find out about our Telemarketing Team.
Alternatively, call David Eaton on +44 (0)1689 873636 to discuss your needs.


	CAD SPAGHETTI Light Field Mapping - New Appearance For 3D Scanning? September 2001