If you are planning to make a 3D game or application, experiment with AR or VR, you may have already thought about 3D scanning. Let's talk about different implementations of this technology, as well as what equipment you need.
Scanning is now being used more and more, and with the development of this technology, it may become even more in demand in the development of games and applications, including for VR. Reconstructing objects will allow you to create the most realistic games, even if you do not have a full-fledged game studio, and will also significantly save time and money.
In this article, you will learn about four ways to scan reality:
- structured light systems;
- ToF cameras and lasers;
- light field cameras.
The first way to scan reality is photogrammetry. This is the most affordable and versatile way to get a three-dimensional model of an object or scene. Photogrammetry determines the shape, size and position of objects from photographs.
How it works?
At the first stage we take a series of photos using a regular camera. We photograph an object or scene from all sides so that the pictures are overlapped.
In the second stage The software finds defining matches on each photo. There are several methods for obtaining such descriptors: SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. Comparative analysis of these tools in Russian is not yet available. To study this issue in detail will help english article at Research Gate.
In the third stage The software maps the descriptors to each other. A three-dimensional point cloud is obtained, which describes the reconstructed object. For each photo, the software builds a depth map, where the distance from the camera to each pixel is calculated.
The software combines data from several depth maps and builds an object mesh.
The algorithm then takes into account the average pixel value and textures the mesh. Result on video.
The finished model cannot be used to create 3D content or VR. After photogrammetry, manual revision is needed, you can read about it in more detail in article on the Exyte website…
What equipment do you need?
Several devices are suitable for photogrammetry – a smartphone camera (the object in the picture above was shot on a Samsung S8 +), a DSLR or mirrorless camera with the following functions and properties:
- fixing the white balance;
- fixing the focal length;
- minimum lens distortion;
- fast shutter speed when shooting handheld.
If you shoot with a smartphone, keep in mind that photogrammetry requires a manual shooting mode and good optics. It is best to use new models to get a quality result. On Android and iOS there are applications that take pictures of an object and show a preliminary scan result in real time. After the object is reconstructed on a smartphone or in the cloud, the result is not impressive. The resulting models are not suitable for 3D content or VR.
In our opinion, the most promising application is display.land…
Even the best smartphone cameras have too small the physical dimensions of the sensor and lenses. Without special software and hardware filters, they are inferior to full-format cameras, especially if the lighting is imperfect.
If you are wondering if a smartphone camera can compete with system cameras, you can read about it in article on the DXO website…
To reverse engineer using MeshRoom requires a powerful computer (like Core i7, 32GB RAM, Nvidia CUDA).
Limitations of photogrammetry
Photogrammetry has several limitations:
- object surfaces should not be reflective, transparent or solid;
- the object should not move and change shape;
- lighting and optical parameters of the camera should not be changed.