The LiDAR sensor by Apple sensor, what is it?
This technology seems to open many possibilities for 3D scanning and augmented reality applications on smartphones. But actually what is LiDAR and how can it be used for 3D acquisition?
This sensor is available on iPhone 12 and 13 Pro and iPhone 12 and 13 Pro Max devices. But many users of these devices are ignoring the possibilities provided by the LiDAR sensor.
I thus propose to you to discover a little more in this article concerning this technology that you have in your hands.
The LiDAR sensor, “Light Detection And Ranging”, is a technology using light to estimate the distance with an object. Thus, it sends out light beams that are reflected once they reach the object in question. The transmitter (i.e. the scanner LiDAR) receives this laser and estimates the delay between the emission and the reception to define the distance.
Focus on augmented reality
The main application of LiDAR technology is mainly focused on augmented reality. In addition to filters, (Snapchat introduced a new filter at the Apple conference in October 2020) augmented reality allows users to see how their new furniture may look in their living room or bedroom.
But the potential of this technology goes far beyond that. Many companies are dreaming of technologies that can mix virtual and real objects. For example, the augmented reality glasses on which Facebook, Snapchat, Microsoft, and most likely others like Apple are working on.
These technological feats will be possible thanks to a 3D cartography of the real world on which virtual objects are superimposed.
Of course, Apple is not the first smartphone manufacturer to integrate this kind of sensor into its devices. Huawei and Samsung, to name a few, have chosen “Time of Flight” sensors that behave in the same way as a LiDAR sensor. This means measuring the distance of objects from the camera.
An improved focus
Thanks to its pulsed light system, the LiDAR sensor turns out to be a real support to other cameras. It helps them to get a quick focus on objects or people in the picture. The LiDAR sensor is particularly useful in low light, since traditional autofocus is often not very effective in conditions where the light is not optimal.
The fact of analyzing the 3D environment, will allow the photo software to better detach the background from the foreground.
Thanks to the “Light Detection And Ranging” sensor, the focus will be 6 times faster than before on iPhone, ensures Apple!
Thus, shots in “Portrait” mode are improved. Indeed, they are not always successful – when the light is a bit low, it still often happens that the artificially created blur in the background protrudes on the main object. Therefore, the accuracy of the pictures improves with LiDAR even under non-optimal conditions.
Many opportunities for 3D
This photo unit, already found in the last generation of iPad Pro, is not a photographic sensor like others. Its role is not to capture the frame, but rather to create a map of a 3D environment. The LiDAR sensor allows you to measure the distance from the object to the camera by using a pulsed light system. This is similar to the function of a radar that uses microwaves, except LiDAR uses infrared waves instead.
A real-time 3D reconstruction becomes possible by capturing the 3D environment scanned by the phone and therefore the LiDAR sensor.
This type of light beam emission sensor can be found in autonomous cars, in robotics and in some drones. Indeed, they map the environment in three dimensions to avoid obstacles. In their case, the sensors also use radio waves to create a vision system to detect curbs and other cars.
On these latest Apple smartphones, the better focus is a big advantage. It is even possible that these latest devices will be able to add more 3D data to photos. An information that outsiders developers could use to imagine new applications.
LiDAR sensor and photogrammetry ?
Photogrammetry is a technique to establish 3D measurements from photographs. Simply, it allows to take pictures of an object so that a software assembles the images in 3D. This enables the digitization of a 3D model that integrates the reliefs, textures and colors of the object.
This technology makes possible to transform any smartphone into a 3D scanner thanks to some mobile apps like the one we develop: MyFit Solutions.
To come back to LiDAR and go even further, this sensor coupled with a photogrammetry technology allows to considerably improve the result of the final 3D models. LiDAR works in poor lighting conditions, even at night. The point cloud, generated by photogrammetric processes, are more visual but often with generalized details while the LiDAR sensor is better at capturing the depths and shapes of the object in a less geometrical way. A revolution to obtain a final rendering more detailed but also well textured and colored in real time!
Clearly, thanks to LiDAR we obtain :
- A real-time 3D reconstruction of the captured object,
- Better remote sensing of depths,
- More efficient photo sensor in low light,
- A support to the cameras to obtain a better accuracy in the 3D reconstruction of objects,
All these improvements coupled with photogrammetry allow us to extend the possible uses for 3D acquisition from a smartphone.
Thus, opening new possibilities in many sectors of activity such as medical, construction or consumer goods where the need for precision is important. In a nutshell, thanks to LiDAR we are expanding the field of possibilities!
Indeed, for us, it is sure to be the advent of many innovative uses! The possibilities are huge and extremely motivating. In fact, it feeds our possibility to work with professionals from different sectors of activity in order to meet their needs, define new uses and innovate in mobile acquisition.
MyFit solutions for 3D
That’s why we developed MyFit Solutions. Today, we are still enriching our technology. MyFit Solutions is a software editor specialized in 3D acquisition from mobiles and product customization for brands and manufacturers. Our technology is independent of the LiDAR sensor, so MyFit Solutions can be used on a wide selection of mobile devices.
This technology, available on more and more mobile devices, allows us to consider the possibility of going even further in terms of precision in the 3D scans taken via our application.
Indeed, we offer manufacturers the possibility of personalize their products on large scale, while remaining competitive. We offer several solutions for 3D acquisition :
- 3D scanning and customization of products for the ear,
- Instant recommendation of the size of a product by taking measurements
- 3D scanning of the human body (face, bust, torso, wrist, foot,…).
- 3D scanning and 3D collaboration on physical objects (mechanical parts, consumer goods,…),
We invite you to discover our activity and the associated use cases on our website : MyFit Solutions.
Our commitment is to facilitate experiences based on the expertise of exact measurements and understanding of the environment through mobile. And you, what use of mobile 3D acquisition are you interested in and want to do?
