Realize a 3D morphological reconstruction from the Iphone LiDAR sensor

Sensors are more and more present in our daily life, they land today on our mobile devices. The 3D LiDAR iPhone is a new generation sensor that equips the latest Apple products. The firm has decided to tackle the market of 3D acquisition from a smartphone. What we were interested in! We have therefore conducted a study on this sensor. Discover in this article all you need to know about 3D morphological reconstruction from the iPhone LiDAR sensor: uses, advantages and limitations.

Table des matières

Scanning with a LiDAR iPhone sensor

What is LiDAR technology?

LiDAR “Light Detection and Ranging” is a detection and localization technology using laser beams to measure the distance between the device and objects. The sensor emits a light beam that reflects off the scanned object. Then, by receiving the return beam, it evaluates the distance thanks to the time spent.

This technology can be used to map an environment and the contours of objects in three dimensions. This makes the LiDAR sensor a promising technology for mobile 3D acquisition!

You will also find on the remote imaging market the “Time of Flight” ToF sensors. In the same way as the LiDAR sensor, the ToF sensor estimates the distance of objects from the camera by measuring the round trip time of a light beam. Apple has bet on LiDAR, so let’s see how this technology is used in the iPhone, iPad or iPad Pro.

How to use the iPhone Lidar?

The iPhone has been equipped with a LiDAR sensor, which greatly improves image capture thanks to precise depth measurements. This new dimension allows to make 3D reconstructions. This means that objects can be reproduced in three dimensions from their surface. Thus, we can create, for example, avatars for augmented reality applications.

However, the potential of this technology could go further! That’s why we’ve been looking into this topic, as we see this tool as a feat to allow us to extend the possible uses for 3D acquisition from a smartphone. In the following article we reveal the results of our study: Pre-study to model and reconstruct in 3D objects from a smartphone equipped with a LiDAR sensor.

Analysis of the 3D LiDAR Iphone reconstruction

How to scan in 3d with an iPhone ?

With the development of 3D Lidar, it is now possible to make 3D scans with its iPhone equipped with the LiDAR sensor. Apple has facilitated the scanning of a person, small objects or entire rooms. To do this, simply install a dedicated application, then scan by moving around the object or in the room. The data collected is processed by a computer to create 3D models.

Which application for Lidar?

Many mobile applications available on the App Store exist for the Lidar including:

  •     3D Scanner App ;
  •     Polycam (one of the best rated applications for liDAR on the iPhone) ;
  •     Canvas ;
  •     Scaniverse ;
  •     Meshroom (Open source software).

In order to analyze the LiDAR scanner designed by Apple, we conducted tests with different types of applications and an iPhone 12 PRO equipped with LiDAR. 

Find here the detailed results of our study. In this study, we were able to deduce that the applications using LiDAR today meet basic needs of 3D reconstruction. However, they do not allow advanced scenarios of 3D acquisition of human morphology. The 3D morphological reconstruction must be very close or even identical to the real thing. Indeed, in concrete application cases, precise measurements are used or morphology is studied.

Analysis of the 3D LiDAR reconstruction

Results of LiDAR technology

The advantage of the LiDAR scanner is its ability to analyze distance and depth, which allows for very accurate depth maps. However, during our study, we identified two points of concern with this sensor:

  1. First, the resolution of the sensor is related to the perspective grid used. This one must be fine enough to ensure a sufficient level of precision.
  2. And secondly, a good quality of the depth map is essential to ensure a good 3D reconstruction.

 In other words, this solution requires the use of high quality components (high costs).

LiDAR Iphone VS LiDAR Scanner

Even if Apple is redoubling its efforts to master LiDAR technology, it still does not match the results of a good quality LiDAR sensor. It is possible that for cost reasons, the Cupertino firm has also chosen to integrate mid-range sensors in the first instance on the current versions of its devices (iPhone 12 Pro, 13 Pro…).

In a nutshell, the rear sensor of the iPhone is a good technology for a BtoC (Business To Consumer) use. It improves the photo or video in augmented reality games. It also allows the creation of 3D scenes. That is to say, to quickly scan a room in a house, or a room in a building to have a visual definition although poorly accurate in terms of details.

The iPhone front sensor VS the iPhone LiDAR sensor

While the rear sensor of the Apple smartphone does not seem to prove its effectiveness in terms of accurate reproduction and scale, this does not seem to be the case with the front sensor (the one used for facial recognition). The security systems of Face ID are so developed on Apple smartphones that it would offer real opportunities for 3D reproduction at scale.

In other words, Apple now has a highly developed LiDAR scanner technology but they are not yet making it available for professional or even medical use. So, discover now an alternative solution to LiDAR for remote 3D acquisition from a smartphone.

An alternative solution to the iPhone LiDAR sensor for 3D morphological reconstruction

What is photogrammetry?

Photogrammetry is a method for creating 3D models by compiling several images. It is based on the principle of stereoscopy, like the human eye (well, both eyes). By analyzing several angles of view of the same point, your brain gives you the vision of relief. Photogrammetry is therefore based on the computer processing of numerous digital photos that combine the measurements taken from different angles.

This method brings new perspectives such as capturing a scene or an object in 3D with a simple photo sensor of a phone! This is what we have developed at MyFit Solutions

Our technology based on photogrammetry, artificial intelligence and augmented reality, allows us to transform any smartphone into a 3D scanner. This technology available as an SDK is finally integrated into a mobile application to meet this need.  

Initially, it is sufficient to perform data acquisition by following guided customer experiences in augmented reality. These experiences are validated to optimize the acquisition and obtain a first time good scan rate close to 100%. Then, an artificial intelligence assembles the photographs and data to digitize a three-dimensional model with the reliefs, textures and colors of the scanned subject.

Iphone LiDAR VS MyFit Solutions

LiDAR and photogrammetry are different data retrieval technologies. The first one sends and receives light beams to create a 3D point cloud while the second one processes and combines hundreds (or even thousands) of photographs to recreate a 3D model. Here are some notable results obtained from our tests on these 2 technologies:

iPhone liDAR

More distant points, therefore a scanner more suitable for large objects

Better surface detection

With the “Time of light” technology the LiDAR sensor has more difficulty to create reconstructions with a real 3 dimensions

MyFit Solutions

More varied and accurate textures for MyFit

A more accurate scan and reconstruction to scale

A real 3 dimensions thanks to photogrammetry coupled with artificial intelligence.

Although some people think that photogrammetry is not sufficient to meet industrial needs, MyFit Solutions has developed an innovative technology to make every smartphone a real 3D scanner. 

 

In fact, production use cases show very interesting KPIs that attest to the robustness of the technology: 

 

  • more than 2000 custom-made products for the human body in 12 months
  • scans received from more than 20 different countries
  • 98% of the scans were good on the first try (via novice users discovering the application)

 

Indeed, we have lifted all technological barriers by coupling photogrammetry with artificial intelligence and augmented reality. As a proof, many uses in production validate the application of our technology.

 

This cocktail of technologies, correctly meshed, allows us to meet the expectations of professionals and to make 3D acquisition possible from cell phones!

To learn more about this innovative technology, contact us.