City digital twins: 3D models from a scanning car

The ScanVan project aimed at demonstrating that the Spherical Camera can greatly simplify the production of 3D photogrammetric datasets, allowing computing 3D models of cities on a daily basis. Conventional cameras always have a limited field of view. On the contrary, the ScanVan camera is capable of perceiving in all directions simultaneously. Mounted on a small car, it permits to digitise a city simply by driving once in each of its streets. To produce the 3D model, the consecutive spherical images are automatically compared. The numerous common points visible in the two images are identified and their position in space is calculated by triangulation. The models are then densified, combined, and finally placed back into the geographical space so that they can be superimposed on the real space.

With a traditional scanning approach, one must first collect a set of captured images and submit it to a processing pipeline to detect the areas requiring more details or coverage. This ends up in an iterative digitisation process for building the image data set until the connectivity and coverage targets are met.

The method and technology developed in this project has the advantage that it transforms the experimental photography acquisition strategies into predicable recording photography campaigns. The use of an omnidirectional central camera significantly simplifies the planning of capture strategies to ensure the connectivity of the images. Thus, optimal diachronic scanning can be envisioned and become a reliable operational step in urban management platform.

The development of smart cities and digital urbanism is only effective if continuous physical sensing of the city itself can be achieved. Failure-proof scanning technology is a basic block for Urban Operating Systems and 4D “Mirror Worlds”. In the approach followed by the ScanVan project, the 4D world will emerge as a series of sparse spatiotemporal zones that get progressively connected forming a denser fabric of representations. Continuously scanned cities will constitute the core structure of this 4D skeleton.

In the course of the project, the research teams designed an omnidirectional central camera, an algorithm for geometry estimation that operates directly on spherical images and a densification technique based on the estimated geometry.

The ScanVan camera was designed as a hyperbolic mirror whose shape has been mathematically calculated to reflect light rays from all directions to two high-resolution sensors. The camera thus behaves like a sphere that perceives its environment. All the rays virtually converge on its centre. To design the central mirror of the device, a manufacturing process using diamond turning was used. The double-sided mirror is produced from a single piece of aluminium with an additional coating. The resulting mirror quality is high enough to require no polishing.

The camera was mounted on a vehicle and tested in the city of Sion. An algorithm for geometry estimation that operates directly on spherical images and a densification technique based on the estimated geometry was developed. The relevance of this device for creating large 3D models of urban environments from a sequence of images captured from a single pass of a trajectory was experimentally tested and methods for producing failure-proof digitisation campaigns were designed and patented.

To explore this flow of new data, a web navigation interface has been developed. The project’s navigation interface allows moving fluidly between different scales of 3D representations and see the spherical images acquired by the ScanVan. When possible, it also includes the option to go back in time when older 3D models are available. It is thus possible to navigate in four dimensions.

A privacy-by-design approach was followed. Algorithms were programmed to erase from the captured spherical images all aspects of reality that could identify people or vehicles. Additionally, an annotation interface makes it easy for anyone to point them out and request their removal. The aim is to make the 4D world thus constructed a public space that respects privacy.

ScanVan - A distributed 3D digitalization platform for cities