How Alteia is leveraging NeRFs in inspection workflows?

Article originally published by Geoweek

Perhaps the most exciting relatively new development in the world of reality capture over the last couple of years has been the development of neural radiance fields, or NeRFs. This technology is similar to photogrammetry in that it takes a series of 2D images and turns them into a 3D model, but uses a neural network to make these creations with significantly fewer images required, generating a radiance field. For the most part, at least from what’s been shared publicly, there haven’t been a ton of in-the-field use cases for this quite yet, which isn’t totally unexpected for a new tool.

That said, it doesn’t mean it’s not being used at all, and one company we have come across who has implemented it at least to some degree into their workflow is Alteia. The France-based company works with operators and owners to provide insights based on visual data using computer vision and artificial intelligence. Earlier this month on their company Linkedin page, they posted about their use of NeRFs and how it has improved some specific workflows with their clients.

To learn more about how they’re using this exciting new technology, Geo Week News spoke with Alteia Chief Product Officer Jean Cesario and Chief Marketing Officer Baptiste Tripard.

Before getting into the NeRF workflow, though, it’s worth taking a second for some quick background about Alteia and their work because, as will be discussed below, in their mind there are still specific use cases for which this technology makes the most sense.

As Jean Cesario tells us, Alteia helps customers in their digital transformation by making sense of any type of visual data, enriched with geospatial and enterprise databases. More specifically, he says, “what we do is we help them realize what we call vision AI to improve the resilience of their infrastructure, increase traceability & compliance, or improve their efficiency. As an example we help them digitize their inspection workflow and therefore contribute to provide more information about their infrastructure and other assets. Thanks to that, they’re able to operate and act more accurately.”

The company was founded in 2018, and according to Jean Cesario, they started using this technology just this year in what was an understandably difficult process.

Jean Cesario tells us that the main challenge was “to change the contractual phase,” as they deal with many large companies with standard contracts in terms of data acquisition and defined workflows in terms of data processing. That leads to a challenge in which they need to have this process implemented and proven before contracts are changed, which is tough to do without clients to use it.

As a result, Alteia ultimately self-funded their early work to provide examples to customers of how this technology can strengthen certain workflows. Working mostly with electrical poles, they would do this work and ask the customers to do their own inventory before comparing the results.

Customers have ultimately seen the return on investment by automating some tedious workflows and opening up experts for more high-value tasks. In our conversation, Cesario is careful to make clear that NeRFs are great for very specific use cases and pieces of full inspection workflows. Baptiste Tripard tells us that one of the biggest benefits of using NeRFs has been gaining accuracy in their ability to identify equipment automatically on complex infrastructure, and how to recognize duplicates between images. Comparing more traditional workflows to what they are starting to use with NeRFs, they’ve seen accuracy in these inventory use cases rise from somewhere around 90 percent in many cases to 98-99 percent.

And, of course, this comes with fewer images needed – Jean Cesario tells us that the example in the LinkedIn post required about 20 images, compared to at least 50 which would be needed for traditional photogrammetry methods. With all of this in mind, it has led to a process which “kind of automated equipment identification because of the high precision achieved,” according to Cesario. He tells us that customers have acknowledged that their “subject matter experts are only there to control the final anomaly diagnosis, and not involved in doing the tedious image equipment detection step.”

In fact, the process of equipment identification is followed by the image Classification at equipment level:

The problem is rescoped to be managed at the equipment level
The model classifies each equipment in a binary manner (ok/ko) according to the anomaly classes defined for that specific equipment
The model makes a prediction that is then validated by the export

The equipment detection step provides value on its own, too. Jean Cesario tells us that one of the major issues with many of their clients is that they simply don’t have an inventory of all of their assets and related equipment, since much of the infrastructure was built 70 years ago. The precision of their workflow allows for that knowledge to come into the customer’s business. Their numbers back this up, too.

That’s not to say that this all comes without any drawbacks, something which both Jean Cesario and Baptiste Tripard acknowledge and why they are considering it as a tool in their toolbox rather than a new, dominant technology.

It is not, at least in its current form, ideal for every use case. Most of their work to this point has been with poles, though Baptiste Tripard mentions that they’ve been contacted about potentially utilizing this with cell tower structures as well. Generally speaking, Jean Cesario tells us that this NeRF workflow is best used on vertical structures, whereas infrastructure like bridges may not really benefit by implementing this more automated workflow.

All of that being said, this is still very much an emerging technology, still in its nascent stages. The usability and functionality of these workflows will improve over time, and should both become even more impactful in existing workflows and open up new possibilities. Speaking about the future, Jean Cesario points to potentially adding Lidar into the mix as Lidar and imagery become more intertwined in other reality capture workflows. He also notes that with the proliferation of smartphones and everyone having quick access to ever-improving cameras, there will be more demand for workflows which don’t require such rigorous image capture and overlap.

For right now, though, Alteia is having early success using it as one tool in part of their infrastructure inspection workflow.