Clearing the mist that's obscuring point clouds

We believe that this requires, in addition to the changes in ‘traditional’ rail infrastructure, a digital transformation. Simply put, a shift from steel and concrete to bits and data. The extensive use of data by all parties increases efficiency, saves costs, and, particularly relevant for the Asset Management area, enables the shift to predict and prevent.

To obtain the ‘digital twin’ of the rail infrastructure as it is, the acquisition and use of geo-data is one of the major developments in this area. In particular, the geometrical quality and resolution of photos and point clouds enables us to not only take the snap-shot of the current railway infrastructure, but also to take a step further on the road to predictive maintenance.

The concept of photos and point clouds

Before 2017, we used airplanes to take photos of the infrastructure, but in 2017 we started with a new measuring system. A helicopter flies 190 meters above the ground and takes photographs and executes laser scans. After flying above a calibration area, the helicopter handles specific areas (hundreds of km’s of infrastructure) to obtain photos, stereo models, and point clouds.

The point cloud is a 3D visualisation of the infrastructure, see picture below with an example of Rotterdam Central Station. We mainly use helicopters for this (airborne laser scanning), but also trains (mobile laser scanning) and sometimes we work from one specific point (terrestrial laser scanning). The advantage of airborne scanning is evident: a large area around the railway track can be visualised in one go. ProRail uses mobile scanning (from trains) already since 2010.

The point clouds do not outrank the photographs! It’s the combination of both that enriches the point clouds (with colour and reflection values) and makes the data set complete (e.g. with GPS-data for photos).

Technological opportunities

The use of point clouds and stereo photos implies some major technological tasks. For one, the amount of data is huge. A typical helicopter flight delivers hundreds of terabyte of data. How do you store, process, and search in these huge heterogeneous data collections? And how do you keep track of the data quality? Simple, straight-forward data storage is not sufficient, since searches in different layers should be provided.

Another area of challenge is analysis of the data and the object recognition. Advanced algorithms are required to visually discern between objects in an efficient way. Point clouds contain a huge collection of points, affecting processing power and memory requirements. For web-based 3D applications, this leads to very slow visualisations.

ProRail is an end-user company and therefore not primary focused on solving the above issues. We benefit from market developments that make collecting large data sets more easy, and increase computer power to run complex algorithms. But of course, we have to define our requirements and make choices about how to make use of these market developments.

Examples of point cloud and photo usages for asset management

We see three main areas for the use photos and point clouds. Firstly, as mentioned before, we make the digital twin of the railway infrastructure. The twin creates a kind of digital copy of the objects outside in our databases. This is very useful for the registration of our assets, incidents, track history, and so on. The challenge here is to have no differences between the databases and the actual railway infrastructure. Flying over the same location several times a year, especially in areas with construction projects, makes the database up-to-date and simplifies comparisons between To-Build and As-Built data.

Even better, by combining several geo-related datasets we create a unique 3D image of the whole railway network (including stations). This proves very useful for training train operations, simulations, and so on.

Secondly, we see opportunities for dealing with incidents. For example, a drone can take photos and create point clouds of a derailment, shortly after the incident. Sharing these data with the emergency services enables joint decision making, reduces misunderstandings, and ultimately contributes to faster recovery. Even better, with point clouds we analyse potential clash-detections and clearance profiles to prevent incidents. The picture below shows an example of a red spot, based on automatic clash detection.

Thirdly, and maybe the most promising area, we improve our predict and prevent capabilities. We explain this in more detail below using our modelling steps.

1. The first recording is a registration of the As-is infrastructure
2. The second recording detects changes compared to the first recording
3. The third recording defines a trend model
4. The fourth (and further) recording validates and improves the trend model

The trend model can be used for predictions and thus for preventive measures. Some examples:

• Availability of the railway tracks benefits from proper maintenance of bushes, trees, plants. Photos and point clouds enable so-called ‘Tree management’. In the first recording, we register trees (heights, width, type). The second recording shows growth, the third shows trends and enables the prediction of, for instance, pruning moments.
• The absolute position of a track (horizontal, as well as vertical) can be monitoring by analysing the first and second recordings. The third recording incorporates movements of the track in a model, which predicts future movements. If thresholds are exceeded, predictive maintenance is in place.
• Inspection of bridges, tunnels and crossovers benefit from the same approach. In fact, the first two steps are sufficient to, for example, monitor the development of rust on steel bridges.

Future developments

At ProRail, we feel we are just at the start of using photos and point clouds for predictive maintenance and incident management. The possibilities are almost limitless. We can only imagine what we can find when we combine the point clouds with other data-sets we already have, e.g. the ultrasonic measurements to detect cracks in the track. But in all cases, it can only have impact if we really implement the predictive models in our operational maintenance processes. This is clearly a joint effort with assetmanagement partners. We passed phases of ‘talking’ and ‘testing’, it’s time for implementation!