Q&A: How augmented reality could change offshore training with ABB

Matthew Farmer 26 May 2020 (Last Updated May 22nd, 2020 12:08)

Augmented reality could replace paper procedures, change training methods, and give detailed advice in real-time. But how can engineers avoid ‘data paralysis’, and could such technology one day replace workers on site? ABB Product Portfolio Manager Chris Naunheimer discusses the ins and outs of their augmented reality procedure monitoring technology.

Q&A: How augmented reality could change offshore training with ABB
The Microsoft HoloLens tool works similar to the famous Google Glass. Credit: ABB

In the last oil price slump, operators took advantage of the situation by improving their technologies, performing necessary maintenance, and making offshore work more efficient. This time around, the need to keep staff numbers low has meant large offshore works are inconceivable to any employer hoping to look after their staff.

But technology is still progressing through the coronavirus downturn, with some companies using it to spur innovation. ABB Group has launched a procedure monitoring technology using augmented reality (AR) to show workers how to perform processes via a tablet computer or HoloLens glasses.

The company has tested this product in refineries and chemical production, with mining and paper companies expressing some interest. Offshore, an AR aid could give workers a virtual overlay to the real world, giving them information on their work as they carry it out.

Chris Naunheimer, product portfolio manager at ABB, explains his belief that the system is the future of training and auditing procedures offshore.

Matt Farmer (MF): For starters, how would you describe the technology you have developed?

Chris Naunheimer (CN): At its core, augmented field procedures are designed to replace paper based procedures that are currently being used in almost every plant manufacturing facility in the world. These may be a single piece of paper that needs to change hands a few times, it may be something where you need feedback from somebody else in the plant during the course of completing the procedure.

It’s a way of doing work that in many industries has gone away but in our industry has not. So what augmented field procedures does is replace the need for those paper based procedures. It’s not a whole lot more complicated than that.

Another use case though is if you have many sites where the same procedures are being done slightly differently across each site. This can help you standardise that procedure across the board and see where variabilities are. You minimise possibility or opportunity for error and overall improve the efficiency of getting things done.

MF: Do you see the best application of AR technology as being for use in procedural work or for training?

CN: There could be opportunities to use something like this as an educational tool for sure.

But there’s also the opportunity of, you know, routine and repetitive work becoming so common that you forget, or don’t even necessarily think about what you’re doing. Bring this back to some of my user experience and research days. When you ask someone ‘How did you complete that procedure?’ or ‘How did you empty that tank?’, they’ll tell you the procedure. They’ll tell you what they thought they did. But then when you say, ‘Hey, you know what, can you show me what you did?’, you get a very different picture of what’s actually happening.

So, to your point, I think it’s good to have as a training tool, for people doing something for the first time. But I also think there’s value in something like this, for people that have done things over and over and over again, right, because procedures change, but people are very slow to change.

If you can take that knowledge from one site and quickly apply it to a dozen sites, without having to go through all kinds of change in management process… It just shows up on their tablet, they get the update, and they have a new process and procedure to follow. They don’t have to go through a training session, they can quickly adapt to change.

MF: You have also said the technology can help with data handling. What did you mean by that?

CN: Let’s say you’re reloading catalyst on a platform. You need to have data available to you out in the field that is only available in the control room. You need to know where the process is at, what are the conditions, is the pressure stable? Maybe the pressure reading is not accessible to where you actually physically are, but you can get a reading from the control room.

If you have something like augmented field procedures, the person in that control room has already gone through the procedure to check if pressure was low enough to add catalysts to it safely. That’s already been completed. You don’t actually have to go try to find that piece of data, it’s available to you in real time. You can stay synchronised with the control system.

MF: After acquiring the technology, how much time and knowledge will users need to take full advantage of it? Will they actually need training to use it as a training aid?

CN: The way the product is built is that end users can basically build the procedures into the software. Users can build those procedures without using code. It’s sort of a drag and drop, no code, procedure-builder-type platform, so they can build those models on their own, whatever their procedure requires.

At the end of the day, if a customer is willing to pay for an engineer to set this up for them, then we’d be happy to set it up for them. But, again, there’s a value in having the customer set it up because they know the procedures, they know the processes, they know what they’re trying to accomplish.

One of the benefits that we promote with our system is the seamless integration with any control system, not just an ABB control system.

MF: Could the availability of data on these procedures lead to over-analysis, where operators spend as much time scrutinising the work as doing it?

CN: In my personal opinion, I’m sure there’s a point where there’s analysis paralysis, where you just spend so much time looking at the data that maybe you’re not taking advantage of the efficiencies. I feel like we’re so far away from that, because we’re not looking at it today at all. I think it’s a long way to get to that point where we’d be spending more time on doing the data analysis than we would be on actually completing the task.

MF: Would this then mean there would be less need for skilled workers on site?

CN: Maybe seeing it as less-skilled workers is not necessarily the right thing. But I think… bringing in workers that can come up to speed quicker. The technology enables many people to complete their job quicker, to be more effective, to apply the skills that they have more effectively.

So there may be less reliance on apprenticeship as the means of learning, which is traditionally the way that things get taught in the refining, petrochemical, oil and gas field.

MF: In what ways would AR improve safety?

CN: Two or three points on health and safety: the procedures are consistent and safe, right? AR means people have the most relevant, up to date procedure that’s accessible. You know that it’s not like they grabbed an old version.

Also, there’s an audit trail. So we can go back and look and see what was done, see what wasn’t done. See what steps were maybe bypassed or what steps were overlooked. Or we can compare times and say ‘Wow, that only took 15 minutes this time when it normally took 30. Somebody skipped some steps, right?

MF: This technology is still in testing, and has a lot of room for growth. What technologies are going to push AR products forward in the future?

CN: The way it’s set up now, if you go up to a piece of machinery or equipment, you scan barcodes, and that shows you that you’re at the right place. Over time, that could be a visual representation of that actual piece of equipment. Where it knows that you’re at the right piece of equipment, just by the way it looks. I don’t think we have that technology in place yet. But we can get beyond just the QR code scanning into visuals.

MF: On that note, could this tie into the Internet of Things?

CN: That would be so cool. It would be cool if the valve could talk. Not just hear from a chat bot, like Sam was talking about, but actually ‘hear from the valve’, and it tells you what state it’s in and then the procedure updates in real time. This is pie-in-the-sky stuff where the valve tells you ‘Hey, you know what, the last time I was worked on, they forgot to do this thing’.