Innovations in AR: Heavy Industry

Augmented reality (AR) is a key pillar of Industry 4.0 (or the fourth industrial revolution), side-by-side with other potentially transformative technologies like machine learning and big data. Indeed, consultancy firm PwC has estimated that industrial manufacturing and design is one of the biggest areas for augmented and virtual reality (VR), with their use in heavy industry having the potential to deliver a $360bn GDP boost by 2030.

In this latest edition of our series on how augmented reality is faring across a range of industries, we’ll be taking a closer look at why AR is proving so useful in heavy industry, in particular the fields of construction, manufacturing and energy.

Construction

AR is proving to be a key tool for the construction industry, whether in the design stage or actually in the construction process itself, leading a 2020 study of the architecture, engineering, and construction (AEC) industry to say that AR and VR would see “strong growth” in the next 5 to 10 years.

On the design side, numerous architectural tools exist to help with space visualisation using augmented reality. One such example is The Wild, which allows designers to view 3D models in both virtual and augmented reality. Such tools can layer virtual details onto a building plan so that plans can be more readily understood by stakeholders. 

That requires highly detailed and accurate 3D models, which is where the technology overlaps with digital twin technology. Using those digital twins, companies like Akular can enable clients to see what a building would look like on-site in the real world before it is built via a mobile application.

When it comes to actual construction, augmented reality again finds a number of uses, not least training workers on safety. That might involve AR headsets that interact with tags on potentially dangerous areas to bring up safety information, but even before workers are on-site, AR can help with training them on how to use heavy machinery – as with the construction equipment training simulators offered by CM Labs or the Arvizio AR Instructor.

Arvizio AR Instructor

“Industries are experiencing a shortage of skilled front-line employees and view augmented reality as a means to accelerate training and efficiently transfer the expertise of experienced workers,” said Jonathan Reeves, CEO of Arvizio. “Arvizio enables organizations to rapidly upskill employees without the need for on-site coaching and mentoring. By delivering no-code authored augmented reality instruction and remote expert connectivity, AR Instructor can substantially increase productivity and reduce errors of workers performing complex operational activities.”

Meanwhile, progress capture and tracking functionality directly compares real-world sites with virtual models to ensure they aren’t deviating – all in real-time. A host of companies provide variations on that technology such as VisualLive, which enables users to witness 3D models in real life via headsets such as the Microsoft HoloLens or mobile devices.

Manufacturing

Much of the technology we’ve covered for construction can equally apply to the manufacturing industry, whether that’s learning how to use dangerous equipment or visualising the layout of equipment and machinery in a factory. None of this is to say there aren’t plenty of bespoke uses for augmented reality in the manufacturing space, however.

One early pioneer was Volkswagen, which was using augmented reality to assist service workers way back in 2013. The MARTA app showed step-by-step instructions on how to repair and replace certain components, overlaying its advice on the car via an iPad app. Along similar lines is Boeing’s more recent use of augmented reality to give technicians real-time, hands-free, interactive 3D wiring diagrams. 

Interestingly, that technology has bled over into the consumer space with AR manuals that assist car-owners with basic maintenance operations by showing precisely where components are located within a car.

In the design space, AR has been deployed by the largest manufacturers to rapidly iterate and do away with expensive and time-consuming physical prototypes. In the case of Ford and its partnership with HoloLens, changes can be made to a design and reflected in real-time to collaboratively sculpt a new vehicle.

AR has been trusted at the very highest levels of manufacturing, too. Lockheed Martin utilised augmented reality in the creation of NASA’s Orion Spacecraft, overlaying information to help with mission-critical procedures such as precisely aligning fasteners.

Nasa Orion HoloLens

Energy

In the energy sector, AR has the potential to remedy significant problems faced by the industry, chief of which is a brain drain caused by an ageing workforce. Indeed, the US Department of Labor estimated in 2019 that 50% of the current energy utility workforce will retire within the next ten years. The institutional knowledge being lost could be replenished more quickly with the help of AR technology.

Shell is duly using the remote collaboration possibilities of AR to educate workers in the field. Expert consultants are able to see through a worker’s eyes via an AR headset, and even draw on the screen of the augmented reality display they are using. That increases safety as workers interact with potentially dangerous heavy oil and gas equipment, as well as allowing experienced but ageing employees the ability to work remotely.

Shell AR
Image credit: Shell

The energy sector is no slouch when it comes to more specific AR solutions either, such as Upskills’s Skylight platform which allows companies to more easily develop bespoke augmented reality apps for use with AR devices, ranging from Google Glass to Microsoft HoloLens 2 and mobile devices. Then there are solutions such as Adroit, which can provide guidance on repairing high-stakes equipment such as oil rigs by scanning and identifying faulty components and machinery.

Final Thoughts

In heavy industry, where the costs of prototyping are enormous and the potential risks from machinery are significant, leaning on the virtual possibilities of augmented reality is common sense – hence the interest in the technology from across the sector.

To find out more about how AR is progressing in other fields, read the previous entry in the series, where we explored the healthcare industry in particular.

Make Real & Kelbray Partner on Immersive Construction Training Solutions

Make Real header

British XR specialist Make Real has a broad product portfolio ranging from training solutions for traffic marshalls to consumer videogames like Loco Dojo. This week the team has announced a new partnership with Kelbray, a civil engineering and construction services company to create a range of virtual and augmented reality (VR/AR) applications.

Make Real

The partnership sees both companies form MRKL LLP as they look to build a range of products catering for the construction industry. Initially, these will utilise Make Real’s SkillShield training application support tool which includes features like timestamped evidence of completed training and native Oculus Quest support.

The training applications MRKL LLP has planned aim to enhance and improve offerings throughout the industry, offered as off-the-shelf licensed packages to customers. So that means software, hardware, device management solutions and training, allowing companies without any XR knowledge to put these products to use.

“After working closely with the Keltbray XR team for the past three years on internal applications, we felt it was the right time to form the official partnership, to co-create and offer validated immersive technology training applications to the wider construction sector. These initial products have been in development over the past year with internal user testing and feedback, until we all agreed that they were ready to go to market. That time is now,” said Ben Dykes, Client Services Director, Make Real Ltd, in a statement.

Make Real

“We are delighted to have established our formal partnership with Make Real after a number of years successfully working with Keltbray Extended Reality, led by Dave Rowe to create applications which push the boundaries of what is possible with Mixed and Virtual Reality in the construction industry,” adds Holly Price, Group Skills and Communities Director, Keltbray.

“Our SkillShield platform provides the missing link in virtual reality training, creating indelible records connecting the trainee irrefutably to the evidence of their training. Added to this, direct access to Keltbray Lifting Services, led by Steve Leyton, has enabled us to produce industry-leading immersive training content that will help transform the industry’s approach to people development, and bridge the growing skills gap in our industry.”

Make Real’s training products already include Slinger SignallerTraffic Marshall and Rear Guard. As further projects are added to its portfolio, VRFocus will keep you updated.

Resolve On Oculus Quest Allows Teams To Review Huge BIM Files In VR

A new meeting and collaboration app, Resolve, is available in Early Access for the Oculus Quest. The app is aimed at design and construction teams, allowing them to review models and designs on their projects while together in VR.

Resolve’s the big draw is it’s ability to display huge BIM (Building Information Modeling) files on the Oculus Quest system. The developers at InsiteVR suggest that other Quest apps offering similar functionality will “run at low frame-rates or even crash when handling similar model sizes.” Resolve can open any BIM file hosted on Autodesk BIM 360 Docs, with support for over 70 different file formats. According to tests run by InsiteVR, the app can handle up to 500 million polygons in one model.

To get a better idea of the app’s offering and what the experience looks like, you can check out the trailer embedded below.

Interestingly, Resolve is available exclusively for the Quest platform, with no PC VR equivalent. There is a PC companion app, but it’s only for non-VR participation if a team member doesn’t have access to a Quest but still wants to join a meeting.

Resolve is the latest of a few new social collaboration apps for enterprise appearing on the Oculus Quest system. In light of the ongoing global pandemic, many apps, like Spatial or Spaces, are offering solutions for better remote collaboration at a time where teams are stuck working from home.

InsiteVR is encouraging design and construction teams with access to Oculus Quest headsets to get in touch and request Early Access through the Resolve site.

The post Resolve On Oculus Quest Allows Teams To Review Huge BIM Files In VR appeared first on UploadVR.

Report Shows How Digital Technology is Transforming Construction

A report from the world Economic Forum has shown that digital technology including 3D printing, augmented reality (AR) and virtual reality (VR) could save the construction industry up to $1.7 trillion (USD) globally within 10 years.

The report was compiled y a close collaboration between the Consolidated Contractors Company, Boston Consulting Group and the World Economic Forum. The report highlights new digital technologies that are disrupting a range of industries, and posits that stakeholders can no longer afford to ignore the changes brought about by these new technologies.

Areas that were pinpointed by the report include business information modelling, 3D printing wireless sensors, digital range-finding and autonomous equipment. There is also note made of the impact of robotics, artificial intelligence and cloud technology.

The Member of the Steering Committee for the Future of Construction Initiative, Consolidated Contractors Company Manager M.I.S. & Business Processes Re-engineering, Aref Boualwan, said: “The three futuristic scenarios we identified in the report are extreme, but conceivable – they analyze how multiple current megatrends could establish different versions of a future world. The scenarios – ‘Building in a virtual world’, ‘Factories run the world’ and ‘A green reboot’ are not designed to predict the future – but to help us prepare for emerging trends.”

Boualwan further commented that the pace of change in this area has been very rapid in recent years: “Initiatives like wireless equipment, cloud & real-time collaboration, 3D scanning/printing and augmented reality and visualization. We have seen digital technologies completely transform global industries in recent years – from social media to e-commerce and digital based mobility companies. But only very recently have digital technologies begun to emerge in the engineering and construction industry. This is the way of the future – and this report will help companies across the industry adequately prepare for it.”

Further information on the report can be found on the World Economic Forum website. For future coverage of news and developments from the VR and AR industry, keep checking back with VRFocus.

Liebherr Bring Virtual Reality To Their Crane Planning Software

The Liebherr Group, one of the largest equipment manufacturers based in Switzerland, has released an update to their lift Crane Planner software which now allows users wearing a virtual reality (VR) headset to virtually walk around the application.

Liebherr Crane Planner

Released as part of the Crane Planner 2.0 release, the new VR mode offers more opportunists then ever before for users to improve their workflow. By taking detailed information and CAD data to build the 3D environment and models, users can now experience them first hand by exploring the detailed models themselves, within the application and virtual space. This offers user a chance to see how the crane set up will look on site, aiding in surveying the work and evaluating possible risks.

One of the more powerful features of Crane Planner 2.0 is how every movement of the cranes and equipment generates planning data which can be used on site. This means that all the needed movements on the job site can be performed within the application and then transfers to the real-world. Of course, with the option to now work in VR, users will have the chance to work from an overhead view point, walking around the model as though it was on a tabletop or switch to a 1:1 scale mode. In both of these models the user will be able to interact with the equipment, generating data on every movement.

By becoming fulled immersed in the virtual environment, there are now more ways for users to view, experience and work. Though many of the benefits including the freedom to work from different angles it also offers many more when it comes to surveying health and safety. If a crane is needed to work in a small space, being able to see it up close and personal will assist in ensuring no collisions or injures occur. Crane Planner 2.0 also has accurate load moment limiter (LML) data for the cranes to ensure that they react accordingly to the weight they are limited to move.

VRFocus will be sure to bring you all the latest from Liebherr in the future so keep reading for more.

AR Startup RealWear Raises $17 Million CAD After Moving To Vancouver

Virtual reality (VR) and augmented reality (AR) startups are getting more support than ever before from financiers and investors, and the industry’s worth just keeps getting bigger. In 2017 we saw investments in VR and AR technologies break records, and with several investments in companies already this year, it’s possible that 2018 might be the biggest year for VR and AR technology and development than ever before.

Now, yet another startup in the AR space has secured the necessary funding they need to continue operation and their ventures into new AR technologies and products. RealWare recently moved from their headquarters from Silicon Valley to a site in Vancouver, and has since raised $17 million CAD.

Oregon Live reports that the company hopes to expand to up to 100 employees by the end of the year, and despite the large amount of money invested thus far, RealWare are still looking for another $3 million to add to the pot.

RealWear create wearable specialised clothing for industry purposes. Waterproof, dust tight, drop proof and rugged pieces of equipment that incorporate AR technology, such as building site helmets with AR visors that allow users to view information and instructions while on the job, such as product maintenance, assembly and inspection.

The RealWear website says it’s technology has the “power of a tablet in a wearable form factor” and it’s the kind of technology that can really help busy employees working on the ground. The technology’s software uses Android OS, making it simple to download and APK file and installing it on an Android device.

When RealWear announced their Vancouver site, they said they would hire at least 20 people for the site, which is based near Fort Vancouver.

The eagerness for investors to spend money on RealWear makes sense, they offer a unique product that’s sure to only become more mainstream in industry use cases as time goes on. We’ve already seen how VR can revolutionise and change the construction industry, with designers and architects evangelising the new technologies and their potential uses. As we get more news on the latest industry uses for VR and AR technology, you’ll read about it on VRFocus.

VR and AR Could Revolutionize Construction, But There Are Still Big Challenges

Paracosm 3D reality capture

The global construction market will hit $10.3 trillion in 2020, and it’s a field where mistakes are costly: rework typically makes up 12 percent of project costs, reaching tens of millions of dollars on big projects. But augmented reality can change that, which is why construction is one of the first industries starting to explore the use of AR in day-to-day work.

For a summary on the state of AR in construction and what’s coming next, I talked to Amir Rubin, founder and CEO of Paracosm. Paracosm offers mobile reality capture for construction teams, and they’ve built both the hardware and the software to power it. (Full disclosure that I’m an investor in Paracosm through my role at the venture capital firm Accomplice).

Big into BIMs

Architecture, engineering, and construction professionals use Building Information Modeling (BIM) to make 3D designs of projects. These BIM models are perfect representations of what the finished project should be. Popular software tools for making BIM models include Autodesk Revit, Graphisoft ARCHICAD, and Bentley Microstation.

As construction teams work on the project, comparing the physical work site to the BIM model is a powerful method for catching discrepancies between the two.

Today, construction professionals use laser tripod scanners to capture the project site, later comparing the data to the BIM model. Faro and Leica Geosystems offer two of the most popular scanners. Although they’re powerful and precise (down to 1mm or better), these systems are slow: they take 2-4 hours to scan a 20k square foot site, then another few hours in the office to post-process the work site data captured. It adds up to about a half to a whole day to scan a small or medium site, plus a day or two of processing. It’s expensive and time-consuming enough to make it pretty infrequent.

The SLAM revolution

One of the most important advances bringing AR into construction projects is Simultaneous Localization And Mapping, or SLAM. SLAM originated in robotics to allow autonomous vehicles to move through environments they’re encountering for the first time. SLAM systems use sensor data, usually through lidar, to track the vehicle’s location and map the space around it. Obviously this process has to happen fast, or else a car moving quickly will be in trouble if it can’t pull in the world and the objects around it. That’s the benefit of SLAM: it works in real time and doesn’t need GPS.

Applied to construction, SLAM lets workers walk through a project site and map it in extreme detail in minutes. This 3D reality capture shows exactly what’s happening in the project site, down to about 2cm to 5mm of accuracy. Note that it’s not as accurate as the tripod system’s 1mm, but they’re slow: tripod scans take days; SLAM scans happen in real time.

To understand why SLAM could be such a big deal for construction, let’s look at a future day on the job site:

  1. A worker walks through the site holding a lidar sensor. This only takes a few minutes. As she’s walking, she’s creating a highly accurate map of everything on the site that’s uploaded to the cloud.
  2. Software in the cloud compares the map she just made of the site to the BIM model, thus comparing the physical reality of the project to date to the desired outcome in the plan.
  3. Any mistakes on the site that aren’t in the BIM model stand out, letting crews fix them early.
  4. The software also tracks completion of the project, keeping teams up to date on progress, expectations, and costs.
  5. Repeat steps 1-4 daily or weekly as the project continues, getting a much better view of how it’s going.

Meet Paracosm

This future workflow is Paracosm’s vision. The team started in robotics, spending years building their own 3D mapping software for that field, before realizing that their biggest opportunity was in construction. They then developed their own hardware to pair with their software, employing lidar (which measures distance to an object using laser lights). Lidar has gotten much cheaper and more powerful in the past few years because of an explosion in use and development around autonomous vehicles and computer vision.

Today, workers can use Paracosm’s scanning hardware to walk through a site and capture accurate 3D data, but eventually, Paracosm wants to automate this scanning with ground robots and then drones. After scanning, Paracosm automatically aligns the coordinate system of the captured data to the project’s BIM model. Once aligned, they can run comparisons with engines like Skur. Then Paracosm’s visualizer provides analytics and insights into how the project is going, whether it’s on spec, and where potential issues could be.

Paracosm’s goal today is to make scanning so easy and effortless that crews can do it every day, gaining a tight feedback loop around quality control. Other players are targeting similar uses in the space, like Shapetrace (which uses off the shelf hardware to overlay drawings, BIMs, and models to reduce errors), and GeoSLAM (which makes both hardware and software for capturing and analyzing survey data).

What’s on the horizon

The architecture, engineering, and construction fields are likely pioneers of AR. Whereas consumer adoption of AR depends on enough people buying headsets and hardware, construction already has lots of interesting and useful applications for it—and the money to invest in its development. We’ve seen some encouraging projects so far.

In June 2016, Microsoft announced a partnership with Trimble, which provides positional technology and 3D modeling, and AECOM, one of the world’s largest engineering firms, to bring HoloLens AR to their projects. The main uses of AR here include holographic displays of BIM models, collaboration among team members, and immersion in construction plans.

In a widely-shared proof of concept in August 2016, Scope AR partnered with the drywall subcontractor Martin Bros. to build a bathroom pod using only AR on the HoloLens. The experiment showed that it’s possible to build indoors in a controlled environment without physical building plans or even a tape measure. Instead, workers use a 3D model floating in AR to place each piece.

Hackathons focused on VR and AR in construction, like AEC Hackathon, are gaining ground with backing from big companies like Autodesk and events throughout the year across the country.

A few early-stage companies offer tools that let users view BIMs in VR, like Iris and VIMtrek, and help designers visualize their projects in more depth than ever before. Once work begins, companies like Scope AR and Daqri allow workers to see and give instructions in AR glasses/helmets or on mobile. These systems are still very new and don’t have widespread usage yet.

Other companies target real estate and interior design in VR, focusing more on finished spaces than those undergoing construction. Floored lets users virtually build out and view commercial real estate spaces on top of floor plans. Matterport uses its own 3D camera to capture real estate interiors, which can be used in VR tours or converted to digital models. Navvis is also capturing indoor spaces, letting users scan areas to digitize them, map them, or even convert them into BIM files.

What’s holding us back?

Ultimately, construction professionals want a scan so accurate that workers can walk through spaces with an AR headset and view the BIM model on top of them, seeing how progress on top of the site itself. They’d be able to use AR/VR visualization to bridge the gap between the BIM model and the actual building being constructed on the ground.

That’s still a ways out, says Rubin, because tracking accuracy is not great enough yet to do the overlay of the BIM in AR. To reach a point where viewing the BIM model on the work site isn’t just a novelty, it needs to be overlayed within an eighth of an inch of accuracy. Otherwise AR elements won’t line up with physical ones, and things will look off.


If you’re in construction, pulling AR and VR into your workflow could be your competitive differentiator. We’re still a few years away from the most powerful use cases, but there are already plenty of tools today that expand your toolkit and cut costs.

Tagged with: , , , , ,