Relative to banking, finance or healthcare, construction is still a small market for artificial intelligence and emerging applications are focused on finding patterns in large datasets which would either be too difficult for humans to process or would take them too long. Over the course of this article, we aim to answer the following questions:

Where is artificial intelligence being applied in the construction and building sector today?
What are some examples of real-life use-cases for AI in construction?
What are the challenges limiting the adoption of AI and machine learning in construction?

We classify the major applications for artificial intelligence in construction and building under the following categories:

Planning and Design
Safety
Autonomous Equipment
Monitoring and Maintenance

1 – Planning and Design

GenMEP by Building System Planning

Building Information Modeling (BIM) is a 3D model-based process that gives architecture, engineering, and construction (AEC) professionals the insights to efficiently plan, design, construct, and manage buildings and infrastructure. Autodesk Revit is one such BIM software (commonly termed 4D BIM in the construction industry) which allows users to design buildings and internal components in 3D and link time or schedule related information to individual components in the 3D model.

Given below is a short 3 minute video from Autodesk Building Solutions giving a live demonstration of the Revit software and its features:

In order to plan and design the construction of an entire building, the 3D models need to take into account the architecture, structural engineering, mechanical engineering, electrical engineering, and plumbing (MEP) – and the work schedule and timing of these respective teams. The challenge for the construction firms here is to ensure that the different models from each team mesh together and don’t ‘clash’ (which could result in unnecessary reconstruction).

The AEC industry is attempting to leverage ML in the form of generative design (explained below) to identify and mitigate clashes between the different models generated by the individual teams in the planning and design phase itself (as opposed to finding clashes during the construction).

Building System Planning, headquartered in California, has launched GenMEP, a generative design add-on to Autodesk Revit focused on the mechanical, electrical, and plumbing (MEP) design aspects of BIM. For example, once a building 3D model is created on Autodesk Revit the GenMEP addon can automatically design the routing of an electrical system within the building model while taking into account the complexity in different building shapes and geometries and ensuring that that too many cables don’t end up along the same route.

The automatic designing here is achieved through Generative Design in which the software uses machine learning to explores all the possible permutations of a solution, quickly generating design alternatives. It tests and learns from each iteration what works and what doesn’t.

In essence, Building System Planning’s GenMEP leverages machine learning to specifically create 3D models of mechanical, electrical, and plumbing systems while simultaneously making sure that the entire routes for MEP systems do not clash with the building architecture.

Here is a video by Brett Young, Founder and CEO of Building System Planning demonstrating how GenMEP can help in autorouting using generative design:

2 – Safety and Efficiency

BIM 360 IQ by Autodesk

Autodesk launched the BIM 360 Project IQ, which is a product development initiative that claims to use connected data and machine learning to predict and prioritise high-risk issues or project subcontractor risk. However at the time of writing, Project IQ is still currently in pilot and requires BIM 360 Field data (data from existing users of Autodesk’s cloud services platform BIM 360 Field) and is is targeted toward general contractors who are already actively using Autodesk’s BIM 360 construction management software.

Using the BIM 360 Field data gathered from the site in the form of audio, video and construction management documents, Autodesk claims that the Project IQ software can provide actionable insights for the challenges that construction managers and superintendents face everyday in terms of risk-mitigation.

Below is a short 1-minute video from Autodesk introducing their BIM 360 construction management software, with a basic overview of its UX and some of its initial features:

Below is a 2 minute video from Autodesk explaining how the BIM 360 Project IQ can be used in a real-life example for managing subcontractor risk using their dashboard:

One Project IQ pilot has been conducted by Autodesk in partnership with the California-based Swinerton Builders who are using Project IQ to monitor subcontractor performance and enhance on-site construction safety.

Swinerton claims that the software could flag issues, assign the right person to fix it and in some cases also detail how the fix needs to be made. The platform was configured by looking at subcontractors who performed well and finding qualities that they shared that the algorithm could potentially look for in other subcontractors.

Another Project IQ pilot currently underway conducted by Autodesk in partnership with Layton Construction (a Utah-based construction management and general contracting firm) and graphics processing unit (GPU) manufacturer NVIDIA to deploy the BIM 360 Project IQ was aimed at improving construction safety in Layton’s sites.

Compact Assist by Volvo Construction Equipment

Swedish firm Volvo Construction Equipment had launched an operator assist tool called the Compact Assist in 2015, an intelligent compaction system for soil and asphalt compactors.

According to Volvo, the Compact Assist software can automatically map (see figure below) passes that the compactor makes and record temperature maps for the area. Volvo claims to have developed AI algorithms for an additional feature in the software called the Density Direct which can potentially adapt to new asphalt mixes and lifts and automatically calibrate to final density requirements.

AI Applications in Construction and Building 1 — Source: Volvo Construction Equipment

According to Dr. Fares Beainy, machine intelligence program leader at Volvo Construction Equipment:

“AI techniques can help inexperienced machine operators to carry out complex tasks, which they otherwise could not. It’s advantageous in an industry that’s finding it increasingly difficult to source highly skilled and experienced operators”

In the 7-minute video below, Dr. Fares explains how the Density Direct feature in Compact Assist leverages AI:

SmartTag by Smartvid.io

Smartvid.io is an industrial photo and video management platform which includes a ‘SmartTag’ engine that can tag photos and videos of the jobsite automatically by leveraging machine learning, speech and image recognition in order to organize the data and make them easily searchable.

Smartvid.io calls its proprietary machine learning technology the Very Intelligent Neural Network for Insight & Evaluation (VINNIE) and claims that it uses a deep learning model to analyze vision and speech to automatically tag construction data and proactively suggest safety measures for the client.

According to this case study from Smartvid.io and Engineering News Record, VINNIE was used to hold an artificial intelligence for construction safety demonstration. The company claims that that VINNIE was able to weed through 1,080 submissions of construction site photos for the 2016 ENR Photo Competition in under 10 minutes and correctly detect 446 images with people.

VINNIE was pitted against a team of human construction safety specialists for the demonstration. The humans required 4.5 hours for the same task and managed around 414 correct detections. In addition, the company claims that VINNIE was also able to flag 32 images containing personnel missing safety hard hats, and 106 images with workers missing high visibility clothing.

In the 2-minute video below from Smartvid.io about the features and applications where their software can help in the construction sector:

However, it must be noted here that experienced human safety professionals can spot thousands of issues in construction data while VINNIE can currently recognize only a few specific categories of issues and is not intended to replace human workers today. This type of automated field monitoring can potentially aid safety efforts by having an additional pair of “eyes” on the construction site identifying risk factors from data dynamically.

The company also claims to have completed successful projects with construction firms like Skanska, Suffolk and Mortenson Construction among others.

3 – Autonomous Equipment

SMARTCONSTRUCTION by Komatsu

Komatsu, one of Japan’s largest construction and mining equipment manufacturers, announced a partnership with NVIDIA with the aim of developing an AI enhanced safety system for jobsites to build 3D visualizations and track the entire construction site the real-time interaction of people, machinery and objects in the site. NVIDIA founder and CEO Jensen Huang believes the ability of construction machinery to perceive their surroundings and be continuously alert to humans and other construction objects in their surroundings through the use of 3D models will help operators work more efficiently and safely.

In 2015 Komatsu initiated its “SMARTCONSTRUCTION” project aimed at using collected data relevant to jobsite workers and objects improve safety. According to the company the project has been initiated in 4000 worksites in the US alone with plans of it being integrated into more in the future.

According to this press release from NVIDIA, their Jetson AI platform will serve as the ‘brain’ of heavy machinery manufactured by Komatsu building upon the SMARTCONSTRUCTION initiative by the Japanese company started in 2015 aimed at creating connected and smart worksites.

This is an 8 minute video from Komatsu America explaining the integrated support features as a part of their SMARTCONSRUCTION initiative:

Yuichi Iwamoto, senior executive officer, CTO at Komatsu says that the company will use NVIDIA technology to create 3D visualizations of construction sites, including the real-time interaction of people, machinery and objects. The SMARTCONSTRUCTION project also involves collection of data through the use of intelligent cameras and drones.

NVIDIA claims that their Jetson AI platform will enable the cameras mounted on Komatsu’s construction equipment to have 360-degree views of the site with the aim of identifying humans and other construction objects in the vicinity to prevent collisions.

At the time of writing however, the project still is underway and no measurable results for the were available.

4 – Monitoring and Maintenance

Comfy by Building Robotics

Building Robotics, a startup based in Oakland, CA, offers ‘Comfy’, a building automation management application that lets users of commercial spaces and office buildings place centralized requests thermostat temperature. The company claims that their platform can change the thermostat settings in a space on-demand, and have preferences automatically applied over time.

According to Building Robotics and Intel (Intel provides IoT gateways enabling the physical connection of the Building Management System (BMS) to the cloud) the Comfy app can collect data from existing BMS in commercial spaces and also take into account user request data to moderate and optimize temperatures in different parts of the building dynamically with the aim of increasing cost savings and improving energy efficiencies in commercial spaces.

Once Comfy receives a request from a user, it automatically provides 10 minutes of hot or cool air to gauge the discomfort levels based on how many additional requests are received. Over time, the app uses user request data to identify patterns and preferences based on location and time of day.

Here is a short 1-minute marketing video from Building Robotics giving an overview of their product Comfy:

The Building Robotics team provides consultation services before the deployment of the app-based solution for commercial properties and claims that in certain cases, 97% of people in the same zone agreed on temperature settings. Although at the time of writing more detailed information on the training and configuration or veracity of the results was still unavailable.

Doxel AI

Doxel is a silicon valley startup claiming to provide AI-enhanced software focused at improving construction productivity. Doxel uses rugged robots and drones equipped with cameras and LiDAR sensors to monitor and scan worksites.

The company claims that the visual data is processed using deep learning algorithms to measure the currently installed quantities and the rate of production by matching against the desired planning and design parameters for the client. The company claims that their AI platform can also detect errors in the construction by comparing visual data from everyday scans of the jobsite to small scale design models.

Doxel claims that it has had a couple of successful collaborations in the past, and one such example is their project for the Kaiser Permanente Viewridge Medical Office project.

Given below is a short video from Doxel which shows how the company uses rugged robots and drones to track construction productivity.

The 2-minute video below is an explainer from Doxel about how the company can help construction firms:

The company claims that their real-time progress tracking system was able to prompt the Viewridge Medical Office project team to take action when pre-defined schedule deviations were detected eventually yielding a 38% increase in labor productivity across all teams involved in the construction, enabling the project to be completed 11% under budget. It is unclear exactly how much of this savings can be attributed directly to Doxel.

Future Potential for AI in Construction and Building

According to The United States Department of Labor’s Occupational Safety and Health Administration (OSHA) construction sites are generally considered one of the more dangerous workplaces settings due to the presence of heavy equipment and uneven terrain and the fatal injury rate for the construction industry is higher than the US national average for all industries. ‘Struck-by deaths’ in construction which are caused by workers being struck in construction sites by an object, equipment or vehicle have risen 34 percent in the U.S. since 2010.

There seems to be a consensus (for example in the UK, UAE and Canada) in the construction sector that AI can go a long way in addressing the safety risks and inefficiencies. In the near future, with data-gathering cameras on heavy equipment and drone-based terrain visualization for the jobsite, it is expected that AI will enable operators to make informed decisions based on higher awareness levels about their surroundings and also help monitor construction equipment towards improving efficiency.

The major applications for AI in the construction sector seem to congeal around using machine learning for pattern recognition and machine vision for image recognition (as we have come to expect in the early stages of AI adoption in most other industries). Enhancing safety of human workers in construction sites using AI is starting to gain prominence among most large construction firms. Although, in the next five years we expect to see many AI vendors working on improving efficiency of the construction processes from planning to monitoring during actual construction.