Using Work-Order Data to Extract Building Performance Metrics

From ASHRAE Journal Newsletter, May 12, 2020

Performance metrics are necessary to understand how operational resources are used in large commercial and institutional buildings. Work-order data in computerized maintenance management systems (CMMS) can be used to obtain these performance metrics.

In a recent Science and Technology for the Built Environment article a method to conduct text analytics on CMMS is developed and demonstrated through a case study in which four years’ worth of data from four large commercial buildings are used.

Researchers Burak Gunay, Ph.D., Associate Member ASHRAE; Scott Bucking, Ph.D., Associate Member ASHRAE; and Saptak Dutta, Student Member ASHRAE, discuss the paper.

1. What is the significance of this research?

Operational performance is a multi-objective construct involving occupant comfort and health, tenant satisfaction, workplace productivity, operational cost savings and energy efficiency. Operational performance metrics act as proxies for building performance and allow operators and facility and energy managers to continuously monitor and conduct ongoing commissioning in building systems. Due to the challenges associated with accessing high-quality operations data, performance monitoring in existing buildings rarely goes beyond crude metrics such as the annual energy use intensity.

Work-order data stored within computerized maintenance management systems (CMMS) represent a resource that can be utilized to extract such performance metrics. While buildings are commonly benchmarked on the basis of energy use metrics, it is rare to find studies where tenant complaints and operator feedback data are used to developing performance metrics. This has traditionally been because of a lack of widely available tools to carry out data and text mining on CMMS databases.

In this paper, a method to conduct text analytics on CMMS data is developed and demonstrated through a case study in which four years’ worth of data from four large commercial buildings are used. Association rule mining technique is employed to identify building, system, and component-level recurring work-order taxonomies and common failure modes. The results highlight the potential of kernel density functions, decision trees, Sankey diagrams, survival curves and stacked line plots to effectively visualize the temporal, spatial and categorical anomalies in the complaint patterns.

2. Why is it important to explore this topic now?

Buildings play a major role in our environmental footprint and economy. While much attention goes into the construction phase of buildings, the operation phase is estimated to account for ten times more embodied energy than the former. It is, therefore, imperative to develop metrics to benchmark operational performance. Traditional benchmarking techniques depend on simplistic energy-use metrics and demonstrate a high-level overview of building performance while building audits derive high-resolution operational performance metrics, which are time-consuming and expensive to conduct.

Better methods for characterizing high-resolution metrics are needed to analyze large amounts of data that are collected from commercial buildings today. Low cost and high-resolution benchmarks can be developed using currently underutilized data streams such as CMMS and tenant survey data in order to convey detailed performance metrics to building operations and maintenance staff.

3. What lessons, facts and/or guidance can an engineer working in the field take away from this research?

Our research demonstrated that complaint logs and operator comments stored within CMMS can be used to extract operational performance metrics in large commercial buildings. Because many large commercial buildings already utilize a CMMS, it is considered a high-value source of data, as there are no additional costs (e.g., sensors, meters) associated with insight extraction. Despite the data contained within these systems, they typically remain underutilized in building operations and maintenance due to difficulties in extraction, analysis and interpretation.

A detailed categorical and spatial breakdown of the analysed data shows that tenant complaints are disproportionately concentrated around only a few floors of a building. Furthermore, analysis of the floor-level complaints shows that only certain complaint categories contribute to a majority of the recorded complaints. Visualizing the shape of a 24-hour probability distribution curve of “too hot” complaints consistently displays a secondary complaint peak at 2:00 p.m. A visualization of the average indoor temperature over 24 hours shows a maximum indoor temperature on average during the same time period. This suggests that the studied buildings could be overheating due to façades that have very large window-to-wall ratios. A decision tree trained with outdoor temperature and temporal factors (weekdays/weekends, mornings/evenings, seasons, etc.) indicates that the studied building has a disproportionate number of cold complaints during summer days where outdoor temperature is below 10°C (50°F). This is a possible indication of a pre-defined, static schedule being used for building controls and over-cooling taking place. Text mining operator logs stored within the CMMS reveal the key operational workflows that building operators perform and expose the top categories of failures that commonly occur within a building, i.e., failed valves, malfunctioning pumps, burnt ballasts, etc.

An engineer working in this field can study these results and potentially replicate them with available CMMS to examine disproportionate complaint distributions, high levels of solar heat gain, efficient building controls scheduling opportunities and top operator workflows and component failures.

4. How can this research further the industry’s knowledge on this topic?

Our research adds value to the current industry understanding of building performance, operational workflows and tenant comfort by developing performance metrics extracted from a CMMS. This adds to the traditional techniques of building benchmarking using metrics.

While traditional methods of benchmarking usually offer insight into energy use, the developed performance metrics allow building operators insight into complaint logging and resolution-related metrics. These include complaint resolution time, complaint distribution shape over a day, complaint probability due to temporal and outdoor thermal variations and categorical/spatial complaint distributions.

5. Were there any surprises or unforeseen challenges for you when preparing this research?

During this research, we consulted with building operations staff and were surprised to discover that little to no analysis on CMMS and tenant survey data was conducted to inform future building maintenance and management workflows. Despite building operators and tenants storing large quantities of insightful data within these databases, it remained challenging to extract insights from them due to their unstructured and voluminous nature.

This highlighted the need for a methodology to be developed for performing data extraction and analytics from CMMS and survey datasets. We also recognized the need to present the developed analytical insights to building managers in an intuitive interface and began working on a prototype application that would potentially be used in commercial buildings in the future.

Only four buildings were analyzed, which is a limiting factor of this study. Additionally, the insights extracted from the dataset are only relevant to buildings in Ottawa, Ontario. Future research into CMMS insight extraction must expand the research scope to larger portfolios of buildings, preferably in multiple climate zones. This will achieve more insightful conclusions from the overall analysis due to the presence of more data points as well as multiple temporal factors and their effects on tenant and operator feedback logged within the CMMS.