Human-in-the-loop machine learning for drone-assisted Structural Health Monitoring

Project Description:

This Research Project is part of the Aura CDT’s Reliability and Health Monitoring Cluster.

Structural health monitoring, such as identifying damage or ice on turbine blades, anticipating its effects and making decisions on maintenance and repair, as well as estimating remaining useful life (RUL), is an important part of extending the lifetime of a wind turbine. While often driven by experts, drones can be used to gather images of the turbine, which can then be analysed by experts or an AI system for automatic analysis [1]. The images can also be annotated with damage information to train machine learning models to automatically detect damage for new images, e.g. unseen types of turbines or parts, or in new physical environments, as will be increasingly relevant with wind farms moving further out to sea.

Since publicly available image data on turbine damage is scarce – and will not likely be available for newer turbine generations in the first instance- this project investigates techniques to achieve high accuracy with as little training data as possible. In active learning for example, the learning agent selects examples from the raw data that are more likely to improve performance when provided with a label/annotation by an expert, e.g. those where its prediction is most uncertain, and then adds them to the training data. In this way, an agent can learn to provide better predictions over time and become increasingly useful in assisting human experts making maintenance and repair decisions. At the same time, successful damage detection should not only provide visual analysis, or potentially recommend the best set of maintenance action/s (and tradeoffs), but also learn from any scenarios encountered, and human feedback received. The system should autonomously improve over time, optimising key objectives such as reduced turbine downtime, extended turbine life, reduced costs and waste, while considering options.

This project envisages an interactive system based on a (deep) reinforcement learning (RL) optimisation framework. We look to develop technical advances in tractable optimisation from dynamic and evolving system knowledge, where new classes (e.g. new types of damage, not previously present in the data) can be added at any point during the lifetime of the system, prompting rapid tuning or re-training, to achieve increasingly better predictions.

Methodology

The research can be structured into three phases.

Phase 1 creates a deep reinforcement learning (DRL) agent [2] that learns to map visual features from drone images to damage types and maintenance & repair actions. This initial phase creates a basic framework for experimentation and is interleaved with an in-depth literature review and understanding of the domain and available frameworks.

Phase 2 will focus on the optimisation of the DRL agent given different environmental conditions (e.g. cold, so blades could have icy, windy/wavy, so maintenance is difficult; rainy, so images could be noisy, etc.) and expert-specified objectives (e.g. minimise downtime vs. minimise risk of broken parts). The focus of the research is on experimenting with different ways of interpreting visual features and choosing actions strategies given tradeoffs.

Phase 3 will develop a framework for interactive policy learning and updates, e.g. when feedback is received from a human operator (e.g. “wrong type of damage detected”, “when windy, prioritise power generation”, “add new type of damage from lightning”, etc.). This part will be based on a Human-in-the-loop Machine Learning (HITL-ML) framework [3], in which the agent and human expert interact to train the detection model more efficiently, going beyond active learning where the learning agent has the initiative. A focus will be on explainability, i.e. for the agent to seek ways of understanding what decisions were made by an expert and why, so that the agent can provide rationales for its own actions in future [4]. This part will use natural language to improve intuitive explainability and the potential to use expert explanations to teach the system how to better identify damage etc., possibly leveraging large language models, such as those behind ChatGPT [5].

References & Further Reading

[1] Chatterjee et al. (2023) Domain-invariant icing detection on wind turbine rotor blades with generative artificial intelligence for deep transfer learning. Environmental Data Science. https://doi.org/10.1017/eds.2023.9

[2] Arulkumaran et al (2017) Deep Reinforcement Learning: A Brief Survey, in IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 26-38, Nov. 2017, 10.1109/MSP.2017.2743240.

[3] Mosqueira-Rey, et al. (2023) Human-in-the-loop machine learning: a state of the art. Artif Intell Rev 56. https://doi.org/10.1007/s10462-022-10246-w

[4] Nishida et al. (2022) Improving Few-Shot Image Classification Using Machine- and User-Generated Natural Language Descriptions. Findings of NAACL, Seattle, USA. 10.18653/v1/2022.findings-naacl.106

[5] Brown et al. (2020). Language Models are Few-Shot Learners. NeurIPS. https://arxiv.org/abs/2005.14165