Embedding human heuristics in machine-learning-enabled probe microscopy

Oliver M Gordon,Philip J Moriarty,Filipe L Q Junqueira

doi:10.1088/2632-2153/ab42ec

Oliver M Gordon, Philip J Moriarty + Show 1 more

Open Access

https://doi.org/10.1088/2632-2153/ab42ec

Copy DOI

Abstract

Scanning probe microscopists generally do not rely on complete images to assess the quality of data acquired during a scan. Instead, assessments of the state of the tip apex, which not only determines the resolution in any scanning probe technique, but can also generate a wide array of frustrating artefacts, are carried out in real time on the basis of a few lines of an image (and, typically, their associated line profiles.) The very small number of machine learning approaches to probe microscopy published to date, however, involve classifications based on full images. Given that data acquisition is the most time-consuming task during routine tip conditioning, automated methods are thus currently extremely slow in comparison to the tried-and-trusted strategies and heuristics used routinely by probe microscopists. Here, we explore various strategies by which different STM image classes (arising from changes in the tip state) can be correctly identified from partial scans. By employing a secondary temporal network and a rolling window of a small group of individual scanlines, we find that tip assessment is possible with a small fraction of a complete image. We achieve this with little-to-no performance penalty—or, indeed, markedly improved performance in some cases—and introduce a protocol to detect the state of the tip apex in real time.

Highlights

One of the major challenges in the drive to fully automate the scanning probe microscope is the need to constantly maintain the integrity of the tip [1, 2]
By comparing a variety of methods based around a common VGG network, we have successfully demonstrated that scanning tunnelling microscopy (STM) images of the H:Si(100) surface can be accurately assessed using partial scans
Given that the majority of the time spent maintaining SPM tips is spent acquiring data, a ‘hybrid’ approach combining individual linescans and long-term recurrent convolutional network (LRCN) prediction would speed up convolutional neural networks (CNNs) routines by approximately 100 times

Summary

Introduction

One of the major challenges in the drive to fully automate the scanning probe microscope is the need to constantly maintain the integrity of the tip [1, 2]. Interactions with the surface can cause the tip to spontaneously and randomly change shape, modifying the interactions and changing the data acquired in a highly nonlinear fashion This frequently results in inconsistent scans containing visual artefacts, often making data unusable or, at best, problematic to interpret. Controlling and maintaining the atomistic and orbital structure of the tip apex is a vital part of state-of-the-art SPM operation This requires a protracted and repetitive routine of voltage pulsing, ‘gentle’ (or not-so-gentle) indenting of the tip into the surface, scanning at relatively high voltages and currents, and/or attempts to pick up adsorbates. This is at present a high-effort, time-consuming and manual process involving only simple sub-processes, making it ideal to automate

Objectives

Results

Conclusion