A Deep Learning Framework for Spatiotemporal Ultrasound Localization Microscopy

Ultrasound Localization Microscopy

Introduction

Ultrasound Localization Microscopy (ULM) bypasses the intrinsic spatial resolution of conventional contrast-enhanced ultrasound imaging via the localization of sparse microbubble (MB) populations across ultrasound images [1]–[3]. As of today, ULM appears to be the only cost-effective, non-invasive, and non-ionizing method for the imaging of the microvasculature in large fields of view in vivo and in several organs such as the brain [4]. Adding tracking algorithms to the detection of MB enabled the mapping of blood flow velocities [5]. The study of the microvascular angioarchitecture and its function at-depth and in vivo could become a powerful tool in the development of novel biomarkers for neurodegenerative diseases, cardiac diseases and cancer.

Nevertheless, the clinical application of ULM is limited by long, motion-free acquisition time (a few minutes) required to output a single highly resolved image. This issue can be addressed in part by increasing MB density [7]. However, higher densities increase the difficulty of precisely localizing MB. Indeed, as they flow throughout the vascular network, MB that are close to one another lead to ultrasound signal interference, preventing their accurate localization with a peak detection algorithm. Several processing techniques have been proposed to tackle this multi-object localization in ultrasound images and [9], efficient filtering methods have been introduced based respectively on background removal, spatio-temporal- interframe-correlation based data acquisition, and separating spatially overlapping MB events into sub populations. Some also use advance pairing techniques that discard unrealistic MB trajectories [10] or graph-based MB tracking on denoised images [11]. In [12], authors exposed the capabilities of neural networks to spatio-temporally filter single MB in in vivo ULM images by training a CNN to perform conventional signal processing methods. Others investigated the application of deep learning-based algorithms to enhance the localization of individual MB when higher concentrations are used.

Those were either based on radiofrequency (RF) data [13] or envelope-detected images [14], [15], [16], [17] and all relied on a per-frame localization. More recently, authors proposespatio-temporal filtering based on CNN methods to localize multiple MB in in vitro data [18] or in small patches containing single MB to perform in vivo ULM [19]. Ideally, multi-object localization would include modelling complexities such as MB interferences, hemodynamic considerations, and blood vessel
shapes a priori.

In this section, we first describe the conventional ULM pipeline used in this study and similar to other approaches described in the literature [7]. We then introduce the proposed novel Deep-stULM framework as a surrogate to the conventional localization and tracking step to recover dense tracks from ULM ultrasound image sequences.

Following an injection of MB contrast agents, a programmable ultrasound scanner was used to acquire several hundred blocks of hundreds of compounded plane wave images acquired at one thousand frames per second and repeated every second for minutes-long acquisitions. After in phase-quadrature complex (IQ) image formation (i.e., beam- forming) [20], tissue was removed using a singular value decomposition (SVD) clutter filter [21], [22], to recover signals from MB. The SVD threshold was heuristically set from what we use in our standard ULM process. Individual MB were then identified as bright local maxima, within correlation maps
resulting from the correlation of the beamformed IQ images with the point spread function (PSF) of the imaging system. MB were then precisely located through subpixel gaussian fitting. MB subpixel positions were accumulated over time on a finer grid corresponding to approximately one tenth of the wavelength, to recover the micro vessel density maps.

The proposed tracking operation took Nt successive frames of Nz×Nx correlation maps as input, i.e., χ ∈ CNt ×N z ×N x with ? ?χi j k ? ? < 1∀(i, j, k) ∈ [[1 : Nt ]] × [[1 : Nz ]] × [[1 : Nx ]] – depicting the flowing MB –, and produced one high-resolution binary tracking, i.e., h (χ; θ ) ∈ {0, 1}r∗N z ×r∗N x – depicting the 2D projection of the MB trajectories –, where r is the resolution factor and θ ∈ is the set of parameters of the CNN architecture. We adopted a supervised learning setting and searched for the optimal tracking operation parameters ˆθ using a limited set of mappings h (·; θ ) and input/output pairs (χ, ψ) ∈ D, the data set and by minimizing the empirical loss as follows:

 θ = argmin θ∈ ∑ (χ,ψ)∈D Loss (h (χ; θ ) , ψ).