
Distance Map Supervised Landmark Localization for
MR-TRUS Registration
Xinrui Song1, Xuanang Xu1, Sheng Xu2, Baris Turkbey3, Bradford J. Wood2, Thomas
Sanford4, and Pingkun Yan1
1Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
2Center for Interventional Oncology, Radiology & Imaging Sciences, National Institutes of
Health, Bethesda, MD, USA
3Molecular Imaging Program, National Cancer Institute, National Institutes of Health,
Bethesda, MD, USA
4Department of Urology, The State University of New York Upstate Medical University,
Syracuse, NY, USA
1. DESCRIPTION OF PURPOSE
Image-guided interventional procedures often require registering multi-modal images to visualize and analyze
complementary information. For example, prostate cancer biopsy benefits from fusing transrectal ultrasound
(TRUS) imaging with magnetic resonance (MR) imaging to optimize targeted biopsy. However, cross-modal
image registration is a challenging task. This is especially true when the appearance of two image modalities
are vastly different. Since registration quality is most reliably evaluated with target registration error (TRE),
it is sensible to directly make use of the anatomical landmark targets from images. Moreover, while image
modalities and thus textures differ, anatomical landmarks are the only information shared across the moving
and the fixed images. Sun et al.1proposed to perform pre-alignment for MR-TRUS registration with manually
labeled landmarks on both images. However, such a procedure is still far from automatic due to the requirement
of manual input at inference time. Heinrich et al.2made use of a landmark detection method specifically designed
for lung computed tomography (CT) registration, which is not generalizable to other tasks.
In this work, we propose to explicitly use the landmarks of prostate to guide the MR-TRUS image registration.
We first train a deep neural network to automatically localize a set of meaningful landmarks, and then directly
generate the affine registration matrix from the location of these landmarks. For landmark localization, instead
of directly training a network to predict the landmark coordinates, we propose to regress a full-resolution distance
map of the landmark, which is demonstrated effective in avoiding statistical bias to unsatisfactory performance
and thus improving performance. We then use the predicted landmarks to generate the affine transformation
matrix, which outperforms the clinicians’ manual rigid registration by a significant margin in terms of TRE.
2. METHODS
Figure 1gives an overview of the proposed method. We first extract the location of selected landmarks from
both TRUS and MR images. Then, the two sets of corresponding landmarks are used to calculate an affine
registration matrix that aligns the two images. The acquisition of landmarks is explained in Section 2.1, and the
step that generates the affine registration from corresponding points is explained in Section 2.2.
2.1 Distance map-guided landmark localization
Anatomically important and stable landmarks are key to registration, as it is to our method. We referred to
previous literature1and consulted clinicians to finally decide on using four landmarks. Two of the landmarks
are the extreme points located on the prostate boundary: the right-most and left-most extreme points observed
from the axial view. The other two (shown in Figure 2) are the entrance point of the urethra into the prostate
at the neck of the bladder, and the point where the urethra exits the prostate at the prostate apex. As the two
Send correspondence to Pingkun Yan (yanp2@rpi.edu).
arXiv:2210.05738v1 [cs.CV] 11 Oct 2022