Category: IPMI 2021 oral

Authors: Suprosanna Shit (TUM)*; Dhritiman Das (Massachusetts Institute of Technology ); Ivan Ezhov (TUM); Johannes C. Paetzold (TUM); Augusto F Fava Sanches (TUM); Nils Thuerey (Technical University of Munich); Bjoern Menze (TUM)

Poster

Abstract: Pressure inference from a series of velocity fields is a common problem arising in medical imaging when analyzing 4D data. Traditional approaches primarily rely on a numerical scheme to solve the pressure-Poisson equation to obtain a dense pressure inference. This involves heavy expert intervention at each stage and requires significant computational resources. Concurrently, the application of current machine learning algorithms for solving partial differential equations is limited to domains with simple boundary conditions. We address these challenges in this paper and present V2P-Net: a novel, neural-network-based approach as an alternative method for inferring pressure from the observed velocity fields. We design an end-to-end hybrid-network architecture motivated by the conventional Navier-Stokes solver, which encapsulates the complex boundary conditions. It achieves accurate pressure estimation compared to the reference numerical solver for simulated flow data in multiple complex geometries of human in-vivo vessels.

Authors: Jiahong Ouyang (Stanford University)*; Ehsan Adeli (Stanford University); Kilian Pohl (Stanford University); Qingyu Zhao (Stanford University); Greg Zaharchuk (Stanford University)

Poster

Abstract:

Multi-modal MRIs are widely used in neuroimaging applications since different MR sequences provide complementary information about brain structures. Recent works have suggested that multi-modal deep learning analysis can benefit from explicitly disentangling anatomical (shape) and modality (appearance) information into separate image presentations. In this work, we challenge mainstream strategies by showing that they do not naturally lead to representation disentanglement both in theory and in practice. To address this issue, we propose a margin loss that regularizes the similarity in relationships of the representations across subjects and modalities. To enable robust training, we further use a conditional convolution to design a single model for encoding images of all modalities. Lastly, we propose a fusion function to combine the disentangled anatomical representations as a set of modality-invariant features for downstream tasks. We evaluate the proposed method on three multi-modal neuroimaging datasets. Experiments show that our proposed method can achieve superior disentangled representations compared to existing disentanglement strategies. Results also indicate that the fused anatomical representation has potential in the downstream task of zero-dose PET reconstruction and brain tumor segmentation.
The code is available at https://github.com/ouyangjiahong/representation-disentanglement.

Authors: Raghavendra Selvan (University of Copenhagen)*; Erik B B Dam (University of Copenhagen); Jens Petersen (University of Copenhagen)

Poster

Abstract: Tensor networks provide an efficient approximation of operations involving high dimensional tensors and have been extensively used in modelling quantum many-body systems. More recently, supervised learning has been attempted with tensor networks, primarily focused on tasks such as image classification. In this work, we propose a novel formulation of tensor networks for supervised image segmentation which allows them to operate on high resolution medical images. We use the matrix product state (MPS) tensor network on non-overlapping patches of a given input image to predict the segmentation mask by learning a pixel-wise linear classification rule in a high dimensional space. The proposed model is end-to-end trainable using backpropagation. It is implemented as a strided tensor network to reduce the parameter complexity. The performance of the proposed method is evaluated on two public medical imaging datasets and compared to relevant baselines. The evaluation shows that the strided tensor network yields competitive performance compared to CNN-based models while using lower resources such as GPU memory. Additionally, based on the experiments we discuss the feasibility of using fully linear models for segmentation tasks.

Authors: Qi Lu (Beijing Institute of Technology); Chuyang Ye (Beijing Institute of Technology)*

Poster

Abstract: Convolutional neural networks (CNNs) have achieved state-of-the-art performance for white matter (WM) tract segmentation based on diffusion magnetic resonance imaging (dMRI). These CNNs require a large number of manual delineations of the WM tracts of interest for training, which are generally labor-intensive and costly. The expensive manual delineation can be a particular disadvantage when novel WM tracts, i.e., tracts that have not been included in existing manual delineations, are to be analyzed. To accurately segment novel WM tracts, it is desirable to transfer the knowledge learned about existing WM tracts, so that even with only a few delineations of the novel WM tracts, CNNs can learn adequately for the segmentation. In this paper, we explore the transfer of such knowledge to the segmentation of novel WM tracts in the few-shot setting. Although a classic fine tuning strategy can be used for the purpose, the information in the last task specific layer for segmenting existing WM tracts is completely discarded. We hypothesize that the weights of this last layer can bear valuable information for segmenting the novel WM tracts and thus completely discarding the information is not optimal. In particular, we assume that the novel WM tracts can correlate with existing WM tracts and the segmentation of novel WM tracts can be predicted with the logits of existing WM tracts. In this way, better initialization of the last layer than random initialization can be achieved for fine-tuning. Further, we show that a more adaptive use of the knowledge in the last layer for segmenting existing WM tracts can be conveniently achieved by simply inserting a warmup stage before classic fine-tuning. The proposed method was evaluated on a publicly available dMRI dataset, where we demonstrate the benefit of our method for few-shot segmentation of novel WM tracts.

Authors: Lasse Hansen (University of Luebeck)*; Mattias Heinrich (University of Luebeck)

Poster

Abstract: As in other areas of medical image analysis, e.g. semantic segmentation, deep learning is currently driving the development of new approaches for image registration. Multi-scale encoder-decoder network architectures achieve state-of-the-art accuracy on tasks such as intra-patient alignment of abdominal CT or brain MRI registration, especially when additional supervision, such as anatomical labels, is available. The success of these methods relies to a large extent on the outstanding ability of deep CNNs to extract descriptive visual features from the input images. In contrast to conventional methods, the explicit inclusion of geometric information plays only a minor role, if at all. In this work we take a look at an exactly opposite approach by investigating a deep learning framework for registration based solely on geometric features and optimisation. We combine graph convolutions with loopy belief message passing to enable highly accurate 3D point cloud registration. Our experimental validation is conducted on complex key-point graphs of inner lung structures, strongly outperforming dense encoder-decoder networks and other point set registration methods. Our code is publicly available at https://github.com/multimodallearning/deep-geo-reg.

Authors: Xiaojie Gao (The Chinese University of Hong Kong)*; Yueming Jin (The Chinese University of Hong Kong); Zixu Zhao (The Chinese University of Hong Kong); Qi Dou (The Chinese University of Hong Kong); Pheng-Ann Heng (The Chinese Univsersity of Hong Kong)

Poster

Abstract: Predicting future frames for robotic surgical video is an interesting, important yet extremely challenging problem, given that the operative tasks may have complex dynamics. Existing approaches on future prediction of natural videos were based on either deterministic models or stochastic models, including deep recurrent neural networks, optical flow, and latent space modeling. However, the potential in predicting meaningful movements of robots with dual arms in surgical scenarios has not been tapped so far, which is typically more challenging than forecasting independent motions of one arm robots in natural scenarios. In this paper, we propose a ternary prior guided variational autoencoder (TPG-VAE) model for future frame prediction in robotic surgical video sequences. Besides content distribution, our model learns motion distribution, which is novel to handle the small movements of surgical tools. Furthermore, we add the invariant prior information from the gesture class into the generation process to constrain the latent space of our model. To our best knowledge, this is the first time that the future frames of dual arm robots are predicted considering their unique characteristics relative to general robotic videos. Experiments demonstrate that our model gains more stable and realistic future frame prediction scenes with the suturing task on the public JIGSAWS dataset.

Authors: Rick Sengers (Eindhoven University of Technology)*; Luc Florack (Eindhoven University of Technology); Andrea Fuster (Eindhoven University of Technology)

Poster

Abstract:

Based on diffusion tensor imaging (DTI), one can construct a Riemannian manifold in which the dual metric is proportional to the DTI tensor.
Geodesic tractography then amounts to solving a coupled system of nonlinear differential equations, either as initial value problem (given seed location and initial direction) or as boundary value problem (given seed and target location).
We propose to furnish the tractography framework with an uncertainty quantification paradigm
that captures the behaviour of geodesics under small perturbations in (both types of) boundary conditions.
For any given geodesic this yields a coupled system of linear differential equations,
for which we derive an exact solution.
This solution can be used to construct a geodesic tube,
a volumetric region around the fiducial geodesic that captures the behaviour of
perturbed geodesics in the vicinity of the original one.

Authors: Zhiming Cui (HKU); Bojun Zhang (Shanghai Jiao Tong University); Chunfeng Lian (Xi’an Jiaotong University); Changjian Li (University College London); LEI YANG (University of Hong Kong); Min Zhu (Shanghai Jiaotong University); Wenping Wang (The University of Hong Kong); Dinggang Shen (United Imaging Intelligence)

Poster

Abstract: Automatic and accurate segmentation of individual teeth, i.e., tooth instance segmentation, from CBCT images is an essential step for computer-aided dentistry. Previous works typically overlooked rich morphological features of teeth, such as tooth root apices, critical for successful treatment outcomes. This paper presents a two-stage learning-based framework that explicitly leverages the comprehensive geometric guidance provided by a hierarchical tooth morphological representation for tooth instance segmentation. Given input CBCT images, our method first learns to extract the tooth centroids and skeletons for identifying each tooth’s rough position and topological structures, respectively. Based on the outputs of the first step, a multi-task learning mechanism is further designed to estimate each tooth’s volumetric mask by simultaneously regressing boundary and root apices as auxiliary tasks. Extensive evaluations, ablation studies, and comparisons with existing methods show that our approach achieved state-of-the-art segmentation performance, especially around the challenging dental parts (i.e., tooth roots and boundaries). These results suggest the potential applicability of our framework in real-world clinical scenarios.

Authors: Juntang Zhuang (Yale University); Nicha Dvornek (Yale University); Sekhar Tatikonda (Yale); Xenophon Papademetris (Yale University); Pamela Ventola (Yale University); James S Duncan (Yale University)

Poster

Abstract:

Dynamic causal modeling (DCM) is a Bayesian framework to infer directed connections between compartments, and has been used to describe the interactions between underlying neural populations based on functional neuroimaging data. DCM is typically analyzed with the expectation-maximization (EM) algorithm.
However, because the inversion of a large-scale continuous system is difficult when noisy observations are present, DCM by EM is typically limited to a small number of compartments ($<10$). Another drawback with the current method is its complexity; when the forward model changes, the posterior mean changes, and we need to re-derive the algorithm for optimization.
In this project, we propose the Multiple-Shooting Adjoint (MSA) method to address these limitations. MSA uses the multiple-shooting method for parameter estimation in ordinary differential equations (ODEs) under noisy observations, and is suitable for large-scale systems such as whole-brain analysis in functional MRI (fMRI). Furthermore, MSA uses the adjoint method for accurate gradient estimation in the ODE; since the adjoint method is generic, MSA is a generic method for both linear and non-linear systems, and does not require re-derivation of the algorithm as in EM. We validate MSA in extensive experiments: 1) in toy examples with both linear and non-linear models, we show that MSA achieves better accuracy in parameter value estimation than EM and MSA can be successfully applied to large systems with up to 100 compartments; and 2) using real fMRI data, we apply MSA to the estimation of the whole-brain effective connectome and show improved classification of autism spectrum disorder (ASD) vs. control compared to using the functional connectome. The package is provided \url{https://jzkay12.github.io/TorchDiffEqPack}