A computational method using multiresolution for volumetric data integration

Abstract

In this paper, we present a new method for integration of 3-D medical data by utilizing the advantages of 3-D multiresolution analysis and techniques of variational calculus. We first express the data integration problem as a variational optimal control problem where we express the displacement field in terms of wavelet expansions and, secondly, we write the components of the displacement field in terms of wavelet coefficients. We solve this optimization problem with a blockwise descent algorithm. We demonstrate the registration of 3-D brain MR images in the size of $257\times 257\times 65$ as an application of the present method. Experimental results indicate that the method can integrate 3-D MR images better than only variational or only wavelet-based methods.

MSC: 68U10, 65D18, 65J05, 97N40.

1 Introduction

The main purpose of this paper is to present an efficient 3-D medical data (image) integration technique. Image integration (sometimes called registration or matching) can be described as finding a spatial correspondence between pixels (or voxels) of two images that maximizes the similarity between the two images. The images could be of the same or different objects and imaging modalities and possibly be taken at different distances, angles, and times. Detecting tumors, locating diseased areas, monitoring changes in an individual, drug discovery, image fusion, feature matching, and motion tracking are some of the important applications of the image registration problem. So far a general theory for image matching has yet to be established. Each application venue has developed its own approaches and implementations. As a result, a single standard method for image integration has not emerged. Therefore, finding reliable and efficient image integration techniques along with fast implementation methods is significantly important and active research area. Some of the well-known image integration algorithms can be seen in [1–5] and in the references therein.

Structure of this paper is as follows. In Section 2, we present an algorithm for integration of 3-D medical data by utilizing the advantages of 3-D multiresolution analysis and techniques of variational calculus. In Section 10, we present some experimental results regarding the integration of MR images as an application of the present method. We complete the paper with a final section where we briefly summarize the paper and discuss the future extensions.

2 Multiresolution approach for deformation field

Assume that both the template $\mathbf{T}(\mathbf{x})$ and reference $\mathbf{R}(\mathbf{x})$ images are defined on the same domain $\mathrm{\Omega }\subset {\mathbb{R}}^3$. Because reference and template images are obtained from different distances, angles, times, sensors, and sometimes even by different individuals, a displacement field may occur between the reference and template images. One of the major goals of this paper is to compute the deformation field in a systematic way.

A deformation field is a vector image that maps reference image pixel coordinates to the coordinates of the corresponding template image pixels. Consider the deformations of the form

$${\varphi }_u(\mathbf{x}):=\mathbf{x}+u(\mathbf{x}),\mathbf{x}=(x_1,x_2,x_3)\in \mathrm{\Omega },u(\mathbf{x})=(u_1(\mathbf{x}),u_2(\mathbf{x}),u_3(\mathbf{x})),$$

(1)

where $u(\mathbf{x})$ denotes a displacement field. Because both the template $\mathbf{T}(\mathbf{x})$ and reference $\mathbf{R}(\mathbf{x})$ images are almost surely ‘the same’ images, we can write

$$\mathbf{R}(x_1,x_2,x_3)\approx \mathbf{T}((x_1,x_2,x_3)+u(x_1,x_2,x_3))=\mathbf{T}(x_1+u_1(\mathbf{x}),x_2+u_2(\mathbf{x}),x_3+u_3(\mathbf{x})).$$

(2)

In this paper, we exploit 3-D Haar wavelets. When expanding the displacement field $u(\mathbf{x})$ in terms of a wavelet decomposition, it is necessary to take into account the wavelet parameters (coefficients) α, which yields that

$${\varphi }_u(\mathbf{x})=\left[\begin{array}{c}{x}_1+u_1(x_1,x_2,x_3,\alpha )\\ x_2+u_2(x_1,x_2,x_3,\alpha )\\ x_3+u_3(x_1,x_2,x_3,\alpha )\end{array}\right].$$

The major goal of this paper is to compute the displacement field $u(\mathbf{x})$ in a systematic way. We express each of $u_i$, $i=1,2,3$, in terms of wavelet coefficients. Therefore, computing these wavelet coefficients will be enough to obtain the displacement field $u(\mathbf{x})$.

A multiresolution analysis of $L^2(\mathrm{\Omega })$, $\mathrm{\Omega }={\mathbb{R}}^3$ is a partially ordered set of closed linear subspaces

$${\{V_{\mathbf{n}}\subset L^2(\mathrm{\Omega })\}}_{\mathbf{n}\in {\mathbb{Z}}^3}$$

with properties:

• (1) ${\bigcap }_{\mathbf{n}}{V}_{\mathbf{n}}=\{0\}$; ${\bigcup }_{\mathbf{n}}{V}_{\mathbf{n}}{\subset }_{\mathrm{dense}}{L}^2(\mathrm{\Omega })$;

• (2) $f(\mathbf{x})\in V_{\mathbf{n}}⟺f(2^{\mathbf{m}}\mathbf{x})\in V_{\mathbf{n}+\mathbf{m}}$, for every $\mathbf{n}\in {\mathbb{Z}}^3$, $\mathbf{m}\in E$;

• (3) $f(\mathbf{x})\in V_{\mathbf{n}}⟺f(\mathbf{x}-2^{-\mathbf{k}})\in V_{\mathbf{n}}$, for every $\mathbf{k}\in {\mathbb{Z}}^3$, $\mathbf{n}\in E$;

• (4) There exists $\mathrm{\Phi }\in V_{\mathbf{0}}$ such that ${\{\mathrm{\Phi }(\mathbf{x}-\mathbf{k})\}}_{\mathbf{k}\in {\mathbb{Z}}^3}$ is a Riesz basis for $V_{\mathbf{0}}$.

Here $\mathbf{n}=(n_1,n_2,n_3)$, $2^{\mathbf{n}}=(2^{n_1},2^{n_2},2^{n_3})$, $\mathbf{0}=(0,0,0)\in {\mathbb{N}}^3$, $\mathbf{x}=(x_1,x_2,x_3)\in {\mathbb{R}}^3$, $2^n=(2^{n_1}{x}_1,2^{n_1}{x}_2,2^{n_1}{x}_3)$. The function $\mathrm{\Phi }(\mathbf{x})$ is called the scaling function of the multiresolution analysis. Detailed information about multiresolution analysis and wavelets might be seen in [6]. Let $\mathrm{\Omega }={(0,1)}^3\subset {\mathbb{R}}^3$ be the open unit cube. For any $\mathbf{n}\in {\mathbb{Z}}^3$, we introduce the function space $V_{\mathbf{n}}$: the space of piecewise constant functions on a uniform grid with mesh size $h=(2^{-n_1},2^{-n_2},2^{-n_3})$. These grids are uniformly spaced in each of the three coordinate directions, but possibly with a different mesh size in the different directions. The volume of these cells is denoted by $h^3=2^{-(n_1+n_2+n_3)}$. The functions in $V_{\mathbf{n}}$ are all constant in each cell defined by

$${\mathrm{\Omega }}_{\mathbf{n},\mathbf{k}}:=[k_1{2}^{-n_1},(k_1+1)2^{-n_1}]\times [k_2{2}^{-n_2},(k_2+1)2^{-n_2}]\times [k_3{2}^{-n_3},(k_3+1)2^{-n_3}].$$

(3)

A basis $V_m$ may be generated from a scaling function Φ. In order to deal with 3-D deformation field, to each component of the displacement field, a multiresolution decomposition is applied. At scale m, i.e., in the space $V_m$, the displacement field $u^m=(u_1^m,u_2^m,u_3^m)$ will be parameterized by the vector

$${\alpha }^m:=[{\alpha }_{x_1,i,j,k}^m,{\alpha }_{x_2,i,j,k}^m,{\alpha }_{x_3,i,j,k}^m]$$

as

$$u^m(x_1,x_2,x_3,\alpha )=\left[\begin{array}{c}{u}_1^m(x_1,x_2,x_3,\alpha )\\ u_2^m(x_1,x_2,x_3,\alpha )\\ u_3^m(x_1,x_2,x_3,\alpha )\end{array}\right]=\left[\begin{array}{c}{\sum }_{i,j,k}{\alpha }_{x_1,i,j,k}^m{\mathrm{\Phi }}_{i,j,k}^m(x_1,x_2,x_3,\alpha )\\ {\sum }_{i,j,k}{\alpha }_{x_2,i,j,k}^m{\mathrm{\Phi }}_{i,j,k}^m(x_1,x_2,x_3,\alpha )\\ {\sum }_{i,j,k}{\alpha }_{x_3,i,j,k}^m{\mathrm{\Phi }}_{i,j,k}^m(x_1,x_2,x_3,\alpha )\end{array}\right].$$

(4)

Elements (scaling functions) of a basis $V_m$ are 3-D functions that are translated across the cubical grid ${\mathrm{\Omega }}_{\mathbf{n},\mathbf{k}}$. These functions are a tensor product of the 1-D scaling and wavelet functions as

$$\begin{array}{c}{\mathrm{\Phi }}^1=\varphi (2^{-j}{x}_1-k_1)\varphi (2^{-j}{x}_2-k_2)\varphi (2^{-j}{x}_3-k_3),\hfill \\ {\mathrm{\Phi }}^2=\varphi (2^{-j}{x}_1-k_1)\varphi (2^{-j}{x}_2-k_2)\psi (2^{-j}{x}_3-k_3),\hfill \\ {\mathrm{\Phi }}^3=\varphi (2^{-j}{x}_1-k_1)\psi (2^{-j}{x}_2-k_2)\varphi (2^{-j}{x}_3-k_3),\hfill \\ {\mathrm{\Phi }}^4=\varphi (2^{-j}{x}_1-k_1)\psi (2^{-j}{x}_2-k_2)\psi (2^{-j}{x}_3-k_3),\hfill \\ {\mathrm{\Phi }}^5=\psi (2^{-j}{x}_1-k_1)\varphi (2^{-j}{x}_2-k_2)\varphi (2^{-j}{x}_3-k_3),\hfill \\ {\mathrm{\Phi }}^6=\psi (2^{-j}{x}_1-k_1)\varphi (2^{-j}{x}_2-k_2)\psi (2^{-j}{x}_3-k_3),\hfill \\ {\mathrm{\Phi }}^7=\psi (2^{-j}{x}_1-k_1)\psi (2^{-j}{x}_2-k_2)\varphi (2^{-j}{x}_3-k_3),\hfill \\ {\mathrm{\Phi }}^8=\psi (2^{-j}{x}_1-k_1)\psi (2^{-j}{x}_2-k_2)\psi (2^{-j}{x}_3-k_3).\hfill \end{array}$$

3 Optimal control formulation of data integration

The state-of-the-art image registration problem can be expressed as an optimal control problem by

$$\underset{\mathit{\varphi }\in \mathbf{\Gamma }}{min}\mathcal{J}[\mathbf{R},\mathbf{T};{\mathit{\varphi }}_u]$$

(5)

for the functional

$$\mathcal{J}[\mathbf{R},\mathbf{T};{\mathit{\varphi }}_u]=C_{\mathrm{sim}}[\mathbf{R},\mathbf{T};{\mathit{\varphi }}_u]+\lambda C_{\mathrm{reg}}[u],$$

(6)

where $C_{\mathrm{sim}}[\mathbf{R},\mathbf{T};{\mathit{\varphi }}_u]$ denotes a similarity measure between the template image T and the reference image R, ${\mathit{\varphi }}_u(\mathbf{x}):=\mathbf{x}+u(\mathbf{x})$ is the deformation field, u is the displacement field, Γ is the set of all possible admissible transformations, $C_{\mathrm{reg}}[u]$ is a regularization term, and λ is a regularization constant.

We choose the $L^2$-norm type similarity measure defined as

$$C_{\mathrm{sim}}[\mathbf{R}(\mathbf{x}),\mathbf{T}(\mathbf{x});{\mathit{\varphi }}_u(\mathbf{x})]:={\int }_{\mathrm{\Omega }}\mathrm{\nabla }\cdot (\mathbf{T}(\mathbf{x}+u(\mathbf{x}))-\mathbf{R}(\mathbf{x}))d\mathbf{x}.$$

(7)

Note that some other similarity measures might be selected depending on the problem. We choose (7) because, as of our best knowledge, this similarity measure has not been associated with any volumetric image registration algorithm in the literature and to test the convenience of this measure in these types of applications.

Without the regularizing term in functional (6), the image registration problem (5) is ill-posed [8]; furthermore, imaging data usually is not smooth due to edges, folding, or other unwanted deformations. Ill-posed problems are widely used in PDE-based image processing problems and inverse problems. An optimization problem is said to be well posed if the solution of the problem uniquely exists and the solution depends continuously on the data of the problem. If one of these two conditions is not satisfied, it is called an ill-posed problem. Image registration is an ill-posed optimal control problem. In order to overcome the ill-posedness of the optimization problem (5) and to assure smooth solutions, we introduce additional regularization terms. The main idea behind adding a regularization term is to smoothen the problem with respect to both the functional and the solution so that well-posedness is assured and efficient computational methods can be defined to determine minimizers. Typical regularization terms associated with image registration problems include curvature, diffusion, elasticity, and fluid. Details about each of these regularization approaches can be seen, for example, in [1] and the references therein.

In this paper, we introduce a regularization term that consists of summation of two different terms defined as follows:

$$C_{\mathrm{reg}}[u(\mathbf{x})]:={\lambda }_1{\int }_{\mathrm{\Omega }}\sqrt{{|\mathrm{\nabla }u(\mathbf{x})|}^2+\beta }d\mathbf{x}+{\lambda }_2{\int }_{\mathrm{\Omega }}log(u(\mathbf{x}))d\mathbf{x}.$$

(8)

Let us further point out that the regularization term (8) has not also been associated with any volumetric data integration problem in the literature. The term ${\int }_{\mathrm{\Omega }}\sqrt{{|\mathrm{\nabla }u(\mathbf{x})|}^2+\beta }d\mathbf{x}$ is known as a perturbed total-variation model and has been used in image restoration problems. This model was obtained by modifying the regularization term mostly known as the Dirichlet regularization term given by

$${\int }_{\mathrm{\Omega }}\sqrt{{|\mathrm{\nabla }u(\mathbf{x})|}^2}d\mathbf{x},$$

(9)

which penalizes non-smooth images. Major shortcomings of (9) is that some image features, like edges of the original image, may show up blurred in the reconstructed image. To overcome this drawback, Rudin, Osher, and Fatemi (ROF) proposed replacing (9) with so-called total-variation (TV) seminorm ${\int }_{\mathrm{\Omega }}\sqrt{|\mathrm{\nabla }u(\mathbf{x})|}d\mathbf{x}$. In the solution of the optimal control problem (5), in order to prevent the degeneracy of the resulting Euler-Lagrange equations, we modify the TV-model as ${\int }_{\mathrm{\Omega }}\sqrt{{|\mathrm{\nabla }u(\mathbf{x})|}^2+\beta }d\mathbf{x}$, where β is an arbitrarily small perturbation parameter. Another regularization term that we use is ${\int }_{\mathrm{\Omega }}log(u(\mathbf{x}))d\mathbf{x}$. This term is added to make the regularization term original and to see its impact on the volumetric data integration problems.

Having said these, we can express the cost function of the optimization problem (5) as

$$\begin{array}{rcl}\mathcal{J}[\mathbf{R},\mathbf{T};{\mathit{\varphi }}_u]& =& {\int }_{\mathrm{\Omega }}\mathrm{\nabla }\cdot (\mathbf{T}(\mathbf{x}+u(\mathbf{x}))-\mathbf{R}(\mathbf{x}))d\mathbf{x}\\ +{\lambda }_1{\int }_{\mathrm{\Omega }}\sqrt{{|\mathrm{\nabla }u(\mathbf{x})|}^2+\beta }d\mathbf{x}+{\lambda }_2{\int }_{\mathrm{\Omega }}log(u(\mathbf{x}))d\mathbf{x}.\end{array}$$

This is a variational [7] convex optimization problem. Necessary and sufficient conditions for the existence and uniqueness of the solutions was given in [4]. Because we set up a connection between this variational optimization problem and 3-D wavelet transforms, for a given scale m, the optimal control problem can be expressed as

$${\hat{\alpha }}^m=\underset{{\alpha }^m\in {\mathcal{A}}^m}{argmin}\mathcal{J}[C_{\mathrm{sim}}(\mathbf{x}),C_{\mathrm{reg}}(\mathbf{x}),{\varphi }_u(\mathbf{x},{\alpha }^m)],$$

where ${\mathcal{A}}^m$ stands for the admissible parameter set. We apply a blockwise descent algorithm. During the minimization, the cost functional $\mathcal{J}$ needs to be evaluated only on ${\mathrm{\Omega }}_{i,j,k}^m$, defined as

$${\mathrm{\Omega }}_{i,j,k}^m:=[\frac{i-1}{2^m},\frac{i+1}{2^m}]\times [\frac{j-1}{2^m},\frac{j+1}{2^m}]\times [\frac{k-1}{2^m},\frac{k+1}{2^m}],$$

which is the support of ${\mathrm{\Phi }}_{i,j,k}^m$. Inside the block, the direction of descent $d\in {\mathbb{R}}^3$ is computed as the opposite of the gradient $\frac{\partial \mathcal{J}}{\partial {\alpha }^m}$ of the cost function $\mathcal{J}$ where

$$\begin{array}{rcl}\frac{\partial \mathcal{J}}{\partial {\alpha }^m}& =& {\int }_{\mathrm{\Omega }}\mathrm{\Delta }({\mathbf{T}}_u(\mathbf{x})-{\mathbf{R}}_u(\mathbf{x}))\mathrm{\nabla }{\mathbf{T}}_u(\mathbf{x}){\left(\frac{\partial u(\mathbf{x})}{\partial {({\alpha }^m)}^t}\right)}^{t}d\mathbf{x}\\ +{\lambda }_1{\int }_{\mathrm{\Omega }}\frac{\mathrm{\nabla }u(\mathbf{x})}{\sqrt{{|\mathrm{\nabla }u(\mathbf{x})|}^2}}{\left(\frac{\partial u(\mathbf{x})}{\partial {({\alpha }^m)}^t}\right)}^{t}d\mathbf{x}+{\lambda }_2{\int }_{\mathrm{\Omega }}\frac{\mathrm{\nabla }u{(\mathbf{x})}^{\prime }}{\mathrm{\nabla }u(\mathbf{x})}{\left(\frac{\partial u(\mathbf{x})}{\partial {({\alpha }^m)}^t}\right)}^{t}d\mathbf{x}.\end{array}$$

4 Experimental results

In this section, we demonstrate the registration of brain MR images in the size of $257\times 257\times 65$ as an application of the present method. The template, reference, and integrated images are shown in Figure 1. Duration of the registration is about 2 minutes, which is quite fast for 3-D medical image integration. We applied the presented method to some other brain MR images and obtained similar results.

Figure 1

Template (top), reference (bottom left) and integrated (bottom right) images.

5 Conclusion

In this paper, we present a method for integration of 3-D medical data by utilizing the advantages of 3-D multiresolution analysis and techniques of variational calculus. We first express the data integration problem as a variational optimal control problem where we express the displacement field in terms of wavelet expansions and, secondly, we express the components of the displacement field in terms of wavelet coefficients. We solve the aforementioned optimization problem with a blockwise descent algorithm. We demonstrate the registration of 3-D brain MR images in the size of $257\times 257\times 65$ as an application of the present method. Experimental results indicate that the method can integrate 3-D MR images better than only variational or only wavelet based methods. In a related work [9], a 3-D wavelet based method was presented for deformable image registration where different similarity measure, different regularization term, and different types of wavelets were used.

In future work, we will investigate the applications of this image matching technique to the registration of noisy and blurred images. Furthermore, we plan to compare the strength of these registration techniques with some well-known image registration methods in terms of speed, quality, and effectiveness in detail.