- Open Access
Methodology for fast interactive segmentation of the peritoneum and diaphragm in multi-modal 3D medical image
Journal of Computational Surgery volume 2, Article number: 4 (2015)
The segmentation of the peritoneum and diaphragm is important for the non-rigid registration and surgical simulation on the abdominal viscera region. However, there has been few works on the peritoneum or the abdominal viscera envelop segmentation. The challenge in segmentation of the peritoneum is caused by its complex shape and connection to the internal abdominal organs with similar intensity value, which limits the feasibility of the deformable segmentation methods. In this paper, we present two semi-automatic tools to perform a fast segmentation of a patient peritoneum and diaphragm based on the low curvature of the peritoneum along cranio-caudal direction. The segmentation of the peritoneum can be achieved by delineating several selected axial slices using 2D B-spline fitting technique, and the remaining slices can be segmented automatically with 3D B-spline interpolation technique. Experiments on the choice of the number of selected slice (NSS) for interactive segmentation are performed and demonstrated that 10–15 slices are enough to reach an accurate segmentation and can be finished within several minutes. The segmentation of the diaphragm is performed in the sagittal view based on the segmentation result of the peritoneum and can be finished within several minutes also. The segmentation duration of these two interactive tools are also evaluated by six users, the experiment shows that they can finish the segmentation within 10 min. The application of the peritoneum and diaphragm segmentation approach for abdominal visualization and registration is also shown. In conclusion, our developed tools for segmenting the peritoneum and diaphragm are efficient and fast and can play an important role for the surgical planning and simulation on the abdominal viscera. This approach can also inspire the segmentation of the other anatomy structures with low curvature.
Registration of abdominal images is extremely important to improve the surgical diagnosis and planning before any intervention. Indeed, much different information is available in arterial CT, venous CT, T1-MRI, and T2-MRI and much more helpful to surgeons if they are visible in a same frame. Practically, this registration is currently still extremely challenging, and there is no work which manages to properly register the entire structures contained in the abdominal viscera.
One of the main issues is related to the sliding between organs and the peritoneum during patient breathing (cf. Fig. 1). Indeed, it is extremely difficult (or impossible) for a patient to exactly reproduce a breathing status between several acquisitions. It is interesting to highlight that even for multi-phase CT acquisitions (arterial, venous, and portal time), which are performed in less than 5 min, the patient-breathing status can be totally different, even if he was told to inspire at the same level. The use of a spirometer might reduce this issue, but it has been already proven in many papers that the breathing status cannot be parameterized by the volume of gas in the lungs only. Indeed, two different contributions of abdominal and thoracic breathing can provide an identical volume of gas in the lungs, whereas the viscera shape and positions are different.
Since the breathing is not reproducible, the position and shape of viscera inside the peritoneum can be very different between two acquisitions, leading to the non-fulfillment of standard assumption of standard intensity-based registration algorithm. Indeed, it is commonly assumed that the neighborhood of each voxel remains differentiable between two images. The discontinuities that appear on the peritoneum boundary because of the sliding of the organs in the peritoneum cannot be properly taken into account. For this reason, several works have focused their work on the design of new regularization approach to handle sliding, with some interesting results on the lungs, as long as their boundary are known, which can be available to a certain extent with appropriate thresholds [1–4]. In the case of the abdominal organs, we already evaluated the superior performance of non-rigid registration by including the peritoneum position knowledge in approach . This confirmed that the peritoneum segmentation is a precious prior knowledge to ensure a proper performance of non-rigid registration in abdominal area. Unfortunately, such automatic delineation of the peritoneum is still unavailable and seems much more difficult to automatize than the lungs only due to the varying anatomical structures along the peritoneum and inside the peritoneum.
We would like to emphasize that the peritoneum segmentation can also be important for breathing and surgical simulation. For instance, Hostettler et al. proposes a breathing model that can be used in radiotherapy and interventional radiology but which relies on a segmentation of the abdo-thoracic viscera and the creation of independent structured grids of the abdominal and thoracic viscera [6, 7]. Bano et. al. developed a patient specific pneumoperitoneum simulator to predict the abdominal organ motion after the gas insufflation [8, 9]. His work relies on the modeling of the thoracic and abdominal viscera, which are easily available if a proper peritoneum segmentation can be realized. Obviously, in case such work had to be used in a hybrid OP room containing CT acquisition device, the segmentation should be performed in a few minutes. All these works clearly show the benefit of the peritoneum segmentation as a prior knowledge for registration or simulation applications. It is interesting to note that the diaphragm delineation is also necessary in the previously mentioned applications and is not as easy as one could imagine (this will be explained in the related work “Related work” section).
Last but not least, the segmentation and visualization of the abdominal organs are critical for clinical application, such as diagnosis and surgical planning [9–12]. The visualization of the abdominal organs is currently achieved with the used of volume rendering technique, available on standard console in radiological department. Due to the existence of the peritoneum, which consists of fat, muscle, and bone, a practical solution is to modify the opacity of the peritoneum to view the inside organs using the transfer function. Although the transparency setting of the peritoneum enables to view the abdominal organs much more clearly, it simultaneously makes some part of the organ transparent since the intensity value between the peritoneum and abdominal organs are very similar. Thus, a preliminary segmentation of the peritoneum can effectively remove or eliminate this issue.
In this paper, we propose a methodology to interactively but quickly delineate the peritoneum and the diaphragm in medical image in order to provide a priori information for registration and simulation algorithm. The remaining of the paper is organized as follows. We firstly present the existing work in the field of semi-automatic segmentation, which are usually dedicated to specific organ only and not to a plurality of organs contained in an anatomical envelop. Secondly, we present our methodology and software interface to perform quick segmentation and provide structured surfaces of both the peritoneum and diaphragm. Since our approach relies on the assumption that their curvature is sufficiently smooth to use interpolation, we analyze the maximum subsampling that can be used, yet that still fulfill accuracy constraints. In a third section, we thus provide experiments with patient data showing that accurate delineation and surface model of the peritoneum and diaphragm can be provided in less than 15 min.
As far as we know, there has been few works on the peritoneum or the abdominal viscera envelop segmentation. Shimizu intended to simultaneously delineate multiple abdominal organs on non-contrasted CT images . The abdominal cavity is roughly extracted using an active cylinder model , but the extraction result is not reported.
Ding et al.  proposed to use a Gaussian mixture model (GMM) to build the intensity value distribution of the abdominal wall, and a 3D flipping-free deformable model is adopted to expand iteratively to the inner boundary of the abdominal wall. The voxels between the skin and bone are identified and used as a sample to estimate the intensity value distribution of the entire abdominal wall. However, there are many cases where the intensity value of the inner abdominal organs is similar to the wall muscles, and the region between the rib and skin is relatively small compared with the entire abdominal wall; the estimation is thus not sufficiently accurate.
The deformable segmentation methods, snake  or level set [16, 17], are widely used in the medical image segmentation [18–22] but seem not adaptable for the segmentation of the abdominal wall. Indeed, the snake algorithm iteratively updates the contours based on the edge information and usually converges to the skin in this context.
The level set has a similar limitation: we can observe the segmentation algorithm stops at the skin instead of the abdominal wall . Xu proposed a texture analysis based on Gabor filters combined with the level set to extract the anterior abdominal wall, obviously the segmentation of the posterior abdominal wall is more difficult since it is attached to many tissues inside with variable intensity value . Huang et al.  proposed to adopt a priori shape model and context of the bone localization for the segmentation of abdominal wall. Although they show some interesting qualitative results, they do not quantitatively evaluate the segmentation result on the entire abdominal wall and it clearly appears that their algorithm does not deal efficiently in the region close to the spine and the psoas muscles.
For the diaphragm segmentation, which corresponds to the delimitation between the thoracic viscera (lungs + heart) and the abdominal viscera, there is no work mostly because the lung segmentation is considered as a trivial step. Practically, this is often true, but the thoracic viscera also contain the heart and the frontier between the heart, and the abdominal viscera is much more difficult to extract automatically. Moreover, in case of atelectasis, the lungs cannot be extracted using a simple threshold.
Regarding semi-automatic algorithms or tools for specific organ segmentation, one can classify them in two branches. The first branch contains algorithms, which are based on a priori information. For instance, it can be manually clicked seeds inside and outside the organ of interest that are used to initialize watershed, active snakes [26, 27], or region growing algorithm . The final boundaries usually converge close to high gradient area and are computed through an optimization of a contour that minimizes several criteria (elastic and intensity-based). Since the boundaries of the abdominal wall are in contact with almost all abdominal organs (lungs, liver, spleen, stomach, bowel, colon) and since it contains highly different structures (bones, muscles, fat, cartilage), this kind of approach is not adapted. The gradient value which separate the abdominal wall to the viscera is varying from zero (typically the liver or the stomach that can have identical gray value in CT image) to several hundreds.
The second branch contains all interactive tools, which allow interpolating a shape from points, lines, or curves manually drawn on the 3D medical image. In such software, the user usually draws several curves in several slices (successive or not), and an algorithm computes the best surface that passes through all curves [29–31]. The surface is usually parameterized by a NURBS [32, 33] and can be closed or not, enabling typical organ surface drawing or open surface. The created surface can be further locally updated or refined using intensity-based method or statistical model. If no further improvement is planned, this means that the control points/lines/curves only define the shape of the final segmentation. Whereas organ surface usually corresponds to an individual organ, manual delineation of open surface can be used to separate two organs that were segmented together with an automatic tool. The main difference between our work and this second branch is not linked to the interpolation techniques but to the human machine interface and the update strategy of the segmentation result, which can be checked in almost real-time in our case.
Since the peritoneum and diaphragm have somewhat smooth curvature, we believe that an interactive tool, which automatically generates a surface from relevant points or lines drawn by the user in some slices only, can be sufficient to properly interpolate the whole peritoneum and diaphragm. Two questions can then be raised regarding this type of approach. What is the minimum number of necessary points/lines that have to be drawn by the user in order to reach sufficient segmentation accuracy? Given a good delineation in one slice (axial for instance), what could be a good strategy to efficiently adapt this delineation to the peritoneum in a further new slice?
Overview and contributions
In this paper, we present two semi-automatic different tools to perform a fast segmentation of a patient peritoneum and diaphragm. Our first contribution is related to the two interface designs, which are specific to the peritoneum and the diaphragm and allow to reduce the necessary time to create the control points, from which the interpolation is performed. These tools allow not only to provide a binary mask of both structures but also to create a nicely structured mesh dedicated to further application like real-time simulation.
The second contribution is various experiments on patient data that assess on average the minimum number of slices, which are necessary to provide a proper segmentation that would not be significantly more accurate with more slices. We will also show that the learning curve is quite fast and that a user can perform the segmentation of both the peritoneum and the diaphragm in less than 10 min. The image data visualization and the mesh generation are implemented in the software VRMed of IRCAD .
Peritoneum segmentation interactive tool
Given an abdominal 3D image, the outline of our segmentation tool is as follows. The user selects some axial slices in which he interactively delineates the peritoneum using control points linked by a 2D B-spline. Then, the remaining slices are automatically segmented in real-time using a 3D B-spline-based interpolation technique. Finally, the user checks these interpolated delineations and add one or more control slice where he thinks the interpolation is not sufficiently accurate. The detailed process is described hereafter.
Step 1: the user selects the first axial slice. Usually, this slice is approximately in the middle position of the image. The choice of this first slice can be quite important and is discussed later in the next section. Then, the user adds control points sequentially along the boundary of the considered region here the abdominal viscera. The number of control points is also discussed in the next section. Once all control points are added on the boundary, a 2D B-spline-based fitting technique is adopted to form a curve linking all adjacent points (cf. Fig. 2a). The B-splines being updated in real-time, the place of control points can be adjusted in order to improve the created curve accuracy.
Step 2: Once this first delineation is finished, the user selects the next slice image. The first created curve and its control point are automatically copied on the current slice (cf. Fig. 2b). If the user did not select a new slice too far from the first one, some small modifications only are sufficient to update the curve which delineates the abdominal viscera. Although allowing the user to modify each control point independently seems to be the standard option, we choose another option in two stages that practically makes the delineation faster. The first stage consists in allowing a linear expansion or shrink along the X- or Y-axis of the control point set. The software displays a rectangular-bounding box around the control points, and the user can independently modify each box side, the remaining ones being static. From the new position of the moved side, the control points are linearly moved in the same direction than the side. The control points close to the moved side will have the same motion than the moved side, and the control points close to the opposite side will remain at the same position (cf. Fig. 2c; where the bottom side of the box has been slightly moved toward the bottom). Usually, the position of the control points is sufficiently accurate after this first stage. In case some local modifications are still necessary, the user passes to the second stage. During this second stage, the user can modify at the same time the position of a set of consecutive control points: the user selects 2 control points surrounding the zone which has to be refined, they will remain static (cf. Fig. 2d, where the two fixed control points are set in red, the red segment indicating the part of the whole curve, which will be moved). Then, he clicks on the frontier that he considers proper, and all the control points between the 2 fixed control points will move so that the curve reaches the clicked point (cf. Fig. 2e). The motion of each control point between the fixed points is linearly dependent to its distance to the mouse click. If the control point is the closest point on the curve to the mouse click, it will move exactly on the mouse click. If the control point is close to one of the fixed point, he will almost not move. The motions of the other control points are linearly computed from their weighted distance between the closest fixed point and the closest point on the curve to the mouse click. Depending on the necessary refinement, the user moves simultaneously 3 to 4 points only or sometime 6 to 8 when a consistent motion occurs on a specific zone.
Step 3: the user selects a new axial slice, on which a copy of the control points from the closest segmented axial slice is performed. Then, the user can refine the control point positions using the two tools described earlier. After delineation of at least four slices with above tools, the boundary of abdominal viscera in all axial slices comprised between the first one and the last one (along cranio-caudal axis) can be automatically generated with the use of 3D B-spline-based interpolation technique. More precisely, since each control point set in each control slice has the same number of points N, it is easy to match each point of the ith slice to its corresponding point in the jth slice (whatever j). Thus, N 3D B-splines are generated and their intersections with the jth slice generate the control points in the jth slice. From the control points in each slice, a 2D curve is finally generated in all axial slices comprised between the first one and the last one. Finally, a 3D mesh can be created that corresponds to the peritoneum and is displayed on a lateral window (cf. Fig. 2f). Since all curves are continuous B-splines, the user can independently select the density of grid points along axial 2D curves and cranio-caudal 3D curves. The resulting mesh can thus be as dense as needed by the further application. Practically, the number of points on the axial 2D curves is set to 100 to ensure a smooth surface visualization.
Step 4: The final optimization step is required since the generation of most of 2D axial curves is based on the 3D interpolation technique. The optimization includes two aspects, one is the accuracy of the segmentation and the second one is the smoothness of the peritoneum. In the lateral window which shows the image and the surface mesh of the peritoneum, we can conveniently and quickly check the accuracy and smoothness by moving the slices and detect any mistake. For instance, if the ribs are not totally removed in some slices, we can directly locate the two slices surrounding this area on which control points had been defined and modify the control point positions. In case this modification is not sufficient, it may mean that the peritoneum is not sufficiently smooth in this area and a supplementary slice should be manually adapted and validated between these two slices using the two tools previously presented. After having defined the new slice, all 3D B-splines are recomputed since the number of control points has increased of 1 in crania-caudal direction. Obviously, if the user has only segmented four axial slices at step 1, he will certainly have to add several supplementary axial slices.
In our approach, the generated curve in the first slice is copied to the two adjacent slices (below and upper in the cranio-caudal direction). These two slices are then copied to the next adjacent slice and so on. Thus, the choice of the first slice for manual segmentation is really important. Based on our experience, if we choose an axial slice below the liver where the peritoneum is quite smooth, then the user may put 15 control points only, which seems sufficient for the current slice. But when we slide to an axial slice close to the lungs, the curve may not perfectly fit the border of the peritoneum even if we modify the position of the 15 control points, since borders with high curvature occur close to the ribs. It thus requires the curve to have stronger local fitting capability, namely to increase the number of control point, which contradicts our design principle. On the other hand, if we put too many control points, for instance 50, the distance between control point of B-spline is very small (<20 pixels) and it is difficult to generate a smooth curve which fit the border very well. Thus, we need to compromise the number of control point and the smoothness of the segmentation result.
We suggest the following principle for the first slice selection:
The first slice should contain the lower part of the liver and the upper part of the psoas muscles
The number of control points of the first slice (and thus of all slices) should be between 20 and 25
The frequency of control points should be higher close to the spine (area with relatively higher curvature), typically 5 points
The two tools we have designed move several control points at the same time. We have adopted this strategy because the border of the peritoneum curve in axial view and in cranio-caudal direction is very smooth. Thus, the variation of corresponding 2D and 3D interpolated curves should also be smooth. If we refine each control point independently, it will probably reduce the smoothness along the cranio-caudal axis since the user will not take care of their proper alignment in this direction. In addition, the manipulation of control points in group is more efficient and quicker compared to one by one operation. Thus, we expand or shrink the curve in group instead of point by point.
Diaphragm segmentation interactive tool
In this section, we describe the interactive tool we have developed to quickly delineate the diaphragm. The principle is based on the same idea; the curvature of the diaphragm is rather smooth, and thus, the number of sample on its frontier can be limited even if millimeter accuracy is necessary. Although it is possible to use our tool without any prior peritoneum segmentation, it is preferable to have one and this will be assumed in the later.
Step 1: the user firstly selects a specific axial slice, so-called in the remaining of the paper axial plane of reference (APR), which is the highest axial slice in the cranial direction that does not contain lung tissue. The software then automatically extracts from the previous peritoneum segmentation, the 2D curve already delineated in this specific slice, and selects 15 sagittal slices uniformly located between the right and left extremities of the 2D curves (cf. Fig. 3a). The user will have to perform the diaphragm segmentation in these sagittal slices (one example sagittal slice is provided in Fig. 3b), the intersection of these sagittal slices, and the 2D curve from the APR providing the beginning and end point of the diaphragm segmentation (cf. the two blue circles in Fig. 3b).
Step 2: The software shows each sagittal slice one by one so that the user delineates the diaphragm boundary on each of them. The standard process is to click at one extremity and to draw the boundary as long as the mouse click is activated until the user reach the second extremity and then release the click action. We choose another solution: we draw 12 rays, which intersect in the middle of the segment, defined by the intersection of the sagittal slice and the 2D peritoneum delineation in the APR (cf. the yellow circle in Fig. 3b). Then, the user presses the “ctrl” keyboard and moves the mouse along the boundary until the mouse crosses the rays, in which length is automatically reduced to fit the mouse position. The choice of this technique over the standard one is discussed after the step descriptions. In fact, most of the time, the ray length can be automatically computed using the high gradient between the diaphragm and the lungs, reducing the user work to few points and for some sagittal slice to a check step only (the blue circles in Fig. 3c show the automatic ray length computation to fit the diaphragm boundary). In other cases, several rays are crossing the peritoneum, directly providing its length (cf. yellow circle in Fig. 3c).
Step 3: Once all sagittal slices have been processed by the user, the software automatically computes in each sagittal slice, the 2D spline passing through all ray end points, and oversamples the curve from 12 points to 100 ordered points (which corresponds to the number of sampled points on the peritoneum delineated in the APR between the two extremities). Then, considering the kth points of the 2D spline in the 15 sagittal slices, the software computes the 3D spline that connects these 15 points and oversamples it from 16 points to 100 points. An accurate quad mesh, describing the diaphragm position, can finally be computed using all the generated points from the two oversampling process (cf. Fig. 3d). Practically, the creation of the 2D and 3D splines is not possible to be close to the points on the APR, since a further point is missing in the caudal direction. To overcome this problem, we artificially add a supplementary differential constraint on these points, i.e., the tangent extracted from the peritoneum segmentation along the sagittal plane for 2D splines and along the middle frontal plane for 3D splines.
Finally, from the two previous segmentations, two masks can be generated and concatenated to create an accurate 3D image containing thoracic and abdominal viscera only, by replacing the voxel values inside these two masks by −1024. Obviously, corresponding 3D surface meshes can be created for simulation applications [6, 8].
The choice of the APR is more motivated by the liver and spleen frontiers than by the real diaphragm position. Indeed, the diaphragm is not so important in our case and is not always visible in the medical image. Regarding the number of sagittal slice (15) and of samples (12) in each sagittal slice, they were chosen empirically and verified later on patient data as an excellent compromise that ensures a very accurate interpolation of the upper viscera boundary.
The choice of the method allowing to select the ray cast length is certainly one of the most important in our context, where duration should be reduced as much as possible. We originally decided to let the user click and draw the frontier, as it is the case in most of standard software. However, we quickly notice that the user often has to redo his drawing when he loses his mental focus, even if almost all the drawing is perfect. Obviously, an eraser tool could allow to correct the mistaken area, but this action also takes time. The first big advantage of our approach is that it allows correcting extremely quickly the area where a mistake occurs only. The second advantage comes from the idea of using a keyboard control to validate the ray length instead of using the mouse left button. Indeed, it removes the hand tremor when the user clicks on the mouse and maintains pushed the left button. We observed that all users needed two to three less time to validate the diaphragm frontier with our interface.
Evaluation of the number of necessary slices and tool efficiency
We have argued that delineating all slices was not necessary for pneumoperitoneum and diaphragm modeling, particularly because nowadays resolution of medical imaging device is close to millimeter. However, how many slices should one select to provide an accurate 3D surface model, which has a comparable accuracy with a delineation of all slices?
The purpose of this section is to quantitatively answer this question. In particular, we want to find the best trade off so that one has to delineate as few slices as possible, keeping in mind that if too few slices are chosen, the segmentation result might not be satisfying. We will also obviously report the time needed by our method to perform both delineation and evaluate the learning curve of our two segmentation tools.
Experimental set up
Using 20 patient CT data (resolution of 512 × 512 × 92 with voxel size of 0.961 × 0.961 × 1.8 mm), our medical staff perform segmentation with our peritoneum segmentation tool for each of them using 50 slices. The length of acquired abdominal-thorax volume data was along z axis and patient size is on average 45 cm. The patient selection was performed randomly among an important database of patient and has an important variability. An expert had previously defined that the segmentation with 50 slices (roughly 1 slice/cm) could be considered as ground truth due to the low curvature of the abdominal wall. This was mathematically confirmed using Shannon theory. Indeed, we have computed for each patient the maximum frequency with a significant magnitude contained in the frontal and sagittal 2D projections of the 3D B-splines, considered in this context as a spatial signal in mm. The mean maximum frequency f max (averaged on our 10 patient data) is close to 0.02 Hz; thus, if the sampling is performed at f sample = 2 × f max (i.e., 0.04 Hz), one sample every 25 mm should be sufficient to properly interpolate the entire skin curvature.
Then, a selection of 20, 10, and 5 slices uniformly sampled among the 50 delineated slices was carried out and a dense 3D mesh S50, S20, S10, and S5 were created for each patient from the segmentation with 50, 20, 10, and 5 slices, respectively, using the 3D-spline-based interpolation described in the previous section. Finally, the surface models S20, S10, and S5 can be compared to the ground truth S50, and the difference between them can be measured by computing the distance from the vertex of the surface models to the closest point belonging to S50.
Figure 4 shows the distance histogram between meshes S20, S10, and S5 with S50 on two patient data. We can see that the distance distribution of points is almost the same for S20 and S10 (peak around 0.6 mm). But for a number of selected slice (NSS) equals to 5, there are many points which distance is larger than 0.8 mm. We also calculate the total average distance and standard deviation between surfaces S20, S10, and S5 with S50 on eight patient data (cf. Table 1). It clearly shows that the mean error is reduced from 1.27 mm (NSS = 5) to 0.84 mm (NSS = 10) which corresponds to the voxel size and thus to the ground truth accuracy. However, there is only a slight improvement in accuracy for the NSS increased from 10 to 20.
From these statistic charts and tables, we can conclude that the NSS for interactive segmentation should be chosen below 20 but above 10.
Evaluation of the interactive segmentation duration
In this section, we have evaluated the time needed by six users to perform the segmentation of the peritoneum and diaphragm of the 20 medical images of patient abdomen from the previous section. We removed from the average value the two first segmentations since they were much longer due to the learning phase. Users were two engineers, two surgeons, and two radiology technicians.
Results show that the peritoneum segmentation is performed on average in 363 s (5 min.) and that the diaphragm segmentation is performed in 215 s (Table 2). No significant difference was reported between the three classes of users. This evaluation also shows that the abdominal wall can finally be fully segmented with an excellent accuracy and in a very reasonable time compared to clinical workflow constraints.
Evaluation of the peritoneum and diaphragm removal for visualization and registration
The removal of the peritoneum and diaphragm in medical images is important for many applications. In this section, we have selected two applications to show the benefits of our methodology: one is the volume rendering of the abdominal organs and the second one is the non-rigid registration of abdominal viscera.
Qualitative evaluation of visualization of patient data
The volume rendering display of medical data is now available on each radiological console and allows a 3D visualization of the patient data, which can be extremely useful to plan resection planes in case of liver tumor dissection or to assess relative positions of vessels. However, since the abdominal wall (which contains the spine, ribs, and muscles) cannot be automatically removed from the medical data; it is often difficult to get a clear understanding of the patient anatomy. We show below on two patient data the benefit of abdominal wall removal. We compare the volume rendering visualization of the original image with the same image after removal of the peritoneum and diaphragm.
The first patient has several tumors in the liver, and a proper visualization of the liver shape is important to assess the position of the resection planes. We show in the left in Fig. 5 the original image where one can see that the ribs and cartilage do not allow to clearly see the liver shape. We highlight that no standard transfer function available in the software allows to completely remove the bones without degrading the image. We have also tried to manually modify the transfer function, but results are still not acceptable for a clinician. On the contrary, the image without the abdominal wall (cf. right Fig. 5) clearly allows a good visualization and understanding of the liver shape.
The second patient has a tumor in the left kidney, and assessment of the positions of the renal artery and vein is important. This is usually done from a back point of view. The left image in Fig. 6 shows that the spine prevents the user from seeing clearly structures surrounding the kidneys. Again, no transfer function allows to remove the spine from image. We have thus tried to crop the volume in order to remove the spine (using the standard bounding box cropping tool: it removes here from the volume rendering a stack of frontal slices in the patient back that contains the spine). However, such process also degrades the structures we are interested in since they are also contained in the frontal slices embedding the spine (cf. middle Fig. 6). Obviously, this problem is overcome if the abdominal wall is removed with our method, as can be checked at the right in Fig. 6.
Quantitative evaluation of non-rigid registration on patient data
We show in this subsection that the removal of the abdominal wall provides a significant benefit for multi-modal registration of patient viscera. This kind of registration can be useful for multi-phase CT, where arterial phase and venous phase images are acquired at a different breathing stage, for CT/MRI information fusion, and for tumor follow-up in several contrasted CT acquired at different times. We have selected 10 pairs of contrasted arterial and venous CT scan of 10 patients. Registration is firstly performed between original arterial image and venous image. Then, a new registration is computed using the arterial and venous image, where the abdominal wall and thoracic viscera have been removed using our two segmentation tools.
A state-of-the-art non-rigid registration algorithm was implemented. A global rigid and affine registration was firstly performed to compensate the body position variation along superior-inferior, anteroposterior and left-right directions. Then a B-spline-based transformation combined with Mattes Mutual Information (MMI) as a similarity metric was used to obtain an optimized transformation for local deformation between multi-phase images. All image processing and registration algorithms are performed on a computer with Intel® Core (TM) i7-2600(3.40GHz) processor with 16GB of RAM. The algorithms are compiled with visual studio 2012 in 64 bits.
Four organs, the liver, the left/right kidney, and the spleen, are segmented semi-automatically by experts. The surface mesh of these organs from target and source image is constructed using marching cube algorithm. The deformation field of registration result is applied on the surface mesh from source image; thus, the new mesh after registration can be obtained. Then, the mesh distance error of the organs from source image to the target image is calculated. The average surface distance for each organ is illustrated in Fig. 7. It clearly shows the distance of four organs before registration (BR) is very big (blue bar), then a little improvement is obtained after the non-rigid registration, but the result accuracy is not clinically questionable, particularly for the liver and spleen since the error distance is larger than 3 mm. On the new images without the peritoneum and thoracic viscera, the error distances of the four organs are significantly improved. The distance of the left/right kidney, the spleen, and the liver is indeed within 1 mm which equals the voxel size.
We also show qualitatively on one patient CT data the benefit of our approach. Left image in Fig. 8 shows the target and source images on the top row and the registration result in the bottom right. Red arrows show that the registration did not perform well at all: the source image after registration is almost the same than the original source image. On the contrary, the green arrows in the right of Fig. 8 show that the source image has been consistently registered and fit the target image.
In this paper, we present two interactive but fast tools for segmentation of the peritoneum and diaphragm. In particular, we carefully describe our methodology and explain why it allows the user to perform both segmentations in a very short time. We provide experimental results on 20 patients, which show that a low number of selected slices (typically 15) are enough to reach an excellent accuracy and can be completed within 10 minutes. We also observed that practicing on one or two patient data was enough for the new users to be efficient. On average, we evaluated with six users that both segmentation could be finished within 600 s (5 min). We finally provide visualization and registration results using our segmentation methods, which exemplify how useful our tools can be in several medical applications. In the future, we will try to integrate template segmentation as a prior in our software to further guide the user during his manual task.
To further reduce the number of selected slices, we also plan to adapt automatically the distance between the selected slices (along the Z-axis for the skin and along the X-axis for the diaphragm) depending on the local curvature of the skin and diaphragm, which can be usually segmented in CT images using a simple threshold. This would be particularly suited for patients with specific pathologies like abdominal or diaphragmatic hernia, which may locally result in much higher curvature than with standard patient and would require a higher spatial sampling frequency in this area.
axial plane of reference
number of selected slice
Pace D, Aylward S, Niethammer M. A locally adaptive regularization based on anisotropic diffusion for deformable image registration of sliding organs. 2013.
Schmidt-Richberg A, Ehrhardt J, Werner R, Handels H. Slipping objects in image registration: improved motion field estimation with direction-dependent regularization. Medical image computing and computer-assisted intervention: MICCAI International Conference on Medical Image Computing and Computer-Assisted Intervention. 2009;12(Pt 1):755–62.
Schmidt-Richberg A, Ehrhardt J, Werner R, Handels H. Fast explicit diffusion for registration with direction-dependent regularization. Biomedical Image Registration. Springer; 2012. p. 220–8.
Schmidt-Richberg A, Werner R, Handels H, Ehrhardt J. Estimation of slipping organ motion by registration with direction-dependent regularization. Med Image Anal. 2012;16(1):150–9.
Zhu W, Nicolau S, Soler L, Hostettler A, Marescaux J, Rémond Y. Fast segmentation of abdominal wall: Application to sliding effect removal for non-rigid registration. Abdominal Imaging. Computational and Clinical Applications. Springer; 2012. p. 198–207.
Hostettler A, Nicolau S, Rémond Y, Marescaux J, Soler L. A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Prog Biophys Mol Biol. 2010;103(2):169–84.
Hostettler A, Nicolau SA, Soler L, Rémond Y, Marescaux J. A real-time predictive simulation of abdominal organ positions induced by free breathing. Biomedical Simulation. Springer; 2008. p. 89–97.
Bano J, Hostettler A, Nicolau S, Cotin S, Doignon C, Wu H et al. Simulation of pneumoperitoneum for laparoscopic surgery planning. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2012. Springer; 2012. p. 91–8.
Shimizu A, Ohno R, Ikegami T, Kobatake H, Nawano S, Smutek D. Segmentation of multiple organs in non-contrast 3D abdominal CT images. Int J Comput Assist Radiol Surg. 2007;2(3–4):135–42.
Bano J, Hostettler A, Nicolau S, Doignon C, Wu H, Huang M et al. Simulation of the abdominal wall and its arteries after pneumoperitoneum for guidance of port positioning in laparoscopic surgery. Advances in Visual Computing. Springer; 2012. p. 1–11.
Soler L, Forest C, Nicolau S, Vayssiere C, Wattiez A, Marescaux J. Computer-assisted operative procedure: from preoperative planning to simulation. European Clinics in Obstetrics and Gynaecology. 2006;2(4):201–8.
Soler L, Nicolau S, Pessaux P, Mutter D, Marescaux J. Real-time 3D image reconstruction guidance in liver resection surgery. Hepatobiliary surgery and nutrition. 2014;3(2):73.
Okumura T, Yamamoto S, Matsumoto M, Takeno Y, Iinuma T, Matsumoto T. The lung region extraction in the chest CT images by the active cylinder model. 1998.
Ding F, Leow WK, Venkatesh S, editors. Removal of abdominal wall for 3D visualization and segmentation of organs in CT volume. Image Processing (ICIP), 2009 16th IEEE International Conference on; 2009: IEEE.
Kass M, Witkin A, Terzopoulos D. Snakes: Active contour models. International journal of computer vision. 1988;1(4):321–31.
Sethian JA. Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science. Cambridge university press; 1999.
Fedkiw SOR. Level set methods and dynamic implicit surfaces. 2003.
Vese LA, Chan TF. A multiphase level set framework for image segmentation using the Mumford and Shah model. International journal of computer vision. 2002;50(3):271–93.
Lie J, Lysaker M, Tai X-C. A binary level set model and some applications to Mumford-Shah image segmentation. Image Processing, IEEE Transactions on. 2006;15(5):1171–81.
Paragios N, Deriche R. Coupled geodesic active regions for image segmentation: A level set approach. Computer Vision—ECCV 2000. Springer; 2000. p. 224–40.
Li C, Huang R, Ding Z, Gatenby J, Metaxas DN, Gore JC. A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI. Image Processing, IEEE Transactions on. 2011;20(7):2007–16.
Brox T, Weickert J. Level set based image segmentation with multiple regions. Pattern Recognition. Springer; 2004. p. 415–23.
Vandemeulebroucke J, Bernard O, Rit S, Kybic J, Clarysse P, Sarrut D. Automated segmentation of a motion mask to preserve sliding motion in deformable registration of thoracic CT. Med Phys. 2012;39(2):1006–15. doi:10.1118/1.3679009.
Xu Z, Allen WM, Baucom RB, Poulose BK, Landman BA. Texture analysis improves level set segmentation of the anterior abdominal wall. Med Phys. 2013;40(12):121901. doi:10.1118/1.4828791.
Huang W, Quan L, Lin Z, Duan Y, Zhou J, Yang Y et al., editors. Abdominal wall extraction using constrained deformable model and abdominal context. Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE; 2014: IEEE.
Williams DJ, Shah M. A fast algorithm for active contours and curvature estimation. CVGIP: Image understanding. 1992;55(1):14–26.
Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage. 2006;31(3):1116–28.
Adams R, Bischof L. Seeded region growing. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 1994;16(6):641–7.
Wimmer A, Soza G, Hornegger J. Two-stage semi-automatic organ segmentation framework using radial basis functions and level sets. 3D Segmentation in The Clinic: A Grand Challenge. 2007:179–88.
Rogers DF. An introduction to NURBS: with historical perspective. Elsevier; 2000.
Rueckert D, Sonoda LI, Hayes C, Hill DL, Leach MO, Hawkes DJ. Nonrigid registration using free-form deformations: application to breast MR images. Medical Imaging, IEEE Transactions on. 1999;18(8):712–21.
marescaux j. IRCAD. 1996. http://www.ircad.fr/.
This research is supported by the China Scholarship Council (CSC).
The authors declare that they have no competing interests.
AH was responsible for the development of the segmentation tools. WZ conducted evaluations of segmentation tools, designed the experiments, analyzed the results, and was responsible for the writing and modification of the manuscript. SN helped design the experiments and draft the manuscript. LS and JM participated in the analysis of experiments results. All authors read and approved the final manuscript.
AH is a researcher in the IRCAD institute. WZ was a PhD student in the University of Strasbourg and worked in the Research and Development (R&D) department of the IRCAD institute. SN is the research director of the IRCAD institute. LS is the manager of the R&D department team of the IRCAD institute. JM is the president of the IRCAD institute.