Evaluation¶

Reference (Ground Truth)¶

Annotation process was handled by a radiology expert and an experienced medical image processing scientist for each individual slice using an open source program 3D Slicer. (See Fig. 2)

Fig. 2: (a-d) Examples of labeling using axial images. (e) Labelling process using multi-planar reconstructions and visualization in 3D slicer. (f) Visualization of annotated (g) portal vein (h) hepatic vein. (h-i) Quality control and continuity check of annotated vessels.

Evaluation Metrics¶

Four evaluation metrics will be used in order to obtain the best results:

1. Sørensen–Dice coefficient: Provides information about the overlapping parts of segmented and reference volumes in mm3. (1 for a perfect segmentation, 0 for the worst case)
2. Connectivity: It is known that the vessels are continuous structures, hence it is expected that the number of connected components in the segmentation results to be very few, ideally one. Therefore, the connectivity measure penalizes the score according to the difference between the number of connected components in both results and ground truth.
3. Area: The ground truth images are manually annotated, and the vessel borders and their widths may show slight variations for each annotator. Accordingly, the area measure evaluates the overlapping areas between the segmentation and ground truth while also providing a tolerance for slight width differences
4. Length: Similar to the area measure, this metric provides some tolerance for slight differences in vessel length.