信息工程与人工智能学院知识库

利用机器学习以及图像处理技术的方法从计算机断层扫描(Computed Tomography，CT)图像中寻找肝脏及肝脏中的肿瘤对于计算机辅助诊断治疗具有重要的意义。近年来，由于计算机性能的提升，深度学习成为医学图像分割的主流。肝脏CT图像是一个三维数据，二维全卷积神经网络（Fully Convolutional Networks，FCN）虽然在语义分割（Semantic Segmentation）任务中表现优秀，但是缺少了利用三维数据中的空间上下文信息这个过程。同时，三维全卷积神经网络虽然将Z轴上的信息与二维平面上的信息进行了综合优化，但由于其参数量巨大，网络性能往往受限于不能采取一定的深度甚至会过度考虑Z轴的信息。

基于以上各自维度的卷积神经网络的不足，本文提出了一种端到端的2.5D卷积神经网络，改进的工作主要由以下两点构成：（1）在二维全卷积神经网络中加入三维卷积部分同时提取空间特征，模型可以同时接受二维和三维图像的输入，并且二维卷积层和三维卷积层可以一起参与训练。（2）为了有效的融合二维和三维信息，本文提出了AFM（Attention FusionModule）模块用于将三维信息生成一个注意力掩模与二维特征图相乘，在训练中使模型得到一个重点关注区域。将模型对MICCAI 2017肝脏肿瘤分割挑战赛的测试集进行预测，通过消融实验表明，本文提出的同时提取二维和三维特征并且将它们融合的这种方式使得该混合全卷积神经网络（Fuse multi-dimensional Network，FMD-Net）模型相对于纯2D卷积神经网络和纯3D全卷积神经网络在测试结果指标上得到了明显提升，将2D Unet 和3D Unet在测试集上的肝脏肿瘤预测结果的Dice per case值分别从0.61和0.55提高到了0.662，与此同时将指标Dice global的值分别从0.774和0.729提高到了0.803，并且将2D Unet的在测试集上的肝脏肿瘤预测结果的Precision这个指标从0.193提高了到了0.253，这个指标对于从CT扫描中精准识别到肝脏肿瘤具有重要意义。

The search for liver and tumors in liver from computed tomography (CT) images using machine learning and image processing techniques is important for computer-aided diagnosis and treatment. In recent years, deep learning has become the mainstream of medical image segmentation due to the improvement of computer performance. The CT image of liver is a three-dimensional data, and although the two-dimensional Fully Convolutional Networks (FCN) performs well in the Semantic Segmentation task, it lacks the process of utilizing the spatial contextual information in the three-dimensional data. Meanwhile, although 3D fully convolutional neural networks integrate and optimize the information on the Z-axis with that on the 2D plane, the network performance is often limited by the fact that it cannot take a certain depth or even over-consider the information on the Z-axis due to its huge number of parameters.

Based on the above deficiencies of the respective dimensional convolutional neural networks, this paper proposes an end-to-end 2.5D convolutional neural network with improvements consisting of the following two main components: (1) By adding a 3D convolutional part to the 2D full convolutional neural network to extract spatial features simultaneously, the model can accept inputs from both 2D and 3D images, and the 2D convolutional and 3D convolutional layers can participate in training together. (2) In order to effectively fuse 2D and 3D information, the AFM (Attention Fuse Module) module is proposed in this paper for generating an attention mask from 3D information and multiplying it with the 2D feature map to make the model get a focused attention region during training. The model is predicted on the test set of the MICCAI 2017 Liver Tumor Segmentation Challenge, and the ablation experiments show that the proposed approach of extracting both 2D and 3D features and fusing them makes the hybrid Fuse multi-dimensional network (FMD-Net) model more efficient than the pure 2D convolutional network and pure 3D convolutional network. The Dice per case value for liver tumor prediction on the test set of 2D Unet and 3D Unet is improved from 0.61 and 0.55 to 0.662respectively, while the Dice global value is improved from 0.774 and 0.774 respectively. The Dice global value was increased from 0.774 and 0.729 to 0.803, and the Precision of 2D Unet's liver tumor prediction in the test set was increased from 0.193 to 0.253, which is important for the accurate identification of liver tumors from CT scans.