Hypothesis / aims of study
Postoperative imaging to localize and measure synthetic pelvic implants, such as mid-urethral slings and/or trans-vaginal meshes, is an important new task in modern pelvic floor reconstruction. This is due to the popularity of various pelvic floor repair procedures involving the placement of such implants and the associated complications, such as dyspareunia, chronic pain and recurrent prolapses. In many instances, these complications are severe enough to warrant additional surgeries to remove these meshes/slings. Hence, an accurate imaging and localization of these implants becomes an absolute necessity, especially if the follow-up surgical procedure is performed years later by a different surgeon. Even for patients who do not have any symptoms or complications, periodic imaging could be helpful in monitoring the change in size of the mesh/sling and its location relative to pelvic organs.
Various modalities can be used to image those pelvic implants, including magnetic resonance imaging (MRI), ultrasonography, computed tomography, and some others. Each of these imaging techniques has its pros and cons (e.g. see [1]), but a common deficiency among all of them is the lack of contrast between mesh/sling and scar tissue. Some recent progress has been made in this direction due to the development of implants from a new material that has high contrast in MRI [2]. However, the current prevalence of implants from conventional synthetic materials dictates the need for improvement of imaging techniques using advanced data processing methods.
Another difficulty encountered by surgeons while using MRI of the pelvic floor is the lack of truly 3D visualization mechanisms. The standard MRI data is presented in the form of sequences of 2D images at multiple depths in three different orientations: transversal (T), sagittal (S) and coronal (C). At the same time, different features of the object can be well pronounced in different orientations. Mentally superimposing this layered information from T, S and C into the same space is an extremely challenging task. It would be much more useful for the surgeon, instead, to have access to a semi-transparent 3D volumetric image, which could be rotated with a mouse.
Therefore, our aim was to develop new mathematical models and machine learning techniques for automatic detection of pelvic mesh in MRI, as well as a mechanism for 3D visualization of the mesh and important anatomic pelvic structures.
Study design, materials and methods
A realistic yet efficient option of acquiring large training data sets is the generation of synthetic data using mathematical models optimized for the specific task at hand [3]. In this work, we followed this approach utilizing the expertise of our diverse group comprised of applied mathematicians, radiologists and pelvic floor specialists. De-identified human MR images from 10 patients (several hundred 2D images per patient) were collected by the medical team and transferred to the mathematicians in DICOM format as part of inter-institutional Material Transfer Agreement (MTA). Images from 5 patients were manually segmented and labeled by the math team and verified by the radiologist. They are currently being used as training data and source of data augmentation (i.e. generating additional synthetic training data). This augmented data set will then be applied to train a Convolutional Neural Network (CNN), which is employed to analyze our MR images. The MR images from the remaining 5 patients will be used as validation and test data sets.
The data augmentation part of our research is in its initial stage; however, some promising directions and ideas have already been identified and are currently pursued by the math team. More specifically, we used optical flow techniques to derive mathematical description of the class of transformations of 2D images of the same patient in fixed orientation sequences. Similarly, we described the transformations of 2D images corresponding to the same depth and orientation across patients. The characterization of these transformation classes allowed us to use many other similar transformations to generate new synthetic image sequences from the same sources.
To create 3D visualizations, we performed segmentation on each 2D image of the same orientation (i.e. (T), (S) or (C)) and labelled the following pelvic floor structures: pubic bone, bladder, urethra, uterus, vagina, bowel, sacrum and the pelvic implant. Then a sequence of such labeled frames was used to synthesize a 3D volume using 3D Slicer (an open source software platform for medical image informatics, image processing, and three-dimensional visualization). The resulting 3D volume was transparent at each voxel except at those corresponding to the labels described above. The image can be rotated with a mouse and provides simple stereo-metric visualization of all important pelvic floor structures and of their relationship.
Additionally, we currently work on obtaining an accurate evaluation of various quantitative parameters related to the implants, such as surface area, shape, dimensions, distance to vital organs, etc. The specific techniques applied in this part of the work include formulation and incorporation of relevant mathematical constraints on the geometry of implants, as well as smart fusion of data extracted from different orientations of MR images of the same subject.
Results
In this preliminary study, we have established a new mathematical framework for automatic segmentation of postoperative MR images of pelvic floor structures, including pelvic implants. Our method is based on machine learning techniques, and its level of accuracy and reliability depends on the size of the training data, which is currently very limited. To augment our training data, we have introduced a novel task-specific approach to generate synthetic training data, and are in the process of its implementation. Using 2D image segmentation (A) and labeling, we developed a mechanism for enhanced 3D (B) stereo-metric visualization of MR images, which are often more desirable for physicians than slice-by-slice image sequences provided by conventional DICOM viewers.
Interpretation of results
Recent advances in artificial intelligence and machine learning have revolutionized many fields of human endeavor. Particularly impressive results have been produced in various tasks of image processing, including automatic image segmentation and feature extraction. To the best of our knowledge, the application of these techniques to imaging of pelvic implants has been very limited.
One of the primary reasons that curtail the progress in this area is the relatively small pool of available data needed to train the algorithms. While creation of such large data sets from thousands of patients may be theoretically possible, in practice it will be a mammoth undertaking, which most probably will require years of work, as well as collaboration and coordination between multiple institutions. Instead, we chose to generate synthetic data using mathematical models optimized for the specific task at hand. After completion of this step, we will use that data to train a convolution neural network to perform automatic segmentation of postoperative MR images of pelvic floor structures, including pelvic implants. Our initial experience with this approach demonstrated its feasibility and yield to advance the state-of-the-art in MRI of pelvic implants.