This PDF file contains the front matter associated with SPIE Proceedings Volume 12469, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

Federated learning is a paradigm that enables organizations to collaborate on machine learning projects without sharing sensitive patient data. This lowers the barrier to international collaboration to build generalizable models that mitigate bias by allowing access to larger and more diverse datasets. The talk will highlight key considerations in federated learning and discuss the results of the largest international federation of healthcare institutions that developed state-of-the-art brain tumor boundary detection model using MRI scans from 71 institutions across six continents.

Building Machine Learning models from scratch for clinical applications can be a challenging undertaking requiring varied levels of expertise. Given the heterogeneous nature of input data and specific task requirements, even seasoned developers and researchers may occasionally run into issues with incompatible frameworks. This is further complicated in the context of diagnostic radiology. Therefore, we developed the CRP10 AI Application Interface (CRP10AII) as a component of the Medical Imaging and Data Resource Center (MIDRC) to deliver a modular and user-friendly software solution that can efficiently address the demands of physicians, early AI developers to explore, train, and test AI algorithms. The CRP10AII tool is python-based web framework that is connected to the data commons (GEN3) that offers the ability to develop AI models from scratch or employ pre-trained models while allowing for visualization and interpretation of the predictions of the AI model. Here, we evaluate the capabilities of CRP10AII and its related human-API interaction factors. This evaluation aims at investigating various aspects of the API, including:(i) robustness and ease of use; (ii) visualization help in decision making tasks; and (iii) necessary further improvements for initial AI researchers with different medical imaging and AI expertise levels. Users initially experienced trouble testing the API; however, the problems have since been fixed as a result of additional explanations. The user evaluation's findings demonstrate that although different options on the API are generally easy to understand, use, and helpful in decision-making tasks for users with and without experience in medical imaging and AI, there are differences in how the various options are understood and used by users. We were also able to collect additional inputs, such as increasing information fields and including more interactive components to make the API more generalizable and customizable.

Auto-segmentation of medical images is critical to boost precision radiology and radiation oncology efficiency, thereby improving medical quality for both health care practitioners and patients. An appropriate metric to evaluate auto-segmentation results is one of the significant tools necessary for building an effective, robust, and practical auto-segmentation technique. However, by comparing the predicted segmentation with the ground truth, currently widely-used metrics usually focus on the overlapping area (Dice Coefficient) or the most severe shifting of the boundary (Hausdorff Distance), which seem inconsistent with human reader behaviors. Human readers usually verify and correct auto-segmentation contours and then apply the modified segmentation masks to guide clinical application in diagnosis or treatment. A metric called Mendability Index (MI) is proposed to better estimate the effort required for manually editing the auto-segmentations of objects of interest in medical images so that the segmentations become acceptable for the application at hand. Considering different human behaviors for different errors, MI classifies auto-segmented errors into three types with different quantitative behaviors. The fluctuation of human subjective delineation is also considered in MI. 505 3D computed tomography (CT) auto-segmentations consisting of 6 objects from 3 institutions with the corresponding ground truth and the recorded manual mending time needed by experts are used to validate the performance of the proposed MI. The correlation between the time for editing with the segmentation metrics demonstrates that MI is generally more suitable for indicating mending efforts than Dice Coefficient or Hausdorff Distance, suggesting that MI may be an effective metric to quantify the clinical value of auto-segmentations.

While there are formal guidelines and target dose levels used in treatment planning for radiation therapy, currently plans are created to adhere to these goals based on physician experience and trial-and-error rather than standard quantitative methods. To introduce uniformity and data driven methods into the radiation therapy treatment planning process we create a web based informatics application which uses algorithmic analysis of historical cases to identify and provide treatment plan templates and treatment benchmarking. The system relies on a database of 360 historical DICOM RT objects from University of California Los Angeles and State University of New York Buffalo; Roswell Park as well as the quantitative features we calculate from each case. To quantitatively identify each case we calculated the overlap volume histogram and spatial target similarity in our feature extraction algorithms. A case undergoing treatment planning when uploaded to our web application will have it’s quantitative features automatically extracted and then our similarity matching algorithm which matches cases based on the similarity/dissimilarity of their quantitative features is used to generate a list of similar historical cases from our database which a physician can then use for reference. Our database also stores treatment outcomes which we will use to establish relationships between the anatomy of the tumor and surrounding organs, the treatment and outcome. These identified relationships will be used in benchmarking and treatment plan assessment. The system aims to increase uniformity of methods and introduce data driven practices into radiation therapy treatment planning.

The deployment of deep learning algorithms in clinical practice faces challenges in data privacy and local hardware constraints. This work presents the tools and design choices of a browser-based edge computing framework to address these challenges. We leverage this framework for 3D medical image segmentation from computed tomography and characterize its speed, memory, and limitations across various operating systems and browsers. Our platform deploys deep learning-based segmentation of a 256×256×256 volume with an average runtime of 80 seconds and average memory usage of 1.5 GB on Firefox, Chrome, and Microsoft Edge using consumer-level laptops.

The use of artificial intelligence (AI) in healthcare has become a very active research area in the last few years. While significant progress has been made in image classification tasks, only a few AI methods are actually deployed in clinical settings. A major hurdle in actively using clinical AI models is the trustworthiness of these models. Often, these complex models are utilized as black boxes in which promising results are generated. However, when scrutinized, these models reveal implicit biases during decision-making, such as having an unintended bias towards particular ethnic groups and sub-populations. In our study, we develop a two-step adversarial debiasing approach with partial learning that can reduce the disparity while preserving the performance of the targeted diagnosis/classification task. The methodology has been evaluated on two independent medical image case studies - chest X-rays and mammograms and showed promises in bias reduction while preserving the targeted performance on both internal and external datasets.

The Medical Imaging and Data Resource Center (MIDRC) is a multi-institutional effort to accelerate medical imaging machine intelligence research and create a publicly available data commons as well as a sequestered commons for performance evaluation of algorithms. This work sought to evaluate the currently implemented methodology for apportioning data to the public and sequestered data commons by investigating the resulting distributions of joint demographic characteristics between the public and sequestered commons. 54,185 patients whose de-identified imaging studies and metadata had been submitted to MIDRC were previously separated into public and sequestered commons using a multi-dimensional stratified sampling method, resulting in 41,556 patients (77%) in the public commons and 12,629 patients (23%) in the sequestered commons. To compare the balance obtained in the joint distributions of patient characteristics from use of the developed sequestration method, patients from each commons were separated into bins, representing a unique combination of the demographic variables of COVID-19 status, age, race, and sex assigned at birth. The joint distributions of patients were visualized, and the absolute and percent difference in each bin from an exact 77:23 split of the data were calculated. Results indicated 75.9% of bins obtained differences of less than 15 patients, with a median difference of 3.6 from the total data for both public and sequestered commons. Joint distributions of patient characteristics in both the public and sequestered commons closely matched each other as well as that of the total data, indicating the sequestration by stratified sampling method has operated as intended.

Previous studies have demonstrated that 3-dimensional (3D) volumetric renderings of magnetic resonance imaging (MRI) brain scan imaging data can be used to identify patients using facial recognition algorithms. We have shown that facial features can be identified on SIM-CT (simulation computed tomography) images for radiation oncology and mapped to face images from a database. We now seek to determine whether CT images can be anonymized using anonymization software that was designed for T1W MRI data. Our study examines (1) the ability of off-the-shelf anonymization algorithms to anonymize CT data, and (2) the ability of facial recognition algorithms to then identify whether faces could be detected from a database of facial images. This study generated 3D renderings from open-source CT scans of two patients from The Cancer Imaging Archive (TCIA) database. Data were then anonymized using AFNI (deface, reface, 3Dskullstrip), and FSL (deface and BET). Anonymized data were compared to the original renderings and also passed through facial recognition algorithms (Face_compare, VGG-Face, Facenet, DLib, and SFace) using a publicly available face database (Labeled Faces in the Wild) to determine what matches could be found. Our study found that all modules were able to process CT data in addition to T1W and T2W data and that data were successfully anonymized by AFNI's 3Dskullstrip and FSL's BET: they did not match the control image across all facial recognition algorithms. Our study demonstrates the importance of continued vigilance for patient privacy in publicly shared datasets and the importance of evaluation of anonymization methods for CT data.

Imaging research requires complex data processing and compliance with hospital, state, federal, and international privacy regulations. Such requirements can be particularly challenging for large hospital networks spanning multiple states or countries. In the present study, we combined a governing process for DICOM deidentification and built a software pipeline to support the approved process in a tertiary-care hospital network. This allows researchers to submit large dataset requests and receive fully-compliant files. This study lends evidence that standardized data management at scale is feasible through control policy and fit-to-purpose informatics tools.

Remote Photoplethysmography (remote PPG) enables contactless monitoring of the cardiac rhythm using video cameras. Prior research has shown the feasibility of video-based atrial fibrillation (AF) and/or flutter (Aflutter) detection in some scenarios, but most exclude patient movement. In this work, we investigate the feasibility of detecting these two cardiac arrhythmias in a regular hospital environment using an RGB camera, where patients were not limited in movement during the recording process. Data of 56 patients was collected before and after a scheduled cardioversion treatment. Using the data and machine learning models, we developed three models: First, a model to detect only AF from the data excluding any Aflutter cases. Here we report a sensitivity of 94.5% and a specificity of 89.3% with an AUC of 0.966. Second, a model to classify if a cardiac arrhythmia (AF or Aflutter) is present or not. There we report there a sensitivity of 95.6% and a specificity of 91.2% with an AUC of 0.975. Finally, we develop a multi rhythm model, where we classify the data in AF, Aflutter and sinus rhythm separately. The performance of arrhythmia detection is close to the second model, but we note that the distinction between AF and Aflutter is still a challenge. Here we theorize that remote PPG is more sensitive to noise during Aflutter, which will lead to features in Aflutter which are closer to those of AF. To confirm this, we will extensively review the reason of misclassification of Aflutter as AF in future work.

In this work, we investigated the feasibility of extracting continuous respiratory parameters from a single RGB camera stationed in a short-stay ward. Based on the extracted respiration parameters, we further investigated the feasibility of using respiratory features to aid in the detection of atrial fibrillation (AF). To extract respiration, we implemented two algorithms: chest optical flow (COF) and energy variance maximization (EVM). We used COF to extract respiration from the patient’s thoracic area and EVM from the patient’s facial area. Using capnography as the reference, for average breath-to-breath rate estimation (i.e., 15-second sliding windows with 50% overlap), we achieved errors within 3 breaths per minute with COF and within 3.5 breaths per minute with EVM. To detect the presence of AF in the respiratory signal, we extracted three respiratory features from the derived COF measurements. We fed these features to a logistic regression model and achieved an average AUC value of 0.64. This result showcases the potential of using camera-based respiratory parameters as predictors for AF, or as surrogate predictors when there is no sufficient facial area in the camera’s field of view for the extraction of cardiac measurements.

Background: Annually and globally, cardiovascular diseases yield the death of 17.9 million people. Continuous and unobtrusive monitoring of vital signs supports an early detection of abnormalities and diseases, such as atrial fibrillation. Here, we analyze capacitive electrocardiography (cECG) recorded in an armchair at home. However, processing such data is challenging, as body movements and other artifacts disturb the signal quality. Methods: In this paper, we suggest video-based pose estimation to assess the reliability of cEGC. In 20 subjects, we measured reference and capacitive ECG synchronized with a video recording for key-point-based movement analysis with the OpenPifPaf pose estimation algorithm. We considered all 17 human body joints to compute a movement index and label all data in windows of 5 s as reliable vs. unreliable, according to that index. Then, we compared the heart rates obtained from complete and reliable cEGC windows with the corresponding windows from the reference ground truth ECG. Result: The left and right hip joints are most significantly influencing the signal’s quality. In addition, the joints’ movement distance from the original position limited to the range 460.84 pixels and 382.22 pixels, respectively, deliver a reliable cECG signal. Conclusion: Video-based pose estimation delivers reliable and unreliable periods of cECG recordings and improves continuous health monitoring at home.

Sports medicine research is a developing field which uses a processed combination of videos, force data, and accelerometer data. In radiology data is processed within the radiology workflow before being stored in PACS. Both processed and raw data can be uploaded to a central database. Currently, in the biomechanics workflow this data must be processed by researchers on local computers. There is currently no centralized method to process this data, each researcher must use unique scripts which require specific computer environments. Also, once data has been stored in a central database there is no method to query that data for further processing. Processed data is a valuable tool for researchers to educate athletes and coaches. Therefore, there is a need for a central, standardized system which can process data within the biomechanics workflow. The integrated biomechanics informatics system (IBIS) is an ideal choice for this system as it already provides a centralized database used by biomechanics researchers. Using an imaging-informatics approach we have developed a web application which allows users to query, retrieve, and process data on the IBIS system. The web application’s efficacy will be measured by testing expert and novice users on the speed and ease of using the web application and comparing the results of web processing to local processing. In the future this web application can be extended to host multiple types of biomechanics data processing on the IBIS system.

In Japan, people wear personal radiation dosimeters to enter radiation controlled areas for occupational radiation exposure management. The cumulative value for one month is the occupational radiation exposure dose value. Managers want to make appropriate announcements to medical staff with high occupational radiation exposure doses to make them awareness of radiation protection, but it is difficult to do so based on occupational radiation exposure doses alone. In this study, we developed an occupational exposure information system that collects and analyzes sensor information and DICOM (Digital Imaging and Communications in Medicine) data to know the situations (location, time, etc.) at that time. In the trial use of the system, it was possible to know the situations (location, time, etc.) at that time.

Primary angle closure disease (PACD) is a leading cause of permanent vision loss worldwide, so early treatment of patients suffering from symptoms of PACD is crucial to prevent vision loss. Gonioscopy is the current clinical standard for diagnosing PACD. However, gonioscopy is a qualitative subjective assessment method. Thus, there is a need for a quantitative method to diagnose PACD. Anterior Segment Optical Coherence Tomography (AS-OCT) is an imaging modality which produces images of anterior structures such as the anterior chamber angle. Adoption of AS-OCT has been slow due to AS-OCT analysis not being standardized and inefficient. Currently, users must annotate each image by hand using proprietary software and use expert knowledge to diagnose PACD based on the key features annotated. Using an imaging-informatics based approach on a dataset of over 900 images we have developed a system to streamline and standardize AS-OCT analysis. This system will be DICOM compatible to promote standardization of AS-OCT images. This system will be attached to a HIPAA compliant database and will require a secure login to protect patient privacy. We have developed a streamlined approach towards annotating key features in AS-OCT images which will be used to validate the results produced by SimpleMind – an open source software framework supporting deep neural networks with machine learning and automatic parameter tuning. SimpleMind is integrated into the system to increase the efficiency of analyzing AS-OCT images and eliminate the need to annotate images for clinical diagnosis.

This study aims to refine an automated workflow for neuroimages that generates three-dimensional (3D) point clouds in the Polygon File Format (.ply) to be deployed into augmented reality (AR) head mount displays (HMD). Our current work involves enhancing and refining core features, along with improving the point-cloud application to optimize the brain MRI intake. Our brain image segmentation algorithm and web-based point cloud generator show promise for clinical workflows, where high-quality point-cloud AR models can be generated from patient MRIs.

Ergonomics for image-guided procedures can be improved by using mixed reality headsets. Such headsets offer the ability to position holographic monitors that display information, such as an ultrasound stream, within the operator’s field of view during procedures. However, one of the barriers of clinical adoption of mixed reality headsets is high latency of information projected on to the headset. Upwards of 40% of the overall latency of the entire system can be due to video cards that capture procedural imaging for wireless streaming. The costs of the video cards can vary widely, from as low as $20 to upwards of several hundreds of dollars. The latencies of four separate video cards with a range of costs were evaluated. Based on these results, we propose an ideal tradeoff between latency and costs for clinical use of wirelessly mirroring procedural imaging into mixed reality headsets.

[Motivation] In order to enable context-aware behavior of surgical assistance systems, the acquisition of various information about the current intraoperative situation is crucial. To achieve this, the complex task of situation recognition can be delegated to a specialized system. Consequently, a standardized interface is required for the seamless transfer of the recognized contextual information to the assistance systems, enabling them to adapt accordingly. [Methods] Our group analyzed four medical interface standards to determine their suitability for exchanging intraoperative contextual information. The assessment was based on a harmonized data and service model derived from the requirements of expected context-aware use cases. The Digital Imaging and Communications in Medicine (DICOM) and IEEE 11073 for Service-oriented Device Connectivity (SDC) were identified as the most appropriate standards. [Results] We specified how DICOM Unified Procedure Steps (UPS), can be used to effectively communicate contextual information. We proposed the inclusion of attributes to formalize different granularity levels of the surgical workflow. [Conclusions] DICOM UPS SOP classes can be used for the exchange of intraoperative contextual information between a situation recognition system and surgical assistance systems. This can pave the way for vendor-independent context awareness in the OR, leading to targeted assistance of the surgical team and an improvement of the surgical workflow.

Medical imaging datasets, such as magnetic resonance, are increasingly being used to investigate the genetic architecture of the brain. These images are commonly used as imaging-specific or –derived phenotypes when conducting genotype-phenotype association studies. When using this type of phenotype, multivariate genome-wide association study (GWAS) designs are considered better suited than univariate methods due to the ability to account for the inherent correlations between the phenotypes related to brain structures as determined from medical images. The main objective of this work is to establish and evaluate a comprehensive pipeline for investigating genotype-phenotype associations of the human brain using canonical component analysis. The proposed pipeline was tested to investigate genotype-phenotype associations between cortical brain region volumes in subjects with attention-deficit hyperactivity disorder as a proof-of-principle. Canonical component analysis, a form of multivariate GWAS and machine learning, was utilized to determine genotype-phenotype associations between cortical brain region volumes in subjects with attention-deficit hyperactivity disorder. Using the developed pipeline, several significant (p-value < 5E⁻⁰⁴) single nucleotide polymorphisms were found that reside in or near several genes like DSCAM or DPYSL2 that are known to be associated with neurological and mental disorders or substance addiction, a common comorbidity for subjects with attention-deficit hyperactivity disorder. These clinically meaningful results show that the proposed pipeline using canonical component analysis can be used to investigate the genetic architecture of the brain.

Objective: To predict right/left/total lung volume, thoracic cavity volume, and heart volume from subject demographics. Methods and materials: A cohort of 4,610 subjects with chest CT scans and basic demographics (i.e., age, gender, race, smoking status, smoking history, weight, and height) was used in this study. The right and left lungs, thoracic cavity, and heart depicted on chest CT scans were automatically segmented using U-Net, and their volumes were computed. Eight machine learning models (i.e., random forest, multivariate linear regression (MLR), support vector machine (SVM), extreme gradient boosting (XGBoost), multilayer perceptron (MLP), decision tree, k-nearest neighbors (KNN), and Bayesian regression) were developed and used to predict the volume measures from subject demographics. The 10-fold cross-validation method was used to evaluate the performances of the prediction models. R-squared (R²), mean absolute error (MAE), and mean absolute percentage error (MAPE) were used as performance metrics. Results: The MLP model demonstrated the best performance for predicting the thoracic cavity volume (R²: 0.628, MAE: 0.736 liters, MAPE: 10.9%), right lung volume (R²: 0.501, MAE: 0.383 liters, MAPE: 13.9%), and left lung volume (R²: 0.507, MAE: 0.365 liters, MAPE: 15.2%), while the XGBoost model demonstrated the best performance for predicting the total lung volume (R²: 0.514, MAE: 0.728 liters, MAPE: 14.0%) and heart volume (R²: 0.430, MAE: 0.075 liters, MAPE: 13.9%). Conclusion: Our results demonstrate the feasibility of predicting lung, heart, and thoracic cavity volumes from subject demographics.

Modern neuro-oncology workflows are driven by large collections of high-dimensional MRI data obtained using varying acquisition protocols. The concomitant heterogeneity of this data makes extensive manual curation and pre-processing imperative prior to algorithmic use. The limited efforts invested towards automating this curation and processing are fragmented, do not encompass the entire workflow, or still require significant manual intervention. In this work, we propose an artificial intelligence-driven solution for transforming multi-modal raw neuro-oncology MRI Digital Imaging and Communications in Medicine (DICOM) data into quantitative tumor measurements. Our end-to-end framework classifies MRI scans into different structural sequence types, preprocesses the data, and uses convolutional neural networks to segment tumor tissue subtypes. Moreover, it adopts an expert-in-the-loop approach, where segmentation results may be manually refined by radiologists. This framework was implemented as Docker Containers (for command line usage and within the eXtensible Neuroimaging Archive Toolkit [XNAT]) and validated on a retrospective glioma dataset (n = 155) collected from the Washington University School of Medicine, comprising preoperative MRI scans from patients with histopathologically confirmed gliomas. Segmentation results were refined by a neuroradiologist, and performance was quantified using Dice Similarity Coefficient to compare predicted and expert-refined tumor masks. The scan-type classifier yielded a 99.71% accuracy across all sequence types. The segmentation model achieved mean Dice scores of 0.894 (± 0.225) for whole tumor segmentation. The proposed framework can automate tumor segmentation and characterization – thus streamlining workflows in a clinical setting as well as expediting standardized curation of large-scale neuro-oncology datasets in a research setting.

Medical image interpretation is central to most clinical applications such as disease diagnosis, treatment planning, and prognostication. In clinical practice, radiologists examine medical images (e.g. chest x-rays, computed tomography images, etc.) and manually compile their findings into reports, which can be a time-consuming process. Automated approaches to radiology report generation, therefore, can reduce radiologist workload and improve efficiency in the clinical pathway. While recent deep-learning approaches for automated report generation from medical images have seen some success, most studies have relied on image-derived features alone, ignoring non-imaging patient data. Although a few studies have included the word-level contexts along with the image, the use of patient demographics is still unexplored. On the other hand, prior approaches to this task commonly use encoder-decoder frameworks that consist of a convolution vision model followed by a recurrent language model. Although recurrent-based text generators have achieved noteworthy results, they had the drawback of having a limited reference window and identifying only one part of the image while generating the next word. This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information, to synthesise patient-specific radiology reports. The proposed network uses a convolutional neural network (CNN) to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information, to synthesise full-text radiology reports. The designed network not only alleviates the limitations of the recurrent models but also improves the encoding and generative processes by including more context in the network. Data from two public databases were used to train and evaluate the proposed approach. CXRs and reports were extracted from the MIMIC-CXR database and combined with corresponding patients’ data (gender, age, and ethnicity) from MIMIC-IV. Based on the evaluation metrics used (BLEU 1-4 and BERTScore), including patient demographic information was found to improve the quality of reports generated using the proposed approach, relative to a baseline network trained using CXRs alone. The proposed approach shows potential for enhancing radiology report generation by leveraging rich patient metadata and combining semantic text embeddings derived thereof, with medical image-derived visual features.

Follow up imaging recommendations are often made in radiology reports. Not adhering to these recommendations can result in treatment delays, negative patient outcomes, additional medical issues, excessive testing, financial loss, and potential legal repercussions. In this paper, we present a generative-discriminative deep learning approach to classify radiology reports based on the presence of follow up recommendations. The generative model aims to produce a distributed representation of words in radiology reports. The discriminative model aims to extract local features that can be used to identify follow up recommendations. Three radiology report datasets were collected: one (n = 41417) with examinations extracted from the information system by regular expression operations, one (n= 14993) with examinations performed between January 1, 2015 to January 10, 2015 and annotated by three radiologists, and one (n = 5093) annotated by our radiologists at the time of dictation. Classification performance of the proposed hybrid model was compared with four traditional classification algorithms (random forest, logistic regression, naive Bayes and support vector machine (SVM)) and three neural network-based models (fastText, Convolutional neural network (CNN), Graph convolutional network (GCN)). A visualization algorithm was used to interpret the classification results of the hybrid model. Precision, recall, accuracy, and F1 scores were calculated for all models on the test set . Experimental results show that the hybrid model had a statistically significant higher F1 score (0.942 for Hybrid-random and 0.951 for Hybrid-report) than did four traditional machine learning algorithms (random forest: 0.914, logistic regression: 0.838, naive Bayes: 0.808 and SVM: 0.890) and the best-performing neural network-based model (fastText: 0.918). Our model can accurately identify radiology reports that contain follow-up recommendations. It can also automate detection of follow-up recommendation, thus streamlining workflows in a clinical setting and ensuring timely and appropriate patient care.

Purpose: A videofluoroscopic swallowing study (VFSS) is the gold standard for the examination of swallowing function. A fluoroscopic unit and contrast medium are used to record an X-ray video of the patient’s swallowing dynamics. This requires clinicians to observe three-dimensional swallowing movements on a two-dimensional video. In addition, the VFSS lacks facial surface information. In this study, we developed a method to synchronize the VFSS video with the three-dimensional movement of the facial surface. Approach: A 44-year-old man with no dysphagia was studied. Five smooth and one textured iron ball with a diameter of 3 mm were attached to the facial surface as markers at the tip of the nose, the upper and lower lips, the left and right corners of the mouth, and the chin. Using an X-ray fluoroscopic unit, the swallowing movements of gelatin jelly containing an iodine-based contrast media were recorded. The patient swallowed the jelly with mastication (right side, left side, both sides) and without mastication (open-mouth swallowing). At the same time, the movements of the facial surface were recorded using three video cameras. The four obtained videos were synchronized in terms of their start point, end point, and playback speed using a computer. Results: We created a synchronized video of the VFSS video and the three-dimensional video of the facial surface. This video could be observed from any viewpoint. In addition, we could analyze the velocity, distance, and angle of each point. Conclusion: This method can be used to objectively analyze mastication using numerical values.

The patients with prostate-specific antigen (PSA) levels of 4 ng/mL and above are considered for a prostate biopsy to rule out prostate cancer (PCa). However, the specificity of PSA test is not satisfied, especially in the PSA gray zone of 4 to 10 ng/mL. In this study, we aimed to assess the feasibility of a combined approach of radiomics and machine learning based on bpMRI images for a non-invasive diagnosis of PCa in PSA gray zone cases, specifically differentiation of PCa and benign prostatic hyperplasia (BPH). Images acquired on a 3-Tesla scanner (T2-weighted and diffusion-weighted imaging) from 103 patients (54 with PCa and 49 with BPH) were annotated to generate volumes of interest enclosing lesions. After image resampling and filtering, 2300 features were extracted. The Wilcoxon rank-sum test and LASSO regression algorithm was applied to select the radiomics features for building models. The binary logistics regression model of selected radiomics features was constructed with 4-fold cross validation and the rad-scores of BPH and PCa were calculated. The AUC of both models from T2WI and ADC showed satisfactory diagnostic performances (AUC > 0.9). The best results in terms of accuracy (80.9%) in test set were achieved by ADC model with 5 radiomics features. These evidences support the hypothesis that machine learning-based bpMRI radiomics models might be a potential and practical pathway to clinicians to better clinical decision-making and reduce the number of unnecessary prostate biopsies.

Medical Imaging and Data Resource Center (MIDRC) has been built to support AI-based research in response to the COVID-19 pandemic. One of the main goals of MIDRC is to make data collected in the repository ready for AI analysis. Due to data heterogeneity, there is a need to standardize data and make data-mining easier. Our study aims to stratify imaging data according to underlying anatomy using open-source image processing tools. The experiments were performed using Google Colaboratory on computed tomography (CT) imaging data available from the MIDRC. We adopted the existing open-source tools to process CT series (N=389) to define the image sub-volumes according to body part classification, and additionally identified series slices containing specific anatomic landmarks. Cases with automatically identified chest regions (N=369) were then processed to automatically segment the lungs. In order to assess the accuracy of segmentation, we performed outlier analysis using 3D shape radiomics features extracted from the left and right lungs. Standardized DICOM objects were created to store the resulting segmentations, regions, landmarks and radiomics features. We demonstrated that the MIDRC chest CT collections can be enriched using open-source analysis tools and that data available in MIDRC can be further used to evaluate the robustness of publicly available tools.

Colorectal cancer (CRC) is the third most common cancer type and the second most common cause of cancer deaths. CT colonography is a nearly ideal safe and accurate method for effective colorectal screening and prevention of CRCs, but the ionizing radiation of CT has been cited as a risk for population screening by CT colonography. Photon-counting CT (PCCT) can be used to address that risk. However, there have been no studies on the performance of automated polyp detection in PCCT colonography. In this preliminary study, we investigated the feasibility of the automated detection of clinically significant polyps from a PCCT colonography dataset. A laxative-free CT colonography examination that was simulated on an anthropomorphic colon phantom was scanned by use of a 16-slice PCCT scanner at 120 kVp and 40 mA. Our previously developed computer-aided detection (CADe) system was used to detect polyps from the PCCT dataset. The polyp detection performance was evaluated by use of 10-fold cross-validation. Our preliminary results show that the CADe system was able to detect the clinically significant polyps ≥6 mm in size from the PCCT colonography dataset at a high accuracy. This indicates that PCCT colonography is indeed a very promising approach for addressing the remaining obstacles of CT colonography in the population screening for CRC.

Photon-counting CT is an emerging technology with several advantages over conventional CT technology, such as the ability to reduce radiation exposure to CT. In this study, we evaluated the effect of the use of photon-counting CT colonography on the performance of our self-supervised 3D generative adversarial learning (GAN)-based electronic cleansing (EC) scheme. We simulated a fecal-tagging CT colonography case by use of an anthropomorphic colon phantom. The empty phantom served as the ground truth for the EC. Both the empty and fecal-tagging versions of the phantom were scanned by use of a photon-counting CT and a conventional CT scanner. We evaluated the performance of the EC scheme by using 100 paired volumes of interest extracted from the corresponding locations on the empty and fecal-tagging phantoms that had not been used for the training of the EC scheme. The peak signal-to-noise ratio was used as the metric for the quality of the EC images generated. Our preliminary results indicate that using photon-counting CT colonography at a low dose generates higher-quality EC images than those obtained by using conventional CT colonography. The results also demonstrate that our self-supervised training scheme generates images of higher quality than those obtained by use of conventional supervised training. Therefore, photon-counting CT colonography combined with our self-supervised 3DGAN EC scheme is expected to provide EC images of the highest quality in low-dose fecal-tagging CT colonography.

Driver action recognition is essential in vehicle safety and smart car systems targeting vehicles as a diagnostic space. In the context of video-based action recognition, attention-based architectures have surpassed conventional methods in deep learning. Former frameworks have produced excellent results on the public Drive&Act dataset. However, present frameworks do not consider the temporal ordering of frames in the video and the spatial layout of the relevant interacting objects yielding poor performance in certain actions that include semantic reversals. This includes so-called conjugate actions that originate when performed backward in time. An example would be moving the hand rightward vs. leftward. We propose a feature engineering approach to model the motion of human pose. We use key points relevant to the action to incorporate the sequential order. We implement video swin architecture on the Drive&Act dataset. Then, we utilize the histogram of oriented displacements on human joint locations and their displacements and train a support vector machine to classify actions in conjugate pairs. Performance increases in two conjugate actions namely fastening/ unfastening seat belt and taking off/ putting on sunglasses. Integrating our module with existing deep learning models increases the overall accuracy by 3% to 72%. Furthermore, our approach can be extended to other action classes.

The number of digital medical images is growing constantly over the years. This opens new possibilities of extracting information from them using computer-assisted methods, such as artificial intelligence. In this context, the application of radiomics has received increasing attention since 2012. In radiomics, medical image data is exploited by extracting numerous features from them that are not directly visible to the human eye. These features provide valuable information for diagnosis, prognosis and therapy, especially in cancer research. In this paper, we introduce a web-based radiomics module for end users under StudierFenster (www.studierfenster.at), which can extract image features for tumor characterization. StudierFenster is an online, open science medical image processing framework, where multiple clinically relevant modules and applications have been integrated since its initiation in 2018/2019, such as a medical VR viewer and automatic cranial implant design. The newly integrated Radiomics module allows the upload of medical images and segmentations of a region of interest to StudierFenster, where predefined radiomic features are calculated from them using the ‘pyRadiomic’ Python package. The radiomics module is able to calculate not only the basic first-order statistics of the images, but also more advanced features that capture the 2D/3D shape and gray level characteristics. The design of the radiomics module follows the architecture of StudierFenster, where computation-intensive procedures, such as preprocessing of the data and calculating the features for each image-segmentation pair, are executed on a server. The results are stored in a CSV file, which can afterwards be downloaded in a web-based user interface.

Pneumoconiosis is an occupational respiratory disease caused by inhaling dust into the lungs. In Japan, 240,000 people undergo pneumoconiosis screening every year. X-rays are used worldwide to classify the severity of pneumoconiosis. It is important to distinguish between type 0/1 and type 1/0, which are eligible for recognition of occupational injury. CT images are expected to provide more accurate diagnosis because they can be confirmed in three dimensions compared to X-rays. We extract micro-nodules from 3D CT images for each severity of pneumoconiosis, and analyze and evaluate the number, size, position and CT values of micro-nodules in each lung lobe.