arXiv subjects

Joseph Bae

Publications and source records attributed to Joseph Bae.

14 recordsLinked to original sources

arXiv2023

Token Sparsification for Faster Medical Image Segmentation

Can we use sparse tokens for dense prediction, e.g., segmentation? Although token sparsification has been applied to Vision Transformers (ViT) to accelerate classification, it is still unknown how to perform segmentation from sparse tokens. To this end, we reformulate segmentation as a sparse encoding -> token completion -> dense decoding (SCD) pipeline. We first empirically show that naively applying existing approaches from classification token pruning and masked image modeling (MIM) leads to failure and inefficient training caused by inappropriate sampling algorithms and the low quality of the restored dense features. In this paper, we propose Soft-topK Token Pruning (STP) and Multi-layer Token Assembly (MTA) to address these problems. In sparse encoding, STP predicts token importance scores with a lightweight sub-network and samples the topK tokens. The intractable topK gradients are approximated through a continuous perturbed score distribution. In token completion, MTA restores a full token sequence by assembling both sparse output tokens and pruned multi-layer intermediate ones. The last dense decoding stage is compatible with existing segmentation decoders, e.g., UNETR. Experiments show SCD pipelines equipped with STP and MTA are much faster than baselines without token pruning in both training (up to 120% higher throughput and inference up to 60.6% higher throughput) while maintaining segmentation quality.

cs.CV↗

arXiv2023

Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation

In medical vision, different imaging modalities provide complementary information. However, in practice, not all modalities may be available during inference or even training. Previous approaches, e.g., knowledge distillation or image synthesis, often assume the availability of full modalities for all patients during training; this is unrealistic and impractical due to the variability in data collection across sites. We propose a novel approach to learn enhanced modality-agnostic representations by employing a meta-learning strategy in training, even when only limited full modality samples are available. Meta-learning enhances partial modality representations to full modality representations by meta-training on partial modality data and meta-testing on limited full modality samples. Additionally, we co-supervise this feature enrichment by introducing an auxiliary adversarial learning branch. More specifically, a missing modality detector is used as a discriminator to mimic the full modality setting. Our segmentation framework significantly outperforms state-of-the-art brain tumor segmentation techniques in missing modality scenarios.

cs.CV↗

arXiv2022

Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentation

Masked Autoencoder (MAE) has recently been shown to be effective in pre-training Vision Transformers (ViT) for natural image analysis. By reconstructing full images from partially masked inputs, a ViT encoder aggregates contextual information to infer masked image regions. We believe that this context aggregation ability is particularly essential to the medical image domain where each anatomical structure is functionally and mechanically connected to other structures and regions. Because there is no ImageNet-scale medical image dataset for pre-training, we investigate a self pre-training paradigm with MAE for medical image analysis tasks. Our method pre-trains a ViT on the training set of the target data instead of another dataset. Thus, self pre-training can benefit more scenarios where pre-training data is hard to acquire. Our experimental results show that MAE self pre-training markedly improves diverse medical image tasks including chest X-ray disease classification, abdominal CT multi-organ segmentation, and MRI brain tumor segmentation. Code is available at https://github.com/cvlab-stonybrook/SelfMedMAE

eess.IV↗

arXiv2022

Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations

Clinical outcome or severity prediction from medical images has largely focused on learning representations from single-timepoint or snapshot scans. It has been shown that disease progression can be better characterized by temporal imaging. We therefore hypothesized that outcome predictions can be improved by utilizing the disease progression information from sequential images. We present a deep learning approach that leverages temporal progression information to improve clinical outcome predictions from single-timepoint images. In our method, a self-attention based Temporal Convolutional Network (TCN) is used to learn a representation that is most reflective of the disease trajectory. Meanwhile, a Vision Transformer is pretrained in a self-supervised fashion to extract features from single-timepoint images. The key contribution is to design a recalibration module that employs maximum mean discrepancy loss (MMD) to align distributions of the above two contextual representations. We train our system to predict clinical outcomes and severity grades from single-timepoint images. Experiments on chest and osteoarthritis radiography datasets demonstrate that our approach outperforms other state-of-the-art techniques.

eess.IV↗

arXiv2022

Lung Swapping Autoencoder: Learning a Disentangled Structure-texture Representation of Chest Radiographs

Well-labeled datasets of chest radiographs (CXRs) are difficult to acquire due to the high cost of annotation. Thus, it is desirable to learn a robust and transferable representation in an unsupervised manner to benefit tasks that lack labeled data. Unlike natural images, medical images have their own domain prior; e.g., we observe that many pulmonary diseases, such as the COVID-19, manifest as changes in the lung tissue texture rather than the anatomical structure. Therefore, we hypothesize that studying only the texture without the influence of structure variations would be advantageous for downstream prognostic and predictive modeling tasks. In this paper, we propose a generative framework, the Lung Swapping Autoencoder (LSAE), that learns factorized representations of a CXR to disentangle the texture factor from the structure factor. Specifically, by adversarial training, the LSAE is optimized to generate a hybrid image that preserves the lung shape in one image but inherits the lung texture of another. To demonstrate the effectiveness of the disentangled texture representation, we evaluate the texture encoder $Enc^t$ in LSAE on ChestX-ray14 (N=112,120), and our own multi-institutional COVID-19 outcome prediction dataset, COVOC (N=340 (Subset-1) + 53 (Subset-2)). On both datasets, we reach or surpass the state-of-the-art by finetuning $Enc^t$ in LSAE that is 77% smaller than a baseline Inception v3. Additionally, in semi-and-self supervised settings with a similar model budget, $Enc^t$ in LSAE is also competitive with the state-of-the-art MoCo. By "re-mixing" the texture and shape factors, we generate meaningful hybrid images that can augment the training set. This data augmentation method can further improve COVOC prediction performance. The improvement is consistent even when we directly evaluate the Subset-1 trained model on Subset-2 without any fine-tuning.

eess.IV↗

arXiv2021

Attention-based Multi-scale Gated Recurrent Encoder with Novel Correlation Loss for COVID-19 Progression Prediction

COVID-19 image analysis has mostly focused on diagnostic tasks using single timepoint scans acquired upon disease presentation or admission. We present a deep learning-based approach to predict lung infiltrate progression from serial chest radiographs (CXRs) of COVID-19 patients. Our method first utilizes convolutional neural networks (CNNs) for feature extraction from patches within the concerned lung zone, and also from neighboring and remote boundary regions. The framework further incorporates a multi-scale Gated Recurrent Unit (GRU) with a correlation module for effective predictions. The GRU accepts CNN feature vectors from three different areas as input and generates a fused representation. The correlation module attempts to minimize the correlation loss between hidden representations of concerned and neighboring area feature vectors, while maximizing the loss between the same from concerned and remote regions. Further, we employ an attention module over the output hidden states of each encoder timepoint to generate a context vector. This vector is used as an input to a decoder module to predict patch severity grades at a future timepoint. Finally, we ensemble the patch classification scores to calculate patient-wise grades. Specifically, our framework predicts zone-wise disease severity for a patient on a given day by learning representations from the previous temporal CXRs. Our novel multi-institutional dataset comprises sequential CXR scans from N=93 patients. Our approach outperforms transfer learning and radiomic feature-based baseline approaches on this dataset.

eess.IV↗

arXiv2021

Mobile Apps Prioritizing Privacy, Efficiency and Equity: A Decentralized Approach to COVID-19 Vaccination Coordination

In this early draft, we describe a decentralized, app-based approach to COVID-19 vaccine distribution that facilitates zero knowledge verification, dynamic vaccine scheduling, continuous symptoms reporting, access to aggregate analytics based on population trends and more. To ensure equity, our solution is developed to work with limited internet access as well. In addition, we describe the six critical functions that we believe last mile vaccination management platforms must perform, examine existing vaccine management systems, and present a model for privacy-focused, individual-centric solutions.

cs.CY↗

arXiv2021

MIT SafePaths Card (MiSaCa): Augmenting Paper Based Vaccination Cards with Printed Codes

In this early draft, we describe a user-centric, card-based system for vaccine distribution. Our system makes use of digitally signed QR codes and their use for phased vaccine distribution, vaccine administration/record-keeping, immunization verification, and follow-up symptom reporting. Furthermore, we propose and describe a complementary scanner app system to be used by vaccination clinics, public health officials, and immunization verification parties to effectively utilize card-based framework. We believe that the proposed system provides a privacy-preserving and efficient framework for vaccine distribution in both developed and developing regions.

cs.CY↗

arXiv2021

COVID-19 Tests Gone Rogue: Privacy, Efficacy, Mismanagement and Misunderstandings

COVID-19 testing, the cornerstone for effective screening and identification of COVID-19 cases, remains paramount as an intervention tool to curb the spread of COVID-19 both at local and national levels. However, the speed at which the pandemic struck and the response was rolled out, the widespread impact on healthcare infrastructure, the lack of sufficient preparation within the public health system, and the complexity of the crisis led to utter confusion among test-takers. Invasion of privacy remains a crucial concern. The user experience of test takers remains low. User friction affects user behavior and discourages participation in testing programs. Test efficacy has been overstated. Test results are poorly understood resulting in inappropriate follow-up recommendations. Herein, we review the current landscape of COVID-19 testing, identify four key challenges, and discuss the consequences of the failure to address these challenges. The current infrastructure around testing and information propagation is highly privacy-invasive and does not leverage scalable digital components. In this work, we discuss challenges complicating the existing covid-19 testing ecosystem and highlight the need to improve the testing experience for the user and reduce privacy invasions. Digital tools will play a critical role in resolving these challenges.

cs.CY↗

arXiv2020

COVID-19 Outbreak Prediction and Analysis using Self Reported Symptoms

It is crucial for policymakers to understand the community prevalence of COVID-19 so combative resources can be effectively allocated and prioritized during the COVID-19 pandemic. Traditionally, community prevalence has been assessed through diagnostic and antibody testing data. However, despite the increasing availability of COVID-19 testing, the required level has not been met in most parts of the globe, introducing a need for an alternative method for communities to determine disease prevalence. This is further complicated by the observation that COVID-19 prevalence and spread varies across different spatial, temporal, and demographics. In this study, we understand trends in the spread of COVID-19 by utilizing the results of self-reported COVID-19 symptoms surveys as an alternative to COVID-19 testing reports. This allows us to assess community disease prevalence, even in areas with low COVID-19 testing ability. Using individually reported symptom data from various populations, our method predicts the likely percentage of the population that tested positive for COVID-19. We do so with a Mean Absolute Error (MAE) of 1.14 and Mean Relative Error (MRE) of 60.40\% with 95\% confidence interval as (60.12, 60.67). This implies that our model predicts +/- 1140 cases than the original in a population of 1 million. In addition, we forecast the location-wise percentage of the population testing positive for the next 30 days using self-reported symptoms data from previous days. The MAE for this method is as low as 0.15 (MRE of 23.61\% with 95\% confidence interval as (23.6, 13.7)) for New York. We present an analysis of these results, exposing various clinical attributes of interest across different demographics. Lastly, we qualitatively analyze how various policy enactments (testing, curfew) affect the prevalence of COVID-19 in a community.

cs.LG↗

arXiv2020

Digital Landscape of COVID-19 Testing: Challenges and Opportunities

The COVID-19 Pandemic has left a devastating trail all over the world, in terms of loss of lives, economic decline, travel restrictions, trade deficit, and collapsing economy including real-estate, job loss, loss of health benefits, the decline in quality of access to care and services and overall quality of life. Immunization from the anticipated vaccines will not be the stand-alone guideline that will help surpass the pandemic and return to normalcy. Four pillars of effective public health intervention include diagnostic testing for both asymptomatic and symptomatic individuals, contact tracing, quarantine of individuals with symptoms or who are exposed to COVID-19, and maintaining strict hygiene standards at the individual and community level. Digital technology, currently being used for COVID-19 testing include certain mobile apps, web dashboards, and online self-assessment tools. Herein, we look into various digital solutions adapted by communities across universities, businesses, and other organizations. We summarize the challenges experienced using these tools in terms of quality of information, privacy, and user-centric issues. Despite numerous digital solutions available and being developed, many vary in terms of information being shared in terms of both quality and quantity, which can be overwhelming to the users. Understanding the testing landscape through a digital lens will give a clear insight into the multiple challenges that we face including data privacy, cost, and miscommunication. It is the destiny of digitalization to navigate testing for COVID-19. Block-chain based systems can be used for privacy preservation and ensuring ownership of the data to remain with the user. Another solution involves having digital health passports with relevant and correct information. In this early draft, we summarize the challenges and propose possible solutions to address the same.

cs.CY↗

arXiv2020

Challenges of Equitable Vaccine Distribution in the COVID-19 Pandemic

The COVID-19 pandemic has led to a need for widespread and rapid vaccine development. As several vaccines have recently been approved for human use or are in different stages of development, governments across the world are preparing comprehensive guidelines for vaccine distribution and monitoring. In this early article, we identify challenges in logistics, health outcomes, user-centric matters, and communication associated with disease-related, individual, societal, economic, and privacy consequences. Primary challenges include difficulty in equitable distribution, vaccine efficacy, duration of immunity, multi-dose adherence, and privacy-focused record-keeping to be HIPAA compliant. While many of these challenges have been previously identified and addressed, some have not been acknowledged from a comprehensive view accounting for unprecedented interactions between challenges and specific populations. The logistics of equitable widespread vaccine distribution in disparate populations and countries of various economic, racial, and cultural constitutions must be thoroughly examined and accounted for. We also describe unique challenges regarding the efficacy of vaccines in specialized populations including children, the elderly, and immunocompromised individuals. Furthermore, we report the potential for understudied drug-vaccine interactions as well as the possibility that certain vaccine platforms may increase susceptibility to HIV. Given these complicated issues, the importance of privacy-focused, user-centric systems for vaccine education and incentivization along with clear communication from governments, organizations, and academic institutions is imperative. These challenges are by no means insurmountable, but require careful attention to avoid consequences spanning a range of disease-related, individual, societal, economic, and security domains.

econ.GN↗

arXiv2020

Clinical Landscape of COVID-19 Testing: Difficult Choices

The coronavirus disease 2019 (COVID-19) pandemic has spread rapidly across the world, leading to enormous amounts of human death and economic loss. Until definitive preventive or curative measures are developed, policies regarding testing, contact tracing, and quarantine remain the best public health tools for curbing viral spread. Testing is a crucial component of these efforts, enabling the identification and isolation of infected individuals. Differences in testing methodologies, time frames, and outcomes can have an impact on their overall efficiency, usability and efficacy. In this early draft, we draw a comparison between the various types of diagnostic tests including PCR, antigen, and home tests in relation to their relative advantages, disadvantages, and use cases. We also look into alternative and unconventional methods. Further, we analyze the short-term and long-term impacts of the virus and its testing on various verticals such as business, government laws, policies, and healthcare.

q-bio.OT↗

arXiv2020

Predicting Clinical Outcomes in COVID-19 using Radiomics and Deep Learning on Chest Radiographs: A Multi-Institutional Study

We predict mechanical ventilation requirement and mortality using computational modeling of chest radiographs (CXRs) for coronavirus disease 2019 (COVID-19) patients. This two-center, retrospective study analyzed 530 deidentified CXRs from 515 COVID-19 patients treated at Stony Brook University Hospital and Newark Beth Israel Medical Center between March and August 2020. DL and machine learning classifiers to predict mechanical ventilation requirement and mortality were trained and evaluated using patient CXRs. A novel radiomic embedding framework was also explored for outcome prediction. All results are compared against radiologist grading of CXRs (zone-wise expert severity scores). Radiomic and DL classification models had mAUCs of 0.78+/-0.02 and 0.81+/-0.04, compared with expert scores mAUCs of 0.75+/-0.02 and 0.79+/-0.05 for mechanical ventilation requirement and mortality prediction, respectively. Combined classifiers using both radiomics and expert severity scores resulted in mAUCs of 0.79+/-0.04 and 0.83+/-0.04 for each prediction task, demonstrating improvement over either artificial intelligence or radiologist interpretation alone. Our results also suggest instances where inclusion of radiomic features in DL improves model predictions, something that might be explored in other pathologies. The models proposed in this study and the prognostic information they provide might aid physician decision making and resource allocation during the COVID-19 pandemic.

q-bio.QM↗