arXiv subjects

Jiho Kim

Publications and source records attributed to Jiho Kim.

32 records · Page 2Linked to original sources

arXiv2024

CookAR: Affordance Augmentations in Wearable AR to Support Kitchen Tool Interactions for People with Low Vision

Cooking is a central activity of daily living, supporting independence as well as mental and physical health. However, prior work has highlighted key barriers for people with low vision (LV) to cook, particularly around safely interacting with tools, such as sharp knives or hot pans. Drawing on recent advancements in computer vision (CV), we present CookAR, a head-mounted AR system with real-time object affordance augmentations to support safe and efficient interactions with kitchen tools. To design and implement CookAR, we collected and annotated the first egocentric dataset of kitchen tool affordances, fine-tuned an affordance segmentation model, and developed an AR system with a stereo camera to generate visual augmentations. To validate CookAR, we conducted a technical evaluation of our fine-tuned model as well as a qualitative lab study with 10 LV participants for suitable augmentation design. Our technical evaluation demonstrates that our model outperforms the baseline on our tool affordance dataset, while our user study indicates a preference for affordance augmentations over the traditional whole object augmentations.

cs.HC↗

arXiv2024

Towards Full Authorship with AI: Supporting Revision with AI-Generated Views

Large language models (LLMs) are shaping a new user interface (UI) paradigm in writing tools by enabling users to generate text through prompts. This paradigm shifts some creative control from the user to the system, thereby diminishing the user's authorship and autonomy in the writing process. To restore autonomy, we introduce Textfocals, a UI prototype designed to investigate a human-centered approach that emphasizes the user's role in writing. Textfocals supports the writing process by providing LLM-generated summaries, questions, and advice (i.e., LLM views) in a sidebar of a text editor, encouraging reflection and self-driven revision in writing without direct text generation. Textfocals' UI affordances, including contextually adaptive views and scaffolding for prompt selection and customization, offer a novel way to interact with LLMs where users maintain full authorship of their writing. A formative user study with Textfocals showed promising evidence that this approach might help users develop underdeveloped ideas, cater to the rhetorical audience, and clarify their writing. However, the study also showed interaction design challenges related to document navigation and scoping, prompt engineering, and context management. Our work highlights the breadth of the design space of writing support interfaces powered by generative AI that maintain authorship integrity.

cs.HC↗

arXiv2023

CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training

A large-scale image-text pair dataset has greatly contributed to the development of vision-language pre-training (VLP) models, which enable zero-shot or few-shot classification without costly annotation. However, in the medical domain, the scarcity of data remains a significant challenge for developing a powerful VLP model. In this paper, we tackle the lack of image-text data in chest X-ray by expanding image-label pair as image-text pair via general prompt and utilizing multiple images and multiple sections in a radiologic report. We also design two contrastive losses, named ICL and TCL, for learning study-level characteristics of medical images and reports, respectively. Our model outperforms the state-of-the-art models trained under the same conditions. Also, enlarged dataset improve the discriminative power of our pre-trained model for classification, while sacrificing marginal retrieval performance. Code is available at https://github.com/kakaobrain/cxr-clip.

cs.CV↗

arXiv2023

KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models

While large language models (LLMs) have made considerable advancements in understanding and generating unstructured text, their application in structured data remains underexplored. Particularly, using LLMs for complex reasoning tasks on knowledge graphs (KGs) remains largely untouched. To address this, we propose KG-GPT, a multi-purpose framework leveraging LLMs for tasks employing KGs. KG-GPT comprises three steps: Sentence Segmentation, Graph Retrieval, and Inference, each aimed at partitioning sentences, retrieving relevant graph components, and deriving logical conclusions, respectively. We evaluate KG-GPT using KG-based fact verification and KGQA benchmarks, with the model showing competitive and robust performance, even outperforming several fully-supervised models. Our work, therefore, marks a significant step in unifying structured and unstructured data processing within the realm of LLMs.

cs.CL↗

arXiv2023

FactKG: Fact Verification via Reasoning on Knowledge Graphs

In real world applications, knowledge graphs (KG) are widely used in various domains (e.g. medical applications and dialogue agents). However, for fact verification, KGs have not been adequately utilized as a knowledge source. KGs can be a valuable knowledge source in fact verification due to their reliability and broad applicability. A KG consists of nodes and edges which makes it clear how concepts are linked together, allowing machines to reason over chains of topics. However, there are many challenges in understanding how these machine-readable concepts map to information in text. To enable the community to better use KGs, we introduce a new dataset, FactKG: Fact Verification via Reasoning on Knowledge Graphs. It consists of 108k natural language claims with five types of reasoning: One-hop, Conjunction, Existence, Multi-hop, and Negation. Furthermore, FactKG contains various linguistic patterns, including colloquial style claims as well as written style claims to increase practicality. Lastly, we develop a baseline approach and analyze FactKG over these reasoning types. We believe FactKG can advance both reliability and practicality in KG-based fact verification.

cs.CL↗

arXiv2023

Text-to-ECG: 12-Lead Electrocardiogram Synthesis conditioned on Clinical Text Reports

Electrocardiogram (ECG) synthesis is the area of research focused on generating realistic synthetic ECG signals for medical use without concerns over annotation costs or clinical data privacy restrictions. Traditional ECG generation models consider a single ECG lead and utilize GAN-based generative models. These models can only generate single lead samples and require separate training for each diagnosis class. The diagnosis classes of ECGs are insufficient to capture the intricate differences between ECGs depending on various features (e.g. patient demographic details, co-existing diagnosis classes, etc.). To alleviate these challenges, we present a text-to-ECG task, in which textual inputs are used to produce ECG outputs. Then we propose Auto-TTE, an autoregressive generative model conditioned on clinical text reports to synthesize 12-lead ECGs, for the first time to our knowledge. We compare the performance of our model with other representative models in text-to-speech and text-to-image. Experimental results show the superiority of our model in various quantitative evaluations and qualitative analysis. Finally, we conduct a user study with three board-certified cardiologists to confirm the fidelity and semantic alignment of generated samples. our code will be available at https://github.com/TClife/text_to_ecg

cs.CL↗

arXiv2022

Graph-Text Multi-Modal Pre-training for Medical Representation Learning

As the volume of Electronic Health Records (EHR) sharply grows, there has been emerging interest in learning the representation of EHR for healthcare applications. Representation learning of EHR requires appropriate modeling of the two dominant modalities in EHR: structured data and unstructured text. In this paper, we present MedGTX, a pre-trained model for multi-modal representation learning of the structured and textual EHR data. MedGTX uses a novel graph encoder to exploit the graphical nature of structured EHR data, and a text encoder to handle unstructured text, and a cross-modal encoder to learn a joint representation space. We pre-train our model through four proxy tasks on MIMIC-III, an open-source EHR data, and evaluate our model on two clinical benchmarks and three novel downstream tasks which tackle real-world problems in EHR data. The results consistently show the effectiveness of pre-training the model for joint representation of both structured and unstructured information from EHR. Given the promising performance of MedGTX, we believe this work opens a new door to jointly understanding the two fundamental modalities of EHR data.

cs.CL↗

arXiv2021

Question Answering for Complex Electronic Health Records Database using Unified Encoder-Decoder Architecture

An intelligent machine that can answer human questions based on electronic health records (EHR-QA) has a great practical value, such as supporting clinical decisions, managing hospital administration, and medical chatbots. Previous table-based QA studies focusing on translating natural questions into table queries (NLQ2SQL), however, suffer from the unique nature of EHR data due to complex and specialized medical terminology, hence increased decoding difficulty. In this paper, we design UniQA, a unified encoder-decoder architecture for EHR-QA where natural language questions are converted to queries such as SQL or SPARQL. We also propose input masking (IM), a simple and effective method to cope with complex medical terms and various typos and better learn the SQL/SPARQL syntax. Combining the unified architecture with an effective auxiliary training objective, UniQA demonstrated a significant performance improvement against the previous state-of-the-art model for MIMICSQL* (14.2% gain), the most complex NLQ2SQL dataset in the EHR domain, and its typo-ridden versions (approximately 28.8% gain). In addition, we confirmed consistent results for the graph-based EHR-QA dataset, MIMICSPARQL*.

cs.CL↗

arXiv2019

Infrared study of Large scale h-BN film and Graphene/h-BN heterostructure

We synthesize a series of CVD h-BN films and perform critical infrared spectroscopic characterization. For high-temperature (HT, Temp = 1400 degrees) grown h-BN thin film only E1u-mode infrared phonon is activated demonstrating highly aligned 2D h-BN planes over large area, whereas low-temperature (LT, Temp = 1000 degrees) grown film shows two phonon peaks, E1u and A2u, due to stacking of h-BN plane at tilted angle. For CVD graphene transferred on HT h-BN/SiO2/Si substrate, interband transition spectrum s1 shifts strongly to lower energy compared with that on LT h-BN/SiO2/Si and on bare SiO2/Si substrate, revealing that residual carrier density n in graphene is suppressed by use of HT h-BN layer. Also the interband transition width of s1 defined by effective temperature is reduced from 400 K for G/SiO2/Si to 300 K for HT h-BN/SiO2/Si. The behaviors of n and effective temperature show that HT h-BN film can decouple CVD graphene from the impurity and defect of SiO2 leading to large scale free-standing like graphene.

cond-mat.mtrl-sci↗

arXiv2019

Infrared study of carrier scattering mechanism in ion-gated graphene

We performed infrared transmission experiment on ion-gel gated graphene and measured carrier scattering rate g as function of carrier density n over wide range up to n=2E13 cm-2. The g exhibits a rapid decreases along with the gating followed by persistent increases on further carrier doping. This behavior of g(n) demonstrates that carrier is scattered dominantly by the two scattering mechanisms, namely, charged impurity (CI) scattering and short-range disorder (SR) scattering, with additional minor scattering from substrate phonon (SPP). We can determine the absolute strengths of all the scattering channels by fitting the g(n) data and unveils the complete n-dependent map of the scattering mechanisms g(n)=gCI(n)+gSR(n)+gSPP(n). The gCI(n) and gSR(n) are larger than those of SiO2$-gated graphene by 1.8 times, which elucidates the dual role of the ion-gel layer as a CI-scatterer and simultaneously a SR-scatterer to graphene. Additionally we show that freezing of IG at low-T (~200 K) does not cause any change to the carrier scattering.

cond-mat.mes-hall↗

arXiv2019

Observation of optical absorption correlated with surface state of topological insulator

We performed broadband optical transmission measurements of Bi2Se3 and In-doped Bi(1-x)In(x)2Se3 thin films, where in the latter the spin-orbit coupling (SOC) strength can be tuned by introducing In. Drude and interband transitions exhibit In-dependent changes that are consistent with evolution from metallic (x=0) to insulating (x=1) nature of the end compounds. Most notably, an optical absorption peak located at hw=1eV in Bi2Se3 is completely quenched at x=0.06, the critical concentration where the phase transition from TI into non-TI takes place. For this x, the surface state (SS) is vanished from the band structure as well. The correlation between the 1eV optical peak and the SS in the x-dependences suggests that the peak is associated with the SS. We further show that when Bi2Se3 is electrically gated, the 1eV-peak becomes stronger(weaker) when electron is depleted from (accumulated into) the SS. These observations combined together demonstrate that under the hw=1eV illumination electron is excited from a bulk band into the topological surface band of Bi2Se3. The optical population of surface band is of significant importance not only for fundamental study but also for TI-based optoelectronic device application.

cond-mat.mtrl-sci↗

arXiv2019

Gate tunable optical absorption and band structure of twisted bilayer graphene

We report the infrared transmission measurement on electrically gated twisted bilayer graphene. The optical absorption spectrum clearly manifests the dramatic changes such as the splitting of inter-linear-band absorption step, the shift of inter-van Hove singularity transition peak, and the emergence of very strong intra-valence (intra-conduction) band transition. These anomalous optical behaviors demonstrate consistently the non-rigid band structure modification created by the ion-gel gating through the layer-dependent Coulomb screening. We propose that this screening-driven band modification is an universal phenomenon that persists to other bilayer crystals in general, establishing the electrical gating as a versatile technique to engineer the band structures and to create new types of optical absorptions that can be exploited in electro-optical device application.

cond-mat.mtrl-sci↗

arXiv2019

Infrared spectroscopic study of carrier scattering in gated CVD graphene

We measured Drude absorption of gated CVD graphene using far-infrared transmission spectroscopy, and determined carrier scattering rate (g) as function of the varied carrier density (n). The n-dependent g(n) was obtained for a series of conditions systematically changed as (10 K, vacuum) -> (300 K, vacuum) -> (300 K, ambient pressure), which reveals that (1) at low-T, charged impurity (=A/sqrt(n)) and short-range defect (=B*sqrt(n)) are the major scattering sources which constitute the total scattering g=A/sqrt(n)+B*sqrt(n), (2) among various kinds of phonons populated at room-T, surface polar phonon of the SiO2 substrate is the dominantly scattering source, (3) in air, the gas molecules adsorbed on graphene play a dual role in carrier scattering as charged impurity center and resonant scattering center. We present the absolute scattering strengths of those individual scattering sources, which provides the complete map of scattering mechanism of CVD graphene for the first time. This scattering map allows us find out practical measures to suppress the individual scatterings, the mobility gains accompanied by them, and finally the ultimate attainable carrier mobility for CVD graphene.

cond-mat.mtrl-sci↗

arXiv2008

Coinductive properties of Lipschitz functions on streams

A simple hierarchical structure is imposed on the set of Lipschitz functions on streams (i.e. sequences over a fixed alphabet set) under the standard metric. We prove that sets of non-expanding and contractive functions are closed under a certain coiterative construction. The closure property is used to construct new final stream coalgebras over finite alphabets. For an example, we show that the 2-adic extension of the Collatz function and certain variants yield final bitstream coalgebras.

math.DS↗