arXiv subjects

Dongju Park

Publications and source records attributed to Dongju Park.

3 recordsLinked to original sources

arXiv2022

Deep learning-based single-shot computational spectrometer using multilayer thin films

Computational spectrometers have mobile application potential, such as on-site detection and self-diagnosis, by offering compact size, fast operation time, high resolution, wide working range, and low-cost production. Although these spectrometers have been extensively studied, demonstrations are confined to a few examples of straightforward spectra. This study demonstrates deep learning (DL)-based single-shot computational spectrometer for narrow and broad spectra using a multilayer thin-film filter array. For measuring light intensities, the device was built by attaching the filter array, fabricated using a wafer-level stencil lithography process, to a complementary metal-oxide-semiconductor image sensor. All the intensities were extracted from a monochrome image captured with a single exposure. A DL architecture comprising a dense layer and a U-Net backbone with residual connections was employed for spectrum reconstruction. The measured intensities were fed into the DL architecture for reconstruction as spectra. We reconstructed 323 continuous spectra with an average root mean squared error of 0.0288 in a 500-850 nm wavelength range with 1-nm spacing. Our computational spectrometer achieved a compact size, fast measuring time, high resolution, and wide working range.

physics.optics↗

arXiv2021

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers

GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data. Here we address some remaining issues less reported by the GPT-3 paper, such as a non-English LM, the performances of different sized models, and the effect of recently introduced prompt optimization on in-context learning. To achieve this, we introduce HyperCLOVA, a Korean variant of 82B GPT-3 trained on a Korean-centric corpus of 560B tokens. Enhanced by our Korean-specific tokenization, HyperCLOVA with our training configuration shows state-of-the-art in-context zero-shot and few-shot learning performances on various downstream tasks in Korean. Also, we show the performance benefits of prompt-based learning and demonstrate how it can be integrated into the prompt engineering pipeline. Then we discuss the possibility of materializing the No Code AI paradigm by providing AI prototyping capabilities to non-experts of ML by introducing HyperCLOVA studio, an interactive prompt engineering interface. Lastly, we demonstrate the potential of our methods with three successful in-house applications.

cs.CL↗

arXiv2021

GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation

Large-scale language models such as GPT-3 are excellent few-shot learners, allowing them to be controlled via natural text prompts. Recent studies report that prompt-based direct classification eliminates the need for fine-tuning but lacks data and inference scalability. This paper proposes a novel data augmentation technique that leverages large-scale language models to generate realistic text samples from a mixture of real samples. We also propose utilizing soft-labels predicted by the language models, effectively distilling knowledge from the large-scale language models and creating textual perturbations simultaneously. We perform data augmentation experiments on diverse classification tasks and show that our method hugely outperforms existing text augmentation methods. Ablation studies and a qualitative analysis provide more insights into our approach.

cs.CL↗