arXiv subjects

Stephen Choi

Publications and source records attributed to Stephen Choi.

4 recordsLinked to original sources

arXiv2024

When Life gives you LLMs, make LLM-ADE: Large Language Models with Adaptive Data Engineering

This paper presents the LLM-ADE framework, a novel methodology for continued pre-training of large language models (LLMs) that addresses the challenges of catastrophic forgetting and double descent. LLM-ADE employs dynamic architectural adjustments, including selective block freezing and expansion, tailored to specific datasets. This strategy enhances model adaptability to new data while preserving previously acquired knowledge. We demonstrate LLM-ADE's effectiveness on the TinyLlama model across various general knowledge benchmarks, showing significant performance improvements without the drawbacks of traditional continuous training methods. This approach promises a more versatile and robust way to keep LLMs current and efficient in real-world applications.

cs.CE↗

arXiv2023

Conversational Factor Information Retrieval Model (ConFIRM)

This paper introduces the Conversational Factor Information Retrieval Method (ConFIRM), a novel approach to fine-tuning large language models (LLMs) for domain-specific retrieval tasks. ConFIRM leverages the Five-Factor Model of personality to generate synthetic datasets that accurately reflect target population characteristics, addressing data scarcity in specialized domains. We demonstrate ConFIRM's effectiveness through a case study in the finance sector, fine-tuning a Llama-2-7b model using personality-aligned data from the PolyU-Asklora Fintech Adoption Index. The resulting model achieved 91% accuracy in classifying financial queries, with an average inference time of 0.61 seconds on an NVIDIA A100 GPU. ConFIRM shows promise for creating more accurate and personalized AI-driven information retrieval systems across various domains, potentially mitigating issues of hallucinations and outdated information in LLMs deployed

cs.IR↗

arXiv2015

Ehrenfest approach to open double-well dynamics

We consider an Ehrenfest approximation for a particle in a double-well potential in the presence of an external environment schematized as a finite resource heat bath. This allows us to explore how the limitations in the applicability of Ehrenfest dynamics to nonlinear systems are modified in an open system setting. Within this framework, we have identified an environment-induced spontaneous symmetry breaking mechanism, and we argue that the Ehrenfest approximation becomes increasingly valid in the limit of strong coupling to the external reservoir, either in the form of increasing number of oscillators or increasing temperature. The analysis also suggests a rather intuitive picture for the general phenomenon of quantum tunneling and its interplay with classical thermal activation processes, which may be of relevance in physical chemistry, ultracold atom physics, and fast-switching dynamics such as in superconducting digital electronics.

quant-ph↗

arXiv2011

Optimized sympathetic cooling of atomic mixtures via fast adiabatic strategies

We discuss fast frictionless cooling techniques in the framework of sympathetic cooling of cold atomic mixtures. It is argued that optimal cooling of an atomic species - in which the deepest quantum degeneracy regime is achieved - may be obtained by means of sympathetic cooling with another species whose trapping frequency is dynamically changed to maintain constancy of the Lewis-Riesenfeld adiabatic invariant. Advantages and limitations of this cooling strategy are discussed, with particular regard to the possibility of cooling Fermi gases to a deeper degenerate regime.

cond-mat.quant-gas↗