arXiv subjects
Publications and source records attributed to Jonathan Lai.
We prove the first instance of a conjecture by Kontsevich-Soibelman that the non-archimedean period map recovers the analytic periods in the case of log Calabi-Yau surfaces. In particular, we show that the K-affine structure, a natural enhancement of the singular integral affine structure on the essential skeleton, determines the isomorphism type of the log Calabi-Yau surface.
Model merging aims to combine multiple expert models into a more capable single model, offering benefits such as reduced storage and serving costs, improved generalization, and support for decentralized model development. Despite its promise, previous studies have primarily focused on merging a few small models. This leaves many unanswered questions about the effect of scaling model size and how it interplays with other key factors -- like the base model quality and number of expert models -- , to affect the merged model's performance. This work systematically evaluates the utility of model merging at scale, examining the impact of these different factors. We experiment with merging fully fine-tuned models using 4 popular merging methods -- Averaging, Task~Arithmetic, Dare, and TIES -- across model sizes ranging from 1B-64B parameters and merging up to 8 different expert models. We evaluate the merged models on both held-in tasks, i.e., the expert's training tasks, and zero-shot generalization to unseen held-out tasks. Our experiments provide several new insights about model merging at scale and the interplay between different factors. First, we find that merging is more effective when experts are created from strong base models, i.e., models with good zero-shot performance. Second, larger models facilitate easier merging. Third merging consistently improves generalization capabilities. Notably, when merging 8 large expert models, the merged models often generalize better compared to the multitask trained models. Fourth, we can better merge more expert models when working with larger models. Fifth, different merging methods behave very similarly at larger scales. Overall, our findings shed light on some interesting properties of model merging while also highlighting some limitations. We hope that this study will serve as a reference point on large-scale merging for upcoming research.
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
We show that the ring of regular functions of every smooth affine log Calabi-Yau surface with maximal boundary has a vector space basis parametrized by its set of integer tropical points and a $\mathbb{C}$-algebra structure with structure coefficients given by the geometric construction of Keel-Yu. To prove this result, we first give a canonical compactification of the mirror family associated with a pair $(Y,D)$ constructed by Gross-Hacking-Keel where $Y$ is a smooth projective rational surface, $D$ is an anti-canonical cycle of rational curves and $Y\setminus D$ is the minimal resolution of an affine surface with, at worst, du Val singularities. Then, we compute periods for the compactified family using techniques from work of Ruddat-Siebert and use this to give a modular interpretation of the compactified mirror family.
In spite of a growing body of scientific evidence on the effectiveness of individual face mask usage for reducing transmission rates, individual face mask usage has become a highly polarized topic within the United States. A series of policy shifts by various governmental bodies have been speculated to have contributed to the polarization of face masks. A typical method to investigate the effects of these policy shifts is to use surveys. However, survey-based approaches have multiple limitations: biased responses, limited sample size, badly crafted questions may skew responses and inhibit insight, and responses may prove quickly irrelevant as opinions change in response to a dynamic topic. We propose a novel approach to 1) accurately gauge public sentiment towards face masks in the United States during COVID-19 using a multi-modal demographic inference framework with topic modeling and 2) determine whether face mask policy shifts contributed to polarization towards face masks using offline change point analysis on Twitter data. First, we infer several key demographics of individual Twitter users such as their age, gender, and whether they are a college student using a multi-modal demographic prediction framework and analyze the average sentiment for each respective demographic. Next, we conduct topic analysis using latent Dirichlet allocation (LDA). Finally, we conduct offline change point discovery on our sentiment time series data using the Pruned Exact Linear Time (PELT) search algorithm. Experimental results on a large corpus of Twitter data reveal multiple insights regarding demographic sentiment towards face masks that agree with existing surveys. Furthermore, we find two key policy-shift events contributed to statistically significant changes in sentiment for both Republicans and Democrats.