arXiv subjects

Andrew Li

Publications and source records attributed to Andrew Li.

30 records · Page 2Linked to original sources

arXiv2024

FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Search

Quantization has become a mainstream compression technique for reducing model size, computational requirements, and energy consumption for modern deep neural networks (DNNs). With improved numerical support in recent hardware, including multiple variants of integer and floating point, mixed-precision quantization has become necessary to achieve high-quality results with low model cost. Prior mixed-precision methods have performed either a post-training quantization search, which compromises on accuracy, or a differentiable quantization search, which leads to high memory usage from branching. Therefore, we propose the first one-shot mixed-precision quantization search that eliminates the need for retraining in both integer and low-precision floating point models. We evaluate our search (FLIQS) on multiple convolutional and vision transformer networks to discover Pareto-optimal models. Our approach improves upon uniform precision, manual mixed-precision, and recent integer quantization search methods. With integer models, we increase the accuracy of ResNet-18 on ImageNet by 1.31% and ResNet-50 by 0.90% with equivalent model cost over previous methods. Additionally, for the first time, we explore a novel mixed-precision floating-point search and improve MobileNetV2 by up to 0.98% compared to prior state-of-the-art FP8 models. Finally, we extend FLIQS to simultaneously search a joint quantization and neural architecture space and improve the ImageNet accuracy by 2.69% with similar model cost on a MobileNetV2 search space.

cs.CV↗

arXiv2024

On the set partitions that require maximum sorts through the $aba-$avoiding stack

Recently, Xia introduced a deterministic variation $ϕ_σ$ of Defant and Kravitz's stack-sorting maps for set partitions and showed that any set partition $p$ is sorted by $ϕ^{N(p)}_{aba}$, where $N(p)$ is the number of distinct alphabets in $p$. Xia then asked which set partitions $p$ are not sorted by $ϕ_{aba}^{N(p)-1}$. In this note, we prove that the minimal length of a set partition $p$ that is not sorted by $ϕ_{aba}^{N(p)-1}$ is $2N(p)$. Then we show that there is only one set partition of length $2N(p)$ and ${{N(p) + 1} \choose 2} + 2{N(p) \choose 2}$ set partitions of length $2N(p)+1$ that are not sorted by $ϕ_{aba}^{N(p)-1}$.

math.CO↗

arXiv2023

Markdown Pricing Under an Unknown Parametric Demand Model

Consider a single-product revenue-maximization problem where the seller monotonically decreases the price in $n$ rounds with an unknown demand model coming from a given family. Without monotonicity, the minimax regret is $\tilde O(n^{2/3})$ for the Lipschitz demand family and $\tilde O(n^{1/2})$ for a general class of parametric demand models. With monotonicity, the minimax regret is $\tilde O(n^{3/4})$ if the revenue function is Lipschitz and unimodal. However, the minimax regret for parametric families remained open. In this work, we provide a complete settlement for this fundamental problem. We introduce the crossing number to measure the complexity of a family of demand functions. In particular, the family of degree-$k$ polynomials has a crossing number $k$. Based on conservatism under uncertainty, we present (i) a policy with an optimal $Θ(\log^2 n)$ regret for families with crossing number $k=0$, and (ii) another policy with an optimal $\tilde Θ(n^{k/(k+1)})$ regret when $k\ge 1$. These bounds are asymptotically higher than the $\tilde O(\log n)$ and $\tilde Θ(\sqrt n)$ minimax regret for the same families without the monotonicity constraint.

cs.LG↗

arXiv2023

Short-lived High-volume Multi-A(rmed)/B(andits) Testing

Modern platforms leverage randomized experiments to make informed decisions from a given set of items (``treatments''). As a particularly challenging scenario, these items may (i) arrive in high volume, with thousands of new items being released per hour, and (ii) have short lifetime, say, due to the item's transient nature or underlying non-stationarity that impels the platform to perceive the same item as distinct copies over time. Thus motivated, we study a Bayesian multiple-play bandit problem that encapsulates the key features of the multivariate testing (or ``multi-A/B testing'') problem with a high volume of short-lived arms. In each round, a set of $k$ arms arrive, each available for $w$ rounds. Without knowing the mean reward for each arm, the learner selects a multiset of $n$ arms and immediately observes their realized rewards. We aim to minimize the loss due to not knowing the mean rewards, averaged over instances generated from a given prior distribution. We show that when $k = O(n^ρ)$ for some constant $ρ>0$, our proposed policy has $\tilde O(n^{-\min \{ρ, \frac 12 (1+\frac 1w)^{-1}\}})$ loss on a sufficiently large class of prior distributions. We complement this result by showing that every policy suffers $Ω(n^{-\min \{ρ, \frac 12\}})$ loss on the same class of distributions. We further validate the effectiveness of our policy through a large-scale field experiment on {\em Glance}, a content-card-serving platform that faces exactly the above challenge. A simple variant of our policy outperforms the platform's current recommender by 4.32\% in total duration and 7.48\% in total number of click-throughs.

cs.LG↗

arXiv2023

The Optical Phase Curves of CoRoT-1 b

Of the three space telescopes launched so far to survey transiting extrasolar planets, CoRoT is unique in that it was the only one with spectral resolution, allowing for an extraordinary opportunity to study the reflective properties of exoplanets at different wavelengths. In this work, I present a systematic lightcurve analysis of the white-light and chromatic CoRoT lightcurves of CoRoT-1 in order to search for the secondary eclipse and orbital phase variation of the transiting extrasolar planet CoRoT-1 b, as well at search for any chromatic difference in the aforementioned effects. I manage to detect a significant secondary eclipse in the white lightcurve, and detect the eclipse marginally in all three of the color channels. However I am only able to significantly detect the planetary phase variation in the red channel lightcurve. The retrieved secondary eclipse depth is higher in the blue and green channels compared to the white and red, suggesting that CoRoT-1 b has a higher geometric albedo at shorter wavelengths. I also attempt to detect the secondary eclipse using TESS, but show that the available volume and precision of the data is not high enough to allow detection of the secondary eclipse.

astro-ph.EP↗

arXiv2023

No Evidence for Additional Planets at GJ 3470 from TESS and Archival Radial Velocities

The nearby M2 dwarf GJ 3470 has been the target of considerable interest after the discovery of a transiting short-period Neptune-sized planet. Recently, claims regarding the existence of additional transiting planets has gotten some attention, suggesting both the presence of a gas giant in the habitable zone, and that the system hosts a remarkable co-orbital gas giant configuration. We show that the existence of these three additional planets are readily amenable to testing with available data from both ground-based radial velocity data and space-based TESS photometry. A periodogram search of the available radial velocities show no compelling signals at the claimed periods, and the TESS photometry effectively rules out these planets assuming a transiting configuration. While it is doubtlessly possible that additional planets orbit GJ 3470, there is no evidence to date for their existence, and the available data conclusively rule out any planets similar to those considered in this text.

astro-ph.EP↗

arXiv2023

Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generation

Automated generation of clinically accurate radiology reports can improve patient care. Previous report generation methods that rely on image captioning models often generate incoherent and incorrect text due to their lack of relevant domain knowledge, while retrieval-based attempts frequently retrieve reports that are irrelevant to the input image. In this work, we propose Contrastive X-Ray REport Match (X-REM), a novel retrieval-based radiology report generation module that uses an image-text matching score to measure the similarity of a chest X-ray image and radiology report for report retrieval. We observe that computing the image-text matching score with a language-image model can effectively capture the fine-grained interaction between image and text that is often lost when using cosine similarity. X-REM outperforms multiple prior radiology report generation modules in terms of both natural language and clinical metrics. Human evaluation of the generated reports suggests that X-REM increased the number of zero-error reports and decreased the average error severity compared to the baseline retrieval approach. Our code is available at: https://github.com/rajpurkarlab/X-REM

cs.CL↗

arXiv2021

Greedy Approximation Algorithms for Active Sequential Hypothesis Testing

In the problem of active sequential hypothesis testing (ASHT), a learner seeks to identify the true hypothesis from among a known set of hypotheses. The learner is given a set of actions and knows the random distribution of the outcome of any action under any true hypothesis. Given a target error $δ>0$, the goal is to sequentially select the fewest number of actions so as to identify the true hypothesis with probability at least $1 - δ$. Motivated by applications in which the number of hypotheses or actions is massive (e.g., genomics-based cancer detection), we propose efficient (greedy, in fact) algorithms and provide the first approximation guarantees for ASHT, under two types of adaptivity. Both of our guarantees are independent of the number of actions and logarithmic in the number of hypotheses. We numerically evaluate the performance of our algorithms using both synthetic and real-world DNA mutation data, demonstrating that our algorithms outperform previously proposed heuristic policies by large margins.

cs.LG↗

arXiv2021

LTL2Action: Generalizing LTL Instructions for Multi-Task RL

We address the problem of teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments. Instructions are expressed in a well-known formal language -- linear temporal logic (LTL) -- and can specify a diversity of complex, temporally extended behaviours, including conditionals and alternative realizations. Our proposed learning approach exploits the compositional syntax and the semantics of LTL, enabling our RL agent to learn task-conditioned policies that generalize to new instructions, not observed during training. To reduce the overhead of learning LTL semantics, we introduce an environment-agnostic LTL pretraining scheme which improves sample-efficiency in downstream environments. Experiments on discrete and continuous domains target combinatorial task sets of up to $\sim10^{39}$ unique tasks and demonstrate the strength of our approach in learning to solve (unseen) tasks, given LTL instructions.

cs.AI↗

arXiv2021

Searching for Fast Model Families on Datacenter Accelerators

Neural Architecture Search (NAS), together with model scaling, has shown remarkable progress in designing high accuracy and fast convolutional architecture families. However, as neither NAS nor model scaling considers sufficient hardware architecture details, they do not take full advantage of the emerging datacenter (DC) accelerators. In this paper, we search for fast and accurate CNN model families for efficient inference on DC accelerators. We first analyze DC accelerators and find that existing CNNs suffer from insufficient operational intensity, parallelism, and execution efficiency. These insights let us create a DC-accelerator-optimized search space, with space-to-depth, space-to-batch, hybrid fused convolution structures with vanilla and depthwise convolutions, and block-wise activation functions. On top of our DC accelerator optimized neural architecture search space, we further propose a latency-aware compound scaling (LACS), the first multi-objective compound scaling method optimizing both accuracy and latency. Our LACS discovers that network depth should grow much faster than image size and network width, which is quite different from previous compound scaling results. With the new search space and LACS, our search and scaling on datacenter accelerators results in a new model series named EfficientNet-X. EfficientNet-X is up to more than 2X faster than EfficientNet (a model series with state-of-the-art trade-off on FLOPs and accuracy) on TPUv3 and GPUv100, with comparable accuracy. EfficientNet-X is also up to 7X faster than recent RegNet and ResNeSt on TPUv3 and GPUv100.

cs.CV↗

arXiv2019

Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data

The ability to segment unknown objects in depth images has potential to enhance robot skills in grasping and object tracking. Recent computer vision research has demonstrated that Mask R-CNN can be trained to segment specific categories of objects in RGB images when massive hand-labeled datasets are available. As generating these datasets is time consuming, we instead train with synthetic depth images. Many robots now use depth sensors, and recent results suggest training on synthetic depth data can transfer successfully to the real world. We present a method for automated dataset generation and rapidly generate a synthetic training dataset of 50,000 depth images and 320,000 object masks using simulated heaps of 3D CAD models. We train a variant of Mask R-CNN with domain randomization on the generated dataset to perform category-agnostic instance segmentation without any hand-labeled data and we evaluate the trained network, which we refer to as Synthetic Depth (SD) Mask R-CNN, on a set of real, high-resolution depth images of challenging, densely-cluttered bins containing objects with highly-varied geometry. SD Mask R-CNN outperforms point cloud clustering baselines by an absolute 15% in Average Precision and 20% in Average Recall on COCO benchmarks, and achieves performance levels similar to a Mask R-CNN trained on a massive, hand-labeled RGB dataset and fine-tuned on real images from the experimental setup. We deploy the model in an instance-specific grasping pipeline to demonstrate its usefulness in a robotics application. Code, the synthetic training dataset, and supplementary material are available at https://bit.ly/2letCuE.

cs.CV↗

arXiv2014

Third sound measurements of superfluid $^4$He films on multiwall carbon nanotubes below 1K

Third sound is studied for superfluid films of 4He adsorbed on multiwall carbon nanotubes packed into an annular resonator. The third sound is generated with mechanical oscillation of the cell, and detected with carbon bolometers. A filling curve at temperatures near 250 mK shows oscillations in the third sound velocity, with maxima at the completion of the 4th and 5th atomic layers. Sharp changes in the Q factor of the third sound are found at partial layer fillings. Temperature sweeps at a number of fill points show strong broadening effects on the Kosterlitz-Thouless (KT) transition, and rapidly increasing dissipation, in qualitative agreement with the predictions of Machta and Guyer. At the 4th layer completion there is a sudden reduction of the transition temperature $T_{KT}$, and then a recovery back to linear variation with temperature, although the slope is considerably smaller than the KT prediction. Some of these effects may be related to changes in the gas-liquid coexistence regions.

cond-mat.other↗