arXiv subjects

Roy H. Jennings

Publications and source records attributed to Roy H. Jennings.

7 recordsLinked to original sources

arXiv2025

Language Integration in Fine-Tuning Multimodal Large Language Models for Image-Based Regression

Multimodal Large Language Models (MLLMs) show promise for image-based regression tasks, but current approaches face key limitations. Recent methods fine-tune MLLMs using preset output vocabularies and generic task-level prompts (e.g., "How would you rate this image?"), assuming this mimics human rating behavior. Our analysis reveals that these approaches provide no benefit over image-only training. Models using preset vocabularies and generic prompts perform equivalently to image-only models, failing to leverage semantic understanding from textual input. We propose Regression via Transformer-Based Classification (RvTC), which replaces vocabulary-constrained classification with a flexible bin-based approach. Unlike approaches that address discretization errors through complex distributional modeling, RvTC eliminates manual vocabulary crafting through straightforward bin increase, achieving state-of-the-art performance on four image assessment datasets using only images. More importantly, we demonstrate that data-specific prompts dramatically improve performance. Unlike generic task descriptions, prompts containing semantic information about specific images enable MLLMs to leverage cross-modal understanding. On the AVA dataset, adding challenge titles to prompts substantially improves our already state-of-the-art image-only baseline. We demonstrate through empirical evidence from the AVA and AGIQA-3k datasets that MLLMs benefit from semantic prompt information, surpassing mere statistical biases. We validate RvTC across two different MLLM architectures, demonstrating consistent improvements and method generalizability.

cs.CV↗

arXiv2022

Reconciling a Centroid-Hypothesis Conflict in Source-Free Domain Adaptation

Source-free domain adaptation (SFDA) aims to transfer knowledge learned from a source domain to an unlabeled target domain, where the source data is unavailable during adaptation. Existing approaches for SFDA focus on self-training usually including well-established entropy minimization techniques. One of the main challenges in SFDA is to reduce accumulation of errors caused by domain misalignment. A recent strategy successfully managed to reduce error accumulation by pseudo-labeling the target samples based on class-wise prototypes (centroids) generated by their clustering in the representation space. However, this strategy also creates cases for which the cross-entropy of a pseudo-label and the minimum entropy have a conflict in their objectives. We call this conflict the centroid-hypothesis conflict. We propose to reconcile this conflict by aligning the entropy minimization objective with that of the pseudo labels' cross entropy. We demonstrate the effectiveness of aligning the two loss objectives on three domain adaptation datasets. In addition, we provide state-of-the-art results using up-to-date architectures also showing the consistency of our method across these architectures.

cs.CV↗

arXiv2021

HPTQ: Hardware-Friendly Post Training Quantization

Neural network quantization enables the deployment of models on edge devices. An essential requirement for their hardware efficiency is that the quantizers are hardware-friendly: uniform, symmetric, and with power-of-two thresholds. To the best of our knowledge, current post-training quantization methods do not support all of these constraints simultaneously. In this work, we introduce a hardware-friendly post training quantization (HPTQ) framework, which addresses this problem by synergistically combining several known quantization methods. We perform a large-scale study on four tasks: classification, object detection, semantic segmentation and pose estimation over a wide variety of network architectures. Our extensive experiments show that competitive results can be obtained under hardware-friendly constraints.

cs.CV↗

arXiv2020

Flip Graphs

Flip graphs are graphs on combinatorial objects in which the adjacency relation reflects a local change in the underlying objects. In this thesis we introduce Yoke graphs, a family of flip graphs that generalizes previously studied families of flip graphs on colored triangle-free triangulations, arc permutations and geometric caterpillars. Our main results are the computation of the diameter of an arbitrary Yoke graph and a full characterization of the automorphism group of this family of graphs. We also show that Yoke graphs are Schreier graphs of the affine Weyl group of type $\tilde{C}_m$. The approach we take in the computation of the diameter is different from the ones used for colored triangle-free triangulations and arc permutations. We show that the approach used for arc permutation graphs does not extend to Yoke graphs. At the heart of our proof lies the idea of transforming a diameter evaluation into an eccentricity problem. The characterization of the automorphism group is a new result for the above mentioned three special families of Yoke graphs.

math.CO↗

arXiv2020

HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNs

Recent work in network quantization produced state-of-the-art results using mixed precision quantization. An imperative requirement for many efficient edge device hardware implementations is that their quantizers are uniform and with power-of-two thresholds. In this work, we introduce the Hardware Friendly Mixed Precision Quantization Block (HMQ) in order to meet this requirement. The HMQ is a mixed precision quantization block that repurposes the Gumbel-Softmax estimator into a smooth estimator of a pair of quantization parameters, namely, bit-width and threshold. HMQs use this to search over a finite space of quantization schemes. Empirically, we apply HMQs to quantize classification models trained on CIFAR10 and ImageNet. For ImageNet, we quantize four different architectures and show that, in spite of the added restrictions to our quantization scheme, we achieve competitive and, in some cases, state-of-the-art results.

cs.LG↗

arXiv2020

Flip Graphs, Yoke Graphs and Diameter

In this paper we introduce Yoke graphs, a family of flip graphs that generalizes several previously studied families of graphs: colored triangle free triangulations, arc permutations and caterpillars. Our main result is the computation of the diameter of an arbitrary Yoke graph.

math.CO↗

arXiv2020

Geodesics in a Graph of Perfect Matchings

Let $\mathscr{P}_{m}$ be the graph on the set of perfect matchings in the complete graph $K_{2m}$, where two perfect matchings are connected by an edge if their symmetric difference is a cycle of length four. This paper studies geodesics in $\mathscr{P}_{m}$. The diameter of $\mathscr{P}_{m}$, as well as the eccentricity of each vertex, are shown to be $m-1$. Two proof are given to show that the number of geodesics between any two antipodes is $m^{m-2}$. The first is a direct proof via a recursive formula, and the second is via reduction to the number of minimal factorizations of a given $m$-cycle in the symmetric group $S_m$. An explicit formula for the number of geodesics between any two matchings in $\mathscr{P}_{m}$ is also given. Let $\mathscr{M}_m$ be the graph on the set of non-crossing perfect matchings of $2m$ labeled points on a circle with the same adjacency condition as in $\mathscr{P}_m$. $\mathscr{M}_m$ is an induced subgraph of $\mathscr{P}_m$, and it is shown that $\mathscr{M}_m$ has exactly one pair of antipodes having the maximal number ($m^{m-2}$) of geodesics between them.

math.CO↗