Searcharxiv⌕ Search

arXiv subjects

Junbo Jacob Lian

Publications and source records attributed to Junbo Jacob Lian.

9 recordsLinked to original sources

Do Language Models Know Which Constraints Belong Together?

Dantzig--Wolfe reformulation strengthens the linear relaxation of a mixed-integer program by convexifying constraint families, and each candidate costs a column generation run. The choice is hard when families strengthen the bound only together, as in classical lot-sizing, network-design, and facility-location formulations. We prove that when an evaluation convexifies at most $s$ of $k$ families, methods that observe only bounds need of order $k^w$ evaluations to find a hidden group of $w$ families in the worst case. A prediction missing $e$ members reduces this to $Θ(k^e)$ through a staged search that need not know $e$. The orders extend to several groups, prices, and growing evaluation costs. We study two predictors: a lookahead at an optimal LP dual, which vanishes on groups whose families share no variable, and language models reading the model, verified by the staged search in JointDW. Such groups appear in three of ten MIPLIB-NL models whose bound can be improved and in all 59 benchmark instances. Given the coefficients of constructed models with up to 16 families, one language model names a true group first in 249 of 252 answers. On the benchmark instances, decompositions chosen by language models from the text solve faster than those of GCG's detector.

cs.LG↗

Admission Without Answers: Label-Free Certification and Experience Learning for LLM-Based Optimization Modeling

Agents that learn from experience improve at optimization modeling by storing solved trajectories and reusing them as skills. A wrong trajectory that enters the library can be retrieved for many later problems, and a stream of new problems carries no ground-truth answers that could keep it out. Existing learners admit trajectories by matching known optima, and label-free substitutes such as execution success or agreement at one instance admit wrong models. We introduce AdmitOR, a label-free admission gate. It generates models from three model families and solves each on the stated problem and on instances with resampled parameters. It keeps the largest group of models whose optimal values agree on every instance and that spans more than one family. A threshold fitted on solver-verified problems maps this agreement to acceptance, abstention, or escalation, with a finite-sample bound on the false-discovery rate of accepted values. Inside a recent skill learner, AdmitOR raises admission precision from 0.871 under majority vote to 0.927. Its library is the smallest of the four compared and has the highest macro accuracy on five benchmarks. An ablation traces the gain to an accepted value external to the learner. One program from the learner's own model already raises precision to 0.923, and agreement across families raises it further. With one candidate per family, resampling can withhold a certificate but cannot change its value, and the values it withholds are wrong far more often than those it keeps, on the stream and on the solver-verified calibration set. The false-discovery bound holds on the calibration set but not on the benchmark stream. An audit traces every false certificate to a benchmark text that omits or rounds the numbers needed to reproduce the labeled answer, or to a label error. A label-free check of the extracted numbers against the text flags most omitted-number cases.

cs.AI↗

ReLoop: Structured Modeling and Behavioral Verification for Reliable LLM-Based Optimization

Large language models (LLMs) can translate natural-language problem descriptions into optimization code, but the code is prone to silent failures: it executes and returns a solver-feasible solution while encoding a semantically incorrect formulation. On compositional problems, the resulting feasibility-correctness gap reaches 90 percentage points. We introduce ReLoop, which combines two mechanisms. Structured generation decomposes code production into a four-stage reasoning chain (understand, formalize, synthesize, verify) to reduce formulation errors during generation. Behavioral verification detects the errors that remain by testing whether the formulation responds correctly to solver-based parameter perturbation, a signal that comes from the solver rather than from LLM self-review and requires no ground truth. The two mechanisms address different error structures: structured generation gives the largest gain on compositional problems (+8.5pp accuracy on RetailOpt-190 with Claude Opus 4.6), and behavioral verification gives its largest gain on localized defects (+4.4pp on MAMO-ComplexLP). With diagnostic execution recovery, ReLoop reaches 100% executable code on Claude Opus 4.6, and relative to direct generation it raises or preserves every reported metric of the three chat-tuned foundation models on all three benchmarks. For the narrowly fine-tuned SFT model we test, the chain-of-thought prompt conflicts with its learned output format and lowers its accuracy on MAMO-ComplexLP; we document and analyze this interaction. We release RetailOpt-190, 190 compositional retail optimization scenarios in which several constraints interact.

cs.SE↗

TwistNet-2D: Learning Second-Order Channel Interactions via Spiral Twisting for Texture Recognition

Second-order feature statistics are central to texture recognition, yet existing mechanisms exhibit a structural tension: bilinear pooling and Gram matrices capture global channel correlations but discard spatial structure, whereas self-attention models capture cross-position relations through weighted sums rather than explicit pairwise products. We propose TwistNet-2D, a lightweight module that computes local pairwise channel products under directional spatial displacement, jointly encoding where features co-occur and how they interact. The core component, Spiral-Twisted Channel Interaction (STCI), shifts one feature map along a prescribed direction before L2-normalized channel multiplication, capturing cross-position co-occurrence patterns that characterize structured and periodic textures. Four directional heads are aggregated through content-adaptive channel reweighting, and the result is injected via a sigmoid-gated residual path with near-zero initialization. TwistNet-2D adds only approximately 3.5% parameters and approximately 2% FLOPs over ResNet-18. To isolate the contribution of architectural inductive bias from that of transfer learning, all models in this study are trained from scratch without ImageNet pretraining. Under this protocol, TwistNet-2D consistently surpasses parameter-matched baselines and substantially larger ConvNeXt and Swin Transformer backbones across four texture and fine-grained recognition benchmarks, while the multi-head structure produces interpretable, orientation-selective representations that align with classical texture analysis.

cs.CV↗

Wasserstein Evolution : Evolutionary Optimization as Phase Transition

Evolutionary algorithms (EAs) serve as powerful black-box optimizers inspired by biological evolution. However, most existing EAs predominantly focus on heuristic operators such as crossover and mutation, while usually overlooking underlying physical interpretability such as statistical mechanics and thermosdynamics. This theoretical void limits the principled understanding of algorithmic dynamics, hindering the systematic design of evolutionary search beyond ad-hoc heuristics. To bridge this gap, we first point out that evolutionary optimization can be conceptually reframed as a physical phase transition process. Building on this perspective, we establish the theoretical grounds by modeling the optimization dynamics as a Wasserstein gradient flow of free energy. Consequently, a robust and interpretable solver named Wasserstein Evolution (WE) is proposed. WE mathematically frames the trade-off between exploration and exploitation as a competition between potential gradient forces and entropic forces. This formulation guarantees convergence to the Boltzmann distribution, thereby minimizing free energy and maximizing entropy, which promotes highly diverse solutions. Extensive experiments on complex multimodal and physical potential functions demonstrate that WE achieves superior diversity and stability compared to established baselines.

cs.NE↗

OPAL: Operator-Programmed Algorithms for Landscape-Aware Black-Box Optimization

Black-box optimization often relies on evolutionary and swarm algorithms whose performance is highly problem dependent. We view an optimizer as a short program over a small vocabulary of search operators and learn this operator program separately for each problem instance. We instantiate this idea in Operator-Programmed Algorithms (OPAL), a landscape-aware framework for continuous black-box optimization that uses a small design budget with a standard differential evolution baseline to probe the landscape, builds a $k$-nearest neighbor graph over sampled points, and encodes this trajectory with a graph neural network. A meta-learner then maps the resulting representation to a phase-wise schedule of exploration, restart, and local search operators. On the CEC~2017 test suite, a single meta-trained OPAL policy is statistically competitive with state-of-the-art adaptive differential evolution variants and achieves significant improvements over simpler baselines under nonparametric tests. Ablation studies on CEC~2017 justify the choices for the design phase, the trajectory graph, and the operator-program representation, while the meta-components add only modest wall-clock overhead. Overall, the results indicate that operator-programmed, landscape-aware per-instance design is a practical way forward beyond ad hoc metaphor-based algorithms in black-box optimization.

cs.NE↗

Twisted Convolutional Networks (TCNs): Enhancing Feature Interactions for Non-Spatial Data Classification

Twisted Convolutional Networks (TCNs) are proposed as a novel deep learning architecture for classifying one-dimensional data with arbitrary feature order and minimal spatial relationships. Unlike conventional Convolutional Neural Networks (CNNs) that rely on structured feature sequences, TCNs explicitly combine subsets of input features through theoretically grounded multiplicative and pairwise interaction mechanisms to create enriched representations. This feature combination strategy, formalized through polynomial feature expansions, captures high-order feature interactions that traditional convolutional approaches miss. We provide a comprehensive mathematical framework for TCNs, demonstrating how the twisted convolution operation generalizes standard convolutions while maintaining computational tractability. Through extensive experiments on five benchmark datasets from diverse domains (medical diagnostics, political science, synthetic data, chemometrics, and healthcare), we show that TCNs achieve statistically significant improvements over CNNs, Residual Networks (ResNet), Graph Neural Networks (GNNs), DeepSets, and Support Vector Machine (SVM). The performance gains are validated through statistical testing. TCNs also exhibit superior training stability and generalization capabilities, highlighting their robustness for non-spatial data classification tasks.

cs.CV↗

Time-Fair Benchmarking for Metaheuristics: A Restart-Fair Protocol for Fixed-Time Comparisons

Numerous purportedly improved metaheuristics claim superior performance based on equivalent function evaluations (FEs), yet often conceal additional computational burdens in more intensive iterations, preprocessing stages, or hyperparameter tuning. This paper posits that wall-clock time, rather than solely FEs, should serve as the principal budgetary constraint for equitable comparisons. We formalize a fixed-time, restart-fair benchmarking protocol wherein each algorithm is allotted an identical wall-clock time budget per problem instance, permitting unrestricted utilization of restarts, early termination criteria, and internal adaptive mechanisms. We advocate for the adoption of anytime performance curves, expected running time (ERT) metrics, and performance profiles that employ time as the cost measure, all aimed at predefined targets. Furthermore, we introduce a concise, reproducible checklist to standardize reporting practices and mitigate undisclosed computational overheads. This approach fosters more credible and practically relevant evaluations of metaheuristic algorithms.

cs.NE↗

Support Vector Boosting Machine (SVBM): Enhancing Classification Performance with AdaBoost and Residual Connections

In traditional boosting algorithms, the focus on misclassified training samples emphasizes their importance based on difficulty during the learning process. While using a standard Support Vector Machine (SVM) as a weak learner in an AdaBoost framework can enhance model performance by concentrating on error samples, this approach introduces significant challenges. Specifically, SVMs, characterized by their stability and robustness, may require destabilization to fit the boosting paradigm, which in turn can constrain performance due to reliance on the weighted results from preceding iterations. To address these challenges, we propose the Support Vector Boosting Machine (SVBM), which integrates a novel subsampling process with SVM algorithms and residual connection techniques. This method updates sample weights by considering both the current model's predictions and the outputs from prior rounds, allowing for effective sparsity control. The SVBM framework enhances the ability to form complex decision boundaries, thereby improving classification performance. The MATLAB source code for SVBM can be accessed at https://github.com/junbolian/SVBM.

cs.LG↗