Searcharxiv⌕ Search

arXiv subjects

Naoki Hamada

Publications and source records attributed to Naoki Hamada.

17 recordsLinked to original sources
arXiv2025

Constraint Qualification for Generic Parameter Families of Constraints in Optimization

Constraint qualifications (CQs) are central to the local analysis of constrained optimization. In this paper, we completely determine the validity of the four classical CQs -- LICQ, MFCQ, ACQ, and GCQ -- for constraint map-germs that arise in generic four-parameter families. Our approach begins by proving that all four CQs are invariant under the action of the group $\mathcal{K}[G]$ and under the operation of reduction. As a consequence, the verification of CQ-validity for a generic constraint reduces to checking CQ-validity on the $\mathcal{K}[G]$-normal forms of fully reduced map-germs. Such normal forms have been classified in our recent work. In the present paper, we verify which CQs hold in each germ appearing in the classification tables from that work. This analysis provides a complete picture of the generic landscape of the four classical CQs. Most notably, we find that there exist numerous generic map-germs for which GCQ holds while all stronger CQs fail, showing that the gap between GCQ and the other qualifications is not an exceptional phenomenon but arises generically.

math.OC↗
arXiv2024

Characterization of generic parameter families of constraint mappings in optimization

The purpose of this paper is to understand generic behavior of constraint functions in optimization problems relying on singularity theory of smooth mappings. To this end, we will focus on the subgroup $\mathcal{K}[G]$ of the Mather's group $\mathcal{K}$, whose action to constraint map-germs preserves the corresponding feasible set-germs (i.e.~the set consisting of points satisfying the constraints). We will classify map-germs with small stratum $\mathcal{K}[G]_e$-codimensions, and calculate the codimensions of the $\mathcal{K}[G]$-orbits of jets represented by germs in the classification lists and those of the complements of these orbits. Applying these results and a variant of the transversality theorem, we will show that families of constraint mappings whose germ at any point in the corresponding feasible set is $\mathcal{K}[G]$-equivalent to one of the normal forms in the classification list compose a residual set in the entire space of constraint mappings with at most $4$-parameters. These results enable us to quantify genericity of given constraint mappings, and thus evaluate to what extent known test suites are generic.

math.GT↗
arXiv2022

B\'ezier Flow: a Surface-wise Gradient Descent Method for Multi-objective Optimization

In this paper, we propose a strategy to construct a multi-objective optimization algorithm from a single-objective optimization algorithm by using the B\'ezier simplex model. Also, we extend the stability of optimization algorithms in the sense of Probability Approximately Correct (PAC) learning and define the PAC stability. We prove that it leads to an upper bound on the generalization with high probability. Furthermore, we show that multi-objective optimization algorithms derived from a gradient descent-based single-objective optimization algorithm are PAC stable. We conducted numerical experiments and demonstrated that our method achieved lower generalization errors than the existing multi-objective optimization algorithm.

math.OC↗
arXiv2022

A Two-phase Framework with a B\'{e}zier Simplex-based Interpolation Method for Computationally Expensive Multi-objective Optimization

This paper proposes a two-phase framework with a B\'{e}zier simplex-based interpolation method (TPB) for computationally expensive multi-objective optimization. The first phase in TPB aims to approximate a few Pareto optimal solutions by optimizing a sequence of single-objective scalar problems. The first phase in TPB can fully exploit a state-of-the-art single-objective derivative-free optimizer. The second phase in TPB utilizes a B\'{e}zier simplex model to interpolate the solutions obtained in the first phase. The second phase in TPB fully exploits the fact that a B\'{e}zier simplex model can approximate the Pareto optimal solution set by exploiting its simplex structure when a given problem is simplicial. We investigate the performance of TPB on the 55 bi-objective BBOB problems. The results show that TPB performs significantly better than HMO-CMA-ES and some state-of-the-art meta-model-based optimizers.

math.OC↗
arXiv2022

Gen\'eLive! Generating Rhythm Actions in Love Live!

This article presents our generative model for rhythm action games together with applications in business operations. Rhythm action games are video games in which the player is challenged to issue commands at the right timings during a music session. The timings are rendered in the chart, which consists of visual symbols, called notes, flying through the screen. We introduce our deep generative model, Gen\'eLive!, which outperforms the state-of-the-art model by taking into account musical structures through beats and temporal scales. Thanks to its favorable performance, Gen\'eLive! was put into operation at KLab Inc., a Japan-based video game developer, and reduced the business cost of chart generation by as much as half. The application target included the phenomenal "Love Live!," which has more than 10 million users across Asia and beyond, and is one of the few rhythm action franchises that has led the online era of the genre. In this article, we evaluate the generative performance of Gen\'eLive! using production datasets at KLab as well as open datasets for reproducibility, while the model continues to operate in their business. Our code and the model, tuned and trained using a supercomputer, are publicly available.

cs.LG↗
arXiv2021

Explicitly Multi-Modal Benchmarks for Multi-Objective Optimization

In multi-objective optimization, designing good benchmark problems is an important issue for improving solvers. Controlling the global location of Pareto optima in existing benchmark problems has been problematic, and it is even more difficult when the design space is high-dimensional since visualization is extremely challenging. As a benchmarking with explicit local Pareto fronts, we introduce a benchmarking based on basin connectivity (3BC) by using basins of attraction. The 3BC allows for the specification of a multimodal landscape through a kind of topological analysis called the basin graph, effectively generating optimization problems from this graph. Various known indicators measure the performance of a solver in searching global Pareto optima, but using 3BC can make us localize them for each local Pareto front by restricting it to its basin. 3BC's mathematical formulation ensures the accurate representation of the specified optimization landscape, guaranteeing the existence of intended local and global Pareto optima.

math.OC↗
arXiv2021

All unconstrained strongly convex problems are weakly simplicial

A multi-objective optimization problem is $C^r$ weakly simplicial if there exists a $C^r$ surjection from a simplex onto the Pareto set/front such that the image of each subsimplex is the Pareto set/front of a subproblem, where $0\leq r\leq \infty$. This property is helpful to compute a parametric-surface approximation of the entire Pareto set and Pareto front. It is known that all unconstrained strongly convex $C^r$ problems are $C^{r-1}$ weakly simplicial for $1\leq r \leq \infty$. In this paper, we show that all unconstrained strongly convex problems are $C^0$ weakly simplicial. The usefulness of this theorem is demonstrated in a sparse modeling application: we reformulate the elastic net as a non-differentiable multi-objective strongly convex problem and approximate its Pareto set (the set of all trained models with different hyper-parameters) and Pareto front (the set of performance metrics of the trained models) by using a B\'ezier simplex fitting method, which accelerates hyper-parameter search.

math.OC↗
arXiv2021

Approximate Bayesian Computation of B\'ezier Simplices

B\'ezier simplex fitting algorithms have been recently proposed to approximate the Pareto set/front of multi-objective continuous optimization problems. These new methods have shown to be successful at approximating various shapes of Pareto sets/fronts when sample points exactly lie on the Pareto set/front. However, if the sample points scatter away from the Pareto set/front, those methods often likely suffer from over-fitting. To overcome this issue, in this paper, we extend the B\'ezier simplex model to a probabilistic one and propose a new learning algorithm of it, which falls into the framework of approximate Bayesian computation (ABC) based on the Wasserstein distance. We also study the convergence property of the Wasserstein ABC algorithm. An extensive experimental evaluation on publicly available problem instances shows that the new algorithm converges on a finite sample. Moreover, it outperforms the deterministic fitting methods on noisy instances.

cs.LG↗
arXiv2019

Simpliciality of strongly convex problems

A multiobjective optimization problem is $C^r$ simplicial if the Pareto set and the Pareto front are $C^r$ diffeomorphic to a simplex and, under the $C^r$ diffeomorphisms, each face of the simplex corresponds to the Pareto set and the Pareto front of a subproblem, where $0\leq r\leq \infty$. In the paper titled "Topology of Pareto sets of strongly convex problems," it has been shown that a strongly convex $C^r$ problem is $C^{r-1}$ simplicial under a mild assumption on the ranks of the differentials of the mapping for $2\leq r \leq \infty$. On the other hand, in this paper, we show that a strongly convex $C^1$ problem is $C^0$ simplicial under the same assumption. Moreover, we establish a specialized transversality theorem on generic linear perturbations of a strongly convex $C^r$ mapping $(r\geq 2)$. By the transversality theorem, we also give an application of singularity theory to a strongly convex $C^r$ problem for $2\leq r \leq \infty$.

math.OC↗
arXiv2019

Characterization of the equality of weak efficiency and efficiency on convex free disposal hulls

In solving a multi-objective optimization problem by scalarization techniques, solutions to a scalarized problem are, in general, weakly efficient rather than efficient to the original problem. Thus, it is crucial to understand what problem ensures that all weakly efficient solutions are efficient. In this paper, we give a characterization of the equality of the weakly efficient set and the efficient set, provided that the free disposal hull of the domain is convex. By using this characterization, we obtain various mathematical applications. As a practical application, we show that all weakly efficient solutions to a multi-objective LASSO with mild modification are efficient.

math.OC↗
arXiv2019

Asymptotic Risk of Bezier Simplex Fitting

The Bezier simplex fitting is a novel data modeling technique which exploits geometric structures of data to approximate the Pareto front of multi-objective optimization problems. There are two fitting methods based on different sampling strategies. The inductive skeleton fitting employs a stratified subsampling from each skeleton of a simplex, whereas the all-at-once fitting uses a non-stratified sampling which treats a simplex as a whole. In this paper, we analyze the asymptotic risks of those B\'ezier simplex fitting methods and derive the optimal subsample ratio for the inductive skeleton fitting. It is shown that the inductive skeleton fitting with the optimal ratio has a smaller risk when the degree of a Bezier simplex is less than three. Those results are verified numerically under small to moderate sample sizes. In addition, we provide two complementary applications of our theory: a generalized location problem and a multi-objective hyper-parameter tuning of the group lasso. The former can be represented by a Bezier simplex of degree two where the inductive skeleton fitting outperforms. The latter can be represented by a Bezier simplex of degree three where the all-at-once fitting gets an advantage.

cs.LG↗
arXiv2019

Topology of Pareto sets of strongly convex problems

A multiobjective optimization problem is simplicial if the Pareto set and front are homeomorphic to a simplex and, under the homeomorphisms, each face of the simplex corresponds to the Pareto set and front of a subproblem. In this paper, we show that strongly convex problems are simplicial under a mild assumption on the ranks of the differentials of the objective mappings. We further prove that one can make any strongly convex problem satisfy the assumption by a generic linear perturbation, provided that the dimension of the source is sufficiently larger than that of the target. We demonstrate that the location problems, a biological modeling, and the ridge regression can be reduced to multiobjective strongly convex problems via appropriate transformations preserving the Pareto ordering and the topology.

math.OC↗
arXiv2018

Bezier Simplex Fitting: Describing Pareto Fronts of Simplicial Problems with Small Samples in Multi-objective Optimization

Multi-objective optimization problems require simultaneously optimizing two or more objective functions. Many studies have reported that the solution set of an M-objective optimization problem often forms an (M-1)-dimensional topological simplex (a curved line for M=2, a curved triangle for M=3, a curved tetrahedron for M=4, etc.). Since the dimensionality of the solution set increases as the number of objectives grows, an exponentially large sample size is needed to cover the solution set. To reduce the required sample size, this paper proposes a Bezier simplex model and its fitting algorithm. These techniques can exploit the simplex structure of the solution set and decompose a high-dimensional surface fitting task into a sequence of low-dimensional ones. An approximation theorem of Bezier simplices is proven. Numerical experiments with synthetic and real-world optimization problems demonstrate that the proposed method achieves an accurate approximation of high-dimensional solution sets with small samples. In practice, such an approximation will be conducted in the post-optimization process and enable a better trade-off analysis.

math.OC↗
arXiv2018

Data-Driven Analysis of Pareto Set Topology

When and why can evolutionary multi-objective optimization (EMO) algorithms cover the entire Pareto set? That is a major concern for EMO researchers and practitioners. A recent theoretical study revealed that (roughly speaking) if the Pareto set forms a topological simplex (a curved line, a curved triangle, a curved tetrahedron, etc.), then decomposition-based EMO algorithms can cover the entire Pareto set. Usually, we cannot know the true Pareto set and have to estimate its topology by using the population of EMO algorithms during or after the runtime. This paper presents a data-driven approach to analyze the topology of the Pareto set. We give a theory of how to recognize the topology of the Pareto set from data and implement an algorithm to judge whether the true Pareto set may form a topological simplex or not. Numerical experiments show that the proposed method correctly recognizes the topology of high-dimensional Pareto sets within reasonable population size.

cs.NE↗
arXiv2017

Simple Problems: The Simplicial Gluing Structure of Pareto Sets and Pareto Fronts

Quite a few studies on real-world applications of multi-objective optimization reported that their Pareto sets and Pareto fronts form a topological simplex. Such a class of problems was recently named the simple problems, and their Pareto set and Pareto front were observed to have a gluing structure similar to the faces of a simplex. This paper gives a theoretical justification for that observation by proving the gluing structure of the Pareto sets/fronts of subproblems of a simple problem. The simplicity of standard benchmark problems is studied.

math.OC↗
arXiv2015

Population Synthesis via k-Nearest Neighbor Crossover Kernel

The recent development of multi-agent simulations brings about a need for population synthesis. It is a task of reconstructing the entire population from a sampling survey of limited size (1% or so), supplying the initial conditions from which simulations begin. This paper presents a new kernel density estimator for this task. Our method is an analogue of the classical Breiman-Meisel-Purcell estimator, but employs novel techniques that harness the huge degree of freedom which is required to model high-dimensional nonlinearly correlated datasets: the crossover kernel, the k-nearest neighbor restriction of the kernel construction set and the bagging of kernels. The performance as a statistical estimator is examined through real and synthetic datasets. We provide an "optimization-free" parameter selection rule for our method, a theory of how our method works and a computational cost analysis. To demonstrate the usefulness as a population synthesizer, our method is applied to a household synthesis task for an urban micro-simulator.

cs.NE↗