Nash-Bargaining HalpernSGD via Limited-Memory MultiLRSGA: A Two-Phase Optimizer for Multi-Objective Learning

Abstract

We propose NB-HalpernSGD via LM-MultiLRSGA, a two-phase optimizer for multi-objective optimization. The process starts with a competitive optimization phase by applying a limited-memory variant of MultiLRSGA, named LM-MultiLRSGA, designed to reduce the memory footprint of the original optimizer while preserving its rotational correction mechanism. It addresses multi-objective tasks by considering the competitive game associated with the original multi-objective problem, approximating a Nash equilibrium. This point defines a Nash-equilibrium-induced disagreement point: we formulate the Nash bargaining problem associated with the original losses and rewrite its Nash product as a logarithmic minimization surrogate. This minimization problem is solved using HalpernSGD, anchored at the computed competitive reference point. Therefore, the method uses the Nash equilibrium as a principled reference point for the bargaining stage and then moves toward a Pareto-oriented solution of the original multi-objective problem. We validate the proposed optimizer on a PINN-inspired neural model, where it outperforms established multi-objective optimizers, including PCGrad, MultiAdam, and DualConeGD. Finally, while the convergence properties of HalpernSGD have been extensively studied, we discuss the convergence and stability properties of the proposed LM-MultiLRSGA phase.