Entropy linear programming
We propose an efficient dual algorithm for ELP based on Fast Gradient Method. The basic idea - to solve properly regularized dual problem.
math.OC↗
arXiv subjects
Alexey Chernov
Publications and source records attributed to Alexey Chernov.
Prediction with Advice of Unknown Number of Experts
In the framework of prediction with expert advice, we consider a recently introduced kind of regret bounds: the bounds that depend on the effective instead of nominal number of experts. In contrast to the Normal- Hedge bound, which mainly depends on the effective number of experts but also weakly depends on the nominal one, we obtain a bound that does not contain the nominal number of experts at all. We use the defensive forecasting method and introduce an application of defensive forecasting to multivalued supermartingales.
cs.LG↗
A shape calculus based method for a transmission problem with random interface
The present work is devoted to approximation of the statistical moments of the unknown solution of a class of elliptic transmission problems in $\mathbb R^3$ with randomly perturbed interfaces. Within this model, the diffusion coefficient has a jump discontinuity across the random transmission interface which models linear diffusion in two different media separated by an uncertain surface. We apply the shape calculus approach to approximate solution's perturbation by the so-called shape derivative, correspondingly statistical moments of the solution's perturbation are approximated by the moments of the shape derivative. We characterize the shape derivative as a solution of a related homogeneous transmission problem with nonzero jump conditions which can be solved with the aid of boundary integral equations. We develop a rigorous theoretical framework for this method, particularly i) extending the method to the case of unbounded domains and ii) closing the gaps and clarifying and adapting results in the existing literature. The theoretical findings are supported by and illustrated in two particular examples.
math.NA↗
New explicit-in-dimension estimates for the cardinality of high-dimensional hyperbolic crosses and approximation of functions having mixed smoothness
We are aiming at sharp and explicit-in-dimension estimations of the cardinality of $s$-dimensional hyperbolic crosses where $s$ may be large, and applications in high-dimensional approximations of functions having mixed smoothness. In particular, we provide new tight and explicit-in-dimension upper and lower bounds for the cardinality of hyperbolic crosses. We apply them to obtain explicit upper and lower bounds for Kolmogorov $N$-widths and $\varepsilon$-dimensions of a modified Korobov class parametrized by positive $a$ of $s$-variate periodic functions having mixed smoothness $r$, as a function of three variables $N,s,a$ and $\varepsilon, s,a$, respectively, when $N,s$ may be large, $\varepsilon$ may be small and $a$ may range from 0 to infinity. Based on these results we describe a complete classification of tractability for the problem of $\varepsilon$-dimensions of the modified Korobov class. In particular, we prove the introduced exponential tractability of this problem for $a>1$. All of these methods and results are also extended to high-dimensional approximations of non-periodic functions by Jacobi polynomials with powers in hyperbolic crosses.
math.NA↗
On Theorem 2.3 in "Prediction, Learning, and Games" by Cesa-Bianchi and Lugosi
The note presents a modified proof of a loss bound for the exponentially weighted average forecaster with time-varying potential. The regret term of the algorithm is upper-bounded by sqrt{n ln(N)} (uniformly in n), where N is the number of experts and n is the number of steps.
cs.LG↗
Prediction with Expert Advice under Discounted Loss
We study prediction with expert advice in the setting where the losses are accumulated with some discounting---the impact of old losses may gradually vanish. We generalize the Aggregating Algorithm and the Aggregating Algorithm for Regression to this case, propose a suitable new variant of exponential weights algorithm, and prove respective loss bounds.
cs.LG↗
Prediction with Advice of Unknown Number of Experts
In the framework of prediction with expert advice, we consider a recently introduced kind of regret bounds: the bounds that depend on the effective instead of nominal number of experts. In contrast to the NormalHedge bound, which mainly depends on the effective number of experts and also weakly depends on the nominal one, we obtain a bound that does not contain the nominal number of experts at all. We use the defensive forecasting method and introduce an application of defensive forecasting to multivalued supermartingales.
cs.LG↗
Supermartingales in Prediction with Expert Advice
We apply the method of defensive forecasting, based on the use of game-theoretic supermartingales, to prediction with expert advice. In the traditional setting of a countable number of experts and a finite number of outcomes, the Defensive Forecasting Algorithm is very close to the well-known Aggregating Algorithm. Not only the performance guarantees but also the predictions are the same for these two methods of fundamentally different nature. We discuss also a new setting where the experts can give advice conditional on the learner's future decision. Both the algorithms can be adapted to the new setting and give the same performance guarantees as in the traditional setting. Finally, we outline an application of defensive forecasting to a setting with several loss functions.
cs.LG↗
Aggregating Algorithm competing with Banach lattices
The paper deals with on-line regression settings with signals belonging to a Banach lattice. Our algorithms work in a semi-online setting where all the inputs are known in advance and outcomes are unknown and given step by step. We apply the Aggregating Algorithm to construct a prediction method whose cumulative loss over all the input vectors is comparable with the cumulative loss of any linear functional on the Banach lattice. As a by-product we get an algorithm that takes signals from an arbitrary domain. Its cumulative loss is comparable with the cumulative loss of any predictor function from Besov and Triebel-Lizorkin spaces. We describe several applications of our setting.
cs.LG↗
Prediction with expert evaluators' advice
We introduce a new protocol for prediction with expert advice in which each expert evaluates the learner's and his own performance using a loss function that may change over time and may be different from the loss functions used by the other experts. The learner's goal is to perform better or not much worse than each expert, as evaluated by that expert, for all experts simultaneously. If the loss functions used by the experts are all proper scoring rules and all mixable, we show that the defensive forecasting algorithm enjoys the same performance guarantee as that attainable by the Aggregating Algorithm in the standard setting and known to be optimal. This result is also applied to the case of "specialist" (or "sleeping") experts. In this case, the defensive forecasting algorithm reduces to a simple modification of the Aggregating Algorithm.
cs.LG↗
Monotone Conditional Complexity Bounds on Future Prediction Errors
We bound the future loss when predicting any (computably) stochastic sequence online. Solomonoff finitely bounded the total deviation of his universal predictor M from the true distribution m by the algorithmic complexity of m. Here we assume we are at a time t>1 and already observed x=x_1...x_t. We bound the future prediction performance on x_{t+1}x_{t+2}... by a new variant of algorithmic complexity of m given x, plus the complexity of the randomness deficiency of x. The new complexity is monotone in its condition in the sense that this complexity can only decrease if the condition is prolonged. We also briefly discuss potential generalizations to Bayesian model classes and to classification problems.
cs.LG↗