Speaker: Gersende Fort
Affiliation: Institut de Mathématiques de Toulouse, France

Abstract

Majorize-Minimization (MM) algorithms are iterative procedures for solving a minimization problem. The sequence of iterates is obtained as follows: at each iteration, a majorizing function G of the objective function F is built and the next iterate is defined as the minimum of this surrogate function G. The surrogate G depends on the current iterate in such a way that this MM strategy ensures a descent property of the objective function F along the iterates. Expectation-Maximization (EM) is a famous special case of MM.

Unfortunately, in the large scale learning setting when the objective function is a sum over a very large batch of examples (possibly a batch of 'infinite' size -- in a sense to be precised in the talk), exact MM can not be applied. This is well known in the EM context for example and many incremental versions were introduced in the literature to deal with this computational intractability.

In this talk, we will first show how (exact) MM is a root finding procedure; this will be a fundamental step to give the intuition of incremental stochastic MM algorithms.
For pedagogical purposes, we will essentially use EM as an illustration of the general MM.

Then, a new incremental technique called "SPIDER" will be introduced in order to reduce the variance of the stochastic approximation.

We will finally derive explicit control of convergence of this novel variance-reduced MM algorithm.

The talk is based on joint works with Eric Moulines (Ecole Polytechnique, France) and Hoi-To Wai (Chinese University of Hong-Kong, 
Hong-Kong).

About Statistics, modelling and operations research seminars

Students, staff and visitors to UQ are welcome to attend our regular seminars.

The events are jointly run by our Operations research and Statistics and probability research groups.

The Statistics, modelling and operations research (SMOR) Seminar series seeks to celebrate and disseminate research and developments across the broad spectrum of quantitative sciences. The SMOR series provides a platform for communication of both theoretical and practical developments, as well as interdisciplinary topics relating to applied mathematics and statistics.

Venue

Priestley Building (67)
Room: 
348 or via Zoom (https://uqz.zoom.us/j/85172010876)