Foundations of Distributed and Large Scale Computing
Optimization
Instructor
Emilie
CHOUZENOUX, Inria Saclay, CVN
Teaching Assistant
Laura GIROMETTI, CentraleSupélec, CVN
Objective
In a wide range of application fields (inverse problems,
machine learning, computer vision,
data analysis, networking,...), large scale optimization problems need
to be solved.
The objective of this course is to introduce the theoretical
background which makes
it possible to develop efficient algorithms to successfully address
these problems
by taking advantage of modern multicore or distributed computing
architectures.
This course will be mainly focused on nonlinear optimization tools
for dealing with convex problems. Proximal tools, splitting techniques
and Majorization-Minimization strategies which are now
very popular for processing massive datasets will be presented.
Illustrations of these methods on various applicative examples
will be provided.
Course outline
The course consists of
eight sessions (3h each) combining lectures and
exercices. The following concepts will be presented:
Logistic informations regarding the course are summarized in [slides].
A selective bibliography can be found in [pdf].
1. Background on convex analysis
- 1.1. Convex sets and functions
- 1.2. Differentiability and subdifferentiability
- 1.3. Proximity operator
- 1.4. Conjugate function
- 1.5. Duality results
[Slides (1st part)] [Slides (2nd part)]
[Slides (3rd part)] [Slides (4th part)]
[Slides (3rd part with annotations)][Slides (4th part with annotations)]
[Video]
TD subject: Solve the exercices in the slides. [Correction]
TP subject: Convex optimization applied to image denoising. [Subject] [Image]
2. Parallel and distributed proximal splitting methods
- 2.1. Fixed point algorithm and Fejér sequences
- 2.2. Minimization of a sum of convex functions
- 2.3. Primal-dual proximal parallel algorithms
- 2.4. Distributed techniques based on consensus approaches
[Slides (1st part)] [Slides (1st part with annotations)]
[Slides (1st part - DR)][Slides (2nd part)]
TP subject 1: Proximal minimization methods for spectroscopy.
[Subject (Part I)] [Subject (Part II)] [Signal]
TP subject 2: Primal-dual techniques for database request optimization.
[Subject] [Code]
3. Parallelization through Majorization-Minimization approaches
- 3.1. Majorization-Minimization principle
- 3.2. Majorization techniques
- 3.3. Half-Quadratic methods
- 3.4. Variable metric Forward-Backward algorithm
- 3.5. Subspace algorithms (memory gradient, BFGS, ...)
- 3.6. Parallel approaches using block-coordinate strategies
[Slides]
TP subject: Parallel MM algorithms for tomography image reconstruction.
[Subject][Data]
Useful informations
- Report: code + answer to questions (jupyter notebook using Python 3), 1 report for 2 students
- Each report is due no later than two weeks after the corresponding lab session.
- Send your reports at lauraDOTgiromettiATcentralesupelecDOTfr with email object LSD_TPnX_Name1_Name2 and file name LSD_TPnX_Name1_Name2.ipynb
- Research paper to study for the exam 2026/2027: [TBA]
- Exam of past year: [Subject]
- Emilie Chouzenoux -
Retour à la page précédente