Seminars

Extrapolation methods use the last few iterates of an optimization algorithm to produce a better estimate of the optimum. They were shown to achieve optimal convergence rates in a deterministic setting using simple gradient iterates. A similar bound is also obtained when extrapolation is applied to stochastic methods with variance reduction, such as SAGA or SVRG. In practice, using the extrapolation step leads to an impressive acceleration in the convergence speed, and the resulting algorithm is usually much faster than optimal methods.

Spinal deformities such as adolescent idiopathic scoliosis are complex 3D deformations of the musculoskeletal trunk. For the past two decades, 3D spine reconstructions obtained from diagnostic scans have assisted orthopedists assess the severity of deformations and establish treatment strategies. However, these procedures required significant manual intervention and were not suited for routine clinical practice. This presentation will expose computational methods recently developed in our lab based on deep learning and statistical analysis to automatically segment the personalized spine geometry from X-rays or pre-operative CT/MRI, classify various deformation patterns in 3D, predict disease progression and perform intra-operative guidance during surgical procedures, with the use of biomechanical simulation models and multi-modal registration. Experiments performed at the CHU Sainte-Justine Hospital on adolescent patients demonstrate the potential clinical benefit of capturing statistical variations in the spine geometry to help diagnose and treat this disease.

Representation learning techniques have gained popularity over the years. Machine Learning community is well aware of several representation learning tools, viz. AutoEncoders, Deep belief networks, Convolutional Neural Networks and Dictionary Learning(similar to matrix factorization and latent factor model). While there has been extensive research on learning synthesis dictionaries and some recent work on learning analysis dictionaries, Transform Learning is a new form of representation learning. It is more generalized analysis equivalent of dictionary learning. Till now, Transform Learning has been restricted to the signal processing community. We start with a standard algebraic model and keep converting our intuitions and observations into mathematical model. We develop formulations aimed at learning representations from data. In this talk, the major part will cover the importance of transform learning and its advantages over other representation learning techniques. Then, Supervised Transform Learning and Deep Transform Learning will be discussed, followed by more robust transform formulations.

Currently there are three basic frameworks in deep learning – stacked autoencoders (SAE), deep belief network (DBN) and convolutional neural network (CNN); SAE and DBN can be applied to arbitrary inputs but CNN can only be applied to natural signals having local correlations (speech, image, ECG, EEG etc.). I am working on developing a new framework for deep learning – deep dictionary learning (DDL). Just as SAE uses autoencoders as basic units and DBN uses restricted Boltzmann machines, DDL uses dictionaries as the basic unit. DDL is formed by stacking one dictionary after another such that the output (features) from the shallower layer feeds into the next (deeper) layer as input. The initial work on DDL was a greedy sub-optimal solution, i.e. each of the layers were solved separately. My work has been on proposing an optimal solution to jointly learn all the layers. This is a solution for unsupervised feature extraction using DDL. Later, I worked on supervised (greedy) versions of deep dictionary learning with a plug-and-play approach. I have developed a framework for multi-label classification problems using the DDL framework. It has been used for solving a practical problem of Non-Intrusive Load Monitoring (NILM).

In the context of linear inverse problems, we propose and study a general iterative regularization method allowing to consider large classes of regularizers and data-fit terms. We were particularly motivated by dealing with non-smooth data-fit terms, such like a Kullback-Liebler divergence, or an L1 distance. We treat these problems by designing an algorithm, based on a primal-dual diagonal descent method, designed to solve hierarchical optimization problems. The key point of our approach is that, in presence of noise, the number of iterations of our algorithm acts as a regularization parameter. In practice this means that the algorithm must be stopped after a certain number of iterations. This is what is called regularization by early stopping, an approach which gained in popularity in statistical learning. Our main results establishes convergence and stability of our algorithm, and are illustrated by experiments on image denoising, comparing our approach with a more classical Tikhonov regularization method.