Skip to content

Latest commit

 

History

History
33 lines (23 loc) · 3.47 KB

README.md

File metadata and controls

33 lines (23 loc) · 3.47 KB

Stochastic Gradient Descent on Riemannian Manifolds.

Jonas_Grabbe_STICKER_with_WHITE_BACKGROUND_The_illustration_rep_3f5fa099-551d-4373-bd21-e28204997242

This project focuses on the exploration and implementation of the algorithmic approach of stochastic gradient descent on Riemannian manifolds. We aim to extend the classical gradient descent algorithm, traditionally applied in Euclidean space, to the context of Riemannian manifolds, with the ultimate objective of optimizing cost functions defined on such manifolds.

Background

A Riemannian manifold is a mathematical structure that generalizes the notion of a curved space to higher dimensions. It is a crucial concept in differential geometry and has found applications in many areas of mathematics and physics. In machine learning, the usage of Riemannian manifolds allows us to define cost functions on a curved space rather than on a flat, Euclidean one, which can offer a more nuanced and accurate representation of the problem at hand.

Stochastic gradient descent is a widely used optimization method in machine learning due to its computational efficiency and scalability to large datasets. Its primary aim is to find the local minima of a cost function, even when evaluations of the function are corrupted by noise. In this project, we aim to extend this optimization method to the case where the function is defined on a Riemannian manifold.

Key Inspirations

The work is primarily inspired by the following papers:

  • "Stochastic gradient descent on Riemannian manifolds" by Silvere Bonnabel. This paper extends the stochastic gradient descent algorithm to the case where the cost function is defined on a Riemannian manifold, proving its convergence to a critical point of the cost function.
  • "Min-Max Solutions for Super Sinh-Gordon Equations on Compact Surfaces" by Aleks Jevnikar, Andrea Malchiodi, and Ruijun Wu. This paper presents the first non-trivial solution of a super sinh-Gordon system on compact surfaces using variational analysis and a linking argument with a suitably defined Nehari manifold.
  • "Decentralized Riemannian Gradient Descent on the Stiefel Manifold" by Shixiang Chen, Alfredo Garcia, Mingyi Hong, Shahin Shahrampour. This paper introduces a decentralized Riemannian stochastic gradient method (DRSGD) and decentralized Riemannian gradient tracking algorithm (DRGTA) for distributed optimization on the Stiefel manifold.

Project Goals

The main objectives of this project are as follows:

  • Implement the stochastic gradient descent algorithm on Riemannian manifolds as described in Bonnabel's paper.
  • Analyze the performance and convergence properties of the implemented algorithm.
  • Apply the algorithm to real-world datasets and compare its performance with traditional gradient descent methods.
  • Investigate the potential for integrating the min-max solutions approach presented by Jevnikar, Malchiodi, and Wu.

Project Structure

The project is organized into the following directories: TBA

References

[1] Bonnabel, S. (2013). Stochastic gradient descent on Riemannian manifolds. arXiv preprint arXiv:1111.5280. Link [2] Jevnikar, A., et al. (2021). Min-max solutions for super sinh-Gordon equations on compact surfaces. arXiv preprint 2102.00709. Link [3] Chan, S., et al. (2021). Decentralized Riemannian Gradient Descent on the Stiefel Manifold. arXiv preprint arXiv:2102.07091. Link