Selected Topics in Constrained Optimization

Abstract

We study the problem of minimization of loss function $f(X)$ over the set of all $n$ x $m$ matrices $X$ of rank $r$. We propose a bi-level extention to Factored Gradient descent (FGD) algorithm and show its theoretical convergence and simulation results. On a side note, we also suggest methods of reducing computation complexity of projection based algorithms by cutting down on the number of projection operation in a standard constrained optimization approach. We also analyse the benefits and limitations of this approach.