Learning Globally Smooth Functions on Manifolds

Smoothness and low dimensional structures play central roles in improving generalization and stability in learning and statistics. This work combines techniques from <PRE_TAG>semi-infinite constrained learning</POST_TAG> and <PRE_TAG>manifold regularization</POST_TAG> to learn representations that are globally smooth on a manifold. To do so, it shows that under typical conditions the problem of learning a Lipschitz continuous function on a manifold is equivalent to a dynamically weighted <PRE_TAG>manifold regularization</POST_TAG> problem. This observation leads to a practical algorithm based on a <PRE_TAG>weighted Laplacian penalty</POST_TAG> whose weights are adapted using <PRE_TAG>stochastic gradient techniques</POST_TAG>. It is shown that under mild conditions, this method estimates the Lipschitz constant of the solution, learning a globally smooth solution as a byproduct. Experiments on real world data illustrate the advantages of the proposed method relative to existing alternatives.