Tehran Institute for Advanced Studies (TEIAS)

/ Convergence and Equilibrium in Generative Adversarial Networks


Convergence and Equilibrium in
Generative Adversarial Networks

September 13, 2020
(23 Shahrivar, 1399)


This Talk is online


Dr. Farzan Farnia

Postdoctoral Research Associate at the Massachusetts Institute of Technology (MIT)


Generative adversarial networks (GANs) represent a zero-sum game between two machine players, a generator and a discriminator, designed to learn the distribution of data. While GANs have achieved state -of-the-art performance in several benchmark learning tasks, GAN minimax optimization still poses great theoretical and empirical challenges. GANs trained using first-order optimization methods commonly fail to converge to a stable solution where the players cannot improve their objective, i.e., the Nash equilibrium of the underlying game. Such issues raise the question of the existence of Nash equilibria in GAN zero-sum games. In this presentation, we discuss theoretical and numerical results indicating that GAN games may have no Nash equilibria. To characterize an equilibrium notion applicable to GANs, we consider the equilibrium of a new zero-sum game with an objective function given by a proximal operator applied to the original objective, a solution we call the proximal equilibrium. Unlike the Nash equilibrium, the proximal equilibrium captures the sequential nature of GANs, in which the generator moves first followed by the discriminator. We show that the optimal generative model in Wasserstein GANs provides a proximal equilibrium. Inspired by these results, we propose a new approach, which we call proximal training, for solving GAN problems. We discuss several numerical experiments indicating the existence of proximal equilibria in GANs.



Farzan Farnia is a postdoctoral research associate at the Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, where he is co-supervised by Asuman Ozdaglar and Ali Jadbabaie. Prior to joining MIT, Farzan received his master’s and PhD degrees in electrical engineering from Stanford University and his bachelor’s degrees in electrical engineering and mathematics from Sharif University of Technology. At Stanford, he was a graduate research assistant at the Information Systems Laboratory advised by David Tse. Farzan’s research interests include information theory, statistical learning theory, and convex optimization. He has been the recipient of the Stanford graduate fellowship (Sequoia Capital fellowship) from 2013-2016 and the Numerical Technology Founders Prize as the second top performer of Stanford electrical engineering PhD qualifying exams in 2014.