Divya Saxena
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Generative Adversarial Networks (GANs) is a novel class of deep generative models that has recently gained significant attention. GANs learn complex and high-dimensional distributions implicitly over images, audio, and data. However, there exist major challenges in training of GANs, i.e., mode collapse, non-convergence, and instability, due to inappropriate design of network architectre, use of objective function, and selection of optimization algorithm. Recently, to address these challenges, several solutions for better design and optimization of GANs have been investigated based on techniques of re-engineered network architectures, new objective functions, and alternative optimization algorithms. To the best of our knowledge, there is no existing survey that has particularly focused on the broad and systematic developments of these solutions. In this study, we perform a comprehensive survey of the advancements in GANs design and optimization solutions proposed to handle GANs challenges. We first identify key research issues within each design and optimization technique and then propose a new taxonomy to structure solutions by key research issues. In accordance with the taxonomy, we provide a detailed discussion on different GANs variants proposed within each solution and their relationships. Finally, based on the insights gained, we present promising research directions in this rapidly growing field.
Supplemental Material
Available for Download
zip
saxena.zip (451 KB)
Supplemental movie, appendix, image and software files for, Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
- Y.-W. Hinton, Geoffrey E. Osindero, and Simon Teh. 2006. A fast learning algorithm for deep belief nets. Neural Comput. 18, 7 (2006), 1527--1554.Google Scholar
Digital Library
- R. Salakhutdinov and G. Hinton. 2009. Deep boltzmann machines. J. Mach. Learn. Res. 5, (2009) 448--455.Google Scholar
- D. P. Kingma and M. Welling. 2014. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations (ICLR’14).Google Scholar
- I. J. Goodfellow et al. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680.Google Scholar
- E. Denton, S. Chintala, A. Szlam, and R. Fergus. 2015. Deep generative image models using a Laplacian pyramid of adversarial networks. In Advances in Neural Information Processing Systems. 1486--1494.Google Scholar
- A. Radford, L. Metz, and S. Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In 4th International Conference on Learning Representations (ICLR’16).Google Scholar
- M. Y. Liu and O. Tuzel. 2016. Coupled generative adversarial networks. In Advances in Neural Information Processing Systems. 469--477.Google Scholar
- T. Karras, T. Aila, S. Laine, and J. Lehtinen. 2018. Progressive growing of GANs for improved quality, stability, and variation. In 6th International Conference on Learning Representations (ICLR’18).Google Scholar
- T. Karras, S. Laine, and T. Aila. 2019. A style-based generator architecture for generative adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition. 4401--4410.Google Scholar
- C. Ledig et al. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 105--114.Google ScholarCross Ref
- C. Spampinato, S. Palazzo, P. D'Oro, D. Giordano, and M. Shah. 2019. Adversarial framework for unsupervised learning of motion dynamics in videos. Int. J. Comput. Vis. (Mar. 2019).Google Scholar
- T. Kim, M. Cha, H. Kim, J. K. Lee, and J. Kim. 2017. Learning to discover cross-domain relations with generative adversarial networks. In 34th International Conference on Machine Learning (ICML’17). 2941--2949.Google Scholar
- Y. Hong, U. Hwang, J. Yoo, and S. Yoon. 2019. How generative adversarial networks and their variants work: An overview. ACM Comput. Surv. 52, 1 (2019).Google Scholar
- Z. Wang, Q. She, and T. E. Ward. 2019. Generative adversarial networks: A survey and taxonomy. arXiv Preprint arXiv1906.01529, 2019.Google Scholar
- Z. Pan, W. Yu, X. Yi, A. Khan, F. Yuan, and Y. Zheng. 2019. Recent progress on generative adversarial networks (GANs): A survey. IEEE Access 7 (2019), 36322--36333.Google ScholarCross Ref
- S. Hitawala. 2018. Comparative study on generative adversarial networks. arXiv Preprint arXiv1801.04271, 2018.Google Scholar
- K. Wang, C. Gou, Y. Duan, Y. Lin, X. Zheng, and F. Y. Wang. 2017. Generative adversarial networks: Introduction and outlook. IEEE/CAA J. Automat. Sin. 4, 4 (2017), 588--598.Google ScholarCross Ref
- J. Gui, Z. Sun, Y. Wen, D. Tao, and J. Ye. 2020. A review on generative adversarial networks: Algorithms, theory, and applications. arXiv preprint arXiv:2001.06937.Google Scholar
- A. Bissoto, E. Valle, and S. Avila. 2019. The six fronts of the generative adversarial networks. arXiv Preprint arXiv1910.13076, 2019.Google Scholar
- T. Motwani and M. Parmar. 2020. A novel framework for selection of GANs for an application. arXiv Preprint arXiv2002.08641, 2020.Google Scholar
- Y.-J. CAO. 2019. Recent advances of generative adversarial networks in computer vision. IEEE Access 7 (2019).Google Scholar
- A. Jabbar, X. Li, and B. Omar. 2020. A survey on generative adversarial networks: Variants, applications, and training. arXiv preprint arXiv:2006.05132.Google Scholar
- L. Jin, F. Tan, and S. Jiang. 2020. Generative adversarial network technologies and applications in computer vision. Computational Intelligence and Neuroscience.Google Scholar
- M. Wiatrak, S. V. Albrecht, and A. Nystrom. 2019. Stabilizing generative adversarial networks: A survey. arXiv preprint arXiv:1910.00927.Google Scholar
- M. Lee and J. Seok. 2020. Regularization methods for generative adversarial networks: An overview of recent studies. arXiv preprint arXiv:2005.09165.Google Scholar
- S. N. Esfahani and S. Latifi. 2019. A survey of state-of-the-Art GAN-based approaches to image synthesis. In Computer Science & Information Technology (CS & IT) Computer Science Conference Proceedings (CSCP). 63--76.Google Scholar
- N. Akhtar and A. Mian. 2018. Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access 6, (2018), 14410--14430.Google Scholar
- A. Chakraborty, A. Manaar, V. Dey, A. Chattopadhyay, and D. Mukhopadhyay. 2018. Adversarial attacks and defences: A survey. arXiv Preprint arXiv1810.00069, 2018.Google Scholar
- F. Huszár. 2015. How (not) to train your generative model: Scheduled sampling, likelihood, adversary? arXiv Preprint arXiv1511.05101, 2015.Google Scholar
- L. Theis, A. Van Den Oord, and M. Bethge. 2016. A note on the evaluation of generative models. In 4th International Conference on Learning Representations (ICLR’16).Google Scholar
- M. Rosca, B. Lakshminarayanan, D. Warde-Farley, and S. Mohamed. 2017. Variational approaches for auto-encoding generative adversarial networks. arXiv Preprint arXiv1706.04987, 2017.Google Scholar
- A. Statnikov, C. F. Aliferis, D. P. Hardin, and I. Guyon. 2011. Support vector clustering. In A Gentle Introduction to Support Vector Machines in Biomedicine, Vol. 1 (Feb. 2011), 136--153.Google Scholar
- T. Le, H. Vu, T. D. Nguyen, and D. Phung. 2018. Geometric enclosing networks. In International Joint Conference on Artificial Intelligence (IJCAI’18). 2355--2361.Google Scholar
- M. Mirza and S. Osindero. 2014. Conditional generative adversarial nets. arXiv Preprint arXiv1411.1784, 2014.Google Scholar
- M. Arjovsky, S. Chintala, and L. Bottou. 2017. Wasserstein generative adversarial networks. In International Conference on Machine Learning. 214--223.Google Scholar
- J. Adler and S. Lunz. 2018. Banach Wasserstein GAN. In Advances in Neural Information Processing Systems. 6754--6763.Google Scholar
- X. Guo, J. Hong, T. Lin, and N. Yang. 2017. Relaxed Wasserstein with applications to GANs. arXiv Preprint arXiv1705.07164, 2017.Google Scholar
- L. Mescheder, S. Nowozin, and A. Geiger. 2017. The numerics of GANs. In Advances in Neural Information Processing Systems. 1826--1836.Google Scholar
- M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems. 6627--6638.Google Scholar
- X. Mao, Q. Li, H. Xie, R. Y. K. Lau, Z. Wang, and S. P. Smolley. 2017. Least squares generative adversarial networks. In IEEE International Conference on Computer Vision. 2813--2821.Google Scholar
- A. Yadav, S. Shah, Z. Xu, D. Jacobs, and T. Goldstein. 2018. Stabilizing adversarial nets with prediction methods. arXiv Preprint arXiv1705.07364, 2018.Google Scholar
- T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. 2016. Improved techniques for training GANs. In Advances in Neural Information Processing Systems. 2234--2242.Google Scholar
- Z. Lin, G. Fanti, A. Khetan, and S. Oh. 2018. PacGan: The power of two samples in generative adversarial networks. In Advances in Neural Information Processing Systems. 1498--1507.Google Scholar
- I. Goodfellow. 2016. Tutorial: Generative adversarial networks. arXiv Preprint arXiv1701.00160, 2016.Google Scholar
- L. Metz, B. Poole, D. Pfau, and J. Sohl-Dickstein. 2016. Unrolled generative adversarial networks. arXiv Preprint arXiv:1611.02163, 2016.Google Scholar
- T. Che, Y. Li, A. P. Jacob, Y. Bengio, and W. Li. 2019. Mode regularized generative adversarial networks. In 5th International Conference on Learning Representations (ICLR’19).Google Scholar
- D. Warde-Farley and Y. Bengio. 2019. Improving generative adversarial networks with denoising feature matching. In 5th International Conference on Learning Representations (ICLR’19).Google Scholar
- M. Arjovsky and L. Bottou. 2019. Towards principled methods for training generative adversarial networks. In 5th International Conference on Learning Representations (ICLR’19).Google Scholar
- S. Huang, Xun Li, Yixuan Poursaeed, Omid Hopcroft, and John Belongie. 2017. Stacked generative adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition. 5077--5086.Google ScholarCross Ref
- Y. Wang, L. Zhang, and J. van de Weijer. 2016. Ensembles of generative adversarial networks. arXiv Preprint arXiv1612.00991, 2016.Google Scholar
- I. Tolstikhin, S. Gelly, O. Bousquet, C. J. Simon-Gabriel, and B. Schölkopf. 2017. AdaGAN: Boosting generative models. In Advances in Neural Information Processing Systems. 5425--5434.Google Scholar
- C. L. Li, W. C. Chang, Y. Cheng, Y. Yang, and B. Póczos. 2017. MMD GAN: Towards deeper understanding of moment matching network. In Advances in Neural Information Processing Systems. 2204--2214.Google Scholar
- H. Kwak and B.-T. Zhang. 2016. Ways of conditioning generative adversarial networks. arXiv Preprint arXiv1611.01455, 2016.Google Scholar
- D. J. Im, C. D. Kim, H. Jiang, and R. Memisevic. 2016. Generating images with recurrent adversarial networks. arXiv Preprint arXiv1602.05110, 2016.Google Scholar
- G. Perarnau, J. van de Weijer, B. Raducanu, and J. M. Álvarez. 2016. Invertible conditional GANs for image editing. arXiv Preprint arXiv1611.06355, 2016.Google Scholar
- A. Jaiswal, W. AbdAlmageed, Y. Wu, and P. Natarajan. 2019. Bidirectional conditional generative adversarial networks. Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 11363. 216--232.Google Scholar
- G. Máttyus and R. Urtasun. 2018. Matching adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition. 8024--8032.Google Scholar
- S. Liu, T. Wang, D. Bau, J.-Y. Zhu, and A. Torralba. 2020. Diverse image generation via self-conditioned GANs. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14274--14283.Google Scholar
- A. Odena, C. Olah, and J. Shlens. 2017. Conditional image synthesis with auxiliary classifier GANs. In 34th International Conference on Machine Learning (ICML’17). 4043--4055.Google Scholar
- L. Chongxuan, T. Xu, J. Zhu, and B. Zhang. 2017. Triple generative adversarial nets. In Advances in Neural Information Processing Systems. 4088--4098.Google Scholar
- X. Wang, Y. Sun, R. Zhang, and J. Qi. 2018. KDGAN: Knowledge distillation with generative adversarial networks. In Advances in Neural Information Processing Systems. 775--786.Google Scholar
- M. Lee and J. Seok. 2019. Controllable generative adversarial network. IEEE Access 7 (2019), 28158--28169.Google ScholarCross Ref
- Y. Saatchi and A. G. Wilson. 2017. Bayesian GAN. In Advances in Neural Information Processing Systems. 3623--3632.Google Scholar
- A. Jaiswal, W. AbdAlmageed, Y. Wu, and P. Natarajan. 2019. CapsuleGAN: Generative adversarial capsule network. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 11131. 526--535.Google Scholar
- S. Lloyd and C. Weedbrook. 2018. Quantum generative adversarial learning. Phys. Rev. Lett. 121, 4 (2018).Google ScholarCross Ref
- A. Zhang, Han Goodfellow, Ian Metaxas, and Dimitris Odena. 2018. Self-attention generative adversarial networks. arXiv Preprint arXiv1805.08318, 2018.Google Scholar
- A. Ghosh, V. Kulharia, V. Namboodiri, P. H. S. Torr, and P. K. Dokania. 2018. Multi-agent diverse generative adversarial networks. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 8513--8521.Google Scholar
- Q. Hoang, T. D. Nguyen, T. Le, and D. Phung. 2017. Multi-generator generative adversarial nets. arXiv Preprint arXiv1708.02556, 2017.Google Scholar
- A. Ghosh, V. Kulharia, and V. Namboodiri. 2016. Message passing multi-agent GANs. arXiv Preprint arXiv1612.01294, 2016.Google Scholar
- H. Ge, Y. Xia, X. Chen, R. Berry, and Y. Wu. 2017. Fictitious GAN: Training GANs with historical models. In Proceedings of the European Conference on Computer Vision (ECCV'18). 119--134.Google Scholar
- S. Arora, R. Ge, Y. Liang, T. Ma, and Y. Zhang. 2017. Generalization and equilibrium in generative adversarial nets (GANs). In 34th International Conference on Machine Learning (ICML’17). 322--349.Google Scholar
- T. D. Nguyen, T. Le, H. Vu, and D. Phung. 2017. Dual discriminator generative adversarial nets. In Advances in Neural Information Processing Systems. 2671--2681.Google Scholar
- I. Durugkar, I. Gemp, and S. Mahadevan. 2019. Generative multi-adversarial networks. In 5th International Conference on Learning Representations (ICLR’17).Google Scholar
- B. Neyshabur, S. Bhojanapalli, and A. Chakrabarti. 2017. Stabilizing GAN training with multiple random projections. arXiv Preprint arXiv1705.07831, 2017.Google Scholar
- G. Mordido, H. Yang, and C. Meinel. 2018. Dropout-GAN: Learning from a dynamic ensemble of discriminator. arXiv preprint arXiv:1807.11346, 2018.Google Scholar
- G. Mordido, H. Yang, and C. Meinel. 2020. MicrobatchGAN: Stimulating diversity with multi-adversarial discrimination. In IEEE Winter Conference on Applications of Computer Vision. 3050--3059.Google Scholar
- T. Chavdarova and F. Fleuret. 2018. SGAN: An alternative training of generative adversarial networks. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 9407--9415.Google Scholar
- A. B. L. Larsen, S. K. Sønderby, H. Larochelle, and O. Winther. 2016. Autoencoding beyond pixels using a learned similarity metric. In 33rd International Conference on Machine Learning (ICML’16). 2341--2349.Google Scholar
- A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey. 2015. Adversarial autoencoders. arXiv Preprint arXiv1511.05644, 2015.Google Scholar
- L. Mescheder, S. Nowozin, and A. Geiger. 2017. Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks. In 34th International Conference on Machine Learning (ICML’17). 3694--3707.Google Scholar
- Y. Pu et al. 2017. Adversarial symmetric variational autoencoder. In Advances in Neural Information Processing Systems. 4331--4340.Google Scholar
- N. T. Tran, T. A. Bui, and N. M. Cheung. 2018. Dist-GAN: An improved GAN using distance constraints. Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 11218. 387--401.Google Scholar
- V. Dumoulin et al. 2019. Adversarially learned inference. In 5th International Conference on Learning Representations (ICLR’19).Google Scholar
- J. Donahue, T. Darrell, and P. Krähenbühl. 2016. Adversarial feature learning. arXiv preprint arXiv:1605.09782, 2016.Google Scholar
- A. H. Li, Y. Wang, C. Chen, and J. Gao. 2020. Decomposed adversarial learned inference. arXiv Preprint arXiv2004.10267, 2020.Google Scholar
- M. Chen and L. Denoyer. 2017. Multi-view generative adversarial networks. In Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 10535. 175--188.Google Scholar
- M. I. Belghazi, S. Rajeswar, O. Mastropietro, N. Rostamzadeh, J. Mitrovic, and A. Courville. 2018. Hierarchical adversarially learned Inference. arXiv Preprint arXiv1802.01071, 2018.Google Scholar
- D. Ulyanov, A. Vedaldi, and V. Lempitsky. 2018. It takes (only) two: Adversarial generator-encoder networks. In 32nd AAAI Conference on Artificial Intelligence. 1250--1257.Google Scholar
- A. Srivastava, L. Valkov, C. Russell, M. U. Gutmann, and C. Sutton. 2017. VEEGAN: Reducing mode collapse in GANs using implicit variational learning. In Advances in Neural Information Processing Systems. 3309--3319.Google Scholar
- D. Bang and H. Shim. 2018. MGGAN: Solving mode collapse using manifold guided training. arXiv Preprint arXiv1804.04391, 2018.Google Scholar
- J. Zhao, M. Mathieu, and Y. LeCun. 2016. Energy-based generative adversarial networks. arXiv preprint arXiv:1609.03126, 2016.Google Scholar
- D. Berthelot, T. Schumm, and L. Metz. 2017. BEGAN: Boundary equilibrium generative adversarial networks. arXiv Preprint arXiv1703.10717, 2017.Google Scholar
- R. Wang, A. Cully, H. J. Chang, and Y. Demiris. 2017. MAGAN: Margin adaptation for generative adversarial networks. arXiv Preprint arXiv1704.03817, 2017.Google Scholar
- G. Zheng, J. Sang, and C. Xu. 2020. MMCGAN: Generative adversarial network with explicit manifold prior. arXiv Preprint arXiv2006.10331, 2020.Google Scholar
- X. Di and P. Yu. 2017. Max-Boost-GAN: Max operation to boost generative ability of generative adversarial networks. In IEEE International Conference on Computer Vision Workshops. 1156--1164.Google Scholar
- Y. Kim, M. Kim, and G. Kim. 2018. Memorization precedes generation: Learning unsupervised GANs with memory networks. arXiv preprint arXiv:1803.01500.Google Scholar
- S. Gurumurthy, R. K. Sarvadevabhatla, and R. V. Babu. 2017. DeLiGAN : Generative adversarial networks for diverse and limited data. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 4941--4949.Google Scholar
- D. Mishra, Prathosh A. P., A. Jayendran, V. Srivastava, and S. Chaudhury. 2018. NEMGAN: Noise engineered mode-matching GAN. arXiv Preprint arXiv1811.03692, 2018.Google Scholar
- X. Di and P. Yu. 2017. Multiplicative noise channel in generative adversarial networks. In IEEE International Conference on Computer Vision Workshops. 1165--1172.Google Scholar
- G. Zhong, W. Gao, Y. Liu, and Y. Yang. 2018. Generative adversarial networks with decoder-encoder output noise. arXiv Preprint arXiv1807.03923, 2018.Google Scholar
- X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel. 2016. InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems. 2172--2180.Google Scholar
- G.-J. Qi. 2017. Loss-sensitive generative adversarial networks on Lipschitz densities. Int. J. Comput. Vis 128, 5 (2017), 1118--1140.Google ScholarDigital Library
- S. Nowozin, B. Cseke, and R. Tomioka. 2016. f-GAN: Training generative neural samplers using variational divergence minimization. In Advances in Neural Information Processing Systems. 271--279.Google Scholar
- M. Uehara, I. Sato, M. Suzuki, K. Nakayama, and Y. Matsuo. 2016. Generative adversarial nets from a density ratio estimation perspective. arXiv Preprint arXiv1610.02920, 2016.Google Scholar
- C. Tao, L. Chen, R. Henao, J. Feng, and L. Carin. 2018. X2 generative adversarial network. In 35th International Conference on Machine Learning (ICML’18). 7787--7796.Google Scholar
- T. Salimans, D. Metaxas, H. Zhang, and A. Radford. 2018. Improving GANs using optimal transport. In 6th International Conference on Learning Representations (ICLR’18).Google Scholar
- M. Lin. 2017. Softmax GAN. arXiv Preprint arXiv1704.06191, 2017.Google Scholar
- Y. Li, N. Xiao, and W. Ouyang. 2019. Improved generative adversarial networks with reconstruction loss. Neurocomputing 323 (2019), 363--372.Google ScholarCross Ref
- F. Juefei-Xu, V. N. Boddeti, and M. Savvides. 2017. Gang of GANs: Generative adversarial networks with maximum margin ranking. arXiv Preprint arXiv1704.04865, 2017.Google Scholar
- Y. Mroueh, T. Sercu, and V. Goel. 2017. McGAN: Mean and covariance feature matching GAN. In 34th International Conference on Machine Learning (ICML’17). 3885--3899.Google Scholar
- N. Park et al. 2018. MMGAN: Manifold-matching generative adversarial networks. In International Conference on Pattern Recognition. 1343--1348.Google ScholarCross Ref
- M. G. Bellemare et al. 2017. The cramer distance as a solution to biased Wasserstein gradients. arXiv Preprint arXiv1705.10743, 2017.Google Scholar
- I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville. 2017. Improved training of Wasserstein GANs. In Advances in Neural Information Processing Systems. 5768--5778.Google Scholar
- H. Petzka, A. Fischer, and D. Lukovnikov. 2018. On the regularization of Wasserstein GANs. In 6th International Conference on Learning Representations (ICLR’18).Google Scholar
- X. Wei, B. Gong, Z. Liu, W. Lu, and L. Wang. 2018. Improving the improved training of Wasserstein GANs: A consistency term and its dual effect. arXiv Preprint arXiv1803.01541, 2018.Google Scholar
- T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. 2018. Spectral normalization for generative adversarial networks. In 6th International Conference on Learning Representations (ICLR’18).Google Scholar
- K. Roth, A. Lucchi, S. Nowozin, and T. Hofmann. 2017. Stabilizing training of generative adversarial networks through regularization. In Advances in Neural Information Processing Systems. 2019--2029.Google Scholar
- Y. Mroueh and T. Sercu. 2017. Fisher GAN. In Advances in Neural Information Processing Systems. 2514--2524.Google Scholar
- V. Nagarajan and J. Z. Kolter. 2017. Gradient descent GAN optimization is locally stable. In Advances in Neural Information Processing Systems. 5586--5596.Google Scholar
- L. Mescheder, A. Geiger, and S. Nowozin. 2018. Which training methods for GANs do actually converge? In 35th International Conference on Machine Learning (ICML’18). 5589--5626.Google Scholar
- N. Kodali, J. Abernethy, J. Hays, and Z. Kira. 2017. On convergence and stability of GANs. arXiv Preprint arXiv1705.07215, 2017.Google Scholar
- C. Daskalakis, A. Ilyas, V. Syrgkanis, and H. Zeng. 2018. Training GaNs with optimism. 6th International Conference on Learning Representations (ICLR’18).Google Scholar
- H. Prasad, Prashanth L. A., and S. Bhatnagar. 2015. Two-timescale algorithms for learning Nash equilibria in general-sum stochastic games. In International Conference on Autonomous Agents and Multiagent Systems.Google Scholar
- A. Fukui, D. H. Park, D. Yang, A. Rohrbach, T. Darrell, and M. Rohrbach. 2016. Multimodal compact bilinear pooling for visual question answering and visual grounding. In Conference on Empirical Methods in Natural Language Processing. 457--468.Google Scholar
- M. Mathieu, J. Zhao, P. Sprechmann, A. Ramesh, and Y. Le Cun. 2016. Disentangling factors of variation in deep representations using adversarial training. In Advances in Neural Information Processing Systems. 5047--5055.Google Scholar
- Z. Xu, Y. C. Hsu, and J. Huang. 2018. Training shallow and thin networks for acceleration via knowledge distillation with conditional adversarial networks. In 6th International Conference on Learning Representations Workshop (ICLRW’18).Google Scholar
- M. Mirza and S. Osindero. 2014. Conditional generative adversarial nets. arXiv Preprint arXiv1411.1784, 2014.Google Scholar
- T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. 1997. Improved techniques for training GANs. arXiv preprint arXiv:1606.03498.Google Scholar
- Y. Freund and R. E. Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci 55, 1 (1997), 119--139.Google ScholarDigital Library
- I. Albuquerque, J. Monteiro, T. Doan, B. Considine, T. Falk, and I. Mitliagkas. 2019. Multi-objective training of generative adversarial networks with multiple discriminators. In 36th International Conference on Machine Learning (ICML’19). 292--301.Google Scholar
- A. Nguyen, J. Clune, Y. Bengio, A. Dosovitskiy, and J. Yosinski. 2017. Plug and play generative networks: Conditional iterative generation of images in latent space. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 3510--3520.Google Scholar
- R. A. Yeh, C. Chen, T. Yian Lim, A. G. Schwing, M. Hasegawa-Johnson, and M. N. Do. 2017. Semantic image inpainting with deep generative models. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 6882--6890.Google Scholar
- I. J. Goodfellow. 2015. On distinguishability criteria for estimating generative models. In 3rd International Conference on Learning Representations Workshop (ICLRW’15).Google Scholar
- S. Mohamed and B. Lakshminarayanan. 2016. Learning in implicit generative models. arXiv Preprint arXiv1610.03483, 2016.Google Scholar
- Y. Bengio, N. Léonard, and A. Courville. 2013. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv Preprint arXiv1308.3432, 2013.Google Scholar
- Y. Bengio, É. Thibodeau-Laufer, G. Alain, and J. Yosinski. 2014. Deep generative stochastic networks trainable by backprop. In 31st International Conference on Machine Learning (ICML’14). 1470--1485.Google Scholar
- H. De Meulemeester, J. Schreurs, M. Fanuel, B. De Moor, and J. A. K. Suykens. 2020. The Bures metric for taming mode collapse in generative adversarial networks. arXiv Preprint arXiv2006.09096, 2020.Google Scholar
- Y. LeCun, S. Chopra, R. Hadsell, F. J. Huang, and E. Al. 2006. A tutorial on energy-based learning. Predict. Struct. Data 1, 0 (2006).Google Scholar
- Z. Dai, A. Almahairi, P. Bachman, E. Hovy, and A. Courville. 2017. Calibrating energy-based generative adversarial networks. arXiv Preprint arXiv1702.01691, 2017.Google Scholar
- S. Mukherjee, H. Asnani, E. Lin, and S. Kannan. 2019. ClusterGAN: Latent space clustering in generative adversarial networks. In AAAI Conference on Artificial Intelligence. 4610--4617.Google Scholar
- D. P. Kingma and M. Welling. 2014. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations (ICLR’14).Google Scholar
- D. P. Kingma, D. J. Rezende, S. Mohamed, and M. Welling. 2014. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems. 3581--3589.Google Scholar
- B. Cheung, J. A. Livezey, A. K. Bansal, and B. A. Olshausen. 2015. Discovering hidden factors of variation in deep networks. In 3rd International Conference on Learning Representations Workshop (ICLRW’15).Google Scholar
- T. D. Kulkarni, W. F. Whitney, P. Kohli, and J. B. Tenenbaum. 2015. Deep convolutional inverse graphics network. In Advances in Neural Information Processing Systems. 2539--2547.Google Scholar
- W. F. Whitney, M. Chang, T. Kulkarni, and J. B. Tenenbaum. 2016. Understanding visual concepts with continuation learning. arXiv Preprint arXiv1602.06822, 2016.Google Scholar
- F. Huszár. 2017. Variational inference using implicit distributions. arXiv Preprint arXiv1702.08235, 2017.Google Scholar
- T. Karaletsos. 2016. Adversarial message passing for graphical models. arXiv Preprint arXiv1612.05048, 2016.Google Scholar
- D. Tran, R. Ranganath, and D. M. Blei. 2017. Hierarchical implicit models and likelihood-free variational inference. In Advances in Neural Information Processing Systems. 5524--5534.Google Scholar
- Y. Rubner, C. Tomasi, and L. J. Guibas. 2000. Earth mover's distance as a metric for image retrieval. Int. J. Comput. Vis 40, 2, (2000), 99--121.Google ScholarDigital Library
- I. Goodfellow, Y. Bengio, and A. Courville. 2017. Deep Learning. The MIT Press.Google Scholar
- A. Gretton, O. Bousquet, A. Smola, and B. Scḧlkopf. 2005. Measuring statistical dependence with Hilbert-Schmidt norms. In Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 3734. 63--77.Google Scholar
- K. Fukumizu, A. Gretton, X. Sun, and B. Schölkopf. 2009. Kernel measures of conditional dependence. In Advances in Neural Information Processing Systems.Google Scholar
- A. Smola, A. Gretton, L. Song, and B. Schölkopf. 2007. A Hilbert space embedding for distributions. In Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 4754. 13--31.Google Scholar
- Y. Li, K. Swersky, and R. Zemel. 2015. Generative moment matching networks. In 32nd International Conference on Machine Learning (ICML’15). 1718--1727.Google Scholar
- G. K. Dziugaite, D. M. Roy, and Z. Ghahramani. 2015. Training generative neural networks via maximum mean discrepancy optimization. In 31st Conference on Uncertainty in Artificial Intelligence. 258--267.Google Scholar
- S. Ioffe and C. Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In 32nd International Conference on Machine Learning (ICML’15). 448--456.Google Scholar
- W. Zellinger, E. Lughofer, S. Saminger-Platz, T. Grubinger, and T. Natschläger. 2019. Central moment discrepancy (CMD) for domain-invariant representation learning. In 5th International Conference on Learning Representations (ICLR’19).Google Scholar
- T. Salimans and D. P. Kingma. 2016. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Advances in Neural Information Processing Systems. 901--909.Google Scholar
- A. Müller. 1997. Integral probability metrics and their generating classes of functions. Adv. Appl. Probab. 29, 2 (1997), 429--443.Google ScholarCross Ref
- B. K. Sriperumbudur, K. Fukumizu, A. Gretton, B. Schölkopf, and G. R. G. Lanckriet. 2009. On integral probability metrics, phi-divergences and binary classification. arXiv preprint arXiv:0901.2698.Google Scholar
- J. C. Butcher. 2016. Numerical Methods for Ordinary Differential Equations. John Wiley & Sons, Ltd, Chichester, UK.Google Scholar
- S. I. Amari. 1998. Natural gradient works efficiently in learning. Neural Comput. 10, 2 (1998), 251--276.Google ScholarDigital Library
- P. Grnarova, K. Y. Levy, A. Lucchi, T. Hofmann, and A. Krause. 2018. An online learning approach to generative adversarial networks. In 6th International Conference on Learning Representations (ICLR’18).Google Scholar
- D. Volkhonskiy, I. Nazarov, and E. Burnaev. 2017. Steganographic generative adversarial networks. arXiv Preprint arXiv1703.05502, 2017.Google Scholar
- W. Zhang. 2018. Generative adversarial nets for information retrieval: Fundamentals and advances. In 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 1375--1378.Google ScholarDigital Library
- D. Saxena and J. Cao. 2019. D-GAN: Deep generative adversarial nets for spatio-temporal prediction. arXiv Preprint arXiv1907.08556, 2019.Google Scholar
- M. Uricar, P. Krizek, D. Hurych, I. Sobh, S. Yogamani, and P. Denny. 2019. Yes, we GAN: Applying adversarial techniques for autonomous driving. arXiv Preprint arXiv1902.03442, 2019.Google Scholar
- M. Zhang, Y. Zhang, L. Zhang, C. Liu, and S. Khurshid. 2018. DeepRoad: GAN-based metamorphic autonomous driving system testing. In 33rd ACM/IEEE International Conference on Automated Software Engineering. 132--142.Google Scholar
- M. Teichmann, M. Weber, M. Zöllner, R. Cipolla, and R. Urtasun. 2018. MultiNet: Real-time joint semantic reasoning for autonomous driving. In IEEE Intelligent Vehicles Symposium. 1013--1020.Google Scholar
- S. Pascual, A. Bonafonte, and J. Serra. 2017. SEGAN: Speech enhancement generative adversarial network. In Conference of the International Speech Communication Association. 3642--3646.Google Scholar
- Y. Xu, Z. Zhang, L. You, J. Liu, Z. Fan, and X. Zhou. 2020. scIGANs: single-cell RNA-seq imputation using generative adversarial networks. Nucleic Acids Res. 48, 15 (2020), 85.Google ScholarCross Ref
- M. Lucic, K. Kurach, M. Michalski, O. Bousquet, and S. Gelly. 2018. Are GANs created equal? A large-scale study. In Advances in Neural Information Processing Systems. 700--709.Google Scholar
- Q. Xu et al. 2018. An empirical study on evaluation metrics of generative adversarial networks. arXiv Preprint arXiv1806.07755, 2018.Google Scholar
- A. Borji. 2019. Pros and cons of GAN evaluation measures. Comput. Vis. Image Underst. 179 (2019), 41--65.Google ScholarDigital Library
Index Terms
- Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
Recommendations
Games of GANs: game-theoretical models for generative adversarial networks
AbstractGenerative Adversarial Networks (GANs) have recently attracted considerable attention in the AI community due to their ability to generate high-quality data of significant statistical resemblance to real data. Fundamentally, GAN is a game between ...
A lightweight ensemble discriminator for Generative Adversarial Networks
AbstractWhile Generative Adversarial Networks (GANs) have brought immense success in various content-generation tasks, they still face enormous challenges in generating high-quality visually realistic images because of the model collapse or ...
Generative adversarial networks in medical image segmentation: A review
Abstract PurposeSince Generative Adversarial Network (GAN) was introduced into the field of deep learning in 2014, it has received extensive attention from academia and industry, and a lot of high-quality papers have been published. ...
Graphical abstractDisplay Omitted
Highlights- A systematic review on medical image segmentation based on GAN is provided.
- The ...
Comments