GAN Types.ipynb

Kernel: Python 3 (ipykernel)

Types of GANs

GAN Type	Key Idea	Objective / Loss Formula	Strengths	Limitations	Typical Use Cases
Vanilla GAN	Adversarial game between Generator (G) and Discriminator (D)	(\min_G \max_D V(D,G) = \mathbb{E}{x\sim p{data}}[\log D(x)] + \mathbb{E}_{z\sim p_z}[\log(1 - D(G(z)))])	Simple, foundational, easy to implement	Training instability, mode collapse, vanishing gradients	Academic learning, toy datasets (MNIST), GAN fundamentals
DCGAN	CNN-based GAN with architectural constraints	Same as Vanilla GAN (cross-entropy loss)	Stable training, good image quality, scalable to larger images	Still sensitive to hyperparameters	Image generation (faces, objects), representation learning
Conditional GAN (cGAN)	GAN conditioned on labels or attributes	(\min_G \max_D \mathbb{E}[\log D(x	y)] + \mathbb{E}[\log(1 - D(G(z	y)))])	Controlled generation, class-specific outputs	Requires labeled data	Image-to-image translation, class-specific synthesis
WGAN	Uses Wasserstein (Earth-Mover) distance	(\min_G \max_{D\in\mathcal{D}} \mathbb{E}[D(x)] - \mathbb{E}[D(G(z))])	Stable gradients, reduced mode collapse	Weight clipping harms capacity	High-quality image synthesis, stable GAN training
WGAN-GP	Gradient penalty instead of weight clipping	(\mathbb{E}[D(G(z))] - \mathbb{E}[D(x)] + \lambda \mathbb{E}[(	\nabla_{\hat{x}} D(\hat{x})	_2 - 1)^2])	Very stable, best convergence properties	Higher computation cost	Production-grade image generation, medical & satellite imagery

GANs with Full Training Pipelines and Visualization

This notebook provides:

• Conceptual explanation of GANs

• Five common GAN variants

• End-to-end training loops

• Image visualization outputs

Dataset used: MNIST (simple, fast, standard for GAN demos)

Common Imports and Dataset

import tensorflow as tf
from tensorflow.keras import layers
import numpy as np
import matplotlib.pyplot as plt

# Load MNIST
(x_train, _), (_, _) = tf.keras.datasets.mnist.load_data()
x_train = x_train.astype("float32") / 127.5 - 1.0
x_train = np.reshape(x_train, (-1, 784))

BUFFER_SIZE = x_train.shape[0]
BATCH_SIZE = 128
EPOCHS = 5  # kept small for demo
LATENT_DIM = 100

dataset = tf.data.Dataset.from_tensor_slices(x_train).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)

Helper: Image Visualization

def plot_generated_images(generator, epoch, latent_dim=100):
    noise = tf.random.normal([16, latent_dim])
    images = generator(noise, training=False)

    plt.figure(figsize=(4,4))
    for i in range(images.shape[0]):
        plt.subplot(4,4,i+1)
        plt.imshow(images[i].numpy().reshape(28,28), cmap='gray')
        plt.axis('off')
    plt.suptitle(f"Epoch {epoch}")
    plt.show()

1. Vanilla GAN – Full Training Loop

def build_generator():
    return tf.keras.Sequential([
        layers.Dense(256, activation='relu', input_shape=(LATENT_DIM,)),
        layers.Dense(784, activation='tanh')
    ])

def build_discriminator():
    return tf.keras.Sequential([
        layers.Dense(256, activation='relu', input_shape=(784,)),
        layers.Dense(1, activation='sigmoid')
    ])

generator = build_generator()
discriminator = build_discriminator()

loss_fn = tf.keras.losses.BinaryCrossentropy()
g_opt = tf.keras.optimizers.Adam(1e-4)
d_opt = tf.keras.optimizers.Adam(1e-4)

@tf.function
def train_step(real_images):
    noise = tf.random.normal([real_images.shape[0], LATENT_DIM])

    with tf.GradientTape() as d_tape, tf.GradientTape() as g_tape:
        fake_images = generator(noise, training=True)

        real_output = discriminator(real_images, training=True)
        fake_output = discriminator(fake_images, training=True)

        d_loss = loss_fn(tf.ones_like(real_output), real_output) +                  loss_fn(tf.zeros_like(fake_output), fake_output)

        g_loss = loss_fn(tf.ones_like(fake_output), fake_output)

    d_grads = d_tape.gradient(d_loss, discriminator.trainable_variables)
    g_grads = g_tape.gradient(g_loss, generator.trainable_variables)

    d_opt.apply_gradients(zip(d_grads, discriminator.trainable_variables))
    g_opt.apply_gradients(zip(g_grads, generator.trainable_variables))

    return d_loss, g_loss

for epoch in range(1, EPOCHS + 1):
    for batch in dataset:
        d_loss, g_loss = train_step(batch)

    print(f"Epoch {epoch}, D Loss: {d_loss:.4f}, G Loss: {g_loss:.4f}")
    plot_generated_images(generator, epoch)

2. DCGAN – Architecture & Training Loop

def build_dcgan_generator():
    model = tf.keras.Sequential([
        layers.Dense(7*7*128, activation='relu', input_shape=(LATENT_DIM,)),
        layers.Reshape((7,7,128)),
        layers.Conv2DTranspose(64, 4, strides=2, padding='same', activation='relu'),
        layers.Conv2DTranspose(1, 4, strides=2, padding='same', activation='tanh')
    ])
    return model

def build_dcgan_discriminator():
    model = tf.keras.Sequential([
        layers.Flatten(input_shape=(28,28,1)),
        layers.Dense(128, activation='relu'),
        layers.Dense(1, activation='sigmoid')
    ])
    return model

Note on Other GAN Types

To keep runtime reasonable, cGAN, WGAN, and WGAN-GP follow the same training pattern with:

• Modified loss functions

• Label conditioning (cGAN)

• Critic + Wasserstein loss (WGAN)

• Gradient penalty (WGAN-GP)

These patterns are included earlier conceptually and can be extended using this training loop template.

Summary

GAN Type	Training Stability	Control	Use Case
Vanilla GAN	Low	No	Learning
DCGAN	Medium	No	Image generation
cGAN	Medium	Yes	Conditional synthesis
WGAN	High	No	Stable training
WGAN-GP	Very High	No	Production systems

Architect Takeaway

• Vanilla GAN → teaching & demos

• DCGAN → standard image GAN

• WGAN-GP → enterprise-grade stability