Machine Learning, Visualized · Vol. XXII

The
Adversaries

Two networks lock horns. The generator forges samples; the discriminator tells real from fake. Each tries to outdo the other, and the equilibrium is photorealism.

The concept

A generative adversarial network (GAN) trains two models simultaneously: a generator that maps random noise to fake samples, and a discriminator that scores each input as real or fake.

The discriminator is trained to maximize the score gap between real and fake. The generator is trained to minimize that gap — to make its forgeries indistinguishable. They're locked in a minimax game, and the optimum is when the discriminator can no longer tell the difference (50% accuracy).

Watch the dynamic on a 2D toy: real points come from a fixed distribution; fake points come from the generator. The discriminator's decision boundary contracts and morphs as the generator improves.

Why ML cares

From 2014 to 2020, GANs produced the sharpest image-generation results in ML — StyleGAN faces, BigGAN ImageNet samples, image-to-image translation (CycleGAN, pix2pix). Diffusion models eventually overtook them on quality, but GANs remain the reference for fast, single-pass generation.

The adversarial framing reappears everywhere: domain adaptation, super-resolution (ESRGAN), audio synthesis (HiFi-GAN), and even RLHF — where a learned reward model plays the role of the discriminator that pushes a language model toward human-preferred outputs.

Try this

Hit Train. The generator's points (orange) start as a tight blob near origin and spread to match the real-data ring. Watch the discriminator's heatmap shift as it loses traction.
Watch the loss strip above the canvas. G-loss (orange) and D-loss (ink) push against each other and oscillate — they don't both fall like a normal training curve. That zig-zag is the adversarial game.
Try spiral or two clusters. Some shapes are harder — the generator may collapse to a single mode (a known GAN failure called mode collapse). Uncovered modes pulse on the canvas.
Slide the discriminator strength knob. Too strong, and the generator gets no useful signal; too weak, and it goes nowhere. Tuning this balance is the eternal GAN-training pain.

· Black points are real samples. Orange points are the generator's fakes. The shaded heatmap is the discriminator's "real probability" — orange means it thinks real, ink means fake. The strip above plots G-loss and D-loss as training proceeds — watch them oscillate.

Before this

Generating realistic images was hard. VAEs gave blurry samples (MSE averages); pixelCNNs were slow. Goodfellow (2014) reframed generation as a game: generator vs. discriminator. The discriminator's learned loss replaced hand-crafted ones — and that move powered photorealistic faces, deepfakes, and the adversarial fine-tuning behind today's diffusion models.

The architecture

Two networks, one feedback loop. G turns noise into fake samples; D classifies real vs fake; D's mistakes are gradient signal pushing G to forge better.

The minimax game

min_G max_D 𝔼[log D(x)] + 𝔼[log(1 − D(G(z)))]

The two networks play a game. D wants log D(x) high (recognize real) and log(1 − D(G(z))) high (recognize fakes). G wants the opposite — to push log(1 − D(G(z))) down, by making D(G(z)) close to 1. They alternate; equilibrium = D guesses randomly because G's fakes are indistinguishable.

Symbol gloss

z: random noise vector drawn from a fixed prior (usually N(0, I)). G(z): the generator's fake sample. D(·): discriminator's score in [0, 1] — closer to 1 means "real." 𝔼: expected value (average over many samples).

Mode collapse

A common failure: G discovers one output that fools D and produces only that. Real distribution has many modes; G covers one. The flashing label on the canvas warns when this happens.

Beyond images

RLHF for language models is GAN-shaped: the language model is the generator; a learned preference model is the discriminator (reward model).

Where you've seen this04 examples

StyleGAN faces

The "this person does not exist" website is StyleGAN samples. Trained on a few hundred thousand celebrity photos, it produces photorealistic faces that have never existed. Cited as the moment GANs hit human-quality.

Image-to-image translation

CycleGAN, pix2pix — turn satellite photos into maps, summer into winter, horses into zebras, sketches into paintings. All variations on the GAN recipe with conditioning.

Audio super-resolution

HiFi-GAN and similar models upsample low-bitrate audio to studio quality. The discriminator's pressure for "realism" produces sharper sound than MSE-trained models.

RLHF — adversarial in spirit

Training a chatbot via human preferences uses a reward model that scores outputs as "good" or "bad." The chatbot is trained to maximize that score — adversarial training, dressed up.