GAN

1. KL divergence
2. JS divergence
3. EM distance

optimal D network = p(r) / (p(x) + p(r))

loss:

–> sum[ log(D(r)) + log(1-D(x)) ]
–> sum[ log(D(r)) - log(D(x)) ]

1. remove the sigmoid
2. change the loss function
3. use RSPOM (or SGD) rather than ADAM