Generative AI Model: GANs (Part 2)
Welcome Folks!
\
In the previous blog, we talked about the core concepts of GANs and the model components, namely Generator and discriminator. It also described the basic adversarial process which is involved in the optimization of the model. GANs work on a very powerful but basic concept of game theory “ Zero Sum Game.” This simply means that the gain of one player in the game is a loss for the other so the total sums up to zero.
\
This is applied in this context in a way that if the generator wins, it implies that the discriminator is not able to identify fake images, and if the discriminator wins, the generator is not doing a good job.
\
To understand how the above adversarial training works and what are the components that help the generator and discriminator achieve their goals, this blog is going to focus on a very basic Loss function “Cross Entropy.”
What Is Cross Entropy?
Cross Entropy is a measure of how far away the predictions are from the actual results (in terms of probability). The higher the entropy, the worse the predictions.
Basic Concepts
Let’s take an example to understand how it works. The task is to identify if an email is spam or not. Cross entropy simply tells how good the guess is compared to the actual scenario. Let’s break it down:
\
Scenario 1: If an email is spam and the guess was that 90% are the chances that the email is spam. This is very close to the actual answer implying cross entropy is lower.
\
Scenario 2: If an email is spam and guess is of 10%. This is very far and thereby has a very high cross-entropy.
Formula
The formula for cross-entropy:
\
L = -(y*log(p) + (1-y)*log(1-p))
\
\
where,
L is the loss
y is the actual label (spam = 1; not spam = 0)
p is the probability of a positive class (probability of it being spam)
\
Continuing with the spam email example, given an email there could be two values of y:
\
Case 1:
\
y = 0 ; email is not a spam
\
\
Above scenario turns y*log(p) = 0
\
Implying L = - (1-y)*log(1-p)
(1-y) = 1
But if
p is close to 0 (good prediction) => log(1-p) is close to zero implying L close to zero
\
p close to 1 (bad prediction) => log(1-p) is becomes very negative and implying L is higher
\
Case 2:
\
y = 1 ; email is a spam
\
Above scenario turns (1-y)*log(1-p) = 0
\
Implying L = - y*log(p)
y = 1
But if
p is close to 1 (good prediction) => log(p) is close to zero implying L close to zero
p close to 0 (bad prediction) => log(p) is becomes very negative and implying L is higher
\
The above scenario explains how the cross entropy actually works in estimating the performance of a model. Wherever the prediction is close to the actual case, it becomes very low. When it is far from the actual label, it becomes very high.
Cross Entropy in GANs
As explained in the last blog, GANs consist of two components: Generator and Discriminator. Depending upon the target for these two functions, the loss function varies for each.
Discriminator:
The role of the discriminator is to distinguish between real and fake images, therefore, both real and fake probabilities matter. Since it receives both real and fake images, the loss functions contain both parts of the loss. Whereas, in binary explained, it is all about one email and whether it is a scam or not a scam.
\
Therefore, the loss function looks like the following:
\
L = - (log(D(real)) + log(1-D(fake)))
\
Let’s break it down:
Real images: D(real) is the probability of it being real
\
Fake images: D(fake) is the probability of it being Fake
Generator
The generator’s task is to generate images that Discriminator classifies as real. Therefore, it tries to minimize the probability that the discriminator correctly identifies the fake images. Therefore, the cross entropy is:
\
L = - log(D(Fake))
\
Where,
D(Fake) is the probability of the Discriminator identifying correctly fake images
\
Cross-entropy is a fundamental concept in machine learning (not limited to the GANs). It provides a clear, interpretable measure of the difference between predicted and true distributions, thereby providing direction to the model in improved performance. By mastering cross-entropy and its applications, you can better navigate the complexities of machine learning and leverage GANs to their full potential.
\
The section ends here, stay tuned for the next post!
Welcome to Billionaire Club Co LLC, your gateway to a brand-new social media experience! Sign up today and dive into over 10,000 fresh daily articles and videos curated just for your enjoyment. Enjoy the ad free experience, unlimited content interactions, and get that coveted blue check verification—all for just $1 a month!
Account Frozen
Your account is frozen. You can still view content but cannot interact with it.
Please go to your settings to update your account status.
Open Profile Settings