Logistic regression

Sigmoid function

from matplotlib import pyplot as plt
import numpy as np

x = np.arange(-10, 10, 0.1)
f = 1 / (1 + np.exp(-x))

plt.ylabel("F(x)")
plt.plot(x, f)
plt.show()

Logistic regression

;

Odds

Since and range of is

We can let be the probability of sth happening (we predict)
And let be the probabilty of sth not happening (we predict)
Then represents odds.

Cost function

is the data set containing many labeled examples, which are pairs.
is true value, either or means happening or not happening

Natural logarithm

Domain of is
Range of is

When y = 0

When (true value, means sth not happening)

If our prediction is close to
(which means our prediction is close to true value),
then will be very small.

Otherwise, if our prediction is close to
(which means our prediction is far from true value)
then will be very large

When y = 1

When (true value, means sth not happening)

If our prediction is close to
(which means our prediction is far from true value),
then will be very large.

Otherwise, if our prediction is close to
(which means our prediction is close to true value)
then will be very small

Conclusion

Our target is to minimize the
Thus we need to get close to 1 when true value is
And get close to when true value is

Ref :
邏輯迴歸 (Logistic Regression)
[資料分析&機器學習] 第3.3講：線性分類-邏輯斯回歸(Logistic Regression) 介紹
Logistic Regression: Loss and Regularization
Machine Learning學習日記 — Coursera篇 (Week 3.2):Cost Function, Simplified Cost Function and Gradient Descent, Advanced Optimization