那个网站直接回做二手发电机,h5手机网站开发demo,有没有专门做淘宝客的网站,首页重庆网站建设图片若未能正常显示#xff0c;点击下面链接#xff1a; http://ihoge.cn/2018/Logistic-regression.html
在线性回归中#xff0c;我们想要建立一个模型#xff0c;来拟合一个因变量 y 与一个或多个独立自变量(预测变量) x 之间的关系。
给定#xff1a;
数据集 {(x…图片若未能正常显示点击下面链接 http://ihoge.cn/2018/Logistic-regression.html
在线性回归中我们想要建立一个模型来拟合一个因变量 y 与一个或多个独立自变量(预测变量) x 之间的关系。
给定
数据集
{(x(1),y(1)),...,(x(m),y(m))}{(x(1),y(1)),...,(x(m),y(m))}
\left\{ \left( x^{\left( 1 \right)},y^{\left( 1 \right)} \right),\; ...,\; \left(x^{\left( m \right)},y^{\left( m \right)} \right) \right\}xixix_{i}是d-维向量Xi(x(i)1,...,x(i)d)Xi(x1(i),...,xd(i))X^{i}\; =\; \left( x_{1}^{\left( i \right)},\; ...,\; x_{d}^{\left( i \right)} \right)
y(i)y(i)y^{(i)}是一个目标变量它是一个标量
线性回归模型可以理解为一个非常简单的神经网络
它有一个实值加权向量w(w(i),...,w(d))w(w(i),...,w(d))w\; =\; \left( w^{\left( i \right)},\; ...,\; w^{\left( d \right)} \right) 它有一个实值偏置量 b 它使用恒等函数作为其激活函数
线性回归模型可以使用以下方法进行训练
a) 梯度下降法
b) 正态方程(封闭形式解) w(XTX)−1XTyw(XTX)−1XTyw\; =\; \left( X^{T}X \right)^{-1}X^{T}y
其中 X 是一个矩阵其形式为(m,nfeatures)(m,nfeatures)\left( m,\; n_{featu\mbox{re}s} \right)包含所有训练样本的维度信息。
而正态方程需要计算(XTX)(XTX)\left( X^{T}X \right)的转置。这个操作的计算复杂度介于O(n2.4features)O(nfeatures2.4)O\left( n_{featu\mbox{re}s}^{2.4} \right)和O(n3features)O(nfeatures3)O\left( n_{featu\mbox{re}s}^{3} \right)之间而这取决于所选择的实现方法。因此如果训练集中数据的特征数量很大那么使用正态方程训练的过程将变得非常缓慢。
线性回归模型的训练过程有不同的步骤。首先(在步骤 0 中)模型的参数将被初始化。在达到指定训练次数或参数收敛前重复以下其他步骤。
第 0 步
用0 (或小的随机值)来初始化权重向量和偏置量或者直接使用正态方程计算模型参数
第 1 步(只有在使用梯度下降法训练时需要)
计算输入的特征与权重值的线性组合这可以通过矢量化和矢量传播来对所有训练样本进行处理 y˙X⋅wby˙X⋅wb\dot{y}\; =\; X\; \cdot \; w\; +b
其中 X 是所有训练样本的维度矩阵其形式为(m,nfeatures)(m,nfeatures)\left( m,\; n_{featu\mbox{re}s} \right)这里我用· 表示∧∧\wedge 。
第 2 步(只有在使用梯度下降法训练时需要)
用均方误差计算训练集上的损失J(w,b)1m∑mi1(y˙(i)−y(i))2J(w,b)1m∑i1m(y˙(i)−y(i))2J\left( w,b \right)\; =\; \frac{1}{m}\sum_{i=1}^{m}{\left( \dot{y}^{\left( i \right)}\; -\; y^{\left( i \right)} \right)^{2}}
第 3 步(只有在使用梯度下降法训练时需要):
对每个参数计算其对损失函数的偏导数
∂J∂wj2m∑mi1(y˙(i)−y(i))x(i)j∂J∂wj2m∑i1m(y˙(i)−y(i))xj(i)\frac{\partial J}{\partial w_{j}}\; =\; \frac{2}{m}\sum_{i=1}^{m}{\left( \dot{y}^{\left( i \right)}\; -\; y^{\left( i \right)} \right)}x_{j}^{\left( i \right)}
∂J∂b2m∑mi1(y˙(i)−y(i))∂J∂b2m∑i1m(y˙(i)−y(i))\frac{\partial J}{\partial b}\; =\; \frac{2}{m}\sum_{i=1}^{m}{\left( \dot{y}^{\left( i \right)}\; -\; y^{\left( i \right)} \right)}
所有偏导数的梯度计算如下
ΔwJ2mXT(y˙−y)ΔwJ2mXT(y˙−y)\Delta _{w}J\; =\; \frac{2}{m}X^{T}\; \left( \dot{y}\; -\; y \right)
ΔbJ2m(y˙−y)ΔbJ2m(y˙−y)\Delta _{b}J\; =\; \frac{2}{m}\left( \dot{y}\; -\; y \right)
第 4 步(只有在使用梯度下降法训练时需要:
更新权重向量和偏置量
ww−ηΔwJww−ηΔwJw\; =\; w\; -\; \eta \Delta _{w}J
ΔbJ2m(y˙−y)ΔbJ2m(y˙−y)\Delta _{b}J\; =\; \frac{2}{m}\left( \dot{y}\; -\; y \right)
其中η表示学习率
代码实现
数据集
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
np.random.seed(123)X 2 * np.random.rand(500, 1)
y 5 3 * X np.random.randn(500, 1)
fig plt.figure(figsize(8,6))
plt.scatter(X, y)
plt.title(Dataset)
plt.xlabel(First feature)
plt.ylabel(Second feature)
plt.show() X_train, X_test, y_train, y_test train_test_split(X, y)
print(fShape X_train: {X_train.shape})
print(fShape y_train: {y_train.shape})
print(fShape X_test: {X_test.shape})
print(fShape y_test: {y_test.shape})
Shape X_train: (375, 1)
Shape y_train: (375, 1)
Shape X_test: (125, 1)
Shape y_test: (125, 1)线性回归分类 源码编译 class LinearRegression:def __init__(self):passdef train_gradient_descent(self, X, y, learning_rate0.01, n_iters100):Trains a linear regression model using gradient descent# Step 0: Initialize the parametersn_samples, n_features X.shapeself.weights np.zeros(shape(n_features,1))self.bias 0costs []for i in range(n_iters):# Step 1: Compute a linear combination of the input features and weightsy_predict np.dot(X, self.weights) self.bias# Step 2: Compute cost over training setcost (1 / n_samples) * np.sum((y_predict - y)**2)costs.append(cost)if i % 100 0:print(fCost at iteration {i}: {cost})# Step 3: Compute the gradientsdJ_dw (2 / n_samples) * np.dot(X.T, (y_predict - y))dJ_db (2 / n_samples) * np.sum((y_predict - y)) # Step 4: Update the parametersself.weights self.weights - learning_rate * dJ_dwself.bias self.bias - learning_rate * dJ_dbreturn self.weights, self.bias, costsdef train_normal_equation(self, X, y):Trains a linear regression model using the normal equationself.weights np.dot(np.dot(np.linalg.inv(np.dot(X.T, X)), X.T), y)self.bias 0return self.weights, self.biasdef predict(self, X):return np.dot(X, self.weights) self.bias
使用梯度下降进行训练
regressor LinearRegression()
w_trained, b_trained, costs regressor.train_gradient_descent(X_train, y_train, learning_rate0.005, n_iters600)
fig plt.figure(figsize(8,6))
plt.plot(np.arange(600), costs)
plt.title(Development of cost during training)
plt.xlabel(Number of iterations)
plt.ylabel(Cost)
plt.show()
Cost at iteration 0: 66.45256981003433
Cost at iteration 100: 2.208434614609594
Cost at iteration 200: 1.2797812854182806
Cost at iteration 300: 1.2042189195356685
Cost at iteration 400: 1.1564867816573
Cost at iteration 500: 1.121391041394467Text(0,0.5,Cost)测试梯度下降模型
n_samples, _ X_train.shape
n_samples_test, _ X_test.shapey_p_train regressor.predict(X_train)
y_p_test regressor.predict(X_test)error_train (1 / n_samples) * np.sum((y_p_train - y_train) ** 2)
error_test (1 / n_samples_test) * np.sum((y_p_test - y_test) ** 2)print(fError on training set: {np.round(error_train, 4)})
print(fError on test set: {np.round(error_test)})
Error on training set: 1.0955
Error on test set: 1.0使用正规方程normal equation训练
X_b_train np.c_[np.ones((n_samples)), X_train]
X_b_test np.c_[np.ones((n_samples_test)), X_test]reg_normal LinearRegression()
w_trained reg_normal.train_normal_equation(X_b_train, y_train)
测试正规方程模型
y_p_train reg_normal.predict(X_b_train)
y_p_test reg_normal.predict(X_b_test)error_train (1 / n_samples) * np.sum((y_p_train - y_train) ** 2)
error_test (1 / n_samples_test) * np.sum((y_p_test - y_test) ** 2)print(fError on training set: {np.round(error_train, 4)})
print(fError on test set: {np.round(error_test, 4)})
Error on training set: 1.0228
Error on test set: 1.0432可视化测试预测
fig plt.figure(figsize(8,6))
plt.scatter(X_train, y_train)
plt.scatter(X_test, y_p_test)
plt.xlabel(First feature)
plt.ylabel(Second feature)
plt.show()
Text(0,0.5,Second feature)转载注明出处 http://ihoge.cn/2018/Logistic-regression.html
