目录
一、The concept of gradient：
① In a univariate function：
②In multivariate functions：
二、Introduction of gradient descent cases：
三、Gradient descent formula and its simple understanding:
四、Formula operation precautions:
一、The concept of gradient：
① In a univariate function：
gradient is actually the differentiation of the function, representing the slope of the tangent of the function at a given point
②In multivariate functions：
a gradient is a vector with a direction, and the direction of the gradient indicates the direction in which the function rises the fastest at a given point
二、Introduction of gradient descent cases：
Do you remember the golf course inside the cat and mouse? It looks like this in the animation:
Let’s take a look at these two pictures. You can easily see the distant hill, right? We can take it as the most typical example, and the golf course can also be abstracted into a coordinate map:
So in this coordinate, we will correspond the following (x, y) to (w, b) respectively. Then, when J (w, b) is at its maximum, which is the peak in the red area of the graph, we start the gradient descent process.
Firstly, we rotate one circle from the highest point to find the direction with the highest slope. At this point, we can take a small step down. The reason for choosing this direction is actually because it is the steepest direction. If we walk down the same step length, the height of descent will naturally be the highest, and we can also walk faster to the lowest point (local minimum point). At the same time, after each step, we look around and c服务器托管网hoose. Finally, we can determine this path:
Finally reaching the local minimum point A, is this the only minimum point? Of course not:
It is also possible to reach point B, which is also a local minimum point. At this point, we have introduced the implementation process of gradient descent, and we will further understand its meaning through mathematical formulas.
三、Gradient descent formula and its simple understanding:
We first provide the formula for gradient descent:
In the formula, 
corresponds to what we call the learning rate, and the equal sign is the same as the assignment symbol in computer program code. J (w, b) can be found in the regression equation blog in the previous section. As for the determination of the learning rate, we will share it with you next time. Here, we will first understand the meaning of the formula:
Firstly, let’s simplify the formula and take b equal to 0 as an example. This way, we can better understand its meaning through a two-dimensional Cartesian coordinate system:
In this J (w, b) coordinate graph, which is a quadratic function, since we consider b in the equation to be 0,So we can assume that
=
,So, such a partial derivative can be seen as the derivative in the unary case. At this point, it can be seen that when>0 and the corresponding w value is in the right half, the derivative is positive, 服务器托管网that is, its slope is positive. This is equivalent to subtracting a positive number from w, and its w point will move to the left, which is the closest to its minimum value, which is the optimal solution. Similarly, when in the left half of the function, its w will move to the right, which is close to the minimum value, So the step size for each movement is .
This is a simple understanding of the gradient descent formula.
四、Formula operation precautions:
This is a simple understanding of the gradient descent formula
just like this:
The following is an incorrect order of operations that should be avoided:
This is the understanding of the formula and algorithm implementation for gradient descent. As for the code implementation, we will continue to explain it in future articles.
Machine Learning —- Cost function-CSDN博客
服务器托管，北京服务器托管，服务器租用 http://www.fwqtg.net
相关推荐: 修复Jenkins fossa扫描的时候报错的问题
学无止境，气有浩然 文章目录 背景 报错 问题 解决 打完收工！ 背景 公司项目最近私有仓库从私有云nexus迁移到了jfog，拉取依赖不再直接拉取，而是每个使用的人需要配置一份账户和密文scret，通过每个人的账号拉取。以此为背景建立Jenkins的Foss…
