ML_Maschinelles_Lernen_Grundlagen/Exercise_1
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity

1.2 and 1.3 finished

Browse Source
This commit is contained in:
paul-loedige 2021-11-10 10:58:28 +01:00
parent 158ab5a68e
commit c0af75025b
1 changed files with 20 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
exercise.ipynb
View File

@ -53,6 +53,21 @@
"\\end{align*}\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$ L = (y-\\Phi w)^T (y - \\Phi w) + \\lambda w^T I w $$\n",
"$$ = w ^T \\Phi^T\\Phi w - y^T \\Phi w - w^T\\Phi^Ty + y^Ty + \\lambda w^T I w $$\n",
"$$ = w^T\\Phi^T\\Phi w - 2 y^T\\Phi w + y^Ty + \\lambda w^T I w $$\n",
"$$ = y^Ty - 2y\\Phi^Tw^T + w^T(\\Phi^T\\Phi + \\lambda I)w $$\n",
"$$ \\frac{\\delta x^TAx}{\\delta x} = (A + A^T)x: \\frac{\\delta w^T(\\Phi^T\\Phi + \\lambda I)w}{\\delta w} = ((\\Phi^T\\Phi + \\lambda I) + (\\Phi\\Phi^T + \\lambda I^T))w = 2(\\Phi^T\\Phi + \\lambda I)w $$\n",
"$$ \\frac{\\delta y^Ty - 2y\\Phi^Tw^T + w^T(\\Phi^T\\Phi + \\lambda I)w }{\\delta w} = -2 \\Phi^Ty + 2(\\Phi^T\\Phi + \\lambda I)w $$\n",
"$$ -2 \\Phi^Ty + 2(\\Phi^T\\Phi + \\lambda I)w = 0, (\\text{to find optimum}) $$\n",
"$$ (\\Phi^T\\Phi + \\lambda I)w = \\Phi^Ty $$\n",
"$$ w^* = (\\Phi^T\\Phi + \\lambda I )^{-1} \\Phi^Ty $$"
]
},
{
"cell_type": "markdown",
"metadata": {},
@ -231,7 +246,8 @@
"The following function will calculate the weights for ridge regression. Fill in the missing code according to the formula for calculating the weight updates for ridge regression. <br>\n",
"Recall that the formula is given by \n",
"\\begin{align*}\n",
" \\boldsymbol{w} &= (\\boldsymbol{\\Phi} ^T \\boldsymbol{\\Phi} + \\lambda \\boldsymbol{I} )^{-1} \\boldsymbol{\\Phi}^T \\boldsymbol{y},\n",
" \\boldsymbol{w} &= (\\boldsymbol{\\Phi} ^T \\boldsymbol{\\Phi} + \\lambda \\boldsymbol{I} )^{-1} \\boldsymbol{\\Phi}^T \\boldsymbol{y}\\\\\n",
" (\\boldsymbol{\\Phi} ^T \\boldsymbol{\\Phi} + \\lambda \\boldsymbol{I}) \\boldsymbol{w} &= \\boldsymbol{\\Phi}^T \\boldsymbol{y},\n",
"\\end{align*}\n",
"where $\\boldsymbol{\\Phi}$ is the feature matrix (the matrix storing the data points applied to the polynomial features).\n",
"Hint: use np.linalg.solve for solving for the linear equation.\n",
@ -256,7 +272,9 @@
" \"\"\"\n",
" ##################\n",
" ##TODO\n",
" #################"
" #################\n",
" identity_matrix = np.identity(y.shape[0])\n",
" weights = (Phi.transpose() * Phi + ridge_factor * identity_matrix)^-1 * np.linalg.solve(Phi.transpose(), )"
]
},
{