1313 lines
213 KiB
Plaintext
1313 lines
213 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# EXERCISE 1 - ML - Grundverfahren WS 21/22\n",
|
|
"\n",
|
|
"**Exercise 1**: Ge Li ge.li@kit.edu\n",
|
|
"\n",
|
|
"**Exercise 2 & 3**: Philipp Becker philipp.becker@kit.edu\n",
|
|
"\n",
|
|
"**Solved by**: Pascal Schindler ujvhi@student.kit.edu, Paul Lödige ucycy@student.kit.edu\n",
|
|
"## Submission Instructions\n",
|
|
"Please follow the instruction from Exercise ZERO!\n",
|
|
"\n",
|
|
"\n",
|
|
"## 1.) Linear Regression"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 1.1) Matrix Vector Calculus (1 Point)\n",
|
|
"Given the following element-wise expression of a matrix-vector product,\n",
|
|
"rewrite it in matrix form:\n",
|
|
"\n",
|
|
"\\begin{align*}\n",
|
|
" g &= \\alpha \\sum_i \\sum_j \\sum_k z_k x_{ij} q_i y_{jk}\\\\\n",
|
|
" &= \\alpha \\sum_i^I \\sum_j^J \\sum_k^K z_k x_{ij} q_i y_{jk}\\qquad (\\text{mit }I,J,K\\in \\mathbb{N})\\\\\n",
|
|
" &= Y\\cdot X \\cdot Q \\cdot Z \\cdot \\alpha\\\\\n",
|
|
" &= \\begin{bmatrix} y_{11} & \\cdots & y_{1J}\\\\ \\vdots & \\ddots & \\vdots\\\\ y_{K1} & \\cdots & y_{KJ}\\end{bmatrix} \\cdot\n",
|
|
" \\begin{bmatrix} x_{11} & \\cdots & x_{1I}\\\\ \\vdots & \\ddots & \\vdots\\\\ x_{J1} & \\cdots & x_{JI}\\end{bmatrix} \\cdot\n",
|
|
" \\begin{bmatrix} q_1 \\\\ \\vdots \\\\ q_I\\end{bmatrix} \\cdot\n",
|
|
" \\begin{bmatrix} z_1 & \\cdots & z_K\\end{bmatrix} \\cdot\n",
|
|
" \\alpha\n",
|
|
"\\end{align*}\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 1.2) Derive Ridge Regression Weights (4 Points)\n",
|
|
"Derive the optimal solution of weights in Ridge Regression using matrix form, i\n",
|
|
".e. $\\boldsymbol{w}= ?$\n",
|
|
"\n",
|
|
"Hint: You will need derivatives for vectors/matrices. Start\n",
|
|
"from the matrix objective for ridge regression as stated here\n",
|
|
"\n",
|
|
"\\begin{align*}\n",
|
|
"L &= (\\boldsymbol{y}-\\boldsymbol{\\Phi} \\boldsymbol{w})^T(\\boldsymbol{y}-\\boldsymbol{\\Phi} \\boldsymbol{w}) + \\lambda \\boldsymbol{w}^T \\boldsymbol{I} \\boldsymbol{w}. \\\\\n",
|
|
"\\end{align*}\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\\begin{align}\n",
|
|
" L &= (y-\\Phi w)^T (y - \\Phi w) + \\lambda w^T I w\\\\\n",
|
|
" &= w ^T \\Phi^T\\Phi w - y^T \\Phi w - w^T\\Phi^Ty + y^Ty + \\lambda w^T I w\\\\\n",
|
|
" &= w^T\\Phi^T\\Phi w - 2 y^T\\Phi w + y^Ty + \\lambda w^T I w\\\\\n",
|
|
" &= y^Ty - 2y\\Phi^Tw^T + w^T(\\Phi^T\\Phi + \\lambda I)w\\\\\n",
|
|
" \\frac{\\delta x^TAx}{\\delta x} = (A + A^T)x: \\frac{\\delta w^T(\\Phi^T\\Phi + \\lambda I)w}{\\delta w} &= ((\\Phi^T\\Phi + \\lambda I) + (\\Phi\\Phi^T + \\lambda I^T))w = 2(\\Phi^T\\Phi + \\lambda I)w\\\\\n",
|
|
" \\frac{\\delta y^Ty - 2y\\Phi^Tw^T + w^T(\\Phi^T\\Phi + \\lambda I)w }{\\delta w} &= -2 \\Phi^Ty + 2(\\Phi^T\\Phi + \\lambda I)w\\\\\n",
|
|
" -2 \\Phi^Ty + 2(\\Phi^T\\Phi + \\lambda I)w &= 0, (\\text{to find optimum})\\\\\n",
|
|
" (\\Phi^T\\Phi + \\lambda I)w &= \\Phi^Ty\\\\\n",
|
|
" w^* &= (\\Phi^T\\Phi + \\lambda I )^{-1} \\Phi^Ty\\\\\n",
|
|
"\\end{align}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Ridge Regression - Code\n",
|
|
"Let's first get the data\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<matplotlib.legend.Legend at 0x7f0dc1a3ed30>"
|
|
]
|
|
},
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEGCAYAAACZ0MnKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAhXElEQVR4nO3df3ScZZ338fc3pVAD3UdIu2uflky6HtgDpdgfsYDYRbRA+SXKI2Ibal2xQZb647iglK6Ww9mwz/OgiwdXwdZF2TZWAe2iriyl+1DXH8Ca0vJLQFqahLQV0lSgECm0+T5/3DPhzmQmuZPMzH3PzOd1zpzMXHPPzHeSdr5zXd/rum5zd0RERKKoiTsAEREpH0oaIiISmZKGiIhEpqQhIiKRKWmIiEhkh8UdQLFNmjTJGxoa4g5DRKSsbNmyZa+7T85ur/ik0dDQQFtbW9xhiIiUFTPryNWu4SkREYlMSUNERCJT0hARkcgqvqaRy5tvvklXVxevv/563KFUvQkTJjBt2jTGjx8fdygiEkFVJo2uri4mTpxIQ0MDZhZ3OFXL3enp6aGrq4vp06fHHY6IRJDI4Skz+ysz2xa6vGJmnzez681sV6j9vNE8/+uvv05dXZ0SRszMjLq6OvX4RAqptRUaGqCmJvjZ2lrQp09kT8PdnwFmAZjZOGAXsAH4G+Bmd//qWF9DCSMZ9HcQKaDWVmhuht7e4HZHR3AboKmpIC+RyJ5Glg8AO9w955xhERFJW7nyrYSR0dsbtBdIOSSNjwHrQ7eXm9ljZna7mR2d6wFm1mxmbWbW1t3dXZooRUTi1tk5svZRSHTSMLPDgQ8Cd6WbbgXeSTB0tQf4Wq7Huftqd29098bJkwetgh+5Ao8RvvTSS3zrW98a1WO//vWv05v9TWII3/ve91i+fPmQx2zevJnf/OY3o4pHRBKkvn5k7aOQ6KQBnAs84u4vALj7C+5+yN37gDXAvKJHkBkj7OgA97fGCMeQOEqZNKJQ0hCpEC0tUFs7sK22NmgvkKQnjUWEhqbMbErovg8DTxQ9giKMEV577bXs2LGDWbNmcc0113DTTTfx7ne/m5NPPplVq1YB8Nprr3H++efzrne9i5NOOokf/vCH3HLLLezevZszzzyTM888M+/zf/e73+X444/njDPO4Ne//nV/+09/+lNOOeUUZs+ezYIFC3jhhRdob2/ntttu4+abb2bWrFn88pe/zHmciCRYZjRkyRJ429ugrg7MIJWC1asLVgQHgrnySbwAtUAP8D9CbWuBx4HHgJ8AU4Z7nrlz53q23/3ud4Pa8jJzD/oYAy9m0Z8jy86dO33GjBnu7n7ffff5smXLvK+vzw8dOuTnn3++/+IXv/C7777bP/WpT/U/5qWXXnJ391Qq5d3d3Xmfe/fu3X7sscf6iy++6AcOHPD3vOc9ftVVV7m7+759+7yvr8/d3desWeNf+MIX3N191apVftNNN/U/R77jimVEfw8RGWjdOvfa2oGfT7W1QfsYAG2e4zM1kVNuAdy9F6jLaltS8kDq64MhqVztBbBx40Y2btzI7NmzAXj11Vd59tlnmT9/PldffTVf+tKXuOCCC5g/f36k53v44Yd53/veR6aWc+mll/L73/8eCBY1XnrppezZs4c33ngj74K6qMeJSAIMNRpSyB5GWtKHp+JX5DFCd2fFihVs27aNbdu2sX37di6//HKOP/54tmzZwsyZM1mxYgU33HBD5OfMt/bhM5/5DMuXL+fxxx/n29/+dt5FdVGPE5EEKMGMqTAljeE0NQVjgqlUwcYIJ06cyP79+wE455xzuP3223n11VcB2LVrFy+++CK7d++mtraWyy67jKuvvppHHnlk0GNzOeWUU9i8eTM9PT28+eab3HXXXf33vfzyy0ydOhWAO+64I2c8Qx0nIglUghlTYYkdnkqUpqaCdvPq6uo4/fTTOemkkzj33HNZvHgxp512GgBHHXUU69atY/v27VxzzTXU1NQwfvx4br31VgCam5s599xzmTJlCg888MCg554yZQrXX389p512GlOmTGHOnDkcOnQIgOuvv55LLrmEqVOncuqpp7Jz504ALrzwQj7ykY9wzz338I1vfCPvcSKSQC0tA1eBQ8FnTIVZUO+oXI2NjZ595r6nnnqKE044IaaIJJv+HiKj0Noa1C06O+GYY4K2ffuCHkZLy5i/6JrZFndvzG5XT0NEpNxk7zHV0xP0LtauLUrxO0xJo4ydcsopHDhwYEDb2rVrmTlzZkwRiUhJlHjGVJiSRhl7+OGH4w5BROJQ4hlTYZo9JSJSbko8YypMSUNEpFxktgvp6AiWAIQVccZUmJKGiEg5CG+eCsGGIZnEUYw9pvJQ0ojBWHa5Pe+883jppZeGPOYrX/kKmzZtGtXzD0XbrIvEKFfx2z1IGO3tJUkYoKQRSaFPuTtU0sgsxMvn5z//OW9/+9uHPOaGG25gwYIFow1vTJQ0RIokxuJ3mJLGMIpwOo1BW6Nv3ryZM888k8WLF/dPl/3Qhz7E3LlzmTFjBqtXr+5/bENDA3v37qW9vZ0TTjiBZcuWMWPGDM4++2z+9Kc/AfCJT3yCu+++u//4VatWMWfOHGbOnMnTTz8NQHd3N2eddRZz5szhiiuuIJVKsXfv3kGxapt1kYSIsfg9QK6tbyvpMtat0VOp3Dujp1KRn2KQ8Nbo7u4PPPCA19bW+nPPPdff1tPT4+7uvb29PmPGDN+7d286nmBr9J07d/q4ceN869at7u5+ySWX+Nq1a93dfenSpX7XXXf1H3/LLbe4u/s3v/lNv/zyy93d/aqrrvIbb7zR3d3vvfdeBwZtuV6qbda1NbrIENate+uDKPtUDQXYAj0fym1r9KQoVY9w3rx5A7Ygv+WWW9iwYQMAzz//PM8++yx1dQN2imf69OnMmjULgLlz59Le3p7zuS+++OL+Y3784x8D8Ktf/ar/+RcuXMjRRw8+3bq2WReJWfbK70zxO1PLKMB2ISOl4alhlKpHeOSRR/Zf37x5M5s2beLBBx/k0UcfZfbs2Tm3Jz/iiCP6r48bN46DBw/mfO7MceFjPOKeY9pmXSRGCSl+hyU2aZhZu5k9bmbbzKwt3XaMmd1vZs+mfw7+elxgxTidxnDbm7/88sscffTR1NbW8vTTT/PQQw+N/sXyeO9738udd94JBCeC+uMf/zjoGG2zLhKzhBS/wxKbNNLOdPdZ/tZOi9cC/+nuxwH/mb5dVEU4ncaArdGvueaaQfcvXLiQgwcPcvLJJ/PlL3+ZU089dQzvILdVq1axceNG5syZw7333suUKVOYOHHigGPC26wvWLCAOXPm9N+X2T59/vz5TJo0qb/9wgsvZMOGDf2F8HzHiUgESSl+hyR2a3Qzawca3X1vqO0Z4H3uvsfMpgCb3f2vhnoebY2e24EDBxg3bhyHHXYYDz74IFdeeSXbtm2LJRb9PUSyZLY9z6z8Dn9O19aWZCFfOW6N7sBGM3Pg2+6+GvgLd98DkE4cf57rgWbWDDQD1MeYkZOss7OTj370o/T19XH44YezZs2auEMSEUhk8TssyUnjdHffnU4M95vZ01EfmE4wqyHoaRQrwHJ23HHHsXXr1rjDEJFswxW/Y5bYmoa7707/fBHYAMwDXkgPS5H++eIYnr8QYcoY6e8gkiWBxe+wRCYNMzvSzCZmrgNnA08APwGWpg9bCtwzmuefMGECPT09+sCKmbvT09PDhAkT4g5FJH6Z/YryfS4lZKg9qcNTfwFsSK8ROAz4vrv/h5n9FrjTzC4HOoFLRvPk06ZNo6uri+7u7oIFLKMzYcIEpk2bFncYIvHKrmNkK9G251EkdvZUoeSaPSUikiiZc2TkElPxuxxnT4mIVId89QqzRBS/wxJZ0xARqSoJXMSXj5KGiEhcEnD61pFS0hARiUNCTt86UqppiIjEIeGL+PJRT0NEJA4JX8SXj5KGiEiBZUoVNTXBzwGnhy6TRXz5aHhKRKSAstfpdXQEtwGaKJ9FfPlocZ+ISAHlW6eXSkE7ee7MHBDzDrZhWtwnIlICeUsVHX1AnoSRwEV8+aimISJSQHnX6TFEgTvhdYwwJQ0RkQI677wc6/R4jRauy/2AMqhjhClpiIgUSGsr3HHHwIlRRh9L+S5NrB/8gAQv4stHNQ0RkQLJuV6PGn7OBcBnBt6R8EV8+ainISJSIHmL4GTVLMpsSCpMSUNEpEAiFcHLcEgqLJFJw8yONbMHzOwpM3vSzD6Xbr/ezHaZ2bb05by4YxURAaC1lZZXP0strw1oHlAEz0ytLdOEAcmtaRwE/s7dH0mfK3yLmd2fvu9md/9qjLGJiAyUXgbe1NsL7GUlN9JJPfV00sJ1bxXBy2hqbT6JTBruvgfYk76+38yeAqbGG5WISB6hCngT63PPlCrjOkZYIoenwsysAZgNPJxuWm5mj5nZ7WZ2dJ7HNJtZm5m1dXd3lypUEalWQ+1Ma1b2dYywRCcNMzsK+BHweXd/BbgVeCcwi6An8rVcj3P31e7e6O6NkydPLlW4IlJthtuxNpWCvr6yr2OEJXJ4CsDMxhMkjFZ3/zGAu78Qun8N8LOYwhORape9nW22ChmOypbInoaZGfAvwFPu/k+h9imhwz4MPFHq2ESkymV6F5ddlj9hVNBwVLak9jROB5YAj5vZtnTbdcAiM5sFONAOXBFHcCJSpYbrXUBZ7Vg7GolMGu7+K8By3PXzUsciItIv1z4h2SpgWu1QEjk8JSKSKJkhqXwnUMqo0DpGmJKGiMhQMkNSwyWMCq5jhCVyeEpEJDGGG5Kqra2KZJGhnoaISC5RhqSqpHcRpp6GiEi2KLOkyvR8GGOlnoaISEaUNRhQFQXvfJQ0RERABe+INDwlItWttTUodg+XLKBqh6TClDREpHpFqV1kVPGQVJiGp0Sk+kStXWRU+ZBUmHoaIlJdRtq7ULIYQD0NEakO6l0UhJKGVJ/Mh0dNDUyaFFyyrzc0BMeVMJQSvWR1ijozCoLexbp1FXXipEJS0pDyFyUJZK6bwZIlwYeHO/T0BJfs6x0dwXFmwz/nKD/xw59jmZdsblbiKCj1LgrOPN9pCitEY2Ojt7W1xR2GjFVmWmRnJxxzTNC2b19wff9+eOONeOODIMG4Q11dcHvfvmCb7JaWnB9C+Xao0KzOMQpPoc38TYaj2sUgZrbF3Ruz28uup2FmC83sGTPbbmbXxh2PFFHmW+JwvYMkJAx468MpX4+loQH+9m/7e0WdHX05n6azs2QRV57sYagoCUO9ixEpq6RhZuOAbwLnAicSnMnvxHijkoLKlSgg2n/+pMrE3tEBt97an/zqyZ0d6mu6NEY1UiMdhgLVLkaprJIGMA/Y7u7PufsbwA+Ai2KOScaqEhNFBC1cRy2vDWir5TVaDn0xdz1FlfKB8v27iUK9i1Ert6QxFXg+dLsr3TaAmTWbWZuZtXV3d5csOBmBuBOFpc8mXFcXXMzeuh6+v4iaWM9qlpGiHaOPFO2sZhlNrB9yqKvVFtMw7nlqrI8Ga6d10merI5lkT3j45CdH/u9GvYuxc/eyuQCXAN8J3V4CfGOox8ydO9clIdatc0+l3MHdLPhZiMv48e51dcFz1tUNfz2VCmKJEmuU5yz0+xniso5FXsurA5predXXsTi4MdL3mUS5fvdj+R1nHleuv4+YAG2e63M4V2NSL8BpwH2h2yuAFUM9RkkjIdatc6+tHduHZuY/fxI/GAv9QZfnkmJnzrtS7Cy/31lYsb5QJPG9lolKSRqHAc8B04HDgUeBGUM9RkkjZuEPgzL/lhjOC5HDyX7QlVeO6cPROJT718Sh0f1es5NJJr7R9M6i/A5K1VOrrY3930u5q4ikEbwPzgN+D+wAVg53vJJGDMb6rTFBiSIjV0dpzJ9Lo+idjLinUchLrkSToKG7JP67KWcVkzRGelHSKLHRDkMl/D98vs5SKlWEFxsimeSvaSwqzQfzGC/rWOQpdrpxyFPsHFvc4VpWQv/dlDMlDSmu0QxDJTxRhOX7smxWwiDSv+N1LPZUTWfog3dx7MkgasIYc8Iro38z5U5JQ4pnhL2LdSzy1Ljn3egrm//7Je1pjFSJivBjvYxoaK1cCvgVTElDCm8UvYt14z/htYe/OfDbZhnULItS0yi2hCWTYYv46kUkipKGFNZIehehD4NU3f7kfmMfxqhmTyVRvtlM+WZPFSLRmA3d0yjrX2hlypc0tMutjEx4B9EoUqkBu7zW1AQfFdnMoC/3/n2SBEPtMjzc9fROv600DTphnjaXTa58u9zqdK8SXQFOk1lfnzvf1NcXKEYpjqamMX+yZx6dyT1D7BovCVZue09JHAp4IpuWliCfhNXWBu1S+Zqagm2f+vq0/VO5Uk9DhlaA3kVY5i592xQpT6ppyNDynV4uW1btQkTKW8WcuU9KILwF9XAJQ1tNi1QVDU/JQCMZjlLvQqTqKGnIQCtXDp8wNE9SpGppeEoC6SGp1o730MBOajhEAztpZdFbx5jpNJkiVU49DekfkmrtvYhm1tDLkQB00EAzawBoSv0mqFuISFVTT6OaZa2/WMmN/Qkjo5cjWWn/WwspRARQT6N65Sh4d5J7WXanH6vhKBEBEtjTMLObzOxpM3vMzDaY2dvT7Q1m9icz25a+3BZzqOUtR8G7ns6ch9anrBQRiUgZSFzSAO4HTnL3kwlO67oidN8Od5+Vvnw6nvDKXGZIKsf6ixauo5bXBrRpiw8RCRs2aZjZcjM7uhTBALj7Rnc/mL75EDCtVK9d8TJDUnkW7DWxntUsIzWuC8M1UUpEBonS03gH8Fszu9PMFppZKccqPgncG7o93cy2mtkvzGx+vgeZWbOZtZlZW3d3d/GjLBfDrcGoraVp3fm0H5xGn5sWeYvIIJH2nkonirOBvwEagTuBf3H3HaN6UbNNBMko20p3vyd9zMr0a13s7m5mRwBHuXuPmc0F/g2Y4e6vDPVa2nsqJN/JLECru0VkgDGdTyP9of0H4A/AQeBo4G4zu9/dvzjSYNx9wTDBLgUuAD6QPoMU7n4AOJC+vsXMdgDHA8oIw8mcQGeohKE1GCISwbBJw8w+CywF9gLfAa5x9zfNrAZ4Fhhx0hjm9RYCXwLOcPfeUPtkYJ+7HzKzvwSOA54r5GtXpOH2klKlW0RGIEpPYxLBENGA6qm795nZBUWI6Z+BI4D70+WTh9Izpf4auMHMDgKHgE+7+74ivH5lGaqOoSEpERkhnU+jUg13Lm+dlFtEhqBzhFeTKNub66TcIjIKSVzcJ2MVYWqt6hgiMhpKGpWoM/d2IIC2NheRMVHSqCSZLUKGm1qrhCEio6SaRqXQ1FoRKQH1NCrFcFNrNSQlIgWgnkalyFfHMNNqbxEpGPU0yt1wdQxNrRWRAlJPo5ypjiEiJaaeRjlTHUNESkw9jXKmOoaIlJh6GuVIdQwRiYl6GuVGdQwRiZF6GmWktRUalp5BTe9+GthJK4sGHqA6hogUmXoaZaK/g3FoGgAdNNDMGgCaWK86hoiUhHoaZSLXRKlejmQlNwY3VMcQkRJIXNIws+vNbJeZbUtfzgvdt8LMtpvZM2Z2TpxxllpnR+6idyf1qmOISMkkdXjqZnf/arjBzE4EPgbMAP4nsMnMjnf3Q3EEWFKtrdTbfDp8cG+iftxu1TFEpGQS19MYwkXAD9z9gLvvBLYD82KOqTRWrqTFr6WW1wY011ovLXdMU8IQkZJJatJYbmaPmdntZnZ0um0q8HzomK502yBm1mxmbWbW1t3dXexYi6+zkybWs5plpGjH6CNFO6t9mfKFiJRULEnDzDaZ2RM5LhcBtwLvBGYBe4CvZR6W46lyDvS7+2p3b3T3xsmTJxfjLZRG1iK+JtbTznT6GEc702lK/Tre+ESk6sRS03D3BVGOM7M1wM/SN7uAY0N3TwN2Fzi05NAiPhFJoMQNT5nZlNDNDwNPpK//BPiYmR1hZtOB44D/LnV8JaPNCEUkgZI4e+r/mtksgqGnduAKAHd/0szuBH4HHASuquiZU9qMUEQSKHFJw92XDHFfC1AdYzL19dDRkbtdRCQmiRueqnqZ4ndHR9CrCFMdQ0RipqSRJJnid6aH4f5W4lAdQ0QSIHHDU1UtV/HbPUgYqmOISAKop5Ek+Yrf+dpFREpMSSNJ8hW5VfwWkYRQ0kgCFb9FpEwoacRNxW8RKSMqhMdNxW8RKSPqacRNxW8RKSNKGnFT8VtEyoiSRtxaWoJid5iK3yKSUEoaccnMmFqyBN72NqirCwrgKn6LSIKpEB6H7HNl9PQEvYu1a5UsRCTR1NOIQ64ZU729QbuISIIpacRBM6ZEpEwlLmmY2Q/NbFv60m5m29LtDWb2p9B9t8Uc6uhpxpSIlKnE1TTc/dLMdTP7GvBy6O4d7j6r5EEVWkvL4PN/a8aUiJSBxPU0MszMgI8C6+OOpSAys6VqaoLaxdKlwUwpzZgSkTKSuJ5GyHzgBXd/NtQ23cy2Aq8Af+/uv8z1QDNrBpoB6pMw5JM9W6qjA+64Q4lCRMqOuXvpX9RsE/COHHetdPd70sfcCmx396+lbx8BHOXuPWY2F/g3YIa7vzLUazU2NnpbW1tB4x+xzA622bS/lIgklJltcffG7PZYehruvmCo+83sMOBiYG7oMQeAA+nrW8xsB3A8EHNGiECzpUSkQiS1prEAeNrduzINZjbZzMalr/8lcBzwXEzxjYxmS4lIhUhq0vgYgwvgfw08ZmaPAncDn3b3fSWPbDS0v5SIVIhEFsLd/RM52n4E/Kj00YxBa2swU6qzE445Jthjat++oIfR0qIiuIiUnUQmjYqg/aVEpAIldXiq/Gl/KRGpQEoaxaIZUyJSgZQ0ikUzpkSkAilpFItmTIlIBVLSKDSdkU9EKphmTxWSZkyJSIVTT6OQ0jOmWllEAzup4RANvU/S+rmH445MRKQglDQKqbOTVhbRzBo6aMCpoYMGmnv+kdbWuIMTERk7JY1Cqq9nJTfSy5EDmns5UsszRKQiKGkUUksLneSeUqvlGSJSCZQ0CiE0Y6q+ZlfOQ7Q8Q0QqgZLGWGVmTHV0gDstfV+iltcGHKLlGSJSKZQ0xiprj6km1rOaZaTGdWl5hohUHK3TGKscxYom1tPU9wPo64shIBGR4lFPY6y0x5SIVJFYkoaZXWJmT5pZn5k1Zt23wsy2m9kzZnZOqH2umT2evu8WM7PSR56D9pgSkSoSV0/jCeBi4L/CjWZ2IsGpXmcAC4FvZc4LDtwKNBOcG/y49P3xa2oKihaplPaYEpGKF0vScPen3P2ZHHddBPzA3Q+4+05gOzDPzKYAf+buD7q7A/8KfKh0EeeQmWZbUxMUw1taghpGe7sShohUrKQVwqcCD4Vud6Xb3kxfz27PycyaCXol1BejtpC9MWFHR3AblDBEpKIVradhZpvM7Ikcl4uGeliONh+iPSd3X+3uje7eOHny5JGGPjydylVEqlTRehruvmAUD+sCjg3dngbsTrdPy9EeD53KVUSqVNKm3P4E+JiZHWFm0wkK3v/t7nuA/WZ2anrW1MeBe2KLUtNsRaRKxTXl9sNm1gWcBvy7md0H4O5PAncCvwP+A7jK3Q+lH3Yl8B2C4vgO4N6SB56habYiUqUsmIxUuRobG72tra3wT9zaGtQwOjuDHkZLi4rgIlIxzGyLuzdmtydt9lT5aGpSkhCRqpO0mkayhddmNDSg0/GJSLVRTyMqrc0QEVFPIzKtzRARUdKITGszRESUNCLT2gwRESWNyLQ2Q0RESSMybYEuIqLZU8PSIj4RkX5KGkPRNFsRkQE0PDUUTbMVERlASWMommYrIjKAksZQNM1WRGQAJY2haJqtiMgAShpD0TRbEZEBNHtqONoCXUSkX1xn7rvEzJ40sz4zawy1n2VmW8zs8fTP94fu22xmz5jZtvTlz4sWoLZAFxHJKa6exhPAxcC3s9r3Ahe6+24zOwm4D5gaur/J3YtwGr4Qrc0QEckrlp6Guz/l7s/kaN/q7rvTN58EJpjZESUNTmszRETySnIh/H8BW939QKjtu+mhqS+bmeV7oJk1m1mbmbV1d3eP7FW1NkNEJK+iJQ0z22RmT+S4XBThsTOA/wNcEWpucveZwPz0ZUm+x7v7andvdPfGyZMnjyxwrc0QEcmraDUNd18wmseZ2TRgA/Bxd98Rer5d6Z/7zez7wDzgXwsR6wAtLQNrGqC1GSIiaYkanjKztwP/Dqxw91+H2g8zs0np6+OBCwiK6YWntRkiInmZu5f+Rc0+DHwDmAy8BGxz93PM7O+BFcCzocPPBl4D/gsYD4wDNgFfcPdDw71WY2Ojt7UVd8KViEilMbMt7t44qD2OpFFKShoiIiOXL2kkanhKRESSTUlDREQiU9IQEZHIlDRERCSyii+Em1k30BF3HCM0iWAfrmqi91wd9J7LR8rdB62OrvikUY7MrC3XrIVKpvdcHfSey5+Gp0REJDIlDRERiUxJI5lWxx1ADPSeq4Pec5lTTUNERCJTT0NERCJT0hARkciUNBLOzK42M89sDV/JzOwmM3vazB4zsw3prfIrkpktNLNnzGy7mV0bdzzFZmbHmtkDZvaUmT1pZp+LO6ZSMLNxZrbVzH4WdyyFoqSRYGZ2LHAWUC3nmr0fOMndTwZ+T7BNfsUxs3HAN4FzgROBRWZ2YrxRFd1B4O/c/QTgVOCqKnjPAJ8Dnoo7iEJS0ki2m4EvAlUxW8HdN7r7wfTNh4BpccZTRPOA7e7+nLu/AfwAGPY0yOXM3fe4+yPp6/sJPkinxhtVcaXPQno+8J24YykkJY2EMrMPArvc/dG4Y4nJJ4F74w6iSKYCz4dud1HhH6BhZtYAzAYejjmUYvs6wZe+vpjjKKiinSNchmdmm4B35LhrJXAdwVkLK8pQ79nd70kfs5JgOKO1lLGVkOVoq4repJkdBfwI+Ly7vxJ3PMViZhcAL7r7FjN7X8zhFJSSRozcfUGudjObCUwHHjUzCIZpHjGzee7+hxKGWHD53nOGmS0lOAf8B7xyFxF1AceGbk8DdscUS8mY2XiChNHq7j+OO54iOx34oJmdB0wA/szM1rn7ZTHHNWZa3FcGzKwdaHT3ctwpMzIzWwj8E3CGu3fHHU+xmNlhBIX+DwC7gN8Ci939yVgDKyILvv3cAexz98/HHE5JpXsaV7v7BTGHUhCqaUiS/DMwEbjfzLaZ2W1xB1QM6WL/cuA+goLwnZWcMNJOB5YA70//bbelv4VLmVFPQ0REIlNPQ0REIlPSEBGRyJQ0REQkMiUNERGJTElDREQiU9IQEZHIlDRERCQyJQ2REjKzd6fPFzLBzI5Mn1vipLjjEolKi/tESszM/oFgP6K3AV3u/o8xhyQSmZKGSImZ2eEE+029DrzH3Q/FHJJIZBqeEim9Y4CjCPbZmhBzLCIjop6GSImZ2U8IztY3HZji7stjDkkkMp1PQ6SEzOzjwEF3/376XOG/MbP3u/v/izs2kSjU0xARkchU0xARkciUNEREJDIlDRERiUxJQ0REIlPSEBGRyJQ0REQkMiUNERGJ7P8D4L4OV0e0FhEAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<Figure size 432x288 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"from typing import Tuple\n",
|
|
"\n",
|
|
"# Load data\n",
|
|
"\n",
|
|
"training_data = np.load('training_data.npy')\n",
|
|
"test_data = np.load('test_data.npy')\n",
|
|
"\n",
|
|
"test_data_x = test_data[:, 0]\n",
|
|
"test_data_y = test_data[:, 1]\n",
|
|
"\n",
|
|
"training_data_x = training_data[:, 0]\n",
|
|
"training_data_y = training_data[:, 1]\n",
|
|
"\n",
|
|
"# Visualize data\n",
|
|
"plt.plot(test_data_x, test_data_y, 'or')\n",
|
|
"plt.plot(training_data_x, training_data_y, 'ob')\n",
|
|
"plt.xlabel('x')\n",
|
|
"plt.ylabel('y')\n",
|
|
"plt.legend([\"test_data\", \"training data\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"As in the lecture notebook, we will use polynomial-features here again.\n",
|
|
"The following functions will be used for:\n",
|
|
"- calculating polynomial features\n",
|
|
"- computing the mean and std of the features (training data) as normalizer\n",
|
|
"- normalize other data (test) features using the normalizer (mean and std)\n",
|
|
"- evaluating the model\n",
|
|
"- calculating the Mean Squarred Error for assigning a performance to each\n",
|
|
"model. <br><br>\n",
|
|
"\n",
|
|
"Note we will use the mean and the standard deviation to normalize our features\n",
|
|
"according to:\n",
|
|
"\\begin{align*}\n",
|
|
" \\boldsymbol{\\tilde{\\Phi}} = \\frac{\\boldsymbol{\\Phi}(\\boldsymbol{x}) - \\boldsymbol{\\mu}_{\\Phi}}{\\boldsymbol{\\sigma}_{\\Phi}}, \n",
|
|
"\\end{align*}\n",
|
|
"where $\\boldsymbol{\\tilde{\\Phi}}$ are the (approximately) normalized features to any input\n",
|
|
"$\\boldsymbol{x}$ (not necessarily the training data), $\\boldsymbol{\\mu}_{\\Phi}$ is the mean of the features applied to the training data and $\\boldsymbol{\\sigma}_{\\Phi}$ is the standard deviation of the features applied to the training data for each dimension.<br>\n",
|
|
"\n",
|
|
"Normalization is a standard technique used in Regression to avoid numerical problems and to obtain better fits for the weight vectors $\\boldsymbol{w}$. Especially when the features transform the inputs to a very high value range, normalization is very useful. In this homework we will use features of degree 10. Since the input range of the data is roughly from -4 to 4 this will lead to very high values for higher order degrees. By normalizing each dimension of the feature matrix, we will map each dimension of the feature matrix applied to the training data to a zero mean unit variance distribution."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def get_polynomial_features(data: np.ndarray,\n",
|
|
" degree: int) ->np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" Function to create Feature Matrix. Extends the feature matrix according to\n",
|
|
" the matrix form discussed in the lectures.\n",
|
|
"\n",
|
|
" :param data: data points you want to evaluate the polynomials,\n",
|
|
" shape: [n_samples] (we have 1-dim data)\n",
|
|
" :param degree: degree of your polynomial, shape: scalar\n",
|
|
" :return polynomial_features: shape [n_samples x (degree+1)]\n",
|
|
" \"\"\"\n",
|
|
" polynomial_features = np.ones(data.shape)\n",
|
|
" for i in range(degree):\n",
|
|
" polynomial_features = np.column_stack((polynomial_features, data ** (i + 1)))\n",
|
|
" return polynomial_features\n",
|
|
"\n",
|
|
"\n",
|
|
"def get_mean_std_features(polynomial_features: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:\n",
|
|
" \"\"\"\n",
|
|
" Function for calculating the mean and standard deviation of the features\n",
|
|
" :param polynomial_features: shape: [n_samples x (degree+1)]\n",
|
|
" :return mean_feat: mean vector of the features,\n",
|
|
" shape:[1 x (degrees+1)]\n",
|
|
" :return std_feat: standard deviation (for each dimension in feature matrix),\n",
|
|
" shape: [1 x (degrees+1)] \n",
|
|
" \"\"\"\n",
|
|
" mean_feat = np.mean(polynomial_features, axis=0, keepdims=True)\n",
|
|
" mean_feat[:, 0] = 0.0 # we don't want to normalize the bias\n",
|
|
" std_feat = np.std(polynomial_features, axis=0, keepdims=True)\n",
|
|
" std_feat[:, 0] = 1.0 # we don't want to normalize the bias\n",
|
|
" return mean_feat, std_feat\n",
|
|
"\n",
|
|
"\n",
|
|
"def normalize_features(polynomial_features: np.ndarray,\n",
|
|
" mean_train_features: np.ndarray,\n",
|
|
" std_train_features: np.ndarray) ->np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" Normalize features\n",
|
|
" :param polynomial_features: features to be normalized,\n",
|
|
" shape: [n_samples x (degree+1)]\n",
|
|
" :param mean_train_features: mean of the feature matrix of the training set,\n",
|
|
" shape: [1 x (degrees+1)]\n",
|
|
" :param std_train_features: std of the feature matrix of the training set,\n",
|
|
" shape: [1 x (degrees+1)]\n",
|
|
" :return norm_feat: normalized features, shape: [n_samples x (degree+1)]\n",
|
|
" \"\"\"\n",
|
|
"\n",
|
|
" # note: features: (n_samples x n_dims),\n",
|
|
" # mean_train_features: (1 x n_dims),\n",
|
|
" # std_train_features: (1 x n_dims)\n",
|
|
" # due to these dimensionalities we can do element-wise operations.\n",
|
|
" # By this we normalize each dimension independently\n",
|
|
" norm_feat = (polynomial_features - mean_train_features) / std_train_features\n",
|
|
" return norm_feat\n",
|
|
"\n",
|
|
"\n",
|
|
"def eval(Phi:np.ndarray, w:np.ndarray)->np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" Evaluate the models\n",
|
|
"\n",
|
|
" :param Phi: Feature matrix, shape: [n_samples x (degree+1)]\n",
|
|
" :param w: weight vector, shape: [degree + 1]\n",
|
|
" :return : predictions, shape [n_samples] (we have 1-dim data)\n",
|
|
" Evaluates your model\n",
|
|
" \"\"\"\n",
|
|
" return np.dot(Phi, w)\n",
|
|
"\n",
|
|
"\n",
|
|
"def mse(y_target:np.ndarray, y_pred:np.ndarray)->np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" :param y_target: the target outputs,\n",
|
|
" shape: [n_samples] (here 1-dim data)\n",
|
|
" :param y_pred: the predicted outputs,\n",
|
|
" shape: [n_samples](we have 1-dim data)\n",
|
|
" :return : The Mean Squared Error, shape: scalar\n",
|
|
" \"\"\"\n",
|
|
" diff = y_target - y_pred\n",
|
|
" return np.sum(diff ** 2, axis=0) / y_pred.shape[0]\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 1.3) Implement Ridge Regression Weights (2 Point)\n",
|
|
"The following function will calculate the weights for ridge regression. Fill in the missing code according to the formula for calculating the weight updates for ridge regression. <br>\n",
|
|
"Recall that the formula is given by \n",
|
|
"\\begin{align*}\n",
|
|
" \\boldsymbol{w} &= (\\boldsymbol{\\Phi} ^T \\boldsymbol{\\Phi} + \\lambda \\boldsymbol{I} )^{-1} \\boldsymbol{\\Phi}^T \\boldsymbol{y}\\\\\n",
|
|
" (\\boldsymbol{\\Phi} ^T \\boldsymbol{\\Phi} + \\lambda \\boldsymbol{I}) \\boldsymbol{w} &= \\boldsymbol{\\Phi}^T \\boldsymbol{y},\n",
|
|
"\\end{align*}\n",
|
|
"where $\\boldsymbol{\\Phi}$ is the feature matrix (the matrix storing the data points applied to the polynomial features).\n",
|
|
"Hint: use np.linalg.solve for solving for the linear equation.\n",
|
|
"If you got confused because of the normalization described before, don't worry, you do not need to consider it here :)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def calc_weights_ridge(Phi:np.ndarray,\n",
|
|
" y:np.ndarray,\n",
|
|
" ridge_factor:float)->np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" :param Phi: Feature Matrix, shape: [n_samples x (degree+1)]\n",
|
|
" :param y: Output Values, [n_samples] (we have 1-dim data)\n",
|
|
" :param ridge_factor: lambda value, shape: scalar\n",
|
|
" :return : The weight vector, calculated according to the equation shown before,\n",
|
|
" shape: [degrees +1]\n",
|
|
" \"\"\"\n",
|
|
" identity_matrix = np.identity(Phi.shape[1])\n",
|
|
" #for Ax=b\n",
|
|
" A = (Phi.transpose().dot(Phi) + ridge_factor * identity_matrix)\n",
|
|
" b = (Phi.transpose().dot(y))\n",
|
|
" weights = np.linalg.solve(A,b)\n",
|
|
" return weights"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"For demonstrating ridge regression we will pick the polynomial degree of 10. In the lecture notebook we have seen that this model is highly overfitting to the data.\n",
|
|
"We will investigate the role of the ridge factor $\\lambda$. For that purpose we first need to calculate the weights for different $\\lambda$ values. <br>\n",
|
|
"We will pick $\\lambda = [1e-{6}, 1e-{3}, 1, 3, 5,10,20,30,40,50, 1e2, 1e3, 1e5] $ to see the differences of the values. <br><br>\n",
|
|
"\n",
|
|
"Practical note. We use here very high values for $\\lambda$ for demonstration\n",
|
|
"purposes here. In practice we would not choose a model where we know from\n",
|
|
"beginning that it is highly overfitting. When choosing an appropriate model, the value needed for $\\lambda$ automatically will be small (often in the range of $1e^{-6}$ or smaller)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Let's do it on polynomial degree 10 and see the results\n",
|
|
"\n",
|
|
"# first we get the mean and the standard deviation of the training feature matrix, which we will use for normalization\n",
|
|
"train_features = get_polynomial_features(training_data_x, 10)\n",
|
|
"test_features = get_polynomial_features(test_data_x, 10)\n",
|
|
"mean_train_feat, std_train_feat = get_mean_std_features(train_features)\n",
|
|
"norm_train_features = normalize_features(train_features, mean_train_feat, std_train_feat)\n",
|
|
"norm_test_features = normalize_features(test_features, mean_train_feat, std_train_feat)\n",
|
|
"\n",
|
|
"\n",
|
|
"# now we can calculate the normalized features for degree 10\n",
|
|
"ridge_factors = [1e-6, 1e-3, 1, 3, 5, 10,20,30,40, 50, 1e2, 1e3, 1e5]\n",
|
|
"weights_ridge = []\n",
|
|
"\n",
|
|
"for lambda_val in ridge_factors:\n",
|
|
" weights_ridge.append(calc_weights_ridge(norm_train_features, training_data_y, lambda_val))\n",
|
|
"\n",
|
|
"# We further have to perform the predictions based on the models we have calculated\n",
|
|
"y_training_ridge = []\n",
|
|
"y_test_ridge = []\n",
|
|
"\n",
|
|
"for w in weights_ridge:\n",
|
|
" y_training_ridge.append(eval(norm_train_features, w))\n",
|
|
" y_test_ridge.append(eval(norm_test_features, w))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We are interested in the mean squarred error on the test and the training data. For that purpose we calculate them here and plot the errors for different $\\lambda$ values in log space. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<matplotlib.legend.Legend at 0x7f0dc1188520>"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEaCAYAAAD+E0veAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA4G0lEQVR4nO3dd3hUZfbA8e8hVKkCQSlKEaRD0IgrKBJsdFBRQeyuXVAUUFfZ9WcXxYIiLiiyuyJlYVlBcBUVRFgLQVFAQFpYAggBJICAQPL+/jgzYRLTM5M7Nzmf55mHmTsz9z2TYe65b7nvK845jDHGGIAyXgdgjDEmelhSMMYYk8GSgjHGmAyWFIwxxmSwpGCMMSaDJQVjjDEZLCkYXxKRySLyVDGVdbqIHBSRmByef1xE3i2OWPxERN4UkVFex2EKxpKCyZaIJInI4cDBMHh7vZhjWCQiRwJl7xaRf4lI3ULsx4lI01yev0lE0gLl7BeR70Wkd/B559z/nHNVnHNphf0shSEiXUUkPRDXARFZJyI3F2cMReGcu9M596TXcZiCsaRgctMncDAM3u7N7kUiUjabbdmeVeckl9ff65yrApwJ1ABeLsh+C+DLQDk1gDeAaSJSI0JlFcT2QFzVgGHARBFpHu5CsvsOTelkScEUWODMeqmIvCwie4HHA80540Vkvoj8CiSISMvA2f4+EVktIn1D9vG71+dWpnNuLzALaJNDTLeJyAYR2Ssic0SkXmD74sBLvg+ccV+TRznpwD+AykCzwD4aBWobZQOPG4vI54Gz9wVA7Syx3CAiW0Rkj4iMCtS6Lg48V0ZEHhaRjYHnZ4hIzdxiCsTlnHPzgb1Au/zsK484HheRmSLyrojsB24Skeoi8raI7BCRbSLyVDBZi0jTwGdODdTapge2S+D/wa7Acz+ISJvAc5ma+HL6jgLPORG5U0TWi8gvIjJORCSvv4sJP0sKprDOBTYBdYCnA9uuDdyvCnwNzAU+DrxmCDAly1lu6OuX5FaYiNQGrgS+y+a5bsCzwNVAXWALMA3AOdcl8LL2gdrO9DzKiQFuBo4F9pOd94DlaDJ4Ergx5P2t0JrG4EAs1YH6Ie8dCvQHLgTqAb8A43KLKbDfMoGkWhvYkNe+8hEHQD9gJlo7mgL8DTgONAU6AJcCfwy89kn0uzwZaAC8Fth+KdCFEzW5a4A92cSf43cUojdwDtA+8LrL8vizmEhwztnNbr+7AUnAQWBfyO22wHM3Af/L8vrJwN9DHl8A/AyUCdk2FXg8u9fnEMMi4FCg7G3ogSs25P1PBe6/DYwOeV8V9KDeKPDYAU1zKecm9GC4L/C+w8DVIc83CuyjLHB64LWVQ55/D3g3cP/PwNSQ504CjgIXBx6vAS4Keb5uoMyy2cTVFUgPxPUbkAbcH/J8jvvKRxyPA4tDnj8lUEalkG2DgIWB+38HJgANssTYDfgJ+EPod13I7+j8kOdnAA97/TsojTerKZjc9HfO1Qi5TQx5bms2rw/dVg/Y6rQ5JmgLmc9Ws9tHVkMDZdd3zg12zqVk85p6hJzVO+cOomerWc+Mc/OVc64GeiY8B01q2akH/OKc+zVk25Ysz2d8LufcITKfOTcEZgea1PahB/Y09KCcne2BuKoBY9GDcH72lVcckPnv3xAoB+wI2d9f0VoewEhAgG8CTYG3BPb7GfA6WkPZKSITRKRaNp8jP9/RzyH3D6GJwxQzSwqmsLKbXjd023bgNBEJ/T92OnrGn9s+CmM7elADQEQqA7WylJUvgYPV3cD1ItIhm5fsAE4OlBF0epbnG4TEUikQS9BWoEeWZFvROZdrrM6534CHgLYi0j8f+8orDsj899+K1hRqh+yrmnOudaD8n51ztznn6gF3AG9IYESXc26sc+5soDXajDQim48Qtu/IRJYlBRMpXwO/AiNFpJyIdAX68Pt25HB4D7hZROJEpALwDPC1cy4p8PxOoEl+d+ac2wO8hTbBZH1uC5AI/J+IlBeR89HPFTQT6CMinUSkPPB/6Bl20JvA0yLSEEBEYkWkXz7jOgqMCYkrt33lFUfWfe9A+wzGiEi1QB/GGSJyYWDfV4lIMMn8giaUNBE5R0TOFZFy6Pd9BK2tZJXXd2SihCUFk5u5kvk6hdn5fWPgANYX6AHsRjs9b3DOrQ13kM65T4FR6OikHcAZwMCQlzwO/C3QLHJ1Pnf7CtBTRNpl89y1aEf7XuAvaHt7MJbVaKf6tEAsB4Bd6Fk4wKto89THInIA+Cqwr/yaBJwuIn1y21c+4sjODUB54Ef0wD8T7acA7QD+WkQOBsq8zzm3GW3Wmhh4/Ra0SejFrDvOx3dkooQ4Z4vsGBMpIlIF7ShuFjiIluo4TPSzmoIxYSYifUTkpEC7+YvASnQ0V6mMw/iLJQVjwq8f2rG6Hb0AbqDzpkoeLXEYH7HmI2OMMRmspmCMMSaDryfBql27tmvUqJHXYRhjjK8sX758t3MuNrvnfJkUAsPx+jRt2pTExESvwzHGGF8RkZzm9fJn85Fzbq5z7vbq1at7HYoxxpQovkwKgaF2E1JTU70OxRhjShRfJgWrKRhjTGT4vk/BGOOtY8eOkZyczJEjR7wOxWRRsWJFGjRoQLly5fL9Hl9fpxAfH++so9kYb23evJmqVatSq1YtbLG06OGcY8+ePRw4cIDGjRtnek5Eljvn4rN7ny+bj4wx0ePIkSOWEKKQiFCrVq0C1+B8mRSso9mY6GIJIToV5nvxZVIockfzd9/BY4+FNyhjjCkBfJkUiuyTT+Dpp+E///E6EmNMEe3Zs4e4uDji4uI49dRTqV+/fsbjo0eP5vrexMREhg4dmmcZnTp1CkusixYtonr16hnxxcXF8cknn4Rl3+Hiy47mkNFHt61fv77gO/jtN2jTBsqWhR9+gAL0zBtjMluzZg0tW7b0OgwAHn/8capUqcLw4cMzth0/fpyyZaNjoOWiRYt48cUX+eCDD3J8jXMO5xxlypTJ9nFO0tLSiImJ+d327L6fEtfRXOTmowoV4OWXYe1aeP318AZnjPHcTTfdxAMPPEBCQgIPPfQQ33zzDZ06daJDhw506tSJdevWAXqQ7t27N6AJ5ZZbbqFr1640adKEsWPHZuyvSpUqGa/v2rUrAwYMoEWLFgwePJjgifX8+fNp0aIF559/PkOHDs3Yb34kJSXRsmVL7r77bs466yy++OKLTI+3bt3KiBEjaNOmDW3btmX69OkZ8SQkJHDttdfStm3bsPztoiN9eqFXL+jeHR5/HAYPhjp1vI7IGN+7/35YsSK8+4yLg1deKfj7fvrpJz755BNiYmLYv38/ixcvpmzZsnzyySf86U9/YtasWb97z9q1a1m4cCEHDhygefPm3HXXXb8b4//dd9+xevVq6tWrR+fOnVm6dCnx8fHccccdLF68mMaNGzNo0KAc4/riiy+Ii4vLeDxr1ixiYmJYt24d77zzDm+88QZJSUmZHs+aNYsVK1bw/fffs3v3bs455xy6dOkCwDfffMOqVat+N+y0sEpvUhDR2kLbttrpPGGC1xEZY8LoqquuymhOSU1N5cYbb2T9+vWICMeOHcv2Pb169aJChQpUqFCBOnXqsHPnTho0aJDpNR07dszYFhcXR1JSElWqVKFJkyYZB+ZBgwYxIYdjygUXXPC75qOkpCQaNmzIH/7wh4xtoY+XLFnCoEGDiImJ4ZRTTuHCCy9k2bJlVKtWjY4dO4YtIYBPk0LYrmhu0QKGDNHTkDvvhLPOCkd4xpRahTmjj5TKlStn3B81ahQJCQnMnj2bpKQkunbtmu17KlSokHE/JiaG48eP5+s14eibDY036+Pc9p/1fUVVOvsUQv35z1C7NgwdCj7sdDfG5C01NZX69esDMHny5LDvv0WLFmzatImkpCSAjDb/cOnSpQvTp08nLS2NlJQUFi9eTMeOHcNaRpAvk0JY1agBzzwDS5dCmL9IY0x0GDlyJI888gidO3cmLS0t7PuvVKkSb7zxBt27d+f888/nlFNOIaeT1mCfQvA2c+bMPPd/+eWX065dO9q3b0+3bt0YPXo0p556arg/BuDTIalBYZv7KC0NzjkHUlJ0RFKYq2PGlGTRNCTVSwcPHqRKlSo457jnnnto1qwZw4YN8zqs0jEkNexiYmDsWEhOhuef9zoaY4wPTZw4kbi4OFq3bk1qaip33HGH1yEVii9rCkW+eC0n114Ls2fDmjVgaz8bky9WU4hupaKmELFFdp5/XoeqjhgR3v0aY4xP+DIpRMxpp8Ejj8DMmbBwodfRGGNMsbOkkNXw4dCwIdx3H2QzRtkYY0oySwpZVaoEL74IK1fCxIleR2OMMcXKkkJ2rrwSunbV6S/27vU6GmNMLooydTbopHL//e9/s31u8uTJxMbGZrqu4Mcffwz3R4gqvpzmIuJE4NVXoUMH+Mtf4LXXvI7IGJODWrVqsSIwC192U2fnZdGiRVSpUiXHNROuueYaXs9lNuWsU1bnNIV1VtE0pXcoqynkpF07nQ9p/HhYtcrraIwxBbB8+XIuvPBCzj77bC677DJ27NgBwNixY2nVqhXt2rVj4MCBJCUl8eabb/Lyyy8TFxfHF198ka/9Z52yOuvjI0eOcPPNN9O2bVs6dOjAwsDAlcmTJ3PVVVfRp08fLr300oh9/qKIvjSVD2GbEC8vTzwBU6dqp/Mnn2gNwhiTsyiYO9s5x5AhQ3j//feJjY1l+vTpPProo0yaNInnnnuOzZs3U6FCBfbt20eNGjW48847c61dTJ8+nSVLlmQ8/vLLL4HMU1YvWrQo0+MxY8YAsHLlStauXcull17KTz/9lPH+H374gZo1axbu7xFhvqwpROw6haxq1dLE8Nln8O9/R7YsY0xY/Pbbb6xatYpLLrmEuLg4nnrqKZKTkwFo164dgwcP5t133813080111zDihUrMm6VKlUC+N2U1aGPlyxZwvXXXw/oZHkNGzbMSAqXXHJJ1CYE8GlNoVjdeSe8+SY8+CD06AEVK3odkTHRKwrmznbO0bp164wz+lDz5s1j8eLFzJkzhyeffJLVq1cXupxomeo63HxZUyhWZctqp/PmzfDSS15HY4zJQ4UKFUhJSclICseOHWP16tWkp6ezdetWEhISGD16NPv27ePgwYNUrVqVAwcOhDWGLl26MGXKFEBXgPvf//5H8+bNw1pGpFhSyI+LLoLLL9cptrdt8zoaY0wuypQpw8yZM3nooYdo3749cXFx/Pe//yUtLY3rrrsuo/N32LBh1KhRgz59+jB79uwcO5qnT5+eaUhqTsNXQ919992kpaXRtm1brrnmGiZPnpxpcZ5o5ssJ8YLCNnV2fmzeDC1bwoAB8O67xVOmMT5gE+JFt1IxIZ4nGjfWKTCmTIF8nCkYY4wfWVIoiIcfhnr1dOnO9HSvozHGmLCzpFAQVarA6NGwfDlEYJ1XY4zxWlQlBRGpLCLLRaS317Hk6NproVMnnWI7NdXraIwxJqwimhREZJKI7BKRVVm2dxeRdSKyQUQeDnnqIWBGJGMqsuC8SCkp8NRTXkdjjDFhFemawmSge+gGEYkBxgE9gFbAIBFpJSIXAz8COyMcU9HFx8PNN2tyCFylaIwxJUFEk4JzbjGQde7pjsAG59wm59xRYBrQD0gA/gBcC9wmItnGJiK3i0iiiCSmpKREMPo8PPOMrr0wbJh3MRhjABCRjGklQGcgjY2NpXdvbYmeM2cOzz33XL72VdSpuMHf03F7Mc1FfWBryONk4Fzn3L0AInITsNs5l+3wHufcBGAC6HUKkQ01F6ecAn/+sw5TnT8fevb0LBRj/GL00tGcU+8cEhonZGxbuHkhy7YvY2TnkYXeb+XKlVm1ahWHDx+mUqVKLFiwgPr162c837dvX/r27ZuvfRV1Km7w93TcXnQ0ZzfVaMbB3Tk32Tn3Qa47EOkjIhNSve7oHTIEzjxTawv5PIMwpjQ7p945XD3zahZu1qmkF25eyNUzr+aceucUed89evRg3rx5AEydOpVBgwZlPDd58mTuvfdeAG666SaGDh1Kp06daNKkCTNnzszX/kvLdNxe1BSSgdNCHjcAthdkB865ucDc+Pj428IZWIGVLw8vvwy9eulCPA8+6Gk4xkS7hMYJzBgwg6tnXs1d8XcxPnE8MwbMyFRzKKyBAwfyxBNP0Lt3b3744QduueWWHA/IO3bsYMmSJaxdu5a+ffsyYMCAXPd97NixUjMdtxdJYRnQTEQaA9uAgWg/Qr4V23oK+dGzp96eeAKuu06blYwxOUponMBd8Xfx5OInGdVlVFgSAui02ElJSUydOpWeeTTn9u/fnzJlytCqVSt27sx7bMu6desypuMGbc6pW7duRrmDBw+mf//+9O/fP1+x5tR8lNd03EOGDAEiOx13pIekTgW+BJqLSLKI3OqcOw7cC3wErAFmOOcKNH9tsa2nkF8vvQSHDsGjj3odiTFRb+HmhYxPHM+oLqMYnzg+oykpHPr27cvw4cMzNR1lJ3RyuvzM/xacjju4psLKlSv5+OOPAZ2O+5577mH58uWcffbZHD9+vNDxR8N03JEefTTIOVfXOVfOOdfAOfd2YPt859yZzrkznHNPF3S/UdOnENS8ua7ONmmSXu1sjMlWsA9hxoAZPJHwREZTUrgSwy233MKf//xn2rZtG5b9BTVv3rzUTMcdVVc051fU1RQARo2C2FidF8nHM88aE0nLti/L1IcQ7GNYtn1ZWPbfoEED7rvvvrDsK1T58uVLzXTcNnV2OL39Nvzxjzq19uDBXkdjTLGwqbOjW6mYOjvqmo+Cbr4Zzj4bRo6Egwe9jsYYYwrMl0khKpuPAMqUgbFjYft2yOfVk8YYE018mRSiWqdO2nT04ouwaZPX0RhTLPzcDF2SFeZ78WVSiNrmo6DnnoOYGBgxwutIjIm4ihUrsmfPHksMUcY5x549e6hYsWKB3mcdzZHy9NPw2GOwcCF07ep1NMZEzLFjx0hOTubIkSNeh2KyqFixIg0aNKBcuXKZtufW0WxJIVKOHNF1ndu1g48+8joaY4zJUOJGH/lCxYp6QdvHH0NgxkVjjIl2vkwKUd+nEHTnnbqu8wsveB2JMcbkiy+TQtQOSc2qRg244w6YPh2SkryOxhhj8uTLpOAr99+v6zq//LLXkRhjTJ4sKURagwZ63cJbb8GePV5HY4wxubKkUByGD9eptceP9zoSY4zJlS+Tgm86moPatNGFeMaOhcOHvY7GGGNy5Muk4JuO5lAjR0JKCvztb15HYowxOfJlUvClLl2gY0edEyktzetojDEmW5YUiouI1hY2boTZs72OxhhjsmVJoTj17w9Nm8Lo0bY6mzEmKllSKE4xMToSadky+Pxzr6MxxpjfsaRQ3G64AerUsakvjDFRyZJCcatUCYYMgfnzYeVKr6MxxphMfJkUfHedQlZ33w0nnaQjkYwxJor4Min48jqFUDVrwm23wXvvwdatXkdjjDEZfJkUSoRhw3QE0iuveB2JMcZksKTglYYNYeBAmDABfvnF62iMMQawpOCtESPg4EF4802vIzHGGADKeh1Aqda+PVx2mU6UN2yYLuFpjCm9Dh/WOdJ27dJ/g7fsHr/wAlx5ZdhDsKTgtREj4OKL4d134Y9/9DoaY0w4HTqU74O8S0lBfv01290cL1OOA5XqkFo+lr1lYknhDKptiOW8CIRsScFr3brBWWdp1r/lFihjLXrG+NLRo/Daa6RNm4H7eReyexcxRw5l+9JjZcrzS7k67CkTyy4Xy47jzdh+vA4pxGbcdnHi8YH0atQoL5x8sg5erFkTHuwQmY9hScFrwYnyBg6EOXN0fiRjjH84R/q/53D4nuFU3rGBb/gD67kg0wE+eJBPLRfL8ZNjKV+rKjVrZT7I16wJDU6GdiGPg89Xr66z5BSHqEkKItISuA+oDXzqnCs9y5RdeSU0bgzPPw/9+mmiMMZEvWPLf2D39cOou+YzttCS52rPp8k9PTjtNGhSE87OctCvVCn6f94RTQoiMgnoDexyzrUJ2d4deBWIAd5yzj3nnFsD3CkiZYCJkYwr6pQtCw8+CPfeC0uXwvnnex2RMSYXBzbuYtN1o2jz1VuUpwbP1H2Nhs/cwduDy1GunNfRFU2kG7AnA91DN4hIDDAO6AG0AgaJSKvAc32BJcCnEY4r+tx8M9SqpdNqG2Oi0o6k35jf7QXSmzaj1VeT+HeDIXz/z/U8su1eBt/k/4QAEU4KzrnFwN4smzsCG5xzm5xzR4FpQL/A6+c45zoBg3Pap4jcLiKJIpKYkpISqdCL30kn6UR5c+fCmjVeR2OMCbF2jWPcxbM53LgVPReOZEPdC1j3z5VcufUVug2oGfVNQgXhxVCX+kDohD/JQH0R6SoiY0Xkr8D8nN7snJvgnIt3zsXHxsZGOtbidc892uhoE+UZExX++18YlrCCHa26cc+nV1Dp5Ipsn/Qfzt7+AW0GtPA6vIjwIilkl1Odc26Rc26oc+4O59y4XHfg91lSc1K7tg5L/cc/YPt2r6MxplRKT9eBgH067mR159sYs+gsOlZayYHnxlF31/fUu/kyr0OMKC+SQjJwWsjjBkCBjoC+nyU1Nw88AGlp8OqrXkdiTKny228waRJ0aHmEpf2e573EZtxSZjJpQ4ZRedt6qj50tw4KKeG8SArLgGYi0lhEygMDgTkF2UGJrSkANGkCV12l8yGVxM9nTJRJTdXxHY0bOebdOot5Sa14noep3DuBmDWrKTd2jF4wUEpENCmIyFTgS6C5iCSLyK3OuePAvcBHwBpghnNudUH2W6JrCqBTX+zfrzOoGmMiYts2/amddhpMe+hb/nOkK7MYQP0zK8OCBZSZ8z6ceabXYRY7cc55HUOBiUgfoE/Tpk1vW79+vdfhRMbFF+sopM2boXx5r6MxpsRYvVrHckyZArHHd/Buo8fomvQOUqsWPPUU3HpriW8mEpHlzrn47J7z5UQ7Jb6mADr1xfbtujqbMaZIjh+HTz6BPn2gTRv497QjzDz7WbaedCYJyf9AHnwQ1q+HO+4o8QkhL76sKQTFx8e7xMREr8OIDOegQwc4dgxWrrSJ8owpoN9+g08/hX/9C95/H3bvhlo1HeMvmskVX40gZusWnWvshRegaVOvwy1WJa6mUKI7moNEtMHzxx9hfo6XbRhjQhw6pEnguuugTh3o1QtmzIBLL4XPnvmKnS26cNU/ryamZg347DOYPbvUJYS8WE0hmh07pv9hGzaExYu9jsaYqLR/P8ybB7NmwYcfamKoVUvnlhzQ+wgX75lOuQnjYNkyzRRPP63TyhTXtKNRKLeaQuluPIt25crpdQv33w9ffgnnRWJJDWP8Z88evcBs1ixYsECXMjj1VLjxRp10+MLTN1N24ni4bZK+uGVLeO01fUHVql6HH9V8WVMoFaOPgg4ehNNPh65dtV5sTCm1Ywf8+9+aCBYt0ms8GzaEK67QRHDeuemUWfARjBunTa5lymifwT336O+nJE1QVES51RR8mRSCSnzzUdCoUVrlXbu2VI6bNqXXli16LjRrls5D5Bw0b65J4IordNFC+WWvXoo8fjxs2gSnnAK33663Bg28/ghRyZKC3+3apbWFG2+Ev/7V62iMiaifftIkMGsWLF+u29q3P1EjaNUqcNKfmAhvvAFTp8KRI3DBBVoruPxyu7YnD9an4Hd16sBNN8HkyfB//6eNp8aUEIcP63Wa77+viWB1YH6Dc8/VxQivuCJkgNCRI/CPGdpE9M03ULmy/jbuvhvatvXqI5QovqwplKo+haD167Xe/Mgj2pRkjE8cPKjNQElJv/83KUkrwqBdABdcoEng8st1+okMmzfrfGBvv60dxy1aaCK44QZdwNgUiDUflRQDBujVOP/7n42gMFEjNTX3g/6ePZlfX768dhA3bAiNGumtcWOd2aVOnZAXpqfDRx9pE9G8eZo1+vXTJqKEBOs4LgJrPiopRozQ+vVbb8GwYV5HY0oB52DfvhMH+NCDffD+vn2Z31Ox4omDfXy8/huaAE45JY8L9PfuhXfe0Y7jjRv1DY89Zh3HxcRqCn7TtauOsNi4kRKxIKyJGr/8ou35q1bpLXh/9+7Mr6tc+cQBPvRgH7wfG1vIk/jly7WvINhxfP75Wiu44grrOA6zQtcUROQ659y7gfudnXNLQ5671zn3enhDNXkaOVKv3Z8+Xa/lN6aAfv1VZ08JHvyDt9DF/qpWhdatdZh/ixbavBM86NesWcSWG+f0ooM1a3SY9Zo18NVXmhQqV9ZRdnffDe3aFfGTmsLItaYgIt86587Kej+7x8WpVHY0BzmnPxYR+P57a1c1OfrtNz3mhp71r1qlfbZBFSvqEM82bTQJtGmjt9NOC8N/rePHtbA1a07cgklg//4Tr6tWTQsfOFATgnUcR1xR+hQkh/vZPS42zrm5wNz4+PjbvIrBMyIwfLgOw/voI+je3euIjMeOH4cNGzKf9a9erQPW0tL0NWXL6uC1jh11GfBgEmjSJAxTAB06BOvWZT7or1mjARw9euJ1devqdBPXX6//tmyp1ZC6de3kJorklRRcDveze2yKy6BB8OijuoagJYVSZ+tWneBz4UL47js9DgePvSI6pr91ax2sFjzzb9YsDM3yu3dnPugHk8CWLVqDBe1BPuMMPdj36pX54G81AF/IKym0EJEf0FrBGYH7BB43iWhkJmfly+voo+HDdebHc87xOiITQSkpOtfPp59qMgi2mNaurWf+3bufaPpp0QJOOqmIBTqnAxm+/lpvK1ZoAgjtca5USase552nM44GD/7NmkGFCkUMwHgprz6Fhrm92Tm3JewRFUCpHH0UtH+/Tn1x6aU6YbwpMfbv15nSP/tMb99/r9urVoULL4SLLoJu3TQJhGXtpb179ergYBL45psTFxecdBLExWnHQ/CMv2VL7XW2hZ98q9B9ClkP+iJSC+gC/M85tzx8IZoCq1YN7rpLm5A2btQqu/GlI0d0srfPPtPawLJl2hdQoQJ07qwXsHfrpmP+i7xS5NGjeuYfTABff60dEqBtT61a6QVi556rt9atS/3ylKVNXjWFD4CHnXOrRKQu8C2QCJwBTHDOvVIsUeagVNcUQIf1NWqkV3e+9ppW3U3UO35c53ILJoGlS3WkUEyMtgQGawKdOunooEJzTq9pCU0A3313ogOibt0TB/9zz9WsY1fKlwpFGX3U2Dm3KnD/ZmCBc+4GEakKLAVeCV+YpsDq1tXTyEcf1fbdK67Qq57PPdfryEyI9HQdERRMAp9/DgcO6HPt2+uQ/G7doEsXrQAWWmgz0Dff6C3YD3DSSXrQHzr0RBJo0MBG/ZjfyaumsMI5Fxe4/ykw0Tk3Letzxa1UX6eQnZ9/hrFjdVqAffv06DJyJPToYe2+Htm4URPAp5/qKKGUFN3erJkmgIsu0ovTY2MLWUB6unY2LF16ohYQ/C0Em4FCawHWDGRCFHpCPBGZC3wMJAOT0JrDPhGpBCQ651pHIuD8KvXNR1kdOKDzIr30EiQn64FgxAgdwmrTBEScc1obePZZTQYA9eppArjoIm3lO/30IhSQmqprT86fr4sR//yzbs/aDHT22UWscpiSrihJoQ7wBFAXGOec+ziwPQE42zn3YgTizTdLCjk4dgymTYMXXoCVK6F+fV3n+fbb7WARAenpuhbAs89qJ/Gpp+qfu39/XSiv0C00zulVaPPn623pUu2QOPlkuOwyrQkmJFgzkCkwmzq7tHJOr3oePVrbMKpVgzvvhPvu01NYUyTHjsF77+lCMGvW6NXBI0fqTA2F7iA+eFCrG8FEsHWrbo+Lg5499XbuudYUZIqkKDWFObnt2DnXt4ixFYklhQJITNSaw8yZOszl+uv14reWLb2OzHcOHdIlgV94QZe2aNcOHn4YrrqqkMfqn346kQQ+/1xHB1WtCpdcokmge3et7RkTJkVJCinAVmAq8DVZ5jtyzn0exjgLzJJCIWzcqH0O77yj6yD26aOnt507WxNEHvbt0/VeXnlFO447d9aF8Hr2LOCf7sgRPfgHE0HwOoGWLU/UBs4/3/qBTMQUJSnEAJcAg4B2wDxgqnNudSQCLShLCkWQkqJz17/+ul69et55mhz69rURS1ns3Akvv6yDu/bv1xP3P/1Jl47Mt6Qk7RyeP197oQ8f1qkiunXTJNCjh85PbUwxCEufgohUQJPDC8ATzrnXwhdi4VhSCINff9Vaw0sv6TTHZ56pzUrXX1/EK6f8b/NmbSKaNElbdK66SpuJOnTIx5uPHtWO4WBt4McfdXuTJjpRXM+eOmdFpUoR/QzGZKdISSGQDHqhCaERMAeY5JzbFuY4C8ySQhgdP65LfY4eDd9+q0sgDh2qU2mcfLLX0RWr1avhued0AbAyZbTjeOTIfFwwvm3bidrAJ5/oEOHy5fXgH2wWatbMmumM54rSfPQ3oA3wITAt5OrmiBCR/mgCqkPIENicWFKIAOd0pNLo0TpyqXJlHcp63306CVoJ9tVXOqx0zhz92HfcAQ88kEsf77Fj8OWXJxLBD4FJhE87TZuDevbUCxSqVCm2z2BMfhQlKaQDvwYehr5QAOecy3PQu4hMAnoDu5xzbUK2dwdeBWKAt5xzz4U8dzLwonPu1tz2bUkhwr7/XttPpk3TGdpq19ZZMlu00Gk1gvcbNfLtEEnn9HqwZ5/V6alr1tQK0r33Qq1a2bxhxw74z380CSxYoBeUlS2rHcPBvoHWra02YKKap9cpiEgX4CDw92BSCHRg/4R2YicDy4BBzrkfA8+PAaY4577Nbd+WFIrJli06lHXtWl1ha+3aE/M2gDaRNG36+4TRvHnULqySng6zZ2syWL5cL9sYPhxuuy3Lif3x4zqFRPAq4u++0+316p1IAhdfbBcFGl8pyoR4ReacWywijbJs7ghscM5tAhCRaUA/EVkDPAd8mFNCEJHbgdsBTi/SnAEm3xo2hAcfzLxtzx5NEMEksXatNsa///6JNSBBp2AIrVUEk8Xpp3syyunoUZgyRS84W7dOc9nEidqvnrE2zM6d2nQ2fz58/DH88ote29Gpk2aRHj1OrJNtTAnjVZ2/Pnr9Q1AycC4wBLgYqC4iTZ1zb2Z9o3NuAjABtKZQDLGa7NSqpQfJTp0ybz96VKdrDq1VrF2rTVD79p14XaVKOtIpNGE0b64d3NWra6N+GA+6T342mh2J5zD3tQSSk/UC4VGTFnJS02X8sdODOj9FsDYQrH2eeqrOVdGjh15IVqNG2OIxJlp5lRSy+7U759xYYGyebz4xS2rYAzNFVL78iYN8KOe0ySmYJIIJIzFRm6bS0zO/PiZGk0P16nowDt7P+ji3+xUqsHOnTiD76pxz+LXH1bSPn8GECQlUOHU213xwIzOWdoTLR2vNp0wZvV7jqae0aah9e7tmw5Q6XiWFZOC0kMcNgO35fbNzbi4wNz4+/rZwB2YiRATq1NFbly6ZnztyRK/qXbdO5/9PTT1x27fvxP1Nm07c37//xGLxOTgWUwHSqnMDNbjp5OqsXXk6t8R358v3Yhlfdxsz/gkJh1aeuG7gkku0p9mYUsyrpLAMaCYijYFtwEDg2vy+2WoKJUzFirrgcJs2eb82KD1drwPIkjw2fpfKwn/tY/OKVE4mlXNapnJW431UTU+lWWoqd62rxpOttzGKC0n454tw1llWGzAmRMSTgohMBboCtUUkGfiLc+5tEbkX+AgdkjqpIFNnWE3BUKZMRlORczpKdPQYHVZaowbc/Se4boh2CwQt3LyQ8TOvZlT8KMYnjieh1gESLCEYk0lxjD4alMP2+cD8wuzTagoG9Nqx6dP1OruVK3VZgTFjdFhp1qWGF25eyNUzr2bGgBkkNE4goVFCpsfGGOXL0yTn3Fzn3O3Vo3QMvImsX3+FV1/V4aTXX68jYCdP1glgH3gg+7Xnl21flikBJDROYMaAGSzbvqx4gzcmytkiO8Y3du3SSV3HjdM16i+4AB56yJaiNqagPL14LRKs+ah02bRJm4UmTYLffoN+/XSCuvPO8zoyY0oeX55fWfNR6bB8OQwcqBOLvvUWXHedzkA9e7YlBGMixZc1BVNyOaezTj//vK5FU60ajBihk9TZstLGRJ4vk4I1H5U8+/bB3Lm6wtl33+mUSaNH6/TVNtecMcXHl0nBrlPwP+d0vfoPPtDbkiU6IWmLFvD22zB4cMgEdcaYYuPLpGD86ehRWLwY5s3TRBBcr75tW20i6tVL+wpsJJEx3rGkYCJq506dePSDD3QW6gMHtAZw0UV6TUGvXjqLtjEmOvgyKVifQvRyDlasONEstGyZbqtfH669VpNAt246M7YxJvrYxWumyH79VUcKzZunt23bdFLUjh2hd2+9tW9va9IYEy1K3MVrxntbtpzoG/jsM72orGpVuOwyTQI9eugs2cYYf7GkYPIlLQ2++kqTwLx5OgEd6PxDd92lieCCC3SNHWOMf1lSMLnavRteegkmTNDFycqW1YP/mDGaCM480+sIjTHh5MukYB3NkZeSAi++qJPPHToEV1wBV18Nl15qSxUbU5L5ckS4zX0UOTt36jUDjRppUujXD1at0mWUr77aEoIxJZ0vawom/H7+WaeVePNN7TS+9lp47DFo3tzryIwxxcmSQim3fbsmg7/+VVcyu+46+NOfrK/AmNLKkkIplZysM5FOnKhzDt1wgyYD66YxpnSzpFDKbN0Kzz2n6xOkp8NNN8Ejj0CTJl5HZoyJBpYUSoktW+DZZ3X1MoBbboGHH9YOZWOMCbKkUMJt3qzJYPJknWbij3/UZGCT0BljsuPLpGDXKeRt40Z45hn4+98hJkYXq3noIWjQwOvIjDHRzK5TKGHWr9d+gubN4b334O67NUG89polBGNM3nxZUzC/t24dPP00TJmi6xUMHaoXodWt63Vkxhg/saTgcxs2wF/+AtOmQcWKMGyYJoNTTvE6MmOMH1lS8LHvv4eEBF3mcvhwePBBm67aGFM0lhR86scf4eKLoUoV+PxzaNzY64iMMSWBLzuaS7ufftI1jsuV0xXPLCEYY8LFago+s3GjrnGcng6LFkGzZl5HZIwpSSwp+MiWLZoQDh/WhNCypdcRGWNKmqhpPhKRJiLytojM9DqWaLRtmyaE/fthwQJo29briIwxJVFEk4KITBKRXSKyKsv27iKyTkQ2iMjDAM65Tc65WyMZj1/9/LMmhJQU+OgjOOssryMyxpRUka4pTAa6h24QkRhgHNADaAUMEpFWEY7Dt1JSdJTRtm3w4YfQsaPXERljSrKIJgXn3GJgb5bNHYENgZrBUWAa0C+/+xSR20UkUUQSU1JSwhht9Nm7Fy65RDuX586Fzp29jsgYU9J50adQH9ga8jgZqC8itUTkTaCDiDyS05udcxOcc/HOufjY2NhIx+qZ1FS47DJYswbef18vUjPGmEjzYvSRZLPNOef2AHfmawclfJbUAwegRw+9Yvlf/4JLL/U6ImNMaeFFTSEZOC3kcQNge0F2UJJnSf31V+jdG775BqZP1/vGGFNcvEgKy4BmItJYRMoDA4E5BdmBiPQRkQmpqakRCdArhw9Dv36wZInOdnr55V5HZIwpbSI9JHUq8CXQXESSReRW59xx4F7gI2ANMMM5t7og+y2JNYXffoMrr4TPPoN33oFrrvE6ImNMaRTRPgXn3KActs8H5hd2vyWtT+HYMU0CH34IEyfCDTd4HZExprSKmiuaC6Ik1RSOH4fBg3WE0euv6xrKxhjjFV8mhZIiLU2XzvznP2HMGLjnHq8jMsaUdr5MCiWhozk9HW67TTuUn3kGHnjA64iMMcanScHvzUfOaa3gnXd0Kc1HcrxUzxhjipcvk4KfOafrKL/5Jjz0kCYFY4yJFr5MCn5tPnIOHn4YXn0V7r8fnn0WJLvru40xxiO+TAp+bT56/HEYPRruugteeskSgjEm+vgyKfjRM8/AE0/Arbfq0FNLCMaYaGRJoRiMGQOPPgrXXQd//SuUsb+6MSZK+fLw5Kc+hddfh+HD4aqrdLRRTIzXERljTM58mRT80qcwYQIMGaKT3E2ZAmW9mKjcGGMKwJdJwQ/+/ne4807o2VOnwC5XzuuIjDEmb5YUImDGDLj5ZujWDWbOhAoVvI7IGGPyx5JCmM2ZoxPcdeqkk9xVquR1RMYYk3++TArR2tH88cfaodyhA8ybB5Urex2RMcYUjC+TQjR2NH/+OfTvDy1bwn/+A9WqeR2RMcYUnC+TQrT58kvo1QsaNYIFC6BmTa8jMsaYwrGkUETffgs9ekDduvDppxAb63VExhhTeJYUimDVKrjkEqhRQxNC3bpeR2SMMUVjSaGQ1q2Diy+GihU1IZx+utcRGWNM0VlSKIRNm+Cii3Qq7E8/hTPO8DoiY4wJD5t4oYC2btWEcPgwLFwILVp4HZExxoSPL2sKXl2n8PPPmhD27oWPPoJ27Yq1eGOMiThfJgUvrlPYvVv7ELZvhw8/hPj4YivaGGOKjTUf5cMvv+goo40bYf58ncLCGGNKIksKeThwQK9DWL1a5zVKSPA6ImOMiRxLCrk4dAh694bERJ3ttHt3ryMyxpjIsqSQgyNHdC6jJUvgvff0vjHGlHSWFLJx9KjOdrpgAUyeDNdc43VExhhTPHw5+iiSjh/X9RA++ADGj4cbb/Q6ImOMKT6WFEKkp+uKaTNnwksv6XKaxhhTmkRN85GIVAbeAI4Ci5xzU4qzfOc0Cbz7Ljz1FAwbVpylG2NMdIhoTUFEJonILhFZlWV7dxFZJyIbROThwOYrgJnOuduAvpGMKyvn4P77YeJEePRRvRljTGkU6eajyUCmgZwiEgOMA3oArYBBItIKaABsDbwsLcJxZXAOHnkExo7V2sGTTxZXycYYE30imhScc4uBvVk2dwQ2OOc2OeeOAtOAfkAymhhyjUtEbheRRBFJTElJKXKMTz4Jzz+vTUdjxoBIkXdpjDG+5UVHc31O1AhAk0F94F/AlSIyHpib05udcxOcc/HOufjYIi5z9sIL8Je/6AijceMsIRhjjBcdzdkdep1z7lfg5nztQKQP0Kdp06aFDuL112HkSL0G4e23oYyNwzLGGE9qCsnAaSGPGwDbC7KDos6SOmkSDBkC/frBP/4BMTGF2o0xxpQ4XiSFZUAzEWksIuWBgcCcguygqOspNGwIV14J06dDuXKF2oUxxpRIkR6SOhX4EmguIskicqtz7jhwL/ARsAaY4ZxbXZD9FrWmcNFFeoFahQqFersxxpRYEe1TcM4NymH7fGB+Yfcbjj4FY4wxv+fL7lUvVl4zxpjSwJdJwRhjTGT4MikUtaPZGGNM9nyZFKz5yBhjIsOXScEYY0xk+DIpWPORMcZEhi+TgjUfGWNMZIhzzusYCk1EUoAthXx7bWB3GMPxspySUkZxlVNSyiiucuyzRF8ZRS2noXMu2xlFfZ0UikJEEp1z8SWhnJJSRnGVU1LKKK5y7LNEXxmRLMeXzUfGGGMiw5KCMcaYDKU5KUwoQeWUlDKKq5ySUkZxlWOfJfrKiFg5pbZPwRhjzO+V5pqCMcaYLCwpGGOMyWBJwRhjTAZLCsYYYzL4PimIyCQR2SUiqyK0/xoisllEjovIERE5L8z7rygi34jI9yKyWkT+rwDvjehnL2p8BSwnSURWisgKEUkM436z/RuJSHcRWSciG0Tk4SLs/zQRWSgiawJ/n/vCXUZgXzl+D+EsJ7C/GBH5TkQ+iGAZ2X7fRS2nIL+J3L67MJZRbL/vsP2GnHO+vgFdgLOAVRHa/9+A5wNlrAZqhHn/AlQJ3C8HfA38IRo+e1HjK2A5SUDt4vj/AcQAG4EmQHnge6BVIfdfFzgrcL8q8BPQKpxl5PY9hLucwP4fAN4DPgj33yu37zsc5RTkN5HTdxfmMort9x2u35DvawrOucXA3qzbRaSxiLwvIomBTN28oPsWkWroF/NwoAznnNsXzjKcOhh4WC5wy9c44Zw+ezgVJb5okMPfqCOwwTm3yTl3FJgG9Cvk/nc4574N3D8ArAHqh7OMwL5z+h7CWo6INAB6AW+FbA5rGbkocjkFOR7k8t2Fs4yw/77DcdzJje+TQnZEpBz6n/oBp3ODPI4e2AuqCZACvAPMA+qJSOUwlxGsrq8AdgELnHNfF2Y/kVJM8TngYxFZLiK3R2D/oeoDW0MeJ5OPg0FeRKQR0AE9Gwx7GTl8D+Eu5xVgJJAesi0Sf6/svu9IfS95/lazfHdhLSOcv588PktYfkNlC/vGKNcfaA3MEhHQz/lF6AtE5BPg1Gze+6hz7v3A/bJo9W0I+sdPRL+AUWEsA+dcGhAnIjWA2SLSxjkXsX6Cgiqm+Do757aLSB1ggYisDZwpRYJks61ItR8RqQLMAu53zu2XwH+KcJaR3fdAGD+LiPQGdjnnlotI19CnwlVGiN993xEqB/L4rWb97sJdRph/PzmWQ5h+QyU1KbRHD7xv5/QC59zF+dhPMpDsnPs6cCaxH00S4Swj9PX7RGQR0B2ImqQQFMn4nHPbA//uEpHZaFNCpJJCMnBayOMGwPbC7ixw9jYLmOKc+1ckygiV5XtYGsZyOgN9RaQnUBGoJiLvAuPCWAaQ4/cdzs8SKsffag7fXVjLCArT7yfHcsL1GyqRzUfADuAyESkDICJtczhzy5Vz7mdga0ibXRXgx3CWISKxgTMIRKQScDGwtqD7iZTiiE9EKotI1eB94FIimxSXAc0CbbPlgYHAnMLsKPCdvw2scc69FIkyAuXk9D2ErRzn3CPOuQbOuUaB/XzmnLsuAp8lp+87rOWEyPa3mst3F84ywv37yamc8P2GitpT7fUNmBr4Qx1Dz85uBSoBM4F1wArg3SLsPw7YE9h/OrAtnGUA7YDvgB8CX+Kfi/LZI/D3LXR8BSijCTrS5Ht0hNejkfz/EdjeEx1tsrEo5QHno00cPwT+H6wAeoazjLy+h3CWE7LPrgRGH0Xgs+T4fRe1nIIcD3L77sJYRlh/37mUE7bfkE2IZ4wxJkNJbT4yxhhTCJYUjDHGZLCkYIwxJoMlBWOMMRksKRhjjMlgScEYY0wGSwrGGGMyWFIwJswCV5luEZG7vI7FmIKypGBMmDnnVqJTNNzgdSzGFJQlBWMiYxc6m6UxvmJJwZjIeA6oICINvQ7EmIKwpGBMmIlId6AyujCT1RaMr1hSMCaMRKQiMBq4G1gJtPE2ImMKxpKCMeH1GPB351wSlhSMD1lSMCZMAosxXYKucwyWFIwP2XoKxhhjMlhNwRhjTAZLCsYYYzJYUjDGGJPBkoIxxpgMlhSMMcZksKRgjDEmgyUFY4wxGf4fKf0HVQmdhOQAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<Figure size 432x288 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"training_error_ridge = []\n",
|
|
"test_error_ridge = []\n",
|
|
"\n",
|
|
"for i in range(len(y_training_ridge)):\n",
|
|
" training_error_ridge.append(mse(training_data_y, y_training_ridge[i]))\n",
|
|
" test_error_ridge.append(mse(test_data_y, y_test_ridge[i]))\n",
|
|
"\n",
|
|
"error_fig_ridge = plt.figure()\n",
|
|
"plt.figure(error_fig_ridge.number)\n",
|
|
"plt.title(\"Error Plot Ridge Regression\")\n",
|
|
"plt.xlabel(\"$\\lambda$\")\n",
|
|
"plt.ylabel(\"MSE\")\n",
|
|
"x_axis = [\"$1e-{6}$\", \"$1e-{3}$\", \"$1$\", \"$3$\", \"$5$\",\"$10$\",\"$20$\",\"$30$\",\"$40$\",\"$50$\",\n",
|
|
" \"$1e2$\", \"$1e3$\", \"$1e5$\"]\n",
|
|
"plt.yscale('log')\n",
|
|
"plt.plot(x_axis, training_error_ridge, 'b')\n",
|
|
"plt.plot(x_axis, test_error_ridge, 'r')\n",
|
|
"# let's find the index with the minimum training error\n",
|
|
"min_error_idx = np.argmin(test_error_ridge)\n",
|
|
"plt.plot(x_axis[min_error_idx], test_error_ridge[min_error_idx], 'xg')\n",
|
|
"plt.legend(['Training Error', 'Test Error', 'Min Test Error'])\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<matplotlib.legend.Legend at 0x7f0dc0f5ca30>"
|
|
]
|
|
},
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAvB0lEQVR4nO3deXiU1f338ffJnslCIAQIhCSIoBDWEDYVBVfcUKx7tPq0En5Wfdqn1Z9iWkFb1FbbulSlYLUqUysuuC8UFdGCYNgMewKEkAWyQNbJPuf5YyZhksxkn8z2fV1XrmTu+8zkhOUzJ99z7nMrrTVCCCF8i5+rOyCEEKL/SfgLIYQPkvAXQggfJOEvhBA+SMJfCCF8UICrO9BVgwcP1omJia7uhhBCeJRt27aVaK1j2h73mPBPTEwkIyPD1d0QQgiPopQ6au+4lH2EEMIHSfgLIYQPkvAXQggf5DE1f3saGhrIy8ujtrbW1V0RViEhIcTFxREYGOjqrgghOuDR4Z+Xl0dERASJiYkopVzdHZ+ntaa0tJS8vDxGjRrl6u4IITrg0WWf2tpaoqOjJfjdhFKK6Oho+U1MCA/g0eEPSPC7Gfn7EMIzeHz4CyGEt9p1rIy/fZVFZW1Dn7+2hL8QQrip1zbl8PdvDuPnhN+ofSv8jUZITAQ/P8tno7FXL1dWVsaLL77Yo+c+88wzmEymLrf/5z//yb333tthmw0bNrBp06Ye9UcI4V5OVdfzcWYh104dQVhw36/N8Z3wNxohLQ2OHgWtLZ/T0nr1BtCf4d8VEv5CeI93t+dR32jm1pnxTnl9j17q2S3p6dA2bE0my/HU1B695EMPPcShQ4eYMmUKl1xyCUOGDGHNmjXU1dWxcOFCHn30Uaqrq7nxxhvJy8ujqamJ3/3ud5w4cYKCggLmzZvH4MGD+frrr+2+/quvvsoTTzxBbGwsY8eOJTg4GICPPvqIP/zhD9TX1xMdHY3RaKSmpoYVK1bg7+/P6tWref755ykrK2vXbujQoT36WYUQ/Udrzb+25jI1PopxsZHO+yae8DFt2jTd1t69e9sdc0gprS1j/tYfSnX9Ndo4cuSITkpK0lpr/cUXX+hFixZps9msm5qa9JVXXqm/+eYb/c477+i77rqr5TllZWVaa60TEhJ0cXGxw9cuKCjQI0eO1EVFRbqurk6fc845+p577tFaa33y5EltNpu11lqvWrVK//rXv9Zaa7106VL91FNPtbyGo3bO1q2/FyFEO5sPleiEBz/Wb2cc6/VrARnaTqb6zsg/Pt5S6rF3vA+sW7eOdevWMXXqVACqqqrIyspizpw53H///Tz44INcddVVzJkzp0uvt2XLFubOnUtMjGUn1ptuuomDBw8ClovbbrrpJgoLC6mvr3d4QVVX2wkh3ItxSy6RIQFcNSnWad/Dd2r+y5eDwdD6mMFgOd4HtNYsWbKEnTt3snPnTrKzs/n5z3/O2LFj2bZtGxMnTmTJkiU89thjXX5NR2vm77vvPu69914yMzP5+9//7vCiqq62E0K4j9KqOj7fXch1yXGEBPo77fv4TvinpsLKlZCQAEpZPq9c2eN6P0BERASVlZUAXHbZZbzyyitUVVUBkJ+fT1FREQUFBRgMBm677Tbuv/9+tm/f3u659sycOZMNGzZQWlpKQ0MDb7/9dsu58vJyRowYAcBrr71mtz8dtRNCuK+3t+XR0KRJLf6xT1cntuU7ZR+wBH0vwr6t6Ohozj33XCZMmMDll1/OrbfeyuzZswEIDw9n9erVZGdn88ADD+Dn50dgYCAvvfQSAGlpaVx++eXExsbanfCNjY1l2bJlzJ49m9jYWJKTk2lqagJg2bJl3HDDDYwYMYJZs2Zx5MgRAK6++mquv/56PvjgA55//nmH7YQQ7qmhycxrm3KYHdbAmP971+lFKs2rE6HPMkxZ5gPcX0pKim57J699+/Yxbtw4F/VIOCJ/L0L0zPs78vnVWzt59ZsXmff9p+0bJCRATk63XlMptU1rndL2uO+UfYQQwo1prVm58TBjhoRzwZbP7TfKze2z7+dbZR83NXPmTOrq6lode+ONN5g4caKLeiSE6G+bDpWyt7CCP/1kEn7xI526OhEk/N3Cli1bXN0FIYSLrfr2MIMDzFxz0zxL8CtluRqpWR+uTgQp+wghhMsdOF7JhgPF3PntWwQfOWQ5qLXlDQD6ZHViWzLyF0IIF/v7xkOENtaRuvXD1ie07tEkb1fIyF8IIVzocHEV7+/I57btnzKw1s61P304yWtLwl8IIVzo+a+yCQ7wZ3H+9/Yb9OEkry2fCv8+3s6/x1s6X3HFFZSVlXXY5pFHHmH9+vU97Fnnli1bxtNPP91hm/fff5+9e/c6rQ9C+Lrsoko+2JHHT3/8jMEH95yu8Tfr40leWz4T/k7Yzt9h+DdfievIp59+SlRUVIdtHnvsMS6++OKed64PSPgL4VzP/mM9IfV1LP7PPy0HnDzJa8tnwr+j7fx7ynY//+nTpzNv3jxuvfXWlvX51157LdOmTSMpKYmVK1e2PC8xMZGSkhJycnIYN24cixYtIikpiUsvvZSamhoA7rzzTt55552W9kuXLiU5OZmJEyeyf/9+AIqLi7nkkktITk5m8eLFJCQkUFJS4rC/y5cv56yzzuLiiy/mwIEDLcdXrVrF9OnTmTx5Mj/5yU8wmUxs2rSJDz/8kAceeIApU6Zw6NAhu+2EED1z8EQlH5cFcOe2DxlUU3H6hO0kr5OC3/p9XL9Xf1c+erufvxO282+1n//XX3+tDQaDPnz4cMv50tJSrbXWJpNJJyUl6ZKSEq316b38jxw5ov39/fWOHTu01lrfcMMN+o033tBaa33HHXfot99+u6X9c889p7XW+oUXXtA///nPtdZa33PPPfrxxx/XWmv92WefacDhPQIyMjL0hAkTdHV1tS4vL9ejR49u2fu/uV9aa52ent7yvWz70FG7tmQ/fyE6t/j1DJ30qzX6ZEhE3wZTGzjYz99nRv6O5kz6ci5lxowZrfbMf+6555g8eTKzZs3i2LFjZGVltXvOqFGjmDJlCgDTpk0jx8GSruuuu65dm++++46bb74ZgPnz5zNw4ECHffv2229ZuHAhBoOByMhIFixY0HJu9+7dzJkzh4kTJ2I0GtmzZ4/d1+hqOyFEx7as/Def7zlO2tb37K/wcdIkry2fCX8nb+cPQFhYWMvXGzZsYP369WzevJldu3YxdepUu/vpN9+aEcDf35/Gxka7r93czraN7uamfI7uD3DnnXfyt7/9jczMTJYuXepw3/+uthNCOGZebeT33xczvKKIRVvXtm/gxEleWz4T/k7Yzr/DPfnLy8sZOHAgBoOB/fv38/33DpZx9cJ5553HmjVrAMudxE6dOuWw7fnnn8/atWupqamhsrKSjz76qOVcZWUlsbGxNDQ0YLSZAW/78zlqJ4Tounf/8RG7h5zBgxv+SWhj6z29nD3Ja8unrvDt4+38W+3nHxoa2urm6PPnz2fFihVMmjSJs846i1mzZvXdN7ZaunQpt9xyC2+99RYXXHABsbGxRERE2G2bnJzMTTfdxJQpU0hISGh1O8nf//73zJw5k4SEBCZOnNgS+DfffDOLFi3iueee45133nHYTgjRNdV1jfwp6Uqm5u9nwb6NrU8q5ZQreR2R/fw9WF1dHf7+/gQEBLB582buvvtudu7c6epu+fzfixB2GY08/e/N/C3pct574zckFxxofd5J2zg42s/fp0b+3iY3N5cbb7wRs9lMUFAQq1atcnWXhBD2GI1kPfgof7/lTyzc/VX74O+nOr8tCX8PNmbMGHbs2NHqWGlpKRdddFG7tl9++SXR0dH91TUhhA1z+m95aO4vCKuvIf3rf7Q+mZBgCf5+qPPbcnr4K6VygEqgCWjUWqcopQYBbwGJQA5wo9ba8Wyl6LLo6Gi3KP0IIU4zDhrPtrjxPP3JXxhsKj99op/r/Lb6a7XPPK31FJu600PAl1rrMcCX1sdCCOFdjEaOj5vMHy+4k/NydvCT3V+1Pt8P6/kdcdVSz2uA16xfvwZc66J+CCGEcxiN6LQ0fjt+AY1+fiz/4gVaXWnjgjq/rf4Ifw2sU0ptU0qlWY8N1VoXAlg/D7H3RKVUmlIqQymVUVxc3A9dFUKIPpKezptj5rB+zCzu/3Y1CWXHT5/rx/X8jvRH+J+rtU4GLgfuUUqd39Unaq1Xaq1TtNYpMTExzuthL5xzzjmdtvn2229JSkpiypQpLRu3CSG8W3aVmccuWsScI9v52Q8fnD7RXOd3YfBDP4S/1rrA+rkIWAvMAE4opWIBrJ+LnN0PZ9m0aVOnbYxGI/fffz87d+4kNDS00/Zaa8xmc190TwjR34xG6s44k/979f0YGur48yd/xQ+b66lcWOe35dTwV0qFKaUimr8GLgV2Ax8Cd1ib3QF8YP8V3F94eDhg2ctn7ty5XH/99Zx99tmkpqaitebll19mzZo1PPbYY6Ra3+mfeuoppk+fzqRJk1i6dClAy/bOv/jFL0hOTubYsWMdtrO3DXR2djYXX3wxkydPJjk5mUOHDjn8fkIIJ7DeOOSpxLnsHTqaP372LEOqbRYyurjOb8vZSz2HAmutG4oFAP/SWn+ulPoBWKOU+jmQC9zQ22/06Ed72FtQ0XnDbhg/PJKlVyd1uf2OHTvYs2cPw4cP59xzz+W///0vd911F9999x1XXXUV119/PevWrSMrK4utW7eitWbBggVs3LiR+Ph4Dhw4wKuvvsqLL77YYbusrCzefPNNVq1axY033si7777LbbfdRmpqKg899BALFy6ktrYWs9ns8HXOP7/L1TchRFelp/NxfDIvz1jI7ds/5pLsrafPuWg9vyNODX+t9WFgsp3jpUD7K5E83IwZM4iLiwNgypQp5OTkcN5557Vqs27dOtatW8fUqVMBqKqqIisri/j4eBISElr2AOqonb1toCsrK8nPz2fhwoUAhISEdPg6Ev5C9L29NX48cN2vSM7fx2+/evn0CReu53fEa67w7c4I3Vm6sj2z1polS5awePHiVsdzcnJabQndUbu236empsbh9s6OXkcI0YeMRk49+jhpC9OJrKtmxdrHCW6y+f/vJnV+Wz6zpbO7uOyyy3jllVeoqqoCID8/n6Ki9vPdXW3XLDIykri4ON5//33AsumbyWTq9usIIbrJaKT+f+7mnuRUisKjWbF2udvW+W15zcjfU1x66aXs27eP2bNnA5YJ49WrV+Pv79+jdrbeeOMNFi9ezCOPPEJgYCBvv/22w9cZMsTupRVCiG4yp/+WB+YuZlPiZJ7+5C9MLTx4+qSb1fltyZbOos/J34vwFkYjpKdDbq6lctMqx41GdHo6vx99Ca9Mv5YHvnmNe75/+/STlQI3WLItWzoLIUQ3WFdtYjJZHh89ankMkIrl5IqJl/PK9Gu5M+NDfmEb/OCWdX5bEv5CCGFHevrp4G9mMlmOp5LOy+Mv4Y9z/w9X7/2GR75c5Vb79nSFx0/4ekrZylfI34fwFrm5Do4fNbNiaAp/uGgRlx/4L3/55C+tr+B1g317usKjR/4hISGUlpYSHR2N9UIy4UJaa0pLS1uuMRDCk8XHW0o97Y7P3saT51tG/H/9+M8EaJu6vpNuxegMHh3+cXFx5OXlITt+uo+QkJCWC92E8GRXXAErVkDLL7NKE3PhLkgpYuHur3jq02daB78HlHpseXT4BwYGMmrUKFd3QwjhZYxGeO2108GvAhsZfPUODGOKuGvrWpZseBX/tiN+N13S6YhHh78QQjiD7WSvf3gtMQszCBpWDv8Zxm+327kHr4eUemxJ+AshRBvNk70h8SUMXrADFdhE8doUarPb3FfEw0o9tiT8hRDCltFIvN/5nJpeR9ScAzScDKf4zWQaSyNIIOd0Ow8s9diS8BdCiGZGI3m/eZgzbniEwwnDqN47nNLPJ6IbAjBQzXIetrRzw106u8vj1/kLIURf0Frz9ssfMf+WpzkxLJJrP/uesI+ioMGPBHJYySJSedPS2M2v3u0KGfkLIXzeoVfeZOnGfL6beTszczN5+tNnGFl+gmf4Q/vGHlzntyXhL4TwWab6Rl54/gNWHjcQEjWSx9a9xG07Pm19xS6Av79lk7Z2u7t5Lgl/IYTPaWwy89YL7/LMoQaKQwdw3b6vWPL1K8SYyto3Nhg8YruG7pLwF0L4jCaz5pPMQp55L4PDdQamle7lpQ1/ICV/n/0nePiKno5I+AshvF59o5m1O/J4acMhckpNjCkvYuX6l7kkewsOdwXz0Iu3ukrCXwjhtYoqajFuyeVfW3MprqxjQkgjK757hUv/+1H7ur4tL5nU7YiEvxDCqzSZNRuzinknI48v9hyn0ayZd1YMd9Yd4fzf/AzVdpP+try41GNLwl8I4fG01uwpqOCjHwv4YEcBxytqGWgI5I5zErm9aBeJj95jf39mW146seuIhL8QwiM1mTU7j51i/b4iPsssJKfURICf4oKxMSxbMJ4Lzx5K0Ftvwn1p7W/J1ZaPjPZtSfgLITxGUUUt32WX8F1WCRsOFnOyup4AP8Xs0dHcPXc0l44fxsCwIMuezFeldz7aB6+f2HVEwl8I4bZOVNSy9chJth45yZYjpRw8UQXAQEMg54+N4aJxQ7lgbAwDQgNPP6ntndc74gMTu45I+Ash3EK5qYE9BeVk5pezK6+MHbllFJbXAhAW5E9ywkAWTo1jzpjBjI+NxM+vzSJNo9GyEX9XRvvgk6UeWxL+Qoh+VdvQRE5pNVknqjhwvJL9xyvZf7yCvFM1LW1GDgolJXEQU0ZGMT1xIONjIwnwt7MPpW3gK2Vzz8UO+NjEriMS/kKIPlfb0ETeqRqOnTSRe9LEkZLqlo9jp0wtGe3vpxg1OIzJI6O4dWY8E4YPYMKIAQwKC+r8m7Qt73Ql+H18tG9Lwl+IPtA8AM3N9aq9v+yqa2yiuLKOExV1nKiopbC8luPlNRSU1ZJfVkN+WQ3FlXWtnhMW5E/i4DAmxg3g2qkjOHNIOGfGhDN6SBjBAf7d60B3yzsgo307JPyF+7NN1kGDLMdOnuzd132Y0G0HoEePWh6DZ2SN2ayprG3klKme0up6TlbXc7K6jtLqekqr6impqqOkqo7iSsvHKVNDu9cIDvBjeFQoI6JCmXdWDCOiDCREGxg5KJSRgwzEhAejlMONFDrXk/JOMxnt26V0d/4QXSglJUVnZGS4uhuiL7UdLl9xBXz6aeuQLy3t/n/2rmp+3ehoy+MevkEkJtofhPbnCsIms6aqrpHK2gYqaqyfaxupqGmgoraB8hrrh6mBspoGykz1lNk8bjLb//MNC/InOjyYweFBxEQEExMRzODwYIZFhjA0MoQhkcEMHxBKlCGwd+Heke6s3rElo30AlFLbtNYp7Y67KvyVUvOBZwF/4GWt9ZMdtZfw92D2Ru7ODPW+ZO8Nos0blZ9uRNu5KZ5Sli3gHdFaU9tgpqqukeq6xpbP1fWNVNU1UVVreVzZfL7W0qayrpGq2gYqaxuptB6rqmvs9EeJCA5ggCGQAaGBRBkCiTIEMSA0kEGGIKIMgQw0BDEoPIjosCAGGoIYHB5MaFA3SzJ9qSflnea/Lxntt3AU/i4p+yil/IEXgEuAPOAHpdSHWuu9ruiPcAJHv6aXlp5u4+7BD636rYE6/0Cqi05R/a/3qAw2UD1iPPFBeygOisYvqBEV1GT93MiA4HJ+9WQh1cNHWkK9rpHq+qaWoDfVNzkccbcVHhxAWLA/ESGBhAcHEB4cwNDIEMvXIQFEhAQSGRJAREgAkSGBRIYGEhESwIBQS9iHBwfYXy3jbqS8029cVfOfAWRrrQ8DKKX+DVwDSPh7Mkf/cd0g5Bv8/KkMDqMiOIzKYAMVIeGWz8FhVAUZqAo2UBlsoCro9Oeq4FDL56BQqoINVAeF0uAfaOfVc4kht9UR3eCHocHEtpwywg7mE6YbGVBfw4jqCsKC/Ambnkz47CTCrKEeFmQJcUvIB7QEfPO5dmvavUlv/91IeadHXBX+I4BjNo/zgJku6ovojX4MfA1UB4VyKjSSU6ERnAqNpCwkgrLQCMpCwikPCacsJIKKkHDKQsOpCA6nIiSM8uBwaoJCOn39kIZaIupMRNSZCKuvIaLORLzpOOH1NYRbj4XXmwivryGs5bPlY0P9PJ6r+38cq09gZEMBj5vTT9/s2553FEZ9M+l+fyTXPIJ4clke9hiXhbzX5xPSbqGjSfvKSqivtxzr6r8bKe/0mktq/kqpG4DLtNZ3WR/fDszQWt/Xpl0akAYQHx8/7Wh3an/CeXrzq3kbTcqPUkMkxWEDKQ4bSFH4IEoMUZSGRVFiiOKkIZISQxSnDJGcDI2kPsDx+u/w+hoG1FQwoKGWAfXVDKgsYwANRNaZiCgvJdJfE1lvIqL0BBH1NUTUVlnO1VUTVl9DoLmpxz9Hdxm5hTRWYSKs5ZiBalay6PSbhqP5Bk8Juz78d9KKBH63uNWEr1JqNrBMa32Z9fESAK31E46eIxO+LtaD/8hNyo+i8IEURMaQFzmEwogYjkdEUxgxmOMRgzkRMYjisIE0+bWfVDQ01BJdU0F01UmidQODJoxl0KRxDDJYJiMHhgUx0DppOdBgqXEHdrGmbTRC+i+ryC01WEfcj5PaPOLupwnpRI5wlMR2xxPIIYdRHT+5K6uUbFdOOeMNo7Plt87485PyTo+4W/gHAAeBi4B84AfgVq31HkfPkfB3gS4Efq1/IEcHDufIwOHkDhxGTpTlc17kUPIHxLSrkUfUVRNbfYqh5UUM03UMnTmVIedMY4jNMsKYiGAMQc6pSNpbNegwUxwFnG249nBU60eT/RVCmDHjhBU2nb1hdOfr/lypJeWdXnOr8AdQSl0BPINlqecrWusOt9aT8O9nbVKy1j+QrMHxHIhJIGtwAlnRI8mOHknegCGYbUbuA03lxJedIK78OCPLi4grP8GIEIi763Zib7uB8GDXXlfolDX5PVjK2quRvxswcgvpPE4u8Zbfnni44zmO7pDA71NuF/7dJeHfT4xGTI88yt66AH4cNobMYWeyZ+hoDkXHtZRnghobOONkHmeWHuOMk3mMLs1n1Kl8Ek4VMqDe5Nb/cf387OdxZ2vye6SDNwWjvrnzmr+b6tJ8RXcEBkJkpOfNaXgICX/hUFFFLd8fOcm2dd+zbV8++wYntAT90MpSJpw4xLiiw4wvOsJZxTkknCokQJvbj/6i/0LqszPd+j+uO1yN2/ymYDx6Lul+T9qs9rHOPbj5BXA9/q3F0yewPZSEv2hRVdfIpuwSNmYVs+lQKYeLqwEw1NcwufAgyfn7mVpwgIknshladdLuaxi5lTRWth79ecB8XLdq/q7kxldFd2u+Qko4Lifh7+OOnTSxbu8J/rP3OBk5p2g0a8IaapmRm8msY5nMOppJ0olDBOgOah82/5ETq3ZztDS8XRNPuCOeR+/A2ZVN7vpgQrojHY78o1Na98Oj/nC9k4S/Dyooq+HDXQV8tKuAPQUVAJw1NIJ55mIu+PuTTDu0kyBz53vCAO1Gbv1aOxe908e7ohpLLyNNrcKkDS3fwi1/exKAm+3tI5yntqGJT34sZE3GMbYcsZRspoyM4uErzubS7K0kPnZPn+yDHh9v/2Xi43vTe+EUqal9msqpAJ7825MAJPy9xuHiKl7ffJT3tudRUdvIqMFh/PqSsVwzZTgJ0WGW0d993dwWt4M67fLl9mvnPnovbJ/Tx+8nwgUk/D2Y1pqMo6dYufEw6/edINDPj/kThnHLjHhmnTEI9a9/wTTr8MzPD5q6uH1BF36Hbz4loz8hPJOEv4facriUP//nIFuPnCTKEMh9887k9tmJxEQEWxq0XdbSWfD3YFWGjP6E8FwS/h4mM6+cP32xn2+zShgSEcyyq8dz4/SR7bdDSE/veolHluEJ4XM84O4OAqCkqo4H3/mRBS98x96CCn575Tg2/u887jx3VOvgNxohMRHj0XNI5Ah+NJHIEYzc0v5FDQZYvdqyNlOCXwifIiN/N2c2a974/ihPf3GAmoYmFs05g/suPJOIEDs3FbGWeoyma1pdfn+URNJYBUCq/xrLWkwp0gvh0yT83VhOSTX/++6PbD1ykjljBrNsQRKjY9pfWNX2XqfpPN7qylsAE2GkqydJfe1KCXwhhIS/O9LaMtp//NN9BPr78afrJ3HDtDiUsnMrPzv7FeRif7F9rh4pwS+EACT83U5lbQMPvZvJJ5mFXDA2hid/MpHYAaGOn2BnYjeeXLuX38cnePF9YIUQ3SITvm5kb0EFVz//HZ/vOc5Dl5/Nq3dOdxz81olde5fZLudhDFS3OiYXYAkhbMnI302s33uC+97cQWRoAG8umsWMUYMcN7a3NaWN5j3V0/3/RG7TCOITlMztCiFakfB3A69vzmHZh3tIGj6Af9yRwpDIEPsN20zsOmQwkLrySlJT4/q+s0IIryDh70Jaa574bD8rNx7m4nFDeO6WqY7vXdvJaL+FXLAlhOgCCX8X0Vqz9MM9vL75KD+dncDSq5Pw9+tgQrYrV+x6wmb6Qgi3IBO+LmA2a373wW5e33yUtPPP4NEFHQR/BxO7rciMrhCiG2Tk38+01jzy4W5Wf5/L4gvO4KH5Z9tfvw9S6hFCOI2Efz979susrgU/dF7qkdsnCSF6SMo+/WjND8d4Zn0W10+L63zE31mpJyFBgl8I0WMy8u8nGw4UsWRtJnPGDOaJ6yb2rtQjE7tCiF6SkX8/yC6q4h7jdsYOjeDF1GQC/Tv4Y+9KqUcmdoUQvSTh72TVdY3cvXobwYH+vHJniv2tmG3l5jo+J6UeIUQfkbKPE2mteXhtJtnFVbzxs5kdb9DWfPWu1vbPS6lHCNGHJPydaPWWXD7YWcD9l47lvDGDHTfsrM4vpR4hRB+Tso+T7D9ewe8/2suFZw/hF3PP7LhxR3V+KfUIIZxAwt8JGprM/GbNLiJDA3j6hsn49fTqXaXk/rpCCKeQso8TvPj1IfYUVLDitmkMCguy36grSzrj7d+RSwghektG/n1sT0E5z3+VxbVThjN/wjC7bYxGSLzjAvxMlSRyBCO3tG8kdX4hhBNJ+Peh+kYz97/9IwPDgli2IMlum+YB/9GmODR+HCWRNFa1fgOQOr8QwsmcFv5KqWVKqXyl1E7rxxU255YopbKVUgeUUpc5qw/97fXNOewrrGD5tROIMtgv96T/sqpdpcdEGOk8bnnQvKRTgl8I4UTOrvn/VWv9tO0BpdR44GYgCRgOrFdKjdVaNzm5L05VVFnLM+uzuPDsIVyaZL/cg9FIbqmdEg+QS7yUeoQQ/cYVZZ9rgH9rreu01keAbGCGC/rRp576/AB1jU387qrxjhulpxOP/St44/0LpNQjhOg3zg7/e5VSPyqlXlFKDbQeGwEcs2mTZz3WjlIqTSmVoZTKKC4udnJXe27nsTLe3pbHz84bxajBYY4b5uaynIcxUN3qsIFqlr8WJ8EvhOg3vQp/pdR6pdRuOx/XAC8Bo4EpQCHw5+an2Xkpu3saaK1Xaq1TtNYpMTExvemq05jNltsxxkQEc9+FY+w3al7PrzWpvMlKFpFADgozCeSwMnqJ5L4Qol/1quavtb64K+2UUquAj60P84CRNqfjgILe9MOVPvqxgF3HyvjzDZMJD7bzx2lnPX8qb5LKm5YHBgM8u7KfeiuEEBbOXO0Ta/NwIbDb+vWHwM1KqWCl1ChgDLDVWf1wpsYmM8+uz+KsoREsnGq3ciVbNwgh3JIzV/v8SSk1BUtJJwdYDKC13qOUWgPsBRqBezx1pc8HOws4XFLNituSHW/h4GiL5uatG4QQwgWcFv5a69s7OLcc8Og1jQ1NZp79Mouk4ZFcZm9pZ2dbNMvWDUIIF5K9fXrove155J408fJPU9rfklG2aBZCuDnZ3qEH6hvNPPdlNpPjBnDRuCHtG0idXwjh5mTk3wNrd+SRX1bD8oUT7N+IXer8Qgg3JyP/btJa8/K3RxgXG8kFY9tce2Cznt8uqfMLIdyEhH83bcwqIauoirvOG9V61N+yXaeDG7NInV8I4UYk/Lvp5W8PMyQimKsnD299Qur8QggPIjX/bjhwvJJvs0p44LKzCApo874pdX4hhAeRkX83vPLdEUIC/bh1hp3avaN6vtT5hRBuSMK/i4or61i7M5+fJMcx0Pa+vLY3YW+78kfq/EIINyXh30Vv/ZBLfaOZn5036vTBtpO8Wp9+A5A6vxDCjUnNvwvMZs2ajDxmnTGI0THhp0/Ym+TV+vStGIUQwk3JyL8Lvj9cSu5JEzdPb1O/dzTJ6+i4EEK4CQn/Lngr4xgRIQHMn9BmAzeZ5BVCeCgJ/06Umxr4bPdxrp0ygpBA/9MTvH5+UFUFQUGtnyCTvEIIDyDh34n3d+ZT32jmpukjW0/wag2lpZbP0dGWiV6Z5BVCeAiZ8O3EWz8cI2l4JBNGDLA/wdvQAOHhUFLimg4KIUQPyMi/A7vzy9lbWMHN0623HJYJXiGEl5Dw78C72/MICvBjwRTr/XllglcI4SUk/B0wmzWfZhYyd2wMA95bI1fxCiG8ioS/Az/knORERR1XVh6Wq3iFEF5HJnwd+CSzkJBAPy7+6+/kKl4hhNeRkb8dTWbNp5nHufDsIYQdzrLfSCZ5hRAeTMLfji2HSympquOqScNlklcI4ZUk/O34OLMQg59m3rXnyySvEMIrSfi30dhk5vOMHC7av4nQw9mWgzLJK4TwMhL+bWw6VMrJJj+u2v01Rm4hkSP40USiPowx+j7LJK8EvxDCw0n4t/HZ7uOE1ZsoPDyaNFZxlEQ0fhwlkbTSJzAaXd1DIYToPQl/G1prvt5fxPknDrCs6Q+YCGt13kQY6eku6pwQQvQhCX8bewsrOF5Ry4WzzyIX+6t5ZIWnEMIbSPjb+Oqt/wAw96HFxPvl220jKzyFEN5Awr+Z0ciXu44xueAAMdWnWG5+EAPVrZrICk8hhLeQ8LcqeewJdg09kwsP/QBAKm+ykkUk+OfJfVqEEF5H9vax2hA0DK38uMga/mB5A0g1/xvMZhf2TAgh+l6vRv5KqRuUUnuUUmalVEqbc0uUUtlKqQNKqctsjk9TSmVazz2nVNvLZ13j64kXMLSylKQTh1qfkCK/EMIL9bbssxu4Dthoe1ApNR64GUgC5gMvKqX8radfAtKAMdaP+b3sQ6/VN5rZmDiVebk7afVOJEV+IYSX6lX4a633aa0P2Dl1DfBvrXWd1voIkA3MUErFApFa681aaw28Dlzbmz70mtFIxjnzqTQrLszPlJuxCyF8grMmfEcAx2we51mPjbB+3fa4XUqpNKVUhlIqo7i4uO97aTRCWhpfhccT1FjPubu/g5oaeOMN2cZBCOHVOg1/pdR6pdRuOx/XdPQ0O8d0B8ft0lqv1FqnaK1TYmJiOutq96Wng8nEfxMmk5K3j7CGWsuNW+QyXiGEl+t0tY/W+uIevG4eMNLmcRxQYD0eZ+e4a+TmUhoayb6hZ/DAN6+1Oi6EEN7MWWWfD4GblVLBSqlRWCZ2t2qtC4FKpdQs6yqfnwIfOKkPnYuP5/v4iQDMzv2x1XEhhPBmvV3quVAplQfMBj5RSn0BoLXeA6wB9gKfA/dorZusT7sbeBnLJPAh4LPe9KFXli9n0+hphNeZmFRovV2jrPARQviAXl3kpbVeC6x1cG450C5FtdYZwITefN9eMxotdf3cXDanrWTGiYMEYL0p+/LlMtErhPB6vneFr3WFDyYThRHRHI6K5dbMdZYVPhL6Qggf4Xt7+1hX+ABsjp8EwOzsDFnhI4TwKb438rdZybMpYRJRNRWMK8qxvwhVCCG8lO+N/K0reTSwOX4ys3Mz8UPLCh8hhE/xvfBfvhwMBnKjhpE/YAjnHN0lK3yEED7Hd8LfaITERLj9dggN5b/jzwVgdkOJ7OEjhPA5vlHzt1nhA0BpKZuHjmVIgJnRP35v2chNCCF8iG+M/G1W+DTbNmwM0w/vwE1uJyCEEP3KN8K/zV49hRHRFEQOYVrWNhd1SAghXMs3wr/NSp7tw88GYFrTKVf0RgghXM43wt+6wqfZthHjCGmoY/yvF7uwU0II4Tq+Ef6pqZYVPQkJoBTbzpjCpCh/Am+TFT5CCN/k3eHfvLzTz88y6bt8ObV1DeyJGUVyylmu7p0QQriM9y71bLu88+hRSEsjs9qfRnME0xIGurZ/QgjhQt478rezvBOTiW3vrQcgOT6q//skhBBuwnvD38GtGLeFxTJqcBjR4cH93CEhhHAf3hv+djZq08D2kUlMlVG/EMLHeW/4t1neCZA7bBSlIVLvF0II7w3/Nss7SUhgW/qTABL+Qgif573hD5Y3gJwcMJshJ4dtI8YRERzAmCERru6ZEEK4lHeHfxs7csuYPDIKfz/ZzE0I4dt8JvzrGps4eKKSiXEDXN0VIYRwOZ8J/wPHK2k0ayaOkPAXQgifCf/d+RUATBgu4S+EEL4T/gXlRIYEMHJQqKu7IoQQLuc74Z9fzoQRA+TOXUIIgY+Ef0OTmf2FlVLvF0IIK58I/4MnKqlvMpMk4S+EEICPhP8e62SvjPyFEMLCJ8I/M7+c8OAAEgYZOm8shBA+wCfCf3dBOUnDI/GTK3uFEALwgfBvbDKzr7CCCVLyEUKIFr0Kf6XUDUqpPUops1IqxeZ4olKqRim10/qxwubcNKVUplIqWyn1nHLy2stDxdXUNpil3i+EEDZ6O/LfDVwHbLRz7pDWeor1439sjr8EpAFjrB/ze9mHDmXmlwPIyF8IIWz0Kvy11vu01ge62l4pFQtEaq03a6018DpwbW/60Jnd+eUYgvwZNTjMmd9GCCE8ijNr/qOUUjuUUt8opeZYj40A8mza5FmPOc3ufMtkr2zjLIQQpwV01kAptR4YZudUutb6AwdPKwTitdalSqlpwPtKqSTAXgLrDr53GpYSEfF27snbFZNHRhE7IKRHzxVCCG/VafhrrS/u7otqreuAOuvX25RSh4CxWEb6cTZN44CCDl5nJbASICUlxeGbREd+d9X4njxNCCG8mlPKPkqpGKWUv/XrM7BM7B7WWhcClUqpWdZVPj8FHP32IIQQwkl6u9RzoVIqD5gNfKKU+sJ66nzgR6XULuAd4H+01iet5+4GXgaygUPAZ73pgxBCiO5TlkU37i8lJUVnZGS4uhtCCOFRlFLbtNYpbY97/RW+Qggh2pPwF0IIHyThL4QQPkjCXwghfJCEvxBC+CCPWe2jlCoGjrq6H900GChxdSf6mfzMvkF+Zs+RoLWOaXvQY8LfEymlMuwtsfJm8jP7BvmZPZ+UfYQQwgdJ+AshhA+S8Heula7ugAvIz+wb5Gf2cFLzF0IIHyQjfyGE8EES/kII4YMk/PuJUup+pZRWSg12dV+cTSn1lFJqv1LqR6XUWqVUlKv75CxKqflKqQNKqWyl1EOu7o+zKaVGKqW+VkrtU0rtUUr90tV96g9KKX/rbWk/dnVf+oqEfz9QSo0ELgFyXd2XfvIfYILWehJwEFji4v44hfWGRS8AlwPjgVuUUt5+67hG4Dda63HALOAeH/iZAX4J7HN1J/qShH//+Cvwv3Rwv2JvorVep7VutD78nta37vQmM4BsrfVhrXU98G/gGhf3yam01oVa6+3WryuxBOII1/bKuZRSccCVWG5C5TUk/J1MKbUAyNda73J1X1zkZ3jv3dpGAMdsHufh5UFoSymVCEwFtri4K872DJbBm9nF/ehTnd7AXXROKbUeGGbnVDrwMHBp//bI+Tr6mbXWH1jbpGMpExj7s2/9SNk55hO/3SmlwoF3gV9prStc3R9nUUpdBRRprbcppea6uDt9SsK/D2itL7Z3XCk1ERgF7LLcr544YLtSaobW+ng/drHPOfqZmyml7gCuAi7S3nsxSR4w0uZxHFDgor70G6VUIJbgN2qt33N1f5zsXGCBUuoKIASIVEqt1lrf5uJ+9Zpc5NWPlFI5QIrW2hN3BuwypdR84C/ABVrrYlf3x1mUUgFYJrQvAvKBH4BbtdZ7XNoxJ1KWUcxrwEmt9a9c3J1+ZR3536+1vsrFXekTUvMXzvA3IAL4j1Jqp1Jqhas75AzWSe17gS+wTHyu8ebgtzoXuB240Pp3u9M6KhYeRkb+Qgjhg2TkL4QQPkjCXwghfJCEvxBC+CAJfyGE8EES/kII4YMk/IUQwgdJ+AshhA/6/yeazS6n8/aYAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<Figure size 432x288 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Let us visualize the newly fitted model with the optimal lambda value here\n",
|
|
"x = np.linspace(-5, 5, 100)\n",
|
|
"new_features = get_polynomial_features(x, 10)\n",
|
|
"new_norm_feat = normalize_features(new_features, mean_train_feat, std_train_feat)\n",
|
|
"y_pred = eval(new_norm_feat, weights_ridge[min_error_idx])\n",
|
|
"\n",
|
|
"plt.plot()\n",
|
|
"plt.plot(test_data_x, test_data_y, 'or')\n",
|
|
"plt.plot(training_data_x, training_data_y, 'ob')\n",
|
|
"plt.plot(x, y_pred)\n",
|
|
"plt.legend([\"test_data\", \"training_data\", \"inference\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"### 1.4) Error Plot (1 Point)\n",
|
|
"In the lecture we have seen and analyzed the plot of polynomial degrees \n",
|
|
"against the error (slide 47).\n",
|
|
"Similarly, now please analyze the relationship between the error and the \n",
|
|
"different values of $\\lambda$, as well as the reason behind it.\n",
|
|
"\n",
|
|
"Hint: Do not forget that we are in log space. Small changes in the y-axis mean high differences in the error values. <br><br>\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"$\\lambda$ is regularization term such that the model cannot fit the training data perfectly anymore. Increasing $\\lambda$ strongly encourages the weight values to decay towards zero, unless supported by the data. In our case, a polynomial feature with degree 10 is highly overfitting to the training data and, therefore, with higher $\\lambda$ values (10-15), we can punish overfitting, which leads to lower MSE values. Increasing the $\\lambda$ value leads to an decrease in the error due to the higher punishment of high weight values and tackles though this the high polynomial degree. Of course, with low $\\lambda$ values, the MSE on the training data is low because there is no punishment of overfitting.<br>\n",
|
|
"On the other hand, if $\\lambda$ is increased too far, the introduced penalty will force the regression to become to simple and underfit the data."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Probability Basics and Linear Classification\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## First Example (Two Moons)\n",
|
|
"\n",
|
|
"Let us start by loading a very simple toy dataset, the \"two moons\"."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<matplotlib.legend.Legend at 0x7f0dc0e6bf40>"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAhuUlEQVR4nO3df5BV5Z3n8feXtllax7JBMMFuCGRkrQQFsbrUWaY0kfiLioG4BnWyht1NFXFHKxOzS8msVY7rjhUyZmPGHROLqBWtmolBo0AmumpgtqwxZZYmKkoMkTE6dEOpASE/RAX87h/3XLncPufec/uec+4593xeVVTfPvec7qev1/s9z/f5Ps9j7o6IiJTXhE43QEREOkuBQESk5BQIRERKToFARKTkFAhERErumE43YDymTp3qs2bN6nQzREQKZcuWLb9x92n1xwsZCGbNmsXw8HCnmyEiUihm9lrYcaWGRERKToFARKTkFAhEREqukGMEIiJpO3jwICMjI7zzzjudbkrLJk2axODgIL29vbHOVyAQEQkxMjLC8ccfz6xZszCzTjcnNndnz549jIyMMHv27FjXKBBIbq17dpTbHt/Orn0HOLm/j5UXncrSBQOdbpaUxDvvvFO4IABgZpx44om8+eabsa9RIJBcWvfsKH/58AscOHgYgNF9B/jLh18AUDCQzBQtCFS12m4NFksu3fb49g+CQNWBg4e57fHtHWqRSPdSIJBc2rXvQEvHRcrg3Xff5YorruCUU07h7LPP5tVXX03k5yoQSEese3aUhas3MXvVj1m4ehPrnh096vmT+/tCr4s6LlIG99xzD5MnT2bHjh1cf/313HDDDYn8XAUCyVw1/z+67wDOkfx/bTBYedGp9PX2HHVdX28PKy86NePWisTT7OZmPO6//37mzZvH/Pnzufrqq1m/fj3Lly8H4PLLL2fjxo0kscukBoslc43y/9WB4OpXVQ1JEaRR3LBt2zZuvfVWnn76aaZOncrevXs599xzmTFjBgDHHHMMJ5xwAnv27GHq1KlttV+BQDIXN/+/dMGAPvilEOLc3LRq06ZNXH755R98yE+ZMiX07j+JyiYFAsncyf19jIYEg2r+Py/zB/LSDsm/NIob3H3Mh/zg4CA7d+5kcHCQQ4cOsX//fqZMmTLu31GlMQIZl3byoY3y/2HjB1/5wXMsuOWJRHKuccUZxxCpSqO4YdGiRaxdu5Y9e/YAsHfvXj7zmc9w3333AfDQQw9x/vnnq0cgndFuPrRR/n/h6k1jutgAb719MNMJZWl09aV7rbzo1KP+n4D2ixvmzp3LjTfeyHnnnUdPTw8LFizgrrvu4uqrr+aUU05hypQpPPDAA0k0X4FAWpfEh2RU/r9RVzrLD2LNY5BWpFXcsHz58g+qhKoefPDBtn5mGAUCaVmaH5JR4wdJ/o5aUeMAzcYx4vwMKZciFzdojEBaluZkr7Dxg6R/R1WjcYC48xg0liDdIJFAYGb3mtkbZvZixPNmZneY2Q4z22pmZ9Y8d7GZbQ+eW5VEeyQd1QHi0X0HqB+eSmqy19IFA3ztstPp7xu7jnrSE8qapbi+dtnpDPT3YcBAfx9fu+z0MXd8WhNJukFSqaHvAX8H3B/x/CXAnODf2cB3gLPNrAe4E7gAGAE2m9kGd/9FQu2SFjRKcdQPEDtgwdeBhNMh1S52EimXqJ+x7tnRyBRUNf0Up6uvsQTpBokEAnd/ysxmNThlCXC/V2ZDPGNm/WY2HZgF7HD3VwDM7IHgXAWCjDWrBAq7860GgadXnZ9Km9rNuUb9TcOv7eWHW6JTN62kn1oZSxDJq6zGCAaAnTXfjwTHoo6PYWYrzGzYzIZb2XBB4mmW4ijinW/U3/T9n+0MLVGF1tNPWhNJukFWgSBsxoM3OD72oPsadx9y96Fp06Yl2jhp/kFfxNVAo/6mww0W6QobB2gk7liCSBKeeuopzjzzTI455hgeeuihxH5uVuWjI8CMmu8HgV3AxIjjkrFmKY40JswkpdUS0B6z0GAw0N83rg/wIpcNSrHMnDmT733ve3zjG99I9Odm1SPYAHwhqB46B9jv7ruBzcAcM5ttZhOBK4NzJWPNUhx5vfMdTwnoVWfPUDpHkrd1Ldx+GtzcX/m6dW3bP7J+GepZs2Yxb948JkxI9qM7kR6BmX0f+AQw1cxGgL8CegHc/S7gUWAxsAN4G/hPwXOHzOw64HGgB7jX3bcl0SZpTZyZkXm88200tlEdxA77m4Y+MkWTwCQ5W9fCj74MB4Me6P6dle8B5i0b148MW4Y6LUlVDV3V5HkHro147lEqgUI6LI8f9M00G9uI+puK+LdKjm285UgQqDp4oHJ8nIEgbBnqtGiJCUlclksuqHxTcmH/SGvHYwhbhjotWmJCEpX1kgvdUL6ZxhaHkrETBls7HkPYMtRpUSCQRGW95EJeB7Hj0lpFXWLRTdBb1wvt7ascH6faZajnz5/PV7/6VTZv3szg4CAPPvggX/rSl5g7d26bDa9QaqiE0kzddGLiWZHz/dr3oEtUxwE23lJJB50wWAkC4xwfqApbhnpkZPzppigKBCWTxibbtZSzb00RZ2xLhHnL2v7g7xSlhkom7dRNN+Tss1TEGdvSfRQISibtO9Ci5+yzpsCZb95gOZI8a7XdSg2VTBapmyLn7LOW1haH0r5JkyaxZ88eTjzxxMzKOJPg7uzZs4dJkybFvkaBoGTyvGZQWSlw5tPg4CAjIyMUcbXjSZMmMTgYv3RVgaBkdAcqEk9vby+zZ8/udDMyoUBQQroDFZFaCgQZynLphbR0w98gIkdTIMhI2vX7WeiGv0FExlL5aEayXnohDd3wN4jIWOoRZKQbZpB2w9+QNaXSpAjUI8hIN8wg7Ya/IUtaUE6KQoEgI90wg7Qb/oYsKZUmRaHUUEa6oX6/G/6GLCmVJkWhQJChuPX7ec4raw5CfFqJVYpCgaCDwj7wAZVodgkt5yFFkUggMLOLgb8FeoC73X113fMrgc/X/M6PAdPcfa+ZvQr8DjgMHHL3oSTalHdRNfn/5pgJ2qikS6SRSstzb1GKq+1AYGY9wJ3ABcAIsNnMNrj7L6rnuPttwG3B+ZcC17t77Qacn3T337TbliKJGkisP1alvHIxJZlK04Q+SUsSVUNnATvc/RV3fw94AFjS4PyrgO8n8HsLrdUPduWVRVVIkpYkAsEAsLPm+5Hg2BhmdixwMfDDmsMOPGFmW8xsRdQvMbMVZjZsZsNFXBa2XtQH++Rje1WiKaFUhSRpSSIQhO3YELU9zqXA03VpoYXufiZwCXCtmZ0bdqG7r3H3IXcfmjZtWnstzoGomvy/unSudviSUJrQJ2lJYrB4BJhR8/0gsCvi3CupSwu5+67g6xtm9giVVNNTCbQr15oNJOqDvxxaGfxVFZKkJYlAsBmYY2azgVEqH/Z/Vn+SmZ0AnAf8h5pjxwET3P13weMLgVsSaFMhqCa/3Fod/NWEPkmLJbE5s5ktBr5FpXz0Xne/1cyuAXD3u4Jz/iNwsbtfWXPdR4FHgm+PAf7B3W9t9vuGhoZ8eHi47XaLZCXszv+2x7eHTjiDSkpQH/KSNDPbElain0ggyJoCgRRJ/Z0/VFI6UaXCtedofEiSFBUItOicSMqiyj57LKzO4uhzVBoqWVAgEElZVHnnYfcxlWNxrxVJkgKBSMqiyjurpcEDDco/VRoqWVAgEElZo30cli4Y4OlV5/OtK87QRELpGK0+WgBaaKzY4pR9qjRUOklVQzkXVXGiahIRaZWqhgpKC43V2LoWbj8Nbu6vfN26ttMtEukKSg11SNx0jxYaC2xdCz/6MhwM/u79OyvfA8xb1rl2iXQB9Qg6oJruGd13AOfI0gLrnh0dc64WGgtsvOVIEKg6eKByXETaokDQAa2kexpVnJTK/pHWjotIbEoNdUAr6R5VkwROGKykg8KOi0hbFAg64OT+vtDFxqLSPVqlFFh009FjBAC9fZXjItIWpYY6QOmecZi3DC69A06YAVjl66V3aKBYJAHqEXRAs3SPJpBFmLdMH/wiKVAgaKL6oTy67wA9Zhx2D10rvtUP76h0T6ublYhISWxdW6mS2z9SGRtbdFNiN0ZKDTVQW+YJldUiYWy5ZyvloM1oApmIjFGdR7N/J+BH5tEkNKlSgaCBsA/lqtoP5yQ/vDWBTETGSHkejQJBA80+fKvPJ/nhrQlkIjJGyvNoFAgaaPbhW30+yQ9vVRSJyBhR82USmkejQNBA2IdyVe2Hc5If3ksXDHywWYlxZPMSDRSLlNiimyrzZmolOI8mkaohM7sY+FugB7jb3VfXPf8JYD3w6+DQw+5+S5xrO6m2zLNR1VCc2b+tVBVpApmIHKVaHZRS1VDb+xGYWQ/wK+ACYATYDFzl7r+oOecTwH9z90+3em2You1HoD0FRCQP0tyP4Cxgh7u/4u7vAQ8ASzK4tjBUEioieZZEamgAqF0NbAQ4O+S8PzGz54FdVHoH21q4FjNbAawAmDlzZgLNzo5KQqUT1j07yv/40TbeevsgAP19vdz8mbnqhWYpxUlgSUqiR2Ahx+rzTT8HPuLu84H/Daxr4drKQfc17j7k7kPTpk0bb1s7QiWhkrV1z46y8qHnPwgCAPsOHGTlg8+Pa6KjjEPKk8CSlEQgGAFm1Hw/SOWu/wPu/lt3/33w+FGg18ymxrm2G6gkVLJ22+PbOXh47D3VwfddKcmsFGgzpSRSQ5uBOWY2GxgFrgT+rPYEM/sw8Lq7u5mdRSUA7QH2Nbu2G2hPAclao7Rjo+e04GGCCrSZUtuBwN0Pmdl1wONUSkDvdfdtZnZN8PxdwOXAfzGzQ8AB4EqvlCuFXttum/JIJaGSpag9L6rPhdGChwkr0GZKiUwoc/dH3f3fuvsfu/utwbG7giCAu/+du8919/nufo67/7TRtSLSnpUXnUpvz9ghuN4JFpmSVHVbwlKeBJYkzSwW6UJLFwxw2+XzmXxs7wfH+vt6ue1z8yPv7lXdlrACbaak/QhEulSr6chWt1CVGAqymZJ6BCICqLqtzNQjEBFA1W1lpkAQk8rqpAxU3VZOCgQxqKxORLqZxghiiCqr+8oPnmPh6k2asp+0rWvh9tPg5v7K1xxOyRdJRE7e6+oRxNCofE69g4RV12epTs2vrs8Chai+EIktR+919QhiaFY+p0k3CSrQ+iwibcnRe12BIIZGW1ZWadJNQhqtz5KTbrRIInK0FpECQQy1+whH0aSbhEStw9I3uTBL+krJjPcGJeUN6VuhQBDT0gUDPL3qfL51xRljege9PcYf3j3E7FU/1uBxu6LWZ4HcdKO70bpnR1m4epPew61qZ8+BHK1FpMHiFtVPuuk/tpffv3OIfQcqG4Bo8LhNUZt0P7wi/PwcLulbNCqPbkH9jmPv/SH6BqXZgG/KG9K3ou3N6zshT5vXL1y9KXR9loH+Pp5edX4HWtSlbj8tYknfGXD9i9m3p4voPRxTfZVPQwY370u7RS1Lc/P6UtOKjRnJUTe62+g9HFNYlU+UHO450IgCQZu0H3FGCrSkb9HoPRxT3DRkAW9QNEbQppUXnXpUfhW0YmNqCrKkb1aSWv9K7+GYonYc65sCE4/reJ6/HQoEbdKKjdIJYQO81//gOYZf28tfLz29pZ+l93BMi24aO0bQ2weXfH38H/z1g88aLI4vT4PFMg45efMXWdQAL8DkY3vZ9/ZBfaCnIcn3btjgc29fqinPqMFi9QgkWzlaX6XIGg3kvvW2SplTk2R6stESE0Vca8jMLjaz7Wa2w8xWhTz/eTPbGvz7qZnNr3nuVTN7wcyeM7OO3+ZrYk3Kmq2vomUkYok7kKt1sHKsm5aYMLMe4E7gEuDjwFVm9vG6034NnOfu84D/Caype/6T7n5GWJclS9W86+i+AzhH7qgUDBLUbC0hLSMRy8qLTsVinqsy0JzqsiUmzgJ2uPsr7v4e8ACwpPYEd/+pu78VfPsMkMsi26h9B3RHlaBGb/4crcaYd0sXDPD5c2bGCgYqA82pHM2NSSIQDAC1NVUjwbEoXwQeq/negSfMbIuZRawjkA1NrMlAozd/jrrKRfDXS0/n9ivOYKC/DwP6+3rp7Tk6NKgMNMdyNDcmicHisJuS0FIkM/sklUDwpzWHF7r7LjM7CXjSzH7p7k+FXLsCWAEwc+bM9lsd4uT+vtBKDN1RJajR+iobb4lYRiKXHchcqN9jWHtrF0xO5sYkEQhGgBk13w8Cu+pPMrN5wN3AJe6+p3rc3XcFX98ws0eopJrGBAJ3X0MwtjA0NJRKzasm1mQk6s0fVaddsFmanaTN52U8kkgNbQbmmNlsM5sIXAlsqD3BzGYCDwNXu/uvao4fZ2bHVx8DFwIdW0Gsdt8Bo7Lo1tcuO13/Y2UlR11lkTJJZEKZmS0GvgX0APe6+61mdg2Au99lZncD/x54LbjkkLsPmdlHgUeCY8cA/+Dutzb7fZpQJpIcpZPKI2pCmWYWi5RY/VIVUEmHqifcnbQMtYiMoZJpAQUCkVJTybSAAoFIqWkvgjZ1yZIoCgQiJbbyolPp6+056phKpmPqoiVRFAhESkwl023ooiVRtAy1SMlpEto4ddGSKAoECVI9tuSN3pMpitq6soBLoig1lBAtYS15o/dkynK0emi7FAgSonpsyRu9J1PWRUuiKDWUENVjS960+p5UGmkccrJ6aLvUI0iI6rElb1p5TyqNVG4KBAlRPbbkTSvvSaWRyk2poYRUu9DqWo/T1rXhm9XIuLXynlRqs9wUCBKkeuxxqs7QrE7O2b8THl4B//oMfPqbnW1bwcV9T2p3vnJTakg6L2yGJg7D9xZyun4RKbUZoUvWEmpGgUA6L3Imphdyun4RaamJEF20llAzSg1J50XN0IRCTtcvKqU26zRaS6jLxq/UI5DOW3QTYOHPFXC6vnSJLlpLqBkFAum8ectg6D8zJhg0m65fkvytdEjUTUgX3pwoEEg+fPqbcNma+NP1S5S/lQ7porWEmkkkEJjZxWa23cx2mNmqkOfNzO4Int9qZmfGvVZKZN4yuP5FuHlf5WujPGwXrQUvOdVFawk10/ZgsZn1AHcCFwAjwGYz2+Duv6g57RJgTvDvbOA7wNkxrxUZK27+VhPVpB1dspZQM0n0CM4Cdrj7K+7+HvAAsKTunCXA/V7xDNBvZtNjXisyVpz8rdJHIrEkEQgGgNrav5HgWJxz4lybiHXPjrJw9SZmr/oxC1dv0mJaRRcnf6v0kXRCAYsYkphHEFb35zHPiXNt5QeYrQBWAMycObOV9n2wsmJ1Ua3qyoqA6qaLqtpdb5T2KVH5n+RE2HIpP/py5XGOU0xJBIIRYEbN94PArpjnTIxxLQDuvgZYAzA0NBQaLKI0WllRgaDAmuVvu2grQcnYeMeWCjoJLYnU0GZgjpnNNrOJwJXAhrpzNgBfCKqHzgH2u/vumNe2TSsrllSJyv8kQVvXwvprjx5bWn9tvBRPQXuhbQcCdz8EXAc8DrwErHX3bWZ2jZldE5z2KPAKsAP4LvDnja5tt031tGlMSZWo/E8S9NgNcPi9o48dfq9yvCpqHKCgk9DMvaUsSy4MDQ358PBw7PPrxwigsrJi6RfVEpGxbj6hwXP7x44DQKWneekdlcdRz+XgBsTMtrj7UP3xUiw6F3eDDu3ZKkWn93AGGo0DXP/ikXMKNHelFIEAmq+sqMoiKTq9hxPSNwUO7A0/Ds3HAQo4CU1rDQW0Z2sOFbAeu5P0Hk7IJV+HCb1HH5vQWzkOjccBCvqeVSAIqLIoZzQruGV6Dydk3jJY+u2jiwyWfvvIXX5UNdqcCwv7nlUgCKiyKGc0K7hleg8nqNECiFHVaC8/Udj3rAJBQHu25ky79dgF7aK3Q+/hDIUFioLOIYASDRY3E7eySDLSzqzggk7zb5fewx1W4JnspZhHIAXUqFa72Yf57adF/A8540h5n0jS2nnPZiRqHoFSQ5JP7cwKHk8XvYSpJElYgWeyKzUk+TXeeuxWu+glTSVJCgo4hwDUI5Bu1Opic6pQkpJTIJDu02oXvcDVHiJJUGpIulN9F706BhC2/kuBqz1EkqAegXS/ZrOUtW+BlJwCgXS/ZmMABa72EEmCUkPS/eKMARS02kMkCQoE0p1q95y1CeCHx56jMQARQIFAulH9vICwIKAxAGlkvJvXF5QCgXSfsDEBAOsBf78U/2NLG0o4wVCDxdJ9osYE/HBQKjpSCRZaRkLClHCCoQKBdJ/I3L8VctMQyVgJJxi2FQjMbIqZPWlmLwdfJ4ecM8PM/snMXjKzbWb2FzXP3Wxmo2b2XPBvcTvtEQHC5wVgQN1Ku11+lycNNFpksNFWlF2q3R7BKmCju88BNgbf1zsE/Fd3/xhwDnCtmX285vnb3f2M4N+jbbZHJHxeQH0QqOriuzyJoAmGY7QbCJYA9wWP7wOW1p/g7rvd/efB498BLwHaKUPSVb+D1Akzws9rdpen5am7jyYYjtFu1dCH3H03VD7wzeykRieb2SxgAfCzmsPXmdkXgGEqPYe3Iq5dAawAmDlzZpvNltJZdFP4piGN7vJKWD1SCppgOEbTHoGZ/cTMXgz5t6SVX2RmfwT8EPiKu/82OPwd4I+BM4DdwP+Kut7d17j7kLsPTZs2rZVfLTK+u7wSVo9krhM9rhKOATTTtEfg7p+Kes7MXjez6UFvYDrwRsR5vVSCwN+7+8M1P/v1mnO+C/xjK40XaUmrd3klrB7JVKd6XOPpHXa5dscINgDLg8fLgfX1J5iZAfcAL7n7N+uem17z7WcBbSgr+aE7x3R1qsdVwjGAZtodI1gNrDWzLwL/CnwOwMxOBu5298XAQuBq4AUzey647r8HFUJ/Y2ZnUCnpeBX4UpvtEUlOFneOJVvK4Cid7HGVbAygmbYCgbvvARaFHN8FLA4e/zOVIu6w669u5/eLpKr6QZHWB3XZB6O1IVBuaK0hkUbSvHNslBrp1kBQ2wPqmww9E+Hwe0eeL3muvlO0xIRIp+RtMDrtCp76iVwH9oI79E1BufrOUo9ApFPylBrJIk0V1gN6/yBMPA5u+HUyv0PGRT0CkU7J01IGWVTw5K0HJB9QIBBJU6N0S57KGLP4kFY5bm4pNSSSljjplryUMWaRptJErtxSj0AkLUVaoiKLNFWWPSAtFtgS9QhE0lKknHjacyZqf0/aPaC4A99lnsxXR4FAJC15qgqKIy9pqnbFmZ9R9sl8dZQaEklKfTpizoX5qQpqVZFTK3F6YkVK22VAgUAkCWG7Xg3fA0wo3oSpZjt45V2c6qQipe0yoNSQSBLC7jABDv4BeB8uW5P/AFAVdbf8yDXw8Ioj+fTquXnLscepTipa2i5l6hGIJKHRnWTRUg5Rf4sf5oMewvprYd2f57PXEKc6KU+T+XJAPQKRJETdYVYVKeXQ7G+BoxeKq8rTgnnNBr6zqpIqCAUCkSSEpSNqFSnl0OxvaSTPAS+sXPR67YUFCgQiyajeST52Q2VVzVpFSznU3y3bhCAtFENeA57KRRvSGIFIUuYtq6yiedl387F+UDvmLavcLd+8Dz5719h8es9EmNB79LE8BzyVizakHoFI0rplYlZVVD497Fhe/26VizakQCAiR0QtuxAV3PL6wV9P5aINKTUkIhVFn0jWiMpFG2orEJjZFDN70sxeDr5OjjjvVTN7wcyeM7PhVq8XkQx0cx49T3s/5FC7qaFVwEZ3X21mq4Lvb4g495Pu/ps2rhcpnyxXyOz2PHq3jd0kqN3U0BLgvuDxfcDSjK8X6ZysN3tPO1WjHcRKq91A8CF33w0QfD0p4jwHnjCzLWa2YhzXY2YrzGzYzIbffPPNNpst0qYsPqSzTtUoj15aTVNDZvYT4MMhT93Ywu9Z6O67zOwk4Ekz+6W7P9XC9bj7GmANwNDQkLdyrUji4qx5366sUzVadqG0mgYCd/9U1HNm9rqZTXf33WY2HXgj4mfsCr6+YWaPAGcBTwGxrhfJnaw2e8+65FF59FJqNzW0AVgePF4OrK8/wcyOM7Pjq4+BC4EX414vkktZ5NPTTtUUefMZSVS7gWA1cIGZvQxcEHyPmZ1sZo8G53wI+Gczex74f8CP3f3/NLpeJPeKvtl7nuYMKCB1nLkXL90+NDTkw8PDzU8USVORNz+//bSItNOMbFfkrF8MDioBVTX+qTCzLe4+VH9cS0yIjFeR8+njGeNII/BlMeguTWmJCZEyanWMI61UUrdPYisIBQKRMmp1jCOtOQ2axJYLCgQiZdTqQHRad+6axJYLGiMQKatWxjjSmtOgSWy5oEAgIs2F7WOc1J17kQfdu4RSQyLSnJZx7mrqEYhIPLpz71rqEYiIlJwCgYhIySkQiIiUnAKBiEjJKRCIiJRcIVcfNbM3gddS/jVTgd+k/DvSUNR2g9reCUVtN6jt4/ERd59Wf7CQgSALZjYctlxr3hW13aC2d0JR2w1qe5KUGhIRKTkFAhGRklMgiLam0w0Yp6K2G9T2Tihqu0FtT4zGCERESk49AhGRklMgEBEpOQWCgJl9zsy2mdn7ZhZZ1mVmF5vZdjPbYWarsmxjRHummNmTZvZy8HVyxHmvmtkLZvacmQ1n3c66tjR8Da3ijuD5rWZ2ZifaWS9Guz9hZvuD1/g5M8vNNltmdq+ZvWFmL0Y8n9fXvFm78/yazzCzfzKzl4LPlr8IOScfr7u7619lnORjwKnA/wWGIs7pAf4F+CgwEXge+HiH2/03wKrg8Srg6xHnvQpMzcHr3PQ1BBYDjwEGnAP8rCDt/gTwj51ua0T7zwXOBF6MeD53r3nMduf5NZ8OnBk8Ph74VV7f6+oRBNz9JXff3uS0s4Ad7v6Ku78HPAAsSb91DS0B7gse3wcs7VxTYonzGi4B7veKZ4B+M5uedUPr5PG/fWzu/hSwt8EpeXzN47Q7t9x9t7v/PHj8O+AlYKDutFy87goErRkAajduHWHsf9isfcjdd0PljQecFHGeA0+Y2RYzW5FZ68aK8xrm8XWO26Y/MbPnzewxM5ubTdMSkcfXPK7cv+ZmNgtYAPys7qlcvO6l2qHMzH4CfDjkqRvdfX2cHxFyLPX620btbuHHLHT3XWZ2EvCkmf0yuNvKWpzXsCOvcxNx2vRzKmu5/N7MFgPrgDlpNywheXzN48j9a25mfwT8EPiKu/+2/umQSzJ/3UsVCNz9U23+iBFgRs33g8CuNn9mU43abWavm9l0d98ddCnfiPgZu4Kvb5jZI1RSHZ0IBHFew468zk00bVPt/+Tu/qiZfdvMprp7ERZGy+Nr3lTeX3Mz66USBP7e3R8OOSUXr7tSQ63ZDMwxs9lmNhG4EtjQ4TZtAJYHj5cDY3o2ZnacmR1ffQxcCIRWYWQgzmu4AfhCUFFxDrC/mv7qoKbtNrMPm5kFj8+i8v/XnsxbOj55fM2byvNrHrTrHuAld/9mxGn5eN07PbKel3/AZ6lE53eB14HHg+MnA4/WnLeYyuj/v1BJKXW63ScCG4GXg69T6ttNpdLl+eDftk63O+w1BK4BrgkeG3Bn8PwLRFRx5bDd1wWv7/PAM8C/63Sba9r+fWA3cDB4n3+xIK95s3bn+TX/Uyppnq3Ac8G/xXl83bXEhIhIySk1JCJScgoEIiIlp0AgIlJyCgQiIiWnQCAiUnIKBCIiJadAICJScv8f6+U9UJfWODMAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<Figure size 432x288 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"from typing import Tuple, Callable\n",
|
|
"\n",
|
|
"data = dict(np.load(\"two_moons.npz\", allow_pickle=True))\n",
|
|
"samples = data[\"samples\"]\n",
|
|
"labels = data[\"labels\"]\n",
|
|
"\n",
|
|
"c0_samples = samples[labels == 0] # class 0: all samples with label 0\n",
|
|
"c1_samples = samples[labels == 1] # class 1: all samples with labe 1 \n",
|
|
"\n",
|
|
"plt.figure(\"Data\")\n",
|
|
"plt.scatter(x=c0_samples[:, 0], y=c0_samples[:, 1], label=\"c0\")\n",
|
|
"plt.scatter(x=c1_samples[:, 0], y=c1_samples[:, 1], label=\"c1\")\n",
|
|
"plt.legend()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let us also define some plotting utility"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def draw_2d_gaussian(mu: np.ndarray, sigma: np.ndarray, plt_std: float = 2, *args, **kwargs) -> None:\n",
|
|
" (largest_eigval, smallest_eigval), eigvec = np.linalg.eig(sigma)\n",
|
|
" phi = -np.arctan2(eigvec[0, 1], eigvec[0, 0])\n",
|
|
"\n",
|
|
" plt.scatter(mu[0:1], mu[1:2], marker=\"x\", *args, **kwargs)\n",
|
|
"\n",
|
|
" a = plt_std * np.sqrt(largest_eigval)\n",
|
|
" b = plt_std * np.sqrt(smallest_eigval)\n",
|
|
"\n",
|
|
" ellipse_x_r = a * np.cos(np.linspace(0, 2 * np.pi, num=200))\n",
|
|
" ellipse_y_r = b * np.sin(np.linspace(0, 2 * np.pi, num=200))\n",
|
|
"\n",
|
|
" R = np.array([[np.cos(phi), np.sin(phi)], [-np.sin(phi), np.cos(phi)]])\n",
|
|
" r_ellipse = np.array([ellipse_x_r, ellipse_y_r]).T @ R\n",
|
|
" plt.plot(mu[0] + r_ellipse[:, 0], mu[1] + r_ellipse[:, 1], *args, **kwargs)\n",
|
|
"\n",
|
|
"# plot grid for contour plots\n",
|
|
"plt_range = np.arange(-1.5, 2.5, 0.01)\n",
|
|
"plt_grid = np.stack(np.meshgrid(plt_range, plt_range), axis=-1)\n",
|
|
"flat_plt_grid = np.reshape(plt_grid, [-1, 2])\n",
|
|
"plt_grid_shape = plt_grid.shape[:2]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 2): Classification using Generative Models (Naive Bayes Classifier)\n",
|
|
"\n",
|
|
"We first try a generative approach, the Naive Bayes Classifier.\n",
|
|
"We model the class conditional distributions $p(\\boldsymbol{x}|c)$ as Gaussians, the class prior $p(c)$ as\n",
|
|
"Bernoulli and apply Bayes rule to compute the class posterior $p(c|\\boldsymbol{x})$.\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"As a small recap, recall that the density of the Multivariate Normal Distribution is given by\n",
|
|
"\n",
|
|
"$$ p(\\boldsymbol{x}) = \\mathcal{N}\\left(\\boldsymbol{x} | \\boldsymbol{\\mu}, \\boldsymbol{\\Sigma} \\right) = \\dfrac{1}{\\sqrt{\\det \\left(2 \\pi \\boldsymbol{\\Sigma}\\right)}} \\exp\\left( - \\dfrac{(\\boldsymbol{x}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x}-\\boldsymbol{\\mu})}{2}\\right) $$\n",
|
|
"\n",
|
|
"and we already saw how to implement it in the python introduction"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def mvn_pdf(x: np.ndarray, mu: np.ndarray, sigma: np.ndarray) -> np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" Density of the Multivariate Normal Distribution\n",
|
|
" :param x: samples, shape: [N x dimension]\n",
|
|
" :param mu: mean, shape: [dimension]\n",
|
|
" :param sigma: covariance, shape: [dimension x dimension]\n",
|
|
" :return p(x) with p(x) = N(mu, sigma) , shape: [N] \n",
|
|
" \"\"\"\n",
|
|
" norm_term = 1 / np.sqrt(np.linalg.det(2 * np.pi * sigma))\n",
|
|
" diff = x - np.atleast_2d(mu)\n",
|
|
" exp_term = np.sum(np.linalg.solve(sigma, diff.T).T * diff, axis=-1)\n",
|
|
" return norm_term * np.exp(-0.5 * exp_term)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Practical Aspect:** In practice you would never implement it like that, but stay\n",
|
|
"in the log-domain. Also for numerically stable implementations of the multivariate normal density the symmetry and\n",
|
|
"positive definitness of the covariance should be exploited by working with it's Cholesky decomposition."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The maximum likelihood estimator for a Multivariate Normal Distribution is given by\n",
|
|
"$$ \\boldsymbol{\\mu} = \\dfrac{1}{N} \\sum_{i}^N \\boldsymbol{x}_i \\quad \\quad \\boldsymbol{\\Sigma} = \\dfrac{1}{N} \\sum_{i}^N (\\boldsymbol{x}_i - \\boldsymbol{\\mu}) (\\boldsymbol{x}_i - \\boldsymbol{\\mu})^T. $$\n",
|
|
"\n",
|
|
"This time, before we use it, we are going to derive it:\n",
|
|
"\n",
|
|
"### Exercise 2.1): Derivation of Maximum Likelihood Estimator (5 Points):\n",
|
|
"\n",
|
|
"Derive the maximum likelihood estimator for Multivariate Normal distributions, given above.\n",
|
|
"This derivations involves some matrix calculus.\n",
|
|
"Matrix calculus is a bit like programming, you google the stuff you need and then plug it together in the right order.\n",
|
|
"Good resources for such rules are the \"matrix cookbook\" (https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf) and the Wikipdia article about matrix calculus\n",
|
|
"(https://en.wikipedia.org/wiki/Matrix_calculus ). State all rules you use explicitly\n",
|
|
"(except the ones given in the hints below). \n",
|
|
"\n",
|
|
"**Remark** There are different conventions of how to define a gradient (as column-vector or row-vector). This results in different ways to write the Jacobian and thus different, usually transposed, matrix calculus rules:\n",
|
|
"- In the lecture we define the gradient as column-vector \n",
|
|
"- In the Wikipedia article this convention is referred to as \"Denominator Layout\". It also contains a nice explanation of the different conventions for the gourmets among you ;) \n",
|
|
"- The Matrix Cookbook uses the same convention (gradient as column vector)\n",
|
|
"- Please also use it here\n",
|
|
"\n",
|
|
"**Hint** Here are two of those rules that might come in handy\n",
|
|
"\n",
|
|
"$\\dfrac{\\partial\\log\\det(\\boldsymbol{X})}{\\partial \\boldsymbol{X}} = \\boldsymbol{X}^{-1}$\n",
|
|
"\n",
|
|
"$\\dfrac{\\partial \\boldsymbol{x}^T\\boldsymbol{A}\\boldsymbol{x}}{\\partial \\boldsymbol{x}} = 2 \\boldsymbol{A}\\boldsymbol{x}$ for symmetric matrices $\\boldsymbol{A}$ (hint hint: covariance matrices are always\n",
|
|
"symmetric)\n",
|
|
"\n",
|
|
"There is one missing to solve the exercise. You need to find it yourself. (Hint hint: Look in the matrix cookbook, chapter 2.2)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Solution\n",
|
|
"\\begin{align}\n",
|
|
" p(\\boldsymbol{x_i}) \n",
|
|
" &= \\mathcal{N}\\left(\\boldsymbol{x_i} | \\boldsymbol{\\mu}, \\boldsymbol{\\Sigma} \\right)\\\\\n",
|
|
" &= \\dfrac{1}{\\sqrt{\\det \\left(2 \\pi \\boldsymbol{\\Sigma}\\right)}} \\exp\\left( - \\dfrac{(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})}{2}\\right)\n",
|
|
" &= \\dfrac{1}{\\sqrt{(2 \\pi)^{K} |\\boldsymbol{\\Sigma}}|} \\exp\\left( - \\dfrac{(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})}{2}\\right)\\\\\n",
|
|
"\\end{align}\n",
|
|
"\n",
|
|
"With K as the dimensions of the quadratic matrix $\\Sigma$\n",
|
|
"\n",
|
|
"##### calculate the likelihood\n",
|
|
"\\begin{align}\n",
|
|
" \\text{lik}(\\boldsymbol{\\theta};D) \n",
|
|
" &= \\prod_i p(x_i)\\\\\n",
|
|
" &= \\prod_i^{N} \\dfrac{1}{\\sqrt{(2 \\pi)^{K} |\\boldsymbol{\\Sigma}}|} \\exp\\left( - \\dfrac{(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})}{2}\\right)\\\\\n",
|
|
" &= \\prod_i^{N} 2\\pi^{-\\frac{K}{2}} |\\boldsymbol{\\Sigma}|^{-\\frac{1}{2}} \\exp\\left( - \\dfrac{(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})}{2}\\right)\\\\\n",
|
|
" &= (2\\pi)^{-\\frac{NK}{2}} |\\Sigma|^{-\\frac{N}{2}} \\exp\\left( - \\dfrac{\\sum_i^{N}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})}{2}\\right)\\\\\\\\\n",
|
|
" \\log\\text{lik}(\\boldsymbol{\\theta};D) \n",
|
|
" &= \\sum_i \\log p(x_i)\\\\\n",
|
|
" &= \\log(-\\frac{NK}{2}(2\\pi))-\\frac{N}{2}\\log(|\\Sigma|) - \\frac{1}{2} \\sum_i^{N}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu}) \\\\\n",
|
|
"\\end{align}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"##### derive $\\mu$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\\begin{align}\n",
|
|
" \\frac{\\partial \\log\\text{lik}(\\boldsymbol{\\theta};D)}{\\partial\\mu}\n",
|
|
" &= \\log(-\\frac{NK}{2}(2\\pi))-\\frac{N}{2}\\log(|\\Sigma|) - \\frac{1}{2} \\sum_i^{N}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})\\\\\n",
|
|
" &= \\frac{\\partial}{\\partial\\mu}-\\dfrac{1}{2}\\sum_i^N (\\boldsymbol{x_i}^T-\\boldsymbol{\\mu}^T) \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})\\\\\n",
|
|
" &= \\frac{\\partial}{\\partial\\mu}-\\dfrac{1}{2}\\sum_i^N \\boldsymbol{x_i}^T\\boldsymbol{\\Sigma}^{-1}\\boldsymbol{x_i} - \\boldsymbol{x_i}^T\\boldsymbol{\\Sigma}^{-1}\\boldsymbol{\\mu} - \\boldsymbol{\\mu}^T\\boldsymbol{\\Sigma}^{-1}\\boldsymbol{x_i} + \\boldsymbol{\\mu}^T\\boldsymbol{\\Sigma}^{-1}\\boldsymbol{\\mu}\\\\\n",
|
|
" &= \\frac{\\partial}{\\partial\\mu}-\\dfrac{1}{2}\\sum_i^N \\boldsymbol{x_i}^T\\boldsymbol{\\Sigma}^{-1}\\boldsymbol{x_i} -2\\boldsymbol{x_i}^T\\boldsymbol{\\Sigma}^{-1}\\boldsymbol{\\mu}+ \\boldsymbol{\\mu}^T\\boldsymbol{\\Sigma}^{-1}\\boldsymbol{\\mu}\\\\\n",
|
|
" &= -\\dfrac{1}{2}\\sum_i^N -2\\boldsymbol{x_i}^T\\boldsymbol{\\Sigma}^{-1} + 2\\boldsymbol{\\Sigma}^{-1}\\boldsymbol{\\mu}\\\\\n",
|
|
" &= -\\sum_i^N -\\boldsymbol{x_i}^T\\boldsymbol{\\Sigma}^{-1} + \\boldsymbol{\\Sigma}^{-1}\\boldsymbol{\\mu}\\\\\n",
|
|
" &= -\\boldsymbol{\\Sigma}^{-1}\\sum_i^N -\\boldsymbol{x_i}^T + \\boldsymbol{\\mu}\\\\\n",
|
|
"\\end{align}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"for Optimum: $-\\boldsymbol{\\Sigma}^{-1}\\sum_i^N -\\boldsymbol{x_i}^T + \\boldsymbol{\\mu} = 0$\n",
|
|
"\\begin{align} \n",
|
|
" 0 &= -\\boldsymbol{\\Sigma}^{-1}\\sum_i^N -\\boldsymbol{x_i}^T + \\boldsymbol{\\mu}\\\\\n",
|
|
" 0 &= \\sum_i^N -\\boldsymbol{x_i}^T + \\boldsymbol{\\mu}\\\\\n",
|
|
" 0 &= -N\\boldsymbol{\\mu}\\sum_i^N \\boldsymbol{x_i}^T \\\\\n",
|
|
" \\boldsymbol{\\mu} &= \\frac{1}{N}\\sum_i^N \\boldsymbol{x_i}^T \\\\\n",
|
|
"\\end{align}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"##### derive $\\Sigma$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\\begin{align}\n",
|
|
"\\frac{\\partial\\log\\text{lik}(\\boldsymbol{\\theta};D)}{\\partial \\boldsymbol{\\Sigma}} &= \\frac{\\partial}{\\partial \\boldsymbol{\\Sigma}} \\left(\\log(-\\frac{NK}{2}(2\\pi))-\\frac{N}{2}\\log(|\\Sigma|) - \\frac{1}{2} \\sum_i^{N}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})\\right) \\\\\n",
|
|
"&= -\\frac{N}{2}\\frac{\\partial}{\\partial \\boldsymbol{\\Sigma}}\\log(|\\Sigma|) - \\dfrac{1}{2}\\sum_i^N \\frac{\\partial}{\\partial \\boldsymbol{\\Sigma}}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})\\\\\n",
|
|
"\\end{align}\n",
|
|
"Because $(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})$ is scalar, we can take its trace and obtain the following form \n",
|
|
"\\begin{align}\n",
|
|
" &= -\\frac{N}{2}(\\Sigma)^{-1} - \\dfrac{1}{2}\\sum_i^N \\frac{\\partial}{\\partial \\boldsymbol{\\Sigma}} \\text{Tr}\\left((\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1} (\\boldsymbol{x_i}-\\boldsymbol{\\mu})\\right)\\\\\n",
|
|
"\\end{align}\n",
|
|
"The rule on page 10 (formula 63) of the matrix cookbook (https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf) allows us to derive the trace\n",
|
|
"\\begin{align}\n",
|
|
" &= -\\frac{N}{2}(\\Sigma)^{-1} - \\dfrac{1}{2}\\sum_i^N -\\left(\\boldsymbol{\\Sigma}^{-1}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1}\\right)^T\\\\\n",
|
|
" &= -\\frac{N}{2}(\\Sigma)^{-1} - \\dfrac{1}{2}\\sum_i^N -\\left(\\boldsymbol{\\Sigma}^{-1}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T \\boldsymbol{\\Sigma}^{-1}\\right)^T = 0 \\\\\n",
|
|
" &= -N \\Sigma^{-1} + \\Sigma^{-1}(\\sum_i^{N}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T) \\Sigma^{-1} = 0\\\\\n",
|
|
"\\end{align}\n",
|
|
"Multiply from both sides with $\\Sigma$\n",
|
|
"\\begin{align}\n",
|
|
"-\\Sigma N \\Sigma^{-1}\\Sigma + \\Sigma\\Sigma^{-1}(\\sum_i^{N}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T) \\Sigma^{-1}\\Sigma &= 0\\\\\n",
|
|
"-\\Sigma N + \\sum_i^{N}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T &= 0 \\\\\n",
|
|
"\\Sigma = \\frac{1}{N} \\sum_i^{N}(\\boldsymbol{x_i}-\\boldsymbol{\\mu})(\\boldsymbol{x_i}-\\boldsymbol{\\mu})^T\n",
|
|
"\\end{align}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### **Implementation**\n",
|
|
"\n",
|
|
"Lets reuse one of the implementations from the zeroth-exercise for that "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def mvn_mle(x: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:\n",
|
|
" \"\"\"\n",
|
|
" Maximum Likelihood Estimation of parameters for Multivariate Normal Distribution\n",
|
|
" :param x: samples shape: [N x dimension]\n",
|
|
" :return mean (shape: [dimension]) und covariance (shape: [dimension x dimension]) that maximize likelihood of data.\n",
|
|
" \"\"\"\n",
|
|
" mean = 1 / x.shape[0] * np.sum(x, axis=0)\n",
|
|
" diff = x - mean\n",
|
|
" cov = 1 / x.shape[0] * diff.T @ diff\n",
|
|
" return mean, cov\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We can now use this maximum likelihood estimator to fit generative models to the samples of both classes. Using those models and some basic rules of probability we can obtain the class posterior distribution $p(c|\\boldsymbol{x})$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Exercise 2.2) Generative Classifier (2 Points)\n",
|
|
"\n",
|
|
"Given a way to fit the class conditional using our Maximum Likelihood estimator, we can implement the generative classifier"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Accuracy: 0.83\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Fit Gaussian Distributions using the maximum likelihood estimator to samples from both classes\n",
|
|
"mu_c0, sigma_c0 = mvn_mle(c0_samples)\n",
|
|
"mu_c1, sigma_c1 = mvn_mle(c1_samples)\n",
|
|
"\n",
|
|
"# Prior obtained by \"counting\" samples in each class\n",
|
|
"p_c0 = c0_samples.shape[0] / samples.shape[0]\n",
|
|
"# LEAVE AS EXERCISE\n",
|
|
"p_c1 = c1_samples.shape[0] / samples.shape[0]\n",
|
|
"\n",
|
|
"def compute_posterior(\n",
|
|
" samples: np.ndarray,\n",
|
|
" p_c0: float, mu_c0: np.ndarray, sigma_c0: np.ndarray,\n",
|
|
" p_c1: float, mu_c1: np.ndarray, sigma_c1: np.ndarray) \\\n",
|
|
" -> Tuple[np.ndarray, np.ndarray]:\n",
|
|
" \"\"\"\n",
|
|
" computes the posteroir distribution p(c|x) given samples x, the prior p(c) and the\n",
|
|
" class conditional likelihood p(x|c)\n",
|
|
" :param samples: samples x to classify, shape: [N x dimension]\n",
|
|
" :param p_c0: prior probability of class 0, p(c=0) \n",
|
|
" :param mu_c0: mean of class conditional likelihood of class 0, p(x|c=0) shape: [dimension]\n",
|
|
" :param sigma_c0: covariance of class conditional likelihood of class 0, p(x|c=0) shape: [dimension x dimension]\n",
|
|
" :param p_c1: prior probability of class 1 p(c=1) \n",
|
|
" :param mu_c1: mean of class conditional likelihood of class 1 p(x|c=1) shape: [dimension]\n",
|
|
" :param sigma_c1: covariance of class conditional likelihood of class 1, p(x|c=1) shape: [dimension x dimension]\n",
|
|
" :return two arrays, p(c=0|x) and p(c=1|x), both shape [N]\n",
|
|
" \"\"\"\n",
|
|
" # TODO: compute class likelihoods \n",
|
|
" # TODO: compute normalization using marginalization\n",
|
|
" # TODO: compute class posterior using Bayes rule\n",
|
|
" p_x_c0 = mvn_pdf(samples, mu_c0, sigma_c0)\n",
|
|
" p_x_c1 = mvn_pdf(samples, mu_c1, sigma_c1)\n",
|
|
" p_x = 1 / samples.shape[0]\n",
|
|
" p_c0_given_x = (p_x_c0*p_c0)/p_x\n",
|
|
" p_c1_given_x = (p_x_c1*p_c1)/p_x\n",
|
|
" return p_c0_given_x, p_c1_given_x\n",
|
|
"\n",
|
|
"\n",
|
|
"p_c0_given_x, p_c1_given_x = compute_posterior(samples, p_c0, mu_c0, sigma_c0, p_c1, mu_c1, sigma_c1)\n",
|
|
"# Prediction\n",
|
|
"predicted_labels = np.zeros(labels.shape)\n",
|
|
"# break at 0.5 arbitrary\n",
|
|
"predicted_labels[p_c0_given_x >= 0.5] = 0.0 # is not strictly necessary since whole array already zero.\n",
|
|
"predicted_labels[p_c1_given_x > 0.5] = 1.0\n",
|
|
"\n",
|
|
"# Evaluate\n",
|
|
"acc = (np.count_nonzero(predicted_labels == labels)) / labels.shape[0]\n",
|
|
"print(\"Accuracy:\", acc)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Lets look at the class likelihoods"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<matplotlib.legend.Legend at 0x7f0dc0e446d0>"
|
|
]
|
|
},
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABG3ElEQVR4nO2deXgUVdb/PzcbJGwhCfsSQBBZZA0I464owuA66KsgrvO6rzO+ozPOT3Ec5tVxfBVn0RkdFySI+waouLGIIJugbLKFJawh7BBCkr6/P073dCd0d7qT6q6q7vt5nnqqu6u66qRS/a17zz3nXKW1xmAwGAyJT4rdBhgMBoMhPhjBNxgMhiTBCL7BYDAkCUbwDQaDIUkwgm8wGAxJghF8g8FgSBKM4BuSAqXU75RSL9lth8FgJ8rE4RvcgFJqE5AJdNFaH/F+9kvgWq31OTbaNQsYAlQCVcBy4E6t9Y922WQwhMK08A1uIg24124jgnCX1roxkAvMAl631xyDIThG8A1u4ingAaVUdrCNSqmJSqmtSqmDSqklSqkzA7aNV0pN9r7+VCl1V43vLldKXeF9fYpS6nOl1F6l1E9KqasiMU5rXQlMBXoGHHewUmq+Umq/UmqHUupvSqkM77a/K6WermHHx0qp+7yv2yql3lVKlSilipRS99Q47mLv37pLKfV/kdhoSG6M4BvcxGKkBf1AiO2LgH5ADjAFeFsp1TDIflOAa3xvlFI9gXxgulKqEfC5d5+W3v3+oZTqVZtxXiEfCywI+LgKuB/IA4YC5wN3eLe9BlyjlErxfj/Pu/0N72cfIy6idt7P71NKDfd+dyIwUWvdFDgJeKs2+wwGI/gGt/EIcLdSqkXNDVrryVrrUq11pdb6aaAB0D3IMd4H+iml8r3vxwLvaa3LgVHAJq31K97jLAXeBUaHsek5pdR+4DBwF/BYgE1LtNYLvMfaBPwTONu7bSFwABFzgKuBWVrrXcAgoIXW+g9a6+Na643Ai959ACqArkqpPK31Ya114EPGYAiKEXyDq9BarwCmAQ/V3KaU+rVSarVS6oBXgJshLeuaxzgETMcvnlcDhd7X+cBpXhfMfu9xxgKtw5h1j9Y6G2iIPDDeUUr18dp0slJqmlJqp1LqIPCnGja9BlzrfX0tfv9/PtC2hh2/A1p5t98MnAysUUotUkqNCmOfwQDIIJjB4DYeBZYC//F/e/31DyKt5ZVaa49Sah+gQhzjDeBRpdQcJPrna+/nW4HZWusLojVKa+0B5iql1gMXAj8AzwPfA9dorQ95/fOBvYXJwAqlVF+gB/BBgB1FWutuIc61Dr876ArkIZPri2AyGIJhWvgG16G1Xg+8CdwT8HETJDSyBEhTSj0CNA1zmBlIK/oPwJtesQbpPZyslBqnlEr3LoOUUj0isU0pNRQZtF0ZYNdB4LBS6hTg9hp/SzEy9vA68K7Wusy7aSFwUCn1oFIqUymVqpTqrZQa5D3PtUqpFl6793u/UxWJjYbkxQi+wa38AWgU8P4z4BNgLbAZOIa0koPi9de/BwxDBmh9nx9CWudXA9uBncCTyHhAKP6mlDqslDqMCPfvtdafeLc9AIwBDiE++DeDfP814FQCwjm11lXAxcggdBGwB3gJcVMBXASs9J5zInC11vpYGBsNBpN4ZTDYjVLqLMS10ymgp2EwWI5p4RsMNqKUSkeSyV4yYm+INUbwDQab8I4L7AfaAM/aaowhKTAuHYPBYEgSTAvfYDAYkgRHx+Hn5eXpTp062W2GwWAwuIYlS5bs0VqfkIkODhf8Tp06sXjxYrvNMBgMBteglNocaptx6RgMBkOSYATfYDAYkgQj+AaDwZAkONqHbzAYDLGmoqKC4uJijh1zV2WKhg0b0r59e9LT0yP+jhF8g8GQ1BQXF9OkSRM6deqEUqGKqzoLrTWlpaUUFxfTuXPniL9nXDoGgyGpOXbsGLm5ua4RewClFLm5uVH3SozgGwyGpMdNYu+jLjYbl44h5ng8cOAA7N8P+/bJ+sABOHYMystPXCoqQClISYHU1BPXWVnQuLEsjRr5XzduDDk50KSJfN9gMFTHCL6hzhw5AkVFsH27LDt2+F9v3w47d4rAHzwI8SzZlJEBLVtCixay+F63awf5+bJ06gR5eebBYHAm5eXlXHfddSxZsoTc3FzefPNNrKg6YATfEJYjR2D1ali/XpYNG/yvd+48cf/mzaFtW1m6dpUWd3a2fJ6d7X/dtCk0bAgNGvgX3/s0713p8chSVeVfV1ZCWZnYdfiwfzlyBA4dgtJSKCmRZfduWa9bB7t2wdGj1W3NzPQ/ALp1gx49/EurVuZhYLCPf//73zRv3pz169czdepUHnzwQd58M9jcOdFhBN8AiKBu3Ag//CDLjz/KesOG6q3zdu3gpJNg5EgR9M6doX17Efg2bURErSI1VZaaUWfNm0d/LK3FlbR584nLpk0wb548OHxkZ/vFv39/GDgQ+vYVd5LBYDWTJk3iL3/5C0op+vTpw+7duxk/fjwAo0eP5q677kJrXe+xBiP4ScquXbBggX9ZtEhaySAt227doF8/uO466N1b3nfp4l7BU0oeFM2by99VE61h2zbpzQQuH38ML78s+6SmQs+eIv4FBbIMGHDiA8ngXu67D5Yts/aY/frBs8+G3r5y5UomTJjAvHnzyMvLY+/evZx11ll06NABgLS0NJo1a0ZpaSl5eXn1ssUIfhKgNaxZA199JS3ZBQvE9w7iPunXD268UVqyp54KvXq5V9jrilLSU2nfHi64wP+570GweDEsWSLL9Onw6quyPSsLhg6Fs86Cs8+GwYOt7eUYEp+vvvqK0aNH/0fMc3JyCDZPiRWRREbwE5Rt2+DLL+GLL2S9fbt83ratCNSdd8KQIdJCNQIVmsAHwWWXyWdaQ3GxPDjnzoU5c2D8ePk8I0NE/7zzxO1VUCA9A4M7CNcSjxXBXDXt27dn69attG/fnsrKSg4cOEBOTk69z2UEP0GorIRvv4UPP4QZM6RFDxKJct55cP75snTpYgYj64tS0KGDLFdeKZ/t2ye9pzlzYNYsePxx+MMfIDcXLroIRoyA4cPl/2EwBHL++edz+eWXc//995Obm8vevXu55JJLeO211xg6dCjvvPMO5513nmnhJzuHD8PMmSLy06dLhEpGBpx7Lvz3f4vAn3qqxK8bYkvz5jBqlCwg/4uZM+Xh++mnUFgoD4ohQ2D0aFk6drTXZoMz6NWrFw8//DBnn302qamp9O/fnxdeeIFx48bRtWtXcnJymDp1qiXncvSctgUFBdpMgFKdsjKYNk0E5NNPJVGpeXP4+c/h0kulFdmkid1WGgLxeGQM4JNP4IMP/IOCQ4dKD2H0aOktGOxh9erV9OjRw24z6kQw25VSS7TWBcH2N20/F1BVJb74G2+U+PCrroKFC+HWW2UgdtcueP11EQ4j9s4jJUX8+o8+Ct9/L3kBf/qTPLx/9Stp6Z9+OvzrX5KkZjDECiP4DmbDBnjwQWn9XXABvPsu/OIX8PnnsHUrTJwo7hsTFuguunaF3/5WxH/tWpgwQXIEbr1VchluuEEGgx3c+Ta4FCP4DqOyUrr9w4eLMDz9NAwaBG+9JS35V16BYcNM5Eei0K0b/O53sGKFRP2MHQvvvSdhnt27w5NPyniAwWAFRvAdws6d8NhjUuPl8sth5Up5v3mzDMpeeaUJn0xklILTThO3zo4dEuffujU89JD08G67zR95ZTDUFSP4NrNuHdxyi9RzGT9eslrff1/S/R95REoZGJKLRo3g+uslxPPHH2HMGHkA9Oghsf2ff27cPYa6YQTfJhYtkkHW7t1h0iS46Sbx5376qST4pJmAWQPSAHjpJdiyReL6ly6FCy+UrOj33zfCb4gOI/hx5ttvJT5+8GCJvHnoIXHbPP+8+HMN1SksFDdXSoqsCwvttsgeWraE//f/5F555RWp/HnFFZIp/eGHRvgTjTlz5jBgwADS0tJ45513LDuuEfw48cMPcPHFEn63ciU89ZS02v70Jwm1NJxIYaG4uzZvFkHbvFneJ6vog5SPvuEGWLVKeoaHD0uPcOBA+OgjI/yJQseOHXn11VcZM2aMpcc1gh9jNm6Ea6+VAmVz54rAb9gADzwgNeENoXn44RNr2B89Kp8nO2lpMG6cVPR89VWJ37/0UgnT/f57u60zRMukSZPo06cPffv2Zdy4cXTq1Ik+ffqQYnGavPEUx4iDByXK5rnnJE7+N7+RxYL6R0nDli3RfZ6MpKXJAO/YsfDiizLQP3Ag3Hwz/PGPpvcYNUvug33LrD1m834w8NmQm4OVR44VpoVvMVqLy6F7d3jmGel+r18PTzxhxD4Y4Xz0oWrNmBo0J5KWBrffLlFfv/oVvPaajAk9+SQcP263dYZwBCuPHCtMC99CVqyQssNz5khZ3I8+kqQpQ3B8Pnqf28bnowdpsU6YUH07SP35CRPib6tbyM6Gv/xFsnYfeECCAiZPhn//WwIFDLUQpiUeK6yYySpSTAvfAsrLxa/cr5+I/j//KVmTRuzDU5uPfuxYSUTKz5fEpPx8eT92bPxtdRvdukn0zscfS+nmoUPh178+8Xob7Of888/nrbfeotSbUh1Llw5aa8cuAwcO1E5n2TKt+/TRGrS+4Qat9+yx2yL7mDxZ6/x8rZWS9eTJ4fdXSq5bzUUp//Fyc/2f5+bWfsx42u8W9u/X+rbb5Bp26aL1l1/abZGzWLVqld0m6FdffVX36tVL9+nTR19//fV64cKFul27djorK0vn5OTonj17Bv1eMNuBxTqEptou6uEWJwt+RYXWf/yj1unpWrdurfXHH9ttkb1Mnqx1VlZ14c7KCi+a+fnBBd8ntunpJ27LyIiNENfFfrcxa5bW3brJ3/bQQ1ofP263Rc7ACYJfV4zgx4GiIq1PO02u3n/9V3K36n2EE+9QhBPZUMer7ZjxtN+NHDmi9S23yN82dKjWmzbZbZH9JJPgGx9+lHz6qYS9rVkDU6fKkptrt1X2U5cQynA++nDfq29YZrDIoEjtd3vmb1aWjDFNnSrjTf36SXVOQ5IQ6knghMVJLfzKSq0ffVT8u336aL1und0WOQurW8ixauGH6lUEjhWEOleiuX3Wr9e6oED+jgcf1Lqqym6L7GHVqlXa4/HYbUbUeDwee1w6wMvAbmBFiO0KeA5YD/wADIjkuE4R/NJSrYcPl6t1/fXSLTYIge6XmoOw9RHDWPjwJ0/WOjU1uLDn5tYu5ono9ikv1/rWW+XvuOwyrQ8fttui+LNx40ZdUlLiKtH3eDy6pKREb9y48YRt4QTfqjj8V4G/AZNCbB8BdPMupwHPe9eOZ9MmGDFCSiT8858yOXicQmYdRWGhhEtu2SKJT75Y+MA4ea3l2mgt7pkJE+oeQun73r33+icAyc2VWb4iPWagzTk5cOiQTBcZjL17ZZrImn9j4LkSMfM3I0MK9/XoIQlbZ54poZzJVJa7ffv2FBcXU1JSYrcpUdGwYUPat28f3ZdCPQmiXYBOhG7h/xO4JuD9T0Cb2o5pdwt/6VKJwMnO1nr2bFtNsZX6uEHsIpjN4ZZIbE7EFn4g06dr3aSJ1m3aSLixwZ3ggEHbdsDWgPfF3s9OQCl1i1JqsVJqsZ1P3JkzZZq59HT45ht5nayESpAKNfVePFu8oQZRg9kcikizdydMkH3r8l03MHKklO9OTZUibIsX222RwXJCPQmiXQjfwp8OnBHw/ktgYG3HtKuF/847Wqelad23r9bbttligqMIlSAVz7DJYIQbRI3U5tTU6MYEEjU5K5CNG7Xu1Enrpk21njfPbmsM0YIDWvjFQIeA9+2B7XE6d1R8+CFcfbXMLzpnDrRta7dF9hOqWFlurr0t3nClGSIpsJaVJUXGohlnGDtWxnU8HlknYpmHzp3l3m/ZUmbXmj3bbosMVhEvwf8IuE4JQ4ADWusdcTp3xEybJpOFFxTAjBmmXr2PUK6MiRPtrXUTbhA1mM3p6fKQMnV5aqdDBxH9jh0laOG77+y2yGAJoZr+0SzAG8AOoAJpzd8M3Abc5t2ugL8DG4AfgYJIjhtPl84nn0jIX0GB1B4xVKc+roxYuUFqG0RNBvdLrNm5U+rv5OVpvXat3dYYIgFTWiE8S5eK77dfP6337o3LKZOGWCYruSURyu0PnrVrRfC7dJEHgMHZGMEPw7ZtWrdrp3WHDlrv2BHz07kCKwUq1qGMThdTtzyUamPBAq0zM6UHnIzJWW4inOAr2e5MCgoK9OIYxoYdPSrhlj/9BPPmQZ8+MTuVa6g5KQmIL7yu/u6UFJG5miglA5+JTqdOMrFLTfLzZdDXTUybJvPmjh0rg93JmIDoBpRSS7TWBcG2JW3xNK1l+sGlS+GNN4zY+7B64vBkn6YwkbJzR42CRx+VjOSXX7bbGkNdSFrB/+c/4e23Zc7PUaPstsY5WC1QiZ6sVBuJ9sB7+GEYNgzuugt++MFuawzRkpSCv3q11A0ZPlymfTP4sVqgkn2awkR74KWmituveXMYPRqOHLHbIkM0JJ3gl5fDmDHQqBG88or4mA1+YiFQyZCsFIpEfOC1bAlTpsC6dTB+vN3WGKIh6eTu0Udh2TLxQbZpY7c1ziMRBcpuEvGBd845Mrj/zDPyezK4g6QS/NWr4emn4aab4OKL7bYmcuI9y1JdBMrtM0EZoueJJyAvT0qGhyo7bXAWSSP4WsN994kr54kn7LYmcnxhkps3y9+webO8d5KgusFGg/U0bw7PPitVNZ9/3m5rDJGQNHH4H3wAl18u9V/uuceSQ8YFN8Rxu8FGQ2zQGs4/H1aulEmCGjWy2yJD0sfhHz8uUTm9esEdd9htTXS4IY7bDTbGkmR2ZykFjz8Ou3ebVr4bSArBf+MNKCqSmPs0qyZ1jBNuiON2g42xwriz4PTTpYzyn/8Mhw/bbY0hHAkv+B6PCH2fPjKjj9twQxy3G2yMFVZnJruVxx6DkhLTync6CS/4H30k0TkPPeTO2h+RhEna7VJI5lDOZHdn+RgyREI1n38+OWokuZWEH7Q9/XTYuVMKpLnNnROMwkJpPW7ZIi6TkSOlkJVVxc4M0WEGrP1MmSL33BdfyECuwR6SdtB23TqZlPn22xNH7Gv6i194wbgU7MQqd5bdvTQruOIKCdV86SW7LTGEIqEFv7BQXAzXXGO3JdYQzF8cqoOWbC4Fu7DCnZUoA78NG8K118J778HevXZbYwhGwrp0tIbu3aF9e/jqK4sNs4lQteWDkYwuBbeSSG6hBQtg6FCJjLv6arutSU6S0qWzZIm4dBLJjx0qzLHmYHSyRMgkCok08DtokLh1PvvMbksMwUhYwf/8c1lfcom9dlhJKH/xbbclZ4SMG4jEN59IeQypqVIvf+bMyHujhviRsII/Zw707AktWthtiXWE8hf/4x+JV43RjdQU9zvuONE3P26c/O8CxT/R8hiGD4ft26XcgsFZJEDsyolUVsoctddea7cl1jN2rBF0J1JzLmBfBFXNVq7vvW9gFvz/z8Bw2wkT3Pt/PuccWc+fD71722qKoQYJ2cJfvhwOHZIJyg2GeBBNBJWPwPDZRKqZ36ULNGsm42gGZ5GQgr90qawHD7bXDkPyUNcBVjcOzNaGUnDqqZLhbnAWCSn4q1dDZqb4SQ2GeBBpBFWk33M7XbvC+vV2W2GoSUIK/saN0q1MxvlqEyFj043UFkEFyRU+m58vA7cVFXZbYggkISVx2zZJuEo2EiVjM9RAp5OpLYJKa3j99eQJn23ZUtalpfbaYahOQgp+SYn/hkskamu9J0Kp3vHj4f77/SKvtbwfPx4oKoQPOsGUFFkXOetJVtvAayINzNZGTo6s9+2z1w5DdRJS8A8dgqZN7bbCWiJpvbs9Y1Nr2L9fpqH0if7998v7LimF6IW3wNHNgJb1wlscJ/oGobhY1hs32muHoToJKfjHjkkhp0Qikta72zM2lYJnnoF77xWRT0mR9b33wrg+D6OqalyAqqOw3EXdlySiqEjWRvCdRUIKfiISSes9ETI2faIfyDPPgDoa4gKE+txgK77kq7PPttUMQw0SUvDT0xMvOiCS1nsizDzlc+MEcv/9oLNCXIBQnxtspbxc1onW03Y7CSn4TZvCwYN2W2Etkbbe3TwwGOizv/de+Rt87p3Xf5iATq1xAVKzoK+Lui9JhC86Jy/PXjsM1UlIwW/ePPHCwcK13hMl9l4pyM4WkX/mmeo+/Y2esajB/4KsfEDJevC/oLOLnmhJxM6dMstcdrbdlhiqobV27DJw4EBdF0aM0HrAgDp9NSZMnqx1bq7W0oaV15MnV9+en6+1UrIO3BbJsbOy/McGeR/NMZyGxxP+vcH5/Nd/aX3SSXZbEQM2Ttb6/XytC5WsNzrvhwYs1iE0NSGrZebny8w7Wtee2h5rCgvhppvg+HH/Z6WlcOON/vc1qyzWrKIYjnDRO25y5wRS839m9//QED3r1sFJJ9lthcUUFUoosC9azBcaDK7paSakS6dHD0n42LXLbktEeAPF3kdFhWyrb7KU22PvDYlHRYXUwu/Tx25LLGb5w36x9+Gy0OCEbOH7anAvWwYXXWSrKWGFt67bAunYMfh8qG6JvTdYgKcSKg9BVTl4KkBXyDpwUQpUGqhUSPGuVZq8TmkI6U0gNdOS7tSKFRKl069f/f80R5EAocEJKfiDB8uA0Zw59gt+KEH2bYPwgl1YGH5ijAkTqruEwH2x9wYvWkPFfijbCcd2w7Fd1ZfyEqg4eOJSVWbN+VUKpDWGtCbyAEhrDBnNoWFLaNBC1oGvM9tAZjt5aATwxRey9sXiJwxZHb2Z3kE+dwmWCL5S6iJgIpAKvKS1fqLG9nOADwFv/h3vaa3/YMW5g9G4sUymPGtWrM4QORMmnOjDB8kV8IlyKMEONotSTf9+os2WlPAcPwCH1sKRTXB4k6yPbPauN0HlkRO/o1KgQUto2ALSs6Fha2hyMqQ3lcUn0KkNQKVDim/JkLXy/sx1pfQGdFXA60p5YFQehopDAetDsj6+Dw5vkAdQ5eEgtqWK6DfK9y4dSduUz40ju9Iupwfo1okzCNN3QnUfPrguNFjpepYiVEqlAmuBC4BiYBFwjdZ6VcA+5wAPaK1HRXPsgoICvXjx4jrZ9bvfwVNPiS+/ceM6HcIyCgsltNAXKpqbK7HlPlEO1Yrv1Cl46z8/X2LsDQ5Fazi2Ew6shoOr/euDq6FsR/V907OhcSevWHaS1mJma2jYyr80yBXRt5vKMullHNstS9k278PKv+iybShd5f9OejNoeooszXpA0x7QvK/8nW58EBQVis/+6Bb5G/pOcNyArVJqida6IOg2CwR/KDBeaz3c+/63AFrr/w3Y5xziLPiffw4XXggffQQXX1ynQ9hOSkrw0sBKSVKSwQFoLT/+vUtg71JZ71sqgugjrYlf7Jr1gCbdoXFnEfmM7IhPVVvDwQnMmFbJ7Tdu462X1nHaKWu8D7o1Jz7sGuRC8wGQMxByvOtGnWP/EHCBYNeXcIJvhUunHbA14H0xcFqQ/YYqpZYD2xHxDzqnvVLqFuAWgI71GHk8+2z5QRQWulfwzYCsA6k8Anu+g5K5UPIt7FsC5V4FVqnQrBe0HQnN+0OzniLymW3rLWS1hfc6RfRfeS2No+TTf0Q+ZAyrvvH4ARH+fd/7H45rnpZBZYCMHGhxOrQ4E1qeKQ+BlHTrjEuAsMr6YkUL/0pguNb6l97344DBWuu7A/ZpCni01oeVUiOBiVrrbrUduz4tfIA774SXX5asv2bN6nwY26jpwwfx77utPo6rOVYCJd/A7rmy3rdUfOAoyO4DuYNEmJoPgOxTIS0zJmaEcu+Bc1x8e/ZA27byu6tZAC8kVeVwYIWIf+lCuc6H1sq21CzIGyIPgNbD5HVKPdqoH3QKMeiaD5dtqvtxHUasW/jFQIeA9+2RVvx/0FofDHg9Qyn1D6VUntZ6jwXnD8m4cTLj0HvvVU90cgtmQNYGqo7Dnm9hx0zY8ZkIPEBqQ8gdDD0fghZnQN5QyIhfKyLaEN7aortiQWGhxODfdFMUX0pt4HXrDISu3tZ22S7pQe2eK+uVj8OKx2Q8oPUwaDsC2gyHrCintUuAsMr6YkULPw0ZtD0f2IYM2o4JdNkopVoDu7TWWik1GHgHyNe1nLy+LXytoXt3KeA0b547x4gMceDIZij+GHbOhF1fSzSKShNRbzMcWp0rgpTawDYTo2nh29Ez9Hgk/6VRI1i0yOKDH98PO7+EHZ/C9k9ksBigWW9odzF0HC0utNp+4KaFX/8Wvta6Uil1F/AZEpb5stZ6pVLqNu/2F4DRwO1KqUqgDLi6NrG3AqXgnnvg7rth7lw466xYn9HgGg6shq3vQfH74k4AaNwFOo/zi3y6c6ZNiyS814cd5TbefRdWr4Y33ojBwTOyoeMvZNEaDqz0iv8MWP1nWPW/8r/rMFrEP6cguPgnQFhlfal3Cz+W1LeFD1BWJq2j/v3h00+tscvgQrSG/cthy9si9AfXyOe5Q6DD5dD+Mmh6sq0m1kakUTrxju7yeKSMgscDP/4IqanWnyMkx/bAtg9hyzuw8wvJK2iUD/nXQJcbT/yfJnmUTsILPsCTT8JDD8HixTBwoAWGGdzD0W2waQoUTZLBQZUKLc+BDldA+0shq53dFlpOvPM33n4brrpKWvdXX2398SOmfC9s+wg2vyXuOV0lUT9dboKOV0pyWhKQ9IJ/8KDc7EOGwIwZ9vjy7RhES1oqj0grvuh1afWhxR/feRx0vEpiwBOYePrwjx+X1r1SUkMnrq37cJTtkP//xpfh4E+Q1khEv9sdElmVwIQTfNtr3odb6loPPxhPPy214j/4wLJDRkwi1qx3JPtXar3oLq3fbKJ1IVp/0Fnr5Y9ofWCt3ZbFnfrMsRANf/yj3M/Tp8fm+PXG49F697daL/il1m82lvvis59pvfktrasq7LYuJhCmHn5StPBBwsX694cjR2DVKsiMTbh0UGqLsDCt/SBE6mv1VEDxB7D2H7B7ltSP6XgldL1VwidNaFbM2LBBInNGjRK3juOpOAgbXoG1z8HhjXJfdb8bTvqlDAwniH8/6V06PmbNgnPPhUcegcces+ywtRJqEM2HSaaqQc2MSJBoisApDY/vE5Ff93fpvjfKh663wUk3SSVHQ0zRGkaMgG+/leicdm4aCvFUwfbp8NOzEoab1gRanSe5F56AyqM17zmXYAQ/gDFjJITsu+/iV687XAvfh1OyJR1BqHjp9FxJgDq2DVCAhtYXSiutzQhIcYoDOfF5+WW4+WZ49lmJHHIt+5bByj9J9FYwXBijH07wHVCCL74895yEs11zjbh34sGECdKKD4eZoSqAUJmPFaVesQfQMnFH5+ug3Sgj9nFk5Uq46y44/3xZO5KiQmk4TEmRdVFh8P2a94Mz3gp9nATLwk06wc/Lg9dfh59+gvvvj885x44Vl01+fuh9tJaeQGGI+zKpiHRCCc8xV00vFw8KC+U+SkmJzf105IiEYDZtCpMnOygqJxCfS/DoZkD7i6SFEn2QlnwwMnJAJ05p2qQTfJCWyW9+Ay++CO+8E59zjh0rLpvJk0O39n0TnCS96PedACkRjqonWAusPvjCMTdvlgaE1feT1pK1vnq13MetW1tzXEsIbNEvuD76uWf7ThCffTVS4HgpfDYYdn9jtcW2kJSCD/D44zIV4g03yNy38aK21n40E5gnJBWHvNUSA8aWGraDjBCx8y6aXi7WhCupYAXPPAOvvCLHGzas9v3jRs0WfeAELIGEaxx0HisDtFn5gJL1kFdh6GSZcvKLM2HeGKme6mKSbtA2kO3b4bTTpOXy3XfxjzQwE5wEoDVsngrfPwBl26UuSp/Hodkpsj2SyJ0kJ5b3ky+bdvRoePNNOZdjCDXIX5O6DsBWHoFV3po96U1h4HNSusGhIb9m0DYEbdvCtGlw4IBMknI4yJSdsSTURCZJN8HJ/hXw5Xnw7RiZr/XC+XDm236xh+AtsAQV+7r64WN1P33zjZQaP/10Gf9ylNhDZG69+hRJS2sEfR6Di76HxifBt2Nh9iVStsNlOO1fF3f69pUWy/LlUgekZjXCWBIsesc3gXlSUHEIltwPn/STwmaDnofhC2Wii2B0HisttDEeWSeo2N90U3U//E03RSb6sbifVq6ESy8VF+SHH0LDhnU/VswI5dZTqUTVOKgtsie7F1zwLfR/GnZ9CdN7weY3LfgD4kioFFwnLFaWVqiNF16QFPHLLtO6vDxup41bCrzj2PmV1h900rpQ+VPe38/XemOyXIDg5OZWL8HhW1JSIrtHrLyfli/XOi9P6zZttN6woe7HiTkbJ2s9NUvuId8yNSu6eynaYxxcp/VnQ2W/hbdrXVlmzd9iAZjSCpHxt79JFMLll0urP93C6TQNXiqPwLLfwtq/QsNWMrmFp9y/3eeXh4RIc4+WSNzC8cjMXrZMBmYbNoSvv4ZutU5IajP1LYtQl8lRPBVyztVPSTz/6W9BU/svlMm0jYK//lUmTamP6JvKmCEomQfzb4DD6+Hke2Dr+1C29cT9MnKhqiwpB2gjHQeMZWb2kiVwwQXQuLGI/UknxeY8jmJKCtUiw/6DEhdiOLZNh/nXgec4/KwQ2l8SCwsjxgzaRsHdd8ukEu+/LwO5hw5F9/1Yx0K7Eu2BlU/AF2fJBBXnfw0FE6GsOPj+x0ujj6NOEHIjrNwcq8zsmTPhvPOgWTOYPTtJxB5CjwNEEvbb7ucwYhk07QFzLoM1E620zFKM4AfhnnvgpZfgiy9kWsRtUQzGxzoW2nWU74XZl8Ly30qo5cjl0Ooc2RZtDH0SJFlNnBhZrzIWkVz//CeMHCmRQXPmQOfO1p/DsQRLvIomsqdRBxg2S2ZOW3ofLL5HirQ5DCP4Ibj5Zpg+Hdavl4lTfvwxsu+FanklZa2c0sXw6QDY+RkM/CucPrX6PLGhfmTpyZtkNXasJDfl54t7JzcXMjKq72N1JJfHAw88ALfdBhdeKGGYHTpYd3xXYEXYb1oWnPE2nPIrGaOaezlUltX+vThiBD8Mw4fL5Ocej8QgT5tW+3dMbL2XTVPg89PFrzVsLnS/60QHdagfWcHE+rW2XI6vDIfHA3v2SGVK3wMgP9/aAdsDB+CKK+Dpp+HOO+Gjj6BJcswEeCJWhP2mpMKAp6Hgb7Btmoh+1TGrLa07ocJ3nLDEMywzHFu2aN2/v4THPfig1hVhJspJ+tmtPB6tV0yQcLXPz9b62J66HWfjZAnTLFQmXDNCog3JXLRI6y5dtE5N1XrixHhYmGSs/7fcv18Nj2vYJmHCMm0X9XCLUwRfa63LyrS+5Ra5YmedpfX27aH3TdrY+qoKrRf8t4j9N2O0rjxmt0VJQzQNDY9H6+ee0zo9XesOHbSeNy/+9iYN6/8tv4evLoqb6BvBt5BJk+SH1LKl1jNn2m2Ng6g4ovVXI+TmXvawqIohbuTnVxd735KfX32/0lKtf/EL2TZqlNZ76tgBM0TB+pfkdzH3Kq09VTE/XTjBNz78KBk3DhYuhJwcGeC6/fboQzcTjsqjMOdS2PGp+OD7/tGxhaUSlUiCBT74AHr1khIJf/mL+OsjDQM11IOTboZ+T8KWt+DHOM6tGgQj+HWgVy9JTrn/fgll69MHvvrKbqtswif2O7+UcrJd/9tui5KScMECJSVSJ+ryy6WG/aJF8OtfJ/gzOdIZr+JFj/+BLjfBij/YaosR/DqSlQX/938Sr5yWJpOq3HknHDxot2VxpKbYd7nObouSlmCF0zIzJXmwVy947z2ZA2LhwvjN5WwbdZnxKtYoJcUBW54N390EexbaYoYR/HpyxhlSafO+++D556F7d5g0KQnq2XuqYN7VRuwdQuDEOkpBmzaSOPW3v8lnS5fC73+fJPWhlj/szEzt1Aw4813IbCOlwCviXI8dI/iWkJUlswEtWCBd6OuvlwfB0qWRHyPWc5Fazve/hm0fQ8FfjdjHkXD3ydixcs/dcQfs2gU7dojgz58PvXvbZbENhMrIdkKmdoNcGDoJDm+EpXGaVDsAI/gWMniw/Lhefhk2bICCArj1Vti5M/z3XFd/56e/wk8Tofv9cPKddluTNIS7T44fh7//XapaPv+8BBOsWyduxrQ0uy2PM/WpixMPWp4FPR+EDS/B1g/iempTLTNG7N8Pjz0m1TcbNJD6PP/zPxLdU5NOneTHW5NYVkSsM9tmwJyLod3FcMa7klloiAuh7pPcXMmO3bQJzj1XosdeeCGJq7W6YTrMquMwcwiUbYNRayGjmWWHNtUybSA7W9w8q1fDZZfBk0+KT/UPfzhxYNc19XeOFsP8cZDdR8rAGrGPK6Huh9JSEf3p02V2rMcfd1FvMRa4YTrM1Ayx6dhuWPmnuJ3WtPDjxI8/wqOPStnl3Fz41a+k2928uUta+J4q+Op82LtY5vZ0wEQPyUao+6RFC/HZK+WSe8ngZ/4NsPkN+PkqaGJNLWrTwncAp54qoXGLFomv/+GHpSLhffdJPL/j57Zd9b+wezYU/CO2Yu+0+GmHcOAADB16Yux8Zqb0JH2fu6a3aBD6/glUGiz7TVxOZwQ/zhQUwIwZEsp5xRUy0PbrX0vyVuvWsamIWG9KF8GP4yF/DHQeF7vzBIufnj8OFt4Ru3M6nM2bpTfYoQNMnQo9ekDLlrItPx9efLH6fZJU1VoToXGQ1RZ6/ga2vgf7V8b8dEbwbaJPH4nX37hRWvgrV0o0z+DB8Mgj4vd3BJ4qWHQHNGwJg/4R2/TMYPHTaFj/gjt/zHWkslLKH4waBV26wHPPwSWXSHb3ypXivtFaXDQ1GwXBErAc11u0AicmV9WVk++C1Ez46dmYn8oIvs106ABPPQVbt8Kzz8qA7s03Q9u2Ek+9bJnNBm54Sfz2/f9iaSRBUELGSWv7k2biQFGRJEd17CgP/KVL4aGH5PPJk2HAgNqPUTMBy3G9RatwanJVXWiQC52vg6LX4VhJTE9lBm0dhtYwb578SN96C8rLxQ00dixcdZU8COLGsT0wrTtknyrz0Ma6+MoHnbwttmAEmUy6qFB+4Ee3SIx13wnOisSIgH37ZCD/jTdkSs2UFLjoIoms+fnPkzCGPlLqM+m4EzmwBqb3gD6PQ+/f1+tQZtDWRSglWbqTJsH27TLHaWWluH3at5cJpl98UULxYs6Kx6HigMzeE49KW30nACHOUzNpxsVd+sOHYcoUcdO0aiU9uo0bYfx4cdNMnw6XXmrEPixOT66KlmanQMtzYPPUmJ7GCL6DycmRhK3vv5d4/kcekQnVb7lFBnh//nPpCWzfHoOTH9sNG16UrmZ2nPLyO4+FrrdxgugHm97QZV36PXvELXPllTLo6iuDcPfdUtBs/XoJ2026uWTrSn0nHXciHa6AAyvh4NqYncISwVdKXaSU+kkptV4p9VCQ7Uop9Zx3+w9KqQi8kYZATjlFWoBr1sjg3X33yQDerbdCu3YwcKBsX7zYosJtP02UuTh7PmjBwaJg8D9g6Ou1J83UVi/F5ggOrWHFCnjiCemxtWolcynMnQs33ihVVrdskblkBw1K8FLFscANyVXR0v4yWRe/H7NT1NuHr5RKBdYCFwDFwCLgGq31qoB9RgJ3AyOB04CJWuvTaju2U334hYUSR2936rrWIvoffywTrM+fL5+1aQPDhkma/XnnycBdVBw/AB92hNYXwplvx8T2ehPK35+VL608G1Lrt22DWbNk+fxzfwLUwIEScTNqlAy8pph+tSEUnw6SuPzh8+t8iHA+fCu8hIOB9Vrrjd6TTQUuBVYF7HMpMMk7/dYCpVS2UqqN1nqHBeePK74CVke9WuJLXYf4i75SUgWxd2/47W9lootPPhHx/+QTeP112a9zZxH/c8+Fs8+WsYCwLcqiSVBxEHqd0FlzDqFEve+E8O4eCwV/+3aYPVsE/uuvpVgZSFmNs8+WiJuRI+M80G44ETcN7re+AFY/JXNNpGXVvn+UWNHCHw1cpLX+pff9OOA0rfVdAftMA57QWn/jff8l8KDW+oTmu1LqFuAWgI4dOw7cHCxP3Ebckrru8Ujr/+uvZZk9WyJCQHoAgwf7l4ICEan/8NlpUFUOI5fZYHkUhPohxyCC48ABcZctWiQ+90WLoLhYtjVrBmedBeecIw/VPn0g1ZQZcgZuKKQWyLbpMHsUDJstVTXrQKxb+MHaijV/bZHsIx9q/S/gXyAunfqZZj1uSV1PSZFyDqeeKgO/Ho9k937zjV+0PvzQv3/37tC/P5zRdx13dlxISfs/k1PlcOHqPDb4jzarYwh3T+0RHFVVEve+cqUsK1bIoPmaNf59unYVgR80CM48U2aQcvR1SjSKCmHJvXDcG6qWngsFE4PfC3Hq7VlG836y3v9jnQU/HFYIfjEQGFvQHqgZNxLJPq6gY8fgLXynp66npIig9+/v/2z/fmm1Llwoy3ffwcnHp+Bpr+h/+TWUHpVU/t69pc56ly7iHurSRQYhHTvQGM7dg4xz7NoloZBFRbKsXSvivno1HDvm/1p+PvTtC9deKwJfUBC8xLUhThQVyhSBnuP+zypKYcGN/veBvb5QeR1OmAwlGJltIb0pHFhV+751wAqXThoyaHs+sA0ZtB2jtV4ZsM/PgbvwD9o+p7UeXNuxnThoW9OHD5K6nijZjFXTB3K0PIt3DsxlxQp/K3fbtur7ZWaK+HfuLFFCrVpJqKhv8b3Pyorvg+HwYRHznUtnsmvxu+zcncauoyezM3Uk2w52Y+NGcb2VlVX/Xrt2Mvdr796y7tULevaUOvMGBxEuOS8jF6rKarToFUGdCVn5cNkm6+2zghl9oVE+nP1Rnb4eU5eO1rpSKXUX8BmQCrystV6plLrNu/0FYAYi9uuBo8CNoY7ndHyiHi5KxylRPFFTcZDUg8to0uthbuxTfVNZmQhlUZG/ZexbL1okA8bB2g7p6dC0qSzNmlV/3aCBbA9c0tJkrbUknFVU+Ne+12VlUoIi2HL8Pw2/C72LPHBatJDB01NOgREj/L2Vzp1lXCYzM3aXtT649l6KFeFa5seDZSNqThB9p8frZ7aDo9tq368OmNIKFuPqHsD2T2HWCDjvc2g9LKqvVlZKctHOnbLs2iXr/fv9YnzgQPXX5eV+IQ8U9qoqOWZaGqSlVpKeUkZaSgXpaVWkZzaiYaOsag+PwCU7+8ReRl6eO7NWXX0vxYqw5TfCoFJBV8n6pFsk38OpfDIQ9i2FMXXT5nAtfCP4FuOWKJ6gLH8YVj0Jo/dDemPbzPB4pFWuNrkswsJior6XtK7uP6v5PhEI5sMHUOmQ1lT8+ScQpIXv5Htoivd/FgPBNykgFuOWKJ6g7F0KzXrbKvYgA8xK4bryCVYT1b30w3hYer/fr6a1vP9hfExss43OY+G0l8Vf7yM9F4a8IpE6NcstBPPhO/0eikF0jg8XdnSdjVujeAA4+BPk1jqWHj9qK58QDDcl2dRCxPeS1lCxX8phAAx4RsT+p4nQ/d7Ea+mHCsf14eYoHYDGJ8HhjTE5tGnhW4xrJ6DQHvkRNO5ityV+oq2I6OIKmsGI+F5SSkS++70i8m+k+MV+wDOJJfa10XmsRN+M8cg6K0RdESdX1Ty+H9KzY3JoI/gW49oJKMpLZVArs43dlviJtiJitC4gh0+RF9W95BP9QJJN7IPhxqqa5XugQWySPYxLJwaMHesCga/JcW/dhQwHZRX5uu2RumiicQHVTLn39QYCz+sAIr6XfD77QJbeb0Q/2nvICRzZDC3PjsmhjeAbhCpvJlKawwLSa/PXBhJNSQW3pdyHwyf2gW4c33swoh/NPWQ3lUegrDhmrlXj0jF48UUyuFgYoum+12VA2KkoJT7fQJ+9z6efnp3cYu829i6V8bTcoFGV9ca08A1CSoasPRX22lEfgnXf246U9/PHVe/O16PAmiPpM756NI5P9I3Yu4vSRbLOGRSTwxvBNwhp3qIxFQfttaO+BHbfw/npaymw5kpqirsRe/dRulDq6GS2isnhjUvHIDTIk3X5bnvtsJLa/PSJNkWewd14qmD315D3s5idwrTwDUJaJjTIhSMu9GHX5D/JV7Uk3bhpMM+Q+JTMhWO7ocPlMTuFaeEb/DQ+CQ6ts9uK+lEt+SoEbvXTG/w4PIeiTmx9F1Izoc2ImJ3CCL7BT7PesH958DrHbiGYG6caSgZyDe4lwTKqAXHnbH0X2o6IaS0rI/gGPzkDJeP2yCa7Lak7tYZVaih6zd3ikOwkYlG97dOgbAfkj4npaYzgG/y0OEPWu+fYa0d9iMRd43ZxSBZCuW0SKYfCx+qnoVEnaH9pTE9jBN/gJ7u3ROvsmGm3JXUnWPJVMEKJQyL6ht1IOLdNtEX1nE7pIhmw7X4vpMQ2jsYIvsGPSoH2l8G2j6AynB/cwdQMt1SpwfcLJg6J6Bt2K+HcNm4siBaO1X+RictPuinmpzKCb6hO/jVQeRi2TbPbkroTWCJ3yGuRi0Mi+oZjQTx6QeHcNomUQ7HnO9jyFpx8t4h+jDFx+IbqtDxbSiRvfgPyr7LbmvoTTbXERPQNW028qozWVvoiEXIofEXvGraGng/F5ZSmhW+oTkoqdLwatk+Ho9vstsYaak6KEUooYuEbTrQxgXj1ghLNbROMzW/CnvnQ909xm1bUCL7hRLrfLRX71jxT+76JhNUik4hjAvHqBSWS2yYYxw/Asv+B5v2hy/VxO60RfMOJNO4M+VfD+hegfK/d1sQPq0UmkcYEfD2VmhOC+4hFhEykPTM3svhuibsf9IIES8QJI/iG4PR8UCZjWPtXuy2JL1aKjJ1jAla6kmorV5ForpZYs+Vt2PQ69Po95A2O66mN4BuCk30qtL8cVj8FR4vttsad2BUvbrUrKVy5ikRztcSao9th4W1S7753/Ht6RvANoRnwtExsvvTXdlviTuwaeLTalRSyR6ISz9USS6rK4ZsrZTrRn70OKelxN8EIviE0jTtDz99KnPDOL+y2xrmEcp/YNfBotSsp0TJb7UBrWHQ77PkWhrwKTbvbYoYRfEN4ev5GJlRedIf49A3Vqc19YsfAo9UCHcueSqKFrYZizTOw8RXo/Yit+S1G8A3hSW0Ip70Eh9bD4rvstsZ5ODESx2qBjlVPpbaHZaI8DLZNkxDMDr+AUx+11RSTaWuonVbnQq+HYeUfodX50Plauy1yDk7Mzo0muziaY1rdO6ntYRmPjN5Ys/MLmDta4u2HvBrXEMxgmBa+ITJOfRRanCl+yINr7bbGXooK4e08mKKIa1x6NPhcSUNfl/fzxzmvlRzuYenEnlO07J4Lsy+BpifDuZ/FLZs2HEbwDZGRkganT4HUBjD7YpkoJRkpKoQFN0JFmL/fKXHpTneZhBtrcGLPKRr2fAezRkKjfDjvC5kv2gEYwTdETlZ7OOtDOLJZWi6VZXZbFH+WPwy6IvR2J8Wlh2olz78W3smTB5edZR/CjTW4OTJo51fw1QXQsKWIfcOWdlv0H4zgG6Kjxenws0Ip+vTtWJmLM5kI28J0WFx6OFuPl5744Iq3yyTcYLBbi6dtfgtmjYBGHWHYbMhqZ7dF1TCCb4iejr+AAc9A8fvi09ceuy2KH+FamE5rfdbFnni7TGqGrYK4l+aPg5RMyMjFNcXTfnoO5l0NuafBBXOlR+wwjOAb6sYp90rkzoYXYcFNydPS7zsBVJAMyZQM57U+I53uMRA7H1o1xxwqSiUrdejrzuo51cRTKdnoS+6VGePO/QwymtttVVCM4BvqTt8/Qp/Hoeg1r3snjG87Ueg8Foa8AukBg3AZuXDay84TpGoukwiw22XixsicYyXw9YWw5v/g5LvgjLchLdNuq0Ji4vAN9aP37yU56/v/Ac8x+NkUSIuyVek23DDbUlFh9Tj8rrfLgzlQUFMyILUJVOy1Jla/vrgtMqd0Ecz9BZSXSIx9HOva1xUj+Ib60+MBSM2UGt9fnCWRPA4brEoqgk1DWPQadL4ets+wLhnLamqb1tApaA3r/wVL7pHpQC+YBzkD7LYqIuol+EqpHOBNoBOwCbhKa70vyH6bgENAFVCptS6oz3kNDuTkOyXmeN418NkgEf3cQXZblZyEco1sn+EfGHUifSdUf1CB/W6mmpTtgoX/Dds+htYXSm6KQ2LsI6G+PvyHgC+11t2AL73vQ3Gu1rqfEfsEpt0ouPBbSGkgLf1NU+y2yDqsTlKKZdKT21wjPpw+rWHxhzDjVNgxU6LUzv3EVWIP9XfpXAqc4339GjALeLCexzS4mexTYfhC8W1+OxZ2fQUDJ0JaI7stqzvBXCT1qeti9fFq4hbXSDCcOD5y/AB8/2vY8G+piTP0dcjuZbdVdaK+LfxWWusdAN51qJQyDcxUSi1RSt0S7oBKqVuUUouVUotLSkrqaZ7BFhq2gPO/hF6/gw0vw6cDYe/3dltVd6yOHol1NIpbk5achtawaSpMO0VKG/f6HVy4wLViDxG08JVSXwCtg2yK5u48XWu9XSnVEvhcKbVGaz0n2I5a638B/wIoKCgIUZnK4HhS0kVgWp0vSTQzh0Df/4Xu90JKqt3WRYfVLpJYuFxqRuU4fYDW6RxcB4vvkGqXOQVwzjTIGWi3VfWmVsHXWg8LtU0ptUsp1UZrvUMp1QbYHeIY273r3Uqp94HBQFDBNyQYrc+DEcvhu5ulW7zlLfHLNu9jt2WRY7WLxOrjhYrKibX/u+ZDJhEeKpVHYdWfYdUTUiiw4O/Q9Vb3NVJCUF+XzkeAL/j0euDDmjsopRoppZr4XgMXAivqeV6Dm2iYB2d9AEMnw+EN4uJZ9lv3FF+z2kVi9fHsSFiyeqJ0u/FUwvqX4ONusOIxmaxk1E9w8h0JI/ZQf8F/ArhAKbUOuMD7HqVUW6XUDO8+rYBvlFLLgYXAdK31p/U8r8FtKCWtv1FrZAKVVU9IxMO2GeIrdTJWR49YfbxoXERWRQe5MSs2GFpD8cfwSV8Jt2yUDxd8A6cXQmYwT7a7UdrBP7aCggK9ePFiu80wxIKdX8Gi2+DQOmg9DPo/Bc372W2VO/mgUwgXUX71uPuarh+QnkVdHjZTUgg++YuSQmhOR2vYPQd+fETWTU6Gfk9ILRyl7LauXiilloQKfze1dAz20Po8GLkCBjwLe5fCJwNg/g1wtNhuy9xHpC4iK1vlbq1XrzVs/1TyRL48Bw7+BIOeh5+vgA6Xu17sa8MIvsE+UjOk6uYlG6Q8w+Y3xIe69AEo22G3de4hUheRldFBbgv91B7Y+r5kgc8aIZP4FPwNLimCbrdJVFkSYFw6BudweBP88AhsLpQSxCf9Enr+RiaTMNSfSF0/keKGKJ2Kw1A0Cdb+DQ6uhsYnQa/fQqdx0uBIQMK5dIzgG5zHoQ0yqFv0mnTBu1wPPX4jk0Eb6o6VPnync2g9rP27JExVHJAY+u73Q/5/yfzMCYwRfIM7ObIFVj8F618ETzm0GS41x9uMSKhQubjihlZ5XfFUwo7PYN0LsH06qFToeCWcfDfkDUl4/7wPI/gGd1O2S8rRrn8ByrZD4y7Q7Q7ociM0yLHbOoPd7F8JRa9C0etwbBc0bCXJUl1vhay2dlsXd4zgGxIDTwUUfyD+2N1zZOKV9pdLGYHWw0yrP5koL5Ws7Q2vwN5FoNKkWmuXG6DtyKQZhA1GOMFPbGeWIbFISZcuescrYd8P0uLfPFWiezLbQKdrofN1kN3bbksNseBYiTzwt7wtVVh1FWT3kVLFncZAw1C1Gw0+TAvf4G6qysVfWzQJtk0HXQnZfSU1vsMV0Kxn0vhuE5KyXX6R3z1LRL7xSd4H/1WSrGf+v9UwLh1DcnCsxNvinwp75gMamnQTt0+HK2QGLmVSTxyNpwpKF0qlzx2fwN4l8nmTbv7eXXZfI/JhMIJvSD7KdsgMRVvf93b/K8Xt0/oCmZqu9TDIbGW3lQaQ7OqdX4nA75gJx/fKgzlvqERktbtYJtYxIh8RRvANyc3xfeLu2fYx7PpSBvxAWoptLpSHQN5QSG9sr53JgNZwZBPsni3LrtlwpEi2NWwFbS6CtiPkf2IisOqEEXyDwYf2wL7vpSW583Mo+Uaif1SqPABa/AzyTpe1yfCtP5VH5XqXLpKlZC4c3SrbMnKg5VnQ8mxZmvc1LjcLMIJvMISi8gjs/gb2zIOSeVD6nXwGkNUeck+TgUHfktnOuBZCUXEIDqzyC/zeRXBgpTxkQa5di5/5Bb5ZTyPwMcCEZRoMoUhrBG2HywKSrbn/Byj5Vh4CpYth67v+/RvkQnY/Ef9mPaWsbtOToUGL5HkQVByCQ2th/woRdN9yJKBOT4NcyBkk5YZzBkFugYyhGGzFtPANhtqoOCQPgX3L/Mv+H6Xcg4/0Zn7xb9JNyhY06gBZHaSnkNbIJuPrQOURGfQ+WiwzlB3eWH0p3+PfNyUDmp4CzXpJ/kOzXhIb36hT8jwAHYZp4RsM9SG9CbQ4XRYfnkqpR3NwrbR2feuSb2DTFE6YHCSjuYh/Zjto2AIycqUV3CDPu86Vz9IaeZcsKWxWn0Jfnip5KFUcgor9cHy/DGAf3+9/X14i5SrKdviXykPVj6NSZSaoxl0kvLVxF2jcVcS9SdeEL0aWSJj/lMFQF1LSvMLXBbio+raqYyKiR7bKAOXR4oB1MRxcJa1k31hB2PNkiPCnZUr5AJTX7x24VjLw7CmXRDRPudigq2o/fmqmuFoy20jLvM1w//us9pLklNXBiHqCYP6LBoPVpDYMeBiEoeqYhIiWl8Jx77ryKFQdkXXlUSllXHlE1trjHQDVJ65VOqQ2kHOnNJDXKQ1lnd4E0rMhI9u/9r1ObWhcL0mEEXyDwS5SG0JWO1kMhjhgYqIMBoMhSTCCbzAYDEmCEXyDwWBIEozgGwwGQ5JgBN9gMBiSBCP4BoPBkCQYwTcYDIYkwQi+wWAwJAmOLp6mlCoBNteyWx6wp5Z97MLYVjeMbXXD2FZ3nGxftLbla61bBNvgaMGPBKXU4lCV4ezG2FY3jG11w9hWd5xsn5W2GZeOwWAwJAlG8A0GgyFJSATB/5fdBoTB2FY3jG11w9hWd5xsn2W2ud6HbzAYDIbISIQWvsFgMBgiwAi+wWAwJAmuE3yl1JVKqZVKKY9SKmSoklJqk1LqR6XUMqVUXGZCj8K2i5RSPyml1iulHoqTbTlKqc+VUuu86+Yh9ovbdavtOijhOe/2H5RSA2JpT5S2naOUOuC9TsuUUo/E0baXlVK7lVIrQmy387rVZpst100p1UEp9bVSarX3N3pvkH1suW4R2mbNddNau2oBegDdgVlAQZj9NgF5TrMNSAU2AF2ADGA50DMOtv0ZeMj7+iHgSTuvWyTXARgJfAIoYAjwXZz+j5HYdg4wLZ73V8C5zwIGACtCbLflukVomy3XDWgDDPC+bgKsddD9Foltllw317XwtdartdY/2W1HMCK0bTCwXmu9UWt9HJgKXBp767gUeM37+jXgsjicMxyRXIdLgUlaWABkK6XaOMQ229BazwH2htnFrusWiW22oLXeobVe6n19CFgN1Jxb0pbrFqFtluA6wY8CDcxUSi1RSt1itzEBtAO2BrwvJkb/3Bq00lrvALnBgJYh9ovXdYvkOth1rSI971Cl1HKl1CdKqV5xsCtS7LpukWLrdVNKdQL6A9/V2GT7dQtjG1hw3Rw5iblS6gugdZBND2utP4zwMKdrrbcrpVoCnyul1nhbH3bbpoJ8ZklsbDjbojhMTK5bECK5DjG7VrUQyXmXIjVLDiulRgIfAN1ibViE2HXdIsHW66aUagy8C9yntT5Yc3OQr8TtutVimyXXzZGCr7UeZsExtnvXu5VS7yPd9HoLlwW2FQMdAt63B7bX85hAeNuUUruUUm201ju83dTdIY4Rk+sWhEiuQ8yuVS3Uet7AH6TWeoZS6h9KqTyttRMKcNl13WrFzuumlEpHBLVQa/1ekF1su2612WbVdUtIl45SqpFSqonvNXAhEDRqwAYWAd2UUp2VUhnA1cBHcTjvR8D13tfXAyf0RuJ83SK5Dh8B13mjJ4YAB3xuqRhTq21KqdZKKeV9PRj5LZXGwbZIsOu61Ypd1817zn8Dq7XW/xdiN1uuWyS2WXbd4jEKbeUCXI48icuBXcBn3s/bAjO8r7sgkRXLgZWIu8URtml/NMBaJBIkXrblAl8C67zrHLuvW7DrANwG3OZ9rYC/e7f/SJioLBtsu8t7jZYDC4CfxdG2N4AdQIX3frvZQdetNttsuW7AGYh75gdgmXcZ6YTrFqFtllw3U1rBYDAYkoSEdOkYDAaD4USM4BsMBkOSYATfYDAYkgQj+AaDwZAkGME3GAyGJMEIvsFgMCQJRvANBoMhSfj/4oBgvvNNc0cAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<Figure size 432x288 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"plt.title(\"Naive Bayes\")\n",
|
|
"plt.scatter(x=samples[labels == 0, 0], y=samples[labels == 0, 1], c=\"blue\")\n",
|
|
"draw_2d_gaussian(mu_c0, sigma_c0, c=\"blue\")\n",
|
|
"plt.scatter(x=samples[labels == 1, 0], y=samples[labels == 1, 1], c=\"orange\")\n",
|
|
"draw_2d_gaussian(mu_c1, sigma_c1, c=\"orange\")\n",
|
|
"plt.legend([\"c0\", \"c1\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"And the final posterior distribution for the case $p(c=1|\\boldsymbol{x})$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"(-1.5, 2.5)"
|
|
]
|
|
},
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAEKCAYAAABXMPIIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABve0lEQVR4nO29eZxkZXX//z63qrqqeu/pmenZ2IRh2EUZQQLiEERhVJafKBijxOCXaPBniEFFYxJj4hb5GuISDUFF/BmRL37RUQYB0ZEhsg2EfXGGbRh6Znqmu6f36q6q+/z+uLeqa7m39uqq7j5vXkVXP3WX51ZPnfrcc85zjhhjUBRFURRFUZoDq9ETUBRFURRFUWZRcaYoiqIoitJEqDhTFEVRFEVpIlScKYqiKIqiNBEqzhRFURRFUZoIFWeKoiiKoihNhIozRVEURVGUJkLFmaIoiqIoShOh4myRIiJfEpEra3i8z4nI5yrc90ERObZWc1EUZWGj9ktZ6Kg4W4SIyDLgA8B/NHouLtcAn2/0JBRFaX7UfimLARVni5M/AzYbY6YaPRGXTcCZIrKy0RNRFKXp+TPUfikLHBVnCxQReUlEPi0iT4vIsIh8X0Qi7svnAr/z2Oe9IvKwiIyIyPMisqFGczlcRIZE5PXu76tEZH/q+MaYGPAw8NZanE9RlPmN2i9lsaPibGHzPuBtwOHAkcBn3fHjgecyNxSRv3Ff/19AD3AB8FItJmGMeR74FPAjEWkFvg/cYIzZkrHZM8Bra3E+RVEWBGq/lEWLirOFzTeNMa8YY4aALwDvdce7gbHURm4Oxz8Af2KMecQYYxtjnjDGvCQiXxCRrSJyi2uYKsIY85/AduABYCXwtzmbjLnzUhRFgSrtFzDsJuuPi8hx1UxE7Zcy16g4W9i8kvH8ZWCV+3wY6Mh47S3AE8aYxzJ3dg3a4caYNwG/Bv68yvn8J3Ac8A1jzHTOax3AgSqPryjKwqEq+wVMAm8HbqnRfNR+KXOGirOFzUEZzw8G+t3nj+OECVIswduwvAm43X1+O3B6pRMRkXbgWuC7wOdEZEnOJkcDucZVUZTFS1X2yxgTN8bsq8VE1H4pc42Ks4XNFSKyxjUknwF+4o5vBt6csd3/AKeLyGvFYa2IHI2TuzHibjOCYwQr5d+Ah40xHwJuA76TekFEwsBJwF1VHF9RlIVFtfarlqj9UuYUFWcLm/8C7gRecB//7I7fCGwUkSiAMeb37mu/xMmduBWI4oQPutx9uoChSiYhIucD5wAfdoc+DrxeRN7n/n4esMUY0++1v6Ioi5Jq7VdNUPulNAIxxjR6DkodEJGXgA8ZY37t8/oXgQFjzLUFjnE88GljzJ+IyOVA2BjzDZ9tPwdgjPlcBXN9ALjMGPNkufsqirLwqIX9ytj2BuCaQvZF7ZfSbAQbPQGlMRhjPlPCNk+IyMsishUYwKnKXY+5nFKP4yqKsjApxX4BiMhm4ERgnYj8hzHmhjrMRe2XUnNUnCkFMcZ8usRNt9RzHoqiKOVijNlY4qZb6jkPRSkXDWsqiqIoiqI0EbogQFEURVEUpYlo6rBmixUxUauj+IaKsogZTe7fb4xZVs4+GzaEzdCwXdK2TzyeuMMYc05Fk1vEqP1SlNKopw2br/arqcVZ1Org1M4LGj0NRWlq7hi+/uVy9xkatrlt89KStj14zZ7SNlSyUPulKKVRTxs2X+2XhjUVRVEURVGaCBVniqIoiqIoTYSKM0VRFEVRlCaiqXPOFKUetHVHePcnzqLvsF7EkkZPpySMbdj74iD/56t3M3Eg1ujpKIrSIOaj/QK1YeWi4kxZdLz7E2dxzPojCQeiiMwP42aMobe3l3d/Am7429saPR1FURrEfLRfoDasXDSsqSw6+g7rnXeGTUQIB6L0Hdbb6KkoitJA5qP9ArVh5aLiTFl0iCXzzrCBY9zmUxhDUZTaM1/tF6gNKwcVZ4qiKIqiKE2EijNFaQJmZqa58lMf5ezzNvDuD1zArv5djZ6SoihKSaj9qj26IKCJMcZgk8Q2yYyfNgYb2zg/jfsfGMjsYS/O/7L+EwuLABYWlgSc5+7P+eomXyj8n5/dTGdnF3dt2sJtd/yCa/7ty1z7lW82elqKUhW2sbFJYJskSdeGzdovg8GGlA0zOTuLY8FAsLCybViG/QoQQET9DI1E7VftUXFWR2yTZMbEmLFjxM20+4gRt53nCTPjPuLOTxIkTdx9JEiSmLO5BggSkCABCRGQIEFCBCREUFoISgshaSEkYUJW2PkpEUISpsWK0CIRLAnM2Vznml9sbuNr3+ph954AK1ck+fgVw7xz40RVx/zZL3/Kd2/8T0SEdWuPYmhokI/+xZUAvO2sc/n8V/4BY4yKZqVhGGPSNmsmw27FTYy4mSFuZkiY6bT9ShrHfiVMnKRry0ye4qoPjlALEiRlw0IE048cGyZhQpZrvyRCixUhQGjBftbUfs1PVJxVgG2SxOwJpu1Jpsw40/bk7MNMMeP+TJgZ32NYBNIGIyAtBCVEq0QdgUTIFUpBAgSxJIglFgGCzj2kWEjGnaRzd0n6J5A2igaDMcb1ss163GySJE0S2yRIkswQhM7PlMGN2eMkTJy4mcYm6Xs9QWkhLFFarChhaaXFihKxWglLGxHLeYStVgIyv/7J/WJzG5/9515iMefOvH93kM/+s7PaqFIDt/35P/Dt736LH3/vFpb0LOHAyAH+9EMXs3LFSgCCwSAd7R0MHxhmSc+S2lyIorgYY5gxU8TsSWL2uGPLjGO/Zuwp14ZNMWOmCogrIeSKnqBrv1okStDqSIujlP0KSAALx4ZZrpfL8YRZ7pf3rHc/a54ZUQEnipCyYU4EIRVRcERhIi0IM0XipJlybJk9TYK473tiEXBt16wNC1tRwtas/YpIG0FpmVeCQ+3X/GV+fVPOEbaxidnjTNqjTNqjTCXHidljTNnjTNnjzJipvH0CBAlbrbRIlPZAD0us1e4HPTLrZcq4Y5tvIgUgaRKzHkA7xkzqrtqOMWOmmLanmDaTjCb3Mx2fIulhDFskStRqJ2q1E7E6iAbaabU6abU6iVjtWE0Wnvjat3rShi1FLGbxtW/1VGzc7n/o95xz1rlpw9Xd1Y0x+V+C8+lLQGkeHPEVc23XKJP2GDHXdk3ZY8TsCTecOIsgtEiUsNVKxGqlM9BL2IrSIlHHflmOlynldQrOQ0+TMbbr8UvZrVmvYEqMTttTTNnjjNgDzJj8QqkBQkSsNloDHUSsdqJWB1Grw7FhgQ6C0tKAK/NH7df8Zf4phBqRunscTw4zkRxhwh5h0v0Zs8ez7hgFy/0gtrMsdBARq332bsp9NNuHsh6kvHkR2qCEKKbjeZtIexlj9gRTtiN0R5OD7I2/nPUlIQgRq4M2q5O2QDet7s/2QDct0pi6Prv3eF+o33gpGGMg51pWLF/B7j27WdG3kkQiwdj4GN1d3RWfQ1n4JEycieQB52GPpu3XZHI078bIuSnqoCuwjL7QYRm2q52Ie1O50L9MRSwnjEkE6Cq6vW1spu0JYiZlwyaYsidcoTvGcGJvXnSkRSK0Wl20BbpoDXTRZnXRHugmanU25MZT7df8ZVGIs2l7ivHkEOPJYcaSw44gsw9kfbACBGkNdNEVWMrKltcQdb05rYFOwtK64A1XPQhKC+2BFtoDPZ6vG2OYNpNMJl0PpT3KZHKUCXuU4ek9WTl3QWmh3eqmPbCE9kAP7YEeOgJLaLEidb2GlSuS9O/O/5isXOEf4i3GqSefxkf/5i/4s/ddRk93DwdGDvDHb34Lt/7yp7zuta/njrtv541vOFX/zSmA47EeTx7IsmETyWFiJtPzIUStdtqsLnrCfbRanY4NC3QStdrnpae+0VhiEQ10EKXDd5u4mXG9k46HcjI5wqQ9yr74K8zM/CG9nWDRanWm7VZ7wLFlrVZnXT/nar/mLwvqE2uMYdIeZSw5yGhykNHEIOPJIabNZHqbkIRpD/SwsuVw2izHK9MW6FYB1gBEhIibk7aElVmvpYTbRPKA6908wLh9gL3xF9k182x6u7C00hHopTPYS2egl47AUqJWe83+lh+/YjgrZwMgErH5+BXDFR9z7eFH8uHLPsr7/9clWJbFMUcdyz9+5gt84u/+mrPP20BXVxf/+qVv1GL6c4aIvASMAUkgYYxZLyJLgJ8AhwIvAe8xxgy7238auMzd/mPGmDvc8ZOAG4AosBn4K2OMEZEwcCNwEjAIXGyMeWmOLm/OmLFjWfZrLDnIpD2S9uRbBGgLdNMTWpFlv1qtzgW9KKdZCUkLoeBSOlma91rCzDCRHMmyX6PJ/eyNv5jeJkCIjkAPHRn2qyPQU7O/pdqv0mhG+zVvxZkxhpiZYDSxjwOJfYwm9zOa3J/2hglCe6CH3tBq5x9/oJf2QM+icN8vBDKFW29odXo8FY4eSw65j0HGEkMMxnalv8BCEqYzsJSu4FK6AsvpCi4jbLVWNI9UXkatVztd+M53ceE735U19vV/+feqjtkEnGmM2Z/x+9XA3caYL4vI1e7vnxKRY4BLgGOBVcCvReRIY0wS+DZwOXA/jnE7B7gdxxAOG2OOEJFLgK8AF8/VhdWDhIkzmtjPSHIfI64Nm7LH0q9HrHY6AkvoazmUjsASOgJLGhYeU8onKC10BZfRFVyWNZ4KR6ftV3KI3dM7eIVnAMfL1hFYQmdgKd1Bx361WV0VlQtR+1UWTWW/5o04s02S0eQgBxJ7OJAY4EBiIO0RS/1jXhF6DV3BpXQGltIW6FZX/gJERNyVVK0sDa1JjydNIm3sUl90L8YeTwu2qNVOV6CP7uBynM9Q6bxz40TVxmyRcj6wwX3+A2AL8Cl3/CZjzDTwoojsAE527147jTH3AYjIjcAFOMbtfOBz7rFuAb4pImK8MpGbEGMMU/YYBxJ7nUdygLHkMGT8++wMLGNN+Ci6AkvpCPTWPWSvNIaghPJEW2bUx7Ffg+yJv5COEgQIuUJtObb5Uwwmb3WrH2q/Kqah9qup1UvCzLB9ahvDiT2MJPalSzlErQ56givTdxWdgV516S9yAhKkO7ic7uByDgofDTiCbTS5nxFXzA8ndrMn/jzTZorR5CBBCREg6PyUUIOvoKlZKiLbMn6/zhhzXc42BrhTRAzwH+7rfcaY3QDGmN0istzddjXOnWWKXe5Y3H2eO57a5xX3WAkRGQF6gcw73aYiaRK8HHuK4cQehhN70qu8gzhfzq+JnEh3cBldgWW0WNEGz1ZpJCJCW8BZSLCi5TWAI9gm7AOMJPY5Niw5wAuxR93IwSBW2nY5P0Ub/vgxL+1XU4uzSXuUF2OP0RHo5aDw0XQH++gJ9lUcolIWFwEJ0hNcQU9wRXpsKjlGSCK0SNipf8QM0+69iyPWnKKVC93rOm63sHXq4BK33rPfGLO+yEanGWP6XQN2l4g8W2Bbr1t+U2C80D5Ny4R9gGen7iMibfSGVtETXEF3YDntgSWaWqEURUTSi59Wh48EnJBoizh12JLEHcHvfgpStTNTN5uletbmK6XbsPlpv5r6G6jV6uTUzgsWRZkKZW6IBjqcciBWOwAGO6voboJJps0kgmR0SAhhlVI7ZBFjjOl3fw6IyK3AycBeEVnp3nWuBAbczXcBB2Xsvgbod8fXeIxn7rNLRII4tRCG6nU9tSBqdfCG9o1EA/6r/RSlHIISwpJAloNitiuDI9ZSHlonKtCikYESaEb71dR+0FT1aUWpF4JFUFqIWG3pZe5Rq4OghEmaBDF7nHG3/ErMniBh/KuML1ZEpE1EOlLPgbcCTwKbgEvdzS4Ffu4+3wRcIiJhETkMWAs86IYQxkTkjeK4lj6Qs0/qWBcBv2n2fLOQhFWYKXUnICHCViutVpeTb211ERZHvE2bSSbsEcaSg0zZY8TNdF4B4sVOs9qvqj1nInIQzhLRFYCNE8/9t5xtBPg3YCMwCfyZMeaRas+tKLVGsNL99wBskukeqLN3pZmtazTXA+gDbnVDdUHgv4wxvxKRh4CbReQyYCfwbgBjzFMicjPwNJAArjCzqzQ+wuxS9NvdB8B3gR+6ybdDOKulaoLaMGUhkWqfFaYVg+1GBZzIQNxMO9u4XrWQFdaoQJPar1qENRPA3xhjHnHV58Micpcx5umMbc7FUZdrgVNwlpueUoNzK0pdsQjQIk4bG4NxQwgz6TYwgNtcOUxIWioWag89/ABf/N//xHPbn+VrX/o657xlYy0vo64YY14AXusxPgic5bPPF4AveIxvA47zGI/hGsc6oDZMWZDk3mw6KRzOzea0mWQ6OZnR57nytoJqv9LjNbNfVd/yG2N2p+4gjTFjwDPMrlBIcT5wo3G4H+h2Y7iKMm8QxA2BOvWn2qxuwhJN92IdSw4xkRxxG0aXFzpYuXI1X/rcV3nHOefVafaKH2rDlMVCQJwe0G2BbjoCS9L9jGfMFBO2U3stZk+QNIniB8tA7VftqemCABE5FHgd8EDOS+llpC6pJaa7PY5xOU4Rt3TStqI0ko6Rn7F04BqCiX4SwVXsX34VY10XpHuNhmnDJkncdprCx+wJYkwQkJB715rvUfvZL3/Kd2/8T0SEdWuP4qv//K8AWNaiD5E2lGptmNovpdnws1+C22tUIhhMOuyZSt+wUl43K4yVIxXUftWfmokzEWkHfgpcaYwZzX3ZYxfPZDi3vsh1AF3BZU2d8KssfDpGfkbf7s9guSugQolX6dv9GQDGui5Ib2fhrKAK05ot1Mw4MUiHDUISZvvzf+Db3/0WP/7eLSzpWcKBkQNzf2FKHrWwYWq/lGaiVPslSNo+OUJtmriZYdpMMZ2cctI7rAghCbPj+R1qv+aAmshcEQnhGLUfGWP+r8cmfktPFaWpWTpwTdqwpbDMFEsHrvHdJyXU2gM9tFndtEiUpEkwZY8xlhxk6wO/5a1nncOSniUAdHd11/MSlBJQG6YsRCqxX45Qi9BqdbqhzzYEIWZPMJYc4p4HfsPZZ71V7VedqVqcuauYvgs8Y4z5ms9mm4APiMMbgZFU5V1FaWaCCe/vX7/xXJyaam10BJbQanUSlBa3+O0048kDzJhYusWU0hjUhikLlWrtlxP6jNIW6KY90OPm2CZJEmcsOUjMnkh37lFqSy08Z6cB7wf+WEQedR8bReTDIvJhd5vNwAvADuA/gb+swXkVpe4kgqvKGi9EUFqIWh1seOPZ3P3ruzlw4AAxe5xdQy8yZY+XnYSr1Ay1YcqCpJb2y4kItLn26zeMj0wyY6Z4deglJuyR9Op1pTZUnXNmjLkX73yMzG0McEW151KUuWb/8quycjYAbImyf/lVFR/zyMPX8ZHL/l8+/OG/wLIsjlx3JO+66F188pOfYHR0lN/c82u+8Z1rue2WO2txCUoR1IYpC5V62K+1hx/JRy77KP/rLz6EZVmsW3cUF130Lq765N+o/aohTd2+SVEaTSpp1mu1UzVc+M53ceE735X+3WC4+/YtzNgxbJIIFtP2JC1WRIvcKopSEXNlvwB+e/tWZkyMhJkBYMoeo0WiC75Pcb3Qd01RijDWdUHVxqwYgjjFbgNRtxtBLF0kskUitFhRreStKErZzIX9gtl2izZJZuwYcRMjbqadbgUS1VaMZaLiTFGajGwjN8WMiTGTjBGUFma7hCiKojQfFgEiVhthWombGNP2FJNm1M1ZizZ6evMGjZcoiw5jG5q8ZzaQMnJON4KwtJI0caaSE2wbu52huC4UVJTFyHyxX6loQEdgCVGrAxCm7HGmEmPsjD2tC6CKoOJMWXTsfXGQ6eTUvDBwMLucPWgi7HlxP2PJIR4av40Hx25TkaYoi4z5Zr8AQhKmzeoiYLfw0vM7eWbq92wduZmXY0+qSPNBw5rKouP/fPVu3v0J6DusF7EKLtJrGoxt2PviILf974c4o+tidk0/ywuxx3ho/LZGT01RlDlkPtovmLVhd33tSda3b+T52CM8O3W/ijMfVJwpi46JAzFu+Nv5K2oCEuSQyHGsCR/FUGI3j4zf0egpKYoyR8x3+wXQG1pFb2gVg/F+OgJL2B7b1ugpNR0qzhRlnhKQIMtCBxXf0IPxZIR7R9eWuPWDFZ1DURSlEL2h8ovhpijdhs1P+6U5Z4qiKIqiKE2EijNFURRFUZQmQsWZoiiKoihKE6HiTFEURVEUpYlQcaYoiqIoitJEqDhTFEVRFEVpIlScKYqiKIqiNBEqzhRFURRFUZoIFWeKoiiKoihNhIozRVEURVGUJqIm4kxEviciAyLypM/rG0RkREQedR9/X4vzKoqiVIvaL0VRmo1a9da8AfgmcGOBbbYaY95Ro/MpiqLUihtQ+6UoShNRE8+ZMeYeYKgWx1IURZlL1H4pitJszGXO2aki8piI3C4ix/ptJCKXi8g2Edk2Y2JzOD1FURRf1H4pijJn1CqsWYxHgEOMMeMishH4GbDWa0NjzHXAdQBdwWVmjuanKIrih9ovRVHmlDnxnBljRo0x4+7zzUBIRJbOxbkVRVGqQe2XoihzzZyIMxFZISLiPj/ZPe/gXJxbUZT6IyIBEfkfEfml+/sSEblLRLa7P3sytv20iOwQkedE5G0Z4yeJyBPua1/PsBlhEfmJO/6AiBw6x9em9ktRFjjNZsNqVUrjx8B9wDoR2SUil4nIh0Xkw+4mFwFPishjwNeBS4wx6vJXlIXDXwHPZPx+NXC3MWYtcLf7OyJyDHAJcCxwDvDvIhJw9/k2cDlOyHCt+zrAZcCwMeYI4F+Br9Ry4mq/FEWhyWxYTXLOjDHvLfL6N3GWqiuKssAQkTXA24EvAB93h88HNrjPfwBsAT7ljt9kjJkGXhSRHcDJIvIS0GmMuc895o3ABcDt7j6fc491C/BNEZFaCSS1X4qyuGlGGzZXCwIURWkiJhMhHh9aVermS0VkW8bv17mJ7ymuBT4JdGSM9RljdgMYY3aLyHJ3fDVwf8Z2u9yxuPs8dzy1zyvusRIiMgL0AvtLvQBFURYWZdiwYvYLmtCGqTibTxy8sjbH2bm7NsdRFgv7jTHrvV4QkXcAA8aYh0VkQwnHEo8xU2C80D6KoijF8LVf0Lw2TMVZo6mV4KrXOVXIKYU5DTjPLTERATpF5P8D9orISveOcyUw4G6/CzgoY/81QL87vsZjPHOfXSISBLrQorGKotSGprRh2vh8Ljh4pf+j2Zmv81bmBGPMp40xa4wxh+Ikyf7GGPOnwCbgUnezS4Gfu883AZe4q5cOw0mafdANH4yJyBvdFU4fyNkndayL3HOo50xRlKppVhumnrNas1iEi991qqdNcfgycLOIXAbsBN4NYIx5SkRuBp4GEsAVxpiku89HcPpcRnGSaG93x78L/NBNvB3CMaCKoij1pKE2TMVZtSwWMVYque+HirVFgzFmC86KJowxg8BZPtt9AWdVVO74NuA4j/EYrmFUFEWpF81kw1SclYuKsfLIfL9UqCmKoihKUVSclYIKstqgXjVFURRFKYqKMz8aLMimDuoovlENib4yNqfnA9SrpiiKoigeqDjLZA4F2VyLr2IUms+cCLfUe68iTVEURVnkqDiDuouyZhNi5eI3/7qINvWmKYqiKIucxS3O6iTK5rsYK5Xc66y5WFNvmqIoirIIWZzirMaibK7E2PiK4n+uwR3r6N92OvHxDkLtY6xafy+9RzyXfr19T6Ju88t8H2oq1FSkKcqioH/6ZLbHLiRm9xKxBlkbuZVV4QcbPS1FmXMWlziroSirhyArRXwVYnDHOnZuPRuTDAEQH+9k59azAdICrZRz1ELA1UWoqUhTlAVL//TJPDX5AWzCAMTspTw1+QEAFWjKomNxiLMmE2XVirAUuV6yZDyYFmYpTDJE/7bTs7xnlc6vUtFWc6GmIk1R5j25XrKE3ZIWZilswmyPXajiTFl0LGxxViNRVq0gq5UYy8TLS+bX5D4+XhsvX+51VCLWairUVKQpyrzEy0vmZ79idu8czkxRmoOFKc4aLMpqIcYyvWLBzjGWb7iXruNmvV+v3nx6npcMxPNYwc4xppZ7nyc6UPjcXnlrKTKvsxqhVhORpgJNUZqKQvlj22MX5nnJ/OxXxBqs6bkVZT6w8MRZDYRZJaKsWkGWKZ5GnlzH7nvPxiQc8ZUY7WT3Zid3LCXQEqOlztHQfsTzBc878uQ6BracTmK0g0B0imSsBYxzPZl5a4CvaKtGqNVEpKkXTVGahmL5Y6V7wwxLA4+XdL6UGAvJOHETAUJZ5x6OH87+5Akq2JR5gVWLg4jI90RkQESe9HldROTrIrJDRB4XkdfX4rxZHLyyamE2dVBH2cJsfEWwbGE2tTz/kcnAltPTwiyFSYQY2HJ6+vdgZ6lCRhjfcbjvqyNPrmP35rNJjHYCQnKqNS3M0udOhth5/wZ2bj3bDZ9KWrQN7liXd8zUe1L2+1LB+5+HttpSyqQp7NcCw8szlsofg3K8YcL+5AkFt0gJQSc0KsRNBylhlnnuXfEz09ukBFv/9MklzkNR5pZaec5uAL4J3Ojz+rnAWvdxCvBt92dtqIEoK4dKxJgfmV6rYOcY7Uc87+sVS4x2EOuzAeg+fyuDN70VE88NbRbeD2D67qPT53TyPIprdBOLkht2KGWxQeq9KsebVrUnTb1oSnncQCPt1zzGK3wI/nliqfG1kVuzPGuF8DpW5nlLtWG59ksXGyjNTE3EmTHmHhE5tMAm5wM3GmMMcL+IdIvISmNM9d+eVQizeoqyQoIsRcprlRm+PPDIifjlXgR6RtPPO9Y/C8DwL08nOdwJlgHb20Bl7je27SgGbz87Q9R5n6tUSl1sUKlIqzrUqQLNk3giQP++7kZPoyloqP2ax3iFLp+Y/DPXohTOH8vMPXMElg0ECu7jd95qbJguNpi/LHQbNlc5Z6uBVzJ+3+WOVWfc5kiY1VqUpdi79bS88KWfoZFQnJ533Js11rH+2bRIG9t2FPt/9Dawc+YaSBA95gV2fu5DRUWcHxKMY4XiTsgzh9zFBl4LDDIpV6TVxIumAk2pjvrYr3mOd1J/yGfNJVhMpz1r4Ai0lEjrnz6ZJyb/jNxwpBBnaeBxfjfypbR3LmnCJXncsjF42dZKFhsoylwwV+LMS3F4foZF5HLgcoCI1e5/xAqFWT1EWTmCLDO8mBzuLHEvQ+8ld9Jx7mO+W3Sc+xh0xBm88a2Y8SgAVvsUrW98iol7XouZaXE2tEu5yzRI6xRmMkqgZzQtCnPDqBKMs3xDtmBMvRf1EGkq0JQGUXv7tQAoz+tkOLb1Rt8QYmr8mcmLSeDY6JCM0xd8iP74aSWV3Mg9X5AxEnQQsQZZGng86ziQLxYVpZmYK3G2Czgo4/c1QL/XhsaY64DrALqCy/I/hU3kLStVlGUKskwCPaMkh7uK7h9YOgodcXZ+7AqS+zsJLB2l5z2/peP0p7O26zj96byxnR+7YlaYlYjVNsUhX/y252upMKpXeY/c/LnlG+5lxfLCxW/LEWkq0JQGUTv7tYCIWIOuWCptWyDLA5a7WjLTk5bidyNfKrnkRiYhGeePu/8ma6xn+vmi5TW0BIfSLMyVONsEfFREbsJJpB2Z63yNUoVZrUSZnyADJwSZzhXLc7dn/y4tM0RP/AOD129Mi6zk/i4Gr98IkCfGcknuL9U7N3u+JZfeCcumYV+2UcwMo6aIAZG9lmf+3O7NZ8NGp/xHKZ60UgUaVBjmVIGmVEbD7VczkZ+Mn2m/4ghgMsKTFtMsDTxeUWsmf++cd5gydb6jojfljXuJv9zr0vZRSrNQE3EmIj8GNgBLRWQX8A+4yQPGmO8Am4GNwA5gEvhgRSeqcyizFsKskCgDNyE/b5Wlc4MdWDpK9MQ/MPXokSQHOwn0Oh6y4ZvPzPN+mZkWxm55M+su/H3B8/UvP8D0QE/+C5YNRpC2KQSwJ6Lp86UF37Lp2e33+ed4xPps9n47P38uVf6j67jnSgp3lirQoAovmgo0JYc5s18LgPxkfEjZr8zVmrnep0KlNQoJHz/vXJAxgtYMMbuXIGOICHHTXpW3q9I5Kko9qNVqzfcWed0AV1R1kgYLs2pFGQDLphm+/XSP8hdCYOkIB3/9W+7vd7Fq2YH0q//97fM9DzddwkqVg//8dp7/14uwp2fFnRWe4fC/voXlZz0KUNqKl5RQ8xFpfvlzuWVBppYXF2hQ5zCnCjQlgzmxXwsEv8r+EWs/b+76dHokV8w8Mfkhz+MVy1vzKrlhMc3RrT+puWAqVv5DUeaS+dEhYL4LswwPlF+YMTnYmSXIMgkv8/Z+hX22zyQlwHZ+71ym93UTXnaAg//89vQ4kHfegmLNx5vmlz/nVSy3mECD8sKcNWmmrihKUSoVMH4esGKrJXNLbtQzD6zSOSpKPWh+cdZEwiw34b37/K15OVhZZAoZl8DSUZL780VMptAauPvELDHVfcrT7LvzDVner2BkmtM/9l8cucQzLzmbd/fzeIYYK0amWCsq1FyB1vOOe/NXc4byV3OmqHWYs2zUe6YsMmqR7F6qgMk919LA4/Tbla2WLJYrViv8vHS6olNpBM0vziqgXsIsN+F98Ka3AuQLNA9RlqLnPb9l6Lsb88KMB//57YAjzDLDkNMDPQzetZ6j3rmFnfe+jvG9vbT3DXLKR2/myHPvK+k6AU7wEHGPD60qul9KqHmJtLF7j2H45jOdFaQ9o7Sd/CRTT7+G5HBnugRHeP2zsNe/rlopYc5iAk3Dm4pSmFolu5ciYLzO1W+fxqrQfzdVb0svsXps6426WlNpCppbnLUUb02US71CmZ79LuMhhn95erY4KyDMVi07ABf+noHOSd8w4+4bzs4SbgCJWJid976O99/210WvqxxyBVshsZYr0sbuPSZ7BelwFxMPHUvvxXflr+h0w74RH5FWSpizGBreVBR/apXsXkqY0e9c+5MnZOWlNRI/sXps641NM0dlcdPc4qxOVJJj5tfvMjncma6+71d/DLJDhcvPejQr5wtmhdJ/7/WuGzS+t/5Jqak5lCLS7vdZQTp8++m+od5Yn12RQNPwpqJUR6FcsUK1x7woFmacD4n1ujJTaXYWlDgrxWtWafJ/sHOMxKhPMr+bCO9Vf8wvyT9FrveqvW8/43uW5W3X3ufkdJzeub3g8Ypx7+jaotuUItKSg/4LG3JrpGXWdfMqXlsKdQ1vKsoCp1DB2NR4rep6zYfE+nIFpBanVeaa8posLnAKrcrsPn8rEop7vJJdCNHMtDB885lAYWF2wpJ+zzywUz56M8FIdmi0JTLFBz7x9aqFGTjiLvUoht8cwX+laKDXbbLuhndTdd0cASvp4rQjT67L27ecNlh+lNvMHqiq64SizAfWRm7FwivlItt+pbxHtT5XsyXW+wlFr/FUCNQRnJIWsf3TJ9d5lspiZsGIs1p4zfyI9dl0rH+W3kvudIq3FqFQWQzwTs5PceS59/Hmz17P0lW7QWyWrtrNh77wRU47785Kpl6QckRaLgf/+e1Y4ZnswWACOxbixT/5NDs/dgVjzx3O8C/z67qlitN6UUigVfr3U5TFzqrwgxzbeiOQLLptteHH1Lki1n7AELH2F+yr2Qi8BKQQJ2G3cMfwdfxu5Etp8VUoBKoo9UK/7VxK8dp0rH+W/T/cWHS7QvXHCgkzcAXTxdvh4huLT6hGpARaoZDnCUv6s8KcqZy5F//9fBKjrc5gIoAZd54n93ex/9+9i+eCfw6foij1YVX4Qd9isJnUIvw4V+UvKiU1t2enLiFunAb1hiAJnHSNmL2UJyY/lNWIPZdmyqFTFh4LwnNWb69ZJoGe0YLbZ5bFyKUUYdZIinnSvOZvT4dwQiOpRyZeYw6F3sdqvWca2pxbRCQiIg+KyGMi8pSI/KM7vkRE7hKR7e7Pnox9Pi0iO0TkORF5W8b4SSLyhPva10VE3PGwiPzEHX9ARA6d8wtdABQTXs0Wfqw3SdNCIfuVEmteNFMOnVI5zWq/FoQ4q5Zycp163nGvR+6ZAQyBpSNZbZEyqbUwe1N0Z8mPcilVoO383rl5ZT9KQUJxet7hXZxWmZdMA39sjHktcCJwjoi8EbgauNsYsxa42/0dETkGuAQ4FjgH+HcRCbjH+jZwObDWfZzjjl8GDBtjjgD+FfjKHFzXgsM798yxX80Yfqwn3q2ovHBauWey2ETsAqcp7ZeKswJ4tWXqWP8svf/rNgJLR0AcQbb0L3/O0r/8OUHLZvtX3su2932agbtPLPk85QizSgRXpSLNj5RAK6W35yyugO0ZofeSO+lY/2xp/UiVpsc4jLu/htyHAc4HfuCO/wC4wH1+PnCTMWbaGPMiTkPxk0VkJdBpjLnP7Wd5Y84+qWPdApyVuitVSscrH+z41us5vvV6wOmBmZlvtZApNyzZzDl0SuU0q/3SnLMyGdt2FMO3n+5UxXfrmgFZBVmnB3p4/l8vApzcrEJes3KFWam07p2k68VxAtM2ybDFyGHtvKlvJ1unDi5p/9M7txctu+HX89OLQM8oB3/u+pK2VZqOpSKyLeP364wx12Vu4N45PgwcAXzLGPOAiPQZY3YDGGN2i0jKR70auD9j913uWNx9njue2ucV91gJERkBeoH9tbjAxYJXSQigJt0DasmG9Tv44Du3sWzJOPuG2vn+L9azZdsRNT1HofIiXttqcdp5y7y0X4tCnNVqlV+qLERq9WG6rllLIq8gqz3dws7vnesZ4qyEcoVZzx9G0wtLg9M2PX9wcrzKEWjFOPjPb89qNeWQcv/P3hRIy0xNw5h1LUq7WEhYWbXoirDfGLO+0AbGmCRwooh0A7eKyHEFNve6YzQFxgvto5SIX1V8i+mmKsi6Yf0OrnzvViJhZ2VpX+84V753K0BNBZpXKyqvf24awmxSSrdh89J+aVizDDzLQsy0YMajntuXF/arHV0vjudV/LBsZ7yWLD/rUQ7/61sILx8GMYSXD6dDvJlh394PbS7cIF5ZMBhjDgBbcHIt9rquftyfqR4Qu4CDMnZbA/S742s8xrP2EZEg0AUM1eMaFip+JSGabTXiB9+5LS3MUkTCST74zm0+e1RGoRCvhjAXJ81kvxaF56xWJIf9V+54UaikRj0JTHvncvmNV0NuK6pU7828Flale2mUeYaILAPixpgDIhIF3oKT8LoJuBT4svvz5+4um4D/EpGvAatwEmcfNMYkRWTMTcZ9APgA8I2MfS4F7gMuAn7j5nUoJVJ+jlVjViMuW+J9E+k3Xg1+JT9UjC0emtV+LXpxNrhjHbt+tIHklOP9ksgUK9+6BYC93z7N6ZnZM0rPO+4l0DOabtWUTb7HslBJjXqTDFsEPYRYMqyO0mZhLnJq5pCVwA/cvA0LuNkY80sRuQ+4WUQuA3YC7wYwxjwlIjcDTwMJ4Ao3rADwEeAGIArc7j4Avgv8UER24NxxXjInV9bkZOaQgQ1YRKxBlgYeZ3/yhKzcMv8cKw/71cBQ3r6hdvp684XYvqH2BsymCPUowTMPev2q/aq//VoU4qx9T8Iz72xwxzp23vNWjD37mom10r/pbUjAYJLOeHK4i8Gb3krbyU8y8eBxeaHNPKykb0mNStk6dXDJeWcjh7Vn5ZwB2JYzXqt8s0Yy3/PN5iqnZq4wxjwOvM5jfBA4y2efLwBf8BjfBuTlexhjYrjGUXHIzSEDZzV/zF7KLvtMUqIrlVu2KvTf9NunlVA+ItnQUN73f7E+6/MBEJsO8P1fFEwbqh2NrnlYzvkbIOTUfs2N/VrUrpT+badnCbNZAmlhlsLEQ0w9/Rp6L7kznU/lm89nrCxhVqiBeCmNyFOUKqwm+1oZPrKTRNjCAImwxfCRndzReVTJ5yo0r0LXoxRnrnJqlIVN4Tpd+T0z9ydPyMqx8s9Hthoa1tuy7Qiu/fGb2DvYjm1g72A71/74TbX94j94pf9jPtGAa1D7NTfUxHMmIucA/4Zz63a9MebLOa9vwInXvugO/V9jzOdrce5qiI+XV0k+OdzJ/h9uJNAzytKP/Jzhm88kuT8/zFlurtm9o2tLLqmREmjFvGiTfa1M9rVm7VPOfCqh328BhE++WWTv4rw3mMucGqU05qMNKzeHLGb38sTkh4hYgxzfer0bDs0PczZD5fst246onRibb4KrFnhdc428bGq/5oaqxZkbp/0WcDbOioSHRGSTMSYnI5ytxph3VHu+WhJqHyM+Xk6Sv3M3mhx2Smi0nfEYE/e8NquMhl+u2eNDqwrWOytHoEH5gqtUigmzWnrNRp5cx9678vP9Vix/rmbnKJkaGa5ScjHmVU7NImC+2rBy6nQ5FA9zLoiyEXMoxp56ZSO7ht6AE4SyWbPkIY49aPOcnb8sShBsar+ah1q4Lk4GdhhjXjDGzAA34VTDnTOir4wV3cYrT2nV+nsRyyt/KYkECuc1mZkWph49kt4Pbc4qG7Hkss2+uWbFhE2lHqtaUY0wK9drNn330ey+7a0kp1pJ9bUzsVb6f/FWBnesA5ycwCduuoxHrr+SJ266jMEd60rKNyvl30M9SOVi9PWOY8lsLsaG9Tuytvv+L9YTmw5kjc1pTo2SS8NtWCV4t2JKUXghq1eYc96WjWhQWNIRZqfgOFsFCLBr6BSeemUjAP1Dx/O7pz/OHY/9I797+uP0Dx0/Z3MrmYz3Te1Xc1GLsGa68q3LLuAUj+1OFZHHcOp+XGWMecrrYCJyOU5vKiIhr5WRtaP3CMdDs+u+DSSn81drDmw5ncRoKvSZv6IpOdhJx+lP55WN6N/XzSqf0GYpHjSY2ybopYjCWgozcN7b3Lw+AEyQ/m2nA7Bz69mYpLP4Ij7eyc6tZxM5PknfmuasmVYoFyPz7jP1fAGtdprv1MyGZdkvq76ehJSIKrZa051Z3v4xu9e3lERT0yRhSsdjlt8sfdfwybQdvI/t/edgu/YrFu/mqf4LmOmN+NqvRt1Upvjghf+j9quJqIU4K6Xy7SPAIcaYcRHZCPwMpzZI/o5OW4XrALpaV5Vcxyj6yhhTBxXOIfNatdl7xHNpkZbbAL3rOGd8+zcvIzGaH/4MdI/6nqsagQZzI9JK9dRVJMwKENlrZYjefOLjHc5ijWROwd9kiJeeO6OgOKvIwDUgF6OmOTVKtdTMhmXZr+CyutdhKyyufgzA70a+1LS5ZSXTAEFW7PuEx3wCT0Z46bkz0sIshV3EfhU9n0u9RNyyzhHvcbVfDaEW4syvWm4aY8xoxvPNIvLvIrLUGNNUffGiA/kCDWD5hnvZvflsTGL2wyahuNOSaF8YlnmHFooJNKBkkZaiGrFWbti0WBi2oDArsggg2DnmKXghlQvobaimp8orBDyXaC7GvGXB2DAvvNoUzYvcsjkQZKUKIk/EgPHQ9WJ87VQt7JffnKsVbftGu+jryhdo+0a7Zv8W86AG20KhFuLsIWCtiBwGvIpTXO1PMjcQkRXAXmOMEZGTcXLdan7bVqn3bHDHOvq3nU58vINQ+xjL/vjetNcMZj1oqTBnqihtuiVRhQINShdpKeYiL60eogyyV2cu33Avu297a35oUxKsWn+v+/fIN2ThqL+3stFhAa/6TPGERaQlzuZvXK/u/+alaWxYpXg1NE951HLDn7mvNx11EmVVCTEPVhz0P+zZ+XqyHa+GFQf9D8P7jmB6ymMlfwH7VS1e11eOTfz+PWdy5Tm3EQnF02OxeIj7dxzBD/7i6yzrHGHfaBffv+dMtjxzvAq1OlO1OHM7rH8UuAMnM/J7bgXdD7uvfwenXcFHRCQBTAGXNLL1SqZAG9yxLi+3qX/Tuey9awN9Z29JC7Ou455LP4/1ebRBSokSD5GWEjOliDQoXajlsn7kRc7f9yhLEpMMBVv5+bIT2dZ1WEn7lroKsxbCDGYFb+ZqTSs8xUGnbkmHmTP/LgBWIM6h6+4paZ4lU0MDk5uLMTYRJhqeoavD+Tcx34s1LlTmow3LxKuh+ROTH2I4fjjHtjuhzabPLTt4JRuOfoIPnvHbfBFQIbUWY7kcecJvANjzyuscD5oYeo96jFWn3UN4x948+yWBOCtO+e8850A9i2rnvgeFxFrqvc78G9y/4wjedsLjacHW1zXClefc5myP+7dRkVYXpEnsiyddravMqUd+qOz9SvlQpj4gT9x0mW85DQnGWbnxriwvWgpPgZbCx4uWopBIy6VUobZ+5EXet+cBwmbWa5P6y/oJtXLKYhTNLStDmGUSHfB9icEd69jzwGlMT3USjo5y6Lp7aptMWwOjUmjp+Q/+8SbPMOfewXYu/YfadR+6Y/j6h40xZS2VCh98kFl91ZUlbfviX11V9vEVJ+fs1M4L6n4ev5wyMBzfen3TizKADUc/kee1sY3jkxooQ6jVWpB5dZYpldyIzKr196ZvPCuh1iIu+spYUUH8g7/4umeoc+9IF5f+x8eyB6uwp/W0YfPVfjV3+6aZePFtPCgnvFmoEK1JhOj/xTkAeQItstfyF2gFvGhQmicthZ+AyhVt5+97NEuYwayzvTcxyXt3P8jO8R5+HT2m6Dlz51mQIg3NKxFmmUYtHB1l3Ym/rP0igBpQrI2JFmtU5gL/YrTCE5N/DjRhI++c0OUHz/htljADsFwDluWt8RBotRBk1YgwL2otzMB7jtUItj86fTtXnnwbkaCHV8x9n30XCXiNa15aTWlucQbOH7qCHIRSBVrRQrTGYvfmswFvgQYFvGglijQoz5sG+aKtJzFZcPsoCT48sbWoOCt59WUVogwKC7PMcMD0VBfbn3AEspdAq1iYVWlANqzfwSfe/zsCgWzPc+bSc10goMwFhYvRBnhq8gNAkwg0H1vuJwJSREJxPnjGb9OioVpBVmsxlolXqszOrc53SLUCLZdKBdtZhzzCZ974E4JWjv3KeZ8LLhLwQ0VaTWh+cQZ1FWivOeJ3bH/inLxlz5mYRIiBLad7hjehiBcNioo0yBdF5Yq1AauTFXbhZNPlPq+XVQ6jClE28uS69KIKv7tJrxIafkvQGynMrnzv1jxhliLlGWt4A2dlUbA2citPTH4I74ogTsHZ7bELGyvOithvPxGQybLOkYpFWS3E2J6BWfsV7Bxj+YZ7Pb8TXr3ZuwzQq4+cTusflS7OCqV8FCLzWr2E2lmHPMInT/5pnjBLkXqfo6+M+S4S+P49ZxafyMErVaBVwfwQZ1VQTKClvvB3PHUWyXgUPwOXGO3wLbUBJXjRIFvYFMlL8xNMmaLtLVNP8+GJrSy3RxmVKDMILQUqg/fTXVFdsmKCLEUxYbb7l29NN5qPj3fy8pZzeXnLuVlCrdQSGo0SZuBdbDaTlGdsy7YjOPqwvbzj9GexLINtC3fcf6QuBlBqyqrwgwzHD2dX/Ez87Fe5fThrRgFRlpnvNDYVZSZh0RL0t58Dk91lnbpSQeZl40eeXJdVTikx6iwc6990bp5Q86vhWKi2Y6nzgPJEm9fig8tf+6t0KNOL1Ps8dVAHt4//EUc//grveN0jWGKwjXDH4yeUvlBDvWgVM3/EWYXeM/AWaHt3HcVLz52RTjY/4ti7AXjusbeDyRcZoXZHDBQSaFCiSIOyhFomKXF1nnmUT3InrTgfsm4zhSG7cmammZ4kxDWcXfJ5aiHIUkQH4A93bEgLs9wZpqv/H0gSjo4WXYLeSGEGhXPGMj1jG9bv4G1v/EPawxYIGN72xj/wzIt9DRdoVnzxNp5fCHiVzugJPe/mmAXytm9IwdkiwizTI9PVOkUiKdg2iGu4JMOAxRIhrnvsnKKnLFeQFbLlmQxsOT2rzqWDM8HEaGdW6otfDcdgZ21yY73mXKpgG18RZHnrAd/Xc9/nsw55hLe99nECrpctIIa3nfA4z/QfVN5K2jp40Ra6DZtfV1bFHzfzC33vrqPY/sQ5rgiQrLymda+9DSuQfVchgTir1t87e6yB4h+GyF4r/SjKvnD2owSu4q60MEvPM+Mxg8UQrdjALrr5DBewSU6s2RxKvbbU+5Rqj+VHqvr/oevuyXv/M0toNFqYgX/OWDIpXPvjN2WV1PBrh6IolZIqneHkmUm6kTnA8a3fy+u3OecFZ0vocem1ACAYMFiWI8pEnNWaxsCeiW7+5cF3cffLr/c81viKYPpRClPLZx+lUszrZRIh9m49jVifTff5W5Gca5NQnO7ztxLrs0t6lEs517Qn1uM5nrAl73328rKl8tLKpknabs0X5o/nLEUNPGiFWmucctZ1AGmvWqGVNinhUewDkSliSvrgFRJHrpdtFQcKHiKMzT5aWC9/W9pxS6DUu5RKcyWmpzrTYeZMr+ah6+7hUPNQdvfDcqjxHZtfLlmmMIPy2jkpSqlsj12YVe0fZvPK3tz16fQ2DSk4W6JtLrYAAJzVmnsmurl402c8Xy9HjJVLrp0O9IySHC7c6zk57HjLUsXJh395Osnhzvyi5RWcP0UpNjjzer1s8bde2Mhnj7qZaMZN8FQyxFcfyBfAfl62Uv5+nmiYs2TmnziDqgVasdYafWuezUo+L2YEMj8ANRdqubgCq7+nmzWBAwU3XWUOwP7KBVm5LuNCoswKT2FPtxbcPxW6zH3/qyqVUQcjUGrjX12tqdQDv/yx1HhDCs6WaY9LWQAA3uKg1qKsFDvc84572f/Djfjl9AEElo6mb547zn2MjnMfK3zQCm6Wc+dazEZ7CbVfDZwEwBWv2cyKyDB7Yj1864WN/Gr6JFiRvYhgYLKbFW0H8o47MNmdXjRQEbpYoCjzU5xBVQItEhohFu/OG/drrZH6x1qKUSjVmwbeH6xSBds1E+fyxY5baBX/xM5+u7ukYxWaTzFK9ZIddOoWXv7dW8F4v4eZocvMfMBIaIS1K+5i1ZInyp5bPT/8pTT+1dWaSj3wK53RsEbmFdhhr1WAXmQuAijF/tZSkGXSce5jxHYvZ/zX6/ESaNIyQ897nFDf2L3HMHzzmST3dxJYOkrPe35Lx+lP5x+0WK5xCeIt8zpKFWrRAUegpURaLuMrgunvvOseO4dPnvzTrNBmZl6aCrT6MX/FGcz+Ycs0DmtX3MVTu87HNi3psVJaA1Ui0qA8t3qhD1jmB3HTzEkwBle13c5q6wCG2aKNAJMmxDUT55Z87HKoJGx5SPtTRF6bTIuuQGgKARLxaFb1/1Q+YCrsHIt389Su8wHKE2hz9KEv1CGgVA+bopRDUzUyr/AG+fbxP2LmoQiXv/ZXLG89wOh0K62hGC2BWRuXEgG1EmUlCzIf0bTsz+8icuSraeGFZcCWLAE2du8xDF6/ETPjfLck93cxeP1GAG+BVs48ioi1UoVapkg7Z/nD+R60gZPS73kqzJn6Ow1MdnPdY+dkhT9VoNWH+S3OUpTpRUt9yW/fczaxeBeR0AiHHLu1YBX6TMoRaZAvZirJgYD8D9ydvIE7eQMAb+98mCv7NrMyNMzueA/X7t3InaMnEansVHlUIshya+zkhiu9ePmpN+XnA5oWtu85u3RxNofCrFCHgNRPFWNKLWmaRuYVCrPUyvm7X3591pf8WYc8kiUCvvHS251QW6Fj1UqUlbhivuP0p9Miy6sW5bZb3pwWZinMTAtjt7yZdRf+3vOYJZc3KkOspa65kEj74yMe5rOrbyZqOV6xVdFhPnvUzcBs6HN8RTDv7+RF6m9akUg7eCUMl7/bQmdhiDOoSKBlfdkb4JXZf2S5pTa8+jpmio9ylnB7CZ1KBVuK20ZP4rbRwoasVCpN6Ify24nkhjBjce+kW7/xLOb4DqzQakwVZEo9KSWvzKvcRs0EXJXCzItMEVDMnhazl7UUZIUKgg/cfSI7v3cu0/u6CS87wPRAt+d20wUEmN/xi4q2zPn7CLVCIu3Kvs1pYZYiGohzxWs2Z4U8U3+LUmx7VV40JYuFI86g4jBnJtFXxugfOp7t/bOhtWIthKByoZY+r48gqla0lXu+Sqi0v9veXUex/bG3pcPLTh6gdxHdUKBwe6pGuMZ1NabSrKTKbaRCn5nlNqoWaHUQZimqDWEWFWUlCLJSu7MM3H0iz//rRdjTjv2aHujBz34FO4rYrxLmUVCspa6rDJG2MuTtrloR8R7PzEUrhAq02rCwxFmKKkXa9j1n+5baKCX0mfsPuJrWIbUUUbWkmoa74Ijgl59+U1ben4PgGLjspNt4Mkz/0PH5oc0G5ivoakylWSlUbqMqcdZAYVaVt6yIKCu3Xd4JS/r54Q2fSAuzWbztV3KyhcgDh3Dkufd5Hi+3V3KxOfoKtRJEWkqg7Y73sLolX4j51UGD0gWaUj0LU5ylqHBFp18Iza8ERzFqKda88EvqrCW1+EDm3k2VFKpME8zOO2uCJFJdjak0K8XKbVREnYTZWYc8woded0dB+1Wxt6wGouyEJf2e4+N7/ZrN52MSIR745nt8xZnXOQoJttS8KxFpqffr2r0b+XxGzhnAlB3iX/dv9D0vlCbQ1HtWPQtbnEFFXjS/UhuR0Ej6H1ylDXjBX+hUItrOWf5wVkHBVdFh/umYH/Hazhf5yo6Laja3Sin0AfV7n/2IxbsaKsq8VmZe++M36WpMpemoebmNOgqzT5zy04L2qyJhVqUo8xNkmbT37Wd8z7Ki26UY31ueMM6cg59QK0mkZQi081oe5qq221llHaDf7ubW4Tfw5o5nshaS3TZ6EiwvHLVRgVZ/Fr44S1GGSPMstSEzrF1xV/r3zH901Qi1TEoRRrkC7orXbM6q9AxOSY2L1vyex0YPS9+BzqUrutQPpN/7bEmchN2Wt33D6jjhvzLz2h+/iUv/4ZKGzUtRvGiGchul2MUPve6OgvbrVrwjAJV6ywqJslIEWYrTO7cjn/g61//tZ5iJzbama4lM0RKJMX4gPzS4dOVeTu/cnjd+7+jaoudLza0ikeYKtPNaHs6qjbkmcIALljzEP+x6j+disqkiAq0UVKBVzuIRZylKEGlepTYKFULN/cdXK7HmRa7I8kvetAT+30NvIzySLFijplZU8gH0e5+BfNHWqDpOLroyU5lP1LTcRgVes1JzzArZr79cu5lbt5+UVyboq7FznDqPXvgIs2pFmZeoOu28OwG4+X//Jft397F05V7e8zf/DuAp2lKvFTt2IbFWikjzE2hXmdvz+jG3Spy/WnWb70r/QgJN88/qS03EmYicA/wbEACuN8Z8Oed1cV/fCEwCf2aMeaQW566YIiItr9RGGcylWPNrrwFO65PM6s4r2pzfj136EqetfrZiwVbLOyHf93nwQOPrOGWgKzMXNvPShhWhJm2c6ijMwEk+XxX1FmgrQ8O8vfPhrLyo1S3D/EvoZv7O/IwemaLf7uaaiXMdsVamMKtUlGVy2nl3pkVaLrmizW87v3NWKtL8BJpfP+ZV1oGshQLloOHN+lG1OBORAPAt4GxgF/CQiGwyxmSWQz4XWOs+TgG+7f6cEwrW+8nMYaqiBEchvP5h1kqwXffYOXz21JuyugOksI1ktd0AiATjXLj2/vT2KcEGeAq0Of1QZfwtVoV3N1SM5VJsZWahTgFKc9PsNqyu9coazLde2Mg/HfMjT/u1O97jWYurRZL0yhTghOa+2HELSIJNnJh3jEqEWTFB5sWbojuzf7/4eq6++HrPbbdOHVzSMUsVaaUKtH66WeMh0FJt/rwKmd82elJNwptK+dTCc3YysMMY8wKAiNwEnA9kGrbzgRuNMQa4X0S6RWSlMabu2d1l1fuZA6GWopjoKVW83f3y6zl26UtZgguc1ifhgHffulxDGAnG+Ytjb+f39xbPfagLTbDyshj3P3kQ7zzjmez32F2ZWUqnAKWpaVobVtd6ZcWos9cMnEr0r+18kYvW/D7rszVlh7h270a+suZHRY/XKnGu4q48cVZPYZYrxsohd99iYq2YSCtFoJ1nHqWVmbwiH6k2f+e1PMw/9s7mo61uGebzq51uAbUqbq6URy2aLa4GXsn4fZc7Vu42AIjI5SKyTUS2zZhY1ZMrVO+nIDt3zz4aQPSVsZIf//mzs/mXX17A3pEubAN7R7q49va3MzBaeqmKZZ0jdbwaDxr8/pbDhvU72Hjac1lfHsaAbeCTl27hE+//nW8+2mJARA4Skd+KyDMi8pSI/JU7vkRE7hKR7e7Pnox9Pi0iO0TkORF5W8b4SSLyhPva191wIiISFpGfuOMPiMihNbyEmtmwprFf84iv7LiIv3v6ffRP9WAbeHWmh79/1UlS3x33r7mVSW7Irlxhdnrn9qLC7E3RnelHLSn1mIXm53ddq5Yd4DzzKF/kZyxhMi3MDDBlW8RMiK91/JhrOm5KC7MUUSvOlX2bgcIrZksqHFzH1J5qaVb7VQvPmYdDOq9McinbOIPGXAdcB9AVXOZdbrkMCtX7+d3Il0oLFcyhR61StjxzPFueOT5v/MpzbiMSmv3Q2SbfcwawrwwhVxEZ76ETpvnYvAnTfOSi+wgFs1eHiUBrxBVkAe9/posoHy0B/I0x5hER6QAeFpG7gD8D7jbGfFlErgauBj4lIscAlwDHAquAX4vIkcaYJE648HLgfmAzcA5wO3AZMGyMOUJELgG+Alxco/nXzIY1pf2aI6opNPurgZM8V2Zeu3cj/7jm5jzhkEs/3ennlQizYuSKp1tvvZB/+fKn6e9fzapVr/LJq7/EhRdWt2ApdY5CnrTTO7eXtLozk6u4K28hgAARsYmK07nA8uls4NdFYIHRlParFp6zXcBBGb+vAXI/BaVsUxcKlV9w6gBJOlTQP31y8QNmenya0Ouz4egn+MFffJ3Nn/gnPnjGb7nj8ROyPGq/eOQkYvHs7gexeIjv33Nm7Sfj8T6lwjQVvfcNorOttB58uSyWTgHGmN2p5HhjzBjwDI5X6XzgB+5mPwAucJ+fD9xkjJk2xrwI7ABOFpGVQKcx5j43fHhjzj6pY90CnJW6K60BTWvDam6/SqVBN6Fv73yYu9b+E08e83HuWvtPAHxm7CJ2JbuxDQzZrUyb7K+tSUJcw9lAdcn/Xnh5tW699UKu/uQ1vPrqQRhj8eqrB3H1J6/h1ltr482s1DPn6z3zWQhQyqenVM/lfKZZ7VctxNlDwFoROUxEWnAU5aacbTYBHxCHNwIjc5FvBk69H4vcL9f89hoVhwqaSKxtOPoJrjznNvq6RrAE+rpGeNsJj/P9e85k41f/jkv/42M8038Q0/EgxjihuZHJKNf+6u2eXreyyH0ffN6LxRCmAee9vf/Jg4pv2CCshJPkW8oDWJoK1bmPy/2O67rrXwc8APSlPufuz1RwxC9EuNp9njuetY8xJgGMAFWUu8+iaW1Y3e1XE5Fambm6ZRhLsvOezhj+LEcMXsPnx89n3IQxOO/CEK18hgvYJCdWdM5Kkv//5cufZmqqNWtsaqqVf/nypyuaQ7mUO+cBq7LONjO2xbV7C3cLaCSl2jDmqf2qOqxpjEmIyEeBO3CWoX/PGPOUiHzYff07OO69jTgKcxL4YLXnLRWvej91aW2SwkuUzNFd6AfP+G1WCBMgEorzwTN+y5Znjk+Lt8xtwqEK6tRUIULr+t7XidHxMF0d5XnPROCNx73Ct/9PnSY1t+w3xhTtSSUi7cBPgSuNMaMFbgz9QoSFQoclp0aUSzPbsDm3X3WkUG7S1HI8V2ZGrThXtd3OppmT8oqoAkQoHO6E6rxmXvT3e6ZL+45Xeu5SV3UW4zttb+Lvx24r2xMzYUcWymKAeWm/alLnzBizGcd4ZY59J+O5Aa6oxbkqIbfej5OrUcPWJsUoJGZqKNz8kvpT46WItw+e8VuWdY7UrRxEzdvKzAHf/umpfPx9v6MlNPtZMqZ4WGAR5ZwhIiEcw/YjY8z/dYf3plY0ui7/1IJ8vxDhLvd57njmPrtEJAh0AUO1mn8z27CG2685wi+/aZV1AICr2m7Pyz1rZXal5nnmUa7efwfL7VEGrE6+0/Ymfh09xvd8lXjNAFatepVXX833iq9a9WpFx6s3v44ew2vG9vGnPJgl0IrZsK7AZN3n1iw0o/2qRVhz3uEVKmhYBXqvcGChRwH8kvr3jTo9KQuJtw3Lt3LlW38xGxJ1y0FsWL+j6kvMpKne+xLZsu0IvvajN7N3sN3J3RtsZ9M9R6d/Tya9LdxiyTlzcye+CzxjjPlaxkubgEvd55cCP88Yv8RdwXQYTu2wB93QwZiIvNE95gdy9kkd6yLgN65gWnQ042eoFqvx/PKbUnW4UiItl1U4KxK/xK2ssEexgBX2KJ8au4O3TD3tuU81fPLqLxGNZguXaHSST179pZqfq1Z8Ts7n47ybXXRjA7vo5odTp7Ir2Y3fp2gx5JtB89qvxde+iRq3NplrCgi079/6uqx6W+DW4rr1dYB/IVVjCx+56L45aU80X9/7LduOyHsfUiHL3DpnMFsDbZFwGvB+4AkRedQd+wzwZeBmEbkM2Am8G8ANGd6MU0csAVzhrnQC+AhwAxDFWeV0uzv+XeCHIrID545z0TY0na+foWJcu3djVjcAmK3DBY5IWxM4kLefjfD33EaU7BSNKAk+PLGVz/POms4ztSqz1qs1M6lVSDOTTXJidi24yTCfm8QzXJyqM7dIaEr7tSjFGdSotUmTkRIPfpXqv/+L9fzNn96TVxYiEDC+KxLrEZpbaO99sfd9oWOMuRfvnAqAs3z2+QLwBY/xbcBxHuMxXOOoLLzPEMwWO/Xro3nNxLl5IgIgiKEH7xDccnvU93z3jq6tOLR54YW31lSMlUO5pTQ8e21msGnmJBhzwsarrANZ3QEWA81qvxatOFuoeHl4MvHzpPrlHiyW0FwpFGrRVOx9VxSlOLeNnpQlCmJ9szeSm2ZO4vVTL/H+6H159srvm3XA6uTxoVUVLQrYOnVwzQvOlnreSvBrhp7iPPMoV3EXqzhAP91c03JuWvhumjmJTTMnVdRfsxS0t2b5LMqcs8XKB9+5LSupPZdc3bbIQnMFSYUu+3rH65qTpyjzkXp9+eaKhT8OP+N7I5lr2aYI8p22NxU8fjEvVD3Ci9Wcr1KvWapLwBoOYAFrcHqSntfycEnHKdRbs1jjc6Uy1HO2iCgWojQ4ZSM62qYXXWgul1wvWSQcn5OcPEVZqLTvSfiW04gOFG4RlMJvUQA49muYVrqZzFutWan3DObGg1aKCCwkzHK9Zm+ZepoPT2xluRmln25amcnrEtAqs2VKIF8IK41Fxdkiwm9BQApLIDYd4uKr358WJ5+8dMuiE2pejcz91tUspnIZitJo/BYFgBMGmqSF85ZfkRYnfz9226xQGzrGU6ClRE+h/LNM8VQroVaOV65cYfapsTvSCyTWcMC3oFYhsZuiWq+ZhjQrQ8XZIuL7v1ift6owl2VLxj3FyZXv3QowbwRaofywYnzwndvy3iPNyVMWFTt3z3kLJz/vWWSvlc4981sUkGIVB1g/8BKfYlacpMpqAL4CDUpfIJArqkoVa+WIsUP27+fEXbtonZlhKNhKbFmQbV2HZW3jl2P24YmteStX/XLyUmVK/LxmhYSZUl9UnC0iMlcVLl8y7ik49g21e4qTUkJ41QiiWlKtuPTzhuUWbdScPEWZJfrKWNF6Z4VCm4VICbTUysJrOm4iKPn+oH66uYq7fMtq/Dp6jG+Ic/3Ii7xtx5MsSUwy2dLCo2vW8PLS/GK/udQ6L+2Q/ftZ/+LLhN3qDL2JSd635wGAtEArlPzvt0I1136lypRUGs5Ur1l9UXG2yEitKixUm+uTl27x3LdQCK/W3rZKhF5qHy/hWU5+mF/4d3Q8TGwm1HDxqSgLlVJyz1ICLdeDlmqA/jW8e6ZlipZcgbZ+5EXet+eBtCBqm5lh/YsvA5Qk0HLJ9HyVKvTuHV3L+pEXeePuFwnkBCLDJsn5+x7le8nTfPdPJf/3080aj2bnQybKlB1mlXWAfrubaybO5c5X3uB7PA1nNhYVZ4uUQrW5PvjObZ7ipFAIr1JvmxeVCD0vsZlLqflhXuHf2HSAb//0VBVjyuKhgtBmLbxnpYQ3c2tz9dvdXBM4m01yIleZuzzFSW4D8JT36YQl/Zy/79G0MEsRNkmO2rmHH7Wcmh4rJex5yP79nPLSSwRtZ65tMzOc8tJLQLbQy80jSwnEXGGWoifh304ps5bZNZzNF/lZ1gKASRPin8YvSCf/Q+EFACrMGo+Ks0WMX20uP3GSG8LL9G755TRUkjBfidDz2ieXUvPDFntRWUWpN9WGN2G2NlcWy6a9xQmhvLIa6RWNA6O+9mtJjiAqpZTF23Y+mRZmKYK2nSf0cvESiJnkikvwLjC7SU4EA1cl78ryks2VMFNqg4ozJY9SxEkpniqoLGHeT9AVEnrFRGCx/LDcMOr9T+Y3NlaURUedvGdQeWmNlLDILFCbZl+YTZwCS8kuuMrZbBo/EcZh1bIDeSsa/RgKtha9jlxyBV2x8VJez63ZlivKsgrMJh0xdg3npj2LV7U5XYQqDWNC6cJMvWa1QcWZ4omfV23D+h185F330dk+7buCMUWlCfOFeoBuWL/Dc16++xgYKOL58gqjnnfGbLHL+bhaVVEaSa0EGhQWaZ4CDdi0/xQ2LTsxb/w88yh/N/BLljDl6y1LMUWQb0TPzAp/lsJQsJVeD6FlI6wfeTFr1WVmYv9eq5MVHsn8CYSvdLyNG8f/CDzuQVMFZlOewjWBA3yl4ycIQosk02Nfar+FUKfl2ZapocKsQL/oxYyKM6VkNqzfwcff97uiXQYMVBUK9Cv5EQgYPnXpFj528VYikWTWOfxCsdf++E1F51BK6QwtOKssWiosq1ELgQbVedEAWOb0DT7PPMpX+ClhvAUdOLbLQF4RWyjeHinFN6JnenrlAhg+uPv3XLz7IeIE6CSWdZ7vtL0pb79JQnyGCxyvnw9XJe+iNZBdWiQs+dcYteJc2bc5T5ypMGtOVJwpJVOs/RM4XqpL/+GSqs6TEkCfeP/vCASyzycCrVH/hQKV5ImVmhenBWeVRUsTCzQo7EVLibSreu4iHPAXZgCv0s0Z8glWLT1QbMq+pATdZ8c2E8xJ7hegjTi4Xq6sGmzRYzgw1pofjpUTvU/kXteq3tLnujI0nH5eSg0zDWU2DhVnSslUm9dVDlu2HeFb0iOTTI9Wpc3Hi3VOyNxOURYtTSDQoEIvGsWr4adKcYB3oj04+Wql8OvoMfz92G0lbRslwYfG/psbx/+ITXIimzjRf+OUNzCDQl0Tctkd72kOUaYes6KoOFNKppCISSalpBBirc6XSbUerVI6Jyy0grNW3OjKK6V8GizQoHSRBtlCrZCISSBO+NDPS5U6ho9o89zWp96YF6v8tvMQY7lcM3EuX2q/hajl3TUhxVQyxL9v31hwm3JsQqOF2UK3YVV1OhWRJSJyl4hsd3/2+Gz3kog8ISKPisi2as6pNI7v/2I9M/H8NNp4wuKrP3xzTYXZhvU7iITjvj0tM6nWo7Vl2xFc++M3kUx6pwjXQ3gqzYHasAqo8Mu11C/z9j2J0mppDRQPzUX2Wmmxds3EuUyb/K+8GRPgqtFL2LT/FEcMpR5V8huOLJDdlk0/3dnnLmEOqWu785U38PevvoeEx7WBkwfcP9XDPz/7Hn41kL8YIEU53rJGC7PFQLVt6K8G7jbGrAXudn/340xjzInGmIXjflhEpEpNBIMG23YT/w2MjIf53//fGb4rO3/wjzex+RvX84N/vIkN63eUfK4r37uVrjquCM1ly7YjEMtbCYplVJgtXNSGVUIVAq0RIu1dU//DJyK/IoRN0szaryG7lU+OvSe/Xhpw3siT3JP4Kjvsv+WexFc5b+TJkoXTeeZRLuJ/SvqCnTQhrhk9t4QtZwVZbp2y20ZPwvKRggZ45/1/5yvMSn2focrcMhVmZVFtWPN8YIP7/AfAFuBTVR5TaTLyappJ8ZWQ1bRz8isoawyMToTBQEf7dM2Lw/qFUTXXbEGjNqxSqmiOXmqYE2Y9OpWGO9/e+TCfX31zVthvyg7x96++Z3blYk6e2nktD2e1h1oTOMAXO26BMWaFXAGBdlVP/gpKcGzYsGnFYOiRKc8CsZmU0/dyd7yH1S3DeeN7Yp7O4LkJYaZQYVY21YqzPmPMbgBjzG4R8VtPY4A7RcQA/2GMuc7vgCJyOXA5QMTSL8VmoFYV+0ttnr7cr/E4cPHV7y9pzpX05iy1M0IzMZn0bnKslExNbdiis19VCjSgbiINHKF2Zd/mvHys3LISuSLoE2t/ldW3E6BV4lzVdruvkAJH1F3VdjurfRYfGGD90Ofzxr1E2Ns7H+bKtZtZGRpmd7yHa/duLFij7N/ZyGePuplohiicSob41guzeWbl5mjVW5TF7AlCUn0IeSFSVJyJyK+BFR4v/W0Z5znNGNPvGr67RORZY8w9Xhu6Ru86gK7gshIyjpR6U8uK/aU0T/cLZZbqwarUazef2jYNxXezI/YII4kSll4tcubShi1K+1WFQIPyvGhQukgDR7isPCbfmwTZZSVKfW2VdcDXm/X2zof5/JLCifm74z0lecNyvX2rW4b5/OqbAfjNDm9xmApbXvGazayIDLMn1sO3XtjIrwZOmntRBgWF2bQ9yQuxx9g1/SxHRk+u/lwLkKL/uo0xb/F7TUT2ishK945zJeD5TWGM6Xd/DojIrcDJgKc4U5qPSsJ9lexTqD/mTFyIhONs/sb1RUVTNU3YKy3HMRcYYxhK7Ob52P8wnNhNi0Q5MvoGnp26v9FTa2rUhs0BqS/iOfKiQbYXqJBQ2xPrYVW09HAf+IcId8f99/Hy0GViG1gVGuautf/k6wUrdKyoFeevl272FWfgCLSUSEu9P+1FWlRlnaPOoixmT/Bi7HF2TT+LwWZVy1qWhw7m2an7qj/vAqPaBQGbgEvd55cCP8/dQETaRKQj9Rx4K/BkledV5pDv/2I9selA1lixcF8l+/h51YwBEaGrfRpLZj1hfgsMKvHaNTPGGAZmdvLg2C/YNr6ZyeQI66Jv5IyuizkkclyjpzffURtWS6rMLUotGChXJKSS2r08RN96YSNTyVDWWCrcl1pMkPu4du9GpuycfewQ1+71L0Xh520zxhFmljhFtFNesLd3PpzeJvf8fsdaEfH39kHh96EQVa3ATLFzt+/ffyI5wlMTW7ln5Ce8Mv00K1sO5/TOd3Nc2xlEA6UL8sVEtTlnXwZuFpHLgJ3AuwFEZBVwvTFmI9AH3CpOrCoI/Jcx5ldVnleZQyoJ91Wyj5+3zbaFUDA7YbeQJ2yhJPbbJsnumed5KfYE4/YwEaudo6N/xOrwkQRESxTWCLVhtaZKL1qKSrxpkJ9X9Sv8w31+/GbHSbSM5u/zm4GTiPrss+c13h66pLEIWtn2q5gXrFRvXzV1vmpW1b+AIB9J7OOl2OPsib+EhcWa8DoOC5+ggqwEqrLwxphB4CyP8X5go/v8BeC11ZxHaTyVhPvK3ccvIT/c4h3q9POEzcfE/kxm7Bi7pp9l5/TTTJtJ2gM9HN/6Zla0HI4l1Tq7lUzUhtWRGos0KF+ogSNg7t3zWu593PkTlpKrBtkhwlL41gveCfkRn1BnIS+Y17FiiRDX/8/bqi68Wm9RZozNQHwnL08/yXBiD0Fp4bDw8RwSOY6w1Vqbcy8C9PZbaRr8vG0ffOe2sjxh8ymxP5Ox5BA7Y0/TP7MdmyS9wVUcFzmD3uBqpFjBN0VpVmok0qByb1omfuKmVNHmh19C/hWv2VxWzltKTH51JMnlr/0Vy1sPMDDZzXWPncPdL7++7HnVvO+ljyiL29O8OvMHdk4/zZQ9RsRqZ130FNaE1xGUltrOYRGg4kxpKvy8beV6wpo5sT8T2yQZiL/MzumnGU7swSLAypYjOCRyLB2BJY2enqLUjjqItBTViLUUpXqkCok4P29bJV6wu19+fUViDOrUiNxHlI0mBnll+hl2z+wgSYLuQB9Htp3M8tAh6umvAhVnStMzXz1hhZhMjrJr5jlenX6OGRMjarVzZPQNrG5ZR4sVafT0FKV+1FCkpaiHWPOj3LBiLb1gftRFjKXwEGUJE2fPzAvsmn6WkeQ+96bycA4OH0NncGn95rKIUHGmzAvmiyesEEmTYO/MS7w68xxDid2AsCx0EAeFj2ZpcDWid5nKYqIOIi3FXIq1UqjGC5ZLXYVYCg9BZoxhJLmPV6efY/fMCySJ02Z1c1T0jaxqWUvI0mKytUTFmaLUEWMMB5J7eXV6O3tnXiBBnKjVwRGRk1gdPpKI1dboKSpKY8kUAnUQauAvaBot2goxJyIsE5+w5ZQ9zu7pHfTPbGfCHiFAkL6Ww1gTXkd3oE/zYeuEijNFqQPjyWF2zzzP7pkdTNnjaYO2qmUtS4Ir1aApihdzINQyKVUA1VLEzbnoKkSB5P698RfZPfO86+WH7mAfx0ZOYEXLYZrgPweoOFOUGjGZHGXPzAvsjj/PeHIYEHqDqzgichLLWw4lKKGix5iPiMj3gHcAA8aY49yxJcBPgEOBl4D3GGOG3dc+DVwGJIGPGWPucMdPAm4AosBm4K+MMUZEwsCNwEnAIHCxMealObo8pVHMsVArRFMJqmrxEWQJM8PAzE72xF9gf3wXBptWq5PDI69nVcsRtAY653iic0cz2jAVZ4pSBePJYQZmXmZP/EXGkoMAdAeWc1T0VFa0HLZY6vrcAHwTx/ikuBq42xjzZRG52v39UyJyDHAJcCywCvi1iBxpjEkC38ZpGn4/jmE7B7gdxwgOG2OOEJFLgK8AF8/JlSnNQa6gaLBYm1cUKBIbt6fZF9/J3vhL7I/vwiZJWFo5OHwMK1sOpzOwdLF4+W+gyWyYijNFKYNUUuxA/GUGZl5iwh4BoCuwnHXRk+lreQ1Ra351IqgWY8w9InJozvD5wAb3+Q+ALcCn3PGbjDHTwIsisgM4WUReAjqNMfcBiMiNwAU4hu184HPusW4BvikiYoxZHI3FlXxUrHlTQvusKXucffGdDMy8zFBiNwabsLSyJnwUK1oOW5R5ZM1ow1ScKUoREmaGwfir7Iu/wr74K8yYKQShJ7iSg8LH0Ndy6EJP7F8qItsyfr/OGHNdkX36jDG7Adym4svd8dU4d5Updrljcfd57nhqn1fcYyVEZAToBfZXcjHKAsRLlCx0wVZiH1NjbEaS+137tTPt4W+1ujgkfCx9LYfRFVi2kAVZJfYLGmzDVJwpSg7GGMaSQ+xP7GJ/fBcHEnswGILSwtLgGpa1HMyy4EHzeum4NWOXk0ez3xhTq95XXt8ApsB4oX0UxZ9C4mW+CLcKG8lP25MMxl91bdirxE0MELqDy1kbfQPLQ4fQZnXNa0FWhg2rpf2CObJhKs4UBcfVPxTvZzDxKoPxV5kxMQDaA0s4JHw8y0IH0R3s04rXpbNXRFa6d5wrgQF3fBdwUMZ2a4B+d3yNx3jmPrtEJAh0AUP1nLyywClV9NRLxFUouvxImDjDiT0MxfvZn3iV8aTz8WiRCEtDq1kWOpje4GotcF0eDbVhKs6URcmUPc5wfDdDiT0MJfqZsp07sBaJ0BtaTW9wNb2h1Qs9XFlPNgGXAl92f/48Y/y/RORrOMm0a4EHjTFJERkTkTcCDwAfAL6Rc6z7gIuA38yHfDOjzr35T41FVK1ImDgHEnvTgmwkuQ+DQbDoDvaxNvoGeoOr6Qz0zmvvWINpqA1ranE2mhzkkfE76AmuoDvYR2dgKQFp6ikrTYgxhnF72DVmjkGL2U4j9aC00BNcwSHhY1kSWkW71aPGrExE5Mc4ibNLRWQX8A84Bu1mEbkM2Am8G8AY85SI3Aw8DSSAK9xVTgAfYXYZ+u3uA+C7wA/dxNshnJVSTc9YcpD7R3+etl/dwT7CVrTR01LmIdP2JMOJvWlBNpYcdMWY0BlYyqGRE1gSXElPcIV+R1ZAM9qwpv4rhqSFieQI++KvACBYdAZ66Q720RVcRldgGVGrQ79MlSym7UlGEvsZSe5jJLGXkcQ+EjhNh1skSk9wBYeGj6MnuJKOwBL991Mlxpj3+rx0ls/2XwC+4DG+DTjOYzyGaxjnEy0SRcTi5emneGn6CQCiVifdwWV0B/roDC6lM9CLJYEGz1RpJpImwVhykJHEPg4kBhhJ7kt79i0CdAWXcVjkBHqCK+kO9i3Y+olzSTPasKYWZ1Grg1M7L2DanuKAe9cwktzHK9PP8PL0kwCEJEJnYCldwaV0BhxjF7Ha9Qt3kTBtTzKWHGLUFWOjycG0VwyEjsASVrYcTlewj55gn4p5Zc6IWG2c0vFObJNkJLk/bcMG4/3snnkecG44OwK9dAV66QwuozPQS3ugRwXbIsERYkOuGNvPaHI/48mhdEg8LK10B/s4OHw03cEVKuYXEU0tzlKErSh9LYfS13IoALaxGU8OuZ6RfYwm9/Ni7LH0P+ighOkILKEjsCRt7NoC3XqHMY+xTZIJe4Tx5DCjiUHGk0OMJYeYNpPpbVqtTroDy+kKH0NXYDkdwV79mysNx5IAPe7NAThh9piZYCSxj5HEAKPJ/eyOv8ArM88CjmBrD3S7NqyXjsAS2gM9ridObyzmI6m/+XhymLHkkGO/EkNM2AfS31shCdMZ6OXQyAl0BZbRFVymOa+LmHkhznKxxHJCAsGlHBQ+Gsi+AxlN7GcsOcSr08+xk0R6v6jV4Qg1q5v2QDdtgW7arK55XRJhoZEwcSaTI2khNpF0fk7aI2kjJli0BbpZElpFZ6CXzkAvHYFe/Tsq8wIRISrtRFvaWdFyGOB8eU/ao2n7NZocYjDeT//MjvR+IQnn269ANxFpU9HWJNjGJmaPM2EfYDx5gAn3MZ4cTqdWAESsdjoCPSxvOcS1X0uJasRHyaAqcSYi78apens0cLIbb/Xa7hzg34AAcL0x5svVnNeLgATpDi6nO7gc3O/oWYM3xETygHvHMpzuG5YiJBHarC5aA520Wp1EAx20Wp20Wh2EJKIfmBpijCFhZpiyx5i0R5m0x5hKjjJhjzKZHMnyhIHQanXQFuimr+UQ2gI9dFiOF1Rd+0otaBYbJiK0BbpoC3SxouU16fEZeyptt8aTBxhPDrEn/iKJmen0NhYB13Z1OXbLtV9Rq5OI1ablX2qMbZJM2eOO/UqOztqy5AiT9ljWd0uLRGgLdLMyfATtgR7arR46Akv0RlIpSrWesyeB/wf4D78NRCQAfAs4G6fWx0MisskY83SV5y5KpsHLxDY2U/aYc1djj6Q9NYPxV+k327O2DRAkYrURtTqIWu1ErHbCVisRq42w1UbEaiUoLfW+lHlDwsSZtieJ2RNM2xPEjPM8Zo8xlRwnZo9n3UGCY8BarU56Q6tptTpdj0AXbVaXijCl3jS1DWuxovRaTlmXFMYYZsxUhv0aZcI+wERymH3xnVniAISItBINpOyXa7ekzX3equHSDGxjM2OmZu2XPUnMHidmTzBlO/Yr+wbSFceu3VoeOoRW9zunzerWumJKxVQlzowxzwDFPtgnAzuMMS+4296E02eq7obND0ssT9EGTnh0yh5L3xFN2ePuh3KMkfg+4mY6b58AQVqsVsISocWKEpYoLVaUkERosSKEJJz1CErLvDCGKS9X3MwQN9PETcz5aU8zY2LMmClm7Cmm3Z8z9lSe8AKnXEXEaica6GBJaCVRq90Vu85dvopbpVHMRxsmIoSllbDVyhJWZb1mjE3MTGbYrzFirg0bSuxh2p7Iq78mCC0SJWxFaXFtl/N7hJC49svKtmHz4abJGINN0rVdjt1K2bEZM03cjrm2a5JpE3NsmJnKO45gOfbLamdpaE36eSrSouJWqQdzkXOW7inlsgs4ZQ7OWxEBCTru50CP5+tJk3DvqlzvkJlg2p5i2p5kxsSYTI4ybPZ4irhMgoQISgsBCRGUEAEJEZAgAYLOTwliEUj/FLGwsBCxECwk4z8HIdUNImV8bWwMNsY4P21jY5MkaZLYJEiaJEmTIEkc2yRImARJEydBnISZIWFmCl+DhF1B2kpnoJeWkPOFERbHsxixWglbbZqUr8x35o0NE7GcfDar3fN1Y2ymzVTafjk2bNK5yXLFyXh8mGkzleOBy8YiQFBaCErKjgUJEErbrlz7ZYmFEMiwYSnbJXl9bWatmEnbLYPt/GeS2CSxXdtl49gt2yRIECdpEiRMnKR7U1noGgIECVkRwhIlarXTHVhGi9Xq2C1pTUdJWjS1RWkARcWZiPwaWOHx0t8aY37uMZ53CI8x38q4InI5cDk4SZPNRkCCvl63TGxj53mbUndwKW+UI4DiJN1H3MQcsWQSJEmkDVGtcYxlIC0ELQkSJEhQQoStaJbRzfX6zd5BRzSXRZkXzKUNa3b7JWKlQ5pdLPPdzhhDgjhxOzbrecqwY4kM+5UwcZLEmTFTJO04SVzhZBIkSVLrNqiCYBEkICnxF0zf3Ebcm8HUzW/QChPytGMRvWlUmpqi4swY85Yqz+HXh8rvfNcB1wF0BZfN2/4nlliEJUqY6iqCG+PePZJM30EajOsNM54tYtI+tfQdqoUlluOBw9K7QGVRMZc2bKHYLxEhRAuhQPUpBymPvTH2rDcfk7ZteedOedUkbb1cGzbreVOUhc5chDUfAtaKyGHAqzhtC/5kDs67IBARNxwQ8L5/VxSl3qgNqwLLTclQ+6UopVPVLYiIXOj2oToVuE1E7nDHV4nIZgBjTAL4KHAH8AxwszHmqeqmrSiKUj1qwxRFaUaqXa15K3Crx3g/sDHj983A5mrOpSiKUmvUhimK0oxo8F5RFEVRFKWJUHGmKIqiKIrSRKg4UxRFURRFaSLmZeNzRVGqZCYOO3c3ehaKoiiVscBtmHrOFEVRFEVRmggVZ4qiKIqiKE2EijNFURRFUZQmQsWZoiiKoihKE6HiTFEURVEUpYlQcaYoiqIoitJEqDhTFEVRFEVpIlScKYqiKIqiNBEqzhRFURRFUZoIFWeKoiiKoihNhIozRVEURVGUJkLFmaIoiqIoShOh4kxRFEVRFKWJUHGmKIqiKIrSRKg4UxRFURRFaSKqEmci8m4ReUpEbBFZX2C7l0TkCRF5VES2VXNORVGaDxE5R0SeE5EdInJ1o+dTKmrDFEVpRvsVrHL/J4H/B/iPErY90xizv8rzKYrSZIhIAPgWcDawC3hIRDYZY55u7MxKQm2YoiximtV+VSXOjDHPAIhIbWajKMp85GRghzHmBQARuQk4H2h6caY2TFEWPU1pv6r1nJWKAe4UEQP8hzHmOr8NReRy4HL31/E7hq9/rs5zWwoslLvhhXItC+U6YG6u5ZBydxhN7r/jjuHrl5a4eSQnlHddzmd4NfBKxu+7gFPKnVOTU5INU/tVFXotzcdcXUc9bdi8tF9FxZmI/BpY4fHS3xpjfl7ieU4zxvSLyHLgLhF51hhzj9eG7pvmK95qjYhsM8b45prMJxbKtSyU64DmvRZjzDk1PJyX28nU8PhVMZc2TO1X5ei1NB/NfB01tGFNab+KijNjzFuqPYkxpt/9OSAit+K4ET3FmaIo845dwEEZv68B+hs0lzzUhimKUoCmtF91L6UhIm0i0pF6DrwVJwlXUZSFwUPAWhE5TERagEuATQ2eU81QG6YoC5qmtF/VltK4UER2AacCt4nIHe74KhHZ7G7WB9wrIo8BDwK3GWN+Vc15a8ychSDmgIVyLQvlOmBhXYsnxpgE8FHgDuAZ4GZjzFONnVVpLAAbtpD+fem1NB8L5Tp8aVb7JcY0PLSqKIqiKIqiuGiHAEVRFEVRlCZCxZmiKIqiKEoToeKM0lu4NCvN2HqiEkTkeyIyICLzPtlaRA4Skd+KyDPuv62/avSclIWJ2q/mYaHYMLVfjUfFmUOqhcu8Wxqf0XriXOAY4L0ickxjZ1UxNwC1rL/VSBLA3xhjjgbeCFwxj/8uSnOj9qt5uIGFYcPUfjUYFWc4LVyMMfWu5F0v0q0njDEzQKr1xLzDLeo51Oh51AJjzG5jzCPu8zGcVUCrGzsrZSGi9qt5WCg2TO1X41FxNv/xaj2hH6ImQkQOBV4HPNDgqShKs6H2q8lR+9UY5qq3ZsOpUQuXZqQpW08oDiLSDvwUuNIYM9ro+SjzE7VfSiNQ+9U4Fo04q0ULlyalKVtPKCAiIRzD9iNjzP9t9HyU+YvaL2WuUfvVWDSsOf9pytYTix0REeC7wDPGmK81ej6K0qSo/WpC1H41HhVn+LdwmQ80a+uJShCRHwP3AetEZJeIXNboOVXBacD7gT8WkUfdx8ZGT0pZeKj9ah4WkA1T+9VgtH2ToiiKoihKE6GeM0VRFEVRlCZCxZmiKIqiKEoToeJMURRFURSliVBxpiiKoiiK0kSoOFMURVEURWkiVJwpiqIoiqI0ESrOFEVRFEVRmoj/H4JXfMfZDQqXAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<Figure size 720x288 with 4 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"p_c0_given_x, p_c1_given_x = compute_posterior(flat_plt_grid, p_c0, mu_c0, sigma_c0, p_c1, mu_c1, sigma_c1)\n",
|
|
"p_c0_given_x = np.reshape(p_c0_given_x, plt_grid_shape)\n",
|
|
"p_c1_given_x = np.reshape(p_c1_given_x, plt_grid_shape)\n",
|
|
"\n",
|
|
"plt.figure(figsize=(10, 4))\n",
|
|
"plt.subplot(1, 2, 1)\n",
|
|
"plt.contourf(plt_grid[..., 0], plt_grid[..., 1], p_c0_given_x, levels=10)\n",
|
|
"plt.colorbar()\n",
|
|
"# plot decision boundary \n",
|
|
"plt.contour(plt_grid[..., 0], plt_grid[..., 1], p_c0_given_x, levels=[0.0, 0.5], colors=[\"k\", \"k\"])\n",
|
|
"\n",
|
|
"plt.title(\"p($c_0$ | x)\")\n",
|
|
"s0 = plt.scatter(c0_samples[..., 0], c0_samples[..., 1], color=\"blue\")\n",
|
|
"s1 = plt.scatter(c1_samples[..., 0], c1_samples[..., 1], color=\"orange\")\n",
|
|
"plt.legend([s0, s1], [\"c0\", \"c1\"])\n",
|
|
"plt.xlim(-1.5, 2.5)\n",
|
|
"\n",
|
|
"plt.subplot(1, 2, 2)\n",
|
|
"plt.contourf(plt_grid[..., 0], plt_grid[..., 1], p_c1_given_x, levels=10)\n",
|
|
"plt.colorbar()\n",
|
|
"# plot decision boundary \n",
|
|
"plt.contour(plt_grid[..., 0], plt_grid[..., 1], p_c0_given_x, levels=[0.0, 0.5], colors=[\"k\", \"k\"])\n",
|
|
"plt.title(\"p($c_1$ | x)\")\n",
|
|
"s0 = plt.scatter(c0_samples[..., 0], c0_samples[..., 1], color=\"blue\")\n",
|
|
"s1 = plt.scatter(c1_samples[..., 0], c1_samples[..., 1], color=\"orange\")\n",
|
|
"plt.legend([s0, s1], [\"c0\", \"c1\"])\n",
|
|
"\n",
|
|
"plt.xlim(-1.5, 2.5)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The color indicates the posterior likelihood for the respective call and the black line indicates the decision boundary. \n",
|
|
"We achieve a train accuracy of 87%.\n",
|
|
"For such a simple task that is clearly not great, but it nicely illustrates a\n",
|
|
"problem with generative approaches:\n",
|
|
"They usually depend on quite a lot of assumptions.\n",
|
|
"\n",
|
|
"### 2.3) Wrong Assumptions? (1 Point):\n",
|
|
"Which untrue assumption did we make?"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"data is independent and identically distributed (i.i.d.)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 3) Stochastic and Batch Gradients\n",
|
|
"\n",
|
|
"In the recap sessions with Prof. Neumann we already saw (or will see) an implementation of a Discriminative Classifier using Logistic Regression. Here we are going to extend this to stochastic and batch gradient descent. \n",
|
|
"\n",
|
|
"We start by implementing a few helper functions for affine mappings, the sigmoid function, and the negative Bernoulli log-likelihood. - Those are the same as used for the full gradient case."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def affine_features(x: np.ndarray) -> np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" implements affine feature function\n",
|
|
" :param x: inputs, shape: [N x sample_dim]\n",
|
|
" :return inputs with additional bias dimension, shape: [N x feature_dim]\n",
|
|
" \"\"\"\n",
|
|
" return np.concatenate([x, np.ones((x.shape[0], 1))], axis=-1)\n",
|
|
"\n",
|
|
"def quad_features(x: np.ndarray) -> np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" implements quadratic feature function\n",
|
|
" :param x: inputs, shape: [N x sample_dim]\n",
|
|
" :return squared features of x, shape: [N x feature_dim]\n",
|
|
" \"\"\"\n",
|
|
" sq = np.stack([x[:, 0] ** 2, x[:, 1]**2, x[:, 0] * x[:, 1]], axis=-1)\n",
|
|
" return np.concatenate([sq, affine_features(x)], axis=-1)\n",
|
|
"\n",
|
|
"def cubic_features(x: np.ndarray) -> np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" implements cubic feature function\n",
|
|
" :param x: inputs, shape: [N x sample_dim]\n",
|
|
" :return cubic features of x, shape: [N x feature_dim]\n",
|
|
" \"\"\"\n",
|
|
" cubic = np.stack([x[:, 0]**3, x[:, 0]**2 * x[:, 1], x[:, 0] * x[:, 1]**2, x[:, 1]**3], axis=-1)\n",
|
|
" return np.concatenate([cubic, quad_features(x)], axis=-1)\n",
|
|
"\n",
|
|
"def sigmoid(x: np.ndarray) -> np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" the sigmoid function\n",
|
|
" :param x: inputs \n",
|
|
" :return sigma(x)\n",
|
|
" \"\"\"\n",
|
|
" return 1 / (1 + np.exp(-x))\n",
|
|
"\n",
|
|
"def bernoulli_nll(predictions: np.ndarray, labels: np.ndarray, epsilon: float = 1e-12) -> np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" :param predictions: output of the classifier, shape: [N]\n",
|
|
" :param labels: true labels of the samples, shape: [N]\n",
|
|
" :param epsilon: small offset to avoid numerical instabilities (i.e log(0))\n",
|
|
" :return negative log-likelihood of the labels given the predictions\n",
|
|
" \"\"\"\n",
|
|
" return - (labels * np.log(predictions + epsilon) + (1 - labels) * np.log(1 - predictions + epsilon))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We are also using the same bernoulli objective and its gradient as before"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def objective_bern(weights: np.ndarray, features: np.ndarray, labels: np.ndarray) -> float:\n",
|
|
" \"\"\"\n",
|
|
" bernoulli log-likelihood objective \n",
|
|
" :param weights: current weights to evaluate, shape: [feature_dim]\n",
|
|
" :param features: train samples, shape: [N x feature_dim]\n",
|
|
" :param labels: class labels corresponding to train samples, shape: [N]\n",
|
|
" :return average negative log-likelihood \n",
|
|
" \"\"\"\n",
|
|
" predictions = sigmoid(features @ weights)\n",
|
|
" return np.mean(bernoulli_nll(predictions, labels))\n",
|
|
"\n",
|
|
"def d_objective_bern(weights: np.ndarray, features: np.ndarray, labels: np.ndarray) -> np.ndarray:\n",
|
|
" \"\"\"\n",
|
|
" gradient of the bernoulli log-likelihood objective\n",
|
|
" :param weights: current weights to evaluate, shape: [feature_dim]\n",
|
|
" :param features: train samples, shape: [N x feature_dim]\n",
|
|
" :param labels: class labels corresponding to train samples, shape [N]\n",
|
|
" \"\"\"\n",
|
|
" res = np.expand_dims(sigmoid(features @ weights) - labels, -1)\n",
|
|
" grad = features.T @ res / res.shape[0]\n",
|
|
" return np.squeeze(grad)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 3.1) Implementation (3 Points)\n",
|
|
"\n",
|
|
"Finally, we can implement our batch gradient descent optimizer. When setting the batch_size to 1 it will become a stochastic gradient descent optimizer.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def minimize_with_sgd(features: np.ndarray, labels: np.ndarray, initial_weights: np.ndarray, schedule: Callable,\n",
|
|
" num_iterations: int, batch_size: int):\n",
|
|
" \"\"\"\n",
|
|
" :param features: all samples, shape: [N x feature_dim] \n",
|
|
" :param labels: all labels, shape: [N]\n",
|
|
" :param initial_weights: initial weights of the classifier, shape: [feature_dim * K]\n",
|
|
" :param schedule: learning rate schedule (a callable function returning the learning rate, given the iteration\n",
|
|
" :param num_iterations: number of times to loop over the whole dataset\n",
|
|
" :param batch_size: size of each batch, should be between 1 and size of data\n",
|
|
" return \"argmin\", \"min\", logging info\n",
|
|
" \"\"\"\n",
|
|
"\n",
|
|
" assert 1 <= batch_size <= features.shape[0]\n",
|
|
" # This is a somewhat simplifying assumption but for the exercise its ok\n",
|
|
" assert features.shape[0] % batch_size == 0, \"Batch Size does not evenly divide number of samples\"\n",
|
|
" batches_per_iter = int(features.shape[0] / batch_size)\n",
|
|
"\n",
|
|
" # setup\n",
|
|
" weights = np.zeros([batches_per_iter * num_iterations + 1, initial_weights.shape[0]])\n",
|
|
" loss = np.zeros(batches_per_iter * num_iterations + 1)\n",
|
|
" weights[0] = initial_weights\n",
|
|
" loss[0]= objective_bern(weights[0], features, labels)\n",
|
|
"\n",
|
|
" for i in range(num_iterations):\n",
|
|
" #--------------------------------------------------\n",
|
|
" # shuffle data\n",
|
|
" idx = np.random.permutation(len(labels))\n",
|
|
" features, labels = features[idx,:], labels[idx]\n",
|
|
" #--------------------------------------------------\n",
|
|
" for j in range(batches_per_iter):\n",
|
|
" global_idx = i * batches_per_iter + j\n",
|
|
" start = j * batch_size\n",
|
|
" stop = start + batch_size\n",
|
|
" batch_features = features[start:stop,:]\n",
|
|
" batch_labels = labels[start:stop]\n",
|
|
" gradient = d_objective_bern(weights[global_idx+1],batch_features, batch_labels)\n",
|
|
" diff = -schedule(i)*gradient\n",
|
|
" weights += diff\n",
|
|
"\n",
|
|
" # log loss (on all samples, usually you should not use all samples to evaluate after each stochastic\n",
|
|
" # update step)\n",
|
|
" loss[global_idx + 1] = objective_bern(weights[global_idx + 1], features, labels)\n",
|
|
" return weights[-1], loss[-1], (weights, loss)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The loss curve is expected to look a bit jerky due to the stochastic nature of stochastic gradient descent.\n",
|
|
"If it goes down asymptotically its fine."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 17,
|
|
"metadata": {
|
|
"pycharm": {
|
|
"name": "#%%\n"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Final loss 0.04803303656958995\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<matplotlib.legend.Legend at 0x7f0dc01e44f0>"
|
|
]
|
|
},
|
|
"execution_count": 17,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA43UlEQVR4nO3dd3zU9f3A8df7MiFAIOy9hyhTQEEUcIKWOuuu4+dsXa1axbq1ravaqtVaHK0L3AtRHCAismTvvQkzEEISst+/P77fXC7ruIRc7nL3fj4e98jd977j/bnAvfP9TFFVjDHGmMp4Qh2AMcaY8GaJwhhjjF+WKIwxxvhlicIYY4xfliiMMcb4ZYnCGGOMX5YojDHG+GWJwoQdEblcROaLSKaI7BSRr0VkeAjj2Swih914ih//CvDY6SJyfbBjDISIXCMiM0Mdh6l7YkMdgDG+ROROYBxwM/ANkAeMBs4Fyn3JiUisqhbUQmhjVfX7mj5pLcZvTLXZHYUJGyKSDDwG3KKqn6hqlqrmq+okVf2Tu88jIvKRiLwjIhnANSLSRkS+EJH9IrJeRG7wOecQ9+4kQ0R2i8hz7vZE9xxpIpIuIr+ISMtqxHyNiMwUkb+LyAER2SQiY9z3/gqcDPzL9y5ERFREbhGRdcA6d9sNbuz73bK08bmGisjtIrJRRPaJyDMi4hGRBHf/Pj77tnDvfppXsRzD3M/goPtzWJkybhSRQ275rnC3dxORH91j9onI+1X9/Ewdoar2sEdYPHDuHAqAWD/7PALkA+fh/KFTD/gReBlIBPoDe4HT3P1nA791nzcATnSf3wRMAuoDMcDxQKNKrrkZOL2S965x47nBPc/vgFRA3PenA9eXOUaB74AUN/5TgX3AQCABeBGYUWb/H9z9OwBri8/plvspn33vACb5iXVmBdtTgAPAb3FqGS5zXzcFkoAMoKe7b2vgWPf5ROB+9/eQCAwP9b8hewTnYXcUJpw0BfbpkatiZqvqZ6paBDQDhgP3qmqOqi4GXsP50gPnS7ybiDRT1UxVneOzvSnQTVULVXWBqmb4ueZn7p1H8eMGn/e2qOqrqloIvInzZXqku5MnVHW/qh4GrgDeUNWFqpoL3AcMFZFOPvs/5e6/Ffgnzpc57vUuF5Hi/8u/Bd4+wrXLOgdYp6pvq2qBqk4EVgNj3feLgONEpJ6q7lTVFe72fKAj0Mb97K39I0JZojDhJA1oJiJHajvb5vO8DbBfVQ/5bNsCtHWfXwf0AFa7VSq/cre/jdMG8p6IpIrI0yIS5+ea56lqY5/Hqz7v7Sp+oqrZ7tMGVSzDFp9zZOJ8Fm0r2X+LewyqOhfIAkaISC+gG/DFEa5dVqnr+1yjrapmAZfgtBntFJHJ7nUA7gEEmCciK0Tk/6p4XVNHWKIw4WQ2kINTreSP75THqUCKiDT02dYB2AGgqutU9TKgBfAU8JGIJKnT9vGoqvYGhgG/Aq6qmWJUGmtl21Nx/jIHQESScO52dvjs097neQf3mGJvAlfi3E18pKo5VYyx1PV9rlH8GX6jqmfg3CmtBl51t+9S1RtUtQ1OVd7LItKtitc2dYAlChM2VPUg8BDwkoicJyL1RSRORMaIyNOVHLMNmAU84TZQ98W5i3gXQESuFJHmbjVVuntYoYiMEpE+IhKDUwefDxQGoVi7gS5H2GcCcK2I9BeRBOBvwFxV3eyzz59EpImItMdph/BtOH4bOB8nWbx1hGuJ+zl5H8BXQA9xuiXHisglQG/gSxFpKSK/dpNXLpCJ+zmJyG9EpJ173gM4yS8Yn6EJMUsUJqyo6nPAncADOI3S24Bbgc/8HHYZ0AnnL+NPgYdV9Tv3vdHAChHJBJ4HLnX/4m4FfISTJFbhNIi/4+cak6T0OIpPAyzS88BFbo+oFyraQVWnAg8CHwM7ga7ApWV2+xxYACwGJgOv+xy/HViI80X90xHiGQYcLvM4iHNHdRdOldc9wK9UdR/Od8RdOJ/tfmAE8Hv3XIOBue5n+wVwh6puOsL1TR1U3DPDGBOmRESB7qq63s8+bwCpqvpA7UVmooUNuDOmjnN7R10ADAhxKCZCWdWTMXWYiDwOLAeesWofEyxW9WSMMcYvu6MwxhjjV0S2UTRr1kw7depUrWOzsrJISkqq2YDCnJU5OkRbmaOtvHB0ZV6wYME+Va1wjrCITBSdOnVi/vz51Tp2+vTpjBw5smYDCnNW5ugQbWWOtvLC0ZVZRMqOzveyqidjjDF+WaIwxhjjlyUKY4wxflmiMMYY45clCmOMMX5ZojDGGOOXJQpjjDF+hX2iEJEkEXlTRF4tXtQ9mFalFbJ+T2awL2OMMXVGSBKFiLwhIntEZHmZ7aNFZI2IrBeRce7mC3BW7boB+HWwY3vqlxxOf+7HYF/GGGPqjFDdUfwPZ0EZL3elsZeAMTira10mIr2BdpSsF2yrZxljTC0LyRQeqjrDnUPf1xBgvapuBBCR94Bzge04yWIxfhKbiNwI3AjQsmVLpk+fflQxHu3xdUlmZmZUlReszNEg2soLwStzOM311JaSOwdwEsQJwAvAv0TkHGBSZQer6nhgPMCgQYO02nO8TJkMEFVzxNicONEh2socbeWF4JU5nBKFVLBNVTULuLa2gzHGGOMIp15P24H2Pq/b4SzobowxJoTCKVH8AnQXkc4iEg9cCnwR4piMMSbqhap77ERgNtBTRLaLyHWqWgDcCnwDrAI+UNUVoYjPGGNMiVD1erqsku1fAV/VcjjGGGP8CKeqp6MmImNFZPzBgwdDHYoxxkSMiEoUqjpJVW9MTk6u1vGrdmbUcETGGFP3RVSiOFpjnv8p1CEYY0zYsURhjDHGL0sUxhhj/LJEYYwxxi9LFMYYY/yKqERh3WONMabmRVSiONruscYYY8qrdGS2iFzg70BV/aTmwzHGGBNu/E3hMdb92QIYBkxzX48CpgOWKIwxJgpUmihU9VoAEfkS6K2qO93XrXGWLDXGGBMFAmmj6FScJFy7gR5BiscYY0yYCWT22Oki8g0wEVCcdSJ+CGpUxhhjwsYRE4Wq3ioi5wOnuJvGq+qnwQ0rNDwCRRrqKIwxJrwEuh7FLKAA545iXvDCCS1LEsYYU94R2yhE5GKc5HARcDEwV0QuCnZg1WED7owxpuYF0ph9PzBYVa9W1auAIcCDwQ2remzAnTHG1LxAEoVHVff4vE4L8DhjjDERIJA2iik+vZ4ALiHC17Ue2qVpqEMwxpiwEUivpz+503kMB4QI7vVULCkhJtQhGGNM2Ai019PPQD4R3uupmFrvJ2OM8YqoXk9Ha9GDZ5CSKKEOwxhjwkogdxTFvZ72AIhIc+B74KNgBhYKTZLiaRgv2A2FMcaUsF5PZdj9hDHGlGa9niqg1khhjDFegfZ6uhA4iTDv9SQiY4Gx3bp1q/45ai4cY4yJCAH1elLVj4GPgxzLUVPVScCkQYMG3XBU56mheIwxJhIE0uvpAhFZJyIHRSRDRA6JSEZtBBcSdkthjDGlBHJH8TQwVlVXBTuYcGFNFMYYUyKQ3ku7oylJ2A2FMcaUVukdhTttB8B8EXkf+AzILX5fVT8JbmihYzcUxhhTwl/V01if59nAmT6vFYjcRGF1T8YY41VpolDVa2szkHBhVU/GGFOav6qne1T1aRF5kQpqY1T19qBGZowxJiz4q3oqbsCeXxuBhAuxWwpjjCnFX9XTJPfnm7UXTniwJgpjjCnhr+ppEn46AKnqr4MSkTHGmLDir+rp77UWRQ2pibmeANQ6yBpjjJe/qqcfi5+LSD2gg6quqZWoqqkm5nqyJgpjjCktkLmexgKLgSnu6/4i8kWQ4wopa6MwxpgSgUzh8QgwBEgHUNXFQKdgBRRqIpYojDHGVyCJokBVDwY9EmOMMWEpkNljl4vI5UCMiHQHbgdmBTes0LLGbGOMKRHIHcVtwLE4EwJOADKAO4IZVChZY7YxxpQWSKK4TFXvV9XB7uN+4NFgBxZK1kZhjDElAql6ukhEclT1XQAReQlIDG5YxhhjwkUgieIC4AsRKQLGAPtV9ZbghhU6HoEiu6Uwxhgvf1N4pPi8vB5n4aKfgcdEJEVV9wc5tpCI8Qh5hZYojDGmmL87igU4cz2Jz89z3IcCXYIeXQjEeSCnoCjUYRhjTNjwN4VH59oMJFykZhaxOzsj1GEYY0zY8Ff1dKqqTvNZO7uUSF0ze3e2U+2UW1BIQmxMiKMxxpjQ81f1NAKYRum1s4uF5ZrZNTF77EltYvk5tYA9Gbm0T6lfc8EZY0wd5a/q6WH3Z7m1s0XkwmAGVV01MXtsrxQPP6fCqp0ZliiMMYbABtxV5B81GkUY6dDI+UhufHtBiCMxxpjwUN1EEbEzXSTGRGzRjDGmWqqbKCJ2oEFibEmi2J2RE8JIjDEmPPjr9bSMihOCAC2DFlGI1fP5RDJzCyK3oMYYEyB/vZ5+VWtRhJH4GKFZgwT2ZeayZtchujZvEOqQjDEmpCqtelLVLWUfQB+f5xHr3etPAOD37y6s9jmy8wpQmzPKGBMBqtpG8VhQoggzPVs19D5PTT9c5ePX7DpE74e+YdLSnTUZljHGhERVE0XUdQka9uS0Kh+zdX82AF8s3lHT4RhjTK2raqK4KShRhKFXrhxY7WM9bjotsponY0wEOOJ6FGXnehKRdsBBYJmq7glWYKF2NI3YHnEyha1rYYyJBIEsXHQdMBT4wX09EpgD9BCRx1T17SDFFlIdmyZV+1ixOwpjTAQJpOqpCDhGVS9U1QuB3kAucAJwbzCDC6X4WA/tU+oBUFTFb3xxM4X1ejLGRIJAEkUnVd3t83oP0MNd4S4/OGGFh0sHdwDgp/X7qnRccYu/5QljTCQIJFH8JCJfisjVInI18AUwQ0SSgPSgRhdi5w1oC8CkJalVOq64jUIjd6YTY0wUCSRR3AL8F+gPDADeBG5R1SxVHRXE2EKudaNEAD5asL1Kx3nbKGxFVWNMBDhiY7aqqojMBPJw5n6ap1FS+e7xCJ2bJbFpXxaH8wqpFx/YinfFVU/W68kYEwmOeEchIhcD84CLgIuBuSJyUbADCxe/H9kVgD++v5jlOw4GdExxepi7aT87qjGy2xhjwkkgVU/3A4NV9WpVvQoYAjwY3LCqR0TGisj4gwcD+0IPRKtkp/ppyopd/OrFmQH1ZPLd5f1fttVYLMYYEwqBJApPmYF1aQEeV+tUdZKq3picnFxj5yw78O6cF2YeOQ6fRuyMwxHdMcwYEwUC+cKfIiLfiMg1InINMBn4OrhhhY/W7h1FsZU7M454jO8dxWeLd9h4CmNMnXbERKGqfwL+A/QF+gHjVfWeYAcWLkSEa4Z1ol+7ZFo0TADgp3V7/R5TnBbaNq5HenY+uQXW/ckYU3cFVIWkqp+o6p2q+kdV/VREtgY7sHDyyK+P5fNbh/Pcxf0B+O3r8/zuX3wHcUKXFMCqn4wxdVt12xqibrpxgOHdm9Ehpf4R9yuuaUqMc7rT1tYdharywtR1/LJ5f61czxgTHaqbKKK20n1Mn1bEx/r/2Iobs+Pc+cYLa2l2wG37D/Pcd2t5dNKKWrmeMSY6VDrgTkTurOwtIGoXkk6I8ZBXUMTdHy7hyQv6EBtTPmkU31EUv1dQS4ki3x0KvnFvVq1czxgTHfyNzG7o573nazqQuiLO/fL/aMF2CgqL+OelA8rtU5IoanddCutdZYwJhkoThao+WpuB1BVDuzaF75znny1OJS7Gw+BOKVw8uL13n+Kv6ziPe0dRWDtf4MU3LlHZgGSMCZqwHDgXzgZ1SmHxQ2d4X3+4YDv3fLyUl35Yz50fLAZK/rIvvqOorTaK4juX4vUwjDGmJliiqIZGiXHltj3zzRo+WbiDoqKScdnF1VTPT11XK3EVz1ZracIYU5MsUVSDxyNMvn14he/lFxV52yiS3Nlmv1+1u8J9a5q3LcQyhTGmBvnr9XSlqr5TWe8nVX0ueGGFv96tG9G4fhzp2aUH0xUUqrfqaUjnpnRulsThvMJaicnyhDEmGPzdUSS5PxtW8ohqIsKc+07j3tG9Sm0vKCypevJ44KRuTdmflceWtOB3WbU2CmNMMPjr9fQf96f1fqpEYlwMN57ShTW7MpizcT+7MnJKVT0JQl5BEXmFRYx4ZjpvXzeEk7s3D1o8JYkiaJcwxkQhf1VPL/g7UFVvr/lw6p4Yj/DPSwcwYe5W/vzpMt6ds5UuzZ2bMRE4t39bpq3ey77MXFbvPBTkROH8tDxhjKlJ/gbcLai1KCJAcj2nJ9Q/vl9Lv/aNAfAInNStGXPuO5Vu93/NzoM5QY1h636nesuqnowxNclf1dObtRlIXXdqrxbe50u2pQOQEOv0eiqeyuONnzdRpMojvz62xq+/JS2LP76/BKi9cRvGmOhQaWO2iEwSkS8qe9RmkHVBvfgYZt47imYNErzbEnwmD2zVyFkA6X+zNgfl+peOn+N9nplbEJRrGGOik7+qp7/XWhQRol2T+rxwWX8uf3UuAAnuNOMAc/58Gtf97xd2ZQSn+ulAdp73ud1RGGNqkr+qpx9rM5BIcULnpt7nDRJKf7xxMR5WpGaQnVdA/Xh/ObrqPNYuYYwJkiN+W4nIJipYf0JVuwQlojouxiNsfvKcCt9btO0AAM9/v477zj6mRq/rmyhO7t6sRs9tjIlugfxZO8jneSLwGyAlOOFEtt0ZuQBMX7O3xhOFb7tE2TsZY4w5Gkec60lV03weO1T1n8CpwQ8t8rx4mbN2RbcWzrpPqsquGugyu+dQ6XPYshTGmJp0xEQhIgN9HoNE5GZsCo9qGduvDf3bN+aQ+9f/Nyt2c+ITU7nytblk51W/p9KhnNLHavSuVGuMCYJA6iie9XleAGwGLg5KNEdJRMYCY7t16xbqUCrVICGWjMP5fL9yN7971xnTOHP9Pv7w3mJeuGwAiT49pQKVUGYNb7ujMMbUpCMmClUdVRuB1ARVnQRMGjRo0A2hjqUyM9fvA+D6t+aX2v7tyt08OmkFT1zQt8rnLNvjyXrHGmNqUiBVT8ki8pyIzHcfz4pIcm0EF4lO8xnBXdaK1IxqnbN8XrBMYYypOYEsXPQGcAinuuliIAP4bzCDimSvXe10IhvQoTFLHjqTq4d2pJ5b3dS3XWD59/fvLqDTuMn858cNABSVuYWwqidjTE0KpI2iq6pe6PP6URFZHKR4Ip5I6XEWj557HI+eexydxk0mLTPPz5Elvlq2C4Anvl7NBQPblXvf8oQxpiYFckdxWES8636KyEnA4eCFFL2+Xr4roP36u7PTAnwwfxurdx0q9b7aLYUxpgYFckdxM/CW2y4hwH7gmmAGFY2GdmnK7I1pZOUWkHSEAXO+aeCZb9b4fd8YY45WIAPulqhqP6Av0EdVB6jqkuCHFl3G9GkFwNVvzCu1fc7GtHJjLHLzK16D+5mL+tK9RQNrozDG1KhA5npKAC4EOgGxxYviqOpjQY0sypzYxZlMcP6WA95tszekcdmrc+jSLIlpd48EcFbK23WIhgmx3oF7xRLiYqgfH0NaVi6qWqMLGD38+XLW7s5k4o0n1tg5jTF1QyBVT58DB3FWvMsNbjjRq0fLhvRv35jF29J56PPlCPDm7C0AbNyXRWGREuMRb0+nXq0b8stmJ6kkxHrILShCgIIiZUWqs4b30K5NK7la1RXHYoyJPoE0ZrdT1UtU9WlVfbb4EfTIotDvR3YF4K3ZW8p9MT/4+XKgZLqOl64Y6H2vfUp9wBl4d8soZ1R62fmfakpBYVFQzmuMCV+BJIpZItIn6JEYzjy2FTPvHcWXtw3nr+cfV+q9zfuc9bBX7TrEsK5NadEwkacvdEZxF/dyEoHj2jhjMfILg9NQkX44PyjnNcaEr0ASxXBggYisEZGlIrJMRJYGO7Bo1a5JfY5rm8z5A9p6tzWpH0er5EQKCotYsi2dHi2dORkvHtyeto3rsWGvk0Q8ArExTrtEsP7y35KWHZTzGmPCl982CnFaQ28GrIK6ltWPj6VLsyQa149j58EcPlm4g1Pd6T9aNCpZl3tHesmQFhEhLsbJ/fkVJIqc/ELembOFs45t5a2uqqoJc7dyfMcm1TrWGFM3+b2jUKdO4x+quqXso5bii2rf3zmCD28eRouGTmK4dcIiAMb2bePd55y+rb3PBYhz7yj+PX1DufM99Ply/jJ5FRPnba12TIM7WZIwJtoEUvU0R0QGBz0SU47HI8R4hH9c0t+77aZTupS6G3jp8pJGbY+Id5ry1AoWREpNd7a9PH0DXyxJrVZMBTY1rTFRJ5DusaOAm0VkM5CF84erqmrV58M21dKleQM2PXF2QOMiEuNiOH9AWz5dtKPce4U+X/K3T1zEr/u1KbfPkRRaojAm6gSSKMYEPQpzRIEkiTy3XaJzsyTAadCOddss9mTklPqST4wL5GbS4Tt3VEVtH8aYyBbIFB5bgPbAqe7z7ECOM7XvrGOdaUBiPE5SOeh2ZV21M4Mhf5vKvM37vfue2btVwOfNLShJDnZHYUz0CWThooeBe4H73E1xwDvBDMpUTaemTptFcYJolOjcKD7wmTNIL9WnZ9SIHs3p2jyJwipMCJXncxdRdqZaY0zkC6Tq6XxgALAQQFVTRaRhUKMyVfLdnSMo8vniv+KEjjz4+Qq+Xr6Lc1/6mYSYkr8HYjxCrMdDYRUG5G3ZVzJ2YndGcEZ8G2PCVyBVSHluN1kFEJGk4IZkqiouxkNCbIz3tccjjDnOqVpasi29VJVTXkERMR6pUu+lCfNKekPX4DyDxpg6IpBE8YGI/AdoLCI3AN8DrwY3LHO0Hj/vOH43sit92pZeXnXepv3ExkipO5AjKSqCFg0TOOvYluzJsHkhjYk2gTRm/x34CPgY6Ak8pKovBjswc3SaNUjg3tG9eOW3x5fa3q99MjEeYdaGfQGfq0iVWI/QvGECezMtURgTbQLqvaSq36nqn4Ance4oTB3RtnE97/O7zujBezcOJb+wiJz8IooCrH4qdNe2aNEwkfTsfHILKl44yRgTmSpNFCJyoohMF5FPRGSAiCwHlgO7RWR07YVojtbCB8/g2d/045ZR3YjxCOf1dyYcvHT8HF6cuo70HP9jI1SdRvDm7lQiaZl5QY/ZGBM+/N1R/Av4GzARmAZcr6qtgFOAJ2ohNlNDUpLiufD4dnjc7rPFkwvO27yfZ79by3dbCio8rqhIKXQfHoHmDZxE8ZYtYmRMVPGXKGJV9VtV/RDYpapzAFR1de2EZoKlS/MG/HL/6dx1Rg8A9h4ufUfxly9Xcsl/ZtPlz19x/ss/U6SKxyMc27YRAFOW76z1mI0xoeMvUfh+exwu854Nz63jmjdM4LbTutOlWRLzdhXy3crd5BcWsScjh9dmbmLuJqdL7dLtB51EIULr5Ho0axDP4E4pIY7eGFOb/A246yciGTiTANZzn+O+Tgx6ZKZW/N/wzjzw2XJueGt+qe1tG9fzrnVRVAQx7gCKxLgYm8bDmChT6R2FqsaoaiNVbaiqse7z4tdxtRmkCZ4rT+zIH49PKLd9wg0ncKdbNZVfWOQdaBcX4yHfEoUxUSWQKTxMhOvXPJbJtw+mQ0p9xs/YSK9WjejYNMm7Wl6uO5obINYjTFqSyorUg3zyu2E0rh9frWu+PXszI3q0oEPT6q20Z4ypPZYoDADHtnFGcN91Zk/vtuLV8mau30esmyjW7ckEYOPeLD5ZuIP0w/neO49A5RYU8uDnK0hJWsfCB8+oifCNMUFk04WbSqUkldwtFM8NNbJnc++2x75cyQtT17F5X1ap47btz+aif89iU5ntxfLdCQn3Z9l4DGPqgoAShYh0FJHT3ef1bPbY6HD+gLaMG9MLgP9e46yG+79rh7D2L2Po376xd7/VuzL4fuVu7wJHM9fvY/6WA4z6+3SmrtpdbgR4foEtfmRMXRLIehQ34Mz19B93UzvgsyDGZMKEiHDziK5sfvIcRrmD9ADiYz18dPNQPrvlJABufmch1781n8nLnPEVCbEl/6yue3M+w5+aVuq8+UWWKIypSwK5o7gFOAnIAFDVdUALv0eYiBcb46F/+8b0aNnAu+3WCYsAOJxfei6o1IM5FPgsflTgsxZGerZVPxkT7gJJFLmq6v3fLCKx2IA74/rD6aUbsjuNm8z9nzor6311+8lcNqQ9UNLGkV9YVGquqJenb6ilSI0x1RVIovhRRP6MM+juDOBDYFJwwzJ1xRm9W9KvXTJXDe1Y7r3uLRvQtblzx5FXWMS63Yc45sEpjP3XTO8+42dsJCMnv9biNcZUXSCJYhywF1gG3AR8BTwQzKBM3REX4+HzW4fz2LnH8fIVA8u9VzwWY9HWdM74x4wKV9br+8i3NtrbmDAWyDiKc4G3VNVWtTN+jTmuFbPvO5WJc7eSmeu0UxQngBvedKYIaZgYy6Gc8rPV/m/WZk7onMIxrRt5B/cZY8JDIIni18A/RWQG8B7wjapWPC91EIhIF+B+IFlVL6qt65qqE3fiwDt9Bu3Fuz2g8gqLaNUokQsGtq2wXeLxL1d6n69+fDSJcTHl9jHGhEYgS6FeC3TDaZu4HNggIq8FcnIReUNE9riLHvluHy0ia0RkvYiMO8L1N6rqdYFcz4SfSwe358vbhnPV0I7cd3avUglg3JhevHLlQE7oXHo22lU7M9hnS64aEzYCXQo1H/ga545iAU51VCD+B5RaDU9EYoCXgDFAb+AyEektIn1E5MsyD+uGW8fFxng4rm0yj517HOf2b+tdXQ+gb9tkRh/XmrH92pQ65vyXZzHoL9+zeFu633PnFxZx+atz+GH1ngrfO/25H5m2eneNlGPVzgxum7iI6WvKX8uYSBfIgLvRIvI/YD1wEfAa0DqQk6vqDGB/mc1DgPXunUIeTvI5V1WXqeqvyjzsf2WE6dC0Phv/djbvXHcCQ7s2BeCKEzp43+/SPMn7fNfBHO8qexWZvHQnszak8dSU8mtpHcjKY/2eTO75aFmNxD1h7lYmLUnlwwXba+R8xtQlgbRRXIPzZX6TqtZEfUBbYJvP6+3ACZXtLCJNgb8CA0TkPlWtcBlWEbkRuBGgZcuWTJ8+vVrBZWZmVvvYuipUZf5xR8nzhBgY0iqW6/rANVOcbf/+ZjHPTVbWHiji4p5xjGwXR2pmER0aecgrhD9MywbAk59VLv6MXCe57MvM5cUPv6dP89L/1Kta5q3bnX/6qbv21Nl/H9H2bzvaygvBK/MRE4WqXlrD16yoS0ulfSNVNQ24+UgnVdXxwHiAQYMG6ciRI6sV3PTp06nusXVVOJR5je/lp0wGYMnekhHeH6zJZ9aeWLYfyCExzkOrRiVrZ/Vo34qRIwd4X2/cm+n8g/rhRwCeXZDL5idPL3W9qpZ58t4lsH07DRunMHLkkICPCyfh8HuuTdFWXghemStNFCIyU1WHi8ghSn+RC6Cq2qia19wOtPd53Q5Irea5TAR67apBTF29mz0ZuUxdvYdmDRLYl5nL9gPOins5+UVsTsv2zimVmVvSCW9rWjanPvtjuQbyo1XoTng4Y+1eNu/LolOzpCMcYUzkqDRRqOpw92dNzxT7C9BdRDoDO4BLcXpTGQPA6b1bcnrvlqgq6dn5NEyM5f3523j9p028+X9DaNO4nnesxcX/me1NFOnZeXy00GlDKF7zG5xlXY+WbzvJ6zM38fh5xx31OY2pKwJpzH47kG2VHDsRmA30FJHtInKdOwbjVuAbYBXwgaquqFrYJhqICE2S4omN8XDFCR2ZdvdI2qfULzUgr0FCLHM27uepKavp/9h3vDB1HQDNG5Ys7xobc/QD+AqKlOR6zgrAOw/mkOquJ25MNAikMftY3xfupIDHB3JyVb2sku1f4UwFYsxRKf7y/neZQXxp7jiMhFgPW9Kymb0hjQEdGld7IF9RkdI0KZ6Dh/P5fpXT5fa1qwcdReTG1B2V3lGIyH1u+0RfEclwH4eA3cDntRZhFYjIWBEZf/DgwVCHYmrJ7ad1574xvWjWoPTa3cU1RQM7NAHgslfncMd7i5i8dCfPfbeWaVvzvQstHUlWbgFfL9/Fxn1Z3DyiKwAZh20iQxM9Kk0UqvqE2z7xjKo2ch8NVbWpqt5XizEGTFUnqeqNycnJoQ7F1JLOzZK4aURX5j9wBp/+fhjn9m9DfEzJP+tLh7Rn5r2jAPhmxW5umbCQF6au462Vebw+cxOH8wrZebDyaqTCImX1rkPe1+PG9GJY16YUBZhkjIkEgXSPvU9EmgDdgUSf7TOCGZgxVTWgQxMGdGjCvaMP8+miHWTlFjCiR3Ma149n4YNn8OK0deTkF9G5WX3+9tVq/jJ5FX+ZvAqAFY+eRVJC6f8OH/yyjXs+XurtXXVm75aAMytuVl7pxZmMiWRHTBQicj1wB0431sXAiTgN1KcGNTJjqqlN43rcMqpbqW0pSfE8PNZpbluZmgGUHs39/NR1XDOsE18t28klg9vTMDGOhVsPAJDrrvF9wyldACdR2LrfJpoE0ph9BzAYmKOqo0SkF/BocMMyJnh6t2nEsyPqccHoURQpdP3zV4yfsZHxMzYCkJFTwIa9mews07Mpy+2GGx8rrNyZwWnPTue7P47AY9OimwgXyKSAOaqaAyAiCaq6Guh5hGOMCWtN63kQEWI8wh/LLOf6wtR1TF66k4Vb00ttP6lbMwCuG+7cWWzYm8UC967DmEgWyB3FdhFpDHwGfCciB7CR1CaC3HF6d648sQPph/PZuj+bqat2886craX2GdqlqXe1vuM7NuG/1w7m2v/+ws6DOaEI2ZhaFUhj9vnu00dE5AcgGZgS1KiMqWVNGyTQtEECXZs3YFTPFjxwTm9un7iIbi0a0K1FAwa43WyL9W7tzGBz+8RFfLRgOw0TYrnu5M4M7NCEqat289pPm7hndM9yxxlTFwXSmO07aU7xnM1h2TdQRMYCY7t163bEfY3xJzEuhvFXVT6grkXDBIZ2acrsjWnMWLsXgPTDeSTGxjDVXR/js0U7LFGYiBBIG8VCYC+wFljnPt8kIgtFJKAR2rXFxlGY2iIivHHNYGbeO4plj5zJiB7N+Xl9GlNX76FnS2d6tA17s1i8LZ37P13mnVqkWFpmLofLdLFdsOUAo/85g50HDzNh7lZy8q0LrgkPgbRRTAE+VdVvAETkTJxV6z4AXsbPWhLGRLJ68TG0i68PwL8uH8DK1Awa1YvjmNaN6DRuMjPX72Pm+n3e/S8Y2JZ2TepzMDuf4//yPSd0TuH9m4YCoKpc+O9ZAAx9YhoAW/dnc+MpXUhJiseYUArkjmJQcZIAUNVvgVNUdQ6QUPlhxkSPholxnNClKce4bRf/vXYwo3o255y+rb0D9oY/9QMz1u5l3CdLAWeG207jJnPHe4vYnJZd7pyv/LiBMc/buFYTeoHcUewXkXtxVrkDuAQ44K59baOOjKnAqJ4tGNXTWfK9qEg5/9+zWLItnavemFdu388Xp9IoMa7C8+zOyKXTuMn8cPdIOpdZA2PPoRxaNEys8DhjalIgdxSX44zK/sx9tHe3xQAXByswYyKFxyN8fstJfH/nCJ6+sC+3ndqNSwc7a3cVz3779pwtAPRs2ZCUpHi+/eMpfH7LSd5znPfSz6XOOXtDGkP+OpVxHy+tpVKYaBZI99h9wG0i0kBVM8u8vT44YRkTeYq72hY7uXtzerVuyMcLtvOyO036Py7pT+82JYtHntglhTkb93PwsDPbrYgzCnzbAaeq6r1ftnHXmT1JrhdHfGwgf/cZU3WBdI8dBrwGNAA6iEg/4CZV/X2wgzMmkp3TtzUA94zuxR2ndycnr4jk+qWroO4Z3YsLXnYauR/+YgU9WjYk1iOM+2SZd5/Bf/0egMuGdOBv5x/nTSbG1JRA2ij+AZwFfAGgqktE5JSgRmVMlEmIjSEhtvyiSgM7NOHDm4fym1dm89bsLeXeH9GjOUu3p3MgO5+J87aSmn6YSwa3Jy0rj4zD+axIPchXy3YBcEnPeIYMK6B+fMX/7acs30nTBgkM7lSz642bui+QRIGqbivzV0pYdvC2AXcmEg3ulMKscaeydvchOjdL4se1e1m8NZ1TejTnvAFtAcjMLeDWCQuZvmYvP7oDAMt6f00eHX7eXG5m3a+W7eT37y70vl752FmVJhMTnQL517DNrX5SEYkHbsdZ6zrsqOokYNKgQYNuCHUsxtSkNo3r0aZxPQCuGprEVUNLv98gIZaXLh/I54tTyS8son/7xrRPqU/G4XwmzNvKBQPbMvqfP7ElLYt/fLeW9XszaVwvjlN6NC+VJAAue3VuqYZ0gJz8QjJy8q2XVZQKJFHcDDwPtAW2A98CtwQzKGNM1SUlxHL5CR1KbUtJiufPZx8DQIv6wgfzt5d6/925zuSHJ3ZJ4fZTu3P5a3NZsi293Ln/8N5ipqzYxca/nW3TqkehQHs9XVELsRhjguiSnvEsyWpEl2ZJ3HBKF056chpxMcL7Nw2lT9tk4mI8XDOsEx8vKEkmszek8aePlrD9gLM2x90fLuHOM3vQrokzIn1laga/bN7PVUM7VtqI7ttby9RNlSYKEXnIz3Gqqo8HIR5jTJAc3zKWu0YO8b6uqC0iMS6GQ7kF/LRuL2mZefzh/cWl3v9k0Q4+X5LK6GNbsfdQLvM27wfgxC5N6dmqYal9VZWnv1nDpCWpTL7t5HI9ukzd4a/jdVYFD4DrgHuDHJcxJsgqarA+qVtTAH77+rxySQLggXOOoXuLBkxetpOVOzO829Myc9m0L4sz//GjtzH9qSlr+Pf0DWw/cJhNaVnlzmXqjkrvKFT12eLnItIQZ0nUa3Gm8ni2suOMMXXXyd2bs/SRM5myfBcb9mTSvGECAzs24eDhfPq2TaZpgwSuP7kL+YVFxHqEtbszOeufM3ht5iYuHNiOtbszufqNeSx5+Exe+XGD97x5tsZ4nea3jcJdi+JOnDaKN4GBqmprPxoTwRolxnHxoPZ+9yle7a9lI2de0Gmr9zDNXYcDoN+j35baP7egkMIi5YP52xjerRntU+p7t+cWFJWb62rWhn1c/upcerduxIQbTqBxfZtBN5T8tVE8A1wAjAf6VDB9hzEmyjWuH8+E60/giyWprN19qNw64/3aN2bJtnTW7DrEu3O2MmWFM/jv9lO7kRgfw9NT1gDw0uUDOadva/IKioiP9TB1lZN0Vu7MYNaGNM7u07pWy2VK83dHcReQCzwA3O/Ta0FwGrMbVXagMSZ6DOvWjGHdmgFOA3aRwtLt6YAz4vycF3/iL5NLhl4lxnl48Yf1qM86mbdMWMgtE8Aj8Jvj27N0x0FEQBUOHs4vdb3JS3dyco9mlc64a2qevzaKOjfDmI3MNia0RIQYodQSsD/ePYoZ6/ZSWKT8qm9rmjZIoLBIyS8sYvG2dC4dP8e7b5HC+/O3AXDZkPZMnLeN+z5ZRvsm9RnevRm7M3K4ZYIzQHDzk+fUbuGiWESN07eR2caEnw5N63Nl046ltsV4hBhPDCd2acrmJ88hO6+A9Ox8mjdMYOv+bLJzCzm2TSMOZOUzZcUufvfOAt6+/gQaJpZ8ZT3+5Ur+fPYxxNTgAEBV5byXZ1FYVMTfzu9D33aNa+zcdVlEJQpjTN1UPz7W2123a/OSqdhf+e3xXPDyzyzcml5uTY7XZ25i4ryt3H5adxokxNKrlbOWR5fmDdi4N5NDeUpVvDt3C09+vZpDOQUAPPT5Cj4rM5VJtLJEYYwJax//bhgrUjP4dsUudh7MIT7Wwx2nd+fK1+aydncmT369utJj/7ViBhNvPJEm9eN4efoGzjq2Fe/M2ULG4Xyeu6R/6ess2M6hnAKS4mPIKyyq0TuVus4ShTEmrIkIx7VN5ri2yaW2T7ptODl5Rew5lMOqXYeIEeFfP6xna1oWWXmFxAis2X2IgY9/5z3mmW/WeJ/3a9+YU3u1oH1KfeZsTGPh1nSGdE7hjWsGc9uEhezLzKt2zDPW7qVL8yTvVCd1nSUKY0ydVLyGR3L9OLq3dKYPKV4MCmD69On8ktuKGWv3sWzHwXLHP/zFCh7+YgUxHqGwyKmmGn1sKxokxBIf6yEjp6S3lbpdtAKds6p4bfRIaXC3RGGMiVh/OqsXfzqr/PZ35mzhxWnr2J2Ry8ndmxEX4+HuM3t656tKrhfHlrRsHv9yJSN7NuflHzawdHs6468axHFtk9m2P5us3AKGdE5h6/5sEmJjaJVcfgr2VTszOKZ13R9JYInCGBN1rjyxI1ee2LHS928e0ZUP5m/n9ZmbeH3mJu/2K16bW+H+9eNjWPjgGSTGxVBQWDJdyZjnf2LJw2eSXM//mI/iXl/Fa44AZOUW8NDnK9i2P5tnftOXjk2TAi1ejbNEYYwxZXRp3oDZ953Kj2v20rpxPZomxfPNil28OG19hftn5xXS68EpNGuQQEyZEWj3f7qMFy8bUGG11duzN/PXr1aRk+8klxWPnkVSgvO1PHtDGh8vdKZ8H/HM9JBWY1miMMaYCrROrselQ0oWgjqubTK3n9ad3Rk5tGtSn9yCQuJjPKRl5fHYpJWs2XWIw/nOnFbnD2jLH07vzohnpvPl0p3MXL+P03q1JDHOQ6N6cSzdns7a3ZnsPZRb6ppfL9/F2H6t2bAni+vfmg9Aswbx7MvMY0XqQY5tU9KgX1SkFKkSWzYzBYElCmOMCVBcjMfbkykhNgaAZg0SeOGyARXuP7ZfGyYtSSU9O997d+CrTXIiH9w8lB/W7OXBz5Zz94dLuPvDJaX2eerCvlz35nxun7iIqXeNBGDb/mxOfvoHAG4Z1ZW7zugZ1JUHIypR2BQexphw8sKl/Xnygj4kJcSSmn6YXzbvJ7egiJO7NyMlKd6bbM7p05rnvl3DgezS81r5Li61YW8Wf3x/MV8uTSW/sGQw4Us/bKBjShK/7t8maOWIqERhU3gYY8KJiHjbHNo0rse5/dtWuF9KUjyLHjqTwiLl9omL2JuZy11n9Ci3uNSni3Z4n/dpm+zt9nvPx0t5efp67q34xuaoRVSiMMaYuizGI7x0xcBy2xc/dAaH8wvJziskK7fAOweVqrJk+0He/2UbE+dt5Xffw9qRzlTtNckShTHGhLnG9eNpXMF2EaF/+8b0dAccbti6IyhTj1iiMMaYOq5efAxPXNCH6dPTgpIo6tyaE8YYY2qXJQpjjDF+WaIwxhjjlyUKY4wxflmiMMYY45clCmOMMX5ZojDGGOOXJQpjjDF+SfESf5FERPYCW6p5eDNgXw2GUxdYmaNDtJU52soLR1fmjqravKI3IjJRHA0Rma+qg0IdR22yMkeHaCtztJUXgldmq3oyxhjjlyUKY4wxflmiKG98qAMIAStzdIi2MkdbeSFIZbY2CmOMMX7ZHYUxxhi/LFEYY4zxyxKFS0RGi8gaEVkvIuNCHU9NEpHNIrJMRBaLyHx3W4qIfCci69yfTXz2v8/9HNaIyFmhizxwIvKGiOwRkeU+26pcRhE53v2s1ovICyJS86vA1JBKyvyIiOxwf9eLReRsn/fqdJlFpL2I/CAiq0RkhYjc4W6P2N+znzLX7u9ZVaP+AcQAG4AuQDywBOgd6rhqsHybgWZltj0NjHOfjwOecp/3dsufAHR2P5eYUJchgDKeAgwElh9NGYF5wFBAgK+BMaEuWxXL/AhwdwX71vkyA62Bge7zhsBat1wR+3v2U+Za/T3bHYVjCLBeVTeqah7wHnBuiGMKtnOBN93nbwLn+Wx/T1VzVXUTsB7n8wlrqjoD2F9mc5XKKCKtgUaqOlud/1lv+RwTdiopc2XqfJlVdaeqLnSfHwJWAW2J4N+znzJXJihltkThaAts83m9Hf+/jLpGgW9FZIGI3Ohua6mqO8H5xwi0cLdH0mdR1TK2dZ+X3V7X3CoiS92qqeJqmIgqs4h0AgYAc4mS33OZMkMt/p4tUTgqqquLpH7DJ6nqQGAMcIuInOJn30j/LKDyMkZC2f8NdAX6AzuBZ93tEVNmEWkAfAz8QVUz/O1awbZIKXOt/p4tUTi2A+19XrcDUkMUS41T1VT35x7gU5yqpN3u7Sjuzz3u7pH0WVS1jNvd52W31xmqultVC1W1CHiVkmrDiCiziMThfGG+q6qfuJsj+vdcUZlr+/dsicLxC9BdRDqLSDxwKfBFiGOqESKSJCINi58DZwLLccp3tbvb1cDn7vMvgEtFJEFEOgPdcRrB6qIqldGttjgkIie6PUKu8jmmTij+wnSdj/O7hggosxvf68AqVX3O562I/T1XVuZa/z2HulU/XB7A2Tg9CjYA94c6nhosVxecXhBLgBXFZQOaAlOBde7PFJ9j7nc/hzWEaW+QCso5EecWPB/nr6frqlNGYJD7n24D8C/c2QvC8VFJmd8GlgFL3S+N1pFSZmA4TnXJUmCx+zg7kn/Pfspcq79nm8LDGGOMX1b1ZIwxxi9LFMYYY/yyRGGMMcYvSxTGGGP8skRhjDHGL0sUJmqISCffmVbdbY+IyN1VPM9mEWl2hH3+XI34fiUii0RkiYisFJGb3O3niUjvqp7PmJpiicKY4KhSonBH344HxqpqP5w5faa7b5+HMyuoMSFhicIYl4hMF5F/isgsEVkuIkPc7U1F5Fv3r/3/4DNvjoh85k62uKJ4wkUReRKo564T8K677UoRmedu+4+IxJS5fEMgFkgDUGf2zzUiMgz4NfCMe2xX9zHFve5PItLLvcb/ROQVd9taEfmVu/1Yn2svFZHuQf0gTcSxRGFMaUmqOgz4PfCGu+1hYKaqDsAZBdvBZ///U9XjcUa93i4iTVV1HHBYVfur6hUicgxwCc7kjP2BQuAK34uq6n733FtEZKKIXCEiHlWd5W7/k3u+DTh3Hre5170beNnnVJ2AEcA5wCsikgjcDDzvXnsQpWcRNeaIYkMdgDG1qLJpCHy3TwRnrQcRaSQijXEWCLrA3T5ZRA747H+7iJzvPm+PM7dOWpnznwYcD/ziLipWj5KJ60qCUL1eRPoAp+MkgDOAa3z3cWcRHQZ8KCULlCX47PKBOhPFrRORjUAvYDZwv4i0Az5R1XWVfA7GVMgShYkmaUCTMttSgE0+r8smE61kOyIyEudLfaiqZovIdCCxgusK8Kaq3nekAFV1GbBMRN5247qmzC4eIN29O6jwFOVPqRNEZC7OXcY3InK9qk47UizGFLOqJxM1VDUT2Ckip4Gz1jIwGpjps9sl7nvDgYOqehCYgVtVJCJjKEk2ycABN0n0Ak70OU++20ANzkR1F4lIi+LrikhH39hEpIGbeIr1B7a4zw/htGGgzloEm0TkN+5xIiL9fI77jYh4RKQrzoSQa0SkC7BRVV/AqcbqG8jnZUwxSxQm2lwFPCAii4FpwKNuvX+xAyIyC3gFZzZWgEeBU0RkIc407Vvd7VOAWBFZCjwOzPE5z3hgqYi8q6orgQdwVhlcCnyHsxayLwHuEZE1bmyPUnI38R7wJ7cxvStO0rpORIpnBPZdtncN8CPOmsg3q2oOTvJb7p63F84ymMYEzGaPNcblVh3drarzQx1LdYjI/4AvVfWjUMdiIovdURhjjPHL7iiMMcb4ZXcUxhhj/LJEYYwxxi9LFMYYY/yyRGGMMcYvSxTGGGP8+n+CFGUVwEfPeAAAAABJRU5ErkJggg==\n",
|
|
"text/plain": [
|
|
"<Figure size 432x288 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAEICAYAAAB7+s71AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABFFklEQVR4nO2dd3xc1bXvv2vUm1Uty5ILBtuATQvFNtUG22A7gEMLJaGFXEKC827qTW6Sl3vvy30vySU3HUJMIJQECAnNAdMvxUAMprggG2PjhizbsorV++z3x5mRR6NzZs5oqqT1/Xzmo9E5e/beczT6zTprr7W2GGNQFEVRUh9PsiegKIqiuEMFW1EUZYSggq0oijJCUMFWFEUZIahgK4qijBBUsBVFUUYIKthKwhARIyLTfc/vFZH/9D0/W0S2Jnd29ojIEb55pw/z9Z8TkedjPS9lbKKCPcIRkV0i0ikibSLSJCJPi8jkZM8rEowxa4wxRzudDxT6oOM3iMjrbsYQkbNE5E0RaRaRRhF5Q0ROi2beNmMMEXdjzJ+NMefHchxl7KKCPTq4yBiTD0wEDgC/GU4nw7UiUx0RGQc8hXVdSoAq4D+A7mTOS1EiRQV7FGGM6QL+BszyHxORLBH5mYjsEZEDInKniOT4zi0QkRoR+Y6I7Af+KCL/LiKPiMj9ItIqItUicmpAf8eKyCsicsh37uKAc6+IyBcDfndlAfvnEaPLYMdMAGPMQ8aYfmNMpzHmeWPMRt/4HhH5gYjsFpE633svdJjrLhFZFPD7v4vIn3y/vub7ech3x3N68DUQkTNEZJ3P0l8nImcEnHtFRH7ks/5bReR5ESmL9cVQRi4q2KMIEckFrgTWBhz+KZZgnQRMx7IufxhwvgLL6pwK3Ow7djHwMFAErAJ+6+s/A/g78DxQDnwV+LOIOLozUoSPgH4RuU9ElopIcdD5G3yPc4EjgXx87zlCzvH9LDLG5Btj/hF4UkRKgKeBXwOlwM+Bp0WkNKDZNcCNWNc3E/jWMOahjFJUsEcHT4jIIaAFWAzcBiAiAvwT8HVjTKMxphX4f8BVAa/1Av9mjOk2xnT6jr1ujFltjOkHHgBO9B2fhyVmPzHG9Bhj/gfL1XB1fN9edBhjWoCzAAPcBRwUkVUiMsHX5HPAz40xO4wxbcC/AlfFwUX0aWCbMeYBY0yfMeYh4EPgooA2fzTGfOT7WzyC9UWrKIAK9mjhM8aYIiALWAG8KiIVwHggF3jX58I4BDzrO+7noM+VEsj+gOcdQLZPvCqBT4wx3oDzu7Gs9pTGGLPFGHODMWYScBzWe/ml73Ql1vvwsxtIByYQW4LH8Y8VeP2Cr31+jOegjGBUsEcRPv/sY0A/lkVZD3QCs40xRb5HoW+BcuBlEQxRC0wWkcDPzRRgr+95O9YXhJ+KiN9EAjDGfAjciyXcYL2vqQFNpgB9WAu4wYR6j+GuZfA4/rH22rRVlCGoYI8ixGI5UAxs8VnCdwG/EJFyX5sqEblgmEO8hSVY/yIiGSKyAOt2/mHf+fXApSKS6wvDu2nYb2YomSKSHfBI8x2XoOPZwS8UkWNE5JsiMsn3+2QsN47f1/8Q8HURmSYi+Vhuo78YY/ps5rEey12S4VuMvTzg3EEsF9ORDu9hNTBTRK4RkXQRuRJrgfipSC6EMnZRwR4d/F1E2rB82P8XuN4YU+079x1gO7BWRFqAF4FhLRIaY3qwFiSXYlnvdwDX+SxWgF8APViW6X3An4f3dmypxrpb8D9u9B0/I+h4p43vuRWYC7wlIu1YQv0B8E3f+XuwfPWvATuBLqwFVTv+N3AU0IQVGvig/4QxpgPr+r/hc0HNC3yhMaYBuNA3bgPwL8CFxph611dBGdOIbmCgKIoyMlALW1EUZYSggq0oihJjROQeXxLWBw7nRUR+LSLbRWSjiJzspl8VbEVRlNhzL7AkxPmlwAzf42bgd246VcFWFEWJMcaY14DGEE2WA/cbi7VAkYhMDNdvShf7yfRkmxxPQbKnoShKitPSX19vjBkfvqUz8xdkmaZGb/iGwKZNfdVY0UR+VhpjVkYwXBXwScDvNb5j+0K9KKUFO8dTwOlFlyR7GoqipDjPNdwVnEEaMU2NXlatdldra9rk/V3GmFPDt3REbI6FDdlTl4iiKEriqQEC69ZPwsqEDYkKtqIoSuJZBVznixaZBzQbY0K6QyDFXSKKoigjERF5CFgAlPlqvf8bkAFgjLkTq0zBMqws5A4OZ+6GRAVbUcYgeYXZXPHthUyYVoJVhTf1McZwYGcjf73tJdqbgwtMphbGmJAlh42VYn5rpP2qYCvKGOSKby9k1ikzyErPGVGCXVpSyhXfhnt/8HSyp5MU1IetKGOQCdNKRpRYA4gIWek5TJhWkuypJA0VbEUZg4jIiBJrPyN13rFCBVtRFGWEoIKtKEpK0NPTzde+s4LFFy/gims/Q01tTbKnlHKoYCuKkhL89YlHGFdQyAurXuGGz93Ez371k2RPKeXQKBFFUcLy99V5/Py3xezbn8bEin6+saKJi5a1R9XnE39/lLsfuAtBOHrmMTQ2NrDiS18D4IJFS/k/P/03jDFj2mcdjAq2oigh+fvqPH7wo1K6uqwb8tp96fzgR6UAwxbtbR9/xO/uvp2H/vg3SopLONR8iM/fdCUTK6yCdenp6RTkF9B0qImS4rEbFRKMukQURQnJz39bPCDWfrq6PPz8t8XD7nPt22+yZNHSATEuKizCbrtCta4Ho4KtKEpI9u1Pi+i4G4wxECTGFRMq2LffKqfR19dHa1srRYVFwx5jNKKCrShKSCZW9Ed03A2nzz2TZ59/mqZDTQAcaj7EefMX8fjfHwXguRefYd5pp6uFHYT6sBVFCck3VjQN8mEDZGd7+caKpmH3OeOomdxy0wqu/eJVeDweZh0zm//4/v/l2z/4OosvXkDhuEJ+8ZPfxGL6owoVbEVRQuJfWIx1lMglF1/GJRdfNujYr2+7I6o+Rzsq2IqihOWiZe1RC7QSPerDVhRFGSGoYCuKoowQVLAVRVFGCOrDVhRFAVq82bzYMd1l6/1xnYsTUVvYIjJZRF4WkS0iUi0i/2zTRkTk1yKyXUQ2isjJ0Y6rKIoy1oiFS6QP+KYx5lhgHnCriMwKarMUmOF73Az8LgbjKooyilj37ltccvWFzDp1Os++sDrZ00lJohZsY8w+Y8x7vuetwBagKqjZcuB+Y7EWKBKRidGOrSjK6GHixCp+/B+3ceGSi5M9lZQlpj5sETkC+BTwVtCpKuCTgN9rfMf22fRxM5YVTrYnP5bTUxRlmBQceoKyup+R3ldLX3ol9eXforXoM1H1GVxe9bb//AUAHo/GQjgRM8EWkXzgUeBrxpiW4NM2LxlamgswxqwEVgIUpo+3baMoSuIoOPQEE/Z9D4/pBCCjby8T9n0PYNiibVdeVQlPTL7KRCQDS6z/bIx5zKZJDTA54PdJQG0sxlYUJb6U1f1sQKz9eEwnZXU/G3afduVVlfDEIkpEgLuBLcaYnzs0WwVc54sWmQc0G2OGuEMURUk90vvsbSun426wK6+qhCcWFvaZwLXAeSKy3vdYJiK3iMgtvjargR3AduAu4CsxGFdRlATQl14Z0XE32JVXVcITtQ/bGPM69j7qwDYGuDXasRRFSTz15d8a5MMG8EoO9eXfGnafduVVr7nyWlZ84xZaWpp5+bWX+M2dv+TpR5+PxVsYNWimo6IoIfEvLMY6SsSuvOprz/0jqj5HOyrYiqKEpbXoM1ELtBI9GvCoKIoyQlDBVpQxiDHGdpfyVGekzjtWqGAryhjkwM5Guvs6R5T4GWPo7uvkwM7GZE8laagPW1HGIH+97SWu+DZMmFYyYnYmN8ZwYGcjf73tpWRPJWmoYCvKGKS9uYt7f/B0sqehRIi6RBRFUUYIKtiKoihxQESWiMhW38Yt37U5XygifxeRDb7NX24M16cKtqIoSowRkTTgdqzNW2YBV9ts7HIrsNkYcyKwAPhvEckM1a8KtqIoSuyZA2w3xuwwxvQAD2Nt5BKIAQp8BfTygUasHbwc0UVHRVEUoK0/izWHZrps/XqZiLwTcGClr5a/H7tNW+YGdfJbrEqmtUABcKUxxhtqVBVsRVGUyKk3xpwa4rybTVsuANYD5wFHAS+IyBqbDWAGUJeIoihK7HGzacuNwGO+vW63AzuBY0J1qoKtKIoSe9YBM0Rkmm8h8Sos90cge4CFACIyATgaa98AR9QloiiKEmOMMX0isgJ4DkgD7jHGVPs3dTHG3An8CLhXRDZhuVC+Y4ypD9WvCraiKEocMMasxtptK/DYnQHPa4HzI+lTXSKKoigjhFjtmn6PiNSJyAcO5xeISHPAno8/jMW4iqIoY4lYuUTuxYopvD9EmzXGmAtjNJ6iKMqYIyYWtjHmNawsHUVRFCVOJNKHfbqvyMkzIjLbqZGI3Cwi74jIOz2mK4HTUxRFSW0SFSXyHjDVGNMmIsuAJ4AZdg196Z0rAQrTx4+c7TAURVHiTEIsbGNMizGmzfd8NZAhImWJGFtRFGW0kBDBFpEKX0UqRGSOb9yGRIytKIoyWoiJS0REHsKq51omIjXAvwEZMBAofjnwZRHpAzqBq8xI2v1TUZSUxUyrGjPmX0wE2xhzdZjzv8UK+1MURYkZZloV7VPy4J3wbUcDmumoKMqIxC/Wh4o7kz2VhKGCrSjKiGNArEu6+ejZO5I9nYShgq0oyojCL9bN4/vY9vJd9B04mOwpJQwVbEVRRgx+sW6ZYNj+xh/p2bmHc340P9nTShgq2IqijAj8Yt1aIXz8zp/o+uAjTv/XM/n8pT3JnlrCUMFWFCXlGRDriR52v/MYHes2cvKK0/jCdbAod3uyp5cwdAMDRVFSGr9Yt1Wm8cmm1TS//SbHXXcCX/5qVkzFuqMvkw0HK2PWXzxQC1tRlJQlUKz3fvwKTa++yIzlM/lf3ysYU5a1HxVsRVFSkkCx3rf3beqfWcXUhUfwrR+Px1fpYsyhgq0oSsoRKNZ1jRupe+IvTJxTyXd+WYknTcakdQ0q2IqipBiBYl3f/hH7HnmAstllfO/OqWRkesasWIMKtqIoKUSgWDf27ab2wXvIrirhB3cfSXZe2pgWa9AoEWUY1HbNYVvnJXR5S8j2NDIj53Eqs99O9rSUEU6gWDd59rP3gbvIKM7lwtsXkldUM+bFGlSwlQip7ZpDdfu1eMkCoMtbSnX7tQAq2sqwCRTr5qxG9t5zJ55MD5++YxEXzFCx9qOCrThiZ0lv67xkQKz9eMliW+clKtjKsBgokQq05LdSc8/voKeHpSuXsGz2fhXrAFSwFVucLGkvmbbtu7wliZyeMkoIFOt9x/ZQc/+deJtbuOD2C7j41HoV6yBUsMcYbv3PTpY0eIGhMbDptCV8jsrIxkyrGnjeMr6fvX+5i759B1j434u59OxDyZtYCqOCPYaIxP/sbDHbJyz0k01t15yIhNVOmIGQc1QxHx34xdpfeW/bmnvo3rGb+f+5gCuXtANjq0aIW2IS1ici94hInYh84HBeROTXIrJdRDaKyMmxGFeJjFD+52CyPY0OvdgLtiHDth8n/F8eXd5SQAaEeUv7lY5zdHpNbdcc1+MqySdQrAdV3vvuGXz+sl5AxdqJWFnY92Lt2Xi/w/mlwAzfYy7wO99PJYY4WZ+Bx+2wOz4j5/FBlq6FwUmwnfpxmluPN9dWmEP5yHXBc+QzSKwnetj11l/peGcjp3z1NL5wvfXZUrF2Jlab8L4mIkeEaLIcuN+3U/paESkSkYnGmH2xGF9xdnc09R5Jbc+ZQ4QuEDtr2i+AgSLb582kj4KI+nGamyX+7sn2NEb0haOkHsFi/cnGp2h5Zy3HXX8Ct6ywPhsq1qFJlA+7Cvgk4Pca3zEV7BjhZH3W9MwH0hxf56F7wHccTGX224Ms12DhtevHbSigk6WeIW30m8xB7f19W/2WDnmNs/tGSRUCxbqtMo29W1+iac3LzLzsGD715VOAbSrWLkiUYNv9d9qaWCJyM3AzQLYnP55zGlU4W5lOyxQm4kW7YKvbihjxhF0wdHJz2M3pmNy/DBojeI7BXxihvnCU1CBYrGt3v0HDC09TNH82c781h3OKVazdkijBrgEmB/w+Cai1a2iMWQmsBChMHx/ZffMYxnIZDLU+LVEdamFnexqZX/yvQGRhdMFWdyCvNv3YIRSw33YOwWRI20DfdmPYuWlCzVUjSpJPsFjvP/AuB596jHGnzeDC/3ca55SoWEdCogR7FbBCRB7GWmxsVv919AQKUjptCL0YMgbOe+imMvONIT7sQKs0lqnmoax8D90hFzA9dA9Y16EI9YURiKbQJ59gsa47VM2BRx8i77gpXPiz05lfNrqFWkSWAL/Cslb+YIz5iU2bBcAvgQyg3hgTckfhWIX1PQT8AzhaRGpE5CYRuUVEbvE1WQ3sALYDdwFficW4Y5ngELc+CjBAhrRiuTsamJ33ALMLHmZ23gNkexoGHQ+0Vt2G+oXDyZec7WkcModJmS87zikWxPJ9KZETLNb17dvY9/B95BxVwcW/OodzK3YAo3eRUUTSgNuxIuRmAVeLyKygNkXAHcDFxpjZwBXh+o1VlMjVYc4b4NZYjKVY2C/kZZAmLZxX8qVBR0NZpbGMvLALBfRb8/ZzeDjiMdyiESXJI1isG3t3U/vg3WRXFnPR7Qs4r2onMHrF2sccYLsxZgeAz7uwHNgc0OYa4DFjzB4AY0xduE61HvYIJVaCFMoqjpTK7LdDWvOJJJbvS3GHmVY1RKybqKXmgZVklORx4e8WsnjqHmBUiHWZiLwT8Lg56LxTZFwgM4FiEXlFRN4VkevCDaqp6XEknoteTouMoWKhA+cyfeorVJZ+wPRxr7B590V4vYcjOTyenmFHXrj1McebUNa+EnsC64L4xfpQeh01d99Jem46F96xiAum1wCpK9b9vR6a9jvnGQRRb4w5NcR5N5Fx6cApwEIgB/iHiKw1xnzk1KkKdpyI96KXW0Ey06qobTiOzU2HRbnLW8rm3RdZcym1qgls37uQrp5CsjObmV71EhNL92KGGAQgO/dGPfdYE+qLUaNE4o+dWDdnNVLzx9/h8XhZdvsSls6yYgxSVazjgJvIuBos4W8H2kXkNeBEQAU70cQ7jdpJkCYeO1Rot+9dOMiCBvB6M9m+dyGVpR8MPNwQ+M8JyRfwcF+MKtDxxU6sW/JarJrW3d0sufMCLjzJcs2OIbEGWAfMEJFpwF7gKiyfdSBPAr8VkXQgEyuC7hehOlXBjhPhfMyxcJcEClKwkA4as6cwouOREDhuMsRb64skDzuxbi1oo+a+O/C2tXLBb86nZGYp0DDWxBpjTJ+IrACewwrru8cYU+2PnDPG3GmM2SIizwIbsRIm/mCMCWk5qWDHiVCJLNWtVw2KjY7GXRJKqAfmktlMV0/RkONZOS0DxeMDydvTHtEc7OaSKPGOZvFVE2uGj61YF3byyQN30lffyOJfLmb88eWcXeR4dz/qMcasxgppDjx2Z9DvtwG3ue1To0TixIycx/HQbXMmjZqec6OOEQ5ckQ9F+5Q8Jp/wOp603kHHPWm9TJm9xvE1do9IcDu/aBluNIiWah0+gZEgfrFuK+6i5qHf01e7n/N+ei4Vp0wcEOuxZl3HE7Ww44TfUtvUfiND07LtCx+5sQrdinQg5VM+BGBP9dl0d44jK6eFKbPXDBx3S3C/bixx/3zjZXGHWnwNtxiprpTICRRrwBLrkh5qHv4DPbs+4YjvXkbVGeNUrOOECnYcqcx+m03tN7luH8oqDCfU4Szg8ikfDgh0W6X1BdIWor5Hfm1/yP6Cxwwn3vESbr+4fthxJb3GKhbmoWdIWdkubymb2m9iS/uVHJv3F02sGQa2Yl3ay96/3U339p1M+eZyzlmuYh1PVLDjjLMve2gtDacY4VBiHU6o/eIcKXavCyXi/nkkS7j7TSb+69lHATU95zL0TsZK4a9uv5Z02mxre2tijT12Yt1e1sfex/5IV/U2Jn/tIhZcXqJiHWdUsOOM0y17ZeYb1PedGHbBy0msnYR6uALthuC+7QTcrdUdS+GOpN42WK6PNOnBY7o1scYFtmI9vp+9T9xH18YPqfrKUs69eryKdQJQwY4zoRM4rFoafl/rpvabBp2PRKzDCXVHxfDmn7vf+Zx/TCfL243VbaZVRS3aw3Fj9Jp8js+7W6NEQhAcCQKHxbr27/fT+X41lTefz8LrdYExUahgJxmnxA9TXkwlg0MyIxHq4Qp0uH6CBXxp6bt89VOrqchqYn93MSs3LOHFmsF7LIcT7mitbbdup+DXaGKNM45iXe6l9uk/0/HOJiZ+YRGLbpqkYp1AVLDjTLhMPKdoBX8Wop9gsY5EqHsPNdFVs5ue/fvoqa+jp6OR/pZ2TGcXpq8PRJCMdDy5OaSNyyczt4SM0vFkVUwkq3Iy6YVFiMiQ/i/rfZcfHvUIOb6QwcrsJr5z6t/oLvawZtNJQ+bRPiUvLm6SUG6nA72n+RYj3a0XKKHF+sCjD9GxYT0TbziPxV+aMqbjrJOBCnacCRc+5hitEJCFGE6sg0XaeL107vqYts2baP94C30HG6wTHg/pZcWklxaTflQZntxsJD0NDJjeXrztnfS3tNLxyQ76N74PxqpVk1Y8jtypM8mdcTR5M2eRlpMDwIppqwfE2k9OWi9fnbKaZxpOAYa6S+LhJgnldprNw5ogEwEhxfqxh2nd8C4V1y5g8ZePGPQ6ta4Tgwp2nAkVPvZq048dX5ed2QxEJtZ9ba00v/0mze+vpb+pGcnMIOuY6eQvPIOso6aSOakCycjADd7uHnr3HqBn1yd0f7ST9g+30Lr+HUjzkH3sDIqOPY2iyibb11ZkHT7u5OeOtZsklHtDXR/uCCvW69+h4nPzOX/FkQDqCkkCKtguCLTQ7DaeDWW9OftXcTzu8fQw+YTXB4l1SKFubaHx1Rdpfmct9PWRPXsGRVcsI+fEWXiyDhd9Kq5oBbpcv++mrMlkHTmZgvPOwHi99Oz8hI73qulYt4H9H/yJqtXCF84xfPV8mFZ++HX7+oqH9NVWmeYYVRLvRUklPK7F+n8dBUQu1nkHOije1UZat5f+LA9NR+TTPiE3xu9i9KOCHYZgH7Q/a9FKxLgegYF9FO1qgtj5V50XwwzZmc1MPuH1QVmITmLt7e2l6fWXaVrzEqa/n7zTT2HckvlkVIwH/AJtlx7vDuv1h2nyTCXrqKkUXbaE7q07KFzzLL95/hN+9SxcMRe+txymT8rgv5uWDcwxcJEyGmtbRTs+BEciDYkGefpBOta/T8Xn53P+V4cv1qXbWvB4rd/Tu72UbmuxxlPRjggV7DDYx/j6yRhSkTw4vdnOvxoqDO2cE37laFkHWtWde3axf9VD9O0/SM4px1F02TIyykt9IjtYaGNFoIA3eabTfewKrm15neyXnudPL3Xzl7Uw/bQKes+cTGbZ4Dm7Fe54RZIoQwkr1k89QMe6jVRct4Dzbx2+G6R4V9uAWPvxeK3jKtiRERPBDrc7sG9n4CeBnb5Djxlj/k8sxo43w4nx7fKWhFzoerXpx/a7xWQ2hxVr4/XS+PLzNL7yAmklRYz/+k3kzJ7pSqhPHB9cP92ZDQcrQ573i/dLnAWXnEXh+R3IC2v4+IXXMe/+F4VzzqD0vAtIy80bmH9wSKCdm0RdJInByQUCWBmMT95P53sfMPHGhSy+ZSowfJ91Wrc3ouOKM1ELdsDuwIuxdlBYJyKrjDGbg5quMcZcGO14iSaUD9qJDGkLGcpnG4bm81v7sRPr/s4Oah+/j67N28g9/WRKrlmOJyd7iOvCTyQCHe61TgIeOLYn7wIKzj2D5lUv0vzaG7Rueo+yxRcy7uQ5iMfj2tqOd9y2W6pbr6KmZz5WUUsvkzJfZXZB/DYOThShxLqttJe9j91L18YtVP7T+Sz64iRgsFhH6o/uz/KQbiPO/VlaLDRSYmFhu9kdeMRi74P20zvIhw1WjK8xhAzlq8x+G1NePGhbrkC/tZ1Y9zY2UPPn39PX0ETJdZeRf84cn1gODquD6ITaicA+Q4l3EwWUXHsJ+efOo+nBJ6l74hGaq9+mYtmVZI6fMPCeUt3atsQ6sB6JVRaXVoaI9kgJGwzlAgFoK+mm5pG76d6ynUm3LuW86yYCQ8U6Un900xH5g14D4PVYx5XIiIVg2+0OPNem3ekisgFrX7NvGWOq7Trz7T58M0C2J/l/0GAftJsoEacKfX73iplWRSWHt+UK5wbp3l9LzQO/h75+yr/5T2TPmGZrVcdDqO0IJd7+eTUxkfJvf4n2N97h0CNPs/v2/6Zs4RKKzlwQ0tpOlQVJy7IeWjyqpmc+szks2PHeuzNWhBPr1sJOav68kp4de5j89Ys59yprESLYDTIcf7T/uEaJRE8sBNvN7sDvAVONMW0isgx4Aphh15kxZiWwEqAwfXxwP0khXBxv8DlLwN3vaO7HUazvvQNJT6f8O7dQfnIuwb7qSITaTWbamkMzXffnH9tJuOWs08g5/hga//Q49c89ReuOaiZe/Dkyiq0vr2BrO3UWJJ1u1wcfHwl1tcOJdUt+KzUP/J7evQeY+p1LmX+JlbRl91kZrj+6fUJuygu09ApZ+93lKSSLWAh22N2BjTEtAc9Xi8gdIlJmjKmPwfgpR6ii+k7/PIH4xbqn/iA1992JZKRT/u1bKD8hc0jbcGI9nNTh4Ne4EfBQwt1EAWVfuZb2f7xH04NPsvuO25hw8WcpOP5TQKq6SLwM3XjCf/wwqVxXO9RnzS/WzdlN1Pzxd/Q3NTPth5/l7CVWFmvgZyBwkVH90cklFoIddndgEakADhhjjIjMwTJTGmIwdsKIxE8ZKlU6cEfzUIkxfW2t1Pzp92AM5d+8eYhYhxLqWNd3COwvnHifOL7W2do+4xSyZ0yj/q6H2P+XB+j4eBvjP30JnoyMiFwkiRDtSZmv2tTUNkzKfHVQO6dF6WTX1Q4n1h0V0FN3gJq778Tb1c2RP7qaMxdYcuAk1uDsj+4ozmTSW3Xq8ogzUQu2m92BgcuBL4tIH9AJXGWMSQl3hxvs/JSb2m8aUg41EDs3ilO51GBXiLe3l71/vRtvSyvl3/oS5SdlD2rvJNaJKMRzff/rnLZ3N/k93TSk5/HXklNZO+4o2/nZW9slTPiXW2h+4nlann2FzgN7qLriRjJKLNGzc5HEO63djtkFD0MrYaNEQt1NJQO7z1iwC6SjAjo/2UXtn/4A6WnM+Om1zJtrXeNQYg32/uiO4kzy67o0MSYBSCrrZmH6eHN6kfuNaeOFU9y0Hw/dzM57IKTP0s3tqd/KPPD4X2h59y3Kvvx5qj59xKDXJVOsj2yo4+zd28nwHjav+oBOTxb5XnsBd4ooadpfQOfGLTT84S8YYOJlnyPv6FkD5+3qcNultofb4SYRMdupEiXi9jPW9mE1+x+5n7TCcUz/8VXMOb4TsBdrNyF8k96qs3WT9GV5qJlbPuR4PJg2ef+7xphTo+kju2qymfKVb7hqu+0H34h6vOGgmY4uCOeP9JLl22zXXWRAKLFueX8dLe++xbhl57oS60SWtzxt7+5BYg3WB6jAa6W/l/W184WDViy5X7RDL0oeS8X//ioH7/gTtX+6m9KFSyievwgRialfG+Ij3MFCfXze3Skh1OD8GWte9w/qVv2NnKMqmPbvV3HqDKvImJNYuwnh08SYxKErBS5w549Mo7r9Wmq75gw548YVAtDTcJC6px4la+aRFH7m/EHnki3WAPk94euSZJl+rmh8Z8hxu/kXV7SSPr6UCd/9MrlzTqThxWfY+8S9eH3jdFQMLR1rVwe8fUpe2L0t3ew2Hwl+N5l15yUD4Xx2f/94YaZVuRbr9nIv9S+spu7Jv1Jw8pEc9eNrbcV6eetGJr1Vx9TX9lO2tcUxhC8QpwVHXYiMPWphuyB08sxh3IRzOUWFGK+Xfav+DGlplH7xSkoqD1uN8RBrO//kix3TQ76mLTOLAheiXdpnb/E6LUg27S+g9ItXkTmlkkN/e4Y9db+h6sqbyCiyqv658WvD8K3t4bg0kh3O51aowcpe3PfUQ3S8vZ6SCz7FpC8v4aSJB4ChYh28oGhHsOWsiTGJQwXbBUOTZ8Bp66lg90k469pvQR76x2v0fLyH0n+6ivGzDv+zxUKs3dZ+CG6Xd6CD7J3d5Pd005aZxZ5xxcxsrBviFgmmLTOLs4s+so0osXORDESRXDCfjMoJ1K98kD2//wWVV3+BnClHALGJ1/YTKNxOC8qb2z9HP9mOAp6scD43+3wGfrb62lrZ+6d76Pl4NxXXn0v55WdwUvk+YKgbpHjT0KQYO4It5/YJuWQ191Cw/3Dp3rbybF1wjAMq2A7YWV3zi/914Jzlsx56ex7KfeJ0297b1EjDS8+Qc+KxVF18eMEuGrGOtqh8sP+yoKebmY117M/Np6qtxZeSbxH479vr8bCu6nCxIKcwQOfwv2Oo+NdbOfib+6i553YmfOZKxp1kre1E4teG8AuSZloV2zZebrvjej9WPLJT5mKiw/ncbsgcKNbd+2vZ+9DdeFvbmPrdyyg669iBz5Sdz9qNz9nOcs470EF+XdcgEya/rovuwg4V7RijTiYbwvknK7Pf5vi8P+IJqjUdHM7l1ro+8NJjABRfs3xg70Q73Ij1otztMdkBxC4FOcPrZVJbCx6s+wsr2A0609IxQGtmFmumTmdH6eHIgLOLPnKct5NfO6NyAhO+fytZR03lwN8epP6F1RifVe/Wrw3OX5CBBG7F5oTf1RHIjJzHw/79Y4GTnxqGWtWBn6vW6g18svJX4O1n+n9dF1aswdnnbHyPviwPDTPGDRHhUOnqSmxRC9sGN/7JUMkxdjiJR/v2rXSu30zhpUsYP/vwnyNYzNyKdaxwsraCv07Sgc60NO7+1LyQ/YVykTj5tcu/fhONDz5J06sv0tWyj8qLPo8n0/q7ROIiAWdrOzuzma6eopBzh6Gujkj//pEQboE0lFVtvF4aXniWpldfJPPIKUz/90vIKCkIK9bg7Iu2E+lANEokcahg2+DWPzmcvQKD/7kOvvAk6eNLGLf4bKycosjFOh576jmlINvhJnoEQos2DPVrN+0voOTaS8mYWM6hR55mT33oxUiIfEFyetVLfLDzUpzWJPzYuTpivVfkcIUarGvR395G7aoH6KreRsn5J1H15SV4MtJdiTUMv0iTpqsnDhVsG2Lhn7SrORxM6/p36d27n9IvXUPJ5E7bNskQa7C3tpw2NovkH9P/ftxY2wOLkYvPJqNifMBi5I3kTJkGRC7aMNjariz9gEOtk6ipn+Pw7qxa5cGujlgky7gNNbT7/ASLdeeeXez72330t7QxacUySpecPOiLP5xYD4w1jCJNGiWSOFSwbYhXuvEg67qvj/rXniVzahVVF04baBNJ5b147lbtJgUZDv9j+ucSLjTQT6QukoHFyN/eR809d1B+4WUUnmq5YSLZhgyGCvesI56lqKBmoD55eloHAvT255Kd2cz0qpeYWLp3oA5MbcNxbG64MOKSqsOJBQ9nVRuvl0NrXqH+xdWkFRcx42c3kDt94rDEOhi3GxVo+dTEoYJtQ7T+STf/mC3vr6O/vomSz31mYKExEldIPMXaj5211V0Y+p94Ue72uIl2ExOY8L0VNKx8kLonHqF7317GL12OpFsf40isbRgs3JWlh+uTh2P73oX2axy9lzPx2OgzKsNZ1OAL2WttYd9Tf6arehs5Jx/HUf+yhLT87JiJdSQbFYyE8qmjARVsByLxTwbfHk8f98qQzQkGWdf9/TS88RKZ0yYzcZF9rY1ki7UTbv4xI7G2h7MYOf6fb+TQY8/R/NyrdNZ/QtVlN5A+rghwFm2wt7Yh9MJkbcNxg3YGml71kmNkiZuIk1A4uc7s6qS3bd7IgVWPYLp7qPrKUkqXnoyIxESsQTfOTVV0VSBK7EIAN+++iNqG4xxf07ppPf31jYz79HmO1rUTyRTrSHE710jD/iQtjeIrllH6pWvordnP7t/9N+3btw60sQv9g8Ghb3YEp7jXNhzH5t0X+aJIhK6eIjbvvoiMtA6HHrwh/+6hxnSyqocsLHa0s/+vf2bfg/eSVlLMzF9/kbJlpwwS6+BQyuF8ZjTyIzVRwY4S2xBAbybb9y4c8k/YUQHGGBrfepn0ieVUnj/Jtk8nARtJYu0nFqIdLNzFFa0UV7SSd9qJVPzgq6QV5FN730orXrv/sBVtJ9rgHLftxy+g2/YvwusdXIfc683EYC1EDiUt7Jd1YP+hLOpgoW6fYGj9YAO7fvtTWj94n3EXLeLYX19H9uSyQdco+DoO9zOj9UGiR0SWiMhWEdkuIt8N0e40EekXkcvD9alXP0ocQwADbo8D//k6d26n95Naxp1/NuKJzLpOJnkHOgaKAk16q468A05W5lCiFW0IkWQzsZwJ319B3lmn0vTqi+y599f01B8caDNcaxugu3Oc7fG+/lxmTf07MNTF4v+y9hMozm5EOlioOyqsTNi9f/sD+x++j7Siccz4+ReY9qV5SHqaowsEovuCbzoiH2+QOmjkh3tEJA24HVgKzAKuFpFZDu1+irWfQFjUhx0ljiGAmc227Rvffw1Pfi5VF9v7d1PRurZbgCrb2kJWcw+NM4tc9eHWrx1J2B8EFI+6/nJyZs+k8f7H2HPHzyhbfCGFc89EPJbq2EWSwOAv02Afd1ZOC92dQ/3SWTktFH5qJ+y0t3e6egpdZVkGj+/HP1dvbw9Na16mac1LIELRFcuY8vmTkDRr3FiJdahoEI38GDZzgO3GmB0AIvIwsBzYHNTuq8CjwGluOlXBjhLbEEBPD5NPsOpCBy429h5qonP9ZsYtmY8ny9rsM9Q/nZ9ku0LsFqAEKNgfeb0It1EkkYf9AaeeQOZRU2m871EOPv04LVvfZ8LSz5I14bCJ7STcMFS8p8xew8fvX4C3//DGrJ60XqbMXgOEFvRQOFn2/rkZr5fWje9R/z9P09/YTO6pxzP11gVkji8cuAaBRCvWoaJBxpJAe3rtPxcOlIlIYB3hlb4NxP1UAZ8E/F4DzA3sQESqgEuA81DBTgxDQgB9kQSFU3YOadvyzloA8ufPHXIulQmVph4YNeA2bjceog3+0L9Cxv/zjbS/+R6HHnmKPXf8jKJ551By7mLSsnMG2oYSbrBENbdyG1XFHupeP5PuznFk5bQwZfYayqd8CBBW0AP7CsWAUBtD+4fV1L/6DL01+8iYUkXpTVcxcf74Qe898PoEo9EgCaM+zI4zdllYwdt7/RL4jjGmP1QNoUBiItgisgT4FVb5uj8YY34SdF5855cBHcANxpj3YjF2KuAPARyU3cjhW2J/KF/z+rfInj2T8ceNHOsaQqep+8U8nKUWLObLj+jgyYITwo4dSTo7HHaR5J95CjknHMOhR5/h0Buv0rJxHaXnnM+4007Hk374Yx9OuEuO3krJ0VsHHWvzVWn0C/r+tWfR21pARkErFfNeJ/fobQNtnAj0qxuvl7bNm2h48wV699SSXl5K6T9dRdVFRzmuc8TKX63RIHGjBpgc8PskIHgh5lTgYZ9YlwHLRKTPGPOEU6dRC3aAc32xb5LrRGSVMSbQV7MUmOF7zAV+R9DtQTKI5V58dqnogxaQtm+l/1ALxdcsd91nKog1WAtNZVtbQqalh6vYZifmy2dsjEq0IbSLpIkCSm+4nIIF82j669McfPpxGv/xMiVnLmTcyafhyTgcARIooBHcFtsKuhPBi5/9nZ20vL+OQ+vW0HewgfQJZZTccDmTLpk54Kf2v8dAYrm46LYOiNu7J2WAdcAMEZkG7AWuAq4JbGCMGUhxFpF7gadCiTXExsJ241xfDtzv2yl9rYgUichEY8y+GIw/LOwK17tJLx4uTZvX4inIY+Ji+1C+VCawQH2gaAdGDYSy1EKJ+aK58VmMhEDhnkT5t26ma/M2mle9wMG/P0rDy89SeMrpFJ46j4ziwZE+wcIaiYCH6gcsa7pz1w5a3l9HW/V6TE8vmUdNoeyyJVQunRqRUEP0X+odxZkh/64QedajAsaYPhFZgRX9kQbcY4ypFpFbfOfvHE6/sRDssM51hzZVwBDBFpGbgZsBsj3xCyFK1BZP/mSHzg1bKFgwD0+GZXXH2hcZbxpnFoVMSw9lqbm57Y6FXxvsd2n3u0lyZs8ke9YMuj/aSesLa2h67SWaXnuJrGOOovDok8k79jjS84Z+5pziud3i7euja/cO2rZ8QNvWjfQ3tSDZWeTO+xQF58xlwryhC5eJEOu8Ax0UHBgs1gboyk+neFcbZVtb6M/yIP1G/dzDwBizGlgddMxWqI0xN7jpMxaC7ca57qaNddBaaV0JUJg+3rZNLAhXQjUSd0k4d0jbBxugr5+800+J1fSTQqiogVAV24p3tbm67Y5WtCHcLu3QtL+A7KOPJPvoI+lraKLt9XfoWPs+dVsegSf/Sua0SWRknk7nnivwdpxNemEOpYtfZ9xJ7tweYH1Bd9fupatmD+37t9O9bRemuwfJSCd79kxyLz+RnJNmUTp1aFnaRAi1n5KPW5Gg/zABclr6Bv5h07u99v+oqJ87GcRCsN041920SSihSqjG2l3SvOVd0ieWUz6nABh51rUddj7NhhnjHC1wt+U3IxFtsHeRQHg3CUATxRQtX0zhxYvo3VNLx/rNtL/1CT11jwKPANDXXM6BR2fQ/FYG2VMMadm5SEYGeDzg7cfb04O3q5O+tlZ6O5roravH23J4p5WMynLyzjiF7NkzyT52eoBIDxZrt9vBxfKz4emzl+Jg68opfkGzHhNPLAQ7rHMdWAWs8Pm35wLNyfRfQ+gSqrFyl/hjr7u37aRw+eKQ23+NJJx8mg0zxlEzt3xI2+JdbYj38C1VuEWrWFT8g9BuEhhsdWdOrSJzahXtb3wJy+X4DvAeUA1mO117d9B9YB+mp3dIP5KVSdq4AtKKC8k54RgyJpaTObmSzKlVePJyA74kwlvU/vdkRzK/yINroRss/7eSWKIWbJfO9dVYIX3bscL6box23GgJVUJ1U/tNtq8JtyO2rTukegMAuXNOGtJ+pFrXbmN3g4UdDlvWsa74B6GtbQgv3AB7msZhSdMC38M/ccPkO27D9PVjenrAGPB4kMwMJG1oCJ/VZz/QOuScUymCRAu1Nw3S7AsYDsHO6s5t6iE+Ww4rTsQkDjucc90XHXJrLMaKJU4lVCPZcSZc7euWre+TMaWS8hMti30k1A0Jh9vY3UiE3cmVEitrG8ILN0BG2SF6DxYPPT7+0CBht+gHhlrc4cYPJlmldBunj6N0a8uggkL+DXeDj9ndG6oPO/FopqMNsdhxxu8O6dnxCYWXLnH1mpFgXYP72F03wu4mZCxS0QZnaxsGC2eweFdc9yI1v12O6T58uy9ZPVRc96Kr8UON5TRXOxK1QQUMrRcSfEz6DWk2/m71YSceFWwbhrPjjK07ZPNGACZecCTgLrNxJOB2D79QGZKT3qobiCBxY4UPZwsyCC3cMFRQN5xr/dx//yJ664vIKDtExXUvUnLuRlfjurmDStY+nXY4Rf64dW0piUUF2wE3O86Ec4e0bttARlUFWVWhfd8wcqxrcL+Hn52wg3V77bekxeGu2sk6j8TaBvfC7efE8bXw2Vr47LOux4h0Lk6k6mdAK/elDirYcaCjAvraWunevptxFy4ccn4kW9d+3FRyc8qQ9OPxOgTjE/p2O1JrGwZfc7fiHQvc/K2TXTrX7Ua7KtDJRwU7TrR/WA3GULHYKhcQ6lY5VS2rWJDb1OMYx+vH62FYt9uRWtt+4inekX4ZJ1usNeV8ZKGCHQPs/NctOzaSVlZM9rRyp5eNCdxEErSVZ5Pb1DOs2+3hWNuB2AmsWxGP5k4p0UJtZ0lradWRhwr2MAnlv/Z2d9O1eTsFC+YOSZYZDe6QSAi18AiH43kDE27825FFIuDRCncg8fobJcuadrKkI10/UJKPCnaM6aiAjs1boa+PnBOtLdzGqjsEnBceA4k0zC8UsRTuWBGr7bqGi5MlPZz1AyW5qGDHiEHhfFuqkdwcKs4uS+KMUoPgCINQNbX97WJxm768dePAmG2ZWayrmsqO0sS5p4bzRRwvn3Ioi3m46wdKclDBjpLgzVaN10v79s3kHHc0kj44ZTmWhedHEv4IAzfxvLHYASV4nIKebhbs/ojjM/e72jAhEo5sqOO0vbvJ7+mOyiLOO9Bhu0lELHzKoRKd/L5sDdcbGahgDwMn/3VHBXTt2Y23tZ2ck44FRkcqeqxwE8/rNosyFG42TAhkOO6TRbnbrS+G3dFbxP4vGKdommh9yqESnTRcb2Shgh1j2j+sBo+HiecNFvWxttjoRDiBcJtFGUiw3zdSK324dzqxct/Y9RNItD5lTXxxh6cX8mtdVsNKEirYUWCbjr69mqwZR5CWnx3ytWPFHRIpkYqLnd83UYtpsdrANpyPOVKfcqiFS/9x/16bKtojCxXsGNLb1Ehv7QGKPvtpQN0hwyWcFZ53oIOS7S14fMaQXenP4Apz8VhMCxWymHegw7UYOvVjgIYZ44blXgl202Q195Bf16VJMiMcFewICeW/bv9HNQAVC6cOOjdWFxvjQd6BjiElQZ3o87lH4uUCCFUrpWxrCyUft9JelhU2KcjJDRSpWIOzm8auPIAmyYw8VLCjJNAd0rrzA9Irxrsq9qQMj+Jdba7Euj/LM2T3m1jjFzq76A4B0vrMIKF0smpj6WOOpTtGST1UsIdJcDhff1cXXVt3ULDoTEDdIfHCjcAkMpa4fUIuZVtbHM+7tWpjFa0RLrPUrr0yctC/VgzoqICO7Vuhv5/yc9UdEk9CCYzBcoMMx5UQrznZEU+rtumIfLw207ELGdQkmZFHVBa2iJQAfwGOAHYBnzXGNNm024W1uV0/0GeMOTWacZNFqPoh7R9W48nLJe/YyY5tlOhpOiLf1odtBOpnJlaowfKpS79x3EbLjnhatW4ySw3hN0JWUpNoXSLfBV4yxvxERL7r+/07Dm3PNcbURzleSuH3X5v+ftq3bSb7+KORNI+jO0St6+ETGKrmTQOvF8QXv+dNFxqPKnAlPrGs1WGXuWm3u3i8o1WC8btXpr6237FNvP37SnyIVrCXc3hr6fuAV3AW7FFDsP+6c89OvO0djF9w5KDjmiwTG4KFMa3fEr5ILepY1+qwi8jwhxSCZcV2FGcOu3RstMQia1RJLaIV7AnGmH0Axph9IuL0tW2A50XEAL83xqx06lBEbgZuBsj2pL5/raMC2ldXQ3oaBScfGf4FSsTEM6NwOP0EWulO7D6nYuB5o+ueh/YfjcgPJ2tUSW3CCraIvAhU2Jz6fgTjnGmMqfUJ+gsi8qEx5jW7hj4xXwlQmD7eKWkt4QT7rwfcIcbQtm0T2cccRVpulrpD4kC8MwqjKSxlRzQWbCzvAjQlffQRVrCNMYuczonIARGZ6LOuJwJ1Dn3U+n7WicjjwBzAVrBTnWB3SM+BffQdbKTiynmDjqs7JHbE6tY+XoWlAvF6oKM4M+INGEL1H02CixZ3Gl1E68xaBVzve3498GRwAxHJE5EC/3PgfOCDKMdNCToqoG3zJhBh3FznbaXUuo4Ou1C14dzax6IfJ2vcH1LYVp5Nfl0X6b4IDb+FnHegI6r+NcFFgegF+yfAYhHZBiz2/Y6IVIrIal+bCcDrIrIBeBt42hjzbJTjJpRQ4Xyt2zaSeeQUMorzNVkmTrRPyKVhxjj6sjxRxVrHoh8na9yfWZnb1ONoIUfbv6JEtehojGkAFtocrwWW+Z7vAE6MZpxUw++/7mmop/eTfVTeNNhrpO6Q2BOrW/to+wm3kBethawLhUooNDU9AgL91x0V0LZmIwCFZxzj+Bp1h4wuwi3kResn14VCJRQq2FHQunU9mUdMInNC0YA7RK3r0U8oKz0WFrIuFCpOqGMsDE7hfL1NjfTsqqF0gS42KoeJlb9dGfmIyBIR2Soi232Z4MHnPyciG32PN0UkrOtYLWyXDHWHrAeg8KxjkzQjJVVRC1kRkTTgdqxgjBpgnYisMsZsDmi2E5hvjGkSkaVY+SdzQ/WrFvYwad7yPplHTCKrotjWHaLWtaKMaeYA240xO4wxPcDDWKU8BjDGvBlQLG8tMClcpyrYIXByh/TU19G7Zy9lC50XGxVFGdWUicg7AY+bg85XAZ8E/F7jO+bETcAz4QZVl4gLgt0hrf/zPohQdPasgeO62KgoI5u0nn7y9rS7bV4fpky0U2XboQ1FzsUS7LPCDaqCHSHGGFqq3yVr5jQyysbZJsuoO0RRxjw1QGBx/EnAELEQkROAPwBLfXktIVGXiEv87pDuvXvoO1DP+PN1sVFRFEfWATNEZJqIZAJXYZXyGEBEpgCPAdcaY1zdoquF7YDffx3sDml56l1IT6fozGN1sVFRFFuMMX0isgJ4DkgD7jHGVIvILb7zdwI/BEqBO0QEXOzGpYIdAaavj9YP3iP3pGNJy89O9nQURUlhjDGrgdVBx+4MeP5F4IuR9KkuERf43SHtH23G29bBhE/PHjin1rWiKIlCBdsGJ3fIoeq3SCssoODkI7Uyn6IoCUcF2yV9rS10btpKyeLjkbShl02ta0VR4o0KdhjaKtOsxcb31oHXS+niE7XQk6IoSUEFOwg7d4jxemlev5asmdPIqipN1tQURRnjqGCHwL/Y2LFjG30HG5hw8eFiWrrYqChKolHBDsBuK7COCmja+Aae/FwKzzxGFxsVRUkaUQm2iFwhItUi4hURx4DvcHVhU41Ad0hfyyE612+m9PwT8WRYYetqXSuKkgyitbA/AC4FXnNqEFAXdikwC7haRGY5tU8V/IuNzevWgjGULjtZrWtFUZJKVIJtjNlijNkaplnYurCpgN1io7evj0PvvknBqUeRVVEMqHWtKErySIQPO9K6sEnHv9jYtmk93pY2yi48Ta1rRVGSTthaIiLyIlBhc+r7xpgnXYzhui6sb7ybgZsBsj3uNy6NNe0TDE1/f5X0ivEUnHwksE+ta0VRkkpYwTbGLIpyDFd1YQPGW4m1txmF6eMdhT2W2LlDunbtoGf3XibdupSTyvclYhqKoighSYRLJGxd2FTCv9jY8O7LePJzKT73BEB914qiJJ9ow/ouEZEa4HTgaRF5zne8UkRWg1UXFvDXhd0CPGKMqY5u2rHDzrruOVhH5/rNjL/wZD41+eCg9irWiqIki6jqYRtjHgcetzleCywL+H1IXdhUxG9dNz3+MpKRTumFpwHNWjNEUZSUQDMdg+hraaZlwzuULD6RU2c0Dzqn1rWiKMlkTO84E+gO8YfyNb3xKngN4y+dB3Sqda0oY4XuXmTn3mTPIiRqYQfQWtBG87o3KDpnFnOO7xx0Tq1rRVGSzZgVbFvr+s3XML19TPjsmYDWu1YUJbUYs4IdTGthB4feXkPhGccw95TeQefUulYUJRUYk4JtF8p36M3XMJ3dTLjqLECta0VRUo8xKdiBtFWm0Tqunaa1r1J4xjHMm9M/6Lxa14qipApjTrDtrOum11+xrOtrzgEOW9cq1oqipBJjTrADaatMoyW/lUNvraHonFnMO60v2VNSFEVxZEwK9iDr+tUXrciQa+YDal0ripK6jCnBDtyzsa0yjeasRg6te5OSRScw91PdSZyZoihKeMaUYMNg67rhpWcBYcLVZwNqXSuKktqMGcEOtq6bqKV1w7uMv/hUTpvVrmKtKErKM2YEGw5nNXZUQN3Lq0jLzeK8Lx+V7GkpiqK4YkwIdqB1DdC+fStd1R8x4aqzyBqXpda1oigjgjEh2HDYum4v93LwhSfJnFDEuTek9F7AiqIogxj1gh1sXbe8+xa9e/cz8cbzSMtMU+taUZQRw6gXbDhsXbcWdVL/P6vJmzWZcz5TqGKtKMqIYlQLdrB13fjyC3jbOqi8+XxEJEmzUhRFGR7RbsJ7hYhUi4hXRE4N0W6XiGwSkfUi8k40Y7oluN71obQ6Dq19jZLFJ3L66Uata0VR4oqILBGRrSKyXUS+a3NeROTXvvMbReTkcH1Gu0XYB8ClwO9dtD3XGFMf5XjDwhjDgRcfw5OVwcKvHTtwXMVaUZR4ICJpwO3AYqAGWCciq4wxmwOaLQVm+B5zgd/5fjoSlYVtjNlijNkaTR/xINi6bhvXTl99IxWfm09OaY7WulYUJd7MAbYbY3YYY3qAh4HlQW2WA/cbi7VAkYhMDNVpojbhNcDzImKA3xtjVjo1FJGbgZt9v3Y/13DXBxGP1uD7GeR8qb3ree6/C+6PrLcyICl3BkGkwjxSYQ6QGvNIhTmAzsPP1Gg7aOmvf+65hrvKXDbPDnLvrgzStSrgk4DfaxhqPdu1qQL2OQ0aVrBF5EWgwubU940xT4Z7vY8zjTG1IlIOvCAiHxpjXrNr6HvTK31jv2OMcfSNJ4JUmEOqzCMV5pAq80iFOeg8YosxZkkMu7OLajDDaDOIsIJtjFkUro2LPmp9P+tE5HGs2wVbwVYURRkF1ACTA36fBNQOo80g4h7WJyJ5IlLgfw6cj7VYqSiKMlpZB8wQkWkikglcBawKarMKuM4XLTIPaDbGOLpDIPqwvktEpAY4HXhaRJ7zHa8UkdW+ZhOA10VkA/A28LQx5lmXQzj6uhNIKswBUmMeqTAHSI15pMIcQOeRkhhj+oAVwHPAFuARY0y1iNwiIrf4mq0GdgDbgbuAr4TrV4wJ6TJRFEVRUoRRnemoKIoymlDBVhRFGSGkjGCnSpp7BPMImXYag3mUiMgLIrLN97PYoV3Mr0c8UmrjMIcFItLse9/rReSHsZ6Db5x7RKRORGwXyhN0LcLNIe7XQkQmi8jLIrLF9//xzzZt4n4txjzGmJR4AMcCRwOvAKeGaLcLKEvmPIA04GPgSCAT2ADMivE8/gv4ru/5d4GfJuJ6uHlvwDLgGaw40nnAWzF+727msAB4KgGfy3OAk4EPHM7H9Vq4nEPcrwUwETjZ97wA+CjRnwt9mNSxsE2KpLm7nIebtNNoWQ7c53t+H/CZGPfvRFxSauMwh4RgrASvxhBN4n0t3Mwh7hhj9hlj3vM9b8WKfAjeASTu12KskzKCHQH+NPd3fWnsycAppTSWTDC+mEzfz3KHdrG+Hm7eW7zfv9v+TxeRDSLyjIjMjuH4kZCIz4IbEnYtROQI4FPAW0GnUuVajFoSVUsESHyaexznEXFKaaTziKCbqK9H8LRsjkWdUhuHObwHTDXGtInIMuAJrKpniSbe18INCbsWIpIPPAp8zRjTEnza5iUaNxxDEirYJkXS3GMwj4hTSiOdh4gcEJGJxph9vtvKOoc+Yp32H5eU2ljPIVAsjDGrReQOESkziS/hG+9rEZZEXQsRycAS6z8bYx6zaZL0azHaGVEuEUmdNHc3aafRsgq43vf8emCI5R+n6xGXlNpYz0FEKkSsbYNEZA7WZ7lhSE/xJ97XIiyJuBa+/u8Gthhjfu7QLOnXYtST7FVP/wO4BOsbuhs4ADznO14JrPY9PxIrYmADUI3lwkj4PHy/L8NaKf84TvMoBV4Ctvl+liTqeti9N+AW4Bbfc8Eqzv4xsIkQUT1xnMMK33veAKwFzojT5/IhrHKXvb7PxU1JuBbh5hD3awGcheXe2Ais9z2WJfpajPWHpqYriqKMEEaUS0RRFGUso4KtKIoyQlDBVhRFGSGoYCuKoowQVLAVRVFGCCrYiqIoIwQVbEVRlBHC/wdEektSzKakSAAAAABJRU5ErkJggg==\n",
|
|
"text/plain": [
|
|
"<Figure size 432x288 with 2 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Generate Features from Data\n",
|
|
"\n",
|
|
"# change this to play arround with feature functions\n",
|
|
"#feature_fn = affine_features\n",
|
|
"#feature_fn = quad_features\n",
|
|
"feature_fn = cubic_features\n",
|
|
"features = feature_fn(samples)\n",
|
|
"\n",
|
|
"num_iterations = 25\n",
|
|
"\n",
|
|
"w_bce, l, l_info = minimize_with_sgd(features, labels, np.zeros(features.shape[1]),\n",
|
|
" schedule=(lambda t: 0.25),\n",
|
|
" num_iterations=num_iterations,\n",
|
|
" batch_size=1)\n",
|
|
"print(\"Final loss\", l)\n",
|
|
"\n",
|
|
"plt.figure()\n",
|
|
"plt.title(\"Cross Entropy Loss\")\n",
|
|
"plt.grid(\"on\")\n",
|
|
"plt.xlabel(\"Update Steps\")\n",
|
|
"plt.ylabel(\"Negative Bernoulli Log-Likelihood\")\n",
|
|
"plt.semilogy(l_info[1])\n",
|
|
"\n",
|
|
"plt.figure()\n",
|
|
"plt.title(\"Bernoulli LL Solution\")\n",
|
|
"pred_grid = np.reshape(sigmoid(feature_fn(flat_plt_grid) @ w_bce), plt_grid_shape)\n",
|
|
"\n",
|
|
"plt.contourf(plt_grid[..., 0], plt_grid[..., 1], pred_grid, levels=10)\n",
|
|
"plt.colorbar()\n",
|
|
"#This is just a very hacky way to get a black line at the decision boundary: \n",
|
|
"plt.contour(plt_grid[..., 0], plt_grid[..., 1], pred_grid, levels=[0, 0.5], colors=[\"k\"])\n",
|
|
"\n",
|
|
"s0 = plt.scatter(c0_samples[..., 0], c0_samples[..., 1], color=\"blue\")\n",
|
|
"s1 = plt.scatter(c1_samples[..., 0], c1_samples[..., 1], color=\"orange\")\n",
|
|
"plt.legend([s0, s1], [\"c0\", \"c1\"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 3.2) Effect of different Batch Sizes and Number of Iterations (1. Point)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Play around with the batch size and number of iterations and briefly describe your observations about convergence speed and monotonicity of the loss curve."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"- the convergence speed and monotonicity increases with the number of iterations\n",
|
|
"- the convergence speed decreases with a larger batch size while the monotonicity increases"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.8.8"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 1
|
|
}
|