{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ECE 417 Lecture 14: K-Means and Expectation Maximization\n", "## Mark Hasegawa-Johnson, October 12, 2017\n", "This file is distributed under a CC-BY license. You may freely re-use or re-distribute the whole or any part. If you re-distribute a non-trivial portion of it, give me credit." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Outline of Today's lecture\n", "* Preliminaries\n", "* Always initialize a GMM from a Gaussian\n", "* K-means clustering algorithm\n", "* Expectation Maximization (EM)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Preliminaries\n", "First let's load some libraries, and some data." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import scipy.stats as stats\n", "import requests\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['5.1', '3.5', '1.4', '0.2', 'Iris-setosa'],\n", " ['7.0', '3.2', '4.7', '1.4', 'Iris-versicolor'],\n", " ['6.3', '3.3', '6.0', '2.5', 'Iris-virginica'],\n", " ['']]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Download data from the UCI repository\n", "r=requests.get('http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data')\n", "# Split at every newline; split each line at commas\n", "dataset = [ x.split(',') for x in r.text.split('\\n') ]\n", "dataset[0::50]" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, 1, 2]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Get a dictionary from labels t indices, and back again\n", "label2class = { 'Iris-setosa':0, 'Iris-versicolor':1, 'Iris-virginica':2 }\n", "class2label = { 0:'Iris-setosa', 1:'Iris-versicolor', 2:'Iris-virginica' }\n", "# Read out a list of the class labels, convert to integers\n", "Y = [ label2class[x[4]] for x in dataset if len(x)==5 ]\n", "Y[0::50]" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 5.1, 3.5, 1.4, 0.2],\n", " [ 5.4, 3.4, 1.7, 0.2],\n", " [ 5. , 3.5, 1.3, 0.3],\n", " [ 5. , 2. , 3.5, 1. ],\n", " [ 5.5, 2.4, 3.8, 1.1],\n", " [ 6.3, 3.3, 6. , 2.5],\n", " [ 6.9, 3.2, 5.7, 2.3],\n", " [ 6.7, 3.1, 5.6, 2.4]])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a numpy arrays for each data subset\n", "X = np.array([ x[0:4] for x in dataset if len(x)==5 ], dtype='float64')\n", "X[0::20]" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYHHWV//H3yYWLBJJAIgIJZEkQBIwggRAuyoYV5aKs\nu4gXEMO6IK6IGlkMsI8iq7irLDdZnR+IRhbQzSo3w0VRJEQCgQRCEAiYwUDCdRIgIYJhyJzfH/Xt\npKenu6s6U1Vd3fN5PU8/M11V/a1T1dV9ui6nvubuiIiINGJQswMQEZHWo+QhIiINU/IQEZGGKXmI\niEjDlDxERKRhSh4iItIwJY8MmZmb2YQc5mNm9hMze8XM7k/4mplm9q0MY8pl2RthZoeZ2Ypmx9EI\nMzvUzJ5Isb3U3ncz297M7jaz18zsv8zsHDP7URptF4WZjQvb8pDw/DYz+0yz4ypJe/toRNOTh5kd\nYmbzzGy1mb1sZveY2f79bHOamf2hYlimX5b9US3eBh0CfAAY4+4HZNB+XWZ2l5n9c1btD2TuPtfd\nd9+U12b9vgOnAiuBbdz9q+5+gbtv0nYQtw1Vfok3i7sf6e4/bWYM5fqzffRXU98IM9sGmA18HpgF\nbAYcCqxrZlzVmNkQd3+r2XHUsAuwzN3/0uxANkXB163UtgvwmCeoNNZ73IbcvWkPYBLwasw0pwCP\nA68BjwHvDcNnAJ1lwz8ahr8L+CuwHlgLvEr0C6kbeDMM+1WYdkfgl0AX8GfgjLL5ngf8ArgGWAP8\nc5XYZgIdwB0hjjnALmXjHZgQ/h8OXB3m9TTwb0R7fn3irbEedgRuBl4GlgKnhOGfrXj9NyteV7X9\nEPt/A7eE2OcD48tet0dYrpeBJ4Dja8T17dD2X0P7l5ct+2nAn8J78N+AhXHTgHuAi4FVwLfC8H8K\n7/UrwK8r1mWieMK02wI/AZ4Lbd0Yhh8GrCibruo2FMZNCO/naqJf1/8bhluI+6WwXTwC7B3GbQ5c\nCDwDvBi2jS3DuFFEP5ReDcswFxiU4DNSGfMy4ExgcYjtf4Etqrwu6/d9Jr0/U39H9Jm5JowfF7aB\nz4b1cTewBdHnaVVYDw8A21NjG6qY3zOhvbXhMYXoc7RfGH9CGL9X2efixrL35ZKwPTwX/t+8xnIN\nDu/hSuAp4Auh3SFh/F2E7wJ6b8evhukPCsOXh23kM2Vt19s+DgNWAF8Nr3seOLnstUcRbaOvAc8C\nZ9bYPt4VYnwVeBT4SMV7VvW9p852XXPb7M+Xf38fwDZhQ/opcCQwsmL8x8KK2j8s3ATCF0oYtyPR\nF/DHgb8AO5S9qX+osrF/q+z5IGAh8HWiPZ5dw5v/wTD+PKIPx9+Habes8QF6DXhf2DAuLZ8vvZPH\n1cBNwNZEH6wngc/WirfKvO4GfkD0AdyHKAlNTfL6OutjFXAA0R7otcDPw7itiDb+k8O4fYk+THvW\naP8uKpJrWPbZwAhg5xDvh8rieQv4Ymh/S+BYoqT4rjDs34B5mxjPLURfqiOBocD7a3zQ6m1DPwPO\nDeO2AA4Jwz8YtpsRRNvku8peczFRgt82vM+/Ar4Txn2H6MtiaHgcSkimMe97ZczLgPtD3NsSJdvT\nmvS+z6T3Z+o8+iaPq0O7WwKfC+vkbURf0vsRHfKCKttQxbxK7Q0pG3Y18NXw/xVEPwQ+XzbuK+H/\n84H7gLcDo4F5wL/XmM9pwBJgbFi/v6d+8ngrrK/BwLeIEsN/E30fHEH0/TAswfZxWGjr/LB9HAW8\nTvhOJEomh4b/R7LxR/SG7SO8bilwDtF32tQw/90TvPc1t+ua70ncxpv1IwQ5kyjrvhVW7vZh3K+B\nLyVsZxFwbMyHpnxDnww8UzHN2cBPyj4Id8fMc2Zp5Yfnw4h+QY0Nz50o4Q0m+oW2Z9m0nwPuqhVv\nxXzGhna3Lhv2HWBmwtfXWh8/Knt+FLAk/P9xYG7F9P8P+EaN9u+ievI4pOz5LGBGWTyV6/42QjIN\nzwcRfXh2aSQeYAegh4ofIl7xQUuwDV1N9IU0pmKaqUSJ/0DK9hyIPnB/ofev+CnAn8P/5xP9eJjQ\n4OejV8xEyePEsuffBTqa9L7PJD557Fo2/p+IvrgnJtmGKsaX2itPHp8Fbg7/Pw78Mxu/DJ9m4xds\nJ3BU2es+SHSYt9p87qQsGRMlgHrJ409l0747TLt92bBVRD/24raPw4A3KpbvJeDA8P8zRN8Z29Ta\nPoh+kLxA7+3yZ8B5Cd77qtt1vUfTT5i7++PuPs3dxwB7E/2iuiSMHkv0xvdhZieZ2SIze9XMXg2v\nHdXArHcBdiy9PrRxDtFudMnyBO1smMbd1xLt7u9YMc0ool8FT5cNexrYKWGsOwIvu/trm/j6Wl4o\n+/91ouQH0bqZXLFuTgDekVL70Hfd7gJcWja/l4k+cDs1GM9YonX1SlxwMdvQWWH+95vZo2b2TwDu\nfidwOdGvy5fM7Ipw7m400S/qhWXt3R6GA3yP6Ffhb8zsKTObERdfHfXWa39en9b7Xq78ff4foh+E\nPzez58zsu2Y2tB9tzwEONbMdiH6gzQIONrNxRIeJF4XpdqTvZ6/yM0rZtMsrpq3nxbL/3wBw98ph\nw4jfPgBWee/zQuXvzT8Sfdk/bWZzzGxKrdjdvaci/vLviarvfZ3tuqamJ49y7r6EKDvuHQYtB8ZX\nTmdmuwBXAqcD27n7COCPRB92iLJ/n+Yrni8nyvojyh5bu/tRdV5TzdiyuIYR7ZI+VzHNSqJDYLuU\nDduZ6JBckvk8B2xrZlvXeH2cJMtRbjkwp2LdDHP3z6fUfrXXLAc+VzHPLd19XoPxLCdaVyPqzTxu\nG3L3F9z9FHffkegX3w9Klx67+2Xuvh+wJ/BO4F+J3uM3iI65l2Ic7u6lD+drHl2RtCvwEWC6mR3e\n8FprTNbve0MxuHu3u3/T3fckOjdwDHBSwlj7jHf3pURfgF8kOkqwhujL8VSiPa7Sl+hz9P3sVX5G\nS56n7DMdpk1D3e0jjrs/4O7HEh16u5EoUVZ6DhhrZuXf64m/J2ps1zU1NXmY2R5m9lUzGxOejwU+\nSXR8EuBHwJlmtl+oZZgQPvRbEW1MXeF1J7Mx4UD0a2CMmW1WMWzXsuf3A6+Z2dfMbEszG2xme2/C\nZcJHhcuNNwP+HbjP3Xv9qnb39URv9rfNbOuwDNOJTh7Wirf89cuJdve/Y2ZbmNlEol32a6pNX0Xd\n9quYDbzTzD5tZkPDY38ze1ed9netMS6pDuBsM9sLwMyGm9nHGo3H3Z8nOgT2AzMbGaZ9X5X51d2G\nzOxjpe2S6KS7Az1hvpPDL+a/EJ3k7QlfVFcCF5vZ20MbO5nZB8P/x4Tt14hOdK8nOrxWuox85iau\nt3qyft8bYmZ/a2bvNrPBRCdluwnrgPhtqCtMWznNHKIfAHPC87sqnkN06ObfzGy0mY0iOs9Z67Mz\nCzjDzMaY2Uiiiyr6LW77qMfMNjOzE8xsuLt3E627niqTzidKpmeF9+4w4MPAzxPMo+p2Xe81zd7z\neI3o3MN8M/sLUdL4I9EVB7j7/xFdiXFdmPZGYFt3fwz4L+Beoo3u3URXPZTcSXSlwQtmtjIMuwrY\nM+wy3hi+0I8hOh75Z6JfBj8i2t1txHXAN4gOs+wHnFhjui8SvSlPAX8Ir/txnXgrfZLouO9zwA1E\nx6F/mzDGJO1vEA6PHQF8IszvBeA/iU4CVnMpcJxFRYqXJYypcp43hHn83MzWEG0HR25iPJ8m+mJa\nQnTc+MtV5he3De1PtF2uJToP9yV3f4roIo8riRLK00THtL8XXvM1okNT94Vl+C1QugZ/t/B8bZjn\nD9z992Hc2Ip5pyXr971R7yC6gnEN0TmKOUSHsiBmG3L314m+C+4Jn+EDw6g5RCef767xHKIT2QuI\nrlB7BHgwDKvmSqJDaw+H6a5vfDFrqrd9xPk0sCy87jSiw4m9uPubRMniSKLvsx8AJ4UjOnHqbddV\nlS6dlE0Qfi2ucPd/a3Ys0prCXsHDRCeRu5sdj0hSTS0SFBnowq/FVA4LieSp2YetRESkBemwlYiI\nNCzzPY9wFdNDZja7yrjDLLoh4qLw+HrW8YiISP/lcc7jS0RXVtQqOJnr7sckbWzUqFE+bty4NOIS\nERkwFi5cuNLdR8dPmUymySNcJ3800SV209Noc9y4cSxYsCCNpkREBgwzi6uWb0jWh60uIbrNQ71i\nk4PMbLFFnazsVW0CMzvVzBaY2YKurq5MAhURkeQySx5mdgzwkrsvrDPZg8DO7j4R+D5REWAf7n6F\nu09y90mjR6e21yUiIpsoyz2Pg4GPmNkyovL4qWbW65YA7r4m3EwQd78VGBpuHyAiIgWWWfJw97Pd\nfYy7jyO63cGd7t7r1h1m9o5wrx/M7IAQz6qsYhIRkXTkXmFuZqcBuHsHcBzweTN7i+iOk59wFZ6I\niBReyxUJTpo0yXW1lQB0zOlk4pjhHDR+45HOeZ0rWbxiNae9v8+d/EUGNDNb6O6T0mpPtyeRljVx\nzHBOv+4h5nVGN4yd17mS0697iIljGr0xsog0SjdGlJZ10PhRXP6pfTn9uoc4cfLOXDP/GS7/1L69\n9kREJBva85CWdtD4UZw4eWcuu3MpJ07eWYlDJCdKHtLS5nWu5Jr5z3DG1AlcM/+ZDYewRCRbSh7S\nskrnOC7/1L5MP2L3DYewlEBEsqfkIS1r8YrVvc5xlM6BLF6xusmRibQ/XaorIjIA6FJdERFpOiUP\nERFpmJKHiIg0TMlDREQapuQhIiINU/IQEZGGKXmIiEjDlDxERKRhSh4iItIwJQ9pio45nX3uQTWv\ncyUdczqbFJGINELJQ5pCHTmJtDZ1BiVNoY6cRFqb9jykadSRk0jrUvKQplFHTiKtS8lDmkIdOYm0\nNiUPaQp15CTS2tQZlIjIAKDOoCRzqsEQkThKHtKHajBEJI7qPKQP1WCISBzteUhVqsEQkXqUPKQq\n1WCISD1KHtKHajBEJI6Sh/ShGgwRiaM6DxGRAaDl6jzMbLCZPWRms6uMMzO7zMyWmtliM3tv1vHI\nwKF6FZHs5HHY6kvA4zXGHQnsFh6nAj/MIR4ZIFSvIpKdTJOHmY0BjgZ+VGOSY4GrPXIfMMLMdsgy\nJhk4yutVLvrNExsuAtBlxyL9l/WexyXAWUBPjfE7AcvLnq8Iw3oxs1PNbIGZLejq6ko/SmlbqlcR\nyUZmycPMjgFecveF/W3L3a9w90nuPmn06NEpRCcDhepVRLKR5Z7HwcBHzGwZ8HNgqpldUzHNs8DY\nsudjwjCRflO9ikh2Mkse7n62u49x93HAJ4A73f3EisluBk4KV10dCKx29+ezikkGFtWriGQn9xsj\nmtlpAO7eAdwKHAUsBV4HTs47Hmlfp71/fJ9hB40fpfMeIinIJXm4+13AXeH/jrLhDnwhjxgkP9N+\ncj8HT9iOUw7d+OV95dxO7lm6ipknH9DEyEQkLbo9iaTu4AnbccEtS7hyblSMd+XcTi64ZQkHT9iu\nyZGJSFrUn4ekrrTHccEtS7jj0Rd5YNkrnHP0Hr32RESktWnPQzJxyqHj2X/cSO5f9gr7jxupxCHS\nZpQ8JBNXzu3kgWWvcMC4kTyw7JUNh7BEpD3osJWkrnSOo3SoqvQc0B6ISJtQ8pDU3bN0Va9zHKW/\n9yxdpeQh0ibUn4eIyADQcv15iIhI+1HyGGCmXngX596wuNewc29YzNQL72pOQP2gzp6kVXS93sW0\n26ex8o1Nv69aGm2kScljgJkyfluunb98QwI594bFXDt/OVPGb9vkyBqnzp6kVXQs7uDBFx+k4+GO\n+IkzbCNNOucxAJUSxo7Dt+C51X/lhMlj+fZHJzY7rE1SShgnTt6Za+Y/o86epHC6Xu/iyOuPZN36\ndWw+eHNu/8fbGbVlY9toGm3onIf027c/OnFD4thx+BYtmzhAnT1J8XUs7qDHo/7werxnk/Yc0mgj\nbUoeA9C5NyzekDieW/3XPudAWok6e5Ii63q9i5uW3kR3TzcA3T3d3Lj0xobOW6TRRhaUPAaY0iGr\nEyaPZd7Zh3PC5LG9zoG0EnX2JEVXvsdQ0uieQxptZEFFggPMvZ0v9zrHUfp7b+fLzQxrk9Tr7EmH\nr6QIHn7p4Q17DCXdPd0semlRrm1kQSfMRUQGAJ0wl35JozYirg3VX4i0PyWPASaN2oi4NlR/IdL+\ndNhqAEqjNiKuDdVfiBSLDltJv6VRGxHXhuovRNqbkscAlEZtRFwbqr8QaW9KHgNMGrURcW2o/kKk\n/Sl5DDD1aiPSaiONeYhIsemEuYjIAKAT5gVVlNqGosQhkoei9XExkCh5pKQotQ1FiUMkD0Xr42Ig\n0WGrFBWltqEocYhkKY0+LgYSHbYqsKLUNhQlDpEsFbGPi4FEySNFRaltKEocIlkpah8XA4mSR0qK\nUttQlDhEslTUPi4GEiWPlBSltqEocYhkqah9XAwkm3TC3Mxuc/cjM4gnVpFPmIuIFFVuJ8zN7L01\nHvsB+yQIdAszu9/MHjazR83sm1WmOczMVpvZovD4ej+XZ8Cb9pP7uXJu75qOK+d2Mu0n9wPJ6kDy\nqBVRPYpIa6t32OoB4ELgvyoeFwIjErS9Dpjq7u8hSjYfMrMDq0w31933CY/zG4pe+jh4wnZccMuS\nDQnkyrmdXHDLEg6esB2QrA4kj1oR1aOItLZ6fZg/DnzO3f9UOcLMlsc17NHxsLXh6dDwaK2ikhZ0\nyqHjAbjgliXc8eiLPLDsFc45eo8Nw0vnQOrVgSSZpr/ymIeIZKfensd5dcZ/MUnjZjbYzBYBLwF3\nuPv8KpMdZGaLzew2M9urRjunmtkCM1vQ1dWVZNYD2imHjmf/cSO5f9kr7D9u5IbEUZKkDiSPWhHV\no4i0rprJw91/4e5P1Bh3Y5LG3X29u+8DjAEOMLO9KyZ5ENjZ3ScC3weqtuvuV7j7JHefNHr06CSz\nHtCunNvJA8te4YBxI3lg2St9zoEkqQPJo1ZE9SgiLczdc3kAXwfOjJlmGTCq3jT77befS21X3L3U\nx31ttl9x99Kqz+9Z2uX7nv8bv2dpV9XnSafprzzmISIbAQs8xe/0zOo8zGy0mY0I/28JfABYUjHN\nO8zMwv8HEO0JrcoqpoHgnqWrep3jOOXQ8Zxz9B7cszRarUnqQPKoFVE9ikhry+zGiGY2EfgpMJgo\nKcxy9/PN7DQAd+8ws9OBzwNvAW8A0919Xr12VechItK4tOs86l1tVT7Tg4Bx5dO7+9X1XuPui4F9\nqwzvKPv/cuDyhLGKiEhBxB62MrP/IartOATYPzxSy17tIo2it7gCvzTayKNIMI3lKJTFs+DiveG8\nEdHfxbM2qZkkHRepcyNpFUnOeUwCDnb3f3H3L4bHGVkH1mrSKHqLK/BLo408igTTWI7CWDwLfnUG\nrF4OePT3V2dsUgJJ0nGROjeSVhF7zsPM/g84w92fzyek+op8ziONTphKX7T7h8tsy09+p9VGkjj7\nuyxpLEchXLx3SBwVho+Fr/wxcTNJOi5S50aSpWZ0BjUKeMzMfm1mN5ceaQXQTtIoeosr8EujjTyK\nBNNYjkJYvaKx4TUk6bhInRtJK0mSPM4D/h64gN73uJIKaRS9xRX4pdFGHkWCaSxHIQwf09jwKpJ0\nXKTOjaTVxCYPd59DVJ+xdXg8HoZJmTQ6YSod6jnn6D2YddpBnHP0Hr3OHaTRRpI4+7ssaSxHYRz+\ndRi6Ze9hQ7eMhieUpOMidW4krSbJ1VbHA/cDHwOOB+ab2XFZB9Zq0ih6iyvwS6ONPIoE01iOwph4\nPHz4sugcBxb9/fBl0fCEknRcpM6NpNUkOWH+MPABd38pPB8N/NajW63nrsgnzEVEiqoZJ8wHlRJH\nsCrh66QJ4mo01AlTgS2eRdclezOtYzdWXrLp9ST9tWTVEqZcN4UnX36yKfOX1pAkCdwerrSaZmbT\ngFuAW7MNSzZVXI2GOmEqqFBP0jF4LQ9usTkdg9ducj1Jf82YO4O13Ws56+6zcp+3tI5E97Yys38E\nDg5P57r7DZlGVYcOW8WLq9FIox5FUnbx3nStfZYjx+zIukGD2Lynh9tXPMeoYTs1VE/SX0tWLeFj\nsz+24fkvP/xL3rntO3Obv2SnGYetcPdfuvv08Gha4pBk4mo01AlTAa1eQceI4fRgAPRgdIwY3nA9\nSX/NmDuj13PtfUgtNZOHmf0h/H3NzNaUPV4zszX5hSiNiqvRUCdMxdM1Ygw3DduK7kFR8ugeZNw4\nbCtWjkheT9JfS1YtoXN173Nfnas7de5DqqrXk+Ah4e/W7r5N2WNrd98mvxClEXE1GmnUo0j6OiZM\noifq2maDHjM6JuR3D9LKvY4S7X1INUnqPMab2ebh/8PM7IxSJ09SPHE1GuqEqZge7llLd0Xy6DZj\nUc/a3GJYsbb6IbJaw2VgS1LnsYjozrrjiK6yugnYy92Pyjy6KnTCXESkcc04Yd7j7m8BHwW+7+7/\nCuyQVgBFkEbtQ1wbefVxoTqOTZBSfx2ZS1AHkld/IHHziY0jxz5SJBtJkke3mX0S+AwwOwwbml1I\n+Uuj9iGujbz6uFAdR4NS7K8jUwnrQPLqDyRuPnXH59xHimQjyWGrPYHTgHvd/Wdm9jfA8e7+n3kE\nWCmrw1Zp1D7EtZFXHxeq42hASv11ZC5BHUhe/YHEzSc2jhz7SJGNcj9s5e6PufsZ7v6z8PzPzUoc\nWUqj9iGujbz6uFAdRwNS6q8jcwnqQPLqDyRuPrFx5NhHimQnydVWB5vZHWb2pJk9ZWZ/NrOn8ggu\nT2nUPsS1kVcfF6rjaEAK/XXkIa4OJK/+QOLmkyiOnPpIkWwlOedxFXARcAiwP9GVV/tnGVTe0qh9\niGsjrz4uVMfRoBT668hDXB1IXv2BxM0nURw59ZEi2UqSPFa7+23u/pK7ryo9Mo8sR2nUPsS1kVcf\nF6rjaFAK/XXkIa4OJK/+QOLmkyiOnPpIkWwlOWH+H8Bg4HpgXWm4uz+YbWjVqc5DRKRxzajzmEx0\nqKq8D/ML0wqgXeRRKzL1wrs494bFvcafe8Nipl5416YFLS0n87qG2dPp+taoqJbkW6Ng9vTG21g8\niyWX7s2UH+/Fk5cWuG5G+iXJ1VZ/W+UxNY/gWkketSJTxm/LtfOXb0gg596wmGvnL2fK+G1TXhop\nqkzrGmZPhwVX0TF8WFRLMnwYLLiqsQQSajhmbNnN2kHGWVt2F7NuRvotyWGr7Yn2OnZ09yND3ccU\nd78qjwArFfmwVR61IqWEsePwLXhu9V85YfJYvv3RiWkvihRQ5nUN39yWrkHet5akx+AbLydr4+K9\nWfL683xspx3ADNz55bPP88637VCsupkBqBmHrWYCvwZ2DM+fBL6cVgDtJI9akW9/dOKGxLHj8C2U\nOAaQzOsafH31WhJfn7yN1SuYMbr3NnvW6FHFq5uRfkuSPEa5+yygByDc56qBrWngyKNW5NwbFm9I\nHM+t/mufcyDSnvKoa+gaMrR6LcmQ5HcjWjJyDJ2bDY32OgDM6NxsKE+OLFbdjPRfkuTxFzPbDnAA\nMzsQ0HWfFfKoFSkdsjph8ljmnX04J0we2+sciLSvPOoaOibsv2GvY8M8MDomJC/rmrH96KrDz6ox\nXFrXkATTTAduBsab2T3AaOC4TKNqQfXqK5Ievopr497Ol3ud4yj9vbcz4fFoaVl51DU8/Lat6F5X\nUUsyyFj0tq0St7Hirdc27nWUmEXDpa3EnjAHMLMhwO6AAU+4e3fMSzJT5BPmIiJFlfYJ89g9DzMb\nDBxF1BnUEOAIM8PdL4p53RbA3cDm4XW/cPdvVExjwKWh/deBac0qPhQRkeSSnPP4FTAN2A7YuuwR\nZx0w1d3fA+wDfCicLyl3JLBbeJwK/DBZ2MklKd4rSgdKcR1GtdKypNLZz+zp8M1t4bzh0d9q9QZp\nzCeNTpYStJGHe5+9l/dc/R7mPze/+gRx6yvhcuTRCVOSeRShM6hWiTNtSZLHGHf/B3f/hrt/s/SI\ne5FHSh0wDw2PymNkxwJXh2nvA0aYWaq9FCYp3itKB0pxHUa1zLKk0dlPKFjbcJmor+9bsJbGfNLo\nZClhG3k48+4z6fEeps+pkWjrra8GliOPTpiSzKMInUG1SpxpS5I8bjOzIzalcTMbHPpAfwm4w90r\nfw7tBJT3CrMiDEtN6aTz6dc9xEW/eWLD1UzlJ7GTTJOH0s0SL7hlCcd3zNtwF97SzRRbZll+dz50\nv9F7WPcb0fCkFs6MH57GfH53Pl0967hp2Fa4hUtTe9b1aqN0mazj1S+PTdBGHu599l7WvLkGgDVv\nrum79xG3vhIuR+z6SEGSeeQRR7vEmYUkyeM+4AYze8PM1pjZa2a2Jknj7r7e3fcBxgAHmNnemxKk\nmZ1qZgvMbEFXV1fDr09SvFeUDpTiOoxqiWVJo7OfWoVp5cPTmE8anSwlaCMPZ959Zq/nffY+4tZX\nwuXIoxOmJPMoQmdQrRJnFpIkj4uAKcDb3H0bd9/a3bdpZCbu/irwe+BDFaOeBcaWPR8ThlW+/gp3\nn+Tuk0aPbvx68STFe0XpQCmuw6iWWJY0OliywfHD0+hUKIVOluLayEP5XkdJn72PmPWVZDlyKVZM\nss4L0BlUq8SZlSTJYznwR09yTW8ZMxttZiPC/1sCHwCWVEx2M3CSRQ4k6jvk+UbmEydJ8V5ROlCK\n6zCqZZYljQ6W9psWPzyNToVS6GQpro08VO51lPTa+4hZX0mWI5dixSTrvACdQbVKnFlJkjyeAu4y\ns7PNbHrpkeB1OwC/N7PFwANE5zxmm9lpZnZamObW0P5S4ErgXzZhGepK0jlSUTpQiuswqmWWJY0O\nlo65CCZ9duOehg2Onh9TdoV4Gp0KpdDJUlwbeXjtzepFeL2Gx6yvJMuRS7FiknVegM6gWiXOrCS5\nq+43qg1PcsVVFlQkKCLSuNzvqlt+eW4jl+q2ksLURkhvSWo40qjzyCOOhHHWrQdIqaYl8/VVIO1Y\nX1EUNZOHmV0S/v7KzG6ufOQXYvYKURshvSWp4UijziOPOBqIs2Y9QIo1LZmur4Jpx/qKoqh52MrM\n9nP3hWZidfNzAAAQ5ElEQVT2/mrj3X1OppHVkNVhqzQ6cpIUXbx3+JKrMHzsxk6FkkxThDgSxlm3\ns6c0ljWP9VUgmXee1WJyO2zl7gvD3znAY8Bj7j6n9EgrgKJoem2E9JakhiONOo884kgYZ916gJRq\nWvrdRgtp1/qKoqh7zsPMzjOzlcATwJNm1mVmDVxv2TqaXhshvSWp4UijniSPOBK0EVsPkMay5rG+\nCqKd6yuKot45j+nAwcD+7r6tu48EJgMHm9lX8gowD4WojZDektRwpFFPkkccCdqIrQdIY1nzWF8F\n0c71FUVRb8/j08An3f3PpQHu/hRwInBS1oHlqRC1EdJbkhqONOpJ8ogjQRux9QBpLGse66sg2rm+\noijqnTD/o7tXvRdVvXFZU52HiEjj8qzzeHMTx4mkJvY6/QR9fuRyrX9KccRNs2TVEqZcN4UnX36y\n78i8ajjarFakKH2TtJp6yeM94S66lY/XgHfnFaAMbHWv00/S50dcG2lIMY64aWbMncHa7rWcdfdZ\nvUfkVcPRhrUiRembpNXUu1R3cLiLbuVja3cfmmeQMjDF9oOQoM+PXPpSSCmOuGmWrFpC5+rorged\nqzt7732k0bdJEnnNJydF6ZukFSW5MaJIU8Rep5+gz49crvVPKY64aWbMndHrea+9j7xqONqsVqQo\nfZO0IiUPKaRE1+nH9PmR27X+KcQRN035XkdJr72PvGo42qhWpCh9k7QqJQ8ppETX6cf0+ZHbtf4p\nxBE3TeVeR8mGvY+8ajjaqFakKH2TtColDymkRNfpx/T5kdu1/inEETfNirXVDwttGJ5XDUcb1YoU\npW+SVhXbn0fRqM5DRKRxuffnIQNUEa7lTyOG2dNZcsHbmfLjvXjygrdXrb/IJY6E4uoB2rFeQFqT\nkof0VYRr+dOIIdRfzBg1krWDjLNGjaxaf5F5HA2Iqwdox3oBaU1KHtJXEa7lTyOGhTNZMnQInZsN\nBTM6NxvKk0OH1K7LyCqOhOLqAdq1XkBak5KH9FWEa/nTiMHXM2N0735Zzho9qnZdRlZxJBRXD9Cu\n9QLSmpQ8pK8iXMufQgxLNtt8w14HsHHvY7PNc40jibh6gHauF5DWpOQhfRXhWv4UYpgxdteqw8+q\nMTyrOJKIqwdo53oBaU1KHtJXEa7lTyGGFfbWxr2OErNoeI5xJBFXD9DO9QLSmlTnISIyAKjOQ9pH\nGvUTcW0UqEZDBq523DaUPKQ50qifiGujYDUaMnC147ah5CHNkUb9RFwbBarRkIGrXbcNJQ9pjjTq\nJ+LaKFCNhgxc7bptKHlIc6RRPxHXRkFqNGTgaudtQ8lDmiON+om4NgpSoyEDVztvG0oe0hxp1E/E\ntVGQGg0ZuNp521Cdh4jIAKA6DxERabrMkoeZjTWz35vZY2b2qJl9qco0h5nZajNbFB6t1xFyE/S7\n4KgIHT0liSNhnG1TgLV4Fl2X7M20jt1YeUnz3pe2WZ+SqSz3PN4CvuruewIHAl8wsz2rTDfX3fcJ\njxw7jGhd/So4KkJHT0niaCDOtijACsvbMXgtD26xOR2D1zbnfaFN1qdkLrPk4e7Pu/uD4f/XgMeB\nnbKa30DR74KjInT0lCSOhHG2TQHW786nq2cdNw3bCjfjxmFbsbJnXe7vS9usT8lcLuc8zGwcsC8w\nv8rog8xssZndZmZ71Xj9qWa2wMwWdHV1ZRhp8fW74KgIHT0liSNhnG1TgLV6BR0jhtNDdBfgHoyO\nEcNzf1/aZn1K5jJPHmY2DPgl8GV3X1Mx+kFgZ3efCHwfuLFaG+5+hbtPcvdJo0ePzjbgAkul4KgI\nHT0liSNBnO1UgNU1Ygw3DduK7kFR8ugeFPY+RuT3vrTT+pTsZZo8zGwoUeK41t2vrxzv7mvcfW34\n/1ZgqJmNqpxOIqkUHBWho6ckcSSIs50KsDomTKKnou+RHjM6JqR2ZWV8DG20PiV7WV5tZcBVwOPu\nflGNad4RpsPMDgjxrMoqplaXSsFRETp6ShJHgjjbqQDr4Z61dFckj24zFvWszS+GNlqfkr3MigTN\n7BBgLvAIUPo5cw6wM4C7d5jZ6cDnia7MegOY7u7z6rWrIkERkcalXSQ4JK2GKrn7HwCLmeZy4PKs\nYmhLi2dFV+CsXhEd/z/86/nvNaRl9nRYOBN8Pdhg2G8aHFN1J1VECiaz5CEZKNU+lC5hLdU+QOsl\nkNnTYcFVG5/7+o3PlUBECk+3J2klRanRSMPCmY0NF5FCUfJoJUWp0UiDr29suIgUipJHKylKjUYa\nbHBjw0WkUJQ8WklRajTSsN+0xoaLSKEoebSSotRopOGYi2DSZzfuadjg6LlOlou0BHUGJSIyAKgz\nqCbomNPJvM7e9/eZ17mSjjmdTYooRlH664jTKnHmRetDWoiSRwITxwzn9Ose2pBA5nWu5PTrHmLi\nmOFNjqyKovTXEadV4syL1oe0GCWPBA4aP4rLP7Uvp1/3EBf95glOv+4hLv/Uvhw0voD3cGyVWpBW\niTMvWh/SYpQ8Ejpo/ChOnLwzl925lBMn71zMxAGtUwvSKnHmRetDWoySR0LzOldyzfxnOGPqBK6Z\n/0yfcyCF0Sq1IK0SZ160PqTFKHkkUDrHcfmn9mX6EbtvOIRVyATSKrUgrRJnXrQ+pMUoeSSweMXq\nXuc4SudAFq9Y3eTIqmiVWpBWiTMvWh/SYlTnISIyAKjOQyRvi2fRdcneTOvYjZWXbGL9hWo4pM0o\neYjUE+ovOgav5cEtNqdj8NrG6y9UwyFtSMlDpJ7fnU9XzzpuGrYVbsaNw7ZiZc+6xuovVMMhbUjJ\nQ6Se1SvoGDGcntCjcg9Gx4jhjdVfqIZD2pCSh0gdXSPGcNOwregeFCWP7kFh72NEA/UXquGQNqTk\nIVJHx4RJ9Jj1GtZjRseEBi5aUQ2HtCElD5E6Hu5ZS3dF8ug2Y1HP2uSNqIZD2pDqPEREBgDVeYiI\nSNMpeYiISMOUPEREpGFKHiIi0jAlDxERaZiSh4iINEzJQ0REGqbkISIiDVPyEBGRhmWWPMxsrJn9\n3sweM7NHzexLVaYxM7vMzJaa2WIze29W8Qwo6nhIRDI2JMO23wK+6u4PmtnWwEIzu8PdHyub5khg\nt/CYDPww/JVNVep4qNR/RKnjIdC9lEQkNZntebj78+7+YPj/NeBxYKeKyY4FrvbIfcAIM9shq5gG\nBHU8JCI5yOWch5mNA/YF5leM2glYXvZ8BX0TDGZ2qpktMLMFXV1dWYXZHtTxkIjkIPPkYWbDgF8C\nX3b3NZvShrtf4e6T3H3S6NGj0w2w3ajjIRHJQabJw8yGEiWOa939+iqTPAuMLXs+JgyTTaWOh0Qk\nB1lebWXAVcDj7n5RjcluBk4KV10dCKx29+ezimlAUMdDIpKDLK+2Ohj4NPCImS0Kw84BdgZw9w7g\nVuAoYCnwOnByhvEMHBOPV7IQkUxlljzc/Q+AxUzjwBeyikFERLKhCnMREWmYkoeIiDRMyUNERBqm\n5CEiIg2z6Jx16zCzLuDpJoYwCljZxPk3olViVZzpapU4oXVibYc4d3H31KqsWy55NJuZLXD3Sc2O\nI4lWiVVxpqtV4oTWiVVx9qXDViIi0jAlDxERaZiSR+OuaHYADWiVWBVnulolTmidWBVnBZ3zEBGR\nhmnPQ0REGqbkISIiDVPyqMPMBpvZQ2Y2u8q4w8xstZktCo+mdJhhZsvM7JEQw4Iq483MLjOzpWa2\n2Mze24w4QyxxsRZlnY4ws1+Y2RIze9zMplSML8Q6TRBnUdbn7mUxLDKzNWb25Yppmr5OE8ZZlHX6\nFTN71Mz+aGY/M7MtKsZnvj6zvCV7O/gSUd/r29QYP9fdj8kxnlr+1t1rFQYdCewWHpOBH4a/zVIv\nVijGOr0UuN3djzOzzYC3VYwvyjqNixMKsD7d/QlgH4h+kBF1+HZDxWRNX6cJ44Qmr1Mz2wk4A9jT\n3d8ws1nAJ4CZZZNlvj6151GDmY0BjgZ+1OxY+ulY4GqP3AeMMLMdmh1UUZnZcOB9RB2Z4e5vuvur\nFZM1fZ0mjLOIDgc63b3yLhFNX6cVasVZFEOALc1sCNGPhucqxme+PpU8arsEOAvoqTPNQWGX8DYz\n2yunuCo58FszW2hmp1YZvxOwvOz5ijCsGeJiheav078BuoCfhEOWPzKzrSqmKcI6TRInNH99VvoE\n8LMqw4uwTsvVihOavE7d/VngQuAZ4HmiHlh/UzFZ5utTyaMKMzsGeMndF9aZ7EFgZ3efCHwfuDGX\n4Po6xN33IdpN/YKZva9JcSQRF2sR1ukQ4L3AD919X+AvwIwmxBEnSZxFWJ8bhENrHwH+r5lxxImJ\ns+nr1MxGEu1Z/A2wI7CVmZ2YdxxKHtUdDHzEzJYBPwemmtk15RO4+xp3Xxv+vxUYamaj8g40/ArB\n3V8iOj57QMUkzwJjy56PCcNyFxdrQdbpCmCFu88Pz39B9CVdrgjrNDbOgqzPckcCD7r7i1XGFWGd\nltSMsyDr9O+AP7t7l7t3A9cDB1VMk/n6VPKowt3Pdvcx7j6OaPf1TnfvldnN7B1mZuH/A4jW5ao8\n4zSzrcxs69L/wBHAHysmuxk4KVx9cSDRLu7zecZZii8u1iKsU3d/AVhuZruHQYcDj1VM1vR1miTO\nIqzPCp+k9qGgpq/TMjXjLMg6fQY40MzeFmI5nOjCnnKZr09dbdUAMzsNwN07gOOAz5vZW8AbwCc8\n/3L97YEbwrY8BLjO3W+viPNW4ChgKfA6cHLOMTYSaxHWKcAXgWvD4YungJMLuk7j4izK+iz9YPgA\n8LmyYYVbpwnibPo6dff5ZvYLokNobwEPAVfkvT51exIREWmYDluJiEjDlDxERKRhSh4iItIwJQ8R\nEWmYkoeIiDRMyUPajpmtt+iOp4+a2cNm9lUzGxTGTTKzy5oU17yU2tnWzO4wsz+FvyPTaFekEbpU\nV9qOma1192Hh/7cD1wH3uPs3mhtZOszsu8DL7v4fZjYDGOnuX2t2XDKwaM9D2lq4FcqpwOmh2vYw\nC/2zmNl5ZvZTM5trZk+b2T+Y2Xct6nPkdjMbGqbbz8zmWHRDx19buDupmd1lZv9pZveb2ZNmdmgY\nvlcYtsiiG+jtFoavDX/NzL5nUV8Mj5jZx8Pww0KbpT46ri1VM1c4Fvhp+P+nwN9ntwZFqlPykLbn\n7k8Bg4G3Vxk9HphKdCO8a4Dfu/u7iaqHjw4J5PvAce6+H/Bj4Ntlrx/i7gcAXwZKezanAZeGm0BO\nIroPVbl/IOo34j1E9yn6nm28Xfa+oa09gV2J7rNWafuyW028QFS9L5Ir3Z5EBrrb3L3bzB4hSjC3\nh+GPAOOA3YG9gTvCTsBgottgl1wf/i4M0wPcC5xrUZ8w17v7nyrmeQjwM3dfD7xoZnOA/YE1wP3u\nvgLAzBaFNv9QK3h3dzPTsWfJnfY8pO2Z2a7AeuClKqPXAbh7D9Bddp+iHqIfVwY86u77hMe73f2I\nyteH9oeEtq4j2pN5A7jVzKY2EO66sv83tFnhxbJDZzvUWC6RTCl5SFszs9FAB3D5Jt7A7glgtIX+\nwc1sqMV0ABSS1VPufhlwEzCxYpK5wMfNbHCI733A/Q3EdDPwmfD/Z8I8RHKlw1bSjrYMh3yGEt11\n9H+AizalIXd/08yOAy6zqOvXIUS9TD5a52XHA582s26icxIXVIy/AZgCPEzUu+JZ7v6Cme2RMKz/\nAGaZ2WeBp8P8RHKlS3VFRKRhOmwlIiINU/IQEZGGKXmIiEjDlDxERKRhSh4iItIwJQ8REWmYkoeI\niDTs/wMS7nz8bFY0NAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plot a scatter plot of the three classes, so we can see how well they separate in the first two dimensions\n", "plt.plot(X[1:50,0],X[1:50,1],'x',X[50:100,0],X[50:100,1],'o',\n", " X[100:150,0],X[100:150,1],'^')\n", "plt.title('Scatter plot of the three classes, in the first two dimensions')\n", "plt.xlabel('Dimension 0')\n", "plt.ylabel('Dimension 1')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Always Initialize a GMM from a Gaussian\n", "Remember from last time that the pdf for a GMM is\n", "$$p_{X|Y}(\\vec{x}|y)=\\sum_{k=1}^K c_{yk}{\\mathcal N}(\\vec{x};\\vec\\mu_{yk},\\Sigma_{yk})$$\n", "\n", "Finding the parameters $\\vec\\mu_{yk}$ and $\\Sigma_{yk}$ is a little tricky. You'll almost always get the best results if you first find the mean and covariance of each Gaussian." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Compute the mean of each class. axis=0 means to compute the average row vector \n", "mu=np.empty((3,2))\n", "for y in range(0,3):\n", " mu[y,:] = np.mean(X[50*y:50*(y+1),0:2],axis=0)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Compute the covariance of each class.\n", "Sigma = np.empty((3,2,2))\n", "for y in range(0,3):\n", " Sigma[y,:,:] = np.cov(X[50*y:50*(y+1),0:2],rowvar=False)\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Create a coordinate system on which we can calculate the Gaussian pdf\n", "coords = np.mgrid[4:8:0.01,2:4.5:0.01].transpose((1,2,0))" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdQAAAEICAYAAADm2wU4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXucHFWd9/8+3dMzPdOZeyZzyeRCEggYhURQkHVNNqAI\nAt7W3+q6qygrqyv7sI+4uCq662X3cb3rw7osK7+Hdb093kBhAUWUsIqghIRAIJiEXEjmkslkLpme\ne/d5/qiqnuru6u7q7uru6pnv+/Wa13RXnTr1rdNV51Pfc/kepbVGEARBEITiCFTaAEEQBEFYDIig\nCoIgCIIHiKAKgiAIggeIoAqCIAiCB4igCoIgCIIHiKAKgiAIggeIoNpQSv2DUuqbZTrXHyil9iul\nJpRSb3CRfq1SSiulasphn1copfYqpbZV2o58UUq9Tyk1aP4+7S7SX6OU+lU5bKsWzLJbV2k7cqGU\nulUp9bEs+8tWL/gVpdQ2pdSxStuRL/nYrZR6u1LqZ1n2P6SU+otsebgSVKXUnyqlHjcfkH6l1H1K\nqVe6OTZHvncopT5dbD6VQCl1WCl1aRFZfBK4RWu9TGt9Vwny9wVa601a64cqbUc+KKVCwBeB15i/\nz3DKft+93CilapVSH1dKPaeUiiqljpvP6WsqZZNZds9X6vxu0Vq/V2v9KfCncDjVk36uH6q1Xtda\nf0trXdTzklNQlVIfAL4M/BPQCawG/gW4upgTlxo/VXYZWAPsrbQRgiOdQJgS/T7KwOvWoR8Arwfe\nAbQCZwBfAV7n8XmEKqdE95/vqIgGaK0z/gHNwATwlixp6jAEt8/8+zJQZ+7bBhwDbgROAP3Au8x9\n1wFzwKx5jrvN7ecADwGjGBXa1bZzPQT8he37NcCvbN818H5gP3DIwda1ZprrTFv7gQ/a9v8D8E3b\n96tNG0bNc59jbv9PIA5MmbbflKFs3gMcAE4BPwF6zO0HU46vSzkuLX+b7e8EjgIngY/ajgkAf2fm\nPQx8D2jLYJf1u9xk+13eAFwB/N609yO29C8HfmOWQz9wC1Br7rvYtGWV+f08YAQ42/x+GLjUVr7f\nB74JnAaeAs4CPmza8QKGV0jqsam/j6083mUeNwK8F3gZsMe09ZZ871vTnqiZ9wTwC4djj9r2TwCv\nwLwXgc+bthwCLk+5d/8R+LX5u27AeL5uN8v0OPBpIGg75t3As2Z+PwXWZLiWS808e3M8z9b9cRp4\nBnhjlnvfKt8a27P2vHnsIeDt5vYNwA5gzLwP/m/K87jB/Pw6YBcwbv5e/+Bwrkz39suBx81jB4Ev\nZrtO85iwWSbLze8fBeaBJvP7p4Avm5/vMMs+Yh4Tt/22PWbZfA/4hnn9e4ELMpxXAV/CuJ/HMe7x\nF5v76oEvAEfM8voVUG/u+z4wYG5/GNiUqZ4kQ/0DXAQ8gnHvPwlsy3b/Odh+GONZfAbjnvs/QNhe\nZ9jSOtbTTvZmKKeLgd+Z1/s74GJz+58Aj6ek/Z/AT2zP7efN+2QQuNVWhtsw6rUPmWX5nw52Z3sG\nriFZT14N7DNtvAXjPv8Lp+tJHJPjpnwtxk1YkyXNJ4FHgRVAh/mDfsp2gfNmmhBGhT0JtNpvZFte\nIQwB+ghQC2w3L3yj7abIJagPAG1WIafYutZM8x2Mh+clwBDJFb5VYVsV66tNu24ybbOE5DC2yt7h\nXNsxKoaXmjfB/wYeTrl5sx2ftN9m+79jPJjnATMsiPwN5u/Qa57v34DvZMjb+l0+bl7be8xy+DbQ\nCGzCeOjOMNOfj/Gw1ph2PAv8jS2/fwR+Ydr1FHC903WY5TsNXGbm9Q2MyvmjNjsOZSkD++9jlcet\nGJXna8y878K4F1diVGpbC7hvrbwd73un/Rj34px5DUHgfRhCrWz37lGzbGvM673T/J0iph2/Bf7S\nTP96jPvtHDP9zcAjGez5DPBQtmfZTPcWDIEIYFRcUaA7tWxTr9G0b5yF57CbhQr/O+bvFzB/h1em\nPI+WoG7DeN4CwLkYleEbXN7bvwH+3Py8DLgo17WaaR8G3mx+/hlGRXq5bd8bU+shUirglPv2CvO3\n/V/AoxnOeRmwE2jBENdzbGX8L+Z9sNLM52IWnI93Yzx71oveblueCfuyPBsrMV6krzDL+NXm945M\n91+GOudpYBVGHfprp3Ihdz2dZm/KedowBPvPTVveZn5vBxrMvM60pf8d8Fbz85cwnJM2s7zuBv5X\nSr32z2Y51qf+nmR/Bq7B1BNguWnHH5vX+z/NvIsS1LcDAznSHASuSLmhDtsucIrkiucE5gORWvDA\nH2K8WQRs276D+TaLO0HdnsXWtWaas23bPgvcnlqpAB8DvmdLF8DwIrY53dAO57od+Kzt+zKMCnet\ny+OT9tts77Vt+63tRnsWuMS2r9s8X5oo2H6XoPm90cz7QluanZgVnsPxfwPcafseMtM/BdyPKSKp\n12GW7wO2fVdhvMWm2tGSoQzsv49VHitt+4eBP7F9/yE24c/jvrXyzldQD9i+N5hpumz37idt+zsx\nRKPetu1twC/Nz/cB16bcf5M4eKnA14Hv2r63YXgOY8B0lntsN/D61LJNvUYMQR0F3kzKiyrGS9Ft\nOHjH2ATVYd+XgS+5vLcfBj6B6W26/cPwQr9qXsMAxkvnZ1jwXtvNdHeQW1B/bvv+ImAqwzm3Y7Ty\nXERyPRYwz3meC7tbzPJoTrXP6bkyv38I+M+UND8F3ul0/2U472HgvbbvVwAHU8uF3PV0mr0p5/lz\n4Lcp234DXGN+/ibwcfPzmRjC1oDxghIF1tuOewXmS7hp4yymV53p98zyDFzDgqC+A9tLk3nuY+QQ\n1Fzt6MPA8hxt0T0YTRgWR8xtiTy01vO275MY4pIprxe01vGU/FbmsNPOC3mmSbXXbkviukybXsjD\nltTjJzDKM59rcWLA9tlelmuAO5VSo0qpUQyBjWFU3E4Ma61j5ucp8/+gbf+UlbdS6iyl1D1KqQGl\n1DhGf/pyK6HWeg7jIXox8AVt3oEZSD3HSQc7Mt0fbvJzvAYHct23hZD4bbTWk+ZH+/nt990ajBeR\nfttv9m8Ynqq1/yu2facwHmqn+2cY4wXKOvcprXULRstCnbVdKfUOpdRuW54vxvY7ZkJrHcV4m3+v\nae9/KaXONnffZNr1W3NE97ud8lBKXaiU+qVSakgpNWbmlXruTPf2tRgtRvuUUr9TSl2Zy2aTHRgV\n6ksxXvYeALZiiN0BnTLYLAeptoWd6kWt9S8wmgf/BTihlLpNKdWEca1hjBe5JJRSQaXUZ5RSB83n\n67C5K+dvY2MN8BbrtzV/31diuy/wtm4spp5OffZSj/82xsslwJ8Cd5nPUweGsO60XeP95naLIa31\ndKYT5/EM9GArC7NOy1l+uQT1Nxhv0dmmdfRh/JgWq81tbkitePuAVSkd5qsxPEMw3k4abPu6XOTp\nxKqU/J3sTboupZQyj7NsyXWe1OMjGE0axzMekYyb67DzAkZzVovtL6y1dnu+bPwrRl/CmVrrJoym\nHmXtVEqtBP4eo8/lC0qpOsdc8sfN710oXt63brEf9wLGs7Xc9ns1aa032fb/ZcrvWa+1fsQh3weB\nlymlejOdWCm1BqNJ9XoMz6wFo3nP+h2zlrXW+qda61djVND7zLzQWg9ord+jte4B/hL4mlJqg4MJ\n38ZoqlultW7GaKpXDunS0Frv11q/DeNl45+BH5jPUy4eATYCbwR2aK2fwfidr8AQW8fTubEph71f\n1Vqfj+HJngX8LUb3zzSw3uGQP8Vo4r8Uo199rbndKh8nm1K3vYDhodrvl4jW+jNZjnHCbd2YrZ7O\nq250OP4BoEMptRlDWL9tbj+J8ZK8yXaNzVpr+0trxnO7eAbs9GMrC1v9n5Wsgqq1HsPoZ/sXpdQb\nlFINSqmQUupypdRnzWTfAW5WSnUopZab6d3O2RoE7PPUHsN4+7vJPM82jGbB75r7dwNvMu3YgPHm\nWggfM/PYhDGo5f86pPke8Dql1CXmNIobMSpAq0JLtT2V7wDvUkptNgXmn4DHtNaHXdqYK/9UbgX+\n0bxpMH+P1+dxfDYaMfrQJkzP5H3WDvNGuwOjiftajBvxUx6ddzfwVvNeuACjP8MrirlvhzAGhRQ8\nx1Jr3Y/Rr/cFpVSTUiqglFqvlNpqJrkV+LB5j6KUalZKvSVDXj8DfgncZXqCteY9e5EtWQSjshky\n83sXxtu5xW7gVUqp1UqpZozBKZhpO5VSrzdFbAajmT5u7nuLTchHzHPYPReLRuCU1npaKfVyDBFx\nhVLqz5RSHaZHNGputs5/WCl1jdNxplezE2OgoiWgj2B4x5kEdRBoN8sgb5RSLzN/gxDGS8o0EDdt\n//+BLyqlekyv9BVm3dCIUa7DGC81/+RgU+q9lrrtm8BVSqnLzLzDypgClPElKwPvV0r1KqXaMPrG\nnerGXPV0rrrrXuAsZUzHrFFK/QnGy8c9kGjx+j7wOYzuiwfM7XEMQfySUmoFGC/zSqnLXF5brmfA\nzn8Bm5RSbzJbIv4HLl7ocw6d1lp/AfgAxqCIIYw3oesxBn+AMTrucYyRlU8BT5jb3HA78CLT/b5L\naz2L8cNcjvE28jXgHVrrfWb6L2G0kQ8C/wF8y+V5UtmB0an+IPB5s0JKQmv9HPBnGIOJTpp2XWXa\nCMbAhJtN2z/ocPzPMfphf4ghMuuBt+ZhY9b8HfgKhgfwM6XUaYwBNxfmcb5sfBCjAjyNcUPbH7L/\ngeE5fMxsFnkXxovEH3pw3o9hlNsIRh/at7Mnz4uC71uzov5H4Nfm73NRrmMy8A6MQR3WqMofYDbR\naa3vxPDGvms2Az6N8Vxk4o0YFdI3MUTnEMYYiMvM/J7BGGH6G4zn5yUYg06sa3oA43fdgyFC99jy\nDmDUAX0YTc9bWXipehnwmFJqAuP+u0E7zz39K+CT5r35cYwXVre8FthrnuMrGH2rU0qpWoxWn0ez\nHLsDo2n9t7bvjRj9smmYdc13gOfN3zbfboAmjGdkBKMZcxhDGMB4jp7CGGRzCuP3DWD0Qx/B8NCe\ncbiepHrS3JZUP2itX8Dwcj/CQj39t+QfvOfbGC96z2M0T6c9Ey7qaSd77ccPA1diOCnDGN0GV2qt\nT6bYcSnw/ZQuww9h1N2Pms/FzzFaIXKS6xlISXsSYwDTZ0wbz8yU1o41AnFJoJRai1HRhFJ+JEEQ\nqgxlBJd5v9kcLBSJUuowxqCbn1falmrF78EPBEEQHNFa/wpjLqcg+IJFHy1DEARBEMrBkmryFQRB\nEIRSIR6qIAiCIHiA9KHaqA3U6/pgY6XNEARBqCrG54dOaq07cqdc3Iig2qgPNnLxcsepfoIgCEIG\n7h/4WmrkoyWJNPkKgiAIggeIoAqCIAiCB4igCoIgCIIHiKAKQoEMb1pHtKstaVu0q43hTQWH+BUE\noYoRQRWEAgkPj3J865aEqEa72ji+dQvh4dEcRwqCsBiRUb6CUCCRgVOs3LGL41u30PrcUUY2rmbl\njl1EBk5V2jRBECqAeKiCUASRgVO0PneUk+edSetzR0VMBWEJI4IqCEUQ7WpjZONqlj+5n5GNq9P6\nVAVBWDqIoApCgVh9pit37KJj9/5E86+IqiAsTURQBaFApttbkvpMrT7V6faWClsmCEIlkEFJglAg\n7XufT9sWGTgl/aiCsEQRD1UQBEEQPEAEVRAEQRA8QARVEARBEDxABFUQBEEQPEAEVRAEQRA8QARV\nEARBEDxABFUQBEEQPEAEVRAEQRA8QARVEARBEDxABFUQBEEQPEAEVViSDG9alxbEPtrVxvCmdRWy\nSBCEakcEVViShIdHk1aGsVaOCQ+PVtgyQRCqFQmOLyxJrJVhjm/dQutzRxnZuDpp5RhBEIR8EQ9V\nWLJEBk7R+txRTp53Jq3PHRUxFQShKERQhSVLtKuNkY2rWf7kfkY2rpaFwQVBKApp8hWWDLNn9yY+\nTzVFGNiwhq4DR6ifnqL20DGObb/A+D4eTTu2dt+xcpoqCEIVIoIqLGrsImpnOtKQJJ7141G6Dhxh\nOtLgKKhWPiKsgiBkQgRVWHRkElE7rf1Dadvqx6OOYpqat4iqIAhOSB+q4DsKnSM6e3avKzEtlnKc\nQxCE6kMEVfAd+c4RLZeQpp5TEATBjjT5Cr7D7RzRSouaNP8KgmBHPFTBl2SbI1oJjzQTfrFDEITK\nIx6q4EtS54g2DAwTammotFmCIAgZEUEVfIfVZ2o189bWBrLOERUEQfAD0uQr+I7p9paEmM6e3Zs0\nR9SPSLOvIAggHqrgQ9r3Pm/0k9qaeN3MERUEQagkIqiCrxBvTxCEaqVqm3yVUkGl1C6l1D0O+5RS\n6qtKqQNKqT1KqZdWwkYhP0olpmPr65L+yoUsYi4IS4tq9lBvAJ4Fmhz2XQ6caf5dCPyr+V/wIV4J\nqVuxTE3XfHDGk/OnYgWosPqD7YOtBEFYfFSlh6qU6gVeB3w9Q5LXA9/QBo8CLUqp7rIZKLimWDH1\nwvP0wmt1ug57gIqhzWcmiasgCIuPavVQvwzcBDRm2L8SeMH2/Zi5rT81oVLqOuA6gHBgmbdWClkp\nRky9brq18vPaW7UHqFj+5H4RU0FYxFSdh6qUuhI4obXe6UV+WuvbtNYXaK0vqA3Ue5GlkINCIx2V\nox/U67xlEXNBWDpUo4f6B8DVSqkrgDDQpJT6ptb6z2xpjgOrbN97zW1ChSlUSKuR1AAVDQPD0uwr\nCIuYqvNQtdYf1lr3aq3XAm8FfpEipgA/Ad5hjva9CBjTWqc19wrlpVrE1Ktz2gNUwEKf6nR7iyf5\nC4LgL6rRQ3VEKfVeAK31rcC9wBXAAWASeFcFTRPIX0yr1Su10773+bRtkYFT4p0KwiKlqgVVa/0Q\n8JD5+Vbbdg28vzJWCXaWipDKMm6CIFRdk69QPVRSTMeaVjIZTm5anQy3MNa0suy2CIKwNKhqD1Xw\nL/mIaaHidXqNyrgvPjvJZO0KZloVrf0jTIZbmIysoCF6oqBzCYIg5EIEVfCcUolpNgFNpaZ2FIDY\nXAfDK5ah4/U0TJygYXrUdR6CIAj5IIIqeIpbMS2VkNqpqR0lHluGjjegApPEOsfgSEFZCYIg5EQE\nVfCEUnilhQqpxfxsCzpejwpMouP1zM+2cHrNKI1HdFH5CoIgOCGCKhSN12JarJCCIaaxuQ6CoSFq\nakcT3438RVQFQfAeEVShKLxs4vVCSC3isYaEmMJCn2o81gB4248qU2YEQQARVKEI/CqmALX1fWnb\nDFGVQUmCIJQGmYcqFIRXYnp6jfJcTN1QiXMKgrC4EQ9VyJtixPRUy1pCc1Eao0MJUZub7iAeb6Cu\nIfMQ3JnVs67OWXe01lU6Lxjp7iAcncR+xmhXG9PtLY5hBwWhEsQaZxh+2z7av30OwYn8n49ij19K\niIcq5IUbMc22xFpoLspMfSunOlcAppjGWggEJtPSzqyeTfy5JZ+0xRKOTjJ4Rm9iSTZrdZnwsDQr\nC/5hfPtRZteOMX5JYXPGij1+KSGCKrjGrZhmZflJAsFR4rEWZqJnGGIaHCUUHkokyVdEUymXqNaP\nR1m5YxfHt25haPOZsjSb4DtijTNELxiEAETPHyS2LL9no9jjlxoiqIIrvBBTq4nXEM95IATMJ8S0\nWCG1U2w+zQdnXKWLDJyi9bmjnDzvTFqfOypiKviK8e1HQZlTxAI6by+z2OOXGiKoQk6KFdPUgUdz\n0x0Y3fdzQA0zofayNtV6Re2+Y0S72hjZuJrlT+5nZOPqRPOvIFSahHdZYwpijc7Lyyz2+KWICKqQ\nFS/E1E6izzQ4Cuc8By2nYLQN+ruLtrXcWH2mK3fsomP3/kTzr4iq4AeSvEuLPLzMYo9fioigChnx\nWkwB4vEGow/1rOPGhu5+Q1Sjywq2MxOl9nqn21uS+kwjA6dYuWMX0+0tOY4UhNIzu2Z8wbu0qNHG\n9jIcvxSRaTOCI6UQUwDO3k88dVt3f9bzrO01+lgPH+vIaZMXuOk/rd13jHaH7ZGBU9KPKviCzq+e\nX9HjlyIiqEIaxYhptoAJM6tnYXg5hKcgEl3YEY3AdD20n0yIpxPWvrHDqxmZCWTMA1g4D3OJJPFY\nPfFYmJraEYnlKwiC50iTr5BEScUUDJE7vsoQQDD+H19F56r+rGJqp7ZpnED/yrQ8DAEl6TzxWD1g\niOncdDeB4LSrc2Q9v8TuFQTBAfFQhQSlENO0fsxIFFa+YAhg6ykCYy10bH6K+jb3wRDq20bpOHcv\nQ3s2EW8ehZE2I0+7x2qeZ+7oKoKhMWJzzYTC/QSCU5kzFgRBKALxUAXXFC2mFpEozauPwckVNPb2\n5SWmFvVtozT29sHJFdB6KllMbecxxLSdYGjMlZi6nX8qCIKQigiqAOT2Tj0TU6CzYY7Tx3poPuMw\np4/1MHUq/1GxU6daEnkw0rbQ/GsnGiE210wwNExsrjnR/FsM0twrCEImpMlXKJuYru0dYupUC0N7\nNtFx7l7q20YJt40kfXdDah5jMW00IdubfaMROLoq0cwbCE4yN91NKNxP87H0uMEg3qkgCMUhHuoS\np5xiCjA73pQknlZ/6Ox4k1uT0/JI9MtO2zzQ6fqkPtNAcIpQuJ94LOz6PKmIdyoIQjbEQ13ClFtM\nAZrXHk3bX982mlc/qlMeRKLJ/ajtJwlEk5eaMjxVGZQkCEJpEEFdopRDTN1Og3l11z7H7Q8MnO3q\n+HzINP80V3OveKeCIORCmnwXCcOb1qXFkI12tTG8aV1a2pKK6fByiEaSxHTqVAtjh1cnHfPqrn2J\nv0xk25eLci40LixNYo0znLjuSQkWLyQQQV0khIdHkwKze73YtWvPNDxFoH9lYuSuNYCotmkh/mc+\nQlmMqKYi3qngJbLwtpCKNPkuEqzA7Me3bqH1uaOMbFztuNh1Id5pXs28Gw8z1THK0J5NNPb2cfpY\nT2IAkZfiaFGu+L6CYCd14e2mB9cQnJBWkaWOeKiLiFyLXRfa1OtEtj5TK+jC2KG1icANpRDTTOTT\n3CveqVAIsvC24IQI6iIi22LXXvab5hqAZA+6MNPXySYqv9yTBMMXvEIW3hYyIYK6SMi22HW5xdQK\nuvD/vfI+LnnVz3jw4dfQN9CT5xWVHvFOhUKQhbeFTIigLhIKXezaSzGFhaALV7/oUQB6uvq45FU/\nY2h4havryAen/lOn5l7xTgUvkYW3hUzIoKRFQvve59O2RQZOEWppyDuvQsUUjKALqf2lPV199HT1\n5W1HKRHvVCgUWXhbyIR4qIsYr0b05hO0oVyDj8Q7FQTBb1SloCqlwkqp3yqlnlRK7VVKfcIhzTal\n1JhSarf59/FK2FopvBrRm6+Y3v/gFex55tyk7XueOZf7H7wCIOd+gCf3bk70uVrRkpyCQxRK88EZ\nRro7mGpKXqFmqinCSHeHeKeCIBREVQoqMANs11qfB2wGXquUusgh3X9rrTebf58sr4n+Jd9ISHZy\neaY93cd4bOfFCdHc88y5PLbzYnq6j7naD9DRfiJpIJNTcIhU8vVOw9FJBjasSYjqVFOEgQ1rCEed\nV6IRBEHIRVX2oWqtNTBhfg2Zf9K2Z5LLO3XCbVOvE/Zm3nNftAeAx3ZezJEXzmDgRDcXnv9IYnuu\n/bAwkOnBh19DXc9gUnAIKC6Yg9V3Wj8epevAEQY2rKF5cJixzna6Dhyh+bfPFZy3IAhLm6oUVACl\nVBDYCWwA/kVr/ZhDsouVUnuA48AHtdZ7HfK5DrgOIBxYVkKLy0O5+02d+kzPfdEeUyx76FrRx7kv\n2sMfNz2xkOBFpO1Ppaerj7qeQcYOraX5jMNZV6MptO+0fjxK8+AwI72dtB4bFDEVBKEoqlZQtdYx\nYLNSqgW4Uyn1Yq3107YkTwCrtdYTSqkrgLuAMx3yuQ24DaA5tKKqvdxy95tmGoC055lzGTjRTdeK\nPgZPdKEOtsEW5/0DJ7rZ88y5aaLaN9CTCA5x+lgP4bYR6ttGPfFOLaaaIox1ttN6bJCxznYau9rS\noksJ1UUhrTPFIn3ugkXVCqqF1npUKfVL4LXA07bt47bP9yqlvqaUWq61PlkJOyuN1/2m2cT0sZ0X\nc+H5j/DJi+7gh7su5d8feQsAb97yc36461J+u/MViWZeKz0sNAf3DfRw30OXJ5p5w20jiWARqbj1\nTp3EdGDDGroOHKF+PErj3kOJwBgiqv6lEoKZi0w2idAuPapSUJVSHcCcKab1wKuBf05J0wUMaq21\nUurlGAOwhstvbXkoVb+p2zVNLfr6exNiCoaIAuw6dg5v3vJzdh07h5ef/5u0PtW+/t7E56HhFUl9\npvVto3Scu5fBF7qhPfv7kNtpMtORhoSYQnIgDBHUyuNH4cyX1GsQgV38KGN8T3WhlDoX+A8giCGU\n39Naf1Ip9V4ArfWtSqnrgfcB88AU8AGt9SPZ8m0OrdAXL39LaY0vAX7oN7WT1F/qwA/GX5p1v9PC\n4sXMO5UgDv5mMYhnviy2e+7+ga/t1FpfUGk7Kk1Veqha6z0k9coltt9q+3wLcEs57fIj5eo3tcgl\npl7hVRCHxVax+Z2lKJ5O2MtB7sHFQ7XOQxVMrAczU6CCyfq2tGMy9pseXQPD7cCCmI4d6WXwCWPO\nqBsxvfnuv+aHuy5N2v7DXZdy891/DcDHfvPutED5fQM9PLl3M2B4p2OHVycWKAfTO41GYHh5xnOn\niulY00omwy1J3uloVzt9Z63Neg1+ZNv6Q9zxp3fyX9d9kzv+9E62rT9UUD6xxhlOXPdkxlVRcu0v\nhNmze5P+hHSkbBYPIqhVjP0hdApU0L9xDTXzU0nHZG3qjUzAia6EqI4d6WXk9xsIt7vvU9zS+yz/\n/shbEqJqDUra0vsskB60oW+ghwcffg0d7ScSTb21TeMM7dm0IKrRCBxfBWHjWtysdxqajTIZWcFo\nl3Eto13tDK/uoWH8dCJNNXgG29Yf4oatj9LZGCWgoLMxyg1bHy1IVMe3H2V27VjGVVFy7XeLCGhh\nSJlVP1XZ5CukkxqoYLSnncbTfdTOTeU+2KLdHLN1oov+39UzM9pM61kHaF5zzHVTrzUI6d8feQuP\nHNrC3v72teszAAAgAElEQVQNvOfi7/PmLT/nB+MvTQracM5Ze3n295u45FU/o6erj70DTca1mIOQ\nhvZsIt48CiNtsPIFiERdN/U2TI9SPzzP8Ooeoq3NTDdGaD/aR8uAcY3VIKYA11y4m3AolrQtHIpx\nzYW7eejgGa7zSazhGYDo+YM0PbiG4ESt6/3ZEBHwFqs8q+UeFRYQD7VKcarE7IEKwtOjaWLqalTv\nefuoaxljZrSFupYxmtfk/1C/ecvP2dR9gKf7z2JT94GEmFr0dPVxzll72fXUBZxz1l56uvrSBiLV\nt40aYnpyBbSegkjU8VyZ+k2bD87QMjBM+HSU6aZlhE9HE2JaTXQsc77uTNszkbSGp8Panbn2pyJe\naOmRsq0+RFCrkEwPmhWooH7yJNPhFmZD9VnzcRqINHakl5nRZupaRpkZbWbsSG/eA5F+uOtS9vZv\n4MXdv2dv/wY+/ug1Sfv7Bnp49veb2PKSx3n295v4yTPpYZgPP7fW8EyXnzD+RyOumnphYVTvaFc7\n040RwuMTTDdGEs2/1fTmPzQRyWu7Ewnv01rDs0YTPX8w0Veaa7+FiGj5kfKuLkRQFwlWoIJlk31E\nJodpPN3H6caehKi6CeDQGqtj5PcbaD3rAN0v20XrWQcY+f2GtNVhsmH1mb7n4u/zhTd9jpef/5uk\nYPhWn+klr/oZF2z+nbH4uL2/FCMYPsdXGc28HSeM/0dXEY8lvyBkG9Vr9Zm2H+1j5bPP0360j+HV\nPZwOVFcvxx2PbWZ6Lpi0bXouyB2PbXadR5L3aWHzQrPtFxH1B1L+1YEIapWR6cGyAhVYzby1c1M0\nnu5jvqbe9ZzT6eG2RJ8pQPOaY1x4/iP09bt/mHcdOyepz/TcF+1JymNoeEWizxRgL010nLuX2fGm\nRB6DL3Qn+kwBiEQJhfuJx8KJNNmaegEmmxqT+kxbBoZZ8fizRLszjxT2Iw8dPIOv7LiIwdMR4hoG\nT0f4yo6L8uo/nV0zvuB9WtRoY3uW/dNnTRdrvuAhIqr+pyoDO5SKagjskO2hyie8oNuISG4XDLc3\n++YK3GAhARz8hVTY1YEf72MJ7GBQXe1fS5xKLsuWC0tEn9y7mY72EwkPFIxm3qHhFZy3aXdi/14W\nPNKpUy3MjjcxUpM+IlnEtLRY99T2jr284ay7+VxvHTe9MMOd+1/FL4Y2VcSmudYRRi//OS33XUpo\npLUiNviZ2bN75X72KdLkWyV4tZKME/nG681Gtnmm1v77Hro80WdqLR4+MpN+K0o0pNJh7xfd3rGX\nGzfex486AzwRruNHXQFu3Hgf2zvSFyMoB+Ov+jU6NMf4q7JGChUE3yEe6iKmXN6pnWzzTGGhz3Ro\nzyYae/sSi4cPToaS8slHTHN5p4JBppeya9ft4HRtnB8vi6CV4q5lEd47Osa163aU3Uudax0h1jIO\nCmItY4y8ZILAZHvJz1tt95B4qf5EBLUKqBbv1MI+z3TLSx5PiKnVZ1rfNkpjb19i8fBUMXWiUDFd\nypWO2y6CFXXj/GNLK3GMF7A4iltbmvnI/EgpzUvCuodntiR7pXMbd1C3601lO7+dahNZofKIoFY5\nxQxEKhWp80x7uo6n9Zlai4ePHe1NHtGLu9CCIGJqp5gBRc/GmvnxsghzAeOemQsYXuqbBuJemZdE\npns23jAMkTGwbl0FRMaINwyXxUtNxW6nH8VVvFT/IYLqc6pt5KV9nmlPVx89XceTFgu3+kwTzbwr\nX1iYc5pHaMFcLPaKxsv74ubQOuIkrzMbR/HR2nWe5O+2BWVu446M28vhpWbDugY/CqvgH0RQq5hi\nvdNSNPdmm2da3zbK7HhTcp9pJGqI6XQ9dcNzaflJv+kCpXq5OtQYJxZIvmfmAopDjXEK8QsL7oKo\nn1jwTi2Uud0n+E1YxUv1FyKoPqbavFOA8zbtTny295nWt40C0Lz2aPpc00jUUzFdLBVMuX7/9rsv\nLzqPYvrxLeoeeUfReZSLsfV1vhFVwT+IoFYpfuw7tXAK2GBRTOAGWPxiWk0vUV6IaDXjN29VqDwi\nqD5heNM6wsOjRAaMtUdnz+5lqinCdKSB1v7cTbOT9W3m2qcL4eLisXozXN+gseHoGmPN0/aFVVfG\njvQyPdxG50v3eHIdP3j0NdQ2jSc8UvAmcAMsTjHdtv4Q73zlU6yoG+fETBO3P7+1YgEVsmEEfniY\nz/XW8bf9s3z71HZ+OvWStHQ6NMnc2Q8R2rcNNddQEltynSPX/svqn+L65gfpDI4xGGvmlrFLHK/F\nrR2joW207AvmPkBY9EhgB58QHh7l+NYtRLvaEmI6sGEN4eikq+Nr5qcYb+5JBJCPx+qZm+4mELTF\nY7UtIL62d6igBcQz8cDA2TwwcHba4uBeBW5YjGL6yj8c44Y/eoyu8DgBBV3h8YoGVMjE9o69fODs\n+xOBH+5crri59W4uq38qLe386t3o5kHmV+92yMkbcp0j2/7L6p/i5ta76a4ZI6Cgu2Ys47XkY8dS\n99YFAxFUnxAZOMXKHbs4vnULp1Z2MrBhDV0HjlA/nr7updPDWzs3RSjcz9x0N/Oz7cxNdxvfzxhb\nSNQ+DCsGzAXEtyRWlnFa8zRbs21qOnta++LgIwfOMBYJ7z6etp6p26kxsLjE1L56y7XrdhAOzift\nDwfnuXad82jXcjO2vo6x9XW8+6yHmQjFkgI/REMxrm9+MCm9Dk0S7zwACuKdB9Ahdy+D+ZDrHLn2\nX9/8IPWB5P76+sBc2rUUa4ewNBFB9RGRgVO0PneUkd5OmgeHHcU0G4HgFMHQGLG5doKhMQLB9CZW\n2oehftLVAuLZRDVVSO3YAzfEm0ddi2kh02OqBacl0FbUjTumzbS9XFhCatEZHOPWlua0wA+dwbGk\n4+ZX715YBk7pknipuc6Ra3+qzbm252OHeKmC9KH6iGhXG6dedAatxwaNhcJPT6SJarbBSPFYPbG5\nZoKhYWJzzQSCk0DKgKThdphqSFpAvFBRzcTUqRYjYIO1OHhDNCGq+YppNXunuQYYnZhpoiucLp4n\nZpocUpeeTPfWM7Q4Bn5448mF3yzhsQXMYBCBuOG5Hd3sWV9qrnO4sWEw1kx3Tbp4DsaaPbEDpC91\nKSMeqk+IdrVxfOsWug4coe34IF0HjjCwYQ1TTRFXx1t9pqFwPzW1w4nmX6K244fbjT7UFQNJC4iP\nHfFuZOnUqRYGd78keXHw46sgGlkyYup2Qe7bn9/KdCz5nXY6VsPtz2/1xA7L03T7l4kPL1uf8E4t\n4ij+btn6xPckj83CYy811znc2HDL2CVMxZNDXU7FQ9wydokndoiXurQRD9UnTLe30HnoWMIjrR+P\n0nXgCNORhsS2bA9rPBYmFO5PNPMGglafav1Ck2t0mdGHao7ytTzT6eG2rF5qPjgtDs7KFwj2L4Pa\n9Niwi0lM853yYo3mvXbdjqJG+Za6Ej/SEEM7BH440hDDekXSTUMLHptFIG5s94hc53BjgzWat5hR\nvuW4VqE6kQXGbVR6gfFCg+A7zT2F7PNPvY6S5DS/1GKxe6blnju6mLygTPduPvit773c81L98DzI\nAuMG4qH6hHJXyoePdXgmqktVTMv1m4mA5pen3wS2lPjpeRBEUKuGXJXq/GwrgeB08sjeaASm66H9\nZOYD7ec4vDpjUIbmtUcBOPbrlxNuG2H5OfsXhLS/22hO3rA/cVy2aTHVLqblEFI/imghQRvyFdCP\nxe7hkuAuPtzRzmeGhvl5bAufCl7p+vgr43u4+oxf8nfdET5/LMq9Q39UUNAGQSgEGZTkA7yooAPB\naeamu5MCO3B8FYQdps6YpHqWmYIy1DYtjEQNt40wcWwlh393rrGhvxtG24ygESbZvNJqDnbvdrBR\nobgZIFRJ3AZtOL1GJf7y4WOxe3g7j/P1liaeCNfx9ZYm3s7jfCx2j6vjr4zv4dP6br7UUcdEQPG5\nrrqCgzYIQiGIoFY5VqW1MAhpIbBD6jqjTthF1Skog7Xs2uFjHRw+1sFE4yi0nDJEdP9Zxv+WU9Dd\nDxQ2x9SNmFbSOy2lkPpdRC1yBTIoVETtvJWdnAwGkgJIDAcDvJWdro7/gH6QIyHNwdoQKMXB2hAv\n1Om8gzZUC35psREWkCbfKsBtZZsc2GGYWA4xtUjrA20eZezQWlh+wlhmbTJlf3c/TDTCfC3UzEJ3\nf0FNvOBvMS21N1pNOAUyCB282NM+0SCaW1ta0gJIfHQ4fXS4E92M8f6OrqRtN3Us50fHBzyzURCy\nIR5qhfGy0k4N7JA0B9Ut0YgRjMEKyuCUR383zIcMMZ0PEfj9yozZVaOYlsojrRZvNBWnQAaxrgOc\nPiNzd0IhDASDjgEkBoPugiX8d6g14Z0CCS/117WtntqZjXJ1XYh36k9EUBcJToEdOLoqP1GNRox+\nV4egDAmsPtOWU3Dm7wkER4nHWpibTh/pW61i6jXVKKJ2MgUyCDZ7G1rwphbnABI3tazPcEQyH+pw\nHm3+txm2C4LXSJPvIsFVYIdcTNc7BmUgNThEyynq5obhaC2Eh5ibhng8edRntYlpqYS01HjZ5Jrp\nN3MKZKBUHFV3wrNzAzwRbiAQmE7aNhdQ7Ay7G1E8EZpDqZTyUIqJ0Bxhr4z0AeKd+hcR1AriphJ3\nWynXOEQhCgSnoD2PoN9O02si0SRBrqs9AsmLdRAKL8xnzTUH0G9iWm1CWop5nJnybjyijW2n3gDF\nr/CXk/mBNxR1/Nyxd2bcF6b0c1OrYaS6UFqqUlCVUmHgYaAO4xp+oLX++5Q0CvgKcAUwCVyjtX6i\n3LZC+uLhYMTunejuSCwePtLdQTg6mRQMf6opwmR9Ew1ThddmdUdrs0ZMSsJhAXKG2yG6jDqMUbxO\n812thcxb+08lFjqvnVvYPxuqZ76mnu6n+3Oa4KWYblt/iGsu3E3HsihDExHueGwzDx08Y8GuHGJ6\n/fqfclXPboJKE9OKu/s2c8vBy5LSbO/YmwgdWEgYu8vqn+Jtbb/g8921jgt3J0QuMEnN8l/CyT+C\nlNaAK+N7eGfgF3yxo5Ybh2a5I76dewLnurYhEwWJd+1xQp0/Y27wNTCb3Ld+ZXwPH9AP0s0Y/TTz\nRXVJmp2ursUsi3mHsvAKN3Nuy7GYeiqpz0escYbht+2j/dvnEJxwHhjoJo3gDdXahzoDbNdanwds\nBl6rlLooJc3lwJnm33XAv5bXxAXsi4fDQiB8++Lh4ehkUjB8a4HxmvniB364XnvUtgA5kAimH5xd\nsMFpvuvcdDeRk0aamvkpTjf2MBsy9s+G6plo6KHlaO4lybwW0xu2PkpnY5SAgs7GKDdsfZRt6w+5\nGnR0/fqf8oaVu6gJaJSCmoDmDSt3cf36nybSbO/Yy40b70ssEJ7vYtXWYtd3LldJC3dvXfVU2hSU\nYPNuVN1gWr+lNffyxy1GHj9uUXxa382V8T15lJZ3hDp+CWjzf7qdKxkjAKxkLM1Ot9eSqSy8xM2c\nW3uacninTs/H+PajzK4dY/ySIxmPc5NG8IaqFFRtYEUSCJl/qW06rwe+YaZ9FGhRSnWX004L++Lh\nQ5vP5PjWLUmB8GEhGP7AhjVJC4zbPb1icCWqtgXI1XPrDDENDVFTuxA5KXW+6/xkN01jfQk7a+em\naDzdx+nGHqIN7Uw09GRcKN2O182811y4m3AolrQtHIrxzle6E7urenbj0B3HVT0LFey7z3o4bYHw\nfBarvr75QceFuz+gU44PTBKI7EcpCET2Q2DhRewD+kEmgil5BB3yKAe1xyEwa5RbYNb4brOzPqWv\noJ65JDtdXUuWsshEvqEI3SwenpomVu/tiGc3xBpniF4wCAGInj9IbFl6S5SbNIJ3VKWgAiilgkqp\n3cAJ4AGt9WMpSVYCL9i+HzO3peZznVLqcaXU47Px0j0U1uLhJ887k9bnjjoKTP14lObB4YIXGM9F\n3dHa3H/R06jAFDregApMJYmphX2+a3h6NE30a+emCE+PMtWw3NV1lKLPtGOZ8zndLt4dTB3Vattu\njdotdrHqzhrnhbu7ST4+2Jw8B9TumXXjLo9ykOqV2r9nsse+3c21ZCsLr3CzUHryyGdN9LynPbfD\nTibvNGFDQDt6oG7SCN5RtYKqtY5prTcDvcDLlVIvLjCf27TWF2itL6gN1HtrpI1oVxsjG1ez/Mn9\njGxc7bjO6VRThLHO9sQC427WQvU6EPj8bAs6Xo8KTKLj9czPtqSlqesLE59ppn7yJNPhlkTzrsVs\nqJ6ZUIur6yjVAKShCedzul28O6ad+w9jtmkdmRalzrVYtdWc+3TQeeHuvUFbmSc8MmOUrVLxJM/M\nVR7lwO6dQpqX2o9zmdi357yWHGXhRMHeaeri4TYvNS1NMM70hudL5qU6PSMJz7PGvL4aneaBukkj\neEvVCqqF1noU+CXw2pRdx4FVtu+95rayY/WZrtyxi47d+1m5Y1fa4uFWn2nqAuOpYlVK5mdbiM11\nEAwNUVt/jGBoiNhcR5Ko1vWFOd3YQ+PpPiKTw4nm3dQ+UzcLpZdyNO8dj21mei45IEA+i3ff3beZ\n1JUNtYYfTZyf+J7vYtWp/aIfcjHvMskjs7B5Zm7yKAep3mnq9i+qS5gipawI8UW1UFa5riVXWaRS\nyMumm0XKM6UphZea6RlJ8jwtUjxQN2kEb6lKQVVKdSilWszP9cCrgX0pyX4CvEMZXASMaa1zDzMt\nAdPtLazcsSsxyjfU0pBYPDyRJtKQ1Ndo9anO15RPUOOxhqQ+05raUYKhIeKxhkRg+/maehpPp/eZ\nztfU03xwhpqxkON12K+1dt+xkk+N+VnoD/nC/isYmG4irmFguokvPHe568W7bzl4GXcd38J8XKE1\nzGvF9ycu4LNjr0uk+enUS/j0yFX0zzcT19A/38ynR65KG+WbKcbt0XAs4Y1ZzAUUR8MLfb+q7kTC\nI0tss80BdZNHWbB5pxYJLxW4J3AuN6urOE4zceA4zdysrkoawZvrWnKVhRe4WTzcMU0wzlyHu1Wd\nvGB2zfiC52lRo43teaQRvKUqFxhXSp0L/AcQxHgp+J7W+pNKqfcCaK1vNafN3ILhuU4C79JaP54t\n33ItMJ7P3Ee3cxpLNT/R7Vu+21GO5Zhj6tXc0mLnk5ZyzqiQnVKviVqpUb1+RRYYN6jKeaha6z3A\nFoftt9o+a+D95bSrkiQm4XuYn1tETJMph5Bmm1vseprUIkXEVKgUVSmoixEvAjsUK6r5VkT5VCyl\nriBKIaSX1T/F9c0P0hkcyxi0wZ6mX5nBCig+qELSOVqf4KaRh+iJjdMXbOKzrdv4CQtj8K4+/XT6\n/saF/XVHa10FVcgWMMHV8TnwIo9slFpIIb97PlY/xfjWX9G045UEp9x33YiYVi8iqD7BCuxg9T9a\ng5SWTfbllY9VqbgV1kIroaUgpje33k19wJg7aQVtABKimprGClZAnKKEwu59Xn36aT4zfC8N2pjv\n2hsb5zPD9wLwk8YX59wPhiB/avheGpjPaqc9YEJs5OLEdivggjWPtJDr9CKPbPhNTAGi5z3FXOcQ\n0fOepunRl7k6RsS0uqnKQUmLEa8DO1iDiHL95UvzwZlFL6ZgBF2whNIiNWiDY5qUYAVumVk9m/iz\nc9PIQwmxtGjQ89w08pCr/ZnSpNmZI3hErqAMufAij0yUo4k3XzGN1U8xveEQKFxPqRExrX5EUMtM\nNgFwCuzgp4Db+dpSDWKaaWk1N0EbOmtyByvIhZOI2umJOY/ItLbn2p8tjduACW6CMuTCizxSKfSl\nMB8Kff6i5z2VVJ65ptSImC4ORFB9RCGBHcpBIV5ptYhpJrIFbbCmwbgJVpCJXEJq0Rd0DkRhbc+1\nP1uahJ05AiYUc5250uaTh0U5hBQKF9OEdxrMHfihHM+KUD5EUH1CpsAOlRTVQpq6qmUkb65RvJmC\nNnw+tBCIwE2wglTcCqnFZ1u3MamShzpMqho+27rN1f5Maex25gqYUMh1puJFHlCevlIobiRvkndq\n4eClipAuPmRQkk/IFNhhOtJA68GhsixWbVFoZbJYxBQWBh7ZR/l+PpQ8KvWewLkQx9XI1XxE1I41\nsCjTKN5c+53SpNqZK2BCPteZiWLzqAYhtZjrGF7wTi1SAj+ImC5OqjKwQ6koR2CHQgWhHILqZyGF\n8olpKsXOKy1UTEtFtcxTLZeIWpRrvMJiFFMJ7GAgHmqV0HxwpmSiWkxFImKaGb8JabWwWIUUFqeY\nCguIoFYR1oPvlbAWW5EsCTHNEuwA4GOxe3grOwmiiaH4LufzqeCVSWKqmCTMr5jmD9F4H5v5E0P3\n8/aJXQkbvrVsC3/fkbpWxIId8cB2x2vJda3UDBPqupe5gStgvj1tdzGBG9yKqJtgG/lQSa801jjD\n8Nv20f7tcwhOlK7VoFznEWRQUlVSTCVgDTQq1itdEmJKcrCDVD4Wu4e38zg1aBRQg+btPM5H6n+S\nlK6WpwhyglrcLW6eD58Yup93TDyRZMM7Jp7gE0P3p6W17Mi0Oku2awWoWb4D1JzxPwUrcMNKxgiw\nELjhyviejLbnOx/aCqTRXTNGQC0E27isPv9yLfYZyIdsK8bMrh0r+eov5TqPIIJadrwSIrswplYO\nTvu8qkDK2WRVaTHNFuwA4K3sJLVBWAFvn9hl+z5JiOdRQIiDKLxdM/PtE7ty2pBqh+MaojmulZph\nVGgUpUCFRqFmOGm3m8ANxQYVcRNsIxflFtJMz0tirdIAJV2jtFznEQxEUBcRpawsyj1fruJiSvZg\nBwBBnEXBvt3wSq3v2nMv1Y0NaXY4XUuOa031SlO/ZwzcoMc8mzfqJthGJsoppJD7xTNprdISrlFa\nrvMIBiKoQk7KPZDCD2KaK9gBQCzNN0zevuAVmnkQ99xLzWWDox2p15LjWiNDJwmY3qmxHwKhUSJD\nJxNiOTifORCGV2QLtpGNcgtprucl4TVaa5XW6JJ4j+U6j7CACKqQkUpEcfGFmJI72AHAdzk/zT/U\nwLeWGSsLJnunCym89FK/tWxLVhsSdqROj7NdS65rnduY3meauj1TIIxbxvIL3JCNfM/hN6/UIslr\ntCiB91iu8wgLyCjfClC775hnQd1LRSWG91dKTJ3IFewA4FPBK4nVxzOOsA1yMuEVJvIgTpAhT2wE\nEufKNsq3ZnYYVZv5WrJda+MRzczKCRw7ausnEl+dAmEUOwI3FTfnqETs63yfldk14wteo0WNNrZ7\nSLnOIywggR1slCOwg4VfBbWS8+SKLZNCxbTQuabVMM+0mCAO5Z4PWgyVWkRC5pUaSGAHA/FQBaDy\nFUOlXjCKjYLkZ5aCmIqQCn5CBLVC+KXZ1w8Vg1M5bO/Yy7XrdrCibpwTM03c/vxWfjG0KWMepYgi\nlS1QgVvv9BND93P+zFO8p7uT2/sH+V3dSxyDLmTj6tNPZ43V60RBYmoGdgjv3gakB3bQoUnmzn6I\n0L5tqDmHwA9lopJLGvrheRH8iwhqBamkqPqlYsgkpjduvI9w0FgUuys8zo0b7wNwFNVSNPVagQqs\nuZVWoALiuI7+YwVdeGNPFxMBxUc62rmz7wkA16J69emn+czwvYkFwntj43xm+F6AjKJaqGcabN5N\noG6Q+dW7CR28OG3//Ord6ObM+0uNCKngd2SUb4Up94NaDesvXrtuR0JMLcLBea5d5zzatBS4CVSQ\ni7dP7OK5UA0Ha0OgFAdrQ+wP1aQFXcjGTSMPJcTUokHPc9PIQ47pC27mDUwSbNgPCuKdB9Ch5MAO\nOjRJvPNAxv2lwsugJIXi9+dF8A8iqD6gHA+sH4U0k3e+os55FKLT9lINRMoYqCDDdieCaP6uY3nS\ntps6lmcMxuBET8y5LFK31x2tLarPNBzYlRTYYX51cmCH+dW7s+73Ej+IKPjzmRH8jQiqTyjFg2tV\nCNVWKZyYacpreynoxzlYQKbtTuwNhRLeKZDwUp8NhbIfaKMv6HzN9u3FLse2rC9qeJ8Bc+pMIJ7k\nhSa80wz7i8Xr8JjFUo3PjOAPRFB9RLEPsl1A/V4hZOs7vv35rUzHkrv3p2M13P781qRtpZwm80V1\nCVOkBBEgxBeV+0AF13etdNz+Vxm2O/HZ1m1MquSymFQ1fLZ1W9FeqRXlKMn7tLB5obn254vfBNSi\nGp4bwd/IoCQfstgf6lwDsayBR/mM8vWaewLnQpyClyMDOFmjUKkREZTiZI37qTrWwCP7KN8vxS/l\npyPu7XDCPi1GNw0teJ8Wgbix3cX+XPhJNFNZ7M+aUF4ksIONcgZ2WMpUOiKSF3NPKxHUodimXYtS\nzjH1s3jaESH1FgnsYCAeqlBW/DD31qKYBbHrjtYys3o25xzRQuaQOp0rJ7kWB8cbIa0WwczEUhRS\nWWC8fIigCksSL+aZvvnwPj5F5jmihcwhtcjXG7UvDh4bSZ8jWqiYVruAWixFIbWwLzDe+uMzK23O\nokYGJQlVhxdRkbyYZ/oB/WDWOaKFzCEtaKBRlsXBi1mLtNrFtFoG6JUSWWC8vIiHKpQNPzX3ejHP\nNFNaa45otjmkXvWHgvPi4A27X1FUntUqpktZPJ1wWmBcvNTSIR6qsCTxYp5ptjzqjtZ6co6cOCwO\nHmzYX/AcUb9NZXHLUvdEnZAFxsuPCKpQFvzknYI380xz5eHFOXKRaXHwQuaIVpuQSpNudmSB8fIj\nTb7CkqPxiOaeNcXPM801V9WLuay5qFEn0KrwOaJQXUIq4ukeWWC8/Mg8VBsyD7V0eOmhejEoqdrX\nQV1KU2BERP2PzEM1EA9VWJI0HtFVKapeBWXwu5iKiArVSFUKqlJqFfANoBPQwG1a66+kpNkG/Bg4\nZG76kdb6k+W0czFTzGTxfBcPLxV/fGIPf9WeuTnWdeAHF0EVisHLyEaZhHR7x17ecNbDfK63jpte\nmOHO/a8q629iF9BY4wwnrpNABEL1UZWCCswDN2qtn1BKNQI7lVIPaK2fSUn331rrKytg36Inn8ni\n9s0RAOgAAAwkSURBVObefBcPLxWX1T/Fza2ZAzvkE/ghV1CFQimHkMLCb/K5FY08Ea7jR11zfDBQ\n2t8kmwcqgQiEaqUqR/lqrfu11k+Yn08DzwLul/AQiqKYyeJ+WDwc4PrmB6kPZA7s4DrwQ5agCoVg\nBWLwsmk3V/Putet2cLo2zo+XRdBKcdeyCBO1cU9/E7crIUkgAqGaqUpBtaOUWgtsAR5z2H2xUmqP\nUuo+pZTjq7ZS6jql1ONKqcdn41MltHTx4DRZ3C35LB6eCS/6/zqD2QM7uA384BRUIR/sAuq1R+q2\nnFbUjXNrSzNxc2WcOIpbW5rz+k3spIpnPv2hxdxbglBpqrXJFwCl1DLgh8DfaK1Tn/4ngNVa6wml\n1BXAXUBa+5HW+jbgNjBG+ZbY5Kon02TxpgfXuOrvOjHTRFc4vaIu5+LhAIOxZrpr0kVzcL6ZxmOa\nwS7n/UlBGRyCKgQi+4mNbXbsSy3lKi8WhbxsPBtr5sfLIswFDEGdCxhe6psG4jmO9HbwULH3liBU\nmqr1UJVSIQwx/ZbW+kep+7XW41rrCfPzvUBIKbW8zGYuOoqdLO528fBcFOul3jJ2CVPxlKAL8RC3\njF2Sdf/Xhi9JCKNTUAWFJhzYleZ5llpMi4lwdHNoXcI7tYij+GjtOsDZ4yxFQAUJRCBUO1XpoSql\nFHA78KzW+osZ0nQBg1prrZR6OcbLw3AZzVyUFDtZ3A+LhwP8dOolgNGX2hkcYzDWzC1jlyS259rf\neEQz21Z8UIVi8aL5+1BjnFggWVDnAoojtdN0lnH6igQiEKqdqgzsoJR6JfDfwFOAVaN9BFgNoLW+\nVSl1PfA+jBHBU8AHtNaPZMtXAjuUhlKGHfQiyEO1Uco5pDL/UygECexgUJUeqtb6V0DWWfla61uA\nW8pjkVApmg/OLBlRFSEVBH9TlYIqVJZt6w9xzYW76VgWZWgiwh2Pbeahg2dkTF+771hJvdRCRfWm\n5v/iTct2EkQTQ/GjifP57NjrSmBh4ZQ6opEIqSB4hwiqkBfb1h/ihq2PEg7FAOhsjHLD1kcBsopq\nqclXVG9q/i/esuxxlNnOUYPmLcseB6i4qJYjLKAIqSB4T9WO8hUqwzUX7k6IqUU4FOOaC/NfLsxr\n8hnp+qZlOxNiaqGUsb0SWLaLmApC9SIeqpAXHcuieW23KHWzrx1LlLJ5rEGcB+Nl2u415Q5OLyIq\nCKVHBFXIi6GJCJ2N6eI5NBGpgDXZySasMRQ1DuIZyz7WrWhbyo0IqSCUD2nyFfLijsc2Mz0XTNo2\nPRfkjsc25zy2UpW7vTnVErYfTZxP6owxrY3tXp+vEmJaisALgiBkRzxUIS+sgUf5jPL1G80HZ/g3\nLiW0PsZVPbsJKk1MK+7u28y/HbyUZtIHOPl9/VALEVFBqBxVGdihVEhgh/JQrr7UpYQIqVBJJLCD\ngXioQtkp5wClxYyIqCD4CxHUKmF40zrCw6NEBk4ltkW72phub6F97/MVtCwdN4Ef/CCq2zv2Vjym\ncCGUQkjzDdYhCEI6MiipSggPj3J86xaiXW2AIabHt24hPDxaYcuSsQI/dDZGCaiFwA/b1h9KS1tJ\nD2t7x15u3HgfXeFxAgq6wuPcuPE+tnfsrZhN2SjVCi+Q328mCEJmRFCrhMjAKVbu2MXxrVsY2nwm\nx7duYeWOXUkeqx/IN/BDpUT12nU7CAfnk7aFg/Ncu25HRexxopQiasfPwToEoZqQJt8qIjJwitbn\njnLyvDNZ/uR+34kpFBb4oRLNvyvqnJcEy7S9XFTiBaPQYB2CICQjHmoVEe1qY2TjapY/uZ+RjasT\nzb9+IlOAh1yBH8o9b/LETFNe20tJuTzRTBT6mwmCkIwIapVg9Zmu3LGLjt37E82/fhPVYgI/QPmE\n9fbntzIdS26gmY7VcPvzW0t+bruA+mGkbrG/mSAIBtLkWyVMt7ck9ZlafarT7S2+avr1KvCDJTSl\nagq2RvOWY5SvH0QzG4shWIcg+AEJ7GBDAjv4m0pPs3GL3wVUELxGAjsYiIcqVA12ofKDuJZLOLet\nP8RVf/Aknz2jgQ8dmuQnvz4vb+9R5pkKQukRQRWqkkxi5rXQVtrbtOaIfq6riSfCdfywd44P5rmg\nu18XhReExYYIqrCoqLQAes01F+7mdFjz42URtFLctSzCe8NjXHPhbtdimG2eqQiqIHiHjPIVBB/T\nsSzKrS3NxM11WuMobm1pzmuOqMwzFYTyIIIqCD5m3/QyfrwswlzAENS5gOGlPje9zHUeMs9UEMqD\nCKog+JibWZXwTi3iKG7Wq1znIfNMBaE8SB+qIPiYwy0kvFOLuYDiUCt0usxD5pkKQnkQQRUEH9P5\n1fM9yeehg2eIgApCiZEmX0EQBEHwABFUQRAEQfAAEVRBEARB8AARVEEQBEHwABFUQRAEQfAAEVRB\nEARB8AARVEEQBEHwABFUQRAEQfAAEVRBEARB8AARVEEQBEHwgKoMPaiUWgV8AyOcqQZu01p/JSWN\nAr4CXAFMAtdorZ8ot61LkW3rD0ncWEEQlhxVKajAPHCj1voJpVQjsFMp9YDW+hlbmsuBM82/C4F/\nNf8LJWTb+kPcsPXRxILWnY1Rbtj6KICIqiAIi5qqbPLVWvdb3qbW+jTwLLAyJdnrgW9og0eBFqVU\nd5lNXXJcc+HuhJhahEMxrrlwd4UsEgRBKA/V6qEmUEqtBbYAj6XsWgm8YPt+zNzWn3L8dcB15teZ\n+we+9nRJDPWW5cDJShvhxOu+E8iwPMoEOwe+trO81rjGt+WZgtjpLWKnd6yptAF+oKoFVSm1DPgh\n8Dda6/FC8tBa3wbcZub3uNb6Ag9NLAlip7eInd4idnpLtdgpVGmTL4BSKoQhpt/SWv/IIclxYJXt\ne6+5TRAEQRA8pyoF1RzBezvwrNb6ixmS/QR4hzK4CBjTWvdnSCsIgiAIRVGtTb5/APw58JRSyhrt\n8hFgNYDW+lbgXowpMwcwps28y0W+t3lvakkQO71F7PQWsdNbqsXOJY/SWlfaBkEQBEGoeqqyyVcQ\nBEEQ/IYIqiAIgiB4wJIUVKVUUCm1Syl1j8M+pZT6qlLqgFJqj1LqpZWw0bQlm53blFJjSqnd5t/H\nK2GjacthpdRTph2PO+z3RZm6sNMXZaqUalFK/UAptU8p9axS6hUp+/1SnrnsrHh5KqU22s6/Wyk1\nrpT6m5Q0FS9Pl3ZWvDyF7FTroKRiuQEjulKTwz4/hSzMZifAf2utryyjPdn4I611psnnfirTbHaC\nP8r0K8D9Wus/VkrVAg0p+/1SnrnshAqXp9b6OWAzGC+oGFPn7kxJVvHydGkn+OP+FDKw5DxUpVQv\n8Drg6xmS+CJkoQs7qwlflGk1oJRqBl6FMS0MrfWs1no0JVnFy9OlnX7jEuCg1vpIyvaKl2cKmewU\nfM6SE1Tgy8BNQDzD/kwhC8tNLjsBLjabqO5TSm0qk11OaODnSqmdZijHVPxSprnshMqX6RnAEPB/\nzOb+ryulIilp/FCebuyEypennbcC33HY7ofytJPJTvBXeQopLClBVUpdCZzQWvs1pizg2s4ngNVa\n63OB/w3cVRbjnHml1nozRtPZ+5VSr6qgLdnIZacfyrQGeCnwr1rrLUAU+LsK2JELN3b6oTwBMJuk\nrwa+Xykb3JDDTt+Up+DMkhJUjIAQVyulDgPfBbYrpb6ZksYPIQtz2qm1HtdaT5if7wVCSqnlZbbT\nsuW4+f8ERr/Py1OS+KFMc9rpkzI9BhzTWluLPfwAQ7js+KE8c9rpk/K0uBx4Qms96LDPD+VpkdFO\nn5Wn4MCSElSt9Ye11r1a67UYzSq/0Fr/WUqyiocsdGOnUqpLKaXMzy/H+C2Hy2mnee6IMtakxWzy\new2QumJPxcvUjZ1+KFOt9QDwglJqo7npEuCZlGQVL083dvqhPG28jczNqBUvTxsZ7fRZeQoOLNVR\nvkkopd4LRYUsLAspdv4x8D6l1DwwBbxVVybsVSdwp/mc1wDf1lrf78MydWOnX8r0r4Fvmc1/zwPv\n8mF5urHTF+VpvkC9GvhL2zbflacLO31RnkJmJPSgIAiCIHjAkmryFQRBEIRSIYIqCIIgCB4ggioI\ngiAIHiCCKgiCIAgeIIIqCIIgCB4ggioIgiAIHiCCKgiCIAge8P8AGtv4hLLEb+oAAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Calculate the Gaussian pdfs at the same set of points\n", "pdf = np.empty((400,250,3))\n", "for y in range(0,3):\n", " pdf[:,:,y] = stats.multivariate_normal(mu[y,:],Sigma[y,:,:]).pdf(coords)\n", "ax=plt.gca()\n", "ax.contourf(coords[:,:,0],coords[:,:,1],np.amax(pdf,axis=2))\n", "ax.plot(X[0:50,0],X[0:50,1],'x',X[50:100,0],X[50:100,1],'o',X[100:150,0],X[100:150,1],'^')\n", "plt.title('Contour plot of the maximum of three Gaussians, with scatter plot overlaid')" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEICAYAAAC6fYRZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4ZHd95/n3t9VS62JdWmp1S2r1BWi7bTe03eAxjiHY\ng4EFw0BmMM8CIQE2G4cMyRMmZMgmy/IYJmF3sjssAwQ8ZMgEwiUXSJiEgDcOAUPGscF2txvabttt\nuy9qSX3RvUtSSy19949zqlRVOlV1SlVSlUqf1/P006pTp0796tSpT/3qe37nHHN3RESkNm2qdANE\nRGT1KORFRGqYQl5EpIYp5EVEaphCXkSkhinkRURqmEJ+FZjZPWb25TV6rleY2TNmdsnMfi7i/mNm\ndvtatKXamNl3zOzdlW7HSpjZz5rZU3nu32tmbmab17Jd1cbMTprZayrdjmIV0+7ws/3CHPe9x8z+\nKd/j1zTkzeydZvZI2Oih8EP4yjIs90/M7PfK0ca1VoaN9GPAZ9z9Knf/Zvad7n7A3b9fwvJjf2mZ\n2dvN7GEzS5jZ+fDvf2tmVsrzr5S7v8Hdv7gayw635VPha/2mmXWWc/nu/kN335/2fFUVZmZ2u5kN\nZE1bs85NsaLau16En+3nVvr4NQt5M/tN4JPAx4EdwG7gD4E3r1UbVmId9JT2AMcq3Qgz+yDwn4H/\nG+gheI/fB7wCaKhg08rOzA4A/wX4BYLXOQ18tqKNqgHr4LNWsoq8Rndf9X9AO3AJeFueebYQfAkM\nhv8+CWwJ77sdGAA+CJwHhoD3hvfdDcwDc+Fz/G04/Trg+8A4QQi+Oe25vg/8r2m33wP8U9ptB94P\nPAM8H9HWveE8d4dtHQJ+K+3+e4Avp91+c9iG8fC5rwun/ymwCMyEbf9QjnXzy8AJYBT4G6AvnP5s\n1uO3RDz2JPCa8O8/AX4v7b7bgYG0278NnAWmgKeAO4DXh+t2PnyOx3O8vwngrQW2gzcCh4FJ4Axw\nT662RLT9ZuCR8LHngE+E0xuBLwMj4fr9MbAj+30GXgT8YzjfReArQEfWc/0WcBSYAP4caMzxOj4O\nfDXt9ovCddQa47PwReCD4d87k9ta2nJGCTpfqfURtZ2wtA2+Gzgdvqb/Pc/z3gk8Eb63Z8ncXt8C\nHAnX7bPA68Pp7wWeDB/zHPAr4fSWsC2LYXsuAe+M2k7CbeMLBJ+Rs8DvAXVpn7v/Afy/4fvyexHt\nvgf4evh+TAGPATfk2EYiMyRHe/tybMdfAi4Ap4APh+/FFoJt68Vp83aHy9we3n5TuA7HgQeBg1lt\n/G2CbesysJnl2/Y/h48dAj4DNGTl0b7w7y6CDJgEfgT8B9KyK/K9jxPSpf4jCIorwOY883wMeAjY\nHq7AB4H/kBYAV8J56gk22Glga47wqicIxd8l6EW+OtxA9md/+NM2tuyQvx/oBJoi2ro3nOdr4Qb0\nknDDSL5p9xCGPHANQQC+NmzXh8K2NWRvpDnWy6sJPsAvDTe2TwM/iNrIczw+fWPKXk+3sxQk+wmC\nty/tNb4o+/Ws9P1Ne76XEHxwDhKE9c9ltyVH2/8Z+IXw76uAW8K/fwX4W6AZqANeBrRlv8/AvvA9\n2EKwff0A+GTWc/0I6Avf9yeB9+V4Hf8d+O2saVPAy2J8Fv4Xljoi7yQI1T9Pu++/R62P7PeZpW3w\nj4Am4AaCALkux/MOAT8b/r0VeGn4980EX2qvDd+XncC14X1vJPjiMeA2gs/cS6Pal2s7Af6a4FdP\nC8Fn+0csfVm8J9xufp0g+KI+a/cQfHHcRfD5+S3geaA+YhsplCEDUesm7bm+FL63reH6fRr4pfC+\nPwZ+P23e9wP3hX8fIuh8vpxgG3x32K4taW08AuxKvsasdr8MuCVcB3sJtr0PZOVRMuT/DPiLcH2+\nmOCLM2/Ir1W5pgu46O5X8szz88DH3P28u18APkrwczhpPrx/3t2/TfBtvD9iORCssKuA/8vd59z9\nH4FvAe8oos3/p7uPuvtMnnk+6u4Jd/8J8N9yLP9/Bv7O3e9393ng/yH4UN4asx0/D/yxuz/m7peB\n3wF+xsz2xn0hMS0QBOD1Zlbv7ifd/dmYj91G1vtrZg+a2biZzZjZqwDc/fvu/hN3X3T3owRfkrfF\nfI55YJ+ZbXP3S+7+UNr0LoIPwYK7P+ruk9kPdvcT4XtwOdy+PhHx3J9y90F3HyX44rgxR1uuIgjG\ndJME4VDIA8ArzWwT8CrgDwhKWoTteSDGMtJ91N1n3P1x4HGCsI8yT/Detrn7mLs/Fk7/JYLt6/7w\nfTnr7scB3P3v3P1ZDzwA/D3ws3EbZmY7CDpkHwg/J+cJeu1vT5tt0N0/7e5X8nzWHnX3r4efn08Q\n/Hq7JWK+QhmSr611Ybt+x92n3P0k8J/SHv/VrHa/M5wGwS/6/+LuD4fb4BcJvnDT2/gpdz8T9RrD\nbfahcB2cJPhSXPa5CNv4VuAj4fr8KcEvw7zWKuRHgG0F6lF9BD+Rkk6F01LLyPqSmCb4sOVa1hl3\nX8xa3s74TeZMkfNktze9LanXFbbpTBFtyX78JYL1WcxrKcjdTwAfIOg5nTezPzOzqNcTZdn76+63\nuntHeN8mADN7uZl9z8wumNkEQc1+W8zn+CWCX0XHzezHZvamcPqfAv8f8GdmNmhmf2Bm9dkPNrMd\n4Ws6a2aTBCWe7OceTvs73/Z1CWjLmtZO0JvPK/ziTBB8gfwsQedj0Mz2s7KQj9vmtxIE7ikze8DM\nfiacvovg18QyZvYGM3vIzEbNbDx8fNz3C4L9RfXAUPiFP04QYNvT5inqcxZ+fgaI8Vkj92cyyraw\nrdmPT37Ovgc0h9vwXoL376/D+/YAH0y+xvB17sp67pyv08yuMbNvmdlwuG1+nOj13E3Q28/OnbzW\nKuT/meCbbdkQvzSDBCsraXc4LQ6PWNausLeUvryz4d8Jgp/3ST0xlhllV9byo9qb8brCkSa70tpS\n6HmyH99C0HM9m/MRueV93e7+VXd/Zfh8DvzHmG1Mvr9vKTDfVwnqibvcvR24l6AUsKxtYa+lO61t\nz7j7OwgC4j8CXzezlvCX3Ufd/XqCX0dvAn4x4rk/Hr6Ol7h7G/CutOcu1jHSesxm9iKCsuDTMR//\nAEH5ocHdz4a3301QRjmS4zFxtsec3P3H7v4WgvX3TYKf/BAExouy5zezLcA3CH557gi/sL/N0jqL\nak/2tDME28U2d+8I/7W5+4E8j4mS+pyFn+l+YnzWyPxMFnqeiwS/drIffxbA3RcI1tk7wn/fcvfk\nl/oZglJOR9q/Znf/Wtqy8j3/54DjwNXhtvm7RG+bFwjKW9m5k9eahLy7TwAfAf7QzH7OzJrNrD7s\nKfxBONvXgA+bWbeZbQvnjzsc6xyQPo70YYJezYfC57kd+FcE9SwIPkj/JmzHPoJe4kr8H+EyDhDs\npPrziHn+Anijmd0R9jA/SLDhP5ij7dm+BrzXzG4MP3gfBx4Of9YV6whwp5l1mlkPQc8dADPbb2av\nDp9jlqUdVck27s360kxx93GCn8afNbO7zKzVzDaZ2Y0EtcOkVmDU3WfN7GaCn7xJTwONZvbGcD19\nmKB8lGzfu8ysO+zJjYeTF83sX5rZS8IvhUmCD2r6L7j0574ETJjZTuDfF15dOX0F+FcWjGVvIdj5\n9VfJD304lPD7eR7/APBrBPsFINh38GsEtdWFHI8ptJ3kZGYNZvbzZtYeljwmWVpHXyDYvu4I37Od\nZnYtwZfWFsJgMbM3AK/Lak+XmbVnTUttJ+4+RFDi+U9m1hYu/0VmFrdEl/QyM/s34S/FDxB8fh6K\nmC9fhkS1NyUtxH8/3H73AL9JZgZ9laD8+vMslWog2C/yvrCXb2bWEm7Hccp3EGybk8ClcN3/ap42\n/hVwT5g71xN0DvLLV7Av979w5TxC0GsbBv4OuDW8rxH4FMEOoqHw70aPt1Puapb2bH8znHaA4MM0\nQTCq4F+nPXYbwcY3RbB3/x6W73jdl+d17CVzdM0waSNjWD665l+HbZgI23Qg7b63EIyOGCdtxEPW\n872P4Cf1KMHP+/6o9ZDjsenrqZHgi2iSYE//v2Npx+tBgp1iU2nPk9wJ2wX8EzAGPFbg/f0RwRfs\nBYIv27tZ2sl8F8HPy6lw+Z/JWk/vCd/78wQ72NLb/uVw+iWCnnRyh+07CEYCJQg+yJ8i3AFM5o7X\nA8Cj4eOPEHzZ5tuxmfEeRrzWd4bvW4JgZ11n2n1fIG0nXcRj94fbz7vD2+0EPbTfTpvn9qz2ZWwn\nLG2Dm9PmSb3erOdrAO4L379JghFIr8zaPo+G78sJ4H/ypZ2L58Ln/FOCTlL6jvs/ZmlUU1/UdhK+\nts8RlFgmCEZXvT3t/c4/MmT56JrDhDt/c2zfkRkS1d6I59pKsJ1dIOidfwTYlDVPcpRbQ9b014fr\nNTlC5i8JR1tlb1sR7X4VQU/+EvBDgh3IkXlE8Ov2WxQxusbCB0oRwppccg9/vp3JFWdmp4F3ufsP\nCs4sZWFmR4A73H2k0m1Z78zsHoKAe1el27Je1fzBBxuZmXUTfPOfrHBTNhR3zzUqR2TN6dw1NcrM\n/gXBwVyfdvfTlW6PiFSGyjUiIjVMPXkRkRpWsZp8w6Ymb6qLO8JIREQAJq9cuOju3YXnDFQs5Jvq\nWrl129sq9fQiIuvSfcOfLXiUazqVa0REaphCXkSkhinkRURqmEJe1oWRAy8k0ZN5hb1ETycjB1Z0\nOheRDUMhL+tC48g4Z287lAr6RE8nZ287ROPIeIFHimxsOq2BrAstw6PsfOAwZ287xNanTjO2fzc7\nHzhMy/BopZsmUtXUk5d1o2V4lK1PnebiDVez9anTCniRGBTysm4kejoZ27+bbY8/w9j+3ctq9CKy\nnEJe1oVkDX7nA4fpPvJMqnSjoBfJTyEv68JsV0dGDT5Zo5/t6qhwy0Sqm3a8yrrQdey5ZdNahkdV\nlxcpQD15EZEappAXEalhCnkRkRqmkBcRqWEKeRGRGqaQFxGpYQp5EZEappAXEalhCnkRkRqmkBcR\nqWEKeVl1uqqTSOUo5GXV6apOIpWjE5TJqtNVnUQqRz15WRO6qpNIZSjkZU3oqk4ilaFyjayauWv7\nAZhpa2F43x56TpyiaXaGhucHGHj1TcHtycSyxzUcH1jrporULIW8lE0y1LPNtjRnBHrTZIKeE6eY\nbWmODPnkchT2IqVTyEtJcgV7uq1DF5ZNa5pMRAZ89rIV9CKlUU1+g1vpGPa5a/tjBXyp1up5RGqV\nQn6DK3YMu0JXZH1RuWaDizuGvdLBrtKNyMqoJy95x7BXU8+9Wtohsp6oJy/LxrA3D49Q39Fc6WaJ\nSBmoJ7/BJWvwOx84TPeRZ9gRjmGfaWupdNMiqTcvUhyF/AY329WRqsHPXdufMYZdRNY/lWs2uK5j\nzwV197TyTJwx7CKyPqgnv8Gp/CFS22KHvJnVmdlhM/tWxH1mZp8ysxNmdtTMXlreZkq5rdaomYkX\nbcn4t1Z0YRKRaMWUa34DeBJoi7jvDcDV4b+XA58L/5cqVI5wjxvg2fO1P3u55OeOGjOfPKgruX8h\nfYeyyEYWK+TNrB94I/D7wG9GzPIW4Evu7sBDZtZhZr3uPlS+pkqpSg33cvTMk8soR9in04VJRKLF\nLdd8EvgQsJjj/p3AmbTbA+G0DGZ2t5k9YmaPzC3OFNVQKU0pAb8apZfVKOXowiQiyxUMeTN7E3De\n3R8t9cnc/fPufpO739SwqanUxUlMKw341a6rl3vZujCJyHJxyjWvAN5sZncCjUCbmX3Z3d+VNs9Z\nYFfa7f5wmlRQKeG+3qTX4FuGR2keHsm4LbJRFezJu/vvuHu/u+8F3g78Y1bAA/wN8IvhKJtbgAnV\n4ytrJQG/1iNiks9ZDukHdcFSjX62q6MsyxdZr1Z8MJSZvQ/A3e8Fvg3cCZwApoH3lqV1siLFBvx6\n7Lln6zr23LJpLcOj6sXLhldUyLv794Hvh3/fmzbdgfeXs2FSvPUa7hMv2lL20TYiEtARrzWikgE/\n0baT6cbMssh0YwcTbcsGWJWFzisvEp/OXVMDign4lYb71B7Led/i3DTTDdu5vNXY3DBO3bl2plu2\n05w4v6LnEpHyUcivc6sZ8PmCPd3mhuBSgQvz3SwuXIVf1UTzpfM0z0ZfQlBE1o5Cfh2LG/DFhHvc\nYM+2uWE8CPjFZmzTNAs7JuDUihYlImWkmvw6Ve6An9pjKw54gCtzHfhiE7ZpGl9s4spcR0nLE5Hy\nUE9+HSpnwJcjiK/MdbAw301d/QU2N4ynbgfLH6f1lJf8HCKyMgr5daTawj1pcaE5FfCwVKNfXGgG\nyluX18gakeIo5NeJag14gIamwWXTgqDXjleRSlNNfh0oV8CXWndfKdXmRSpHPfkqV2rAj3bspX4+\nAdsupqbNz3azuNjMlubo4S+Xd8/Fes4tpxtizVcOY73dNCamSX/GRE8ns10dkac0EJGAevJVrCw9\n+C3TXG7ayvxssCN0frabxYUONm2azpjt8u651L+4ipm3VI2Jac69oD91+uDkWScbR1QSEslHPfkq\nFSfg45Rn6rnA/CwsLnRwOXEVsJlNdePUN14ASg/qy7vn1qRH3zSZ0JWfRFZAPfkqVK6ATwoC/QpQ\nD1yhvvFC0b32fNaqR68rP4kUTyFfZUoN+Kidq0GpZjMwD2zmcn1XaY0sszhnoGw4PqArP4msgEK+\nipQj4LMla/B0jMJ1TwX/j3fCUG9Jbc222r359Cs/dR95JlW6UdCL5KeQrxKrEfAAi4vNQbD3hhfq\n6h0KbieuWlE7K0VXfhJZGe14rQKrFfAAXPvM8mm9ua/MuLc/2CF7cqC7YJvWSsPxAaIKTLryk0hh\nCvkKW62AT5VPRrZB4wy0JJbuTLTAbBN0XUyFerbk9ImTuxm7vCnn4zOeg/nULIsLTSwuNLK5YSzv\na9MVoURWl8o1Va6kgIcgfM/uCoIZINHCpqGd7Ng1lDPg0zW0TQbzN8+nHs/ZXWGoZz7H4kITEAT8\n/Gwvm+pmAXSCMpEKUk++ggr14nMFfL7yzLIdoC0J2HkGzu6iffcAU0N9dB88RlNnvIOImjrH6T54\njAtHD0D7OIx1BstL79mHzzF/ehd19RMszLdT3zjEprqZ3AuOSSckEymNQr5Cyh3w+Ua37N1/krE6\nY+L5vbS/4GTsgE9q6hyntX+Qief3wrbzmQGf1JIIA76LuvqRWAGvUo3I6lO5pgLWNOD7LzAz2sHU\nQB/tLzjJ1EAfM6PFjUhJfzxjnUuln3SJFhbm26mrH2Fhvj1VuimFevEipVNPfo2tVcAn6+0zox1c\nOHogVaJp7BzLuF1I9uMnFjyoyaeXbMI6fbJEs6lumvnZXuobh2gfmM7/BCKyqtSTX0NrHfAAc5Nt\nGYGerLHPTbbFafKyx6dq/LNpPfXZJurrl2rwm+pmqG8cYnGhMedyC5Vq1IsXKQ/15NdIJQIeoH3v\n6WXzNHWOx67LRz2elkRmXb7rIpsSmScpC3r0pe94FZHSKOTXQKUCPspre45HTr9/+NqCjy1WrqGT\n2uEqsnYU8is0cuCFNI6MZxxxWc6LWMQK+PAgpL37T6YmzYx2MDfZtqwHnivcs+9fjbAvlko1IuWj\nmvwKNY6MZ5wgK9dFLFbai4+yrAffOMOmoZ2p0TLJnaQNbZOpWV7bc7xgwJdqLa8QJSLFUU9+hZIn\nyMp3EYtylmmiSjR7959kpnucC0cP0No/yNTA0oFOqx3shay0VKNevEh5qSdfgnwXsVj1gA/r7+kH\nKrX2D65qwFfTSctEJB6FfAlyXcRirQIeMg9Uujy4gwNMLpt/NUWVatSLF6keKtesUPpFLFqGR2ke\nHuHsbYfY8fwATZMRh/2Hyh3wyQOV3nz9QwwO9/HdH7yOO1719/T1DK7gVVWOAl5kdagnv0I5L2LR\n0lz0slYS8LB0oNKbr38IgL6eQe541d9zYWR70W0oJKpUU85evIisDvXkVyhqmGR9RzNbh3KPVY/q\nxa804CE4UCm7/t7XM6hevIikqCdfJuUaKhk34KHw2Pe1pl68SPUpGPJm1mhmPzKzx83smJl9NGKe\n281swsyOhP8+sjrNrU6rcV74pHwBf9937+ToEwczph994iD3fffO1O1C8zx+7EYGh/sy7p8Z7WDi\n5O7U7bilmlzGeruZacs8c+VMWwtjvd3qxYussjg9+cvAq939BuBG4PVmdkvEfD909xvDfx8rayvX\nsVJ3tEZJ9uD7egd4+NFbUyF+9ImDPPzorfT1LgVnoXm6u87z3R+8jsHhPu4fvjbygKo48vXiGxPT\nDO/bkwr6mbYWhvftoTGhM1SKrLaCNXl3d+BSeLM+/KfruYXiXKM1Wyl1+PQSzcHrjwLw8KO3curM\nCxg+38vLX/ZganqceZI7a7/7g9expe9cxgFVUFovPlmmaZpM0HPiFMP79tB+boSJHV30nDhF+4+e\nirUcEVm5WDV5M6szsyPAeeB+d384YrZbzeyomX3HzA7kWM7dZvaImT0yt7j+z1C4kjJNuQI+6eD1\nR+nZPsTw+T56tg9x8Pqj3NX2WOpfrnnS9fUMct01xzIOqMqlmBE16ZomE7SfG2Gsfwft50byDjMV\nkfKJFfLuvuDuNwL9wM1m9uKsWR4Ddrv7QeDTwDdzLOfz7n6Tu9/UsKn0KwdVs3LvaM21k/XoEwcZ\nPt9Lz/ZBzp3vwZ7tzLj/rrbHMuYZPt+7rEY/ONzH48cPLrtyVClHuGbvbJ1pa2FiRxdbB84xsaOL\n+XGVakTWQlFDKN193My+B7we+Gna9Mm0v79tZp81s23ufrF8Ta0u+Xrx5d7Rmi/gH370Vl7+sgf5\n2C1/wjcOv4Y/evBtALz10D8A8I3Dr0nNc/D6o6nHQNDDTx5AFXXlqGwr7cUna/A9J07RNJmg9djz\nGQeSicjqiTO6ptvMOsK/m4DXAsez5ukxMwv/vjlc7kj5m1sdVrMOX4zBof5UwEMQ7L98619yeOC6\n1DyHB67LqMEfvP4oL3/ZgwwOBa/hwsh22l98fNmVo86d6V1xu7J78bMtzamAh7QDx7qKu9asiBTP\ngv2qeWYwOwh8EagjCO+/cPePmdn7ANz9XjP7NeBXgSvADPCb7v5gvuW212/3W7e9rQwvYW2Va7hk\nqWWapGTdPZevT7407/25zh+fXaqJ24vX+WlEVtd9w5991N1vijt/nNE1R4FDEdPvTfv7M8Bn4j5p\nrVqrMk1SoYBfqTi1+DhlmmwKeJG1pyNei5Dei486wGeuvonppswdnznLNKf3wEhXxvSJU/2ce+zg\nsvmj3NX2GB/+21/nG4dfkzH9G4dfw4f/9teBoBcfdbDT4HAfjx+7kfuHr2Xi5O7UjlYIAz7RElx1\nKlRoyORE206mGzsyevHjPV0MXrM31msRkdWjkI8pu0yTfYDPhWs7mGrtY/OV/ENDU2WalktwvgdG\nutjbf4GJU/2MPb2Pxq5gR2ScUxYc6n+SP3rwbamgT+54PdT/ZGqe9IOdgKUdrV3nAWhom+TC0QNL\nQZ9ogbO7oDF4HXHKNPVzCaZbtjPeE3xpjfd0MbK7j+bJqdQ86sWLVIZOULZC6Qf4bJkfZ7axg9ap\nQRrml0I+b5mmK9wvfb6HoR83cXm8na3XnKB9z0DsMk1yBM0fPfg2Hnz+EMeG9vHLt/4lbz30D6la\nfPrBTtddc4wnnz7AHa/6e47RFryOcEfrhaMHWGwfh7FO2HkGWqLHsUeVaZpnx2kaucLI7j4SW9uZ\nbW2h6/QgHcPBa1TAi1SOevIx5NrZmjzAZ6Z5G42z4wUDftnO1q4RtnRMcHm8gy0dE7TvKT4M33ro\nHzjQe4KfDl3Dgd4TGQGflDzY6fBPbuK6a46lAj71OjrHg4C/uB22jqYCvpgjWzuGR2icSjDbdhWN\nU4lUwItIZSnkSzDT1sJ4XxdN0xeZbexgrj73AV5Ro2m2Lmzh8ng7WzrGuTzezsSp/qLPLPmNw6/h\n2NA+Xtz7NMeG9vGRh96zbJ7B4T6efPoAh17yCI8fP5hRg4fghGSMdcK288H/iZaiR9OM93Qx29pC\n4+QlZltbUqUb9eJFKkvlmgJy9eKTB/gkSzT189NMtfbROjXI5b7ZgsvdurCFsaf3pUo0yZr80dbz\ny047kEuyBp8s0XzkofdkHOgELLta1GBDXepgp6bOcWZGOzh35CVLJZrmBJzexWLjEJvqln6Z5BtN\nk6zBJ0s0ydt15yfoyvkoEVkL6snnkW9M/GxLM1dNL9XgG+ZnaJ0aJLFteW8+qhc/O9KZCniA9j0D\nGQcpxXF44LqMGnz2gU4QHOyUDPj7h69N1eDnJoOSzbkzvZk1+JYE9Y1DLC40ppZR6Dzx022tGTX4\njuERuk4PkujdFvk4EVk7BQ+GWi3r4WCocp26IDvkS70ISPr4+EIHOyVFHfRUyiX9dNCTSGWU/WCo\njWqtT11QTC0+GeyPH7uR7q7zGZf7Gxzu48LIdm44cCQ1zylrzDiz5MxoR9CD78o8tdBKz02TTQEv\nUj1UrlmBuGeYjJKrF78ShcbAA5yyxoxx8MmLgiTHwSeV66LcCniR6qKefIRqOQFZIbnGwCd79uk1\n+AtHD9DaP8jUQB+LvWdzjoNP0vVaRWqDevJZynVB7mKUckHu7DHw6QGf1NQ5Tmv/IBPP7w3Gw2cF\nfLFXespFvXiR6qOefBmUcrWnUqWPgX/y6QP09ZxddrDTzGgHUwN9S+PgmxN5D3jSjlaR2qGQT1OJ\nXnwpssfA9/Wc5Tvff0PGNVqTNfhUiaY5EZybZucZtozML1umdrSK1BaVa0pUyV58+hh4gGO0ZYyB\nB5ibbMuswbckYOcZ6oauWrY81eFFao968qH11osHUsMkYakG39Q5njFccmzzzLJ3ecvIPDSMZUxb\nacCrFy9S3RTyJYhzMZDVluvKTlDawU6ggBepBRuyXDNy4IUkepYu7jF3bT8zbS2M9UZfESmqFz/d\n1MniQuYpDBYXmrjcEpZKSrwoSBxff+h1kScbmzi5O3bA56KAF6kNGzLkG0fGOXvboVTQJ0821piY\njr2MzVfgU2RSAAAP3klEQVRmmJ/tTQX94kIT87O9SwcZFbgoSCnuH76W+4evXXbBj+RO1rHLy9/W\nXAGvHa0itW1DlmtahkfZ+cBhzt52iLaL40zs6KLnxCmaJpcfIJSrFn+5b5b6hSHmZ3upq59gYb6d\n+sYh5pM7OPNcFCTb/cPXxh4rnz3+Pc6BTsUGvHa0itSODRnyEAT91qdOc/GGq9k6cC4y4AvZVDcT\nBnwXdfUjzL9gInOGrhGYagsvCjKe96Ig+YI+X909/UAntp1f9YBXL15kfdmwIZ/o6WT0+hewdeAc\nEzu6aJq6tCzoC51pcnGhiYX5durqR1iYb4fEZGbIjnTBTHPGRUEKBX2xZkY7mDjdH/tAJ1DAi2wk\nGzLkEz2dnL3tUKpE0zR1ieF9e3KWbKIka/D14cU1NtVNMx8eZERLIgj48z2wfZjeG46navLAii7z\nFyXygh95DnQCBbzIRrMhd7zOdnWw4/mBVKAnL8o929IcexmLC42pgIegdMPOMzAbjrhJXAXbh1O1\n+fY9A2y95gSzI525Flm0qAt+5DrQCRTwIhvRhuzJdx17btnBT02TiYxefKGDnzZnHUwEBCGbDNzd\np5bd3b5noCy9+NTwyKzzwUP0gU6ggBfZqDZkT34lpxJOynUAVL7TCkeNWV+pfMtSDV5Esm3IkC+H\nK3Nblx0MRaIFRuJd13Ti5O6cBzIlDfyPm7n45NVAEO4nB7phqBdOXL1seRomKSJRFPIR4pynZlPd\nbMbBUCRagp2eWVdcSsrugec6kKmhbTI1T2PnGJcGdnLyx+FRskO9MN4ZHGgV2nK6YVUCXr14kdqw\n4WrypZRq0m2qmwkOfgoPhuJse+ZO0AgnB7pTZ6OMOpApeYrg1BdC6zh0NAXBfqkVrtRDxyj0DgH5\nT1OggBcR2IAhX4rsenz6wVBRByJFWVZTbx9PHch0broeprPu7x0KA74BNs8VDPh8pylQwItsPBuq\nXBOnF1/MKYXTD4biYmdQsilGoiU4gCl5IFPU44d6gx785jm4Us+mp3cq4EUktg0V8uWUfjDU5oYR\n6huHgpp83KBP1vB3noHu88H/2Y9P1uA7RuHqp9lUN87iQgfzs8tH2CjgRSSKQn6FCh4MVchsU+SB\nTBmPT1wFHaNsmR9hy+kG6hsvBEG/mHnQlgJeRHLZMDX5cu1wTSp4MFQhEQcyZT9+S8MpyDo7QX3j\n0mUEC50mWAEvIgV78mbWaGY/MrPHzeyYmX00Yh4zs0+Z2QkzO2pmL12d5haWfUEQCM5Vk35BkLHe\nbmbaMssqM20tDL24t6TnLuaiHFEXFWGkC07vSQ2LjBqLv7jQxJW5rbSecqabOpmrz7x/rr6J6aZO\nBbyIAPHKNZeBV7v7DcCNwOvN7Jased4AXB3+uxv4XFlbWYTsC4IkT0aWfkGQxsQ0w/v2pII+edGQ\nzVeix7gXI3bQp11UBEid0KxubqkN2WPxk/sBWi4G82y+MsNUa18q6Ofqm5hq7aPj9CSFKOBFNoaC\n5Rp3dyB59E19+C+7TvAW4EvhvA+ZWYeZ9br7UFlbG0P6BUG2PnWasf272fnAYeo7lurYyROSDe/b\nQ/u5kdRFQ+a6r5SlDcmgz3eqg/SLitjFdnyxibr6C2xuWLoId/ZY/MXL7bRNDdIwH4R8w/wMrVOD\nTLX20Tg7zuX6DnqfKnwmTQW8yMYRqyZvZnXAo8A+4A/d/eGsWXYCZ9JuD4TTMkLezO4m6OnTuCn6\nTInlkH5BkG2PP5MR8ElNkwnaz40w1r8jddGQue78wydbT3lRF+8u3KufYm5TO77YjG2azgj4pPSx\n+E2zF1MBn9QwP0Pj7DgzzdtiXfxEAS+yscQaXePuC+5+I9AP3GxmL17Jk7n75939Jne/qWFTzFEo\nK5Do6WRs/262Pf4MY/t3L6u/Q1CimdjRlbpoSNQ8q+3KXAe+2IRtmsYXm7gy17Fsni2DjSxebqdp\n+iKzjR2RNfjL9R2xXocCXmTjKWp0jbuPm9n3gNcDP0276yywK+12fzhtzSVr8DsfOEzL8CjNwyMM\nvPqmjAuCJGvw2RcNuWp6cFlPebVcmetgYb47VaJJ3gZSPfotg41MtfbRGpZo6uenM27P1Tdxqbkv\n1sVPFPAiG1Oc0TXdZtYR/t0EvBbIvhjp3wC/GI6yuQWYqEQ9HoILgiQDHoLSTfYFQWZbmjOCMFmj\nv7J59X5dZFtcaM6owW9uGKeu/gKLC820nnJaTzlXNjelAh2WavBXNjfR/uxlNk/UR76O7IufKOBF\nNq44Pfle4IthXX4T8Bfu/i0zex+Au98LfBu4EzgBTAPvXaX2FtR17LmM23PX9i+7IMjWoQvZDwtq\n8jOFd7wWW5fPpaFpcNm0rUNjwNL4++aZ0eWPm5+h+/h4OH/060h/rQp4kY0tzuiao8ChiOn3pv3t\nwPvL27TqVa6gT19eXHHPA69wFxHYQEe8phvr7aYxMZ3R451pa2G6qS2y9xyl1KAvJtihuIt8KOBF\nJGlDnrumXAdDJWvnxc6vgBeRtbIhe/LlPhiq2NCOq9hL9CngRSTbhuzJQ+bBUO3nRoLbVXTdUwW8\niJTDhuzJw/KDoZqmLhU8WnQtKNxFpJw2ZMjnOhiq58QpoDznrynWSn5FKOBFpJCaLtfkOod8roOh\nZluaK1KyUcCLyGrZkD35QgcRtT97uahrva7USr9QFPAiEldNh3zD8YEVXxFqtYK+lF8KCncRKVZN\nl2tKVc7STfuzlxXwIrLmaronXw6l9OjL9SWhgBeRlVLIxxAV1unBv1o7axXuIlIqhfwKrfYoHAW8\niJSDQr7KKNxFpJwU8lVC4S4iq6HmR9esh/BcD20UkfVJPfkKUriLyGqr+Z48VF+YNhwfqLo2iUht\nUk9+DSnYRWStbYiePFQ2YNVzF5FK2TAhD2sf9Ap3Eam0DRXysDZBr3AXkWqxIWvypZydMt8yRUSq\nzYYMecgM5ZUEvkJdRNaDDRvy6RTYIlKrNlxNXkRkI1HIi4jUMIW8iEgNU8iLiNQwhbyISA1TyIuI\n1DCFvIhIDVPIi4jUMIW8iEgNU8iLiNSwgiFvZrvM7Htm9oSZHTOz34iY53YzmzCzI+G/j6xOc0VE\npBhxzl1zBfiguz9mZq3Ao2Z2v7s/kTXfD939TeVvooiIrFTBnry7D7n7Y+HfU8CTwM7VbpiIiJSu\nqJq8me0FDgEPR9x9q5kdNbPvmNmBHI+/28weMbNH5hZnim6siIgUJ/aphs3sKuAbwAfcfTLr7seA\n3e5+yczuBL4JXJ29DHf/PPB5gPb67b7iVouISCyxevJmVk8Q8F9x97/Kvt/dJ939Uvj3t4F6M9tW\n1paKiEjR4oyuMeALwJPu/okc8/SE82FmN4fLHSlnQ0VEpHhxyjWvAH4B+ImZHQmn/S6wG8Dd7wXu\nAn7VzK4AM8Db3V3lGBGRCisY8u7+T4AVmOczwGfK1SgRESkPHfEqIlLDFPIiIjVMIS8iUsMU8iIi\nNUwhLyJSwxTyIiI1TCEvIlLDFPIiIjVMIS8iUsMU8hFGDryQRE9nxrRETycjB15YoRaJiKyMQj5C\n48g4Z287lAr6RE8nZ287ROPIeIVbJiJSnNjnk99IWoZH2fnAYc7edoitT51mbP9udj5wmJbh0Uo3\nTUSkKOrJ59AyPMrWp05z8Yar2frUaQW8iKxLCvkcEj2djO3fzbbHn2Fs/+5lNXoRkfVAIR8hWYPf\n+cBhuo88kyrdKOhFZL1RyEeY7erIqMEna/SzXR0VbpmISHG04zVC17Hnlk1rGR5VXV5E1h315EVE\naphCXkSkhinkRURqmEJeRKSGKeRFRGqYQl5EpIYp5EVEaphCXkSkhinkRURqmEJeRKSGKeRFRGqY\nQl5EpIYp5EVEaphCXkSkhinkRURqmEJeRKSGKeRFRGqYQl5EpIYVDHkz22Vm3zOzJ8zsmJn9RsQ8\nZmafMrMTZnbUzF66Os0VEZFixLnG6xXgg+7+mJm1Ao+a2f3u/kTaPG8Arg7/vRz4XPi/iIhUUMGe\nvLsPuftj4d9TwJPAzqzZ3gJ8yQMPAR1m1lv21oqISFHi9ORTzGwvcAh4OOuuncCZtNsD4bShrMff\nDdwd3rx83/Bnf1rM81fINuBipRsRg9pZXmpneamd5bOnmJljh7yZXQV8A/iAu08W2yoAd/888Plw\neY+4+00rWc5aUjvLS+0sL7WzvNZLO4sRa3SNmdUTBPxX3P2vImY5C+xKu90fThMRkQqKM7rGgC8A\nT7r7J3LM9jfAL4ajbG4BJtx9KMe8IiKyRuKUa14B/ALwEzM7Ek77XWA3gLvfC3wbuBM4AUwD742x\n3M8X3drKUDvLS+0sL7WzvNZLO2Mzd690G0REZJXoiFcRkRqmkBcRqWFrEvJmVmdmh83sWxH3Vc0p\nEQq083YzmzCzI+G/j1SojSfN7CdhGx6JuL8q1meMdlbL+uwws6+b2XEze9LMfibr/mpZn4XaWfH1\naWb7057/iJlNmtkHsuap+PqM2c6Kr89yKepgqBL8BsGRsm0R91XTKRHytRPgh+7+pjVsTy7/0t1z\nHbBRTeszXzuhOtbnfwbuc/e7zKwBaM66v1rWZ6F2QoXXp7s/BdwIQYeJYBj1X2fNVvH1GbOdUB3b\nZ8lWvSdvZv3AG4H/mmOWqjglQox2rhdVsT7XAzNrB15FMEQYd59z9/Gs2Sq+PmO2s9rcATzr7qey\npld8fWbJ1c6asRblmk8CHwIWc9yf65QIa61QOwFuDX9ifsfMDqxRu7I58A9m9mh4mohs1bI+C7UT\nKr8+XwBcAP5bWKb7r2bWkjVPNazPOO2Eyq/PdG8HvhYxvRrWZ7pc7YTqWp8rtqohb2ZvAs67+6Or\n+TylitnOx4Dd7n4Q+DTwzTVp3HKvdPcbCX72vt/MXlWhdhRSqJ3VsD43Ay8FPufuh4AE8L9VoB2F\nxGlnNaxPAMJy0puBv6xUG+Io0M6qWZ+lWu2e/CuAN5vZSeDPgFeb2Zez5qmGUyIUbKe7T7r7pfDv\nbwP1ZrZtjduJu58N/z9PUEe8OWuWalifBdtZJetzABhw9+QJ975OEKbpqmF9FmxnlazPpDcAj7n7\nuYj7qmF9JuVsZ5Wtz5Ksasi7+++4e7+77yX4WfSP7v6urNkqfkqEOO00sx4zs/DvmwnW3chattPM\nWiw4pz/hz/XXAdln8qz4+ozTzmpYn+4+DJwxs/3hpDuAJ7Jmq/j6jNPOalifad5B7hJIxddnmpzt\nrLL1WZK1Gl2TwczeByWdEmFNZLXzLuBXzewKMAO83df+cOEdwF+H295m4Kvufl8Vrs847ayG9Qnw\n68BXwp/uzwHvrcL1GaedVbE+wy/11wK/kjat6tZnjHZWxfosB53WQESkhumIVxGRGqaQFxGpYQp5\nEZEappAXEalhCnkRkRqmkBcRqWEKeRGRGvb/A9umHtUAwMsGAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ax=plt.gca()\n", "ax.contourf(coords[:,:,0],coords[:,:,1],pdf[:,:,0])\n", "ax.plot(X[0:50,0],X[0:50,1],'x')\n", "plt.title('Contour plot of just Gaussian 0, with scatter plot overlaid')" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Principal Components\n", "It's going to be useful, in a few minutes, if we know the principal components of each of these three Gaussians. Remember that the PCA directions, $\\vec{v}_{yi}$, and variances, $\\lambda_{yi}$, are the eigenvectors and eigenvalues of the covariance matrix:\n", "$$\\lambda_{yi},\\vec{v}_{yi}=\\mbox{eig}(\\Sigma_y)$$\n", "\n", "which means that\n", "$$\\Sigma_y \\vec{v}_{yi}=\\lambda_{yi}\\vec{v}_{yi}$$" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VfWd//HXJyFhzwIJW9hkVVRcSAGFWkVtCwatVq1V\nVNSK2Kpjp9PF6UzHdqadqb/OdHNGRK27VVRUiCBqXSq2IIugVJR9B9kSwk6Wz++Pc6DxmpCbcJN7\nw3k/H488cnPP95zzud+b+77nfs9yzd0REZHjX1qyCxARkaahwBcRiQgFvohIRCjwRUQiQoEvIhIR\nCnwRkYhQ4EtSmdl/mtmdya6jLmb2RTP7JMk1vGdmJ8fZNt/MPjaz1k1QV28zczNrEf4908yub+z1\nJnJdZna7mf0yETWlMgV+LcxsjZkdMrO8mPvfD/+5eyensmNjZt3MbEMS1vuImf1HzH35wHXA/eHf\n55pZlZntMbPdZvaJmd1QrX2mmd1tZsvNbG/4HP2hKZ4Ld3/H3Qc29nrq8CvgZ7F3mtkjNbT9EfCI\nu+8P27xlZgfCvt1uZlPNrGu1ZQw1sxlmVmpmO8M3lxtqWG6d3H20uz/akHmPJnzun2ikdT0AXGNm\nnRKwrJSlwD+61cA3D/9hZqcCbZJXTt3CF8XdR2kyBnilicoBwMzSa5k0HphxOJRCm9y9HZAF/BB4\nwMwGhdOeAy4GrgaygdOA+cD5jVH3YYe3XFPANOA8M+tigUlm1gvAzPLMbLKZtTGzlsD1wBMx898W\n9u0AIAf4dTjvWcAbwNtAP6AjcCvw1UQ/gBTqy89w9wPATIINkOOXu+unhh9gDfAvwLxq9/0K+DHg\nQO/wvpbh/euAT4FJQOtwWi5QDGwDSsLb3ast7y3g34F3gd3Aq0BeOK0VwQt2B1AKzAM6x1H33cDd\nR5k+Fbgszj4YA3wU1rYR+Kdq074PbAY2ATeGfdIvnPYIcB8wA9gLTADKgUPAHmB62O4NYFy1ZZ4L\nbIipYRtwOXABsB/oEWftPwSei7nvt8Dvwts3AEvDx7YKuCW2jnAZW4DHY2sj2IJeGc7/EXBptWnj\ngdnh/0UJwYbD6GrTOwAPh31XArxYbVoRsCh8zv8CDI55DK8B14e3e4V9vRKYcrgtcA6wIma+t4Bv\nVfv7O8CS8PZs4H/r8dpIDx/b9rDvvhM+/y1i1xX2xbsEby47gP8I778x7P8SYBbQq9ryTw4f506C\n19Q/E7z5HAr/j/YAi2tYVxrBa3YtsBV4DMgOp/UOa7ye4LW6HfhxzOO6BngzmbnT2D/awj+6OUCW\nmZ0UbqVexee3mv6LYIvpdIKtowLgJ+G0NIIXdi+gJ0Fg3Rsz/9UE4dMJyAT+Kbz/eoKt2B4EW1wT\nw/kbzMwyCMLgtThneYggCNsDpxAENGb21bDOC4H+BGEc62rg50B7ghfek8A97t7O3ceGbU4FahwX\nN7M0M7uUYEv0w3Ad77n7+jhrfxoYY2btw+WlA1cCT4XTtxKEaxZB///azM6sNn8XgmDuRfCGFWsl\n8EWC5+inwBPVh0iAYeFjywPuAR4yMwunPU7wSfFkguf98Jb2GcAfgFsInvP7gWnhFvthSwk+2Rzm\ngIW/q8L7au3XcD15wNeB982sDXAWwaeneN1M0HdnAIUEb8hHM4zgjaEz8HMzu4QgxC8D8oF3gD+G\ntbUHXif4FNqN4DX1J3d/BfgF8Ez4P3Ta59YSvLmMB84D+gDt+PzrbSQwkOBT4U/M7KRq02L79rij\nwK/b4wQf8y4k+IfYeHhC+AKeAHzX3Xe6+26Cf8qrANx9h7s/7+77wmk/B74Us/yH3X2ZB8MaUwje\nOCDYkulIsNVc6e4L3L3sGB/LOQRbRrvjbF8ODDKzLHcvcfeF4f1XhnUvcfe9BJ8qYr3k7u+6e5UH\nH5drkkOwhVxdNzMrJdgC+zfgWnf/hKAvNsdZN+6+FlgIXBreNQrY5+5zwukvu/tKD7xN8Onqi9UW\nUQX8m7sf9M8OOR1e/rPuvil8fM8Ay4Gh1ZqsdfcH3L0SeBToCnQO3xRGAxPDPi0P1w/B/9L97j43\nfM4fBQ4Cw6stdzeQE/7v3UXQ938m2Mq+IwzwmvoV4Hdh3y4m6Mt/JPgUmkY9+pbg+f+Nu693953A\nf9bRfpO7/97dK8K+nAj8p7svdfcKgtfM6eHwVBGwxd3/290PuPtud58bZ13XAP/j7qvcfQ9B/1wV\nM4z0U3ff7+6LCfqhesDvJngDP24p8Ov2OMHW6niCLdXq8gm21BaEO7tKCbZM8gHC8dT7zWytmZUR\nvDBzYsa0t1S7vY9gq+TwemcBT5vZJjO7J9xC/xwzK662/h8BPzr8t5kVV2s6hmCYBTO7JtyBt8fM\nZtby2L8ezrPWzN4Ox3oh2PKqvqW9toZ549kSLyH4BFDdJnfPcfcO7n66uz8d3r+DIDTr4yn+vg/m\nav6+dY+ZjTazOeEOylKCx1l9B/22o7xRYWbXmdmiav1+Ssz8R55Xd98X3mxH8Iltp7uX1LDYXsD3\nqj13pWH7btXatAdKwzeqieEbG+6+3d0nhOuqqV8B7gj7tsDdr3H3w0ONVdSvb+N5/quL/V/oBfy2\n2mPcSfAppYDg8a6sRy2xdVWvZS3QguCTxWG1vd4g6LNdDVx3s6DAr0P4glpNEAhTYyZvJxhmOTl8\nIeW4e7YHO8YAvkfw8XGYu2cRbGFD8M9d13rL3f2n7j4IOJtgy6fGHUruXnR4/QRDTP9VrZ6iak2P\nBL67Pxl+NG7n7qNrWe48d7+EYNjhRYJPIBBsDfao1rRnTbPX8TfABwTDYfF4HRhqZt3jbA/wLHBu\nOM+lhIEfDpE8TzAO3Tnstxl89nmp9TKy4ZboA8BtQMdw/iXE8bwShF8HM8upZdrPqz13Oe7ext3/\nWK3NSQRbpn8v1H18zHLi7tfwDeKvBG/u8Yrn+f/MamL+Xk8wVFj9cbZ297+E0/rEuZxYmwjeTKrX\nVUGwHyAen+vb440CPz43AaPC4Ysj3L2K4IX/68OHc5lZgZl9JWzSnuANodTMOhAMUcTFzM4zs1PD\nTwNlBMMrVXXMdrTlnQC0dPelcbbPDD8FZLt7eVjD4fVPAcab2aBwCCGex/Upn38hz+DzQ1w1cvfX\nCfY9vGBmQ8yshZm1N7OJZnZjLfNsI9ip9zCwutpjzyTY2b4NqDCz0cCX46kj1JYgfLYBhIcvnhLn\n49hMcDTI/5lZrpllmNnhDYEHgIlmNiw8CqetmV1UbT9EK2AIde+DeY/gk2RBnI/nBwTP5/fNrGO4\nrtPM7Ola2k8hGD7qbma5BJ8q62MScJeF5xSYWbaZXRFOKwa6mtmdZtYyfI6HhdM+BXqbWW259Ufg\nu2Z2gpm14+9j/hVx1vUlgufmuKXAj0M41ju/lsk/BFYAc8Jhm9cJtuoBfgO0JvgkMIf6HQ7ZhWBH\nWhnBvoO3CYZ5Guoiwq37ergWWBM+rokEY6S4+0yCx/YGwWN/I45lPUSwP6DUzF4M73uMYMdqvCcH\nXU7wGJ4h+Oi9hGCn4etHmecpgh2+R4Zzwn0YdxAEVwnBcM+0OGvA3T8C/ptgy/hTgp2k78Y7P0G/\nlgMfE+w8vjNc7nyCHaL3hnWtIBhKPGws8Ja7b6qjvkMER++Mi6eYcMt6VPizysx2ApOp/f/lAYLh\nxsUE+0liP/nWtb4XgF8SDFeWETyPo8Npuwn2l40lGH5ZTrATFoJPbAA7zGwhn/cHgtfInwk+lR8A\nbo+npvDNdAzB/pbjlrnrC1CiwMxmAPe6e31DP97lO9Df3VfUc75fAFvd/TeNUdfxxMzmAje5+5I4\n2h4++uWMmnY6y2eZ2e0Eh/z+INm1NCYFfkSY2Q+A3zfWi7+hgS8iTSeuwDezNQSHLFUCFe5eGDPd\nCE5qGUOw53u8//0QPokABb5I6qvPac7nufv2WqaNJjgBpz/BSRb3hb8lItw9niNURCSJErXT9hLg\nsfDY4DkERwjU95hpERFpRPFu4TvwuplVEpwJODlmegGfPbliQ3jfZ87eM7MJhKept23bdsiJJ57Y\noKJFRKJqwYIF2909vyHzxhv4I919Y3is+Wtm9rG7/7m+KwvfKCYDFBYW+vz5tR3pKCIiNTGzus5s\nrlVcQzruvjH8vRV4gc9eMwSC68tUP/OuO9WuOSMiIslXZ+CHZ/sdPtOvLcEZibHHAU8DrgvPDhwO\n7ArPKBQRkRQRz5BOZ4LT2Q+3f8rdXzGziQDuPongjLwxBGcG7iO43KyIiKSQOgPf3VdRwzWiw6A/\nfNsJLs8qIiIpStfSERGJCAW+iEhEKPBFRCJCgS8iEhEKfBGRiFDgi4hEhAJfRCQiFPgiIhGhwBcR\niQgFvohIRCjwRUQiQoEvIhIRCnwRkYhQ4IuIRIQCX0QkIhT4IiIRocAXEYkIBb6ISEQo8EVEIkKB\nLyISEQp8EZGIUOCLiESEAl9EJCIU+CIiEaHAFxGJCAW+iEhEKPBFRCIi7sA3s3Qze9/MimuYdq6Z\n7TKzReHPTxJbpoiIHKsW9Wj7D8BSIKuW6e+4e9GxlyQiIo0hri18M+sOXAQ82LjliIhIY4l3SOc3\nwA+AqqO0OdvMPjCzmWZ28rGXJiIiiVRn4JtZEbDV3RccpdlCoKe7DwZ+D7xYy7ImmNl8M5u/bdu2\nBhUsIiINE88W/gjgYjNbAzwNjDKzJ6o3cPcyd98T3p4BZJhZXuyC3H2yuxe6e2F+fv6xVy8iInGr\nM/Dd/S537+7uvYGrgDfcfVz1NmbWxcwsvD00XO6ORqhXREQaqD5H6XyGmU0EcPdJwOXArWZWAewH\nrnJ3T0yJIiKSCJasXC4sLPT58+cnZd0iIs2VmS1w98KGzKszbUVEIkKBLyISEQp8EZGIUOCLiESE\nAl9EJCIU+CIiEaHAFxGJCAW+iEhEKPBFRCJCgS8iEhEKfBGRiFDgi4hEhAJfRCQiFPgiIhGhwBcR\niQgFvohIRCjwRUQiQoEvIhIRCnwRkYhQ4IuIRIQCX0QkIhT4IiIRocAXEYkIBb6ISEQo8EVEIkKB\nLyISEQp8EZGIUOCLiERE3IFvZulm9r6ZFdcwzczsd2a2wsw+MLMzE1umiIgcq/ps4f8DsLSWaaOB\n/uHPBOC+Y6xLREQSLK7AN7PuwEXAg7U0uQR4zANzgBwz65qgGkVEJAHi3cL/DfADoKqW6QXA+mp/\nbwjv+wwzm2Bm881s/rZt2+pVqIiIHJs6A9/MioCt7r7gWFfm7pPdvdDdC/Pz8491cSIiUg/xbOGP\nAC42szXA08AoM3sips1GoEe1v7uH94mISIqoM/Dd/S537+7uvYGrgDfcfVxMs2nAdeHROsOBXe6+\nOfHliohIQ7Vo6IxmNhHA3ScBM4AxwApgH3BDQqoTEZGEqVfgu/tbwFvh7UnV7nfgO4ksTEREEktn\n2oqIRIQCX0QkIhT4IiIRocAXEYkIBb6ISEQo8EVEIkKBLyISEQp8EZGIUOBL/ZUfSHYFItIACnyp\nny1L4PdnwvLXk12JiNSTAl/it3EhPFoE7pDbK9nViEg9KfAlPuvmwmOXQMv2cMMMyOuf7IpEpJ4U\n+FK3VW/D45dCu05ww0zocEKyKxKRBlDgy9Etfw2euhJyesL4GZDdPdkViUgDKfCldkunwx+/CXkD\nYPzL0L5zsisSkWOgwJeaffgcTLkeup0O10+Hth2TXZGIHCMFvnzewsfh+W9Bz7Pg2hegdU6yKxKR\nBFDgy2e99wBMuw36ngfXPBsclSMixwUFvvzdu7+DGf8EA8fAN5+GzDbJrkhEEqjBX2IuxxF3ePse\neOsXcPKlcNkDkJ6R7KpEJMEU+FHnDq/fDe/+Bk67Gi65F9LSk12ViDQCBX6UVVXBKz+C9+6Hwptg\nzK8gTaN8IscrBX5UVVVC8Z2w8DEY/h34ys/BLNlViUgjUuBHUWUFvHgrfDgFzvk+nPdjhb1IBCjw\no6biEDx/EyydBqP+Fc75p2RXJCJNRIEfJeUHYMp1sHwWfOU/4axvJ7siEWlC2kOXon416xP+MHs1\n7p6YBR7aG1wEbfmrUPRrhb1IBNUZ+GbWyszeM7PFZvY3M/tpDW3ONbNdZrYo/PlJ45QbDZVVzvKt\nu/lZ8Ufc8fQi9h6sOLYFHiiDJ74Oa96Br90HhTcmplARaVbiGdI5CIxy9z1mlgHMNrOZ7j4npt07\n7l6U+BKjJz3NmDRuCJPeXsX/m/UxH28uY9K1Q+ib367+C9u3Mwj7LR/A5X8ITqwSkUiqcwvfA3vC\nPzPCnwSNM0htzIxbz+3L4zcNY8feQ1xy77u8smRL/RayZxs8OhY+XQLfeEJhLxJxcY3hm1m6mS0C\ntgKvufvcGpqdbWYfmNlMMzu5luVMMLP5ZjZ/27Ztx1B2dIzol0fx7SPp26kdE59YwH/N/JiKyqq6\nZyzbDI+MgR0r4epnYODoxi9WRFJaXIHv7pXufjrQHRhqZqfENFkI9HT3wcDvgRdrWc5kdy9098L8\n/PxjqTtSuuW0Zsotwxk3vCeT3l7JtQ+9x/Y9B2ufoXQdPDwayjbBuOeh76imK1ZEUla9jtJx91Lg\nTeCrMfeXHR72cfcZQIaZ5SWsSqFli3T+42un8qsrTmPhuhKKfjebhetKPt9wx0p4eAzs3wnXvQS9\nRzR9sSKSkuI5SiffzHLC262BC4GPY9p0MQtO1TSzoeFydyS+XLl8SHemfvtsMloY37j/rzw+Z+3f\nD93c+nEQ9of2Bt9S1b0wucWKSEqJ5yidrsCjZpZOEORT3L3YzCYCuPsk4HLgVjOrAPYDV3nCDiCX\nWCd3y6b4ti/y3SmL+NcXl/D+uhJ+cRa0+uNlkNYCbpgBnU5KdpkikmIsWblcWFjo8+fPT8q6jxdV\nVc69b67gjT/N4PGWv6R122xa3FAMHfsmuzQRaSRmtsDdG/TxXWfaNmNpacYd/bbxXJtfUurtuGjP\nP/OnrQ04Vl9EIkGB35ytfBOe+DotsruRfuNMMjr25qZH5/Pfr35CZZVG1ETksxT4zdWyWfDUNyD3\nBLhhBt169uW5iWfzjcIe/P6NFYx/+D1K9h5KdpUikkIU+M3RRy/B09dA50EwvhjadQKgVUY6v7x8\nMP912anMXb2Tot/P5oMNpUkuVkRShQK/uVn8DDw7HgrODI6zb9Phc02uGtqT5yaeBcDl9/2Vp99b\n18RFikgqUuA3JwsegRdugV4jYNxUaJVda9PB3XOYfvtIhvXpwI+mfsgPn/uAA+WVTVeriKQcBX5z\nMWcSTP8H6HcBXPMstKz7aJwObTN55Iah3D6qH8/MX88Vk/7K+p37mqBYEUlFCvzmYPav4ZUfwolF\ncNWTkNE67lnT04zvfXkgD15XyJodexl772zeXqYL14lEkQI/lbnDm7+A1++GUy6HKx6BFi0btKgL\nBnVm+m0j6ZLVivEPv8d9b61MaKkikvoU+KnKHV77V3j7l3DGOLhsMqRnHNMie+e15YVvj+DS0wvo\n1L5hbxwi0nzpS8xTUVUVzPw+zHsQvnAzjL4H0hLz3tw6M53/vvI0wmvdiUiEKPBTTVUlTLsDFj0B\nZ98BF/4MEhzOCvvjR1VlJWnp6ckuQ5oJDemkkspymHpzEPZf+lGjhL0cP5bP/Qv/d/PV7C2t4XsR\nRGqgwE8VFQeDE6qWPA8X3A3n3aWwl6PK7VbAwb17WT73L8kuRZoJBX4qKN8PT18NHxcH4/Ujv5vs\niqQZyOvRiw4FPVg2Z3ayS5FmQoGfbAf3wJNXwIo/wdjfwbBbkl2RNCMDho9kw9K/aVhH4qLAT6YD\nu+CJy2DtX4LDLodcn+yKpJkZOHwE7lWsmPfXZJcizYACP1n27YRHL4aNC+GKh2HwlcmuSJqhjj16\nkdutu4Z1JC4K/GTYsxUeKYKtS4NLJQy6JNkVSTNlZgwcPoL1f1vCvl3H36Ww3Z3ST/fxt3c28upD\nf2PTcg1dHQsdh9/Udm2Exy6Bso1wzRToc26yK5JmbsDwkcyZ+gwr5s1h8AVfTXY5x+RwwG9cVsqm\nZSVsXFbKvrLgi3zaZGXSe3DHJFfYvCnwm1LJWnh0bDCcM24q9Dor2RXJcSCvZ29yu3bjkzmzm13g\nuzslm/excVkJm5aXsnF5KfvDgG+bnUnBwFwKBuTQrX8OOZ3b6KTBY6TAbyrbV8BjF8OhvXD9S1Aw\nJNkVyXHCzBgwfCTvvfQc+8p20Sar9u9JSDavcnZu3huEexjy+3eXA9A2pyU9TsylYEAu3frnkN2p\ntQI+wRT4TeHTj4JhHK8KvpKwy6nJrkiOMwOGj2TuC1NYMe+vDD4/dbbyvcrZsWkvm5YHwzOblpdy\nYE8Q8O1yW9JzUEe6DcihYEAOWXkK+MamwG9smxfDY1+D9Mwg7PMHJrsiOQ7l9zqBnM5dWTbn3aQG\nvlc52zfuYdOycAt+RSkH91YA0L5DK3qf0pFuA4JhmvYdWyngm5gCvzGtnwdPfB1aZQXfP9uxb7Ir\nkuNUMKwzgnnTp7J/dxmt22c1yXqrqpwdG/awMdzBunlFKQf3BQGfldeKE07Lp6B/MAaflRf/F/dI\n41DgN5Y1s+Gpb0DbfLh+GuT0THZFcpw7PI6/Yt4cTh315UZZR1VlFds37DlyFM2mFbs4tD8M+PzW\n9DkjDPgBubTv0KpRapCGU+A3lrV/gayCYMs+q2uyq5EI6HRCX7I7d2HZ3HcTFvhVlVVsW7/nyA7W\nzctLOXSgEoDsTq3pN6QT3foHY/DtchXwqa7OwDezVsCfgZZh++fc/d9i2hjwW2AMsA8Y7+4LE19u\nM3LO92H4rdCyfbIrkYgwMwYMG8GCl19k/57dtG5X//+9ysoqtq3bHY7Bl7J5ZSnlYcDndG5D/y90\nDnay9s+lbY6+Na25iWcL/yAwyt33mFkGMNvMZrr7nGptRgP9w59hwH3h7+gyU9hLkxswfCTzpj3P\nynlzOOW8C+tsX1lZxba1u4Mt+GWlbFq5i4qDQcDndmnDwKFd6BYeB982WwHf3NUZ+O7uwJ7wz4zw\nx2OaXQI8FradY2Y5ZtbV3TcntFqRetjzzmwOLF1K3oSbk11Kk+ncpx9Z+Z1ZNmd2jYFfWVHF1jVl\n4SGSJWxeuYuKQ1UAdOjWlpOGd6FbeBx8m6zMpi5fGllcY/hmlg4sAPoB/+vuc2OaFADrq/29Ibzv\nM4FvZhOACQA9e2onpjSuvbPfYecTT5I1ZjSZ3bsnu5xjsnHPRtpnticr8+hH3xw+WmfhjGkc2LOH\njJZt+HRN2ZHj4Les3EVFeRDwHQvactLZ3Y6cydq6vQL+eBdX4Lt7JXC6meUAL5jZKe6+pL4rc/fJ\nwGSAwsLC2E8JIgnV4cYbKXnqj+x48EG63n13ssupt7JDZby65lWmr5zOwq0L+eEXfsi4QeOOOk9F\neSW53QZTVTmVKb94hn27+1JZXgUGHQvaMWhkNwoG5NK1fzat2yngo6ZeR+m4e6mZvQl8Fage+BuB\nHtX+7h7eJ5I0GZ07k33ZZex6fip5t95KRufOyS6pTuWV5byz8R2KVxXz1vq3KK8qp3dWb24/43bO\n73n+59pXHKpky+qyIxca+3R1GRXllZDWnt1bl3Dal885Mgbfqm1GEh6RpJJ4jtLJB8rDsG8NXAj8\nMqbZNOA2M3uaYGftLo3fSyroePO3KH3uOXb+4Q90vuuuZJdTI3dn8bbFFK8q5pU1r7Dr4C46tOrA\nlQOvZGyfsQzqOOjIGanlhyrZsmrXkTNZP11TRlWFYwZ5PdpzyrkFFAzIZeW88/jwTzMovKgrrdq2\nS/IjlFQRzxZ+V+DRcBw/DZji7sVmNhHA3ScBMwgOyVxBcFjmDY1Ur0i9ZHbvTnZRESXPTKHjLbfQ\nokOHZJd0xPqy9RSvKqZ4VTHrdq+jZXpLRvUYRVHfIs7qdhYZaRmUH6xkw8clR46D/3R1GVWVQcDn\n92zP4PN6UDAgh679cmjZ+u8v55atzmHxq9NYteA9Bp0zKomPUlJJPEfpfACcUcP9k6rdduA7iS1N\nJDE63jKBXdOmsfORR+n0j8n9gvjSA6XMWjOL6aums3jbYgxjaJehfOvUb3FhrwvJrGrFllW7mD9/\nHZuWlbJ1TRlVVY6lGZ16tee083sEY/B9s8lsXfvLt2v/gbTvmM8nc2Yr8OUInWkrx72WffrQ/qtf\noeTJJ+l4042kZzft5YMPVR7i7Q1vM33ldN7Z+A4VVRX0y+nHnWfeyZe7fZWqLa3YtKSEV6Z+zLa1\nu6mqctLSjPxe7Tn9wp4UDMihS99sMlvF/3INjtY5m0WzXubgvr20bNO2ER+hNBcKfImEvFtuYffM\nV9j5xBPkf6fxP4xWeRXvb32f4lXFzFozi92HdpPXOo+r+4zjrLTzsU1t2fTyLqavW4ZXOWnpRqde\nWZzx5Z50G5BDlz71C/ia9B82kgUvv8TKBe8x6IvnJeiRSXOmwJdIaHXiibQ77zxKHnucDtePJ71d\n42zxrt61muJVxby86mU27tlIFrl8JfPrDKIQX9Ga7W/uYbHvJC29hM4nZDHkq73o1j8I+IyW6Qmt\npVv/gbTr0JFlc95V4AugwJcIyZt4C2u+cRWlzzxNx5tuSthydx7YyczVMyleWcyyT1dSsLsfwyou\npmB3fyq2puMO21tU0eWEFgwZ05uC/jl07pNNRmZiAz6WpaUxYNgIFr8+k4P79tGyTZtGXZ+kPgW+\nREbr006j7dlns+PhR8i95hrSWjX86o4HKg7w1vq3mPHxLNZ9vJ3OZSdw2t6vMbIsDzDSW6TRqU8W\n3b6QGwT8CVm0aOSAr0n/4SNYOHMaqxa+x0kjz23y9UtqUeBLpHSceAvrrrue0ueep8O4a+o1b5VX\n8ddV83jrvblsWV5GXmkvBuwrYiBGWgvo2jeXgi8Flwru1DuLFhlNH/CxCgacRLvcDiybM1uBLwp8\niZY2X/gCrYcMYceDD5J75RVY5tEvL7B/9yEWLvqYBe9/zJ61VWTv7UQOp5CVXkl2rwxOPLkn3Qd2\noHOvLNIz0proUcTP0tLoP2wEH/zpFQ7t30dmaw3rRJkCXyLFzMibOJH1N99M6UsvkXvFFZ+Zvq/s\nEJuWl7L44ng/AAAL+UlEQVTyo82sWrqFqp3B5Qiq0tqTll9C7pnlnP2FM+jRJ4/0FqkX8DUZMGwE\n778ynVUL53HiiC8luxxJIgW+RE7bkSNodfLJ7HjgQXIuvZRVH+5k/dISNizbya4t+wE4lHaALVmr\nqDxxN6ec2ocxw0bRqV1+kitvmG4nnkTbnFyWzXlXgR9xCnyJHDMj79aJbLjtdkpffpnZH2VTtukQ\nm9qtYEPPTzjYuZSRpw3hln5F9Mnpk+xyj1laWjr9h53Nkjde49CB/WS20peJR5UCXyJp4+kF7O6e\ny+Zf/Qv/d2070numceEJF3BHn3EM6TyENGsewzXxGjB8JItmvczq9+cz8KwvJrscSRIFvkTGp3s/\nZcbqGRSvKmZZyTK+OMS4/aUK/l+bSxn6jTto1eL4/RLughMH0SY7h2V/na3AjzAFvhzX9pbv5fW1\nr1O8qpi5m+fiOIPzBnPX0Lv4yuUXUrroWrpPnUPLa76f7FIbVVpaOv2Hns3f3v4T5QcOkHEM5yBI\n86XAl+NORVUFczbPYfrK6byx7g0OVB6goF0BEwZPoKhPEb2zex9pmz5hApt//GP2/vnPtPvS8b1D\nc8DwkSx+bQarF81nwPCRyS5HkkCBL8cFd2fpzqVMXzmdmatnsuPADrIysxjbdyxj+47l9PzTj3yJ\nSHXZF49l2//ey/b7JtH2nHNqbHO86H7SybTOyuaTOe8q8CNKgS/N2uY9m3l59csUryxm5a6VtEhr\nwZe6f4miPkWc0/0cMtOPfmKVZWTQ8Vvf4tOf/Tv75r5H2+HDmqjyppeWnk7/oWfx0TtvUn7wABkt\nNawTNRZ8d0nTKyws9Pnz5ydl3dK87T60m9fXvs70VdOZt2UeAGd0OoOiPkV8pfdXyG5Zv+vdVx08\nyIoLLqBl3370euThxig5ZezYsI69paV0H3QyaWnJv/SD1J+ZLXD3wobMqy18aRbKq8r5y8a/MH3V\ndN5a/xYHKw/Ss31Pvn36tynqU0SP9j0avOy0li3peMONbL3nHva9/z5tzvjcF7wdNzp270nH7j2T\nXYYkiQJfUpa7s2T7Eqavms4rq1+h5GAJOS1zuLTfpRT1LWJw3uCEjbnnfuNKdkyezI5J99Pm/kl1\nzyDSDCnwJeVs2L3hyJeIrClbQ2ZaJuf2OJeiPkWMLBhJRnpGwteZ1rYtHcZfz7bf/JYDH31Eq0GD\nEr4OkWRT4EtK2HVwF6+ufZXilcUs3LoQgCGdhzD+5PFc2PtCsjKzGr2G3KuvZseDD7F90v10/91v\nG319Ik1NgS9JU15Zzp83/pnilcW8veFtyqvKOSH7BO444w7G9BlDQbuCJq0nPSuL3HHXsGPS/Rxc\nsYKW/fo16fpFGpsCX5qUu7N422Kmr5zOrLWz2HVwFx1adeDKgVcyts9YBnUclNRj4Ttcfz07H32M\n7ZMnU3DPPUmrQ6QxKPClSawrW0fxqmKKVxWzfvd6Wqa3ZFSPURT1LeKsbmeRkZb4cfmGaJGbS+5V\nV7Hz0UfJv+02MnvqiBY5fijwpdGUHChh1ppZTF81nQ+2fYBhDO0ylAmDJ3BBzwtol9ku2SXWqMMN\n4yl58kl2PPAAXf/935NdjkjCKPAlocqrynlz3ZtMXzWd2RtmU+EV9Mvpx51n3slFfS6iS9suyS6x\nThmdOpFz+dcpefY58r79bTK6dk12SSIJocCXxHL42ZyfkZGWwdUnXc3YvmMZmDuw2V2jpuNNN1Ey\n5Vl2PPQHuvzLj5NdjkhC1Bn4ZtYDeAzoDDgw2d1/G9PmXOAlYHV411R3/1liS5XmICM9g8dGP0bP\n9j1pkdZ8tycyCgrIvvhiSp99lryJt9AiLy/ZJYkcs3hekRXA99x9oZm1BxaY2Wvu/lFMu3fcvSjx\nJUpz0ye7+X8tIEDehJvJ6NIZyzz6BdhEmos6A9/dNwObw9u7zWwpUADEBr7IcSWzd2/y77gj2WWI\nJEy9vrjTzHoDZwBza5h8tpl9YGYzzezkBNQmIiIJFPcgq5m1A54H7nT3spjJC4Ge7r7HzMYALwL9\na1jGBGACQE8d3ywi0qTi2sI3swyCsH/S3afGTnf3MnffE96eAWSY2ef2crn7ZHcvdPfC/Pz8Yyxd\nRETqo87At+B4uoeApe7+P7W06RK2w8yGhsvdkchCRUTk2MQzpDMCuBb40MwWhff9M9ATwN0nAZcD\nt5pZBbAfuMqT9VVaIiJSo3iO0pkNHPWsGXe/F7g3UUWJiEji1esoHRERab4U+CIiEaHAFxGJCAW+\niEhEKPBFRCJCgS8iEhEKfBGRiFDgi4hEhAJfRCQiFPgiIhGhwBcRiQgFvohIRCjwRUQiQoEvIhIR\nCnwRkYhQ4IuIRIQCX0QkIhT4IiIRocAXEYkIBb6ISEQo8EVEIkKBLyISEQp8EZGIUOCLiESEAl9E\nJCIU+CIiEaHAFxGJCAW+iEhE1Bn4ZtbDzN40s4/M7G9m9g81tDEz+52ZrTCzD8zszMYpV0REGqpF\nHG0qgO+5+0Izaw8sMLPX3P2jam1GA/3Dn2HAfeFvERFJEXVu4bv7ZndfGN7eDSwFCmKaXQI85oE5\nQI6ZdU14tSIi0mDxbOEfYWa9gTOAuTGTCoD11f7eEN63OWb+CcCE8M+DZrakPutPkjxge7KLiIPq\nTKzmUGdzqBFUZ6INbOiMcQe+mbUDngfudPeyhqzM3ScDk8PlzXf3woYspympzsRSnYnTHGoE1Zlo\nZja/ofPGdZSOmWUQhP2T7j61hiYbgR7V/u4e3iciIikinqN0DHgIWOru/1NLs2nAdeHROsOBXe6+\nuZa2IiKSBPEM6YwArgU+NLNF4X3/DPQEcPdJwAxgDLAC2AfcEMdyJ9e72uRQnYmlOhOnOdQIqjPR\nGlynuXsiCxERkRSlM21FRCJCgS8iEhFNEvhmlm5m75tZcQ3TUuKyDHXUeK6Z7TKzReHPT5JRY1jL\nGjP7MKzjc4dnpVB/1lVn0vvUzHLM7Dkz+9jMlprZWTHTU6Uv66ozFfpyYLX1LzKzMjO7M6ZN0vsz\nzjqT3p9hHd8NL2ezxMz+aGatYqbXvz/dvdF/gH8EngKKa5g2BpgJGDAcmNsUNdWzxnNruj9Jda4B\n8o4yPVX6s646k96nwKPAt8LbmUBOivZlXXUmvS9j6kkHtgC9UrE/46gz6f1JcOLqaqB1+PcUYPyx\n9mejb+GbWXfgIuDBWpok/bIMcdTYnCS9P5sDM8sGziE45Bh3P+TupTHNkt6XcdaZas4HVrr72pj7\nk96fMWqrM1W0AFqbWQugDbApZnq9+7MphnR+A/wAqKplem2XZWhKddUIcHb4sWmmmZ3cRHXVxIHX\nzWyBBZeqiJUK/Ql11wnJ7dMTgG3Aw+FQ3oNm1jamTSr0ZTx1Qur8fwJcBfyxhvtToT+rq61OSHJ/\nuvtG4FfAOoJL1Oxy91djmtW7Pxs18M2sCNjq7gsacz3HIs4aFwI93X0w8HvgxSYprmYj3f10giuU\nfsfMzkliLUdTV53J7tMWwJnAfe5+BrAX+FET1xCPeOpMdl8eYWaZwMXAs8mqIR511Jn0/jSzXIIt\n+BOAbkBbMxt3rMtt7C38EcDFZrYGeBoYZWZPxLRJ9mUZ6qzR3cvcfU94ewaQYWZ5TVhj9Vo2hr+3\nAi8AQ2OaJLs/gbrrTIE+3QBscPfDFwJ8jiBYq0uFvqyzzhToy+pGAwvd/dMapqVCfx5Wa50p0p8X\nAKvdfZu7lwNTgbNj2tS7Pxs18N39Lnfv7u69CT4+veHuse9SSb0sQzw1mlkXM7Pw9lCCftvRVDVW\nq6OtBd9JQPix/stA7BVHk36Zi3jqTHafuvsWYL2ZHb7y4PnARzHNkt6X8dSZ7L6M8U1qHyZJen9W\nU2udKdKf64DhZtYmrOV8gkvTV1fv/qzX5ZETxcwmwjFdlqHRxdR4OXCrmVUA+4GrPNxN3sQ6Ay+E\n/4stgKfc/ZUU7M946kyFPr0deDL8eL8KuCEF+zKeOlOhLw+/uV8I3FLtvpTrzzjqTHp/uvtcM3uO\nYHipAngfmHys/alLK4iIRITOtBURiQgFvohIRCjwRUQiQoEvIhIRCnwRkYhQ4IuIRIQCX0QkIv4/\ndmPLmiZYE+oAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Find the principal component directions and variances of each Gaussia\n", "Lambda, V = np.linalg.eig(Sigma)\n", "\n", "# Let's show the mean vectors plus/minus each variance-scaled principal component direction\n", "ax=plt.gca()\n", "plt.xlim(4,8)\n", "plt.ylim(2,5)\n", "plt.title('Means +/- sqrt(PC variance)*(PC direction)')\n", "for y in range(0,3):\n", " for i in range(0,2):\n", " x0 = mu[y,0]+np.sqrt(Lambda[y,i])*np.array([-V[y,0,i],V[y,0,i]])\n", " x1 = mu[y,1]+np.sqrt(Lambda[y,i])*np.array([-V[y,1,i],V[y,1,i]])\n", " ax.plot(x0,x1)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Now initialize the GMM\n", "\n", "Now we're going to initialize the GMM. We'll create two Gaussians per class. Each of the two Gaussians will have exactly the same covariance matrix and exactly the same mixture weight, but they will have slightly different mean vectors, set at $$\\vec\\mu_{y0}=\\vec\\mu_y+0.1\\sqrt{\\lambda_{y0}}\\vec{v}_{y0}$$\n", "\n", "and at $$\\vec\\mu_{y1}=\\vec\\mu_y-0.1\\sqrt{\\lambda_{y0}}\\vec{v}_{y0}$$\n" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEICAYAAAC3Y/QeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHihJREFUeJzt3XuUHWWd7vHvQwi3cEkkLYSEJAhjBEIgnDYwyAEmw105\nyCjIRRAEMQoojIwyOseBUWYxs1TQ4UAERGAQmAyKYiQGjtzEmMQGkpAA8XBNiEE6IQHCTcHf+aPe\nhsrO7t7V3bt77+56Pmv1yq6qt6p+u+qtp2rXru4oIjAzs3LZqNEFmJlZ/3P4m5mVkMPfzKyEHP5m\nZiXk8DczKyGHv5lZCTU0/CXNkvSpLqZPl/S/Cy7rXkln1K86K0LSMZKWS1onaXIfrSMk7dJHyz5V\n0gN9seyBKu3L9zVBHUskHdToOvqbpAsl3ViwbY9zr+7hL+kZSQcXaRsRR0TE9Wm+DQ7CiJgWEd+o\nd42DWQPC7FvA2RGxZUQ83NuF9eVJXNL4dCLZuC+W3xOSNpH0dUlLJb0qaUW6KDq0UTWlfflUo9af\nq2P3iLi3VjufwHumaQ4Caw6SNo6It7oxyzhgSQ/XNSQi3u7JvI0gSYAi4i91XOytwGjgFKDj5DkV\n+DBwZx3XY53oQZ8fHCKirj/AM8DB6fWpwANkV4drgKeBI3Jt7wXOAHYF3gDeBtYBa9P064Bvptcj\ngJlAe1rWTGBM5bI6qelC4L+BG4FXgEeA9wP/CLwALAcOzbXfBvgBsBJYAXwTGJKm7QzcDawGVgE/\nAoZXvP/zgUXAS8B/AZtVqWlTYC0wMTeuBXgdeG8a/giwILWbA0zKtd0R+EnaHquBy7vYjtsAN6S2\nzwL/BGyU20e/AS5Ny/kmsAtwX6p/FfBfndS/DgjgVeDJNH7XtC/Wkp0U/ldunuuAK4E70jwHVyzz\n4lT7G2nZl6fxAUwD/l9a7v8hC+GO+T4NPJb6xWxgXCf9YFla1rr089cU66MXp230eto2nfaPbtZz\ncFrmmGrTc+0uAJ4k67uPAsdU9O0bc8Pj03vcOLd/n0rzPg2clMZ3uo/T/Luk1x8mOym9THacXFhl\nXZ9K23YV8LUu3sd1wBXArLT9fwNsD1yWttXjwOROsuQO4Nu5abcA19J5n7+XXB507OeK93gWWZ96\nOo37AHAX8CKwFDiui/dyb9rvc9J6fw5sS5YHLwO/A8bn2u+Xxr2U/t0vN22ntC9eSeu/vGKf7pvW\nsxZYCBxULfe62qdV30NPAr5GR83vsFOBPwOfAYYAnwP+QDpwKwpfb+fkOktH+G8LfAzYAtiKLMx/\nWm0jVKnpwtRBDiP7tHMD2YHwNWBoqu/pXPvbgO8Dw4D3AvOBz+Y28CFk4dcC3A9cVvH+5wM7AO8h\nC4FpndR1LXBxbvgs4Jfp9WSyE9M+adt9Ki170zS8kCywhwGbAft3sR1vAH6Wttt44PfA6bn2bwHn\npG2zOXBz2jYb5ZfdyXvIB8VQ4Angq8AmZFewrwATcvvzJeBDHcvu5KA6o8o6ZgLDgbFkJ7HD07Sj\n0zp3TfX/EzCnk1rHkwvGbvTRZcDuaflDa/SP7tRzCXBvgWPq2NSfNgI+QXbiHJXr21XDP9X3cm77\njwJ2T6873ccV+/QgYI/UbhLwR+CjFeu6OvWbPYE3gV07eR/XkYXS/0jrvJvsODwlbftvAvd0kiXb\nkx0PU4GTyE5oW3XR5++ldvjfRXaMbp621XLgtLTtJqdad+vkvdyb9vPOZBcDj5IdVwfzbsb8MLV9\nD9nJ7eQ07YQ0vG2a/lvgO2TH9gFkx8yNadposouyI9M+OCQNt1TJ0MLHbUT/hP8TuWlbpI2+fZXC\nq+3A60jhX2U9ewFrugqN3LQLgbtyw0eRna07rua3SnUNB7Yj68Cb59qfQK5TViz7o8DDFe//k7nh\nfwemdzLvwaQr5jT8G+CU9PpK4BsV7ZcCB5JdsbaTC7EuOvkQ4E/5Tgx8lhQ6qf2yimXcAFxFjSvS\nKkHxP4HnSZ8qch3ywtz+vKHG8jbYj2kd+XCaAVyQXs8incjS8EbAa1S52qbz8K/VR/8lN73L/tHN\neq4BbskNv4fs6u4l4I0uttEC4Ohc3+4q/NeSXTRtXrGMTvdxfp9WmXYZcGnFuvKfwOcDx3cy73XA\n1bnhc4DHcsN7kK7cc8fSwbnhj5EF9KqK/nAqPQv/qbnhTwC/rljG94F/7qKffi03/G1gVm74KGBB\nen0yML9i/t+mmsaSXXwNy027iXfD/yvAf1bMOxv4VOX77GqfVvvpj6d9nu94ERGvpZdbdnchkraQ\n9H1Jz0p6meyKe7ikIQUX8cfc69eBVfHu/ebXc3WNI7u6WylpraS1ZJ3gvamO7STdkr6Ye5nsVtLI\ninU9n3v9Gp2/33uALSTtI2k82QnttjRtHPCljhpSHTuSXQHuCDwbxe5Tjkzv59ncuGfJrig6LK+Y\n58uAgPnpiYtPF1gPqbblsf498VrrKqqzbToO+G5uG71IVvtoiqvVR/M1d9k/ulnParKr8Y51vxgR\nw8mujDftGC/pFEkLcsucyIZ9bgMR8SpZqE1L9f5C0gfS5EL7OPXNeyS1S3opLaun/R02PA4rh7ua\n9+dkFzNLI6IeX/BW7td9Ko63k8g+cXSm6HvZgfWPP3j3uNiB7CL21Ypp+bqOrahrf3L9Jqdbx20z\nPecfNaZ/CZgA7BMRW5N9PILszdbTcrIru5ERMTz9bB0Ru6fp/5pq3SPV8cme1pBOPjPIrhxPAGZG\nxCu5Oi7O1TA8IraIiJvTtLGdPLVSuR1Xkd3WGJcbN5bsXnXVeSLi+Yj4TETsQPYp4YqCj1r+AdhR\nUr5fdbmuAvXXspzslkt+O20eEXPqsOxq89XqH92p51fAByWN6WzFksaR3VY5m+w2wXBgMe/2uVfJ\nPq10WC+sImJ2RBxCFhaPp2V1Zx/fBNwO7BgR2wDTqf8xV9TFZLdRR0k6ITe+2n7tcrtUmW85cF/F\nftsyIj7X66qz42JcxbiO42IlMELSsIpp+br+s6KuYRFxyQZvppvHbTOF/x+BMZI26WT6VmRn07WS\n3gP8c18UEREryZ6y+LakrSVtJGlnSQfm6lgHvCRpNPAPvVzlTWRXZyel1x2uBqalKy9JGibpw5K2\nIvtovRK4JI3fTNKH0nzrbcfcCeZiSVulMPl7sk8sVUk6NhdIa8gOkiJPuMwju/L7sqSh6Rnto8i+\nnCvqj0B3njGfDvyjpN0BJG0j6dhO2raTvY8eP8NeoH8Urici7iT79PfTtJ83kTSU7Au+DsPItn97\nWt5pZFf+HRYAB0gaK2kbsocYSG23k3R0CpY3yfrtX9K0ovt4K+DFiHhD0hTgxEIbqs4kHUB2P/4U\nsu+//iMdf1A9OxYAf5fuGOwCnF5jFTOB90s6OfXdoZI+KGnXOpR/R1r2iZI2lvQJYDeyi71ngTbg\norT/9yc7ZjrcCBwl6TBJQ9KxflC1C4buHrfNFP53kz0d8rykVVWmX0b2xcwqYC7wyz6s5RSyLywf\nJduIt/Lux6yLgL3J7sv+guyJmx6LiHlkVyk7kN0v7hjfRvYl5OWphifI7hF2BPpRZF8+LwOeIzuB\nQPXteE5ax1NkT7bcRPZlc2c+CMyTtI7squ+LUeC574j4U6rrCLL9dAXZdxiP15o357vAxyWtkfS9\nAuu8Dfg34JZ0G25xWn+1tq+RntxJH6H3rdaugE77R3fqSY4hC54bye7PP012IXBYWt6jZPeTf0sW\ncnuQfTfU8Z7uInuibBHwYFpWh43ITvR/ILv9dCDZF9pQfB9/HvgXSa8AXye7kOhXkrYmu599dkSs\niIhfkz1t9UNJonqfv5Tsu64/AteTPYXTqfSJ+1DgeLLt9TzZfty0q/mKiIjVZE/ufYnsVt+XgY9E\nREetJ5I92PEi2UXtDbl5l5M9RPBVsguA5WQXnNWyu1vHbccTDWZmViLNdOVvZmb9xOFvZlZCDn8z\nsxJy+JuZlVDD/rDbyJEjY/z48Y1avZnZgPTggw+uioiW3i6nYeE/fvx42traGrV6M7MBSVLlbwv3\niG/7mJmVkMPfzKyEHP5mZiXk8DczKyGHv5lZCQ3e8H/gMnj6/vXHPX1/Nt7MrOQGb/iP3hv++9R3\nTwBP358Nj967kVWZmTWFhj3n3+d2OgCOvS4L/NbToe0H2fBOB9SY0cxs8Bu8V/6QBX3r6XD/v2f/\nOvjNzIDBHv5P359d8R/w5ezfyu8AzMxKavCGf8c9/mOvg6lfe/cWkE8AZmaDOPxXPLT+Pf6O7wBW\nPNTIqszMmsLg/cJ3/3M3HLfTAb7vb2bGYL7yNzOzTjn8zcxKyOFvZlZCDn8zsxJy+JuZlZDD38ys\nhBz+ZmYlVDP8JW0mab6khZKWSLqoi7YflPSWpI/Xt0wzM6unIr/k9SYwNSLWSRoKPCBpVkTMzTeS\nNAT4N+DOPqjTzMzqqOaVf2TWpcGh6SeqND0H+DHwQv3KMzOzvlDonr+kIZIWkAX7XRExr2L6aOAY\n4MoayzlTUpuktvb29p7WbGZmvVQo/CPi7YjYCxgDTJE0saLJZcBXIuIvNZZzVUS0RkRrS0tLzyo2\nM7Ne69bTPhGxFrgHOLxiUitwi6RngI8DV0j6aF0qTKbf9yRznly13rg5T65i+n1P1nM1ZmalUORp\nnxZJw9PrzYFDgMfzbSJip4gYHxHjgVuBz0fET+tZ6KQx23D2TQ+/cwKY8+Qqzr7pYSaN2aaeqzEz\nK4UiT/uMAq5PT/NsBMyIiJmSpgFExPS+LLDDfjuP5PITJ3P2TQ/zyX3GcuO8ZVx+4mT223lkf6ze\nzGxQqRn+EbEImFxlfNXQj4hTe19WdfvtPJJP7jOW7939BF+YuouD38yshwbUb/jOeXIVN85bxhem\n7sKN85Zt8B2AmZkVM2DCv+Me/+UnTubvD53wzi0gnwDMzLpvwIT/oudeWu8ef8d3AIuee6nBlZmZ\nDTyKqPbLun2vtbU12traGrJuM7OBStKDEdHa2+UMmCt/MzOrH4e/mVkJOfzNzErI4W9mVkIOfzOz\nEnL4m5mVkMPfzKyEHP5mZiXk8DczKyGHv5lZCTn8zcxKyOFvZlZCDn8zsxJy+JuZlZDD38yshGqG\nv6TNJM2XtFDSEkkXVWlztKRFkhZIapO0f9+Ua2Zm9VDzP3AH3gSmRsQ6SUOBByTNioi5uTa/Am6P\niJA0CZgBfKAP6jUzszqoGf6R/Vdf69Lg0PQTFW3W5QaHVU43M7PmUuiev6QhkhYALwB3RcS8Km2O\nkfQ48Avg050s58x0W6itvb29N3WbmVkvFAr/iHg7IvYCxgBTJE2s0ua2iPgA8FHgG50s56qIaI2I\n1paWlt7UbWZmvdCtp30iYi1wD3B4F23uB94naWQvazMzsz5S5GmfFknD0+vNgUOAxyva7CJJ6fXe\nwKbA6vqXa2Zm9VDkaZ9RwPWShpCdLGZExExJ0wAiYjrwMeAUSX8GXgc+kb4oNjOzJqRGZXRra2u0\ntbU1ZN1mZgOVpAcjorW3y/Fv+JqZlZDD38yshBz+ZmYl5PA3Myshh7+ZWQk5/M3MSsjhb2ZWQg5/\nM7MScvibmZWQw9/MrIQc/mZmJeTwNzMrIYe/mVkJOfzNzErI4W9mVkIOfzOzEnL4m5mVkMPfzKyE\nHP5mZiVUM/wlbSZpvqSFkpZIuqhKm5MkLZL0iKQ5kvbsm3LNzKweNi7Q5k1gakSskzQUeEDSrIiY\nm2vzNHBgRKyRdARwFbBPH9RrZmZ1UDP8IyKAdWlwaPqJijZzcoNzgTH1KtDMzOqv0D1/SUMkLQBe\nAO6KiHldND8dmNXJcs6U1Caprb29vfvVmplZXRQK/4h4OyL2IruinyJpYrV2kv6GLPy/0slyroqI\n1ohobWlp6WnNZmbWS9162ici1gL3AIdXTpM0CbgGODoiVtenPDMz6wtFnvZpkTQ8vd4cOAR4vKLN\nWOAnwMkR8fu+KNTMzOqnyNM+o4DrJQ0hO1nMiIiZkqYBRMR04OvAtsAVkgDeiojWPqrZzMx6qcjT\nPouAyVXGT8+9PgM4o76lmZlZX/Fv+JqZlZDD38yshBz+ZmYl5PA3Myshh7+ZWQk5/M3MSsjhb2ZW\nQg5/M7MScvibmZWQw9/MrIQc/mZmJeTwNzMrIYe/mVkJOfzNzErI4W9mVkIOfzOzEnL4m5mVkMPf\nzKyEHP5mZiVUM/wlbSZpvqSFkpZIuqhKmw9I+q2kNyWd3zelmplZvdT8D9yBN4GpEbFO0lDgAUmz\nImJurs2LwBeAj/ZFkWZmVl81r/wjsy4NDk0/UdHmhYj4HfDn+pdoZmb1Vuiev6QhkhYALwB3RcS8\nnqxM0pmS2iS1tbe392QRZmZWB4XCPyLejoi9gDHAFEkTe7KyiLgqIlojorWlpaUnizAzszro1tM+\nEbEWuAc4vG/KMTOz/lDkaZ8WScPT682BQ4DH+7owMzPrO0We9hkFXC9pCNnJYkZEzJQ0DSAipkva\nHmgDtgb+IulcYLeIeLmvCjczs56rGf4RsQiYXGX89Nzr58m+DzAzswHAv+FrZlZCDn8zsxJy+JuZ\nlZDD38yshBz+ZmYl5PA3Myshh7+ZWQk5/M3MSsjhb2ZWQg5/M7MScvibmZWQw9/MrIQc/mZmJeTw\nNzMrIYe/mVkJOfzNzErI4W9mVkIOfzOzEnL4m5mVUM3wl7SZpPmSFkpaIumiKm0k6XuSnpC0SNLe\nfVOumZnVQ83/wB14E5gaEeskDQUekDQrIubm2hwB/FX62Qe4Mv1rZmZNqOaVf2TWpcGh6Scqmh0N\n3JDazgWGSxpV31LNzKxeCt3zlzRE0gLgBeCuiJhX0WQ0sDw3/FwaZ2ZmwEOzn+W5pWvWG/fc0jU8\nNPvZhtRTKPwj4u2I2AsYA0yRNLEnK5N0pqQ2SW3t7e09WYSZ2YD03vFbM/vqxe+cAJ5buobZVy/m\nveO3bkg93XraJyLWAvcAh1dMWgHsmBsek8ZVzn9VRLRGRGtLS0t3azUzG7DGTBjBYZ+ZyOyrFzPv\n9qeYffViDvvMRMZMGNGQeoo87dMiaXh6vTlwCPB4RbPbgVPSUz/7Ai9FxMq6V2tmNoCNmTCCiQeM\npu2OZ5h4wOiGBT8Ue9pnFHC9pCFkJ4sZETFT0jSAiJgO3AEcCTwBvAac1kf1mpkNWM8tXcPi+1fQ\neuR4Ft+/gtETRjTsBFAz/CNiETC5yvjpudcBnFXf0szMBo+Oe/wdt3pGTxjR0Fs//g1fM7N+8MIz\nL68X9B3fAbzwzMsNqafIbR8zM+ulvQ8bt8G4MQ287eMrfzOzEnL4m5mVkMPfzCyZ/7NbWbZ40Xrj\nli1exPyf3dqgivqOw9/MLNl+5/cz87JL3jkBLFu8iJmXXcL2O7+/wZXVn7/wNTNLxk6cxEfOvYCZ\nl13CnoceycI77+Aj517A2ImTGl1a3fnK38wsZ+zESex56JHM/fEt7HnokYMy+MHhb2a2nmWLF7Hw\nzjvY92PHs/DOOzb4DmCwcPibmSUd9/g/cu4FfOi4T75zC2gwngAc/mZmyfNP/n69e/wd3wE8/+Tv\nG1xZ/Sn7szz9r7W1Ndra2hqybjOzgUrSgxHR2tvl+MrfzKyEHP5mZiXk8DczKyGHv5lZCTn8zcxK\nyOFvZlZCDn8zsxKqGf6SdpR0j6RHJS2R9MUqbUZIuk3SIknzJU3sm3LNzKweilz5vwV8KSJ2A/YF\nzpK0W0WbrwILImIScArw3fqWaWZm9VQz/CNiZUQ8lF6/AjwGjK5othtwd2rzODBe0nZ1rtXMzOqk\nW/f8JY0HJgPzKiYtBP4utZkCjAPG9L48MzPrC4XDX9KWwI+BcyPi5YrJlwDDJS0AzgEeBt6usowz\nJbVJamtvb+9F2WZm1huF/rCbpKHATGB2RHynRlsBTwOTqpwk3uE/7GZm1n399ofdUpj/AHiss+CX\nNFzSJmnwDOD+roLfzMwaq8j/4fsh4GTgkXRbB7Kne8YCRMR0YFfgekkBLAFO74NazcysTmqGf0Q8\nAKhGm98Cg++/tzczG6T8G75mZiXk8DczKyGHv5lZCTn8zcxKyOFvZlZCDn8zsxJy+JuZlZDD38ys\nhBz+ZmYl5PA3Myshh7+ZWQk5/M3MSsjhb2ZWQg5/M7MScvibmZWQw9/MrIQc/mZmJeTwNzMrIYe/\nmVkJ1Qx/STtKukfSo5KWSPpilTbbSPq5pIWpzWl9U66ZmdVDzf/AHXgL+FJEPCRpK+BBSXdFxKO5\nNmcBj0bEUZJagKWSfhQRf+qLos3MrHdqXvlHxMqIeCi9fgV4DBhd2QzYSpKALYEXyU4aZmbWhLp1\nz1/SeGAyMK9i0uXArsAfgEeAL0bEX+pQn1kpXLv4WuavnL/euPkr53Pt4msbVJENdoXDX9KWwI+B\ncyPi5YrJhwELgB2AvYDLJW1dZRlnSmqT1Nbe3t6Lss0Gl4nbTuT8+85/5wQwf+V8zr/vfCZuO7HB\nldlgpYio3UgaCswEZkfEd6pM/wVwSUT8Og3fDVwQEfMr23ZobW2Ntra2HhduNth0BP5xE45jxtIZ\nfOvAbzFl1JRGl2VNRtKDEdHa2+UUedpHwA+Ax6oFf7IM+NvUfjtgAvBUb4szK5Mpo6Zw3ITj+P6i\n73PchOMc/Nanitz2+RBwMjBV0oL0c6SkaZKmpTbfAPaT9AjwK+ArEbGqj2o2G5Tmr5zPjKUz+Oyk\nzzJj6YwNvgMwq6eaj3pGxAOAarT5A3BovYoyK5uOWz4dt3qmbD9lvWGzevNv+Jo1gcWrF68X9FNG\nTeFbB36LxasXN7gyG6wKfeHbF/yFr5lZ9/XbF75mZjb4OPzNzErI4W9mVkIOfzOzEnL4m/XC6muu\n4dW56/+pq1fnzmP1Ndc0qCKzYhz+Zr2w2cQ9WHHeee+cAF6dO48V553HZhP3aHBlZl0r8vf8zawT\nw/bdh9GXXsqK885jxAnHs+bmWxh96aUM23efRpdm1iVf+Zv10rB992HECcez6oorGXHC8Q5+GxAc\n/ma99Orceay5+RZGfv5zrLn5lg2+AzBrRg5/s17ouMc/+tJLafnCF965BeQTgDU7h79ZL7yx+JH1\n7vF3fAfwxuJHGlyZWdf8t33MzAYQ/20fMzPrMYe/mVkJOfzNzErI4W9mVkIOfzOzEmrY0z6S2oFn\n67jIkcBA/E/jB2Ldrrn/DMS6XXPfGhcRLb1dSMPCv94ktdXj8af+NhDrds39ZyDW7ZoHBt/2MTMr\nIYe/mVkJDabwv6rRBfTQQKzbNfefgVi3ax4ABs09fzMzK24wXfmbmVlBDn8zsxIaEOEv6RlJj0ha\nIGmDPwWqzPckPSFpkaS9c9MOl7Q0TbugiWo+KdX6iKQ5kvYsOm+D6z5I0ktp+gJJX89Na9Zt/Q+5\nehdLelvSe4rM24c1D5d0q6THJT0m6a8rpjddny5Yd9P16wI1N12f7hcR0fQ/wDPAyC6mHwnMAgTs\nC8xL44cATwLvAzYBFgK7NUnN+wEj0usjOmouMm+D6z4ImFllfNNu64q2RwF3N3pbA9cDZ6TXmwDD\nK6Y3XZ8uWHfT9esCNTddn+6PnwFx5V/A0cANkZkLDJc0CpgCPBERT0XEn4BbUtuGi4g5EbEmDc4F\nxjSynjpo2m1d4QTg5kYWIGkb4ADgBwAR8aeIWFvRrOn6dJG6m61fF9zWnRkofbpHBkr4B/B/JT0o\n6cwq00cDy3PDz6VxnY3vD7Vqzjud7CqvJ/PWW5F175c+2s+StHsa1/TbWtIWwOHAj7s7b53tBLQD\nP5T0sKRrJA2raNOMfbpI3XnN0K+L1txsfbrPDZTw3z8i9iL7GHmWpAMaXVABhWqW9DdkB8lXujtv\nH6m17oeAsRExCfgP4Kf9WFtnim6vo4DfRMSLPZi3njYG9gaujIjJwKvAQLifXLjuJurXRWpuxj7d\n5wZE+EfEivTvC8BtZB/H8lYAO+aGx6RxnY3vcwVqRtIk4Brg6IhY3Z15+0qtdUfEyxGxLr2+Axgq\naSRNvq2T46m45dOgbf0c8FxEdPwv77eSBVRe0/VpitXdbP26Zs3N2Kf7Q9OHv6RhkrbqeA0cCiyu\naHY7cEp6QmJf4KWIWAn8DvgrSTtJ2oTs4L+9GWqWNBb4CXByRPy+O/M2uO7tJSm9nkLWh1bTxNs6\nTdsGOBD4WXfnrbeIeB5YLmlCGvW3wKMVzZqqTxetu9n6dcGam6pP95eNG11AAdsBt6V9szFwU0T8\nUtI0gIiYDtxB9nTEE8BrwGlp2luSzgZmk31zf21ELGmSmr8ObAtckdq9FdlfFaw6bz/UXLTujwOf\nk/QW8DpwfEQE0MzbGuAY4M6IeLXWvP1QM8A5wI9SqDwFnNbkfbpo3c3Yr2vV3Gx9ul/4zzuYmZVQ\n09/2MTOz+nP4m5mVkMPfzKyEHP5mZiXk8DczKyGHv5lZCTn8zcxK6P8DDS9NORhcV2YAAAAASUVO\nRK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "mu_gmm = np.empty((3,2,2))\n", "Sigma_gmm = np.empty((3,2,2,2))\n", "c_gmm = np.empty((3,2))\n", "ax = plt.gca()\n", "plt.title('Initial mean vectors for the three Gaussian mixture models')\n", "for y in range(0,3):\n", " # We need to find the maximum eigenvalue, because numpy doesn't sort them\n", " imax = np.argmax(Lambda[y,:])\n", " #imax=0\n", " for k in range(0,2):\n", " mu_gmm[y,k,:]=mu[y,:] + (-1)**k * 0.1*np.sqrt(Lambda[y,imax])*V[y,:,imax]\n", " Sigma_gmm[y,k,:,:]=Sigma[y,:,:]\n", " c_gmm[y,k] = 0.5\n", " ax.plot(mu_gmm[y,k,0],mu_gmm[y,k,1],'x')" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAEICAYAAAAZeSDaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8JGd95/HPT9eopdE9h6TRaMZm7LE9MPaAY4g5TGwg\n2LCQDZBAQrwQgoEACwlZNmQJAXK9kt0lEFhwDCSGcAVwYIEFgzFgII5NbM8B4/HYY3sOjaQ5dE9L\nGmmkZ/+oqp7qVl+SqlXdmu/79dJL3VXVVU9XV9e3n6fqqTLnHCIiIiutKu4CiIjIhUkBJCIisVAA\niYhILBRAIiISCwWQiIjEQgEkIiKxiDSAzOw7ZvZf8oy/1cz+tMh5/cjMfi+60pWWmf22mX0v5jLk\nXf+rkZkdNrMXFDHdVjNzZlazhGUkzOybZjZmZl9Z5GudmW1b7DLLVaHt3Myeb2Z9K1mmclSpn3ux\n5TazXjM7Y2bVOca/38w+V2g+BQOo2C84gHPuRufcZ/zXvc7Mfpox/s3OuT8vZl5FlOsSM/uSmZ0y\ns3Eze8zMPmpmPVHMf7Gcc593zr2oFPP2N4qT4Z2nmdX6w1IducLrv8D8VnwnYWa3++/j5RnD/84f\n/rqVLM8ivRLYCHQ4514Vd2HyMbM/MLNB/zvxj2a2Jsr5Z27n5bajzbbf8be9v4irTPlkK28lcM4d\ndc6tdc7NLWc+FdkE52/w9wP9wC7nXDPwbOBx4Dlxlq2ERoAbQ89v9IetuKXUInyPAjdnzOc38D63\ncrYFeNQ5dy7uguRjZr8K/DFwA16ZLwY+EGuhVoFlbO8VI7b36JzL+wccBl7gP34d8FPgf+Ht/J4E\nbgxN+yPg94DLgWlgDjgDjPrjbwf+wn/cBnwLOOXP61tAT+a8cpTpc8A3C5S70PxT78t//n7gc/7j\nen8ZQ8Ao8B/AxtA6eAKY8N//b4fXTWh+HwGOAePAg8BzM5b1ZeCz/nz2A1fneS8OeC/wldCwrwL/\nw/sIF64z4BPAHaFxfwPcDTQCU8C8/9mcAbrDn40//fOBvoz19d+BfcBZoMZ/3R3+On4S+K953sPt\n/nZzAmjzh70U+A7eNvU6f1iV/16PACf9ddQSms/v+OOG/Pef+hz91/4xXqAN+eu43R+31V+PNTnK\nd7m//kb9z+Nl/vAPADPArL+u3pDltdXAn/jLnfA/782hz26b//glwG5/mzgGvD80j0Vvc1nK8QXg\nr0LPrwcGC33H/WnvAV7hP362X+6X+M9vAPZkbufAj/3pkv66+c1guwHe5X9+A8Dr8yw353sD3ggc\n8Mc9DDzdHx58xsHw/xz6DNP2O8At/mc34w/7pj9tzm0X7/v5Vf/zGCfLfghve74VuMsvxz3Alozv\nbPC5t+Btx6fwtt334m2rWfeTWZbVDXwDGAYOAW8MDZ/C38b9YbuA00Ct//x3/XU4Anw3SxnfCjwG\nPLnI7XUroe8TcJG/Dib8dfIx/P1p3u2uiA3zMOkBNOtvGNXAW/BqIZZlB/g6Qjvk0IcWBFAH8Aqg\nAWgCvgJ8PdvONEuZBvF3WHnKXWj+qfcV2uiCAHoT8E3/tdXAM4BmvJ33OLDdn64L2JHt/QKv9ctQ\ng/dlHATqQ8uaBm7y5//XwH153osDnoq3827FC9cT/jCXbZ35ZX/UL9dz8TbKHn/c8wmFS+Znk20a\nf33tATYDCbwv0IPA+4A6vF/bTwC/muM93A78BXAb8BZ/2JeB15AeQL+L9yW7GFgL/Cvwz/64K/C+\nqM8D1gAfAs5xfvt8B3Af0OOP/wfgi9m+MBllq/WX+Sf+e7ke74sUfM6pbSPHe/tvwM+B7YABV+I1\n1wWf3bbQOn2av+52+p/hry11m8tSjr3Ab2Z8B1xQlgLflw8CH/UfB2H6N6FxH8mxnafeX+g9nvNf\nU4u3jU/i/+jIWGa+79OrgOPAL/nrdBv+ztMf1+2vx9/EC8CuYvY7/vO8267/ec8Cv+ZPm8ixPU9w\nflv8SK71ghc+/xdvP7QV73v5hlzlzbKsHwMfx/uRchVekF3vj/sBfiD5z/8ncKv/+OV42/XlePuh\n9wL3ZpTxLqA9eI8Uv71uJT2A/h3v+7jGXycTlCiADoXGNfiF6MyyAyy4IWSMuwoYybYzzTLtOeDF\noedvw/u1cwb4ZJHzT72vzJ0M3k7wXmBnli/MKF6wJTLG5d2Q8H6BXBla1vdD464ApvK81uF9AT+F\nt6N6M/BJf5jLtc6AZ+L9ajoCvCZjJ7GUAPrdjHkfzZjHe4B/yvEebscLoOf4G2sr3gadID2A7gZ+\nP/S67Xg7gxq8HcaXMj6PGc5vnweAG0Lju0Kv3UruAHou3g+EqtCwL+L/4qNwAB0EXp7vs8sx7sPA\n3y11m8syv8dJ/17U+svfmu91/rQ3APv8x3fitWTc5z+/B/j1bNt55vvzt5up8HrGqwk9K8sy832f\nvgu8o1C5/Wn3BOs/s3w5tu28267/ef+4wDJvz9gW1+LVZNJqvng/JmaAK0LTvgn4Ua7yZixnsz/f\nptCwvwZu9x//HvAD/7Hh1VSe5z//DqEaO16QTHI+yB1+kC1he93qT1sD9OLtkxtD036BIgJoKceA\nBoMHzrlJ/+Haxc7EzBrM7B/M7IiZjeOlfGuusyoyDOHtXIJyfMw514q3gmojmP8/430BvmRm/Wb2\nt2ZW65xL4v3iejMwYGb/z8wuy/H+/sjMDvhnTo3iVcPXhSYZDD2eBOqLaIf9LN4xlJv9x3k55+7H\n+2VneLWN5ToWerwF6Daz0eAP75fzxgJl+imwHq/57FvOuamMSbrxAjNwBG8j3+iPS5XB/zyGMsr0\ntVB5DuB9efOWKZivc24+Y7mbCrwusJkijmOZ2TPN7If+iTNjeNtRsE0se5vD+wHWHHre4v+fKOI9\n/DtwqZltxPux9llgs5mtA67B+/4Ua8ilHy+bJMs+osB7y7lOzexmM9sT+pyfSvp3q5Bitt1j2V+a\nJrwtnsH7sdedMc06vH1S5jZd7LbVDQw758KfYfj1dwC/bGZdeDWPeeAn/rgtwEdC73EYb18QXnbO\n91lge80s44j/eYbLWFApT0JwBca/C+/X7TOddxLB8/zhVsS87wZ+fZnzT+LV4AKdwQPn3Kxz7gPO\nuSuAa/GOVdzsj/uuc+6FeAH4CF5NJI2ZPRd4N94B9jY/HMeKfG/5/MRf7ka8WkNeZvZWvCpxv1+e\nQLbPJuf6yPG6Y3jtxq2hvybn3E2FyoXXtv4usodoP94XJxD8ujqBdzxhczDCzBrwmpnCZboxo0z1\nzrnjBcrTj7ezDX8fevGagIpxDHhKEdN9Aa8tf7NzrgXvGILB8rc533685r/AlcAJ59xQjulT/B+T\nD+I1Y/7COTeDVyP7Q+Bx59zpIt7fouV5b1nXqZlt8ad5G17TYivwC85/t7Jt25nDitl2C+2/IH1b\nXIvXlNWfMc1pvFp45jYdbFuFltMPtJtZU7bXO+dGgO/hBflv4dXKgnkeA96U8T4Tzrl7Q/PKt/yc\n22uGAaDNzBozylhQKQPoBNBjZnU5xjfhVdVHzawd+LNFzPv9wHPN7ENmtgnA/6V2+SLmvwd4tX86\n89V4p9riz+tXzOxpfm1pHG8DmjezjWb2cn9Fn8X7xTnPQk14O81TQI2ZvY/0X6ZL4m9Y/wnvAHne\nDdfMLsVr8not3oH7d5vZVf7oE0CHmbWEXrIHuMnM2s2sE3hngeL8DJgws/9uXj+ZajN7qpn9UhFv\n5e+BF5L9V/UXgT8ws4v8L/VfAf/i/6L+KvBSM3uOv119kPRt+FbgL/2dFGa2PvO07xzux/uV/m5/\ne3g+3nr+UhGvBa9p9M/N6xpgZrbTzDqyTNeE92t22syuwdth4Jd1udsceIH+BjO7wszagD/FayoK\nlnG7md2e47XgNbW9zf8PXpNu+Hk2J/COoSxagff2KeCPzOwZ/jrd5n+ujXg7zVP+PF6PVwMKlydz\nv5NZxuVsu2E3hbbFP8drskyrUTjvNOUv422XTf57+EO8H2G5yht+/TG8HwJ/bWb1ZrYTeEPo9eAF\nxc14+7AvhIbfCrzHzHYAmFmLmS2mG0HO7TWjjEeAB4APmFmdmT0H7/tTUCkD6Ad4v8gGzSzbr6cP\n47X/n8Y7cHxnsTN2zj2K147bA+w1swng3/B+LQQdXQvN/0/xfmGN4J3pFP7gOvF2duN4zTj34DWR\nVOFtPP141dnr8E7EyPRdf3mP4lVFpymuSl+Qc26/c25/vmnMa8r7HN5B5L3Oucfwmhj+2czWOOce\nwdvRP+FXz7v997cX71jP94B/KVCOObxf6VfhnUV0Gm+n0ZLvdf5rh51zd+cI0X/0y/Jjf77TwNuD\n94531s4X8H51jeCdcRX4CN4vtu/528R9eNtJofLM4H1hbvTfx8eBm/31VIwP4e1kvoe3zXwab9vL\n9PvAB/2yvY/0ZtHlbnM45+4E/hb4Id529yTpP7w2431PcrkHb6fz4xzPs3k/8Bl/O/qNPNNlk/O9\nOee+Avwl3mc9AXwd72yvh4H/jddkeALvIHn4PWXb73wauMIv49eXs+1m+ALe+h3GO2nktTmmezte\nC8MTeC0XX8DbznOVN9Nr8I659ANfA/7MOff90PhvAJfgnfG4NxjonPsa3tmvXzLvMMQvSO/KUUi+\n7TXTb3H+mPOfUcQhAjh/9pqIrGL+L+y9eCc5zMZdnkrn1yT7nHPvjbsslWzVd7ASkVQt7/KCE4qs\noIq8EoKIiFQ+NcGJiEgsVAMSEZFYVMQxoLqqhEtUNxWeUEREABg/d+q0c2593OXIpyICKFHdxLXr\nyvoq+CIiZeXOwY8XdTWCOKkJTkREYqEAEhGRWCiAREQkFgogWVWGdlxMsrM9bViys52hHUu6XJmI\nlJACSFaV+qFRjl+3KxVCyc52jl+3i/qh0ZhLJiKZKuIsOJFiNQ4Os+me3Ry/bhdtB48ysr2XTffs\npnFwOO6iiUgG1YBk1WkcHKbt4FFOX3kJbQePKnxEypQCSFadZGc7I9t7Wbf3MUa29y44JiQi5UEB\nJKtKcMxn0z27Wb/nsVRznEJIpPwogGRVme5oTTvmExwTmu5ojblkIpJJJyHIqtKx/4kFwxoHh3Uc\nSKQMqQYkIiKxUACJiEgsFEAiIhILBZCIiMRCASQiIrFQAImISCwUQCIiEgsFkIiIxEIBJCIisVAA\niYhILBRAUlZ0R1ORC4cCSMqK7mgqcuHQxUilrOiOpiIXDtWApOzojqYiFwbVgKTsJDvbGb7iItr6\nTjB8xUXU1VWRGE8CUPdIX8ylE5GoKIAkdjOX9aQeTzU3MrhtC52HjpAYT5KYOJP2PDwtKJBEKpkC\nSFZcZoiETTc2pMIGIDGepPPQEaYbG1LDss1LQSRSeRRAsmLyBU+gbeDUgmGJ8WTW8Mmct0JIpLLo\nJAQpylL758xc1pP6K7WVWIaIREcBJEVZbP+clQqdbMsVkcqgJjgpSrH9c8ohANQcJ1IZFEBStHD/\nnHV7H0sLn3IIHhGpLAogKVqys52R7b2s2/sYI9t7aRgcora1Ie5iZaVakEj5UwBJUYJjPkGzW8Pg\nEH3XX512yrSIyGLoJAQpynRHayp8Zi7roba1IdU/p1ypWVCkvKkGJEXp2P8EkL5TL6Z/johILgog\nKYpqEyIStcgDyMyqgQeA4865l2aMM+AjwE3AJPA659xDUZdBohV1+Iw9ZU3OcS2Pn410WdkM7biY\n+qHRtLP4kp3tTHe0pmp6IlJ6pagBvQM4ADRnGXcjcIn/90zgE/5/KUNRBU++wMk3banCKOhUGxzT\nCp9gISIrJ9KTEMysB3gJ8Kkck7wc+Kzz3Ae0mllXlGWQaCw3fMaesib1t5x5lEK4U+2pqy5JCyMR\nWTlR14A+DLwbaMoxfhNwLPS8zx82kDmhmd0C3AJQX7U22lJKTssJnlIERjDPqGtD+TrVisjKiKwG\nZGYvBU465x6MYn7Ouducc1c7566uq0pEMUspYKnhs9yaThwyO9VmXmhVREovyia4ZwMvM7PDwJeA\n683scxnTHAc2h573+MMkZksJn5UMniiXEz7ms37PY6nmOIWQyMqKLICcc+9xzvU457YCrwZ+4Jx7\nbcZk3wBuNs+zgDHn3ILmN1k5S7lqdVw1nqiWGe5UC+ePCU13tEYyfxEpTsn7AZnZmwGcc7cC38Y7\nBfsQ3mnYry/18iW3pQTPapDtVOvGwWEdBxJZYSUJIOfcj4Af+Y9vDQ13wFtLsUxZnAs1fESkfOhK\nCBegxYRPlMEz1ryJ2pkkDdPnb2I3Wd/KbF0jLeM6FChyoVEAXWBKHT4TWyznuPmZSSbrNnC2zaip\nG6X6RAuTjRtoSJ5cVJlW4moJIlJ6CqALRCmDJ1/ohNXUeTWfudn1zM+txa1NUF17ioYct/UWkdVN\nt2O4AJQqfCa2WNHhE6ipG8WqpnDzDVjVFDV1o4ueh4isDgqgVa4U4bOU4Amcm2nFzSewqkncfIJz\nM62peUZNd0QVKW9qglvFig2fxQTPcpybaWVudj3VtaeoqRtNPYfzzXMicuFQAK1SUYZPVLWT+bmG\nVPjA+dCZn2sAvKa4piMukmWJSPlTAK1CUYVP1M1idYn+BcO8EFLtR+RCpABaZYoJn5Ws9YiI5KIA\nWkWWEz7DrVupnU3CutOpYbPT65mfb2BNw5GsrznbO1NUudYcrStquiiMdK2nPjlJy88Opobpbqci\n5Ulnwa0Sy6351M4mOZtoY3baOylgdno983OtVFVNLpj2bO9M0eGz2OkL1bwKdUKtT04yuG1L6srW\nwZWv69XXSKTsqAa0CkTS7LbuNFXTxvxcK2eTa4EaqqpHqa0/BRRf28nnbO9MyWtDifEknYeOcPy6\nXbQdPMrI9l7d7VSkTCmAKtxywydc46itP+WHTy0w6z2PIHhWWsvPDjIzM6+7nYqUOTXBVbAowwfw\nm99qgFmghrO1HcsrYBYrEWi626lIZVANqEKVInzm51qpqh5l/tLjMNAFo/6Ou2vl7hm43H5AU82N\nnLhye6rZrWFwKHX3U9WERMqLAqgCLSd8ch3kn59vOB8+cD50kmuXVMZSKOYq2OeSsznvdqoAEikv\nCqAKU4rwAeCyx5jPHLaCNZ+o6G6nIpVDAVRBShU+Z3tnYGgd1E9BY/L8iGQjTCeg4zRbe04VXvbh\nXkbOVuWcR2oZzKZGz88lmJ+rp6ZupOD8RWR10UkIFaIU4ZPWP6d+Co5v9gIDvP/HN7Nx80BR4QNQ\n1zxO1cCmBfPwQuf8MubnEoAXPrPTXVRVTxc1/4LL19WvRSqKakAVoFThk6YxCZuOeYHRNkzVWCvr\nr/o5ifbiO3Am2kdZv3M/p/btYL5lFEbavXkGNSJ/GbNHN1NdO8bcbAu19QNUVU8VnLfugiqy+qgG\nVOZWJHwCjUlaevvg9AaaevoXFT6BRPsoTT39cHoDtA2nN8f5y/DCp4Pq2rG08FnOGXCq/YhUHgVQ\nGVvR8AE2Nswy0ddNy0WHmejrZmq4tbiChkwNtzLR1w3rTno1oKA5LpBsZG62heraIeZmW1LNcfmo\n9iOyOimAytRKhs/WnlNsbJjl1L4drN+5n7ZtT6aa0hYTQlPDral5sP7k+Sa9jGNCtfUD1NQNUVs/\nwOx0V1EhlI9qPyKVSQFUoRYTPvkuBhqcYDAz3sz6nftTzW7B8ZyZ8eaiy5Q5j9RxpWk/YKYTsOlY\nqtmtqnqK2voB5ufqi16GiKweOgmhDBWq/Sw2fLLJPLOtZevRBdMk2kcXdRwo2zxoTJ4/DtTh3+ph\n6PwFSauqp6iqnsp5/KdQ85tqPyKVSwFUZuIIn1xe2PnIgmF3DV5W1GtzWcl7A4lIeVMAldjQjoup\nHxpN64mf6wZpJQ0fvxPo1u2HU4OmhluZGW9Oq7lkC52wYPxygyhsqbUfEalsOgZUYvVDoxy/blfB\nG6SVvOZTP0XVwKbUSQXBCQN1zeOAFyyFwqfcqPlNpLKpBlRiwcUw890gbUWa3bYfZmr9KKf27aCp\np5+Jvu7UCQNLCZ4Xdj6y6FqQmt9EJEwBtAIaB4dpO3g00hukLeWYT9BJdOzJrbRcdJiXXXHfssux\nXDr5QOTCpSa4FZDvBmlLqf0s9YSDoJNoy0WHOdu/kf7B7mLfwqIc7ltfkvmKyOqiGlCJBcd8st0g\nrba1Ie9r891QrhiZ4RN0En3ZFffRP9jN3T9+ETc873t0d/YvaznFyNb8ptqPyIVNNaASm+5ozXqD\ntDOXb8n7uuUe98k81TroJBo0u3V39nPD877HqaENRb2PcqHwEVk9VAMqsWw3SKttbaBtIHdfnKjD\nB7xOopknG3R39i+59pPrBIRszW9R1n5EZPVQAJWZUoRPpZ1enYtqPyKrS2RNcGZWb2Y/M7O9Zrbf\nzD6QZZrnm9mYme3x/94X1fIrRTEXGc0URfjcefdN7Ht4Z9q4fQ/v5M67bypq/N79Vy04aWFquJWx\nw715y77Y2s9I13qmmtOvoD3V3MhIl05sEFltojwGdBa43jl3JXAV8GIze1aW6X7inLvK//tghMsv\ne0vt75Op2PAJ6+7q4/4Hr02FzL6Hd3L/g9fS3dVX1Pj1HSe5+8cvSoVQZkdWWN7Zb0HTW31yksFt\nW1IhNNXcyOC2Law9cGTJ8xaR8hRZE5xzzgFn/Ke1/t/S7zC2ykTZ2TRTrvAJN73tvGIfAPc/eC1H\njl3E4MkunvmMe1PDC40PTlq4+8cvYk33ibSOrLkspvYTSIwn6Tx0hMFtW2g5McTYxg46Dx2JpO+U\niJSXSM+CM7NqM9sDnATucs7dn2Wya81sn5l9x8x25JnXLWb2gJk9MDNf+JbNlWw5x32KCZ/Aziv2\n0blhgMGT3XRuGEiFS7Hjuzv7ufzS/Yw9uXXBHVOjqP0EEuNJWk4MMdKzkZYTQ7T87OCS5y0i5SvS\nAHLOzTnnrgJ6gGvM7KkZkzwE9DrndgIfBb6eZ163Oeeuds5dXVe1vBuWxa2Ux32yyXXSwb6HdzJ4\nsovODf0MnuzCHm/nlc0P8crmh7KOzzwm1D/Yzd5HdhZ1x9Sl1H4CU82NjG3soK3vBOPrWtM67orI\n6lGSs+Ccc6Nm9kPgxcAvQsPHQ4+/bWYfN7N1zrnTpShHOVjp4z75wuf+B6/lmc+4lw8+63bu2P0C\nPnnvqwB4xa7vp43fecW+1HPwakZBx9Wg2a2+fSTVsfXEZG1R7yGbzNpPcMyn89AREuNJmvY/mdaR\nV0RWjyjPgltvZq3+4wTwQuCRjGk6zcz8x9f4yx+KqgyVJurjPvlOt+4f6EmFD3ih88Zrv8LuvsvT\nxoePCT3zGffSP+AF6KmhDbQ89ZGi7phabO0nW5+f6caGVPjUPdKX6rg73VH8rcFFpDKYd+5ABDMy\n2wl8BqjGC5YvO+c+aGZvBnDO3WpmbwPeApwDpoA/dM7dW2jeLbUb3LXrXhVJOVdSqa7ztpjjPmFB\nU1s2Xx1/et7XQvbOp8V0PF1Kp1P1+RFZnjsHP/6gc+7quMuRT5Rnwe0DdmUZfmvo8ceAj0W1zHJW\n6ouMZlpO+BRjqeGTi654ICK6FlyJZetYeeqyViYTiziwfnQLDHUA52s/Y0d6OPGQd5JAMeHz3m++\nnTt2vyBt+B27X8B7v/l2vjr+9KwdTfsHu9m7/ypveYd7F550kGz07rTqK9T0Nta8icn69HmMdnbQ\nf+nWtGGq/YhcGBRAJRCu/WTrWDnR1E3NufRTy/PWfhrPwMlO2ua8WtPYkR5GHt1GfUfxB+V39Rzg\nk/e+KhVCwUkIu3oOAAs7mqZOOug4yV2Dl1HXPM6pfTtSIXT44FY4vhnqc58in9n0VjuTZLJxAy65\nFvDCZ6i3m4bxiaLfh4isHroWXMQym94yO1aOdnfQNNFP3ez5HXfBprcO7zyNkUe3MXlyPWdHW2i7\n9BAtW/qKbnp7xa7vA/DJe1/FvU/uYv/ANt547VdwT/FCLNzR9PJL93Pg0R3c8LzvsR/vJIPgpINT\n+3Yw3zIKI+2w6Rg0JoHimt4apkdJDJ1jqLebZFsL002NdBztp3Xw/Hkoqv2IXDhUA1oB4Y6V9dOj\naeFTrK1XPsKa1jHOjraypnWsqPDJ9Ipd32dH1yF+MXApO7oOpcInEHQ03f3zq7n80v0LrpQd3FGV\n0xugbThv+OQ68aB1cIj6iSTTzWupn0gqfEQuYAqgCOU68SDoWJmYPM10fSsztec71hZ74sHYkR7O\njrawpnWUs6MtjB1ZfOfWO3a/gP0D23hq16P8YuCSrB1NDzy6g11Pe4ADj+7gGw+nX8pvariVsaM9\nsO6kVwNKph/bCuQ76220s4Pppkbqx88w3dTIaKd3bEvhI3LhURNcRPKFz+C2Layd9JrdamcnmWjq\nXtAMF8gWPm1zaxh5dFuq2S04BrSv6eSCy+XkEhzzCZrdGh8+lbWjaXCH1P666lRH00T7KFPDrZzY\n87TzzW4NSTi+mdraAaguXKMLwmeotzvV7BY8B9igABK54CiASmy6sSEVPgB1s1M0TfRzribB2e7p\n4uYx1J4KH4CWLX1c2nSS/oGeogNod9/lXPOMf081uwWvC+ZxamhDKnzuGrwsraNpon2UE8e60o75\n0JiktnaA+bl6qkIBlK/D6WRzU9oxn+D/dPXSr6QgIpUrso6opVTuHVFL2ednpW8wV+ydTqO6y6ma\n3kRKoxI6ouoYUIkVe623bArd42cxCvXzyXfDuWKudK1bbIvIYimAlmmlr3S9VPn6+QTjv/OjG1P9\nfIIbzo2cXbiJRHW1A9V+RC5sOga0DFFd6Xoxltr8lqufT3Cq9X6aU/18mnr6mejrZr7r+PljPr7l\nXGg0TOEjIqoBrbDlHPtZrlz9fILjPkE/n7Ent3qdTZcYPoUofEQEVANasjhqP8uV2c+nu/N46koH\n4DW7TfR1n+/n05BcUmdTHfcRkWIogFZQnLWfbP18vvOjG9P6+Zzat+N8s5vfz4dNx1gzNLtgfksN\nH9V+RCSgJrgliKv2s5zTr/P18wGYGW9OP+bTmPT6/UwXfzt0hY+ILIZqQCskjjPfwq7csQdI7+eT\naB9N3eFmNCy8AAAQe0lEQVR0pGZq4dbQmCy69qPwEZHFUg1okUpd+ylF81ug2E6mgahOOhARyUYB\nlMfQjotJdqbfOG6quZGRrsIdMwOTiXbGehrShs3PJTg30+Y9Cd1sLhC+2VxYrgAp5K7By/jqfS9a\ncEO5qeFWDu+9POtrojzpQLUfEclGAZRH/dAox6/blQqhsWu2M7htC/XJyazTZ6v91JybYna6i/k5\n71jK/FyC2eku5rrOeBP4N5sLQmgpN5vLJwitzBvKpS4umuWGcgofEVkJOgaUR+PgMJvu2c3x63bR\ndvAow9u20HnoCInxZOEX++pmp6itH2B2uovq2jHmZlu858HBfv9mc5zshIlmRqYa0i48mumuwcuK\nOhkhs7YUvqFcU0+/d1uF8MVFfQofEVkpCqACGgeHaTt4lNNXXkJb34mc4ZPr2M/EFqOKKT98Oqiu\nHUq7ejTghdBEM0w1sqZ1NGf4BHKFUKEmunBHU9adVPiISKwUQAUkO9sZ2d5LW98J76ZyE2cWVQMC\nr9ltbraF6toh5mZbqKqeBEJnwA11wFQDJJKpm80VE0KLteCGciXqaKrwEZFiKIDySHa2c/y6XWx8\nso/EeJLExBkGszTD5TvzLTjmU1s/QFX1FFXVk8zOdkHynLfzH+rwmt82DELHUOrmc0DBEFqMwwe3\npjqWlrKjqYhIsXQSQh7THa1sumd3KmwS40k6Dx1hurGhwCs9E1uM+bn6VPgA3v9wB8/k2lT4gBc6\nbZceYnqoPddsF+1w33pveRk3lGPTMaoH1i6Yfjnho9qPiBRLNaA8OvY/saDfT2I8uagmuJq6kYUD\nG883fdF7ZMHoli19kdV+Un18Ok6nDfea3GYho3wKHxFZKQqgZSrHi45C7s6lkPt+PsvpZKrwEZHF\nUhNcHku52VwguPTOuZm2VB+glGQjDK3L+rpswTF2uDdrJ9Kxw70A9P3bNZw+cEnq9Yf71sNAFxy6\nZMG8lhI+OulAREpBAVRiVdXTCzqicnxz1g6guWTrRHpq3w7qmscBqG8f4UzfJg7/h3/1hIEuGG33\nOrmGKHxEpJyoCW4Ziml+q6pe2BGV3oUdQMMO961PuyZcZifSib7u1G0UDveth6ZRaE14oXOmCc7V\nQuswdA0A+W+hrfARkbgogHJYTvNbpqrq9I6oc3nCJ5D1GE7LaKoT6YnJWpgMTdM14IdPHdTMKHxE\npOypCW4FZHZEJdm4+JkkG73Oo0En0sx5DHR5NZ+aGe//QJfCR0TKmmpAJZa1I2q4Q2gxko05O5HS\nmDx/zMdvdqt6dBPzo+3MVldRW59+e4flBA8ofEQkOqoBZVFM81uxp18X7IhajBydSNM6s7YOs2Z2\niDVH66itP0VV9Sjz8+kdZhU+IlJOVAMqsWwdUdcMzXK2dxHXk8voRAqkd2bd9tiC5jbVfESk3EVW\nAzKzejP7mZntNbP9ZvaBLNOYmf29mR0ys31m9vSolr8U2W44l+xsT91wbqRrPVPN6cdaFntDulzy\nHZ9ZIMtN6xjqgKNbWHO0jupDGxf0NQrf9K7piGMy0c5Mbfo0M7UJ5iebCy5e4SMipRBlE9xZ4Hrn\n3JXAVcCLzexZGdPcCFzi/90CfCLC5S9a5g3nkp3t9F1/deqGc/XJSQa3bUmF0FRzY94b0pVMxk3r\nqg9tgJOdVM8EzXoL+xrNTnfReHoqVfOpOTfFRFN3KoRmahOcaegu+F4UPiJSKpE1wTnnHBD0fKz1\n/zLbfV4OfNaf9j4zazWzLufcQFTlWIzMG86NbO9Nu9J1cPHRwW1baDkxxNjGjtT4mfXLvwRPUAs6\n2zuTf8LQTevsdAtz8wmqa09RUzcKZO9r1DzWT93s+c6udbNTNE30M9HUTf30KGdrWwveXE/hIyKl\nFOkxIDOrBh4EtgH/xzl3f8Ykm4Bjoed9/rAFAWRmt+DVkqivWnjF5qiEbzi3bu9jJKbTr1CQGE/S\ncmKIkZ6NqRvSRX39t+Ka4yaYqWrBzTdgVZOp8AmE+xolJk+nhU+gbnaK+ulRphrW5b25Hih8RKT0\nIj0Lzjk355y7CugBrjGzpy5jXrc55652zl1dV7WIM8YWKbjh3Lq9jzF8xUVZj/mMbexI3ZAuc/xK\nOTfTiptPYFWTuPkE52bSrw03P5dg/mwLicnTTNe3LjjeA1B3qoazta0F34vCR0RWQknOgnPOjZrZ\nD4EXA78IjToObA497/GHxSK44dyme3bTODhMXV1V2g3ngmM+wfPghnRrJ/uz1jBK5dxMK3Oz61PN\nbsFzgJq6Udb01zPR1EXThFeu2tlJJpq6U8/BC59s7yWzGU7hIyIrJcqz4NabWav/OAG8EHgkY7Jv\nADf7Z8M9CxiL6/gPnL/hXOPgMLDwhnPTjQ1ZjwmdqylcI1vOrQ0yzc81pB3zqakbpbr2FFXJBpqO\nOM7VJNLDxj/eE5Sz5fGzOd9L+OZ6Ch8RWUlR1oC6gM/4x4GqgC87575lZm8GcM7dCnwbuAk4BEwC\nr49w+YvWsf+JBcPCN5xrGziVdfzM1LmSly2sLtGf9twLtxH/Dxqmhhe+ZnaKutmpVB+fXO8leK8K\nHxFZaVGeBbcP2JVl+K2hxw54a1TLLHdNR1zqvkBRza9YxXQuDSh8RCQOuhSPL9vld3J1RJ1MtC+Y\nNpcomuKajjiFj4isOgqgPHJ1RG09Or6o+Sw2QMKvWWzwKHxEpFLoWnB5RN0RNcoTEzIpeESk0iiA\nCsjWEbWcLCZ4QOEjIuVDTXAFlEtH1GwUPiJSyVQDyiNXR9TOQ0doeTz6S/IUa7HBAwofESk/CqA8\n8nXejKMpTsEjIquJAiiPQp03Wx4/uyK1oKUEDyh8RKS86RjQMi01HIqdt8JHRFYr1YAiEGVNaLmB\npuARkUqhAIpIEBxLCaKoalEKHxGpJAqgiIXDJFcYlaLZTuEjIpVGAVRCpTw+FFDwiEil0kkIFUzh\nIyKVTAHkq6Sded0jfRVVXhGRbNQEV0EUOiKymqgGVCEUPiKy2iiAQspxJ6/mNhFZrdQEV6YUOiKy\n2qkGlCHuHb9qPCJyoVAAZRFHACh4RORCoya4HOoe6WPmsp6SL0NE5EKlGlAepQoI1XZERFQDKigI\niuXUhhQ2IiILKYCKlBkiuQJJYSMiUhwF0BIpaERElkfHgEREJBYKIBERiYUCSEREYqEAEhGRWCiA\nREQkFgogERGJhQJIRERioQASEZFYKIBERCQWkQWQmW02sx+a2cNmtt/M3pFlmueb2ZiZ7fH/3hfV\n8kVEpLJEeSmec8C7nHMPmVkT8KCZ3eWcezhjup84514a4XJFRKQCRVYDcs4NOOce8h9PAAeATVHN\nX0REVpeSHAMys63ALuD+LKOvNbN9ZvYdM9uRZx63mNkDZvbAzPxUKYopIiIxivxq2Ga2FrgDeKdz\nbjxj9ENAr3PujJndBHwduCTbfJxztwG3AbTUbnBRl1NEROIVaQ3IzGrxwufzzrl/zRzvnBt3zp3x\nH38bqDWzdVGWQUREKkOUZ8EZ8GnggHPuQzmm6fSnw8yu8Zc/FFUZRESkckTZBPds4HeAn5vZHn/Y\nnwC9AM65W4FXAm8xs3PAFPBq55ya10RELkCRBZBz7qeAFZjmY8DHolqmiIhULl0JQUREYqEAEhGR\nWCiAREQkFgogERGJhQJIRERioQASEZFYKIBERCQWCiAREYmFAkhERGKhAFqmoR0Xk+xsTxuW7Gxn\naMfFMZVIRKQyKICWqX5olOPX7UqFULKznePX7aJ+aDTmkomIlLfI7wd0oWkcHGbTPbs5ft0u2g4e\nZWR7L5vu2U3j4HDcRRMRKWuqAUWgcXCYtoNHOX3lJbQdPKrwEREpggIoAsnOdka297Ju72OMbO9d\ncExIREQWUgAtU3DMZ9M9u1m/57FUc5xCSEQkPwXQMk13tKYd8wmOCU13tMZcMhGR8qaTEJapY/8T\nC4Y1Dg7rOJCISAGqAYmISCwUQCIiEgsFkIiIxEIBJCIisVAAiYhILBRAIiISCwWQiIjEQgEkIiKx\nUACJiEgsFEAiIhILBZCIiMRCASQiIrFQAImISCwUQCIiEgsFkIiIxEIBJCIisVAAiYhILBRAIiIS\ni8gCyMw2m9kPzexhM9tvZu/IMo2Z2d+b2SEz22dmT49q+SIiUllqIpzXOeBdzrmHzKwJeNDM7nLO\nPRya5kbgEv/vmcAn/P8iInKBiawG5JwbcM495D+eAA4AmzImeznwWee5D2g1s66oyiAiIpUjyhpQ\nipltBXYB92eM2gQcCz3v84cNZJnHLcAt/tOzdw5+/BeRFzR664DTcReiCCpndCqhjKByRq0Syrkl\n7gIUEnkAmdla4A7gnc658aXOxzl3G3CbP88HnHNXR1TEklE5o1UJ5ayEMoLKGbVKKWe5i/QsODOr\nxQufzzvn/jXLJMeBzaHnPf4wERG5wER5FpwBnwYOOOc+lGOybwA3+2fDPQsYc84taH4TEZHVL8om\nuGcDvwP83Mz2+MP+BOgFcM7dCnwbuAk4BEwCry9y3rdFWM5SUjmjVQnlrIQygsoZtUopZ1kz51zc\nZRARkQuQroQgIiKxUACJiEgsyiqAzKzazHab2beyjCuby/gUKOfzzWzMzPb4f++LqYyHzeznfhke\nyDK+LNZnEeUsl/XZamZfNbNHzOyAmf1yxvhyWZ+Fyhn7+jSz7aHl7zGzcTN7Z8Y0sa/PIssZ+/qs\nZCXpiLoM78C7gkJzlnHldBmffOUE+Ilz7qUrWJ5cfsU5l6uzXDmtz3zlhPJYnx8B7nTOvdLM6oCG\njPHlsj4LlRNiXp/OuYPAVeD9mMPrivG1jMliX59FlhPKY/usSGVTAzKzHuAlwKdyTFIWl/EpopyV\noizWZyUwsxbgeXjdDHDOzTjnRjMmi319FlnOcnMD8Lhz7kjG8NjXZ4Zc5ZRlKJsAAj4MvBuYzzE+\n12V8VlqhcgJc6zcbfMfMdqxQuTI54Ptm9qB/WaNM5bI+C5UT4l+fFwGngH/ym14/ZWaNGdOUw/os\nppwQ//oMezXwxSzDy2F9huUqJ5TX+qwoZRFAZvZS4KRz7sG4y5JPkeV8COh1zu0EPgp8fUUKt9Bz\nnHNX4TVlvNXMnhdTOQopVM5yWJ81wNOBTzjndgFJ4I9jKEchxZSzHNYnAH4T4cuAr8RVhmIUKGfZ\nrM9KVBYBhNeJ9WVmdhj4EnC9mX0uY5pyuIxPwXI658adc2f8x98Gas1s3QqXE+fccf//Sbx262sy\nJimH9VmwnGWyPvuAPudccHHdr+Lt6MPKYX0WLGeZrM/AjcBDzrkTWcaVw/oM5Cxnma3PilMWAeSc\ne49zrsc5txWvqvsD59xrMyaL/TI+xZTTzDrNzPzH1+Ct46GVLKeZNZp3Tyb8JpgXAZlXE499fRZT\nznJYn865QeCYmW33B90APJwxWezrs5hylsP6DHkNuZu1Yl+fITnLWWbrs+KU21lwaczszbDsy/iU\nXEY5Xwm8xczOAVPAq93KX25iI/A1/3tRA3zBOXdnGa7PYspZDusT4O3A5/3mmCeA15fh+iymnGWx\nPv0fHC8E3hQaVnbrs4hylsX6rFS6FI+IiMSiLJrgRETkwqMAEhGRWCiAREQkFgogERGJhQJIRERi\noQASEZFYKIBERCQW/x9sUdcbjN2lKwAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Now let's calculate the GMM model of each of the three classes\n", "gmm_gaussians = np.empty((400,250,3,2))\n", "gmm_pdf = np.zeros((400,250,3))\n", "for y in range(0,3):\n", " for k in range(0,2):\n", " gmm_gaussians[:,:,y,k] = stats.multivariate_normal(mu_gmm[y,k,:],Sigma_gmm[y,k,:,:]).pdf(coords)\n", " gmm_pdf[:,:,y] = gmm_pdf[:,:,y] + c_gmm[y,k]*gmm_gaussians[:,:,y,k]\n", "\n", "ax=plt.gca()\n", "ax.contourf(coords[:,:,0],coords[:,:,1],gmm_pdf[:,:,0])\n", "ax.plot(X[0:50,0],X[0:50,1],'x')\n", "plt.title('Initial Gaussian Mixture Model of class 0, with scatter plot overlaid')" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbcAAAEICAYAAADcC7pLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXmYHVd55/957+3bq3pRt1q9qCXZlm3ZCGwJOxgcsBTZ\n4NjYwLDMsCRghoSB4PmRYOIMYJJASEIgIZDxEMfAxCFA+AWwARvbLAaLxbGDbcmy5U2StXe31Gr1\nenu/98wfVXX7LlX31t2Xfj/P0093V52qeuvU8q33nPOeV4wxKIqiKEotESi3AYqiKIpSaFTcFEVR\nlJpDxU1RFEWpOVTcFEVRlJpDxU1RFEWpOVTcFEVRlJqjoOImIveJyLvSrL9NRD7uc18PisjvFc66\n0iIiTSJyt4hMiMi3ctjeiMi59t93iMinstg2q/KlQkQOi8hV5bajGIjItIick2Z9zZ57MRCRHSJy\nvETH6hGRn4vIlIj8nc9tinY9q/3d50X8Oy1DuQ328xT0WP/nIvK1TPvJKG7ZXERjzDXGmH+xt7tB\nRH6ZtP59xpi/8LMvH3adJyLfFJEREZkUkf0i8r9FZMBev8OuzLuStrvYXv5g3DIjIqdEpC5uWche\nlmsg4JuBHqDLGPOWHPeREbd6VkqPMWaVMeYFqNyPC7+42V/J4lyA+n4vcBpoM8bcVIT9F41Kvi65\nYow5aj9PkXz2U5XNkrb6PwIMAtuMMW3AbwIHgVfGFR0BXiEiXXHL3gU877LbMeCauP+vsZflykbg\neWPMUh77KDrxgq4oxUAsKvldsxF42uiMFkWhbO8YY0zaH+AwcJX99w3AL4G/xXrxHwKuiSv7IPB7\nwIXAHBABpoFxe/0dwKfsv1cD92AJ0Jj990Dyvjxs+hpwdwa7dwDHgduAD9jLgsAJ4E+BB+PKGuAW\n4Ftxy74NfMyqIs9jXGjbOQ7sA15nL/8EsAAs2uf/HpdtXwb8h73tEHArUJ9k07nJ9eZyfK96/j/A\nD4AprA+BTUn7/gCwHzhkL7sA+DFwBngO+K9x5Rvsa34UOGnXaVOaevl94Bn72E8DL3W5lzzPHxDg\n74FTwCTwJPBie9219j6n7Gv54bjjXgfssff5EHBR3Lo/sctP2ed3pY97/93x95ldX/H3yDFga/z1\nwvICFu3rP+1sb5/7h4G9wATw/wONHsfdBPwUGMXyKL4OdHiUTVdXTcDfAUfsY/7SuW7At4Bhe/nP\ngS328hT7gX8FosCsvexmu+zL7XoeB54AdiQ9v38J/Mre7lyPd8tH7Os5BvyzUyfYz6+PZ821vl2O\ndTnwa/t8fw1cHvesxG9/VdJ2OV1P0tyLLra9GnjW3s+twC7sd1+6eyHNdXG9th7H7ge+j/XcHwB+\nP275LNAZV3abbUPI/v+/Yz3nY8APgY0Z3jHx77TXArux7tljwJ/HbXuWXbbO/v9su06msN5RtwJf\ny/j8+njAD5MobotYL68g8H4s70nibujfiyv7y6R93cGyuHUBbwKagVb7gnw36eHwErdh4IYMdu/A\nErfLgUfiXow/xBLgZHF7MdaLuwNLeE/ay4zH/kP2zfBRoB7YaVf+Znv9n6e7AMAlWC+HOvtiPgP8\nYZJNacUtQz2PYglIHdYD8c2kff8Y6MR6AbbYN9i77fLOTfwiu/zfYz0Anfa1uhv4aw973oIlIr+B\n9eI9F/umT7qXPM8fuBp4zL4WgvVi67PXDQGvsv9ezbJwbsN6wV+GdW++yz5eA7DZPr/+uIdnk5v9\nSedyDtbLKYD1sB/BfuHa68aAgJ/rZdvyn/Z+Ou3zfZ/Hcc/FeuE1AN1YL6jPe5RNV1f/B+s5WmfX\nyeVAQ9yLqdU+xueBPW7Pqdt7wP5/HdY9dq1dP6+2/++Oe36PAlvsaxzyeLc8Bay36+RXLL8fdsTV\ndaZnLcXepON02tfqd21b3mb/3+Vz+6yuJ2nuRZd9r7HP5c32ef4RsMTyezTtvZB8XTJdW5fj/xz4\nItAIbMVyNnba636KLXb2/58FbrP/fr19TS606/QW4CGvd4zLM7IDeIl971yE9b59Q9zzGS9u/wF8\nzj6fK+z6yihuuTQVHDHGfMlY7aH/AvRh9S1lhTFm1BjzHWPMjDFmCusrb7vPzddgCRwAInKjiIzb\nnZBfSjrOQ0CniGwG3gl81WOfc1gv7f9m/3zfXubFy4FVwKeNMQvGmJ9ieZ9v83MCxpjHjDEPG2OW\njDGHgX/C//n74S5jzH8aq1n061g3bjx/bYw5Y4yZxfrKPGyM+Wfbnt3Ad4C3iIhgfb3+kV1+Cvgr\n4K0ex/094DPGmF8biwPGmCPJhTKc/yLWw3kB1ofTM8aYobh1LxKRNmPMmDHmcXv5e4F/MsY8YoyJ\nGKvvdx7rOkWwHowXiUjIGHPYGHMwUwUaqw9tyq67K7A+jAZF5ALb1l8YY6KZ9hPHPxhjBo0xZ7Du\nteRr4hz3gDHmx8aYeWPMCNaD7XVvuNaV3Qz434EPGmNO2HXykDFm3j7G/zXGTNn//zlwsYi0Z3Eu\nvwPca4y51xgTNcb8GHgUS+wc7jDG7LOv8aLHfm41xhyz6+QvcX9+8nrWsLyE/caYf7Vt+TcsT+l6\nn9t74XU9092LyVwL7DPGfNuuo88T927L8l5wtvF1bUVkPVZ3zp8YY+aMMXuAL2O9JwG+gV3H9nvg\nrfYygPdhvUOesd8xfwVsFZGNcYeIf8ck2/igMeZJ+97ZC/yb23mJyAasD+WP23Xwc6y6zkgu4hZf\n8TP2n6uy3YmINIvIP4nIERGZxPqC6PAaIZPEKJaoOnbcaozpwLoxQi7l/xW4Efgt4C6X9Q5fxbqw\n6UTQoR84lvRyO4L1RZsRETlfRO4RkWH7/P8KS7QLxXDc3zOkXqNjcX9vBC6zPxDGRWQceAfQi/W1\n2Aw8Frfufnu5G+ux+j7Tku787ZfXrViexykRuV1E2uxN34T1QjgiIrtE5BVx53BT0jmsx/LWDgB/\niPWgn7IHIvVnstFmF9ZX5hX23w9iPYTb7f+zIdM1AWKj974pIifsuvkaHvdGmrpag/U1nnItRCQo\nIp8WkYP2/g/bq7K5/zZiffzE1/criXsuSbzHvIgvcwTruUomr2eNZa87nmy298Lrenreix62xerA\nWK5K7P9s7gW7fDbXth9wPlgd4uvlO1hjFvqw7v8o8Iu4c/xC3PmdwWo5iK9Tz+svIpeJyM/sAYET\nWGLpZeOYMSacZGNGitnJazKsvwmruegyYw0IucJeLj72/QDwxixs+VfgD7C+NGfSlPsFy55ophGI\ng8D6pI7yDVhNcn74R6yvx/Ps8/8o/s49mUz17Ge7Y8AuY0xH3M8qY8z7sZonZ7Ha7Z117cYYrw+a\nY1j9BJlIe/7GmH8wxlwCvAg4H/hje/mvjTGvB9YC3wX+Pe64f5l0Ds32VzrGmG8YY16J9VAa4G98\n2AjL4vYq++9dZBa3XK+Jw1/Z+3iJXTe/Q5p7w6OuTmO1PLhdi7djNStdBbRjNQMRdww3+5OXHQP+\nNam+W4wxn06zjRvr4/7egPVcJZPpWct0nEGs6x5PNs9qttcz7b2YxBBxdWB7SPF1kuleSLYt07WN\nZxCrVas1blmsXowxY8CPsFqy3o7VteEc7xjwP5LOsclYLWVetsXzDazWsfXGmHasfnw3G4eA1SLS\nkmRjRoopbieBARGp91jfivXSHBeRTuDPstj3nwOvEpHPicg6ABFZg9X+m4Ix5hDWy+hj6XZqX7jr\nsTqrM93Qj2B9rd1shw3ssLf9ps9zaMXqTJ22m7ne73O7ZDLVsx/uAc4Xkd+1zyUkIr8hIhfaX8tf\nAv5eRNYCiMg6EbnaY19fBj4sIpfYo+TOTWqqcPA8f/vYl4lICAhjvaSjIlIvIu8QkXa7CWcS62sS\n28b32duJiLSIyGtFpFVENovIThFpsPc162wndshImrrZheXxNxljjmN9AP02Vp/xbo9tTmL1yeVK\nK9YAgQn7/v5jr4JedWVft/8LfE5E+u0v+lfYddCK1Uw2iuWV/5UP+5OXfQ24XkSutvfdaNflQJbn\n+gERGbDfAR/DGpiRTKZnLVN934t1f79dROpE5L9hfQjc49PGbK+n573oUvYHwBYReaNYowr/P6wW\nE4dM90KybZmubQxjzDGswS5/bV+/i4D3YF1bh29gtWS9meUmSbDE6CMisgVARNpFJJuQp1Ysr3FO\nRF6GJZ5uNh7Bau7+hP38vxKfzcnFFLefYo1qGhaR0y7rP481mOE08DBWU5cvjDHPY3XWDgBPiMgU\nVmf0IOAaJG6M+aUxxu2rMLncPmPMPh/lFrAq+Rr7HL4IvNMY86zP0/gw1gWdwnoY3B5qP2Sq54zY\nzRKvwWpTH8RqbvkbrH4qsEYaHgAetps6foLldbvt61tYfSffwDq372J1KieT7vzb7GVjWE0Qo1id\n2WANCjhs2/E+rOZTjDGPYg10utXe7gDWYBvs8/g01nUaxvL6PmKvW4/1gHvVzfNYL5df2P9PAi8A\nvzLecThfwerfGxeR73rtOw2fAF6KNdrtB8Cdacqmq6sPY42e/DVWs9HfYD3zX7XLnsAaqfiwD/v/\nGrjFXvZh+8X4eiyPewTrS/6Pyf6d8g0s7+AFrCbUlHgyH89a2vo2xoxi9SvfhFU/NwPXGWP8Pi9Z\nXc8M92Jy2dNYg7A+bdt2Hta7zCHTvZBwXch8bZN5G5Z3N4jVZfNnxpifxK3/vm3TsDHmiTi778K6\nn75pP4tPkRhKlYk/AD5pv7v/lOUWGDfejvW+P4PlBGXqMgKWRzkqyopERL6MNbz/h+W2ZaUhIoex\nRgX+JFNZRckWDeBVVjTGmJqb5khRlCqdoURRFEVR0qHNkoqiKErNoZ6boiiKUnPUXJ9bfaDJNAXd\nRtwqiqIoXkwujZw2xnhNzlB11Jy4NQVbuXxN0TLMKIqi1CT3D3/R18wf1YI2SyqKoig1h4qboiiK\nUnOouCmKoig1h4qbsqIZ3XIO4d7E2cHCvZ2MbslnakhFUcqNipuyomkcHefE9m0xgQv3dnJi+zYa\nR8fLbJmiKPlQc6MlFSUbWobPsG7Xbk5s38bq544ytnkD63btpmX4TLlNUxQlD9RzU1Y8LcNnWP3c\nUU5ffB6rnzuqwqYoNYCKm7LiCfd2MrZ5A2ue2M/Y5g0pfXCKolQfKm7KisbpY1u3azfde/bHmihV\n4BSlulFxU1Y0c10dCX1sTh/cXFdHmS1TFCUfdECJsqLp2vdCyrKW4TPa76YoVY56boqiKErNoeKm\nKIqi1BwqboqiKErNoeKmKIqi1BwqboqiKErNoeKmKIqi1BwqboqiKErNoeKmKIqi1BwqboqiKErN\noeKmKIqi1BwqbkrVolm0FUXxQsVNqVo0i7aiKF7oxMlK1aJZtBVF8UI9N6Wq0SzaiqK4oZ6bUtWE\nezs586KzWX38JGdedDb19QGaJsOx9fXPHi+jdYqilAsVN6WqWLhgIPb3bFsLw+dupPfAEZomwzRN\nTSf8n1weVOwUZaWg4qZUPMkC5TDX0pwgZE2TYXoPHGGupTnBe3Pbl4qcotQ2Km5KReIlaPGsHhpJ\nWdY0GfYUtuT9q8ApSu2iA0qUopBrDNrCBQO+hK0QlOo4iqKUHhU3pShkG4NWSlFLPq6iKLWHNksq\nRcFvDJqKi6IoxUA9N6VopItBK5en5kal2KEoSuFQz00pGuHeTsY2b2DNE/sZ27yB5uFRQh3N5TZL\nUZQVgHpuSlFw+tjW7dpN95799Bw6zvGdlzLb1lJu01xR701RagsVN6UozHV1xPrYFi4YSIhBUxRF\nKTbaLKkUha59L1j9anHNkH5j0BRFUfJFPTel4FTSYJFsqEabFUVxp+yem4gEgUeBE8aY65LWCfAF\n4FpgBrjBGPN46a1U/FIMgZjY1OC6vP3gfMGPlczolnNoHB1PGOkZ7u1krquDrn0vFP34iqLkRtnF\nDfgg8AzQ5rLuGuA8++cy4B/t30oFUghh8xKyTGWLJXROMLrTfxg/UEZRlMqlrOImIgPAa4G/BD7k\nUuT1wFeNMQZ4WEQ6RKTPGDNUSjuV9OQratkIWqZ9FFrkNCGqolQn5e5z+zxwMxD1WL8OOBb3/3F7\nWQIi8l4ReVREHl2IzhbeSsWTfIRtYlNDQYQteZ+FRhOiKkr1UTZxE5HrgFPGmMfy3Zcx5nZjzKXG\nmEvrA00FsE7JRK6DRhxBK4YIxR+jkCQHoydPCK0oSuVRTs/tN4HXichh4JvAThH5WlKZE8D6uP8H\n7GVKGcnVWyumoBXrWMnB6E4TpQqcolQ2ZRM3Y8xHjDEDxpizgLcCPzXG/E5Sse8D7xSLlwMT2t9W\nXvLx1qqR+GB0WO6Dm+vqKLNliqKkoxJGSyYgIu8DMMbcBtyLFQZwACsU4N1lNG3Fk62wVaugxeM2\n3L9l+Iz2uylKhVMR4maMeRB40P77trjlBvhAeaxSHKpV1CY2NZQkFk5RlMqj3KMllQqnnMI20baO\nmcbE5r+Zxg4m2lIGzCqKoiRQEZ6bUplkI2y5itrURvFcF12YYaZ+LfOrhbr6cYIn25lpWUtz+FRO\nx8pE/bPHi7JfRVFKj4qb4koxhS2doMVTVz8OQGSxm2hkFWZVE83Tp2ieG8/qeIqirDxU3JQU/Apb\nNqLmV9CSqasft4Qt2owEZoj0TDCF0HrE5LQ/RVFWBipuSgKFFrZcRc1haaEDE21CAjOYaBNLCx0x\nj05RFMULFTclRiGFLV9RA0vYIovdBEMj1NWPx/639j+u3puiKJ6ouCkVJ2oO0UhzTNhguQ8uGmkG\n1HtTFMUbFbcVTqUKG0B902DKMkvgxmPHK5T3piMlFaW2UHFbwRRK2AotaoqiKPmi4rZCyUfYznSc\nRWgxTGt4JCZsi3PdRKPNNDQf8dzX/IYFX8dsOFrvq1whGOvrpjE8Q/wRNdO2UmlEWucZfduzdH3j\nQoLT2T8f+W5fjegMJSuQfD220GKY+abVnOlZC9jCFukgEJhJKTu/YSH245dsyuZLY3iGk2cPxGb5\nd7IANI5qn55SOUzuPMrCWRNMXun98VjM7asRFbcVhh9hyziL/5rTBILjRCMdzIfPtoQtOE6ocSRW\nJFtBS6ZUAtc0GY6lsRnZel4svY1OjKxUCpHWecKXnoQAhC85SWRVds9GvttXKypuKwi/wpYOpxnS\nErIlIAQsxYQtX1GLp1QCp5m2lUpmcudREHvgVMBk7X3lu321ouK2QshX2KY2SsLAkcW5bqwu20Wg\njvlQV0mbE/3gJyNA/bPHNdO2UrHEvK46W5zqTFbeV77bVzMqbiuAQghbPLE+tuA4DS2HoOMMjHfC\nUF/etiZTbMHUTNtKJZPgdTlk4X3lu301o6Mla5xCCxtANNps9bmdfwLLN7KTo4dX5WZkGUmXaVub\nJ5Vys7Bxctnrcqgz1vISbF/NqLjVMMUQNoCG5iOpHlXfUFa2VQL1zx6ny2W5ZtpWKoWef7ikrNtX\nMypuNUqxhG1+wwKMroHwLLSEl1eEW2CuCbpOA3DWwIjr9rFjH97A2Hwg7T4YXQONs1j9ehbRSBPR\nSCN19WNp968ZuBVlZaN9biuUnIUNLME5sd4SI7B+n1hPz/ohzhoYyShsAPVtkwSG1qXswxIzEo4T\njTQBlrAtzvURCM4B6MTJiqJ4op5bDZLJa/MStnTTaCU0Q7aEYd0xS4xWnyEw0UH31idp6vQf+NzU\nOU73RfsY2buFaPs4jHVa+4z35OzjLB5dTzA0QWSxnVDjEIHgrPeOfaJzSSpKbaOeW41RaGHzjFtr\nCcPqM3B6La0Dg1kJm0NT5zitA4Nweq21r3hhizuOJWxdBEMTvoRNmyQVRVFxqyGKIWxe9DQvEpjo\noP3sw0wd72f2TId/Q21mz3Qwdbyf9rMPW56b00QZT7iFyGI7wdAokcX2WBNlPqjXpii1jzZL1gil\nFraRvVvovmgfTZ3jNHaOJfzvh9kzHQnbTESM1cwZ3zRp98M5TZGB4AyLc32EGodoP546j6WiKIqD\nem41QKmEzRkssjDZliBkTv/ZwmSbf5uT9hHrx5uL88zmmgiFlvvYAsFZQo1DRCONnvvN1CSpXpui\nrAzUc6tySilsDu1nHU1Z39Q5nlW/m9s+aAkn9rt1nSYQTkzPYXlw+Q8oURSltlFxq2LKIWzpeHXv\nsynLfjx8ga9ts8ErBEAHkiiK4qDNkhXG6JZzUuY1DPd2MrrlnKIeN0HYRtdAuCUhZm32TAcThzek\nbPfq3mdjP254LS8H2iSpJBNpnefUe59YERMJrzRU3CqMxtHxhIl7vZJnFtJrS/HYGmcJDK2LjYB0\nBn/Uty3PR5dO0JLJVeBKmZFbWZmsxCSeKwVtlqwwnIl7T2zfxurnjjK2eUNK8syiChtw1ubDzHaP\nM7J3C60Dg0wd748N/iiGJ3b4eLfvsrk2SarXpiSTnMSz7YGNBKf1g6pWUM+tAkmXPLPowmY3QzoB\n1hOHzooFaVdSE6Oi5MtKTeK5UlBxq0C8kmeWStggMcB66ng/Wyhtigy3Jkn12pRCsZKTeK4UVNwq\njFyTZxZa2JwA6//6yvu4Zsd9PPDz1zA43J/l2ZQfFTbFjZWcxHOloOJWYXglz5y+cGPW+8pF2GA5\nwPp1L3oYgP7eQa684keMjK7N2oZMuPW3FdJrUxQ3VnISz5WCDiipMLr2vZCyLNTRzOoh71gzN68t\nV2EDK8A6uX+tv3eQ/t5BTxsqEfXaFC9WchLPlYJ6bhVOrv1sfvAKzi7nwBH12hRFKQRlEzcRaRSR\n/xSRJ0Rkn4h8wqXMDhGZEJE99s+flsPWclHMASTphO3+B65l79MXJSzf+/RF3P/AtQAZ1wM8sW9r\nrI/OmaUkORA8mxCAZNoPzjPW181sW2Imgdm2Fsb6utVrU5QVTjk9t3lgpzHmYmAr8Nsi8nKXcr8w\nxmy1fz5ZWhMrl3wHkLjheGz9fcd55LHLYwK29+mLeOSxy+nvO+5rPUB316mEQShugeDJZOO1ATSG\nZxg+d2NM4GbbWhg+dyONYc0YoCgrnbL1uRljDDBt/xuyf7zfZCuMTF6bG/n0s8U3RV70or0APPLY\n5Rw5djbDp/q47JKHYsszrYflQSgP/Pw1NPSfTAgEh/y9NoCmyTC9B44wfO5G2k+OMtHTRe+BI7T/\n53M571tRlNqgrH1uIhIUkT3AKeDHxphHXIpdLiJ7ReQ+EdnisZ/3isijIvLoQrT6Z4zPpTmyUMLm\ncNGL9tK7dojhU/30rh1KEC4/68ESuIb+kwmB4F5k67U5NE2GaT85ythAD+0nR1XYFEUByjxa0hgT\nAbaKSAdwl4i82BjzVFyRx4ENxphpEbkW+C5wnst+bgduB2gPra1p76/QA0i8Bo/sffoihk/10bt2\nkOFTfcjBTt607ScAfHvypSnr9z59UYrADQ73JwSCN3aO0dQ5XhCvzWG2rYWJni5WHz/JRE8Xrb2d\nCTO6KNVHLq0W+aJ9tLVHRYQCGGPGReRnwG8DT8Utn4z7+14R+aKIrDHGnC6HnaWgWM2RftPWwHIf\nmtPUKAc7+dJDbwHgTdt+ghzsTFjvlIflJsvB4X7ue/Aa12zdyeTqtTl9bL0HjtA0GaZ136FYALwK\nXOVSDvHKhJdNKnrVS9nETUS6gUVb2JqAVwN/k1SmFzhpjDEi8jKsZtTR0ltbGorZHOmGl9c2ODTA\nZZc8xCdffoe1YJv1a/fxC3nTtp+w+/iFrn1wg0MDsb9HRte6Zus+eawPutJ/m/gd+j/X0hwTNlgO\neJ/r6lBxqwAqUcSyJfkcVOyqB7HGdZThwCIXAf8CBLFE69+NMZ8UkfcBGGNuE5EbgfcDS8As8CFj\nzEPp9tseWmsuX/OW4hpfBAo17D+ffrZ43tz2eNr13558adr1bklK85mNROePrGxqQchyoZbuu/uH\nv/iYMebScttRKMo5WnIvMZ8gYfltcX/fCtxaSrsqkWwzaseTi7BlIpOw+SXX5shkaukFUy2sVDFL\nxqkHvQcrD52hpAKIf1G4BSYvhJqYaUqcONmzOfLoRhjtSlg+cWSAk49flFLeiz27fpPv7L4qYdl3\ndl/FLXf/TyAxQNthcLifJ/ZtBSyvbeLwhliyU7C9tnCLleXbJxNt65hp7Ejw2sZ7uxg8/yzf+6gU\ndmw6xB1vv4sfvPdr3PH2u9ix6VBO+8mUObpYmaUXLhhI+FES0bqpPFTcykzyw5AcmDxyQQdTrf3U\nLaUPcYg1R7ZMw6leGO3irIERJo4MMPb8uTR2WX1Qfpojtw08w5ceektM4L6z+yq+9NBb2DbwDN+e\nfGlKgPbgcD8P/Pw1dHedijVH1rdNMrJ3y7LAhVvgxHpotM7Dj9cWWggz07KW8V5LrMd7uxjd0E/z\n5FSsTDV8Me/YdIgPbn+YntYwAYGe1jAf3P5wTgKXKXN0oTJLq5jljtZZZVARoyWVZeIDkxsWx5lr\n7KB1apD6xWVxS9sc2WWPtznVy9Cvm5gfb2f1+Qdo33jcd3OkM+T/Sw+9hYcObWPf0Ln8/uXfwmyy\nBDI+QPvC8/fxzPNbuPKKH9HfO8i+4TbrPOwBJCN7txBtH4exTlh3DFrCvpsjm+fGaRpdYnRDP+HV\n7cy1ttB1dJCOYescq0HYAG64bA+NoUjCssZQhBsu28ODB8/2vZ9MmaPzzSytL+TCsnDBQNXco7WI\nem5lxOtl4gQmzzavoXFuPKOwpQwi6RqloWOC+fEOGjomaN/o7wGLH0Typm0/YUvfAZ4aOp8tfQdi\ngufQ3zvIhefvY/eTl3Lh+fvo7x1MGUTS1DluCdvptbD6DLSEfdnh0H5wno7hURqnwsy1raJxKhwT\ntmqie5X7eXst9yJT5uhsM0urd1Z8tG7Lh4pbBTLb1sJ4fxdNM6eZa+xgIdTkWdZtdOTqSAPz4+00\ndIwzP97OxJGBrAeRfGf3VewbOpcX9z3PvqFz+dOHb0hYPzjczzPPb2HbSx7lmee38P2nU6cFPfzc\nWZbHtuaU9TvckvXoyPHeLuZaW2icnGautSXWRFlNX8Qj0y1ZLXcjU+Zov5mlVczKg9Z36VFxKxNe\nN7sTmNxUJMXaAAAgAElEQVQ6NUjLzCitU4NMtfazEGryNTqS0S7Gnj+X1ecfoO83drP6/AOMPX9u\nyiz+6XD62H7/8m/xiqse4GWX/EfCRMlOH9uVV/yIS7f+2kpkGt+/hjVRMifWW02R3aes30fXE40k\nCnW60ZFOH1vX0UHWPfMCXUcHGd3Qz1SgulrT73hkK3OLwYRlc4tB7nhkq+99ZMocnW69ClploPVf\nWlTcykC6m3yupZlVM8t9bPWLs7RODbJUl+q9uXltTZH6WB8bQPvG41x2yUMMDmV+sJwh/ruPX5jQ\nx3bRi/Ym7GNkdG2sjw1gH1bm7oXJtti+Th7ri/WxAdASJtQ4RDTSGCuTKVh7pq01oY+tY3iUrqOD\nhPv8j7isBB48eDZf2PVyTk61EDVwcqqFL+x6eVb9bZkyR3utnzt/Ll/zlQKiAlc6yhbEXSyqIYg7\n3Q1erBxtxUKDtSsLfXlWB5V4H2sQt5IXxUxl40YuwvbEvq10d52KeWZgNUWOjK7l4i17Yuv3seyp\nzZ7pYGGyjbG61JAFFbbi4txTO7v38Ybz7+azAw3cfGyeu/ZfwU9HXBNpFJ3F1WOMX/MTOu67itDY\n6rLYUMnoSMrio82SJSTXKbZKTbo4Nmf9fQ9eE+tjcxKRjs2n3k46C0nxiO9H29m9j5s238edPQEe\nb2zgzt4AN22+j53dqRNVl4LJK36FCS0yeUXa2fJWNOplFxf13CqcfPK05Uq6ODZY7mMb2buF1oHB\nWCLSkzOhhP1kI2yZvDZlGbeX4nvO2cVUfZTvrWrBiPDdVS28b3yC95yzq+Te2+LqMSIdkyAQ6Zhg\ncfUYM529RT+u3kNKPCpuJaIcXls+fW3xcWzbXvJoTNicPramznFaBwaZOHQW7WcfThG2bNDmSG/8\nft2vbZjkLztWE8X6GIoi3NbRzkeXxoppXgoTmxqY35borY1d+RANu99YkmMnU+mCp82TxUPFrYIp\nh9fmkBzH1t97IqWPzUlEOnF0IHFkJNrPliu5NlU9E2nne6taWAxY98xiwPLe3jgcLaR5CbiJSbR5\nFFomwLl1BWiZINo8SmCmK6V8sYm3sdKFTiksKm4loFr62hzi49j6ewfp7z2RkHjU6WOLNUWuO7Yc\n05bF9FqZWAnCVqh+l1tC5xAlMU9eFOFj9ecUZP/g7z5d3LzLc3kpvLd0OPZXmsip91YcVNwqlHxG\nSOZLuji2ps5xFibbEvvYWsKWsM010TC6mLI/7WdbpliDCA61RokEEu+ZxYBwqDVKrv5STh9dTdPL\nXpuD2MsrhEoVOaWwqLgVmWJ7bcVokrx4y57Y3/F9bE5W7fazjqbGsrWECypstfQlW4pRcV13X1OQ\n/eR7PzY89M6C2FEKVORqGxW3CsTXNFtFxi042yGfIG1YGcJWTcO8K61ZvNRMbGpQgatBVNwKyOiW\nc2gcHadl2Jq2auGCAWbbWphraWb1UKqH5fZSmWnqJBqZIxBcDoaORppYbAsBp61kpC3Ty6ltsJKR\nzo120vPSvQU5j28//Brq2yZjnhoUJkgbalfYdmw6xLte+SRrGyY5Nd/GV17YXrYA6nRYgd4/57MD\nDfzx0ALfOLOTH86+JKGMCc2weMGDhJ7dgSw2F8UOP8dIV+bqpie5sf0BeoITnIy0c+vElSnnkY0d\n46EddDwbzLyBUjVoEHcBaRwd58T2bYR7razZziTIjeEZ3/uoW5plca4vNsFwNNLE4lxfLMlnfDJS\nICUZaT78ePgCfjx8QUqi0WyDtL2oRWFbuGCAV75qgg/+1iP0Nk4SEOhtnCxrALUXL3/x83zogvtj\ngd53rRFuWX03Vzc9mVBuacMeTPtJljbs8dhT/vg5hleZq5ue5JbVd9NXN0FAoK9uwvU8srVjpXuw\ntYaKWwFpGT7Dul27ObF9G8NXXsLwuRvpPXCEpsnUvF1eD9J8/xyhxiEW5/pYWuhica6PUOPQ8jD7\nrlFYOwynemkYGohlAHDL2ZauaTG5XHzZ+ESjYwfOthKO9p1IycfmJWy5jIysJpJn2X/PObtoDC4l\nlGkMLvGec9xHDpaaiU0NTGxq4Mb2B5gORRICvcOhCDe2PxAra0IzRHsOgEC05wAm5P/DzC9+jpGu\nzI3tD9AUSOzfbQosJpxHoexQqhcVtwLTMnyG1c8dZWygh/aTo67ClolAcJZgaILIYhfB0ASLZ08k\nFugahaYZX8lIvQTOETSv9fFB2tH28byFrRa8Nq+0MWsbJl3Ley0vBY6gxX9E9QQnuK2jPSXQuye4\nfH8tbdiznDpHTFG8Nz/HSFcm3t54vJZnY4d6b7WD9rkVmHBvJ2dedDarj59koqeLpqnpFIHLNPN/\nNNJEZLGdYGiUyGI7hCcTxWW0C2abE5KR5iJw6Zg902EFZzuJRpvDMRtWmrBlGhxyar6N3sZUITs1\n3+ZSurikezk/TYdroPd/OW1dt5gnE7ADvwNRy6M5urVgfW9+jpGpzMlIO311qUJ2MtJeEDtA+95q\nAfXcCki4t5MT27fRe+AInSdO0nvgCMPnbmS2zX/GZaePLdQ4RF39qNUkeWI9hO19jHZZfW5rhxOS\nkU4cKdzovNkzHZzc85LERKO2DStF2LJJ8PmVF7YzF0n8TpyL1PGVF7YXxJZ4LyzTTzo+smpTzGtz\niCL8r1WbgCRPxqHA3pufY2Qqc+vElcxGE6d7m42GuHXiyoLYod5bbaCeWwGZ6+qg59DxmKfWNBmm\n98AR5lqafTdPRiONhBqHYqMlA8FZWGsFSNMShvAqq8/NHi3peGxzo51pvbdscEs0yrpjBIdWQX3q\nXIW1JGy5DOF3RkW+55xdeY2WLPZL9UhzBOMS6H2kOUI9YNpGlj0Zh0DUWl4g/BwjUxlnVGQ+oyVL\nca5KedFkpQUmn6DtbGclKXQAt1v8mkOte2zliEurNQ8h1/jMShyAVOq4t0p4HjRZqeJJPi/IXF4M\nh493F0Tg0oka1LawlVLUaknMCjnRQPK+KlHslOpDxa2E+Hm5LS2sJhBMDOIm3GI1S3ad9t7QOcbh\nDZ4B2O1nHQXg+K9eRmPnGGsu3L8sbEN9VpPnuftj26WLYat2YSuVqFWioOUSpJ2NmH08cg9XBnfz\nke4uPj0yyk8i2/iL4HW+t5/aKFwX3csb5GfcPNDC3x4Pc+/Ib+UUpF0NVMozUWvogJIKIxCcSwji\nJtxiDeZoTJ0ZBFK9Lq8A7Pq25dF8jZ1jTB9fx+FfX2QtGOqD8U4rQNymGMJWCfgdJJIPfgd4lAu/\nQdpTGyX245ePR+7hHTzKlzvaeLyxgS93tPEOHuXjkXt87+O66F4+Ze7m73obmA4In+1tyDlIW1m5\nqLgViEK9MAPB2YQg7vhUMl7EC5xbALaTqubw8W4OH+9munUcOs5Ygrb/fOt3xxnoGwLSN0PmI2zl\n/EIttqhVuqA5ZApczkXQ4nkrj3E6GEgIFB8NBngrj/nex4fMAxwJGQ7Wh0CEg/UhjjWYrIO0lZWN\nNktWAMkvkvggbtacSitsDin9Zu3jTBw6C9acslLTzCSt7xuC6VZYqoe6BV/C5kUlC1uxBa3acAtc\nDh28vGB9aEEMt3V0pASKf2zUf0bwPib4QHdvwrKbu9dw54nhgthYSWiTZPFQz60A+HmBZvMijA/i\nZqxzOcbNL+EWazsnANtt+6E+WApZwrYUIvD8OhU2n1SDh+aGW+BypPcAU2e7N3nnwnAw6BoofjLo\nPzD6F6HVMa8NiHlv/xFcXTA7ldpHxa3CSAniDiUFcWfC6aNzCcCO4fSxdZyB854nEBwnGulgcS51\n1GQ1CluxmiCrVdQcvAKXg+2FC9K+ucM9UPzmjk2+9/En3e6jdz/ssbzQlKrvWL224qLNkhVGxiDu\nTMw1uQZgJ2wfXgUdZ2hYHIWj9dA4wuIcRKOJI+eqTdiKJWiloFDNgumumVvgskgUaThVkGMDPN7Y\nTCAwl7BsMSA81uh/+q7p0CIiSfUhwnTDItX7aaGUGhW3PCn0C7XOZQYQWsL+hA3cwwWStm+oPwJJ\nSbNDjcvxcpnijFaCsBVT1IqZjNYrZmxqo8CZN0D+mZHSsjT8hrz3sXj8Xa7LSxH/pl5b7VA2cROR\nRuDnQINtx7eNMX+WVEaALwDXAjPADcaYx0ttK6QmIgVrLsnpvu5YItKxvm4awzMJU23NtrUwvqGN\n5tnc3yoNR+vTzlSSgEsyU0a7ILyKBqxBI26xdNFIE9FII6uHzjDT1End0iz1i8vrF0JNLNU10ffU\nUEYTCvng7th0iBsu20P3qjAj0y3c8chWHjx49rJdGUTtxk0/5Pr+PQTFEDHC3YNbufXg1Qlldnbv\ni02dlWviy6ubnuRtnT/lb/vqU5KAxgQnMEPdmp/B6d+CJC/5uuhe3hX4KZ/rruemkQXuiO7knsBF\nWdngRk5CWn+CUM+PWDz5GlhYl2Lnh8wD9DHBEO18Tq5MsdPXudh1seRSF4Ui34SoxSL5+Yi0zjP6\ntmfp+saFBKfd+739lFESKWef2zyw0xhzMbAV+G0ReXlSmWuA8+yf9wL/WFoTl0lOROpMkhyfiLQx\nPJMwUbKTrLRuKf8Oe99JQZOSmToTLQcXlm1IjqVz+vlaTltl6pZmmWrtZyFkrV8INTHd3E/H0cwp\nXAotbB/c/jA9rWECAj2tYT64/WF2bDpk2eVD2N6wbjd1AYMI1AUMb1i3mxs3/TBWZmf3Pm7afF8s\n0WguiS+d5Jl3rZHlJKBdd7N9/ZMJ4hJs34M0nEzp43Liur7XYW3/vQ7hU+ZurosWJrN6toS6fwYY\n+3eqneuYIACsYyLFTr/n4lUXXuTitWWbELVccZqTO4+ycNYEk1ceyauMkkjZxM1YOFHDIfsn+Q5+\nPfBVu+zDQIeI9JXSTof4RKQjW8/jxPZtCZMkw/JEycPnbuTMup5YstJ4DygfGo7WZxa5uGSm8tw5\nlrCFRqirX56xJDmWbmmmj7aJwZid9YuztE4NMtXaT7i5i+nmfs+kq/EUuqnlhsv20BiKJCxrDEV4\n1yuf9NUMeX3/Hly6bri+f/ll55ZoNNvEl65JQIMRPmTi9hGYIdCyHxEItOyHwPJH0YfMA0wHM2xf\nKupPQGDBqrfAgvV/nJ1NSe3ZTSwm2OnrXNLURaHIJSFqpKlwo0a9cPPawpeehACELzlJZFVqC42f\nMkoqZR0tKSJBEdkDnAJ+bIx5JKnIOuBY3P/H7WXJ+3mviDwqIo8uRIt3gzqJSE9ffB6rnzvq+rJv\nmgzTfnI0r2SlmXBEzvMnPIUEZjHRZiQwmyBsDvGxdI1z4ykCXL84S+PcOLPNa3ydRzH6ELpXuR/T\nbxLQYPLIwKTlE5saPPeVTeLLnjr3JKB9LO8j2J4YXxbvsfSReftSkeytxf/vZU/8cj/nkq4uCkXW\nCVExhC9+quB2xOP2jEzuPLpsQ8C4emZ+yiiplFXcjDERY8xWYAB4mYi8OMf93G6MudQYc2l9oKmw\nRsYR7u1kbPMG1jyxn7HNG1zztM22tTDR0xVLVuonl1uhO8qXFjow0SYkMIOJNrG00JFSpmGwkeh8\nO00zp5lr7Ig1QToshJqYD3X4Oo9idY6PTLsf028S0Ihx72+KILEBI14JLjMlvoyfyeOpoHsS0H1B\nu95jnoo1UlEkmuCxZNy+VMR7bZDivQ3hXifxy/OtCzeyfT48E5HGeW8pZYJR5s59oWjem9szEvPI\n6uzzqzMpnpmfMoo7FRHnZowZB34G/HbSqhPA+rj/B+xlJcfpY1u3azfde/azbtfulESkTh9bcrLS\nZOEoJksLHUQWuwmGRqhvOk4wNEJksTtB4BoGG5lq7ad1apCWmdFYE2RyH5ufpKvFHPV1xyNbmVtM\nDP7NJgno3YNbSc7oZAzcOX1J7P9sE1+6TU31JxliuxI8FYc4jyXT9qUi2WtLXv45uZJZkuqKEJ+T\n5brKty6SybWvLdeEqMX23uJJ8MgckjwzP2UUd8ombiLSLSId9t9NwKuBZ5OKfR94p1i8HJgwxmQe\nrlcE5ro6WLdrd2y0ZKijOZaINFampTmhb8rpg1uqK524RSPNCX1sdfXjBEMjRCPNsbkhl+qaaJ1K\n7WNbqmui/eA8dRMh1/OIP1co/nDmH4Vexd/tv5bhuTaiBobn2vi7567xnQT01oNX890T21iKCsbA\nkhG+NX0pn5l4bazMD2dfwqfGrmdoqZ2ogaGldj41dn3KaMl08y0ebYzEPBWHxYBwtNHqL5SGUzFP\nxSE+vizT9iUjzmtziHlvwD2Bi7hFrucE7USBE7Rzi1yfMBIy37qIJ9cWjZwTogajLHZnzryRLV7P\nycLGyWWPzKHOWMuzKKO4U7ZkpSJyEfAvQBBLZP/dGPNJEXkfgDHmNjsU4FYsj24GeLcx5tF0+y1V\nstJs4qr8xkwVK/7J70vC72ixUsToFCpuLd94tWLGpCne1EpMWzXFs2my0gJhjNkLbHNZflvc3wb4\nQCntKietR0xBX6bZvCBU2BJRUSsfKmxKIdAZSgqIVxD3TJP/IO58BS7bF0M2D3m1CFu8qF3d9CQ3\ntj9AT3DCM0A7uczfhq7kHvIPoI7nuuhe/ijwE/ojkwwG2/jM6h18v3V5/NTrpp7i5rEHXdc74R9+\nAqgzBUf72keG88hn+0xUmrBFmmaZ3P5L2na9kuCs/+4FFbbyo+JWQJwgbqe/yhlgsmpmMKv9JEyZ\nlEX5bKkkYSuGt+YEVzcFrNgsJ0AbiAmcW5lPmbshSt4vbWdWmddNPcVfjN5Lc8SKpxuITPLp0XsB\n+H7ri3nd1FN8evRemo37+vkNC7xu6ik+dfq+WJyZE0CdbGd8cHRk7PIEe5wA60z78CLf7TNRacIG\nEL74SRZ7Rghf/BRtD/+Gr21U2CqDihgtWSsUOojbGQCS6Sdb2g/O17ywgRVc7YiWQ3KAtmuZpMDk\nbJjfsBD7cbh57MGYcDk0myVuHnvQ13qnTKYA6kzB0X6CsNOR7/Ze5HofZ0O29zxYXtvcuYdA8B0m\noMJWOai45UC6l3EpgrjzIdsHvBqEzSsVjVcgdvzynrrMgcl+SBa0ePoj7iPbnOWZ1qcrk01wtJ8g\n7HTku70bleitOYQvfjKhPjOFCaiwVRYqbgXGLYi7XHPWxZOLt1YtwuZFpgDtqY3iKzDZCzcvzY3B\noHvQubM80/p0ZWJ2+giOzudcC7F9PKXw1iB3YYt5bUF/Qd4qbJWHilsB8Qri9jNLSbHIpTmmGgeO\nuOEVoP23oStj/Zl+ApPd8J2lAfjM6h3MSGL39ozU8ZnVO3yt9yoTb6ef4Ohcz7VQ20PpRA3yGxGZ\n4LU5uHhvpfgIVHJDB5QUEK8g7rmWZlYfHCl5FudcHu5aETZYHjSSMhIybvDDPYGLIIrvEYDZiJqD\nM+rRazRkpvVuZZLt9BMcne25JpPP9qUSNCjMMP/F7tFlr80hKchbRa2yKVsQd7EoRRB3ri/nUolb\nrg93LQmbG/mEWOQiasXEdwqkMlNKUQNNNpoPGsSt5Ez7wfmiCVw+D3WpHtRyCVu+AdmVJmyVTqkF\nDUonalCbwlaLqLiVmEILXL4P9YoQtgyBzR+P3MNbeYwghgjCN7mEvwhelyBqwgyN/JI5XoWh8HOF\nfmLkft4xvTtmw9dXbePPupPnEV+2IxrY6Z7B2k+G67pRQr33sjh8LSx1JazKNUg7G0HzE1ifDeX0\n1kqVIVszcWePDigpA7kM8nDbXoUtPY7Hli7r88cj9/AOHqUOgwB1GN7Bo3y06fsJ5ep5kiCnqMd/\nhm6/fGLkft45/XiCDe+cfpxPjNyfUraeJwmaU56z6PvJcF23ZhfIovU7Dj+ZtuPJJdbSCZrvq5vI\nOfO5QyGeAb94PSulypCtmbizR8UtBwolCn5EKr5MoR7mUo7wKrewZQpsfiuPkdxoKcA7pnfH/T9D\niBcQIMRBhMLm/HrH9O6MNiTY4ZXB2k+G67pRJDSOCEhoHOpGY6syBWnnO3kA+Ausz0QpRQ28n/dS\nZcjWTNy5oeJWIbiJWDEe4FL2F5Rd2Mgc2BzEI1t33HLLW1vO2Fxo782PDTE7TJpzyXCurUcM9W0P\nJghp3ZpdMbHqMx5B2maiYP1ofgLrvSiHqKV7XkqVIVszceeGitsKodTxOJUgbH4CmyMpPlPi8mWv\nzd4H0YJ7b5lsSLDD61wynGvrEUO0eRRaJnB2KwKB0Li1nNyzkmdDLscotahB5o/AUmXI1kzcuaPi\ntgIo9eiucg73j8dPYPM3uSTFbzLA11dZ2ZgSvbblEoX03r6+altaG2J2JIftxJ2Ln3Nd3JzYx5a8\nPNus5LmQzTHKJWp+npdSZcjWTNy5o6Mla5hyDFmupDg2P4HNfxG8DiIkjJaMH6kY5HTMa4vtgyhB\nRigUzrHSjZasWxhF6r3PJd25xpoUm6Zx7dxrmgbcg97zHcmYjJ9jlGO6umyflVJlyNZM3LmjQdx5\nUKjZ7AtNOeNwKqI5MksqPY4tn4DtcsSc5Uq55mDVuDULDeJWKpZyP6QqbIVnJQibippSDFTc8qD+\n2eMV4b1VwkPqVg87u/fxnnN2sbZhklPzbXzlhe38dGSL5z6KMXtLITJHf2Lkfi6Zf5Lf7+vhK0Mn\n+XXDS1wDrNORLtO2FzkJmx3E3bhnB+AexG1CMyxe8CChZ3cgix6B3kWmnJkyKuF5UYqPilsVUykP\nqZew3bT5PhqDVhLO3sZJbtp8H4CrwBVjvshCZI52Aqz/S38v0wHho91d3DX4OIBvgcuUaduNXD22\nYPseAg0nWdqwh9DBy13LLG3Yg2lPX6YYlDv1U6U8L0pp0NGSeVKOB6Ya0my855xdMWFzaAwu8Z5z\n3EfsFYNMQcl+miTfMb2b50J1HKwPgQgH60PsD9WlBFinw0+m7XhybooMzBBs3g8C0Z4DmFBqELcJ\nzRDtOZC2TKEpx6jHZCr9eVEKj4pbASjVg1OJoubVLLu2wX00l9vyYs3yX4jM0UEM/6t7TcKym7vX\neAZeu+En0zZYopZPH1tjYHdCEPfShtQpuJY27MlYJl+KPRFBNlTiM6OUBm2WLBDOA1ToPrhqfTBP\nzbfR25j6Uj81755RuhgM0c46FyHLJnP0vlAo5rUBMe/tmVAo/YZxDAbbGHARuPjs2vmmsFk1GGbh\nNw5AwA4HCEQtz+zo1li/WsxrS1MmV8otYslU63OjFA713ApMIb4UnX1U+gOaTsi/8sJ25iKJ305z\nkTq+8sL2hGXFzHFXiMzRN/auc13+Bx7L3UiXaTtfb82ZOivBI3NI8sz8lPFLJXln8VTDc6OUBvXc\nikTyA+YlBNX6IGbyUJ1BI9mMlswGP8P/8808DXC6TpDkyGcRTtf5Dz9wy7T999Gr+OFYdqM2k4kf\n6m/aRpY9ModA1FqeRRkvKknAkqnWZ0gpLhrEreREuafYyjcBqUOp49wKkUG7mPFrlSxiyaioFRYN\n4lZWPJUQ2+eQbxxbw9F6rl79eNoYtFxi1JKP4QsfiUbzFbZqEi8vVqKoabLS7FFxU6qWQsSxXRfd\ny6dO3xfbR3IMWi4xag7ZemnxiUYjY4nxZ/mImgpa9ROfrHT1984rtzlVgQ4oUbKikry2THFsue4j\nPgYtmxg1Z3BIToNE0iQaXcnCpgNENFlprqjnppSFQoySLEQcm1fZ/sgkDUfr08aoFaL/zMEt0Wjz\nnlfktc9qFbaVLmbJuCUrVe8tM+q5KVWLV7xaNnFsmfZRiGNkxCXRaLB5f86zh1Ta8Hy/qJeWiiYr\nzR0VN8U3ldQkCYWJY8u0j0IcIxNeiUZzjT+rJqolprNcaLLS3NFmSaUqaT1iuGdj/nFsmWLhChEr\nl4k6OYWR3OLPHKpJ1FTI/KPJSnNH49wU3xTScytEn1uhYt3KRSHi1apF1FTQKh+Nc1OUCqH1iKlK\ngStUEHYlC5uKmVJuyiZuIrIe+CrQAxjgdmPMF5LK7AC+BxyyF91pjPlkKe2sZbIJDE322rJNRFos\n3nxqL3/Q5d1k6DvI20cAdT4UclYRL1Hb2b2PN5z/cz470MDNx+a5a/8VJb0m8YKmQcdKuSmn57YE\n3GSMeVxEWoHHROTHxpink8r9whhzXRnsq3lyDQzNNhFpsbi66UluWe0dxJ1NkHe6AOpcKfQ0Wek8\nNeeafHZtK483NnBn7yIfDhT3mqTzzjToWCk3ZRstaYwZMsY8bv89BTwD+J9qXcmLfAJDKyERKcCN\n7Q/QFPAO4vYd5J0mgDpbnFn6C+2pZWqCfM85u5iqj/K9VS0YEb67qoXp+mhBr0n8yMZ0wqZBx0ol\nUBGhACJyFrANeMRl9eUisldE7hMR109QEXmviDwqIo8uRGeLaGnt4BYY6pdsEpF6UYj+op5g+iBu\nv0HebgHU2VAMQYPs4tXWNkxyW0c7UTuDQRThto72rK5JPMlClk0fWj73lqIUirIPKBGRVcB3gD80\nxiQ/iY8DG4wx0yJyLfBdIKWNwxhzO3A7WKMli2xy1eMVGNr2wEZf/SOVkIgU4GSknb66VAE7udRO\n63HDyV739QkB2C4B1IGW/UQmtrr2vRVzRn6HXIT/mUg731vVwmLAErfFgOW9vXE4mmHLwg7+yPfe\nUpRCUVbPTURCWML2dWPMncnrjTGTxphp++97gZCIrCmxmTVHvoGhfhORZiJf7+3WiSuZjSYFWEdD\n3DpxZdr1Xxy9MuZpNQZ2IyTWhWAtj/fIiuGZJZPPzCK3hM6JeW0OUYSP1Z8DuHtixQie1qBjpVIo\n52hJAb4CPGOM+ZxHmV7gpDHGiMjLsMR4tIRm1iT5BoYWOxGpX344+xLA6nvrCU5wMtLOrRNXxpZn\nWg/5JfAsFIVooj3UGiUSSBS3xYBwpH6OnhIOy9egY6VSKFsQt4i8EvgF8CTgvF0+CmwAMMbcJiI3\nAu/HGlk5C3zIGPNQuv1qEHdxKObUW4UI6K42ihmjpjFmSi5oEHeBMMb8EkgbgWuMuRW4tTQWKeWi\n/eij7iMAAAj+SURBVOD8ihE4FTVFKQ1lH1CilIcdmw5xw2V76F4VZmS6hTse2cqDB8/2LF//7PGi\nem+5CtzN7T/gjaseI4ghgnDn9CV8ZuK1RbAwd4o9k4iKmqKkouK2Atmx6RAf3P4wjaEIAD2tYT64\n/WGAtAJXbLIVuJvbf8BbVj2K2P5/HYa3rHoUoOwCV4qpsVTUFMWbiohzU0rLDZftiQmbQ2Mowg2X\nZZ9ipdBkM2Lwjaseiwmbg4i1vBw4tquwKUr5Uc9tBdK9KpzV8nLgCEQ6Ty6I+2Aor+WFptQTF6ug\nKYp/VNxWICPTLfS0pgrZyHRL2u2K3e/mRjqRiyDUuQhZJP04pbxtKTUqaoqSPSpuK5A7Htma0OcG\nMLcY5I5HtpbRqvQkC8vEpgbunL4koc8NwBi4c/qSgh6rXKioKUruqLitQJxBI9mMlnQoh/fmRvvB\nef6JqwhtinB9/x6CYogY4e7BrfzTwatopzIEKhdU1BQlfzQTt5ITlSBwtYaKmlJONIhbUZSCoYKm\nKMVBxa2EjG45h8bRcVqGz8SWhXs7mevqoGvfC2W0LJVMQd6V0jxZKRnBs6UYopZtYL6i1DIa51ZC\nGkfHObF9G+HeTsASthPbt9E4Ol5myxJxgrx7WsMEZDnIe8emQwnlyu11ONmnexsnCchyRvCd3fvK\napcXxZqJH/xfM0VZKai4lZCW4TOs27WbE9u3MbL1PE5s38a6XbsTPLlKIJsg72K9rP1QKRnB01FM\nQYunkgPzFaUcaLNkiWkZPsPq545y+uLzWPPE/ooTNsgtyLsczZSFyAheDMoh9tUQmK8opUQ9txIT\n7u1kbPMG1jyxn7HNG2JNlJWEVzC3nyDvUuKV+bvUGcGhdB6aF7leM0WpVVTcSojTx7Zu12669+yP\nNVFWmsDd8chW5haDCcv8BnmX8gVfqIzguVDMbNa5kM81U5RaRJslS8hcV0dCH5vTBzfX1VFRzZP5\nBHk7xL/wi9VcWcqM4JUgYOkoxDVTlFpCg7iVklEJoQN+qXQxU5RCo0HcipIjyYJRCWJXKhHbsekQ\n1//mE3zm7Gb+5NAM3//VxVl7VRrHpij+UXFTyoaXsBRa9MrthTkxaJ/tbePxxga+M7DIh7NMDlup\nCWYVpVJRcVMqjnKLUaG54bI9TDUavreqBSPCd1e18L7GCW64bI9vYUoXx6bipiip6GhJRSky3avC\n3NbRTtTOMxdFuK2jPasYNI1jU5TsUHFTlCLz7NwqvreqhcWAJW6LAct7e25ule99aBybomSHipui\nFJlbWB/z2hyiCLeY9b73oXFsipId2uemKEXmcAcxr81hMSAcWg09PvehcWyKkh0qbopSZHr+4ZKC\n7OfBg2ermCmKT7RZUlEURak5VNwURVGUmkPFTVEURak5VNwURVGUmkPFTVEURak5VNwURVGUmkPF\nTVEURak5VNwURVGUmkPFTVEURak5VNwURVGUmqNs02+JyHrgq1jT6xngdmPMF5LKCPAF4FpgBrjB\nGPN4qW1diWjWZ0VRqplyzi25BNxkjHlcRFqBx0Tkx8aYp+PKXAOcZ/9cBvyj/VspIpr1WVGUaqds\nzZLGmCHHCzPGTAHPAOuSir0e+KqxeBjoEJG+Epu64kiX9VlRFKUaqIisACJyFrANeCRp1TrgWNz/\nx+1lQ0nbvxd4r/3v/P3DX3yqKIYWljXA6XIb4cZr/y3gMY39NI8Nf/Gx0lrjm4qtzyTUzsKidhaO\njeU2oJCUXdxEZBXwHeAPjTGTuezDGHM7cLu9v0eNMZcW0MSioHYWFrWzsKidhaVa7KwlyjpaUkRC\nWML2dWPMnS5FTgDx6YoH7GWKoiiK4knZxM0eCfkV4BljzOc8in0feKdYvByYMMYMeZRVFEVRFKC8\nzZK/Cfwu8KSIOCMVPgpsADDG3AbcixUGcAArFODdPvZ7e+FNLQpqZ2FROwuL2llYqsXOmkGMMeW2\nQVEURVEKis5QoiiKotQcKm6KoihKzVHV4iYiQRHZLSL3uKwTEfkHETkgIntF5KXlsNG2JZ2dO0Rk\nQkT22D9/WiYbD4vIk7YNj7qsr4j69GFnpdRnh4h8W0SeFZFnROQVSesrpT4z2Vn2+hSRzXHH3yMi\nkyLyh0llyl6fPu0se32uFMoe55YnH8Sa2aTNZV0lTd2Vzk6AXxhjriuhPV78ljHGK9C0kuoznZ1Q\nGfX5BeB+Y8ybRaQeaE5aXyn1mclOKHN9GmOeA7aC9aGIFQ50V1KxstenTzuhMu7PmqdqPTcRGQBe\nC3zZo0hFTN3lw85qoSLqsxoQkXbgCqxQF4wxC8aY8aRiZa9Pn3ZWGlcCB40xR5KWl70+k/CyUykR\nVStuwOeBm4Gox3qvqbtKTSY7AS63m1LuE5EtJbIrGQP8REQes6czS6ZS6jOTnVD++jwbGAH+2W6O\n/rKItCSVqYT69GMnlL8+43kr8G8uyyuhPuPxshMqqz5rlqoUNxG5DjhljKnUeQ4B33Y+DmwwxlwE\n/G/guyUxLpVXGmO2YjXvfEBEriiTHZnIZGcl1Gcd8FLgH40x24Aw8L/KYEcm/NhZCfUJgN1s+jrg\nW+WywQ8Z7KyY+qx1qlLcsALAXycih4FvAjtF5GtJZSph6q6MdhpjJo0x0/bf9wIhEVlTYjsxxpyw\nf5/C6id4WVKRSqjPjHZWSH0eB44bY5yJwL+NJSLxVEJ9ZrSzQurT4RrgcWPMSZd1lVCfDp52Vlh9\n1jRVKW7GmI8YYwaMMWdhuf8/Ncb8TlKxsk/d5cdOEekVEbH/fhnWNRktpZ0i0iJWTj3sZqnXAMmZ\nFcpen37srIT6NMYMA8dEZLO96Erg6aRiZa9PP3ZWQn3G8Ta8m/rKXp9xeNpZYfVZ01T7aMkEROR9\nkNfUXSUhyc43A+8XkSVgFnirKf20MT3AXfYzVwd8wxhzfwXWpx87K6E+Af4n8HW7ieoF4N0VWJ9+\n7KyI+rQ/Zl4N/I+4ZRVXnz7srIj6XAno9FuKoihKzVGVzZKKoiiKkg4VN0VRFKXmUHFTFEVRag4V\nN0VRFKXmUHFTFEVRag4VN0VRFKXmUHFTFEVRao7/ByTOOY5LSeHuAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ax=plt.gca()\n", "ax.contourf(coords[:,:,0],coords[:,:,1],np.amax(gmm_pdf,axis=2))\n", "ax.plot(X[0:50,0],X[0:50,1],'x',X[50:100,0],X[50:100,1],'o',X[100:150,0],X[100:150,1],'^')\n", "plt.title('Initial GMM of all three classes, with a scatter plot of the data overlaid')" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# Re-training the GMMs: K-means clustering" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "K-means clustering is an algorithm that clusters some input data vectors, $\\vec{x}_n$, into $K$ different clusters.\n", "\n", "The algorithm alternates between two steps.\n", "1. Each datum is assigned to the cluster with the closest centroid:\n", "$$k(n) = \\arg\\min\\Vert \\vec{x}_n-\\vec\\mu_k\\Vert$$\n", "\n", "2. Each cluster centroid is re-estimated as the average of the data in its cluster:\n", "$$\\vec\\mu_k = \\frac{1}{N_k} \\sum_{n:k(n)=k}\\vec{x}_n$$\n", "\n", "We go back and forth betwen these two steps until the total cluster distance, $D$, stops decreasing, where\n", "$$D = \\sum_{n=1}^N \\Vert \\vec{x}_n-\\vec\\mu_{k(n)}\\Vert^2$$\n", "\n", "\n", "All of these steps have to be done separately for each of the three classes." ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt0VNd98P3vT1cQEhLSSLIMCAESINux8RWMBcZGbmKv\n5rGbPs7rt3nSJE9Tkjxvs+K3SWOvJE3aXLpyeZw2WUnbuI6XkzZ247Rxksdp3sSAbcDG2IDvCJAA\niYtBoyu6IXT7vX/srWFQdBlJM5oZ6fdZa5bOzDmzZ++zpd8c/c4++4iqYowxJvmlxLsCxhhjosMC\nujHGzBIW0I0xZpawgG6MMbOEBXRjjJklLKAbY8wsYQHdmCkQkWIR2SkinSLyUIw/q15EqmP5GWZ2\nsIA+x4jIn4jIPhHpEpEzIvIbEamKd72S0FagGVioqp+Od2Wiyf+ONIhIt4j8QkTy410nExkL6HOI\niPwl8A/A3wHFQCnwfeC/xbNe4UQkLd51iNAy4KBO4cq8RG6jiFwJ/AD4IO53pAf4x7hWykROVe0x\nBx5ALtAF3DvONpm4gP+Of/wDkOnXbQZOAZ8GgsAZ4CN+3TrgLJAaVtYfAW/45RTgQeAo0AI8CeT7\ndWWAAn8GnAB2+tf/FGjw2/81UA9UT6K8D/nymoHPh9UrFficf28nsB9Y6tetAZ4BWoHDwPvH2E+P\nAf1An9+n1RHuuwf8fvrXMcr9c6DG1+sgcJ1/PbztNwF7gHbfB98DMvw6Af7e908H8CZwlV93ly+z\nEzgNfGaMOvwd8HjY85W+nTnx/h22RwR/5/GugD1mqKPhPcAAkDbONl8GXgKKgELgReArft1m//4v\nA+k+QPQAi/z6o8AdYWX9DHjQL3/Kl7vEB74fAE/4dcMB+MfAAmA+cIUPlFVABvC/fQCtnkR5/+LL\nuga4AFT69X/lA91qHwCvAQr8Z58EPgKkAdfivgyuGGNfPQZ8dZL77hu+vvNHKe9eH2hv9PUqB5b5\ndfVhbb8eWO/rWIb7Arjfr3s37gsqz5dRCZT4dWeAjX55Ef7LYpR6/BJ4YMRrncD18f4dtkcEf+fx\nroA9Zqij4QPA2Qm2OQrcFfb83UC9X94MnCfsCwF3JLjeL38VeNQv5wDdYQGpBtgS9r4SH6CHg5IC\nK8LWf3E4QPvnWbijxOpJlLckbP3LwH1++TBw9yht/7+AXSNe+wHwpTH21ciAPtG+6wPmjbPvfwt8\naox1oYA+yrr7gaf88u3AER/wU0ZsdwL4GC7nP97vwHbg4yNeOw1sjvfvsD0mflgOfe5oAQIT5G8v\nx6U5hjX410JlqOpA2PMeINsvPw68T0QygfcBB1R1uKxlwFMi0i4i7biAPIjL0Q47OaIeoeeq2uPr\nPyyS8s6OUc+luOA70jJg3XCZvtwPAJeNsu1oJtp3TaraO877x6rXJURklYg8LSJnRaQDlyIJAKjq\nDlwK5vtAUEQeFpGF/q1/jPuvqkFEnheRm8f4iC5g4YjXcnFH6SbBWUCfO/bgUg/3jLPNO7jANqzU\nvzYhVT2IC2J3An+CC/DDTgJ3qmpe2GOeqp4OLyJs+QwunQKAiMzHpUUmU95YTuLywqO9/vyIMrNV\n9RMRlAkT77uJTp6OVa+R/gk4BFSo6kLc+QAJfYjqd1X1elzaahUuxYSqvqKqd+NSQr/AnXcYzdu4\nNBQAIrISl/Y6EkHdTJxZQJ8jVPUcLpXxfRG5R0SyRCRdRO4UkW/6zZ4AviAihSIS8Nv/2yQ+5nFc\nfnsTLoc+7J+Br4nIMgBf/t3jlPMfwHtFZIOIZAB/Q1jQmkJ54R4BviIiFeJcLSIFwNPAKhH5oN8v\n6SJyo4hURljudPfdI8BnROR6X6/y4faNkIM74dklImuA0BeOr+86EUnHpbx6gSERyRCRD4hIrqr2\n+/cPjVGPn+D2/UYRWQB8Bfi5qtoRehKwgD6HqOpDwF8CXwCacEeFf4E7YgOXB98HvIE7cXjAvxap\nJ4BbgR2q2hz2+neAXwG/E5FO3MnDdePU823gk8C/447Wu3D5+gtTKW+Eb+OOTn+HC2w/xJ2k7AT+\nALgPd2R9losnMSMxrX2nqj8Dvob7UuzE9clo478/g/sPqBN34venYesW+tfauDhC6Ft+3QeBep+m\n+TgunTRaPd7263+C2+cLgP8VaTtMfImq3eDCJDYRycYN06tQ1ePxro8xicqO0E1CEpH3+rTQAtyw\nxTdxoz2MMWOwgG4S1d1cvEinAjfs0P6dNGYclnIxxphZwo7QjTFmlphwkiARWYq7LLsYN5b2YVX9\njp+B7ae4K/PqcfNetI1XViAQ0LKysmlW2Rhj5pb9+/c3q2rhRNtNmHIRkRLcfBAHRCQHN1fEPcCH\ngVZV/bqIPIib0+OB8cq64YYbdN++fZG2wRhjDCAi+1X1hom2mzDloqpnVPWAX+7EXWa9GHfS6kd+\nsx8x/hWI0/L8kSYee+E4dcEuLOdvjDGjm9S8zCJShpuFbi9QrKpn/KqzXDqPRvh7tuJuBkBpaemU\nKvm7t8/yk70nACjJnUdVeYCqigC3lAcIZEd63YcxxsxuEY9y8Rd3PA98TVV/LiLtqpoXtr5NVReN\nV8Z0Ui4nWnrYVdfEC3XNvFDXwrnz/QBUlixkow/uN5XlMz8jdUrlG2NMooo05RJRQPdzQzwN/FZV\nv+1fO4ybUvOMz7M/p6qrxysnWjn0wSHlrdPn2F3XzO7aZvY3tNE3OERGago3lC3ilvIAGysCXHl5\nLqkpMnGBxhiTwKIW0EVEcDnyVlW9P+z1b+GmUx0+KZqvqp8dr6xYnRTt6Rvg5eOtvFDXzK7aZg6d\ndfMI5WWlc8vKQCjAL83PivpnG2NMrEUzoFcBu3CXXg/P0PY5XB79Sdw0oQ24YYut45U1U6Ncmjov\n8EJdc+gI/myHm4Z6WUGWC+7lATasDJCblR7zuhhjzHRFNeUSLfEYtqiqHG3qYnetC/B7jrbQ3TdI\nisC7luRRVV5AVXkh1y3LIzPN8u/GmMRjAX0M/YNDvHayPRTgXzvZzuCQMj89lZuW57Oxwo2gWV2c\ng8s2GWNMfFlAj1BHbz97j7Wyu7aJXXXNHGvqBiCQnemO3isKqSoPcFnuvDjX1BgzV0Ua0Cc1Dn02\nWjgvnTuuKOaOK9ww+nfaz4dy77tqm/nFa+4uYuVF2VT5k6vrVhSQnTnnd50xJsHM+SP08QwNKYfO\ndrK7roldtc28fLyVCwNDpKUI15bmUVVeSFVFgGuW5JKWavOcGWNiw1IuMdDbP8iBhjZ2+SP4t945\nhyrkZKaxfmWBy7+XB1geWGD5d2NM1FhAnwFt3X28eLQldAR/qu08AJfnzqOqIkBVRSG3rCygwKYn\nMMZMgwX0GaaqnGjtYVetO3p/8WgzHb0DAFzhpyeoqghwY1k+89JteKQxJnIW0ONscEh58/Q5N3qm\ntpkDJ9roH1Qy0lK4sWwRVeWFbKwIcEXJQlJsegJjzDgsoCeYnr4B9h5vZXdtMy/UXZyeYFFWOhv8\n1au3lNv0BMaY32fDFhNMVkYat60u4rbVRQAEO3tDc8/srm3m12+4mYjLCrJc/r08wM0rA+TOt+kJ\njDGRsSP0BKCq1AW7XHCva+alYy30+OkJrl6SF5oe+LrSRWSk2fBIY+YaS7kksb6B4ekJmthd18zr\np84xOKRkZaSybnl+6OrVVcXZNjzSmDnAAvos0tHbz56jLaH8+7FmNz1BUU4mVT73XlURoHihTU9g\nzGxkAX0WO9XWE8q/v3i0hdbuPgBWFWeH5n5ft7yABTY9gTGzggX0OWJoSDl4poPdde7ofXh6gvRU\n4drSRaH7r1692KYnMCZZWUCfo3r7B9lX3xa6/+pbpzsAyJmXxoaVBT7AF1JWkGX5d2OShA1bnKPm\npaf6aQcCALR29/kba7sUzW/fbgRgcd780NH7LeUB8hdkxLPaxpgosCP0OURVqW/pCY2eefFoC51+\neoIrL19IVUWAjeWF3FC2yKYnMCaBWMrFTGhgcIg3Tp8L3b3pQEMbA0NKZloKN5blhy5wsukJjIkv\nC+hm0rovDLD3eAu7a90MkkcauwDIX5DBhuHpgSsKWZw3P841NWZusRy6mbQFmWncvqaY29e4uzc1\ndvSGxr7vqmvmaT89wfLAgkvy73b3JmMSgx2hm4ioKkcau9hV60bP7D3eSk/fIBmpKdy8soDqK4qp\nriyiJNeO3o2JNku5mJjqGxhiX0MrO2qCPFPTSENLDwBXLV5IdWUx1ZXFXHn5QhsaaUwUWEA3M0ZV\nOdrUxTMHg2yraeTAiTZUoSR3ngvuVxSzfkU+mWk2csaYqbCAbuKmuesCOw4F2XawkV21zZzvH2RB\nRiq3ri6kurKY21YXscjGvRsTMQvoJiH09g/y4tFmnjkYZHtNI8HOC6QI3FCWzx3+6H15YEG8q2lM\nQrOAbhLOkL8t37aaRp452Bi6a9PKwgVUX1HMHZXFXFu6iFQb827MJSygm4R3srWH7TWNbKsJ8tKx\nFgaGlPwFGdy+pojqymI2VgRsxkhjsIBukkxHbz/PH25iW00jzx4K0tE7QEZaCrf4IZFb1hRzWa7N\n927mJgvoJmn1Dw7xSn0r2w4GeabmLCdbzwNw9ZLc0JDIypIcGxJp5gwL6GZWUFVqg108c7CRbTWN\nvHayHVU3W2R1ZRHVVxSzbnmB3WvVzGoW0M2sFOzs5dlDQZ45GGR3XRO9/UNkZ6Zx6+pC7qgsZvPq\nQvKybEikmV2iFtBF5FHgD4Ggql7lX8sHfgqUAfXA+1W1baIPs4Buoul83yAv1DWzzZ9Ybe66QGqK\ncGPZIqori7njimKWFdiQSJP8ohnQNwFdwI/DAvo3gVZV/bqIPAgsUtUHJvowC+gmVoaGlNdPtbOt\nppHtNcHQkMiKomw/z0wxa5fm2ZBIk5SimnIRkTLg6bCAfhjYrKpnRKQEeE5VV09UjgV0M1NOtvb4\nI/dG9h5rZWBICWRfHBJZVREgK8OGRJrkEOuA3q6qeX5ZgLbh56O8dyuwFaC0tPT6hoaGSNtgTFSc\nO9/Pc4eDbKsJ8tzhIJ29A2SmpVBVHvBDIosoWmhDIk3imrGA7p+3qeqiicqxI3QTb30Dfkikv1r1\nVJsbEnnN0jyq17hRM2susyGRJrFYysWYCQzP8T4c3F872Q64IZF3+Lz7TcvzbUikibtY37HoV8CH\ngK/7n7+cYjnGxI2IsPqyHFZflsP/c1s5wY5eN0tkTSNPvHyCx16sJ2d4SOQVxWxeVURuVnq8q23M\nmCIZ5fIEsBkIAI3Al4BfAE8CpUADbthi60QfZkfoJlmc7xtkd10z2w42sv1QI81dfaSlCDeW5Ycm\nEistyIp3Nc0cYRcWGRMlQ0PKa6fa2eavVh2+efaq4uzQDTzWLskjxYZEmhixgG5MjDS0dLOtxt3A\n4+X6VgaHlEB2Jlv8SdWq8gDzM+zuTCZ6LKAbMwPO9fTz3JEgzxxs5PnDTXRecEMiN1YEqK4s5vbK\nIopybEikmZ5YnxQ1xgC5WencvXYxd69dTN/AEC8fvzgkcltNEIC1S/NCo2ZWFWfbkEgTM3aEbkwM\nqCqHznaG8u6vnzoHwNL8+W6emcpiblyeT3qqDYk0E7OUizEJpLGjl+01bkjk7rpm+gaGyJmXxm2r\nXd791lWF5M63IZFmdBbQjUlQPX0D7Kp1QyJ3HArS0u2GRK5bkR+6gcfSfBsSaS6ygG5MEhgcUl47\n2cYzB93Re13QDYlcc1lOaEjk1YtzbUjkHGcB3ZgkdLy5m+3+pOor9a0MKRTmZLq7M1UWc0t5gHnp\nNiRyrrGAbkySa+vu47kjQbYdDPL8kSa6LgwwLz2FjRXu7ky3rSmiMCcz3tU0M8ACujGzyIWBQfYe\nc0Mitx1s5J1zvYjAtUvzQlMRlBfZkMjZygK6MbOUqnLwTAfbfN79zdNuSOSygiyqK4u5ekmuBfYE\ntH55/pTn3bcLi4yZpUSEKy/P5crLc/lUdQVnzp0PDYn81z0N9A0OxbuKZhSPfeTGmN9IxQK6MVPQ\n2NjIvffey6uvvsrWrVt56KGHYvZZZWVlPPLII1RXV4+6viR3Pv9j/TL+x/pldF8Y4My53pjVxUxd\nSW7sp4CwgD7HPP7443z729/m0KFD5OTksHbtWj7/+c9TVVUV76ollYcffphAIEBHR0dCpTcWZKZR\nXpQ95fefOXOGj33sY+zbt48zZ85w/PhxysrKoldBE1MzmkMXkSbc/OlTEQCao1ideIpXW4qBy3B9\n0AEosBDIAU5NsczZ0i+TbccyoB94JzbVucS7gHqgM8Ltp9MnacAioAdYA7wJ9E2xrGiYLb9fML22\nLFPVwgm3UtWkeAD74l2HZG4LkAt0AfeOs00m8A+4IPWOX8706zbjgv6ngSBwBvgIsA9YB5wFUsPK\n+iPgDb+cAjwIHAVacDdHyffrynBfLH8GnAB2+tf/FPfF0wL8NS6gVU+ivA/58pqBz4fVKxX4nH9v\nJ7AfWOrbsQZ4BmgFDuNu3DLafnoMF8z7/D6tjnDfPeD307+OUe6fAzW+XgeB6/zr4W2/CdgDtPs+\n+B6Q4dcJ8Pe+bh24YHyVX3eXL7MTOA18ZoLflzS/H8vm2t9KMrcl7o1MpJ0xm9sCvAcYANLG2ebL\nwEtAEVAIvAh8xa/b7N//ZSDdB4ge4FW//ihwR1hZPwMe9Muf8uUu8YHvB8ATft1wAP4xsACYD1zh\nA2UVkAH8bx+kqidR3r/4sq4BLgCVfv1f+UC32gfAa4AC4ABwEvcllQZci/syuGKMffUY8NVJ7rtv\n+PrOH6W8e32gvdHXqxx3VAaXBvTrgfW+jmW4L4D7/bp3476gXvVlVAIlft0ZYKNfXoT/shjnd8EC\nehK2Je6NTKSdMZvbAnwAODvBNkeBu8Kevxuo98ubgfOEfSHgjtRr/PJXgUf9cg7QHRaQaoAtYe8r\n8QF6OCgpsCJs/ReHA7R/noU7Gq6eRHlLwta/DNznlw8Dd4/R9l0jXvsB8KUx9tXIgD7RvusD5o2z\n738LfGqMdaGAPsq6+4Gn/PLtwBG/f1JGbHcC+BiwMMLfFwvoSdiWZJq78+F4VyCK4tGWFiAgIuOd\nCL+cS89xNPjXQmWo6kDY8x7gN375ceB9IpIJvA84oKrDZS0DnhKRdhFpxwWcQVxOf9jJEfUIPVfV\nHl//YZGUd3ZEPYfPFC7FBd+RXgXWDZfpy/0A7pxDJCbad02qOt7wk7HqdQkRWSUiT4vIWRHpAP4O\nl5tFVXfgUjBZQFBEHhaRhf6tf4z7r6pBRJ4XkZsjbFe82d/9JCRNQFfVWdOxcWrLHlzq4Z5xtnkH\nFyyHlTLxSb//AlDVg7ggdifwJ7gAP+wkcKeq5oU95qnq6bBtws/On8GlUwAQkfm4tMhkyhvLSWDl\nKK//J/D8iDKzVfUTEZQJE++7iUYfjFWvkf4JOARUqOpC3PmA0DAbVf2uqi7Dpa1W4VJMqOorqno3\nLiU0fJP3hGd/95OTNAHdTI+qnsOlMr4vIveISJaIpIvInSLyTb/ZE8AXRKRQRAJ++3+bxMc8jstv\nb8Ll0If9M/A1EVkG4Mu/e5xy/gN4r4hsEJEM4G8IC1pTKC/cI8BXRKRCnKtFpAB4GlglIh/0+yVd\nRG4UkcoIy53uvnsE+IyIXO/rVT7cvhFycCc8u0RkDRD6wvH1XSci6biUVy8wJCIZIvIBEclV1eET\npmNefSQi83C5foBM/9wkAQvoc4iqPgT8JfAFoAl3VPgXuCM2cHnwfcAbuBOHB/xrkXoCuBXYoarh\nw7O+A/wK+J2IdOJOHq4bp55vA58E/h13tN6Fy9dfmEp5I3wbd3T6O1xg+yHuJGUn8AfAfbgj67Nc\nPIkZiWntO1X9GfA13JdiJ65P8kfZ9DO4/4A6cSd+fxq2bqF/rY2LI4S+5dd9EKj3aZqP49JJYzmP\n2+fg/hs4H2k7TJzF+0TBKCcO3oM7cVWHHyUxYr0A3/Xr32CCs/UJ3pbNwDngNf/4YrzrPEY7HsUF\n1LfGWB/TPsHlvweA5TFuR1L0h6/rUuBZ3FDEtxnlhGqy/K1E2JaE7xtgHu4E/Ou+HX87030S950w\norGpuBNDK3DD1V5nxLAx3Imd3/gdsx7YG+96T6Mtm4Gn413XCNqyCbhunEAY9T4B3os7ubcAl2J5\nFX8hXAzbkRT94etawsVx6jm40S3J+rcSSVsSvm/8fs72y+nAXmD9TPZJoqVcbgLqVPWYqvbh/uUe\nmRu9G/ixOi8BeSJSMtMVjUAkbUkKqroTd7HNWGLRJ3dz8SKdCtyww4lOLI4rgnYkDVU9o6oH/HIn\nbqTP4hGbJcXfSoRtSXh+Pw+nqtL9Y+TvbEz7JNEC+mIuHb52it/v2Ei2SQSR1nODiLwhIr8RkStn\npmpRF/U+UdWPqhtpkquqW1T18LRqGLmk6w8RKcNdCLV3xKpk+VsJGactkAR9IyKpIvIaLrX3jKrO\naJ/Y5FzxdQAoVdUuEbkLdyKsIs51msuSrj9EJBs35PJ+Ve2Id32mY4K2JEXfqOogsFZE8nDXSlyl\nqm/N1OfP6ORcgUBAbeY2Y4yZnP379zdrBJNzzegRellZGXbHImOMmRwRiWiW2qRIuTz2wnH2HGth\nY0UhmyoKKS3IineVjDEm4SRFQB8YUt463cFv324E3L0TN1YE2FhRyM0rC1g4Lz3ONTTGmPhLmptE\nqyrHm7vZVdvMrtom9hxtobtvkNQUYe3SvFCAv2ZJLmmpiTZ4xxhjpi7Sm0QnTUAfqX9wiFdPtLOr\ntomdtc28caodVciZl8YtKwNUVQQsPWOMmRVmfUAfqb2njxfqWthd18TOI82cbnfTT1h6xhiT7OZc\nQA9n6RljzGwypwP6SOOlZzasLLDRM8aYhGYBfRyWnjHGJBML6BGy9IwxJtFZQJ8iS88YYxKNBfQo\nae/p48WjLS7Aj5KeqSovZEO5pWeMMbFjAT0GLD1jjIkHC+gzwNIzxpiZYAE9Diw9Y4yJBQvocWbp\nGWNMtFhATzCWnjHGTJUF9AQ3XnqmqtwdvVt6xhgDFtCTiqVnjDHjsYCexCw9Y4wJF7WALiJLgR8D\nxYACD6vqd0QkH/gpUAbUA+9X1bbxyrKAPjWWnjFmbotmQC8BSlT1gIjkAPuBe4APA62q+nUReRBY\npKoPjFeWBfTpuzQ908yeo82jpGcCXLMkz9IzxswSMUu5iMgvge/5x2ZVPeOD/nOqunq891pAjz5L\nzxgz+8UkoItIGbATuAo4oap5/nUB2oafj8UCeuxZesaY2SfqAV1EsoHnga+p6s9FpD08gItIm6ou\nGuV9W4GtAKWlpdc3NDRE2gYzTZaeMWZ2iGpAF5F04Gngt6r6bf/aYSzlklRGpmfePNXOkKVnjEl4\n0TwpKsCPcCdA7w97/VtAS9hJ0XxV/ex4ZVlATyxjpWdK8y/eucnSM8bEXzQDehWwC3gTGPIvfw7Y\nCzwJlAINuGGLreOVZQE9cVl6xpjEZRcWmWmx9IwxicMCuomqsdIzKwoXUF1ZzJY1RVy/bJEdvRsT\nAxbQTcwMp2d2Hmli+6EgLx1roX9QyZ2fzubVhWypLObWVYXkzrfcuzHRYAHdzJiuCwPsOtLEtpog\nzx4O0trdR1qKcGNZPlsqi6iuLKYssCDe1TQmaVlAN3ExOKS8drKNbTVBdtQEOdzYCcBKn5q53VIz\nxkyaBXSTEE629rC9pvGS1ExeVjqbV/nUzOpCGxZpzAQsoJuE09nbz67aZrbVNPLc4aZQauam5fls\nqSymurKIZQWWmjFmJAvoJqENDimvnmhj+6Eg22saOdLYBUB5UTZbKovYsqaY60ptzLsxYAHdJJkT\nLT1sP9TI9poge49fTM3ctrqILZVFbFplqRkzd1lAN0mrs7efnUea2V7TyLOHg7T19JOWIqxbkc+W\nNcVssdSMmWMsoJtZYTg1s63GpWZqg5emZqori7mudBGpKRLnmhoTOxbQzazU0NLN9pog2w81svdY\nKwNDyqJQaqaYTasC5FhqxswyFtDNrNfR2++uVvUXNLX39JOeKqxbXhA6sWpzzZjZwAK6mVMGh5QD\nJ9rYVuNOrNb51ExFUXZoSOS1lpoxScoCupnTGlq6Q3n3l4+71Ez+ggw2ry6kurKYjRWWmjHJwwK6\nMd658y41s+PQ6KmZ6spiluZbasYkLgvoxoxiYHCIAyfa2V7TyLaaRo42dQOwqvhiambtUkvNmMRi\nAd2YCNQ3d4fy7q/UX0zN3La6iOrKIjauKiQ7My3e1TRznAV0Yybp3Pl+nj/SxI6aRp493MS58y41\ns35FAVvWuGGRlpox8WAB3ZhpGBgcYn+Dm2tmW00jx3xqZnVxjhsSWVnM2qV5lpoxM8ICujFRdLy5\n200DXBPk5fpWBoeUggUZbLbUjJkBFtCNiZHh1Mx2Pw3wufP9ZKSmsG5Fvru/amURSxZZasZEjwV0\nY2bAwOAQ+xraQkfvx5pdambNZRdTM9cssdSMmZ6oBXQReRT4QyCoqlf51/KBnwJlQD3wflVtm+jD\nLKCb2e5YU1dorplX6ttCqZnb1vjUTEUhCyw1YyYpmgF9E9AF/DgsoH8TaFXVr4vIg8AiVX1gog+z\ngG7mknM9/Tx3JMj2miDPHQ7S0TtARmoK61cWUF1ZxO1rLDVjIhPVlIuIlAFPhwX0w8BmVT0jIiXA\nc6q6eqJyLKCbuap/cIh99W2h+6seHyU1s3ZJHimWmjGjiHVAb1fVPL8sQNvw81HeuxXYClBaWnp9\nQ0NDpG0wZtY62tTFjho3JHJfg0vNBLIzQtMAb6wIWGrGhMxYQPfP21R10UTl2BG6Mb+vvaeP5480\nsc2nZjp7B8hIS+HmFT41U1nM4rz58a6miSNLuRiThPoHh3ilvtWdWK1ppL6lB3CpmeEhkddYambO\niXVA/xbQEnZSNF9VPztRORbQjYmcqnK0qZsdhxrZVhNkX30rQwqB7ExuX1MYSs1kZVhqZraL5iiX\nJ4DNQABoBL4E/AJ4EigFGnDDFlsn+jAL6MZMXXtPH88dbmJbTSPPH2kKpWY2rCxgS2UxW9YUcbml\nZmYlu7Ar9pDxAAAWl0lEQVTImFmsf3CIV463upt4HGqkwadmKksWcvXiXFJS4lxB83s+vGE5qy/L\nmdJ7Iw3o9r+aMUkoPTWFDeUBNpQH+Os/rORo08W5Zp49HIx39cwo3nvN5cDUAnqkLKAbMwWNjY3c\ne++9vPrqq2zdupWHHnooZp9VVlbGI488QnV19ajrRYTyomzKi7L52K0rY1YPk/jsH7M55vHHH+eG\nG24gOzubkpIS7rzzTnbv3h3vaiWdhx9+mEAgQEdHR0yD+Uz79a9/TVVVFXl5eVx22WV89KMfpbOz\nM97VMhGa0Ry6iDThTqJORQBojmJ14ilebSkGLsP1QQegwELc/4GnpljmbOmXybZjGdAPvBOb6lzi\nXbg5kyKNrNPpk3xgADfdhwArgAvAiSmWN12z5fcLpteWZapaOOFWqpoUD2BfvOuQzG0BcnF/pPeO\ns00m8A+4IPWOX8706zbjgv6ngSBwBvgIsA9YB5wFUsPK+iPgDb+cAjwIHAVacCOk8v26MtwXy5/h\ngsZO//qf4r54WoC/xgW06kmU9yFfXjPw+bB6pQKf8+/tBPYDS3071gDPAK3AYdzordH202O4YN7n\n92l1hPvuAb+f/nWMcv8cqPH1Oghc518Pb/tNwB6g3ffB94AMv06Av/d16wDeBK7y6+7yZXYCp4HP\nRPh78z7gzbn0t5LMbbGUy9xxMzAPeGqcbT4PrAfWAtfggscXwtZfhvtiWIwLwN/HBfG9QDdwe9i2\nfwI87pc/CdwD3ApcDrT594a7FagE3i0iVwD/CHwAKAn7zGGRlFcFrAa2AF8UkUr/+l8C/zcuwC0E\n/ifQg/uSeMbXuQi4D/hHX5dLqOqHgZ8A31TVbFXdFuG+y8cd2W8dWaaI3Av8De6LbCHw33BfViMN\nAv8v7mjvZt++/+XX/QGwCXgLt8/eH1bGD4GPqWoOcBWwY5SyR7MJeDvCbU28xftbK5G+3WZzW3DB\n8ewE2xwF7gp7/m6g3i9vBs4DaWHrg0CNX/4q8KhfzsEF+GX+eQ2wJex9JbijyDQuHlGvCFv/ReCJ\nsOdZuKPh6kmUtyRs/cvAfX75MHD3GG3fNeK1HwBfGmNfPQZ8dRL7rg+YN86+/y3wqTHW1Q+3fZR1\n9wNP+eXbgSN+/6SM2O4E8DFg4SR+Z+7AfVmumunf17A62N/9JB7JdIT+cLwrEEXxaEsLEBCR8UY2\nXc6l5zga/GuhMlR1IOx5D/Abv/w48D4RycT9m35AVYfLWgY8JSLtItKOCziDuJz+sJMj6hF6rqo9\nXHq0Gkl5Z0fUM9svL8UF35FeBdYNl+nL/QDuyDoSE+27JlXtHef9Y9XrEiKySkSeFpGzItIB/B3u\naB1V3YFLwWQBQRF5WEQW+rf+Me6/kgYReV5Ebp7gc9bj+vS/q+qRieoVQ/Z3PwlJE9BVddZ0bJza\nsgd3cuuecbZ5Bxcsh5Uy8Um//wJQ1YO4IHYnl6ZbwAXnO1U1L+wxT1VPh20Tfnb+DLBk+ImIzAcK\nJlneWE4Co43t+0/g+RFlZqvqJyIoEybedxONPhirXiP9E3AIqFDVhbjzAaGJXVT1u6q6DLgCWAX8\nlX/9FVW9G5dOGr7Se1Qici3wK+B/qur2COoUM/Z3PzlJE9DN9KjqOVwq4/sico+IZIlIuojc6W9Y\nAvAE8AURKRSRgN/+3ybxMY8Dn8LlXX8W9vo/A18TkWUAvvy7xynnP4D3isgGEcnA5ZbDZ6OabHnh\nHgG+IiIV4lwtIgXA08AqEfmg3y/pInJjWO59ItPdd48AnxGR6329yofbN0IO7oRnl4isAUJfOL6+\n60QkHZfy6gWGRCRDRD4gIrmqOnzCdGi0SojIVcD/B3xSVf/PJOpvEoAF9DlEVR/CnRT8AtCEOyr8\nC9wRG7g8+D7gDdwIiQP+tUg9gTtRuUNVw4dnfQd3xPc7EekEXsKNjBmrnm/jTnz+O+5ovQuXr78w\nlfJG+Dbu6PR3uMD2Q2C+qnbiTirehzuyPgt8Azd6JRLT2neq+jPga7gvxU5cn+SPsulncP8BdQL/\ngrsV5LCF/rU2Lo4Q+pZf90Gg3qdpPo5LJ43m00Ah8EMR6fIPOymaLOJ9omCUEwfvwZ24qgMeHGW9\nAN/169/AD+1KxEcEbdkMnANe848vxrvOY7TjUVxAfWuM9THtE1z+ewBYHuN2JEV/+LouBZ7FDUV8\nm1FOqCbL30qEbUn4vsGNInsZeN23429nuk/ivhNGNDYVd2JoBZDhd8wVI7a5C3ciTnDDxPbGu97T\naMtm3LTEca/vBG3ZBFw3TiCMep8A78Wd3FuAS7G8ir8QLobtSIr+8HUt4eI49Rzc6JZk/VuJpC0J\n3zd+P2f75XRgL7B+Jvsk0VIuNwF1qnpMVftw/3KPzI3ejbthtarqS0Cev8lGoomkLUlBVXfiLrYZ\nSyz65G4uXqRTgRt2OK3LmiNoR9JQ1TOqesAvd+JG+iwesVlS/K1E2JaE5/dzl3+a7h8jf2dj2ieJ\nFtAXc+nwtVP8fsdGsk0iiLSeG0TkDRH5jYhcOTNVi7qo94mqflTdSJNcVd2iqoenVcPIJV1/+BvQ\nXIs7IgyXLH8rIeO0BZKgb0QkVURew6X2nlF30V24mPaJzbYYXweAUlXtEpG7cCfCKuJcp7ks6fpD\nRLJxQy7vV9WOeNdnOiZoS1L0jaoOAmtFJA93rcRVqvrWTH3+jE7OFQgEtKysbMY+zxhjZoP9+/c3\nawSTc83oEXpZWRlTuWNR94UB5qWnkmo3xjXGzEEiEtEstUmRcvnO9lqe3HeSW1YG2FgRoKoiwJJF\nWfGuljHGJJSkCOgbVhbQ2t3Hrtomfv3mGQBWBBawsSLAxopC1q8sIDszKZpijDExk1Q3iVZVaoNd\n7KptZldtEy8da6G3f4i0FOG60kUuwK8q5F2Lcy09Y4yZNSK9SXRSBfSRLgwMsr++jV11LsC/ddqd\nGM+dn84t5QVsrCikqjzA0nxLzxhjktecCOgjtXRd4IWjLew60sSu2mbOdrjZSpeHp2dW5JMzLz1m\ndTDGmGibkwE9nKpSF+xiZ20zu2ubeOlYK+f7B0lLEa4tzWNjRSEbKwJcvSTP0jPGmIQ25wP6SBcG\nBtnf0Mbu2mZ21Tbz1jvnUIWF89K4pTwQCvCWnjHGJBoL6BMYTs/srnXpmTPnXHqmrCArFNxvXllg\n6RljTNxZQJ8EVeVoUxc7jzSzu66ZPUdbON8/SGqKcO1Sn55ZFeDqxbmkpSba9DfGmNnOAvo0XBgY\n5EBDO7tqm9hd18ybpy9Nz1RVBNhUUWjpGWPMjLCAHkWt3X284IdGjkzPVPnRMzevLGChpWeMMTFg\nAT1GXHqm2x291zaz51gLPX0X0zPDAf6aJZaeMcZEhwX0GdI3MMSBE22hAP+GT8/kzEtjw0p3cdOm\nikJKCyw9Y4yZGgvocdLW3ccLR5vZdcSlaN7x6ZllBVlU+eGRG8otPWOMiZwF9ASgqhxr7g5dufrS\nsRa6fXpm7dI8qsoDbFoV4JoleZaeMcaMKWoBXUTmATuBTNzsjP+hql8SkXzgp0AZUA+8X1Xbxitr\nrgX0kfoGhnj1RJubXKyumTdOtbv0TGYaG8oLqKooZFNFgGUFC+JdVWNMAolmQBdggb/1UzqwG/gU\n8D6gVVW/LiIPAotU9YHxyprrAX2ktu4+XjzaEho9c7r9PACl+Vl+aGSAm1cGyJ1v6Rlj5rKYpFxE\nJAsX0D8B/BjYrKpn/F2rn1PV1eO93wL62FSV483doamB9xx16ZkUwaVn/NH7NUvzSLf0jDFzSlQD\nuoikAvuBcuD7qvqAiLSrap5fL0Db8PMR790KbAUoLS29vqEhojspzXn9g0O8esJd3LSztpk3T7Uz\n5NMzN68sCM0euawgC7f7jTGzVayO0POAp4BPArvDA7iItKnqovHeb0foU9feczE9s/PIxfTM0vz5\nbmqC8gAbVgbIzbL0jDGzTaQBfVL3bVPVdhF5FngP0CgiJWEpl+AU62oikJeVwV3vKuGud5WgqtS3\n9ISC+69ee4fH954gReCapRenBl5r6Rlj5pRITooWAv0+mM8Hfgd8A7gVaAk7KZqvqp8dryw7Qo+N\n/sEhXjvZzq4jLj3zhk/PZI9Iz5RZesaYpBTNUS5XAz8CUoEU4ElV/bKIFABPAqVAA27YYut4ZVlA\nnxnnevp58WgzO/0J1lNtLj2zZNH80NH7LZaeMSZp2IVFBnCjZxqG0zO1bmrgrgsDpAhcvSSPTRUB\nqioKubbU0jPGJCoL6GZU/YNDvH6yPXT0/vrJi+mZ9SsK2LQqQFV5gOWBBZaeMSZBWEA3ETnX08+e\nYxfTMydbXXpmcd58H9wLuaW8gLysjDjX1Ji5ywK6mZKGlm4X3I+4i5s6Lwwg4emZ8gDXli4iI83S\nM8bMFAvoZtoGBod4/VQ7O/3Mka/59MyCjFQ/eqaQ21YX2dTAxsSYBXQTdefO97MnbO6ZE609AFQU\nZbOlspjqyiKuLV1Eaorl3o2JJgvoJubqm7vZfijI9ppGXj7eysCQkr8gg82rC6muLGZjRYAcm/fd\nmGmzgG5m1Lnz/ew80sT2mkaePdzEufP9pKcK61cUsGVNEVsqi+2m2sZMkQV0EzcDg0Psb2hj+6Eg\n22oaOdbUDcDq4hy2VLrgvnZpnqVmjImQBXSTMI43d7O9ppFtNY28Ut/G4JBSsCCDzauLqK4sYuOq\nQrIzJzWtkDFzigV0k5DO9fTz3JEg22uCPHc4SEfvABmpKaxbkU91ZTFbKotYsshSM8aEs4BuEl7/\n4BD76tvYXtPI9kNBjje71Myay8JSM0vySLHUjJnjLKCbpHO0qcunZoLsb3CpmUB2BretdsF9Y0WA\nBZaaMXOQBXST1Np7+nj+SBPbfGqms3eAjLQUbl5RQLU/er88b368q2nMjIjm9LlLcfcPLQYUeFhV\nvyMi+cBPgTKgHjd9btt4ZVlAN1PRPzjEK/WtbK9xY97rW9wFTZUlC0PB/erFuZaaMbNWNAN6CVCi\nqgdEJAd3b9F7gA8DrWE3uFikqg+MV5YFdDNdqsrRJjdqZntNkH0NrQwpFOZkcvvqIrZUFlFVESAr\nw1IzZvaIWcpFRH4JfM8/Nofdgu45VV093nstoJtoa+vu47kjQbbVBNl5uInOCy41c8vKArb4UTMl\nuZaaMcktVjeJLgN2AlcBJ4ZvEi1u4uy28JtGh71nK7AVoLS09PqGhoaIP8+YyegbcKmZbf7ofXiu\nmSsvXxiaa+aqyy01Y5JP1AO6iGQDzwNfU9Wfi0h7eAAXkTZVXTReGXaEbmaKqlIX7GKbz7sfONHG\nkEJRTqYbErmmmFvKA8zPSI13VY2ZUKQBPaJEo4ikA/8J/ERVf+5fbhSRkrCUS3Dq1TUmukSEiuIc\nKopz+MTmlbR29/HsoSDbDzXyf14/wxMvnyQzLYWq8kAoNVO8cF68q23MtERyUlRwN4luVdX7w17/\nFtASdlI0X1U/O15ZdoRuEkHfwBAvH/epmUONobs0vWtxbujo/arFC+0WfCZhRHOUSxWwC3gTGPIv\nfw7YCzwJlAINuGGLreOVZQHdJBpVpTbYFcq7HzjRhioUL8zk9jUu735LeYB56ZaaMfFjFxYZMwUt\nXRd49rCbBnjnkSa6+waZlx6WmllTRJGlZswMs4BuzDRdGBhk77HW0HQEp9tdaubqJblsWePy7lde\nbqkZE3sW0I2JIlXlcGMn22vcHO+vnWxHFUpy53H7miKqK4u5eWWBpWZMTFhANyaGmrsusMPffm9X\nbTM9fYPMT0+lqiJAdWURt60poijHUjMmOiygGzNDevsHeelYS2iumXfO9QJwzdI8qv3t9ypLciw1\nY6bMAroxcaCq1JzpZMchl3d//ZRLzSzOm8/ta9xcM+tXWGrGTI4FdGMSQFPnBZ7191bdVdvM+f5B\nsjJS2VjhRs3cvqaIQHZmvKtpEpwFdGMSTG//IHuOtYRmijxzrhcRWLs0L3T7vdXFlpoxv88CujEJ\nTFU5eKYjlHd//dQ5wKVmhud4X7cin8w0S80YC+jGJJVgRy87DrlpgHfXNdHbP8SCjFQ2rSpkS2Ux\nt60upMBSM3OWBXRjklRv/yAvHm1mW02QHTVBzna41Mx1pYtCY95XFWdbamYOsYBuzCygqrz9Tkdo\nrpk3T7vUzNL8+aGrVdctLyAjLSXONTWxZAHdmFmosaOX7TVBdhxqZHddM739Q2RnprFpVYAta4q5\nbU0R+Qsy4l1NE2UW0I2Z5c73haVmDjXS2HGBFIEli7KwmzIlnm/88dWsW1EwpfdG7QYXIvIo8IdA\nUFWv8q/lAz8FyoB63NS5bVOqqTFmSuZnpPqbcxSjehVvnXapmfqW7nhXzYwiZ156zD8jkjsWPYa7\nIfSPw157ENgednOLB4EHol89Y0wkRIR3LcnlXUty410VE0cTnklR1Z3AyBtX3I27ixH+5z1Rrpcx\nxphJiuieoqMoVtUzfvksUDzWhiKyFdjqn3aJyOEpfmYAaJ7iexONtSXxzJZ2gLUlUU2nLcsi2Wiq\nAT1EVVVExjyzqqoPAw9P93NEZF8kJwWSgbUl8cyWdoC1JVHNRFumOni1UURKAPzPYPSqZIwxZiqm\nGtB/BXzIL38I+GV0qmOMMWaqJgzoIvIEsAdYLSKnROTPgK8Dd4hILVDtn8fatNM2CcTaknhmSzvA\n2pKoYt6WGb2wyBhjTOzYBBDGGDNLWEA3xphZIuECuoi8R0QOi0idvwp15HoRke/69W+IyHXxqGck\nImjLZhE5JyKv+ccX41HPiYjIoyISFJG3xlifFH0SQTuSoj8ARGSpiDwrIgdF5G0R+dQo2yRLv0TS\nloTvGxGZJyIvi8jrvh1/O8o2se0TVU2YB5AKHAVWABnA68AVI7a5C/gNIMB6YG+86z2NtmwGno53\nXSNoyybgOuCtMdYnS59M1I6k6A9f1xLgOr+cAxxJ4r+VSNqS8H3j93O2X04H9gLrZ7JPEu0I/Sag\nTlWPqWof8O+4aQbC3Q38WJ2XgLzhMfEJJpK2JAUdffqHcEnRJxG0I2mo6hlVPeCXO4EaYPGIzZKl\nXyJpS8Lz+7nLP033j5GjTmLaJ4kW0BcDJ8Oen+L3OzaSbRJBpPXc4P/1+o2IXDkzVYu6ZOmTSCRd\nf4hIGXAt7ogwXNL1yzhtgSToGxFJFZHXcBdbPqOqM9on077030zLAaBUVbtE5C7gF0BFnOs0lyVd\nf4hINvCfwP2q2hHv+kzHBG1Jir5R1UFgrYjkAU+JyFWqOuo5m1hItCP008DSsOdL/GuT3SYRTFhP\nVe0Y/hdNVf8LSBeRwMxVMWqSpU/GlWz9ISLpuAD4E1X9+SibJE2/TNSWZOsbVW0HngXeM2JVTPsk\n0QL6K0CFiCwXkQzgPtw0A+F+BfypP1u8HjinF2d+TCQTtkVELhNxd/oVkZtw/dEy4zWdvmTpk3El\nU3/4ev4QqFHVb4+xWVL0SyRtSYa+EZFCf2SOiMwH7gAOjdgspn2SUCkXVR0Qkb8AfosbJfKoqr4t\nIh/36/8Z+C/cmeI6oAf4SLzqO54I2/LfgU+IyABwHrhP/anwRCJu+ofNQEBETgFfwp3wSao+iaAd\nSdEf3i3AB4E3fc4W4HNAKSRXvxBZW5Khb0qAH4lIKu4L50lVfXom45dd+m+MMbNEoqVcjDHGTJEF\ndGOMmSUsoBtjzCxhAd0YY2YJC+jGGDNLWEA3xphZwgK6McbMEv8/6iPYh3RNdHkAAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "for y in range(0,3):\n", " # Perform K-means clustering for the y'th class\n", " # Start out with the dummy assumption that all data are in the 0'th cluster\n", " Xy = X[50*y:50*(y+1),0:2]\n", " k_index = np.zeros((50),dtype='int')\n", " # Find the resulting value of D, then set Dprev to 100 times that\n", " D = np.zeros((2))\n", " for n in range(0,50):\n", " D[-1] = D[-1] + np.sum(np.square(Xy[n,:]-mu_gmm[y,k_index[n],:]))\n", " D[0] = 1.5*D[-1]\n", "\n", " # Now iterate until convergence\n", " while D[-2]-D[-1] > 0.05*D[-2]:\n", " # Assign each datum to the nearest cluster\n", " for n in range(0,50):\n", " d0 = np.linalg.norm(Xy[n,0:2]-mu_gmm[y,0,:])\n", " d1 = np.linalg.norm(Xy[n,0:2]-mu_gmm[y,1,:])\n", " k_index[n]= np.argmin([d0,d1])\n", " # Now re-compute the centers (and the variances), based on the data assigned to each one\n", " for k in range(0,2):\n", " c_gmm[y,k] = np.count_nonzero(k_index==k)/50\n", " mu_gmm[y,k,:]=np.mean(Xy[k_index==k,:],axis=0)\n", " Sigma_gmm[y,k,:,:]=np.cov(Xy[k_index==k,:],rowvar=False)\n", " # Re-calculate D\n", " D = np.append(D,[0])\n", " for n in range(0,50):\n", " D[-1] = D[-1] + np.sum(np.square(Xy[n,:]-mu_gmm[y,k_index[n],:]))\n", "\n", " # After convergence, plot D\n", " plt.subplot(3,1,y+1)\n", " plt.plot(D)\n", " plt.title('Convergence for class {}'.format(y))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The K-means algorithm is guaranteed to converge to a local minimum of D\n", "Notice that we can be certain that the K-means algorithm converges, because both of the steps reduce the value of D.\n", "* The first step assigns each datum to the nearest centroid. Either a particular datum stays in the cluster it's already in, or it moves to a centroid that is closer, thus reducing the total value of D:\n", "$$k(n)=\\arg\\min\\Vert\\vec{x}_n-\\vec\\mu_k\\Vert^2$$\n", "* The second step changes the mean, $\\vec\\mu_k$, to be the average of the data in the cluster. But that's exactly the value that minimizes $D$! In other words, the minimum-mean-squared error centroid, $\\vec\\mu_k$, is equal to the average of the data in the cluster:\n", "$$\\arg\\min \\sum_{n:k(n)=k}\\Vert\\vec{x}_n-\\vec\\mu_k\\Vert^2 = \\frac{1}{N_k}\\sum_{n:k(n)=k}\\vec{x}_n$$\n", "\n", "Since each of the two steps either leaves $D$ unchanged or reduces $D$, therefore the algorithm only continues while it's possible to reduce $D$.\n", "\n", "But this algorithm doesn't find the GLOBAL optimum, it only finds a LOCAL optimum. It's possible that if you started with some other initial values of $\\vec\\mu_k$, you might end with an even lower value of D." ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEZNJREFUeJzt3X+Q1PV9x/HXK0AHf8PIGumdFyyNtMmFH2YLVB00OP4i\nVqdTZdSqEycRSTWKE6Y66TQzTqcztmGCTRwliFYz/hqiJKYMinZqoIw9bg6E4xBxICreBctBUQNp\nTNF3/9g9crcc7PeOvdvbzz0fMzu33+/3fbfvz3zhdd/77Gd3HRECAKTlM9VuAABQeYQ7ACSIcAeA\nBBHuAJAgwh0AEkS4A0CCCHcASBDhDgAJItwBIEEjq/XA48aNiwkTJlTr4QGgJm3YsGFvROTK1VUt\n3CdMmKCWlpZqPTwA1CTb72apY1oGABJEuANAggh3AEgQ4Q4ACSLcASBBhHstWveA9PbanvveXlvY\nDwAi3GtT3bnST772+4B/e21hu+7canYFYAip2jp3HIezZ0nXPl4I9PzXpZZHC9tnz6pyYwCGCq7c\na9XZswrBvvafC18JdgDdEO616u21hSv2WX9b+Fo6Bw9gWCPca1HXHPu1j0uz/+73UzQEPIAiwr0W\ndWzsOcfeNQffsbGaXQEYQnhCtRZdsODIfWfPYt4dwGFcuQNAggh3AEgQ4Q4ACSLcASBBhDsAJIhw\nB4AEEe4AkCDCHQASRLgDQIIIdwBIEOEOAAkqG+62R9tutr3Z9lbb9x2l7iLbm4o1ayrfKgAgqyxv\nHPaxpNkRccD2KEnrbL8YEU1dBbbHSHpI0uURscv2GQPULwAgg7LhHhEh6UBxc1TxFiVlN0haERG7\nit+zp5JNAgD6JtOcu+0RtjdJ2iPplYhYX1JyjqSxtn9he4PtmyvdKAAgu0zhHhGfRMRUSfWSpttu\nLCkZKenLkr4q6TJJf2/7nNKfY3ue7RbbLZ2dncfZOgDgaPq0WiYiPpD0qqTLSw61S1odEQcjYq+k\ntZKm9PL9SyMiHxH5XC7X354BAGVkWS2TKz5hKtsnSLpE0pslZS9IusD2SNsnSpohaVulmwUAZJNl\ntcx4SU/YHqHCL4PlEbHS9nxJioglEbHN9kuSWiV9KmlZRLQNWNcAgGNyYTHM4Mvn89HS0lKVxwaA\nWmV7Q0Tky9XxClUASBDhDgAJItwBIEGEOwAkiHAHgAQR7gCQIMIdABJEuANAggh3AEgQ4Q4ACSLc\nASBBhDsAJIhwB4AEEe4AkCDCHQASRLgDQIIIdwBIEOEOAAki3AEgQYQ7ACSIcAeABBHuAJAgwh0A\nEkS4A0CCCHcASBDhDgAJItwBIEGEOwAkqGy42x5tu9n2Zttbbd93jNo/s33I9jWVbRMA0BcjM9R8\nLGl2RBywPUrSOtsvRkRT9yLbIyT9k6SXB6BPAEAflL1yj4IDxc1RxVv0UvotSc9L2lO59gAA/ZFp\nzt32CNubVAjuVyJifcnxOkl/KenhMj9nnu0W2y2dnZ397RkAUEamcI+ITyJiqqR6SdNtN5aUPCDp\nnoj4tMzPWRoR+YjI53K5/nUMACgry5z7YRHxge1XJV0uqa3bobykZ21L0jhJc2wfioifVaxTAEBm\nZcPddk7S/xWD/QRJl6jwxOlhEXF2t/rHJa0k2AGgerJcuY+X9ERxNcxnJC2PiJW250tSRCwZyAYB\nAH1XNtwjolXStF729xrqEfG1428LAHA8eIUqACSIcAeABBHuAJAgwh0AEkS4A0CCCHcASBDhDgAJ\nItwBIEGEOwAkiHAHgAQR7gCQIMIdABJEuANAggh3AEgQ4Q4ACSLcASBBhDsAJIhwB4AEEe4AkCDC\nHQASRLgDQIIIdwBIEOEOAAki3AEgQYQ7ACSIcAeABBHuAJCgsuFue7TtZtubbW+1fV8vNX9tu9X2\nFtuv2Z4yMO0CALIYmaHmY0mzI+KA7VGS1tl+MSKautW8LenCiNhv+wpJSyXNGIB+AQAZlA33iAhJ\nB4qbo4q3KKl5rdtmk6T6SjUIAOi7THPutkfY3iRpj6RXImL9Mcq/LunFSjQHAOifTOEeEZ9ExFQV\nrsin227src72V1QI93uOcnye7RbbLZ2dnf3tGQBQRp9Wy0TEB5JelXR56THbkyUtk3R1ROw7yvcv\njYh8RORzuVx/+gUAZJBltUzO9pji/RMkXSLpzZKaBkkrJN0UEW8NRKMAgOyyrJYZL+kJ2yNU+GWw\nPCJW2p4vSRGxRNJ3JZ0u6SHbknQoIvID1DMAoIwsq2VaJU3rZf+Sbve/IekblW0NANBfvEIVABJU\nM+G+ZM1OvbZzb499r+3cqyVrdlapIwAYumom3CfXn6Y7nn79cMC/tnOv7nj6dU2uP63KnQHA0JPl\nCdUh4byJ4/TgDdN0x9Ov68YZDXpy/S49eMM0nTdxXLVbA4Ahp2au3KVCwN84o0E/+I8dunFGA8EO\noGZsXP2u2rfv77Gvfft+bVz97oA8Xk2F+2s79+rJ9bt05+w/1pPrdx0xBw8AQ9UZE07V6kfaDgd8\n+/b9Wv1Im86YcOqAPF7NTMt0zbF3TcXMnHh6j20AGMrqJ43VZbc2avUjbWqcVae2tR267NZG1U8a\nOyCPVzNX7q3tH/YI8q45+Nb2D6vcGQBkUz9prBpn1all1TtqnFU3YMEu1dCV+/wLJx6x77yJ47hq\nB1Az2rfvV9vaDuXnTFDb2g7VTRrLlTsA1LKuOfbLbm3UjKv+6PAUTemTrJVCuAPAINjzzkc95ti7\n5uD3vPPRgDxezUzLAEAtO/eyzx2xr55pGQBAXxDuAJAgwh0AEkS4A0CCCHcASBDhDgAJItwBIEGE\nOwAkiHAHgAQR7gCQIMIdABJEuANAggh3AEgQ4Q4ACSLcASBBZcPd9mjbzbY3295q+75eamz7B7Z3\n2G61fe7AtAsAyCLLh3V8LGl2RBywPUrSOtsvRkRTt5orJH2+eJsh6eHiVwBAFZS9co+CA8XNUcVb\nlJRdLenHxdomSWNsj69sqwCArDLNudseYXuTpD2SXomI9SUldZLe67bdXtwHAKiCTOEeEZ9ExFRJ\n9ZKm227sz4PZnme7xXZLZ2dnf34EACCDPq2WiYgPJL0q6fKSQx2Szuq2XV/cV/r9SyMiHxH5XC7X\n114BABllWS2Tsz2meP8ESZdIerOk7OeSbi6umpkp6cOI2F3xbgEAmWRZLTNe0hO2R6jwy2B5RKy0\nPV+SImKJpFWS5kjaIek3km4ZoH4BABmUDfeIaJU0rZf9S7rdD0m3V7Y1AEB/8QpVAEgQ4Q4ACSLc\nASBBhDsAJIhwB4AEEe4AkCDCHUjEY22PqXl3c499zbub9VjbY1XqCNVEuAOJaDy9UQvXLDwc8M27\nm7VwzUI1nt6vt4JCjcvyClUANWD6+OladOEiLVyzUHMnzdXy7cu16MJFmj5+erVbQxVw5Q4kZPr4\n6Zo7aa5+1PojzZ00l2Afxgh3ICHNu5u1fPty3Tb5Ni3fvvyIOXgMH4Q7kIiuOfZFFy7SHdPuODxF\nQ8APT4Q7kIi2fW095ti75uDb9rVVuTNUgwtv6Dj48vl8tLS0VOWxAaBW2d4QEflydVy5A0CCCHcA\nSBDhDgAJItwBIEGEOzBIml94TrvaWnvs29XWquYXnqtSR0gZ4Q4MkjMnnqOVD9x/OOB3tbVq5QP3\n68yJ51S5M6SI95YBBklD42RdueBerXzgfk25dI42v7xKVy64Vw2Nk6vdGhLElTswiBoaJ2vKpXPU\n9PyzmnLpHIIdA4ZwBwbRrrZWbX55lWb+1XXa/PKqI+bggUoh3IFB0jXHfuWCe3X+3BsPT9EQ8BgI\nhDswSN7f+VaPOfauOfj3d75V5c6QIt5bBgBqCO8tAwDDGOEOAAkqG+62z7L9qu03bG+1fVcvNafZ\n/jfbm4s1twxMuwCALLK8iOmQpG9HxEbbp0jaYPuViHijW83tkt6IiL+wnZO03fZTEfG7gWgaAHBs\nZa/cI2J3RGws3v+1pG2S6krLJJ1i25JOlvQ/KvxSAABUQZ/m3G1PkDRN0vqSQw9K+lNJv5K0RdJd\nEfFpL98/z3aL7ZbOzs5+NQwAKC9zuNs+WdLzkhZExEclhy+TtEnSH0qaKulB26eW/oyIWBoR+YjI\n53K542gbAHAsmcLd9igVgv2piFjRS8ktklZEwQ5Jb0v6k8q1CQDoiyyrZSzpUUnbIuL7RynbJeni\nYv1nJU2S9MtKNQkA6Jssq2XOl3STpC22NxX3fUdSgyRFxBJJ/yDpcdtbJFnSPRGxdwD6BQBkUDbc\nI2KdCoF9rJpfSbq0Uk0BAI4Pr1AFgAQR7gCQIMIdABJEuANAggh3AEgQ4Q4ACSLcASBBhDsAJIhw\nB4AEEe4AkCDCHQASRLgDQIIIdwBIEOEOAAki3AEgQYQ7ACSIcAeABBHuAJAgwh0AEkS4A0CCCHcA\nSBDhDgAJItwBIEGEOwAkiHBHZvuWLdPBpvU99h1sWq99y5ZVqSMAR0O4I7PRjV9Sx913Hw74g03r\n1XH33Rrd+KUqdwag1MhqN4DacdLMGapbvFgdd9+tsddfp/3PPKu6xYt10swZ1W4NQImyV+62z7L9\nqu03bG+1fddR6i6yvalYs6byrWIoOGnmDI29/jrtfehhjb3+OoIdGKKyTMsckvTtiPiCpJmSbrf9\nhe4FtsdIekjSVRHxRUnXVrxTDAkHm9Zr/zPPatzffFP7n3n2iDl4AEND2XCPiN0RsbF4/9eStkmq\nKym7QdKKiNhVrNtT6UZRfV1z7HWLFyt3552Hp2gIeGDo6dMTqrYnSJomqfR/8zmSxtr+he0Ntm+u\nTHsYSn7btqXHHHvXHPxv27ZUuTMApRwR2QrtkyWtkfSPEbGi5NiDkvKSLpZ0gqT/kvTViHirpG6e\npHmS1NDQ8OV33333uAcAAMOJ7Q0RkS9Xl+nK3fYoSc9Leqo02IvaJa2OiIMRsVfSWklTSosiYmlE\n5CMin8vlsjw0AKAfsqyWsaRHJW2LiO8fpewFSRfYHmn7REkzVJibBwBUQZZ17udLuknSFtubivu+\nI6lBkiJiSURss/2SpFZJn0paFhFtA9EwAKC8suEeEeskOUPd9yR9rxJNAQCOD28/AAAJyrxapuIP\nbHdKqtXlMuMk7a12E1UwXMctMfbhOPahOu7PRUTZFSlVC/daZrsly1Kk1AzXcUuMfTiOvdbHzbQM\nACSIcAeABBHu/bO02g1UyXAdt8TYh6OaHjdz7gCQIK7cASBBhPsx2H7H9pbih5C09HLctn9ge4ft\nVtvnVqPPSssw7otsf1g8vsn2d6vR50CwPcb2c7bftL3N9p+XHE/1nJcbd5Ln3PakbmPaZPsj2wtK\namrynPMxe+V9pfhmaL25QtLni7cZkh4ufk3BscYtSf8ZEVcOWjeD518kvRQR19j+A0knlhxP9ZyX\nG7eU4DmPiO2SpkqS7RGSOiT9tKSsJs85V+7H52pJP46CJkljbI+vdlPoH9unSZqlwhvlKSJ+FxEf\nlJQld84zjns4uFjSzogofXFlTZ5zwv3YQtK/Fz+AZF4vx+skvddtu11HfkpVLSo3bkk6r/gn6ou2\nvziYzQ2gsyV1SvpX26/bXmb7pJKaFM95lnFLaZ7z7q6T9Ewv+2vynBPux3ZBRExV4c+y223PqnZD\ng6TcuDdKaoiIyZJ+KOlng93gABkp6VxJD0fENEkHJd1b3ZYGRZZxp3rOJUnFqairJP2k2r1UCuF+\nDBHRUfy6R4V5uOklJR2Szuq2XV/cV9PKjTsiPoqIA8X7qySNsj1u0ButvHZJ7RHR9TGSz6kQet2l\neM7Ljjvhc97lCkkbI+K/ezlWk+eccD8K2yfZPqXrvqRLJZW+R/3PJd1cfDZ9pqQPI2L3ILdaUVnG\nbfvM4oe4yPZ0Ff4d7RvsXistIt6X9J7tScVdF0t6o6QsuXOeZdypnvNurlfvUzJSjZ5zVssc3Wcl\n/bT473mkpKcj4iXb86XCh5RIWiVpjqQdkn4j6ZYq9VpJWcZ9jaRv2j4k6X8lXRfpvBruW5KeKv6Z\n/ktJtwyDcy6VH3ey57x4EXOJpNu67av5c84rVAEgQUzLAECCCHcASBDhDgAJItwBIEGEOwAkiHAH\ngAQR7gCQIMIdABL0/xZ5ojS9I+biAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ax=plt.gca()\n", "for y in range(0,3):\n", " for k in range(0,2):\n", " ax.plot(mu_gmm[y,k,0],mu_gmm[y,k,1],'x')" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8XHd95//XR5Yl636xbMmW5YjasZM4PzuhJg1JTSCU\ncGmgdGFL2kdhoQ9I4UG70OVXumW3/dFud/tr2V+XLWybTWlLgVJKuWRbNqTQXJ2wCeRim8SOjU2U\n2IolWRfrbsmyPr8/zpnxzGguZ6SRZjR6Px8PPTxz5syc7xyfec93PufyNXdHRETKS0WxGyAiIoWn\ncBcRKUMKdxGRMqRwFxEpQwp3EZEypHAXESlDCncpODPbaWZldYytmf2Omd2V5fH3m9lDK9ikkrNa\n/9/zabeZvdbMnsvy+JfM7JMFa9wSlGW4m1mPmf1Mwv07zGzEzG4pZrsKycyazOzTZvaimU2a2Utm\n9g9m9qrw8UozczM7a2brEp5XZWZDZjaXMO3RcN49Kcv4p3D6T6/cO8ufmVWY2X81s+Hwvf1hoZfh\n7v/J3T8YLq/kQszM/sDMPp8y7VEze29xWpRduvauBu7+kLvvyT1n8ZVluCcys38D/A/gZ9394WK3\npxDMbAPwIHAV8BagEbga+Crw5pTZx4DbEu7fDgymedkTwHsSlrEZ2A8MF6zhy+dDBOvhWmAf8K/M\n7P3FbdLql9gpKFdmVlnsNiwbdy+7P6AH+BngVwmCbH+O+R8Ffh94HJgE7gE2An9HEI5PANsT5r8G\n+BeC4HseeEfCY28DDoXPewn4nYTHdgJOEKJngHPAv094/Ebg6fC5/cCnMrT3g0AvUJPlPVWGy/qP\nwN8lTL8H+A/AXMr7/92wvRXhtI8CnwH6gJ/OsIxa4L+FzxsFHgGqY+8zYb73A8eAceAU8P6ExzYD\n9wLnw/X5SMJjnwBeDtfH88BrM7Tj+8CvJNy/E3g04rZyBtgX3v434TrbHd7/VeBr4e0/AD4f3n45\nnG8i/HtV+B4fDtfHeeDHwG1Zlpv2vYX/b78Trqcx4Elga/jYZ8P2jgE/AG4Kp98OzAIXw/Y8BfwR\ncAm4EE77dIRt90sEHaH7CD4HC9Z3uK3857Bdo8A3gZbE7Tth3m3At8Jl/Sj2f5SuvRnW0Z5wnZ4H\nfkjQQQO4mWD7r0iY918DT4e3K8L1e4rg8/+V1DYC7yPYbh9I0+5s2+vPAD0J93+S4PM+TpAX/wB8\nciXzLuM2VuwGLMubCsL96wQBuS/C/I8Cx4GfAFrCjf448Lrww/Zl4C/CeevDDes94WM/CQxxORBu\nDTfKCoJe5CBwe8qGdRewAXglMANcGT7+A+AXw9sNwE9laO/XgM/leE+xcL8mXA+NBF9YfcBeFob7\ne8MN/Q3htKcJQitbuP9P4H5gC7AO+GlgfZoPy1vDdWvh+pkG9oaPfYogtNYDVcBrwul7gBeBjvD+\nK4CfyNCOSeAnE+7/FDAScVv5MvCR8PZfhR/mDyQ89uvh7cRwT3p/4bT3E4TVr4Tr4teB0xmWmfG9\nAb8NHAauDLeh64DW8LF3A63h/+1vhdthdWr7Uv9fE+7n2na/BIwArw6XXZ3hs3I63K7qCDoLadcL\n8BhBByG2rQ8Ct2Rqb8pyqoAXgI+H28bPEHwR7Ay3ox7gdQnzfxP4v8PbHwuX3Rku+3PAF1M+g39N\n0DmpSdPubNtrPNwJOjJngH8btvGOcBv45ErkXK6/ci7LvIGgJ/7DiPP/lbv/2N1HgH8GTrj7g+4+\nR/BtfH0438+Fj33B3efc/SmCDfydAO7+gLs/5+7z7n6YoNeQWuv/pLtfcPengecIvgQg2DCuNLON\n7j7u7k9kaGsbQegCYGb7zey8mY2l2dkzBXyboGfziwQfgtkMr/sF4D1h7X2Du/8g08oKf7K/F/i3\n7n7W3S+5+6PufjF1Xnf/p3Ddurs/QPCFcCDhPW8l+GU06+6PhNPnCD6Ye8ys0t1fcPcfp2mHEXxI\nRxMmjxF8OUbxMJf/fw4Af5hw/5bw8ahOuftfufsl4G+AbWbWlma+bO/t/cAn3P1H4TZ0yN2HAdz9\ni+4+HG6Tf0zwhb0zj/Zl3XZD33T3/xMueybD6/yNux9190mCX3x3hP8PcWb2CuAGgl+msW39rwm+\noKK4mSDgP+XuF939Xwi24zs8SNavEGzPmFkz8MZwGgS/bD/h7r3ufgH4PeBfm1li3v0/7j7l7tOp\nC86xvaa20YHPhG38CvBMxPe37Mo53D8E7AI+l7jhmdnnzGwi/Pt4wvz9Cben09yvD29fAdwchul5\nMzsPvIug94qZvdrMHjKzc2Y2SvBhTfqAu3tfwt2phNd+H0GP6LiZfd/M3pLhvQ3Flhe+3pPu3gz8\nAkFvItUXCHpr7wlvZ/I1gvr8h4EvZpkPoJ3gw3cqx3yY2e1m9kS4w/N8uIzYOvl/CXqx95vZKTP7\nzfA9HSfogf0+MGBmf2dmHamvHX7QpwiCLqaJ4GdyFA8DrzGzToLQ/RpwwMx2EgRw1M4BJHzhhm2C\ny/+3iW3O9t66yLBOzezjZvZ8uF2NEPSc0315ZJJ12w2djvA6ifO8SLDNtabMsxUYDL8AEuftjNjW\nrcBL4f9vuud/GXiHma0H3gE84e5nwse2A/+U8B5j/4ebM7yHJDm219Q2nknTxpJQzuHeD7ye4Bv3\nz2IT3f397l4f/v3xIl73NHC/uzcn/NW7+6+Fj3+FoCTU5e5NBD8JLdOLJXL34+5+B8FG+P8BXw93\nnqa6H3iTmdVGbPODBB/sZnf/P1mWPwF8l6Bm/aUcr9lP8AtgR7aZzKyGIDD/EGgPv4S+Q7hO3H3M\n3X/D3buBtwO/FTuqyd2/5O43E5Qt1oWvkU7irx/C2xkPV0vk7s8ThPqHgYfdPVb7/xXgYMoHN/60\nKK+dY7mZ3ttp0qxTM3sd8O8IgqyZoHw4weVtK0o7c227mV4nVVfC7e0EpcXUHe8vA21mVpcyb2/E\n5bwMdKX8Iog/392PEHyZvhH4JYKwjzlDUF5MfJ8bEjtVGf5fc26vKc4S7FdItD3H+1ox5RzuuPvL\nBAH/JjP7bwV62X8k+Dn9S2a2Pvy7wcx2h483AMPufsHMbiSow0ViZu82szZ3nycoMzgwn2bWvyao\nX37DzPaY2bpwo9yf7nXDDfl2gvDM5bcI6qJZe3Bh6eHzwKfNrCNsw81hTypRNUEP/xxwycxuJ/g/\nAcDM3mpmO8IP8SjBTsB5M7vazF5nZtUEv5ymSb8uIPg18jEz22pm2whC8PMJy3jUzP5jlrfzCPBr\nXC7BPJRyP9UA4Gb2E1leM6Mc7+1zwB/E1omZXWdmrQTb1RzB//t64JMEPfeYfqA7JQz7CWrHMbm2\n3ajeY2ZXhcH9e8BXU8PS3V8g2On6X8ys2syuI/hlGus0pGtvou+F7/djYTtvJTgi6u8T5vky8BsE\n+wi+ljD9rnC52yE48svM3hbxvWXdXlM8ClSY2a9ZcOjxLxDsWygJZR3uAO7+EsFOkXdaAY5/dvdR\ngt7CLxN8c/cRfMvHyiEfAv7QzMYJ9th/NY+XfwtwLHzufwXe5e4L6uNhnfAWgp2+3+byERf7yPBl\n4u7PuvvRCO+v190fi9je3yA4quApgp7bfyGlhxP2hH+DoNY/TFDf/VbCLLsJduROEOwE++/ufpBg\nff4xQZj1EfRU/0OGdvwZwX6S54Aj4bL+MuHxrvC1M3mYIDwfyXA/ibuPE/yfPxH+9E/7pZpFtvf2\nKYI6+P0E/693E5SH7iU4yuVHBDsTxwi2v5i/JwilYTP7fjjt08Avhm38kwjbblRfJAjpswS/Oj6a\nYb53EewY7iMI30+4+0NZ2hsX1vvfSrCfYBD4U+CX3P1HCbN9meCz/V0P9pXF/AnBET/3h5+l7xEc\nHJBThO01tY0/D3yAoEz28wT/dyXBMvw6ESkLZtZNcKREuh1ikicze5TgSK3PF7stkl35HsAvArh7\nD+mPdBApa2VflhERWYtUlhERKUPquYuIlKGi1dyrKmq8prIx94wiIhI3dnFg0N035ZqvaOFeU9nI\nTZvfVazFi4isSvf1fibSWbAqy4iIlCGFu4hIGVK4i4iUIYW7rApD13Qz2Z584cHJ9laGrukuToNE\nSpzCXVaFDUNj9B7YFw/4yfZWeg/sY8PQWJFbJlKadPkBWRXq+ofpPHiY3gP7aDlxmpFdXXQePExd\n/2oY4lVk5annLqtGXf8wLSdOM7h3By0nTivYRbJQuMuqMdneysiuLtqOnGJkV9eCGryIXKZwl1Uh\nVmPvPHiYTUdOxks0CniR9BTusipc2NiYVGOP1eAvbNQlLETS0Q5VWRU2Hu1ZMK2uf1h1d5EM1HMX\nESlDCncRkTKkcBcRKUMKdxGRMqRwFxEpQwp3EZEypHAXESlDCncRkTKkcBcRKUMKdxGRMqRwl2Wn\nUZREVp6uLSPLZvraTgCsvooze69nU89paiYmma6v41x3F5t6TjN9bSc1z/YWuaUi5UfhLgUTC/NU\nNROTbOo5zbnuLhoGhxlva40HferzFPQihaFwlyXJFOipaiYmaRgcZrRjM019A/Fgz/R6CnmRpVG4\ny6JEDfX4/PV1jLe10tQ3wHhbKxsmJjMGfOLrK+RFFkc7VCUv09d2LirYYzX2lr6BeIlmur4u0vJE\nJH8Kd4lkMaEeM1tbk1Rjj9XgZ2trIi9bRPKjcJeclhquTQODC0owNROTNA0MrlgbRNYa1dzXuKFr\nutkwNJY0XN1keysXNjZSW3GxiC0TkaVQz32N2zA0Ru+BffGTjCbbW+k9sA9rrCpyyxZS710kOvXc\n17i6/mE6Dx6m98A+Wk6cZviaK5Lq4yKyOinchbr+YVpOnGZw746sx6BHMbI7e4+/5fjsol8b0Bmt\nIhEp3IXJ9laGr7ki8jHoiXKFebb5lxr0IpKZau5r3GR7K2ded33ex6CP7K7KO9gL9RqqvYvkpnBf\n48b3dOV1DHohQj3da4pIYakss0bFer/pjjWvSVOWWe4AHtldpTKNSAGp574G5VPWWI6eerZliUhh\nRA53M1tnZs+Y2bfSPGZm9qdmdtLMjpjZKwvbTCmUfIO9VKW+Dw0IIpIsn577R4BjGR57M3Bl+Hcn\n8OdLbJcsg6jBvpK99XTLXoxMJ2NtGBorZPNEVo1INXcz2wb8LPCfgX+XZpafA77g7g48bmbNZrbF\n3c8WrqmyFPkEez7Gd8xHmq/h1PJWAFNPxhrZ1UXnwcNJl1UQWUuifuI+DXwcyPRJ7gROJ9w/E05L\nYmZ3mtmTZvbk7Px0Xg2V5Rc12Md3zMf/ospn3sVKPBmr5cRpBbusaTnD3cxuBwbc/amlLszd73b3\n/e6+v6oi2uVeZemi9NqjBHu+gb7Y5y+2NDPZ3srIri7ajpxiZFfXghq8yFoSped+M/A2M+sBvgLc\namZfSpmnF+hKuL8tnCZFVohgX2qop3u9QovV2DsPHmbTkZPxEo0CXtaqnOHu7r/t7tvcvRu4A3jA\n3X85ZbZ/BN4THjVzIzCqenvxFSrYV4MLGxuTauyxGvyFjY1FbplIcSz6JCYz+yCAu98F3Au8BTgJ\nTAHvK0jrZNGWGuzLHerjO+YLupN149GeBdPq+odVd5c1K69wd/eHgIfC23clTHfgw4VsmCxeqQe7\niCw/XX6gzKxUsNe/YjR+e/K5XVQ2j1Hd2RefNvbDbrhQR0XbmazLWu5DJEXWKoV7GVnuYE8M9ESV\nzWPM9GwHoLqzj5neDhjbDI0DOduTrZ261ozI4inc15DFBnumUI+J9dhnerZzcaiF+fEGqrtforqz\nj4kXmhbXWBFZEv0mLhO5eu3LFewx1Z19VDSMMz/eSEXDeDzwoz4/XxqNSSQ79dzLwHIEe76hPNPb\nwfx4AxUNY8yPNzDT25FUg8+XSjIiS6Oe+ypXKsE+07Od6u6XqN97jOrul5jp2R7U3hfxeiKydOq5\nr1GFCnaAufON8Ro7XK7Bz51vXFLvXUQWT+G+ii22117IYAeo23NiwbTqzj4Fu0gRKdxXqWIF+23b\njyfd/85LuyM9T0RWlsJ9Fcon2Merr2DdpSlq587Fg33+/GaYqaOi/QUgWrCnhnrq9OUO+dHNbVRN\nTQfju4ZHyky2t3JhY2PaSw+IrHXaobrK5DNMHsC6S1NcrGxmqnITEAb7VAtUBwNg5wr227Yfzxjs\nqfMtp6qpac51dzFdXwdopCWRXNRzLzOp5ZjauXNMARfXN0NfPcxXQu0IFc0DkYK9GNIdBlkzMcmm\nntOc6+7i0tS8RloSyUHhvoosts5+6Yr+MNjXQ8VFKpqzXxagWKGeS83EJA2Dwwzu3UHbkVMKdpEs\nVJZZJZayA3X+/Oagx15xEeYrmT+/OWOvfSWDPd+Lhk3X1zHR3KyRlkQiULivAvnW2RPFa+y1I1R0\nnILaEZhqYfrkFQVs4fKbrq/jXHeXRloSiUjhXuKWcqXH8R3zMFMXr7EDNF7/I9a39zM3unCEolIo\nx2S67MBsbQ2bek5rpCWRiFRzL2FLDnaIH+6YqGbniwumLWewF+LKkE0DgwsuFqaRlkQyU8+9RBVi\n/FMAH2/FZ2qBy4c9zp1vZObMlvg82YL9+NE9DPS3J00b6G/n+NE9WedJXUYqDdIhsrz0CStBS6mx\nQ8pZqFUX8JGtbGhxIAjd6eM7WVc/Gem1WjYO8vhjt8TDe6C/nccfu4WWjYML5pk737ioZcRfJ8uV\nIHWJX5H8qCxTYqIGe9TLC1j1FLS8zPTxnazv6OdiXzs1u09S2Ryc/JOrHLO5vZ8bb36Yxx+7hR07\nj3Pq5G5uvPlhNrf3L5jn4CO3LlhGupKMeu0iy0+fshKy1GDPpOGqs6zv6Gf2zDbWd/RHDvaYze39\n7Nh5nGPP7WPHzuNJwR5zaKY57TKi0vXbRQpL4V4iChHsmS4KNne+kYt97VRtO8PFvvZ4+SSqgf52\nTp3czdV7DnPq5O4F9fVCLCMblWRE8qeyTAlYzmDf0OJMH98ZL5NUNo0zfXwnB17zQKRlxmrssVLM\npva+pPsA9x55Vdpl0HQ2KAslUElGZGXok1Zkyxns9a8Y5dJEXVKNvbJ5jJrdJxkZaou23KG2pCCP\n1dcTn59pGcxuiLQM7UgVKTz13ItoqUfFRFG97eyCaW/Z+4PIz999zXMLpm1u74+H/Xde2p12GRdG\nDGtIPgZdvXaRlaNwL4J8Q30lxkFdjEzXcM/npCX12kWWh7pSizR0TfeC65pMtrcydE131uflE+wj\nu6uyBvvY5ub4CUoxPlOLj5fe9VbUaxdZWfrELdKGobGkC1flGjxi+trOvIM9m/Ed8/ETlGIB7zO1\n+MhWarYPZXxeoS4zkE+vPVOwq9cusnxUllmk2IWreg/so+XE6ayDRxSyDAOXSzGxE5R8ZCteex6m\nmqm9+kd5H2Oer+Uux4jI0qnnvgR1/cO0nDjN4N4dtJw4vaLBHmPVU1B7HibaoPZ80YI9k8WUY9Rr\nF1k6hfsSTLa3MrKrK+3gEfmWYSD/YIegFMNUM9QPYheaCnryUKpswV6ocoyIFIbKMosUq7HHSjG1\n/cPx+xWbavJ6rSiXE8gU7D6yFWt5mYarzsYv2JV4zHmipdTb8w32xVKvXaQw1HNfpAsbG5Nq7LEa\n/PierrxeZ7HBDsDshniww+WThy5N1OXVhlwWE+zaiSpSXOq5L9LGoz1J96ev7aSCYFCJKKJe/Ctj\nsMOCk4QgCPhC1d1z1dcLGewiUljquRdAoWvrMdmCPWa5TlbKd8dpzGKDXb12kcLKGe5mtsHMvm9m\nh83sOTP7vTTzvNbMRs3sUPj3u8vT3NJTyJOSYsZ3zEcKdptqZ6a3I2naTG8Hk8/tit+ffG5XfJ5Y\nYJ94/moefehWYOEoSt95aTf3HnlV1lGUIL86+0xlC9P1yaWi6fo6RjcH16dRsIsUXpSyzAxwq7tP\nmNl64FEz+7a7P54y30F3v73wTSxdhTwpKSaf3vpMbw0zPdsBqO7sY6a3g5me7VR3vxSft7J5LGme\nE89fzZFn9rP3+ieBYBSlg4/cGt8Jm7hTNpN8yzFNL05wrruLTT2nqZmYZLq+Ln5fRJZHznB3dwcm\nwrvrwz9fzkatBoUeWCNKqENyGaa6sw+AmZ7tXBxqYX68gerul+LT081zJJynp3aenrAnX7P7ZMaR\nmhJl661nL8fMsqnnNOe6u2gYHGa8rTUe9Oq1iyyPSDV3M1tnZoeAAeC77v5EmtluMrMjZvZtM9uT\n5nHM7E4ze9LMnpydn15Cs4snn+PXC1mCgfT19erOPioaxpkfb6SiYTwp2KPOU9k8lnMUpcUHe6Bm\nYpKGwWFGOzbTMDisYBdZZpHC3d0vuft1wDbgBjO7NmWWp4Ht7r4X+AxwT4bXudvd97v7/qqK/I4F\nLwWFLsPkE+qZdpzO9HYwP95ARcMY8+MNC2rwUebJNYrSYoI91XR9HeNtrTT1DTDe1rqgBi8ihZXX\noZDuft7MHgTeBDybMH0s4fa9ZvZnZtbm7tGOC1wFCt1bjyLXkTCJNfbEmjsklGNyzJN64lNsFKVs\nA1zHZAv2xF57Yo29ZmKSDROTDG7rpLp9MO0lG0Rk6XKGu5ltAi6GwV4DvAH4o5R5OoB+d3czu4Hg\nF0HmSxOWqUIFe9TDG+fONybV2GP/zp1vTL6dZZ5MoyhdmqjjwohlXHbUYAeYra2JBztA6+MnqG4f\n5MLGRoW7yDKJ0nPfAvyNma0jCO2vuvu3zOyDAO5+F/BO4ENmNgdMA3eEO2LLQpRe+2KuC5Mq32PW\n6/acWDCturMvqaaea550oyhVNo8VLNgh+cSuWJ29rn9YwS6yjKIcLXMEuD7N9LsSbn8W+Gxhm1Ya\nViLYV2LUpKhyHb+eb7An0g5UkZWjM1SzyBbso5vbmK6vSwr2uYoaZipb4vezHQkzP7iN+YmWpGBP\nPQEpl8QTlDK9xsyZLQt2kM6db4yfpJT4eCzYM43mlCnYJ6u2UjHckDRtdNNG+n/iisjvRUQKS+Ge\nQa4ee9XUNAM7upgLj/qZq6hhqmoL6+YvALl769Vbh2BsczycYzs687kuTOwEpWyvsa5+kunjO+MB\nHtuBuq5+Munx8eeDsI9daZKqC0nLytZjb+gbZ2RrB6ObNgJBsI9s7WDD+ER8HvXaRVaWLhyWRpRS\nzIXOi9TOnmWqagtVc6PMVjZRO3uWyvnprMF+uacenmWa5QSkVKmX7P1O+G+214jtIM10klJl8xg0\nnU0azclaXg4GAQnlLsUE+85HtnYw1dTITF0tLS/30XQumK5gF1l5CvcU+dTYK+enqZobZWb9Rqov\nDuUR7IHqzr4wlBupaBhLCuUo116/bftx2H6cbwy9Pe1rxCSepFS17UxSz37ihSaseioI9om2YNCP\nvII90HRuKAj2+jqqJyYV7CJFpnDPU2qNfbayieqLQ8xWNjHbMYkxteA5UU9A6p6qYNdVx/Jqz4nn\nr15wglJqwKeepFTZNJ50DHviaE5MNePVU1j1VF47T0c3bWSmrpbqiUlm6moZ3bSRjgeP5PVeRKRw\nFO4JcvXaU4N9qmpLvBQz2zEZ1KpTShrZgj12ctFbX/1w/IJeQOSAT7wIWE/tfNqTmDKdpETT2aDH\nnjCak1VP4dVT+MhWaqfPEhzVulC6YB/Z2hEvxcTur79qlI3PvxjpvYhIYWmHaiifYAe4VLEhqcZu\n1VNYy8swuyE+T7ZDHFtm1rH3+id566sfBoJA33v9kwz0Zb/UbqKBvi3svf7J+JdBdWcf1d0vJR0d\nk+4kJZrOXm5nOJpT7AvJqqeomz7LpYoNpJPucMcLDfVJNfaOB4+w+anjTG7ZGPm9iEhhWbHONWqq\navebNr+rKMtOJ99wj8lUY88W7EsZyzSTqINrLOdx7KAau8hyu6/3M0+5+/5c86ksw8oFe6FD/fjR\nPbRsHGRze3982tz5Ri5N1MXPPJ05s4V19ZMLa+yzGxYM07fUYBeR0rHmyzKrNdghGGjj8cduiY+k\nlHoMOxTmOPaowa5eu0jpUM99EUoh2AE2t/dz480P8/hjt8CmwbQDbcSOc586duUSjmPPTcEuUlrW\ndM99sb32dFY62GM2t/ezY+fxrANtXBgxiB3HXntewS6yBqjnnkE+5ZilBPu7W76X8bEvjtyU8/kD\n/e2cOrl7wTHsMRMvNBXkOPZMFOwipWnNhns+oyrFFDLYs4V64jzZAn6gv53HH7uFyp0vpB1oAyjI\nceyZKNhFSteaLstkkk85Jp1swf7ulu9FCvbE+TMZGWrjxpsfTjvQBoSHPaY5jt1aXs54HHtUCnaR\n0rYme+7L2WvPFeyFtPua54IbLzXHp1U2jyWVZVIPdwRoPHMBuLBgOuhYdpFyoZ57iqi99mIHe0ym\nk5cynaykk5RE1oY1F+7T13bGB9pIml5fx+jmtrTPSddrTzcIxkB/O8eP7gHg0Ydu5cTzVwOXg/2e\nQ6/n97/14SW/h5jjR/dkHYgjlYJdZO1Yc+EOwUAb57q74gE/XV/Hue4uZjdejPT8+leMLhgE47rq\n8zz+2C20bAzGC93ccZYjz+yn4cVgMI97Dr2ez3/vHezd9nzB3kfLxsG0A3HMTuR4YurrKNhFys6a\nrLnXTEyyqec057q7aBgcZrytlU09p7nQuTDcM52wlDoIxuPn9nHjzQ/HLwWw66pj/GRND5//3jt4\n4oXrOHZ2B++96eu8/br7C/Y+Ds00px2II93g1tl67bko2EVWnzXVc0/ckVozMUnD4DCjHZtpGBym\nZmIyyzMvS6y1Jw6CsWPn8aRrvAC8/br7uXrLKY6evZKrt5wqaLCna8P6jv60wZ6NrhkjUp7WVLgn\nmq6vY7ytlaa+AcbbWjl39cIdkLnGQY0NgnH1nsOcOrk7fo0XCOrs9xx6PcfO7uCaLT/i2Nkd3HPo\n9Xm3M9eJTKkDcfhM7YJ5MvXaVY4RKV9rsiwTq7Fv6jlNzcQkGyYmGdjRFb8+exSx+vaB1zzA5vZ+\nNrX38fhjt8RLM7Eae6wUE7sPFKQH/52Xdi8YiOPi9PyCAUMU7CJr05rpuSeWZGZra+LBDkGJpnY2\n8wAVMYlTMYvvAAAPNklEQVQlmdggGLFSTOwiXiNDwRE3R85clVRjf/t19/Pem77OkTNXFew9pQ7E\nkW7AkMVSsIusbmtmsI5sJy5FvY5MPse2F+K49kwlmXTHtqc7rn2xvXYFu0jpijpYx5rpuecrV709\nm+U6YakQtANVZG1QuC/SzJktXFd9Pmla4klMuXzjmds40rsradqR3l1845nbgKDXft+33sbTP7gh\naZ7/9cBtjD/1fyVNy6fXnot67SLlYU2E+2KuJZPLuvrJpFGQYldojJ3ElMvOzT186p8/EA/4I727\n+NQ/f4Cdm3vi5ZjN7X38+OTueMD/rwdu42J/O5VNyZf0TaVyjIisyaNlEi32WjKVzWPsD0dB2rHz\nOKdO7o4fKfPFkZtylmb2dp7gN9/4F3zqnz/Am659hPuefQ2/+ca/4HDt5UsgvPJV3wfgxyd3c/bl\nbVycqmN9ez81O1/M812KyFqzJnruyyU2CtKx5/alPYkpl72dJ3jTtY/w1Sd/ljdd+0hSsMe88lXf\np6Z2kumpeqiaTQp29dpFJBOF+xLERkFKdxJTFEd6d3Hfs6/hF/b/b+754evTPv/pH9zA9FQdVM3A\nbBXTJ68AFOwikp3CfZHmzjfGT1ras/dwfKDqWEDnOrM0VmP/zTf+BfNXDi14PgTB/uOTu1nf3k/j\nqw6xvr2fi/3t8YAXEclE4Z5GlMMgL03UUbnzhYwnMUH2gD850M11N30vXopJ9/yB/o6kGnvNzheD\ngB9sWfB66rWLSKI1v0M1qokXmpJ2qlZvO7tgns3t/Qvq7hkDvnuCzSRfmzf1+RV7j1OT8rRL685T\n0Z58CKaCXURS5ey5m9kGM/u+mR02s+fM7PfSzGNm9qdmdtLMjpjZK5enubkNXdPNZHtr/P70tZ0L\nBuJIN1jHXEUNM5VBj3gpl8eNKnEwj5gTz1/Now/dCgRnoaYOCDLxQlMw4PV48P58vHXBhcIS30c2\nCnaR8hYlxWaAW919H3Ad8CYzuzFlnjcDV4Z/dwJ/XtBW5mHD0Bi9B/bFAz52kbCqqcsXBEsdrGOu\nooapqi2sm08/rmg2mYa5yyU2mEcs4E88fzVHntnPSPWl+GumDgjiM7XBhcGqwnZWXYDBrcxV1Cx4\nHzoTVWRty1mW8eDiM7H6wfrwL/WCND8HfCGc93EzazazLe6+sHaxzOr6h+k8eJjeA/toOXGa4YSr\nP8YkDtax3keZrWzK64qQhbDrqmMAHHlmPy+f6WLwXDvV3S9R3dkXnydxQBDfMApTzVjCFR8bz1xg\nruIsU1VbqJq7/D42HVt4DZxE6rWLlL9I9QczW2dmh4AB4Lvu/kTKLJ3A6YT7Z8Jpqa9zp5k9aWZP\nzi5jkNb1D9Ny4jSDe3dkHIgjNljHzPqNVM2NLinYF9t733XVMdo29TN4roOKhvGkYI+pbB4Lgn2i\nDWrPL7iUb+X8NFVzo/H3oWAXEYgY7u5+yd2vA7YBN5jZtYtZmLvf7e773X1/VUXqrsLCmWxvZWRX\nF21HTjHe1rqgvg6XB+uovjjEbGVTvLSRTbpjy2MWE/Annr+awXPtVDSMMT/ewExvx4J5xp/fAlPN\nUD8IU834TG3SPoG5ihpmK5vi7yPdexWRtSevo2Xc/byZPQi8CXg24aFeoCvh/rZw2oqbbG+l98A+\nOg8epq5/mHW1FUkDc0DyYB0XOi9SOT/FVNWWJZdmYgGf6TLAifPN9HYw07M9XoqJ3Qeo7uy7vPN0\nZGu8FOPVU/jIVuYqgnbGauyxdje/OLbgvSZSr11k7cgZ7ma2CbgYBnsN8Abgj1Jm+0fg18zsK8BP\nAaPFqLcDXNjYGA92uFxfn62tiQde4mAdF6iicn46PlhHLNwbTlWkPd499ZDIdKL04ufONybV2GP/\nzp1v5OJs+AUzuyGpxm7VU9ROX27npYoN8WAPdqDOLnivMQp2kbUlSs99C/A3ZraOoIzzVXf/lpl9\nEMDd7wLuBd4CnASmgPctU3tz2ni0Z8G0monJpLBrGrh85caW47OM7A4CPmqvPUrA51K358SCadWd\nfZeDHbCG4aTHg3LM5XZWz40Aycezp75XULCLrEVRjpY5AlyfZvpdCbcd+HBhm1YYNc/2LsslfwsR\n8Kmvl81SxkIVkbVnTV5+INdJTDG5TmbKFchRTLzQlPV1Gk5VaJBrEcnbmgz3fE5iihLwiwn5KM/L\ntmwFu4hksyavLZN4ElPD4DDT61qXfKRMYlCnK9fk+wWgYBeRpViT4Q6XT2Ia7dhMU98ANGUO9kxH\nzmSylHJNrl8KCnYRiWJNlGXShV3sJKamvgHG21rZ0Ls+62usxMXEChHsIiKwRnvuiScx1UxMsmFi\nknPdXdRcyl6aybcHH1WUL46owa5eu4jAGg33xJOYIPlEp/mW7HX3WBAXIuSj/hpQsItIvtZkuCee\nxBQTP/lnAEZ2V+V8jcRgzifo8ynv5FOGUbCLSKI1E+7LdTITLE89XsEuIkuxJnao5quYOy5bjs8q\n2EVkydZUuOcThPmGbCHkuzwFu4hksmbKMosVu7DYci8jHwp1EcllzYX7YmrvsfAtZMgv9leBgl1E\nolhz4Q6L37m61JBfaplHwS4iUa3JcF+qYuxwVbCLSD7W1A7VRKslLGue7V01bRWR0rFmwx1KP+BL\nvX0iUrrWdLhDaQaoeusislSquXM54JfrDNZ82yEislRrvueeqFjhqp66iBSaeu4pVrIXr0AXkeWi\ncM9guUJegS4iK0HhnkNiGC8m6BXmIlIMCvc8KKhFZLXQDlURkTKkcBcRKUMKdxGRMqRwFxEpQwp3\nEZEypHAXESlDCncRkTKkcBcRKUMKdxGRMqRwFxEpQwp3EZEylDPczazLzB40s6Nm9pyZfSTNPK81\ns1EzOxT+/e7yNFdERKKIcuGwOeBj7v60mTUAT5nZd939aMp8B9399sI3UURE8pWz5+7uZ9396fD2\nOHAMKO54dCIiklVeNXcz6wauB55I8/BNZnbEzL5tZnsyPP9OM3vSzJ6cnZ/Ou7EiIhJN5Ou5m1k9\n8HXgo+4+lvLw08B2d58ws7cA9wBXpr6Gu98N3A3QVNXui261iIhkFannbmbrCYL9b939G6mPu/uY\nu0+Et+8F1ptZW0FbKiIikUU5WsaAvwSOufufZJinI5wPM7shfN2hQjZURESii1KWuRl4N/BDMzsU\nTvsEsB3A3e8C3gl8yMzmgGngDndX2UVEpEhyhru7PwpYjnk+C3y2UI0SEZGl0RmqIiJlSOEuIlKG\nFO4iImVI4S4iUoYU7iIiZUjhLiJShhTuIiJlSOEuIlKGFO4iImVI4Z7G0DXdTLa3Jk2bbG9l6Jru\n4jRIRCRPCvc0NgyN0XtgXzzgJ9tb6T2wjw1DqVc6FhEpTZGv576W1PUP03nwML0H9tFy4jQju7ro\nPHiYuv7hYjdNRCQS9dwzqOsfpuXEaQb37qDlxGkFu4isKgr3DCbbWxnZ1UXbkVOM7OpaUIMXESll\nCvc0YjX2zoOH2XTkZLxEo4AXkdVC4Z7GhY2NSTX2WA3+wsbGIrdMRCQa7VBNY+PRngXT6vqHVXcX\nkVVDPXcRkTKkcBcRKUMKdxGRMqRwFxEpQwp3EZEypHAXESlDCncRkTKkcBcRKUMKdxGRMqRwFxEp\nQwp3EZEypHAXESlDCncRkTKkcBcRKUMKdxGRMqRwFxEpQwp3EZEypHAXESlDOcPdzLrM7EEzO2pm\nz5nZR9LMY2b2p2Z20syOmNkrl6e5IiISRZQxVOeAj7n702bWADxlZt9196MJ87wZuDL8+yngz8N/\nRUSkCHL23N39rLs/Hd4eB44BnSmz/RzwBQ88DjSb2ZaCt1ZERCKJ0nOPM7Nu4HrgiZSHOoHTCffP\nhNPOpjz/TuDO8O7Mfb2feTaf5RdJGzBY7EZEoHYWltpZWGpn4VwRZabI4W5m9cDXgY+6+9hiWuTu\ndwN3h6/3pLvvX8zrrCS1s7DUzsJSOwtrtbQzikhHy5jZeoJg/1t3/0aaWXqBroT728JpIiJSBFGO\nljHgL4Fj7v4nGWb7R+A94VEzNwKj7n42w7wiIrLMopRlbgbeDfzQzA6F0z4BbAdw97uAe4G3ACeB\nKeB9EV737rxbWxxqZ2GpnYWldhbWamlnTubuxW6DiIgUmM5QFREpQwp3EZEytCLhbmbrzOwZM/tW\nmsdK5tIFOdr5WjMbNbND4d/vFqmNPWb2w7ANT6Z5vCTWZ4R2lsr6bDazr5nZ82Z2zMxenfJ4qazP\nXO0s+vo0s90Jyz9kZmNm9tGUeYq+PiO2s+jrc6nyOolpCT5CcGZrY5rHSunSBdnaCXDQ3W9fwfZk\n8jp3z3SiRSmtz2zthNJYn/8duM/d32lmVUBtyuOlsj5ztROKvD7d/ThwHQQdJYLDob+ZMlvR12fE\ndkJpbJ+Ltuw9dzPbBvws8LkMs5TEpQsitHO1KIn1uRqYWRPwGoJDfXH3WXc/nzJb0ddnxHaWmtcD\np9z9xZTpRV+fKTK1c9VbibLMp4GPA/MZHs906YKVlqudADeFPyW/bWZ7VqhdqRz4FzN7KrycQ6pS\nWZ+52gnFX5+vAM4Bfx2W4z5nZnUp85TC+ozSTij++kx0B/B3aaaXwvpMlKmdUFrrM2/LGu5mdjsw\n4O5PLedylipiO58Gtrv7XuAzwD0r0riFftrdryP4efthM3tNkdqRS652lsL6rAReCfy5u18PTAL/\nvgjtyCVKO0thfQIQlo3eBvxDsdoQRY52lsz6XKzl7rnfDLzNzHqArwC3mtmXUuYphUsX5Gynu4+5\n+0R4+15gvZm1rXA7cffe8N8BgjrhDSmzlML6zNnOElmfZ4Az7h67EN7XCEI0USmsz5ztLJH1GfNm\n4Gl370/zWCmsz5iM7Syx9bkoyxru7v7b7r7N3bsJfv484O6/nDJb0S9dEKWdZtZhZhbevoFg3Q2t\nZDvNrM6Ca+oT/iy/DUi9smbR12eUdpbC+nT3PuC0me0OJ70eOJoyW9HXZ5R2lsL6TPCLZC51FH19\nJsjYzhJbn4uyUkfLJDGzD8KSLl2wIlLa+U7gQ2Y2B0wDd/jKn97bDnwz3OYqgS+7+30luD6jtLMU\n1ifArwN/G/5E/zHwvhJcn1HaWRLrM/wyfwPwqwnTSm59RmhnSazPpdDlB0REypDOUBURKUMKdxGR\nMqRwFxEpQwp3EZEypHAXESlDCncRkTKkcBcRKUP/P7mda9tLSBM4AAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Now let's calculate the GMM model of each of the three classes\n", "gmm_gaussians = np.empty((400,250,3,2))\n", "gmm_pdf = np.zeros((400,250,3))\n", "for y in range(0,3):\n", " for k in range(0,2):\n", " gmm_gaussians[:,:,y,k] = stats.multivariate_normal(mu_gmm[y,k,:],Sigma_gmm[y,k,:,:]).pdf(coords)\n", " gmm_pdf[:,:,y] = gmm_pdf[:,:,y] + c_gmm[y,k]*gmm_gaussians[:,:,y,k]\n", "\n", "ax=plt.gca()\n", "ax.contourf(coords[:,:,0],coords[:,:,1],gmm_pdf[:,:,0])\n", "ax.plot(X[0:50,0],X[0:50,1],'x')\n", "plt.title('K-means GMM class 0, with scatter plot overlaid')" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXmcHOV17/09PUvPvmhWaSQhJFlii8RqY7CMDA4xNibx\n8sZLYodcGye+9oXcxHtybeL4jd8bO7nBS8zFzhvi6zhODJgtgB2zmcVghEAyICQkGCQNmhlpNJpd\ns/Vz/6iqnuqe6u7q7urp6unz/XzmM91VT1Wd2n596tRzniPGGBRFUZTlRaTYBiiKoijBo+KuKIqy\nDFFxVxRFWYaouCuKoixDVNwVRVGWISruiqIoyxAVdwUAEfn/RGRIRA7nsOyjInK1/fmjIvJQFstm\n1X6pEJEfiMj1xbZDARE5LCLbi21Htvi1W0QqRGRcRNammJ/TPVJ0cReRXhF5q+v7+0VkWEQuKaZd\nQSIizSLy9yLyqohMiMhBEfmxiFxgz68UESMiR0SkwrVctS24c65pj9ptz0zaxl329DflYN+pwLXA\nZmPM6tz3NON2NoqIJlaUCCLyVhHpTZr2FRG5uTgWpcfL3lLAGDNvjGkwxhwMcr1FF3c3IvIHwLeB\ndxhjHi62PUEgIjXAg8BpwNuBJuB04N+BK5KajwKXu75fCRzzWO0+4MOubXQC5wPHczTzFGDQGOO1\nrdAgIhERCdU1q2SHiFQW24ZCE5Z9DM2NIiJ/BPwt8FvGmMfTtHtURL4sIk/YXvDtItImIv8qIqMi\n8qT78UZEzhCRn4vIcRF5UUTe45p3lYg8ay93UET+h2veRtsT/rD9eHVURD7nmn+hiOy0lx0Qka+l\nMPlqoAt4lzHmeftXesIY82NjzJeT2v4fXKJtf/6+xzr/BfiAS+g+CNwCzKY5bi12qOGo/bT0ebF4\nG3AvsNZ+NPyex7JtInKPveyw/ZTQk2pbafiFvb5x+++ChU3I/xKREyLysojEf+Ds8/1XIvJLYMK2\ns0VE/sl+0jlsXw8R1zIftc/1sIjcKyJr0hyXN9vX0oiIHBKRD2W7/yLyEfuYjtn2v9+evklEfmGv\n+5iI/NC1TLrr8koR2WOv77CI/Hc/BzfdciLybte1vt85xvaxcpY5ICIftac3A3excF2Mi8iHgc8A\nv2d/f9pum/J82Ov/hYh8Q0SOA3/hYfdXROTfxHqaHRORHSLyGyn2scZe1xER6RORvxPrCdfL3k6P\n5VPdB7X2sTnN1bZbRKZEpM3+fpWI7LKv00dF5CxX28Mi8mkR+TXWdZq83Tfa19kJ2/ZviEiVPc95\ncl9nf+8Qkbtte54ATk1z2lNjjCnqH9AL3AoMAFt9tH8U2AusB1qBF+3vbwEqgR8C37XbNgB9WCJZ\nCZwHDGGFHwAuBc7E+pHbiuUlX2nP2wgY4EagBjgXmAZeZ89/CviA/bkReEMKe28BvpdhnyrtbZ1h\nH4cmoA3oB7YAc0n7fzXwAPCb9rSdwAV2+zel2MYPgdtsW9cD+4E/sOe9FehNY18H8C6g1rbtNuCW\nZJvszx8FHkqxno3WJZcw7aNYP0r/BagA/htwKGndvVhPO1X2sboL+AegDuuH82ngI3b799jXw2a7\n7fXAIynsORUYB37XbtsOnG3P+wFwfab9t7+PuK6LlcAZ9ucfA5+1r68a4GKf1+VR4CL78wrgXJ/3\nkudywEXACeAy25Y1rm29074eBOt+mAK2pLougK8ANydNS3c+PgrMAR+3z2+th91fsa+Bd9nn+HNY\n12elPf8wsN3+/NfA4/Y56QSeBL7k5zr2cR98H/hLV9vrgLvtzxdg3ZsX2PvxX4ADQLXLxqeB1c4+\nJtl9AfAG+3yvx3r6/mTS/b/OpRn/ah/PLcARUtxTafc12wWC/sO6cUeBO4CIj/aPAp91fb8BuMv1\n/V3ADvvz7wEPJi3/j8Cfp1j3t4CvGZcQAd2u+TuB99qfHwe+CLRlsPch4Cuu7+dj3WijwPPJJxe4\nGfgI8EngO1jhHC9xvxrL0z8TeMGe5ynuWDfMHLDJNe0TwM/93hRJ6zsfOJpsk+tmzlbcX3R9b7KP\nRbtr3V90ze/BEqCoa9qHgP+0P/8n9s3qOrbTQI+HPf8D+HEKW+Pinm7/bXtP2NddTVK7H9rnsCdp\netrrEnjNPi6NWd5LnsvZ6/6az3XcDXwi1XVBkrj7OB8fBV7OsM2vAI+6vlcAg8Ab7e9ukXwVuNzV\n9h3Afj/XsY/74G3APte8J4EP2p+/i/0j4pp/gIUf7MPAh5Pmx+32sOVTzrVH4v3v2LjR1fZvyEHc\nwxKW+TiwCfieiIgzUUS+53rE+oyr/YDr85TH9wb78ynAxfaj0AkROQG8D8u7ch6VHrIf0UawLsR2\nt2HGmH7X10nXuv8Qy9PeKyK/EpG3p9i3IWd79vp2GGNasLzFqEf772N5dKlCMg63YMXnP4El8uno\nxLphXnVNexXrxsyIiDTY5+KgiIxiPTW0Z1ouC5KPMSwcZ4BDrs+nYB23Adc5/TaWx+jM/7Zr3jEg\nhuVRJbMG6wZNS7r9N8aMAh/AOg/99uP0JnvRP8O6WXeIyK/Feqfk2JjyusT6obgKOGhfn2/IZGOG\n5VLupx3KeVKs8NAJrGsqm3Ob6XxA4vlLRbyNMWYe68lmlUe7VeR4HZP5Pvg50CIi54nIBqz7+w57\n3inAZ5PO2cqkbafcTxE5TUT+Q0T67Wvoy3gf5y7bRve6XvVol5GwiPsA1iPjNqzHOwCMMR811lvk\nBmPM3+Sw3kPA/caYFtdfgzHmk/b8H2GFhNYYY5qB72E9nmbEGLPXGPN+rAvmb4FbxXp5msz9wNtE\npM6nzQ9iXUgtxphfptn+OJaX+jEsLzMdg8C8vV6HtVg3kB8+jRXCeL0xpgnr8T0XTADLHcL6AVjh\nOqdNxpgtrvkfSTrntcaYJz3WewjY4GP7afffGHOvMeatWDf7fuB/29OP2NfwSizxv0msnklpr0tj\nzJPGmKuwrq27sa7TjKRZznM/RaQWy0n4KtBlOx0/Y+Ee8DpfydMynY9U60km/l7Ejtf3YD2JJPMa\nqa/jTNtJex8YY+awQmkfwHqPdacxxomfH8IK2bjPWZ0x5t9d60q3/f8NPIflkTdhPfV7ac0AljPi\nfk/k2UUyE2ERd4wxr2EJ/NtE5H8FtNo7gTNF5IMiUmX/vV5ENtvzG4HjxpiTInIh8H6/KxaRD4lI\nuzEmhhVzNVgnJZl/wvIebxORM8Xq01qL9Wi/CGM9h10J/I4PMz4LXGKMSesZGWNmsW7iv7a90FOB\n/07mHwWHRqwbeNh+ufRFn8slMwgYEVmf4/LY+/ow8HURaRKrB81GEXmz3eRG4M9F5HSIv0B7b4rV\n/QDrenuP/VKrXUS2erRLuf8islJE3mn/eM9gvUyL2fN+VxZevJ7AukbmSXNd2i/2PigiTfZ5G3Ot\nz3nxtqi7a7rlsMIyHxWRt9jHa7V9D0SBaqxY/byIXIl1DzoMAO0i0pg0bZ3zhO3jfPjl9SLy2/ZL\nxk/Z9j/l0e5fgS/a56oDK7TmXMde9sbxeR/8EOsp6oP2Z4fvAp8QkQvEosE+7/U+968RSycm7Gvz\nj9LYeDvwl/Y5PQsrzJU1oRF3AGP187wUeK+IfDWA9Y0AvwX8PtZLiX4sL8UJh3wc+KqIjAFfwOqe\n6Je3A3vsZb8OvM8YM+NhwxRwCdZLvnuxYu0vYr3A9fwxMcY8Z4x5wcf+9RljHvNp73/FEp9erJvx\nn0kf9nHzd0AzVojpcaz9yBpjzBjW8X/SfrT1/IHzwe8D9cALwDCWt9Vtb+PHtr0/th9/d2NdA172\nvIL1QvGzWN1IdwJevTTS7X8Flmd/xJ5/EZaXDtYLtKdEZALrJd4njDEHfVyXfwC8atv/EbsdWN7c\nCJYH6IXncsbqfXYN8A17+QexnlZPYInbT+z9fy+Wx+8cn+ewnmx77fPVCfwb1g/CcRH5ld005fnI\ngp/Y6zmOJa7vtj3pZP4S2GUfg91YcfGvprE3mUz3weNYMe8OrKcY7HU/gaUX37H3cR8L58UPf4Z1\nfsawvPh/S9P241idRQawfpj/KYvtxBE7YK8oSsgRKwt4gzHmf2RqW0qIyFeA1caYq4tty3IiFJ3t\nFUXJjDHm5mLboJQOoQrLKIqiKMGgYRlFUZRliHruiqIoy5CixdyrI7WmtrKpWJtXFEUpSUZnB48Z\nYzoytSuauNdWNnFR5/uKtXlFUZSS5L6+b/rKWNWwjKIoyjJExV1RFGUZouKuKIqyDFFxV0qCoTPW\nMdG1ImHaRNcKhs5YVxyDFCXkqLgrJUHN0Ch927bGBX6iawV927ZSMzRaZMsUJZzo8ANKSVA/cJye\nR3bRt20rrfsOMbxpDT2P7KJ+INeysYqyvFHPXSkZ6geO07rvEMe2bKB13yEVdkVJg4q7UjJMdK1g\neNMa2ncfYHjTmkUxeEVRFlBxV0oCJ8be88guOnbvj4doVOAVxRsVd6UkONnWlBBjd2LwJ9t0CAtF\n8UJfqColQdsLvYum1Q8c17i7oqRAPXdFUZRliIq7oijKMkTFXVEUZRmi4q4oirIMUXFXFEVZhqi4\nK4qiLENU3BVFUZYhKu6KoijLEBV3RVGUZYiKu6IoyjJExV0pOFpFSVGWHh1bRikYU2f1ACAN1Rze\ncg4dvYeoHZ9gqqGeo+vW0NF7iKmzeqh9rq/IlirK8kPFXQkMR8yTqR2foKP3EEfXraHx2HHG2lfE\nhT55ORV6RQkGFXclL1IJejK14xM0HjvOSHcnzf2DcWFPtT4VeUXJDxV3JSf8inq8fUM9Y+0raO4f\nZKx9BTXjEykF3r1+FXlFyQ19oapkxdRZPTkJuxNjb+0fjIdophrqfW1PUZTsUXFXfJGLqDvM1NUm\nxNidGPxMXa3vbSuKkh0q7kpG8hXX5sFji0IwteMTNA8eWzIbFKXc0Jh7mTN0xjpqhkYTytVNdK3g\nZFsTdZHZIlqmKEo+qOde5tQMjdK3bWs8yWiiawV927YiTdVFtmwx6r0rin/Ucy9z6geO0/PILvq2\nbaV13yGOn3FKQnxcUZTSRMVdoX7gOK37DnFsy4a0fdD9MLw5vcffuncm53UDmtGqKD5RcVeY6FrB\n8TNO8d0H3U0mMU/XPl+hVxQlNRpzL3MmulZw+C3nZN0HfXhzddbCHtQ6NPauKJlRcS9zxs5ck1Uf\n9CBE3WudiqIEi4ZlyhTH+/Xqa17rEZYptAAPb67WMI2iBIh67mVINmGNQnjq6balKEow+BZ3EakQ\nkWdE5G6PeSIi3xCR/SKyW0TODdZMJSiyFfawkrwfWhBEURLJxnO/DtiTYt4VwOvsv48B38nTLqUA\n+BX2pfTWvbadC6mSsWqGRoM0T1FKBl8xdxFZDbwD+H+BP/Vo8tvA940xBnhCRFpEZKUx5khwpir5\nkI2wZ8PYhpivdo0HChsBTE7GGt60hp5HdiUMq6Ao5YTfO+7vgc8Aqe7kHuCQ6/the1oCIvIxEdkh\nIjtmYlNZGaoUHr/CPrYhFv/zSzZtc8WdjNW675AKu1LWZBR3EbkSGDTGPJ3vxowxNxljzjfGnF8d\n8Tfcq5I/frx2P8KeraDnunyuoZmJrhUMb1pD++4DDG9asygGryjlhB/P/WLgKhHpBX4EXCoiP0hq\n0wescX1fbU9TikwQwp6vqHutL2icGHvPI7vo2L0/HqJRgVfKlYziboz5vDFmtTFmHfB+4AFjzO8n\nNbsT+LDda+ZCYETj7cUnKGEvBU62NSXE2J0Y/Mm2piJbpijFIeckJhH5YwBjzI3APcDbgf3AJPCH\ngVin5Ey+wl5oUR/bEAv0JWvbC72LptUPHNe4u1K2ZCXuxpiHgIfszze6phvgE0EapuRO2IVdUZTC\no8MPLDOWStgbTh2Jf554fhOVLaNEe/rj00Z/vQ5O1hNpP5x2W4XuIqko5YqK+zKi0MLuFnQ3lS2j\nTPeuBSDa0890XzeMdkLTYEZ70tmpY80oSu6ouJcRuQp7KlF3cDz26d61zA61EhtrJLruINGefsZf\nac7NWEVR8kKfiZcJmbz2Qgm7Q7Snn0jjGLGxJiKNY3HB97t8tmg1JkVJj3ruy4BCCHu2ojzd101s\nrJFI4yixsUam+7oTYvDZoiEZRckP9dxLnLAI+3TvWqLrDtKwZQ/RdQeZ7l1rxd5zWJ+iKPmjnnuZ\nEpSwA8ydaIrH2GEhBj93oikv711RlNxRcS9hcvXagxR2gPoz9y2aFu3pV2FXlCKi4l6iFEvYL1+7\nN+H7zw5u9rWcoihLi8bcS5BshH0segqTlR3WZ1vYYyc6iQ2cGm/jR9gvX7t3kbCnmx40I53tTDXU\nAws9ZbTSkhI25hunGfzoTuYbpouyvBsV9xIjmzJ5ABXzk8xWtsQFPnaiEyZbIWoVwM4k7H7Fu9AC\nXz05xdF1a+ICr5WWlDAyur2XmVNGGH1Lb1GWd6PivsxIDsfUzR2lau4Es1UtxPo3WMJeN0ykZdCX\nsBcDr26QteMTdPQe4ui6NRzdsjE+vK8ODKaEhfnGaSbO64cITJzXn7X3ne/yyai4lxC5xtnnTxmA\nyBzEqiAyR6Ql/bAASxVqyZba8Qkajx3XSktKKBnd3gtirC9isva+810+GRX3EiGfF6ixE50Qq4TI\nLMQqiZ3oTOm1L6WoZzto2FRDPeMtLVppSQkdca+70hbnSpOV953v8l6ouJcA2cbZ3cRj7HXDRLoP\nQN0wTLYytf+UAC0sPFMN9Rxdt0YrLSmhJMHrdsjC+853eS9U3ENOPiM9jm2IwXR9PMYO0HTOS1R1\nDTA3srhCURhCMamGHZipq6Wj95BWWlJCycza0QWv26HSWNOXYHkvtJ97iMlb2IFI1yuL5tVufHXR\ntEIKexAjQzYPHls0WJhWWlLCQte3Lyjq8l6o5x5Sgqh/CmDGVmCm64CFbo9zJ5qYPrwy3iadsO99\n4UwGB7oSpg0OdLH3hTPTtkneRjJapENRCoveYSEknxg7JGWhVp/EDK+iptV65Js70cTU3o1UNEz4\nWldr2zGeeOySuHgPDnTxxGOX0Np2bFGbuRNNOW0jvp40I0HqEL+Kkh0algkZfoXd7/ACEp2E1teY\n2ruRqu4BZvu7qN28n8oWK5aXKRzT2TXAhRc/zBOPXcKGjXs5sH8zF178MJ1dA4vaPPKLSxdtwysk\no167ohQevctCRL7CnorG045Q1T3AzOHVVHUP+BZ2h86uATZs3Mue57eyYePeBGF3eHa6xXMbftHx\n2xUlWFTcQ0IQwp5qULC5E03M9ndRvfows/1d8fCJXwYHujiwfzOnn7mLA/s3L4qvB7GNdGhIRlGy\nR8MyIaCQwl7TapjauzEeJqlsHmNq70a2vfkBX9t0YuxOKKajqz/hO8A9uy/w3AbNR6ywkAsNySjK\n0qB3WpEppLA3nDrC/Hh9Qoy9smWU2s37GR5q97fdofYEIXfi6+7lU22DmRpf29AXqYoSPOq5F5F8\ne8X4Ibr6yKJpb9/ylO/lN5/x/KJpnV0DcbH/2cHNnts4OSxIY2IfdPXaFWXpUHEvAtmK+lLUQc2F\nVIU6sklaUq9dUQqDulI5MnTGukXjmvgpHpGNsA9vrk4r7KOdLfEEJQczXYcZC994K+q1K/kQZBGL\nckHvuBypGRpNGLgqU/GIqbN6shb2dIxtiMUTlByBN9N1mOFV1K4dSrlcUMMMZOO1pxJ29doVvwRZ\nxKJcUHHPEWfgqr5tWzMWjwgyDAMLoRiJTiKtr2GGVxEbbccMr6Lu9Jey7mOeLYUOxyiKm6CLWJQL\nKu55UD9wnNZ9h9IWjyiUsDtIdBLqTsB4O9SdKJqwpyKXcIx67YqboItYlAsq7nkw0bWC4U1rPItH\nZBuGgeyFHaxQDJMt0HAMOdkcaPJQMumEPahwjKK4KUQRi3JBe8vkiBNjd0IxdQPH498jHbVZrcvP\ncAKphN0Mr0JaX6PxtCPxAbvcfc7d5BNvz1bYc0W9dsVNuiIWrXdl9xRZbqjnniMn25oSYuxODH7s\nzDVZrSdXYQdgpiYu7LCQPDQ/Xp+VDZnIRdj1JaoSBIUoYlEuqOeeI20v9CZ8nzqrhwhWUQk/+B38\nK6Www6IkIbAEPqi4e6b4epDCriheFKKIRbmgnnsABB1bd0gn7A6FSlbK9sWpQ67Crl67ogRLRnEX\nkRoR+ZWI7BKR50XkLz3abBeRERF51v77YmHMDR9BJiU5jG2I+RJ2mexiuq87Ydp0XzcTz2+Kf594\nflO8jSPY+148nUcfuhRYXEXpZwc3c8/uC9JWUYLs4uzTla1MNSSGiqYa6hnptManUWFXlODxE5aZ\nBi41xoyLSBXwqIjca4x5IqndI8aYK4M3MbwEmZTkkI23Pt1Xy3TvWgCiPf1M93Uz3buW6LqD8baV\nLaMJbfa9eDq7nzmfLefsAKwqSo/84tL4S1j3S9lUZBuOaX51nKPr1tDRe4ja8QmmGurj35Xyo1Bj\nKqmTkEhGcTfGGGDc/lpl/5nUS5QHQRfW8CPqkBiGifb0AzDdu5bZoVZiY41E1x2MT/dqs9tu01sX\no9f25Gs3709ZqclNOm89fThmho7eQxxdt4bGY8cZa18RF3q9IZcvSzEwXqbtlfP15euFqohUAE8D\nG4FvG2Oe9Gh2kYjsBvqATxljFg0nKCIfAz4GUFPRmLPRxSRob92vqIN3fD3a028LexORxtEEYffb\nprJlNF5FqXr14YCF3aJ2fILGY8cZ6e6kuX9QhX0ZstRi7ge3TeV2vfkSd2PMPHC2iLQAPxGRs4wx\nz7ma7ATW2qGbtwO3A6/zWM9NwE0AzdVdJef9F0vY0700ne7rJjbWSKRxlNhYI9N93YvEO1Ob5CpK\nlc1jCQKfi7AnM9VQz1j7Cpr7BxlrX0HN+ATZZQMoYSOMYp6OchP6rLpCGmNOiMiDwNuA51zTR12f\n7xGRfxCRdmOMv36BJUCQYZggRB1IiLG7Y+7gCsdkaJOc+ORUUUpX4NohnbC7vXZ3jL12fIKa8QmO\nre4h2nXMc8gGJZyUmpinw9mX5SzyGcVdRDqAWVvYa4HfBP5nUptuYMAYY0Tk9Vi9cFIPTbhMCUrY\n/XZvnDvRlBBjd/7PnWhK/JymTaoqSvPj9ZwclpTb9ivsADN1tXFhB1jxxD6iXcc42dak4h5ilpOY\np2I5i7xY70vTNBDZAvwzUIEl2v9ujPmyiPwxgDHmRhH5JPBxYA6YAv7UGPN4uvU2V3eZizrfF8Au\nFB4/F3ku48IksxQFNvwShMfuxXK8iZYL5SDmmSiF6/O+vm8+bYw5P1M7P71ldgPneEy/0fX5W8C3\nsjWyFFgKYS8VUQcV9uWEivlilpMnr8MPpCHdxT/S2U715BQne2bj0+YitcxHaojODQPpRT12bDXU\nTND0G73xadN93cydaKL+zH2+7Jt4fhOVLYm9X5LXMX14JRUNEwkvSOdONDE/Xk909ZGE+Y6wm+k6\na9wanzVQJ6pX0dg/hjsSN9LRxsnGBrpeftXXvoSF7etf5uoLdtLRMMHR8XpufupcHnp5fdbrmW+c\nZuh9z9P2ozOpGI9mPb8QqJj7Z+qsnpIXeB1+IAWZboTqySkGN6xhLmL1+ZiL1DJZvZKK2Ekgs7ce\nXTUEo53x7FHnRWc248I4CUrp1lHRMMHU3o3xoYCdF6gVDRMJ88detDJSnZEmqT6ZsK10Hntj/xjD\nq7oZ6WgDLGEfXtVNzdh4vE0p3Cjb17/Mddsep6txgohAV+ME1217nO3rX856XZkqBy1FZSFn2Olc\nhp9WSv/HUD13D/yc1JM9s9TNHGGyeiXVcyPMVDZTN3OEythUWmFfCMHYWaZpEpCSSR6y92f2/3Tr\ncF6QpkpSqmwZheYjVrm+uhMw2YK0vmYVAbHJHIqxPPbhVd1MNjcxXV9H62v9NB+1ppeCsANcfcFO\naqrmE6bVVM1z9QU7s/LekysHNT24LsE7zzQ/W0pdhMJMKYdpVNyTyCbGXhmbonpuhOmqNqKzQ1kI\nu0W65CI/Y69fvnYvrN3LbUO/kzaJKV2S0vgrzUh00hL28Xar6EdWwm7RfHTIEvaGeqLjEyUn7AAd\n9tOM3+mp8Koc5B57PNP8dKiQF4dSDNOouGeJ++XpXKSWmcpmorNDzFQ2M9M9gTC5aJlUL0yTk4vW\nTUbYdNqerOzZ9+LpGZOYUiUpJcTY7WpOTLZgopNIdDKrl6cjHW1M19cRHZ9gur6OkY42uh/cndW+\nFJuj4/V0NS4W8qNZjI+fqnKQ451nmu9GhTxclJrAq7i7yHQzJQv7ZPXKeChmpnvCilUnhTTSCbuT\nXPTONz4cH9AL8C3w7kHAeutinklMqZKUaD5ieeyuak4SncREJ61C21NHsHq1LsZL2IdXdcdDMc73\nqtNGaHuxdF6o3vzUuVy37fGE0MzJ2Qpufupc3+vIVDnIc34Ejr/nGA1Pvz5345UloZQEXl+o2mQj\n7ADzkZqEGLtEJ5HW12CmJt4mXRfH1ukKtpyzg3e+8WHAEvQt5+xgsD/9ULtuBvtXsuWcHfEfg2hP\nP9F1BxPqqHolKdF8ZMFOu5qT84Mk0Unqp44wH6nBC6/ujicbGxJi7N0P7qbz6b1MrGzzvS9h4KGX\n13PDIxcxMFZPzMDAWD03PHJRVvH2dJWDps7q4eSmqcXzK2LMtR8NYA+UpaBUnqgyJjEVirAlMWUr\n7g6pYuzphD2fWqap8Ftco5D92KG0YuxLQakIgZIdxbzOA0tiKgeWStiDFvW9L5xJa9sxOrsG4tPc\nfdiBQPqxg5bIy4ZM19Nlbc/x7lMf5muro3zm8DS3vnIJ9w+dtUTWJTLXPMTIW39O888vp3KktSg2\nlCKlEJ4p+7BMqQo7WIU2nnjsknglpeQ+7BBMP3a/wh72i72Q+O1Pflnbc3x6/T3c1hVhZ02U27oi\nfHr9PVzW9lza5QrF2EWPQdUsYxc9WpTtlzJhfyore3HPhTAIO0Bn1wAXXvwwTzx2CSdf7Ul4cerg\n9HM3w6uIjbYnvDx1UGHPnWwThK5Z+xBj1THuaKjHiHB7Qz3j1TGuWftQ4YxMwVzzELHmURCINY8w\n1zy85DZ++nZlAAAgAElEQVQohaOsxT1Xr92LpRZ2h86uATZs3MvM4dVUdQ94ZrieHBZw+rHXnVBh\nz5N8sj47q0e5saWZGNaImzGEG1ua6az2n5kcFGMXPZb0Xb33bAmz964x9xRkE47JR9g/1Jp68Mz/\nM3xRxuUHB7o4sH9z2kIbQfRjT0U5CXsQN/Ke+WbuaKhnNmKJ+2zE8t7fPeC/IlcQuL12IMF7X+rY\nux8nSt/5ZE/ZinsuN2qQwp5O1N1t0gn84EAXTzx2CZUbX/EstAEE0o89FeUi7EF6Z38eXU+MxBo2\nMYQvRLMfnCwfkr32hemP0nrvOwuyzWyehDMtGyaxD+vL1bIV93TkcxFCemH3I+rJ7VMJ/PBQOxde\n/DDPTrcAiYU24r1jPPqx0/oa89O1VMa8xd0PYbyYg6YQj9y9DTHmI4lFUGYjQm9DjJbAt5aaWMPE\ngtfuIBBrGPdsnwv53kd+1x0moQ8TZSnuhfTagxT2TGw+w65BfnBBFipbRhPCMsndHQGaDp8ETi6a\nDtqXHQobR2356TsKtu5saPvxBwqy3kIKerptFlvgw+i9l6W4p8PvxVlsYXdIlbyUKllJk5RSE+aX\nY2GmGIKeyoZii3yYKDtxnzqrJ15ow6npCVYR55m6WmBxrwUvr92rCMbgQBfDQ+1sPuN5Hn3oUjq7\nj7DptD1xYb/92cvYffg0vnjltwPZl70vnMncXJ1nIQ48BjBTYV9cjOOm/suKlkCUDj+JTrGaScYu\nfpTGx7YROVlbEDtSbcMt6KZyEnPqg8grb0Hm6hKWv6JqN9fW3k93ZIT+WDPfmLqMe2e3ZG1Hum24\nCYMXHxbKTtzBKrRxdN2aeNHmqYZ6jq5bQ+38ESp9dFpoOHWEuRMm4eXl2dETPPHYJVx4sTVWTGf3\nEXY/cz7n1fZCqyXsNz/+Hq6+6NbA9qO17RhTv7g0boOTxGQNCpbFespI2N0Dg3U1TvDpunsAllzg\n03m7V1Tt5lP19/L19gY70WmWT1Xcy8TKygRhjK3ZAe2DDL/peSKHUr94z0fsJs/6NXMd6bdhVj4L\nDQOYlc8irjZXVO3mS/V3UStWtbJVFSN8qf4umCBrgU+1DS9U4C3KUtxrxyfo6D3E0XVraDx2nLH2\nFXT0HkoomeeQKmEpuQjGE0e3cuHFD8eHAth02h7Oq+3l5sffw5OvnM2eIxu4+qJb+Z2z7w9sP56d\nbvEsxHFyOPlNWXqvPRPLQdghRTGOijmuWfvQkoi73/DFtbX3M145n5Do9McnRri29v64KJrKSWh7\nyXop2vYS5sjZKT3aXMMmpnISs+HltNtIZ8e1tffHhd2hVmYT9sOvHX731aEYAh+2uHtZJTG5Y6q1\n4xM0HjvOSHcnjceOJ4Ro0uGOtbuLYGzYuDdhjBeA3zn7fk5feYAXjryO01ceCFTYvWyo6h7wFPZ0\nlIOH4yQcpSq6UcgEouHN1fE/v3RHRjwTnbojC9eeWfks4Az6Z+zvweJnG+nauO11k2p6PnYoiykr\ncXcz1VDPWPsKmvsHGWtfwdHTF7+AzFQH1SmCcfqZuziwf3N8jBewXqDe/uxl7DmygTNWvsSeIxu4\n/dnLsrYzUyJTciEOM73Yo0nltS/3cExyFungTJNnu1TTcyUXQXfzvLR4Jjq9IFavqLgnG7Gvz0jM\n8mgrF79nyRU/28jUpj/m/VI/1fRc7UhFGF70FpOyFHcnxt7Re4jW/kE6eg8xWb0yXuzaD058e9ub\nH+DMLbviY7w4Au+OsX/1XX/L1Rfdys2PvycngffiZwc3JxTiqDmlb6EWqkvgy03Y0w0N8N2D2zk5\nnxiJPDlfyXcPbs97u/kKupvPNW2Ie+0OMYTPNW4Akj1Zh2A9Wj/byNTmG1OXMWWqEuZOmSq+MeX/\nHliKfV2ulE3M3X2zz9TVxl+mghWiqZuxClSkS+xxh2ScIhhOKMYZxGt4qJ3OrgF2Hz4tIcbu/N99\n+LTAwjPJhTicBCVmaiCanxdXSsLutwujE1e/Zu1DdFaPMjjTxHcPbs853l4oz/Bg7Tx4JDq9Wjdv\neWP1gwuerEMkZk0PCj/byNDm3tktMEF+vWWWYl+XKWVTrCOdAPgdRyabvu1B9GtPFZLx6tvu1a89\nV6+9FIS9mH3Sy/1xv5RY6ndKS3HvaLGOPMkUb09HoRKWgqCUX6AWO8lIRV0pJcoy5h4E04dXcnb0\nRMK0wYEu9r5wpq/lb3vmcnb3bUqYtrtvE7c9czlgee333X0VO59KLJp8xwOXM/b0byRMy8Zrz0TY\nvPZ8htcNiqBi6dliKieJve4/An1R6ubztXdzX9tfcc7p/8BP2/6Kz9fenfU6rqjazTfbb6Bu6/f4\ndvsNXFG1uwCWKrlQFuJeCGGoaJhIeIHqjNDY2nYsw5IWGzt7+dpPr4kL/O6+TXztp9ewsbM3Ho7p\n7Orn5f2b4wJ/xwOXMzvQRWVz4pC+yZR6OCYMgg7FE3UHd+JO0Hy+9m7eF93B91qb2FkT5XutTbwv\nuiMrgXeSlP6uK8p4RPh6V5Qv1d+lAh8SykLc05HrWDKVLaPxHjLP794az07t7BrwNQ77lp59fPq3\nvsvXfnoN//Krd/K1n17Dp3/ru+yqa4+3OfeCX7F+415e3r+Z/7jj3cwOdFHVNUDtxlez28kSIQyC\n7lDsEMyixJ2Avff3Rp/mWGUkIVFqqDLCe6NP+17HtbX382q14UB1FYhwoLqKQ9WGa2uDz+dQsqfs\nxT0fnCpIe57f6pnElIktPft421m/4N93vIO3nfWLBGF3OPeCX1FbN8HUZANUzyQI+3Lw2sPipTsU\n21t3KHTiTgXGM1GqYlG3w9R0R0b4XEfiNfuZjvask5QKRSm/XwoCFfc8cKogeSUx+WF33ybue+7N\n/O75/8Htv77Mc/mdT72eqcl6qJ6GmWqm9p8ClL6wh0nQHZZS1Mc2xFL+jW8cx7QnJu6Y9mC99/6K\nCs9EqYGKCt/reLSyNe61A3Hv/fHKpa3kFBbCEtZ00N4yOTJ3ookn9i+MJ9PR1b8oNJOu14wTY3dC\nMRc2PZywPFjC/vL+zfFQzNT+U5h1fgAqTqRcd5gpJ0HPtcdVVXQXXok7M5ufYebkG/MaJ8jhc40b\niCVV4oonSg35W8dnOzvwqub16Y4O8PfqqWCUu9cOKu6e+Lkp58frqdz4SsokJiCtwO8fXMfZFz0e\nD8V4LT840J0QY3f+zx5rJdKVKO5h99rDJOphFHQ3kYpBRBLXIxIjUjGYsI18RH5nTR1EEgu2zEaE\np2vqfD/Oj1fNLnjtC4YyXj2rIYEQUBZJTNkmMKW6QbMt0JEvyyVZKQzCHnZBz5UgvPjlRrG89qW6\nnwJLYhKRGuAXQNRuf4sx5ktJbQS4AXg7VpWIq40xO3MxPF+GzlhHzdAo9QNWebmps3rihTiaB61n\nRa9iHXORWuYjNUTnhmk8ECn4Desu5uGw78XTGexfyZu2P8DPDm5eVBBk/JVma9yYmRqk8ThmbAVU\nJ5bMc+9HOoK+EJOLYNz81LncW7fN9/LXnnIfV3U/QwWGeYQ7+8/hG6++LaHNZW3P+Ro6INNY6b/X\n8AB/213Np/pn+MH4pYvS4TMVn3CW/9NjM/z/0bdwJ4l5B0uFVPQxf87PkX2XE5lI/AH1UyQj32MR\nFH624deOQgp7uuIozv003zjN0Puep+1HZ1IxnkVRhQLg52d/GrjUGLMVOBt4m4hcmNTmCuB19t/H\ngO8EamUW1AyN0rdtKxNdK4CFQcKqJxdig06xjqmGesASxMnqlVTEvOuKpiNVmbtMOMU89r14OmAJ\n++5nzmc4Oh9fZ0XDBFN7NzJ3whq10EzXYYZX2YKO9f/YqviAZ+79WErvxSmC0dU4QUSsIhjXXfJL\nLmt7ztfy155yH+/q3kmlGESgUgzv6t7JtafcF29zWdtzfHr9PXRHR4kIdEdH+dTGe7nwrBcTBu3K\nJOxfqr+L29uEnTVRbm8Tz37ZqfqXJy9/Z7Pw1xN3c9X0r7M4WsFRU/cwYIi97kFPO1dVjBCRhSIZ\n7v3M91gEiZ9tZGrTunem4Ne8U7hk8szU53t0ey8zp4ww+pbegtrih4zibiyckuhV9l9yLOe3ge/b\nbZ8AWkRkZbCm+qN+4Dg9j+yib9tWjm7ZmFBxycFdrONkZRuT1SupmzmSdtCwoNl02h62nLOD3c+c\nz0M/v5zdz5xPdN1Boj398TbugiCjuzZghlchra9ZA4RhFbqumznCZPXKhP3o2JO+K1rQXnu6Ihh+\nuKr7Ga/QLVd1PxP//pH1D1NTMZfQxin84BevIhgTlfMJ60jXv/yTTYuXn6yY51NTD/i2ISikog+R\nGURAZIZY/cI5TVckw90mn2MRFH62kanNUjgysZpJpk+1CpdMrz9ArGZBK9xe+8R5/RCBifP6mW+Y\nLrhd6fAVsBORChF5FhgE/tMY82RSkx7gkOv7YXta8no+JiI7RGTHTAGFtH7gOK37DnFsy4aUhTic\nYh3TVW1Uz43kJey5eu+bTttDe8cAx452E2kcSxB2h8qWUUzNCIy3Q92JuLA7sdbK2BTVcyPx/Vhq\nYQfyLoKRqm91BSbujQdR+CGfIhhjG2Ksinkvvyq29P26La99Abf37udYLYeCIEvhrTtMnvVrENsG\nMZ7e++j23oQ2xfbefYm7MWbeGHM2sBp4vYjkNEaqMeYmY8z5xpjzq7MYOz1bJrpWMLxpDe27DzDW\nviIefnHjFOuIzg4xU9nsayx3r5eZDrkI/L4XT+fY0S4ijaPExhqZ7ute1GbsxZUw2QINx2CyBTNd\nl/ASbS5Sy0xlc3w/vPa10ORbBGMe7+pR7ulBFH7IpQiGaX+J8Y3Wg+tzVd7LP1fZ4tuGIHB77cAi\n793PsSrpgiCtL9H86tL9oMa99grbhopY3Htf5LVX2uJeaYruvWf1qt0YcwJ4EHhb0qw+YI3r+2p7\n2pIz0bWCvm1b6XlkFx2798fDL27RcxfrqJkbioc2sinW4cXPDm72JfI/O7iZu355STwU07BlD9F1\nB5nuXRsX+PFXmhl7cWU8FBNpOoa0voYZXhxjr5s5Qs3cEJ0HFu+rm6C9dicRKd8iGLdMn0dypy1j\nrOkOQRR+yLUIhtXvHD6zYr3n8p9pW+/bhiBI9trjbLC8dz/HKuiCII4X7f7LRKZttO6dobr2aRZd\nHCk850KR4LWnsCHBa3e1Kab37qe3TAcwa4w5ISK1wG8C/zOp2Z3AJ0XkR8AbgBFjzJHArfXBybYm\neh7ZFe8t48TXZ+pq4+EZd7GOk1RTGZtaVKwjVY+Z8VeaPbtEuvEj8HMnmhJi7M7/uRNNzM7YIaKZ\nmoQYu0QnqZtasHM+UhN/V2DdTDOL9tWhEMLukGsRDOfl51enrgSs8U6c3jK3TJ8Xnw7BFH7IpQiG\nu3/5wep5KjyWP1g9D4trqxcMt9e+MA2osATVz7HKtSBIRcUALT5DIZkE/sS6AeYzbGOu7diCx+xQ\nEWOu/agvG4IglQ2xhtdwos8za0cXvHaHSmNNLxIZ+7mLyBbgn4EKLE//340xXxaRPwYwxtxod4X8\nFpZHPwn8oTFmR7r1LlU/dz99rP0W63CTSeBzJV3oJyz92fPttx6GsVvSUcx+6/lQqD7vmu25mGIO\nNRBYP3djzG7gHI/pN7o+G+AT2Rq5FNQ+11eQJBo/Hny260tHPrVQgySfYxl2UYfSFfZCoKJe2pTl\n8AOZkpgcMiUzBSHwuYo6LH2R61yF3Y+o+026yaseZwbGNsS4avrXfGrqAVbFRngt0szXay/lzuhC\nklKm+X7biEwSrXuY6clLMGZxUo6fdaQj32NVasKeLsEoaMI2QFgqylLcnSQmJ+7ufjGZjB+Bh+zD\nNJlE3dl2KkpB2P166k5CjdM320m6YYK4IPlpkw+OsP/1xN3U2QH01bER/nrCKl5xZ/Q3Ms4HfLUB\na3CwSMUAVdFdzJx8Y4ItfteRinyOVamJuoM7wajh6ddnXiBHSkXYoUyH/HUnMQ13dzJVkX8S0/gr\nzfG/TPNLSdhzHZo3mxCMn6SbTzZ5t/lk0/3xoXLz5VNTD8QF1aGO2XiSUqb5ftuITFJZvd/Kxq3e\nj0hi90I/60iHn+PpRakKe7oEoyApJWGHMvXcYSGJaaS7k+b+QWhOfUFkO9aMH/FOt610LLWwZ0su\ncfWUSTcVI/Hjvuq4dxt3ApHXOfLzkjG+jRTJSM70TPP9tkkc0tcs8t79rCMd2SZ8laqoO3glGAXt\nvZeasEOZeO5eJ8ZJYmruH2SsfQU1fVUeSy6wFKPvBSHsQbFUwg7wWoX3j+FrkWbPz6naeOEugpHR\njgzb8GNDpjYLXnvM/h5b5L3nuq8O2SR8lbqwp0swCopSFHYoE3FPxp3E1No/GA/RZEpiKpTANx6I\nBCbsxboQcxF2R3S/XnspkyT+uE5SxddrL41/99PG7/aSp/ndRhB2pirE4SRK+d1OMu7rJ4iEr1LB\nT4JRPpSqsEOZhmXcSUyQmOgUa03/i+/cREHEeP3+WCy1sGfrtWcr7MnHznlJmK53iJ82uW7f7zaC\nsDNTIY4g9tVvwlepe+1QuCSnUhZ1h7Io1gHZCVa+YpWObLz/bG6+Ygh7IY+T4p9cnyiXg7gXgrAL\ne2BJTMuFQiUzQWHCNctJ2FXUC4dWYgqWsAt7NpSNuGdD696ZomVTZutNLSdhz5TYc/34PXxwZmH8\nmR9Wn8f1DW/Pah354scGP3b4sVMiQ9TW38fUxNswsbaEefEkp5bCJSn5rXxVCmRKcgrqPiq1SkzL\nhmxO4FKOFe3eZjaEWdhz6XvuTuxJ5vrxe/jQzA4qMQhQieFDMzu4fvwe3+vIF782+LHDj53R2l+A\nzFr/XThJTqtjqSst5YtX5atPr7/Hd3WtsJGqilLtc32BeuslVYmp3FkKgc/2hyTIC7JQwp4tmRJ7\nPjjz9KJR38We7ncd+eLHBj92+LFTIkNEKkYQwfofGYrP80pyyqYqlZ9r7Zq1Dy2qfJVNda0w4ZXk\nFLSoQ4lWYlpO5HJCC+HFZzPutZuwxwRzja97Jfa4SVetye868sWPDX7s8GNnsrfufG88EGHVfO5V\nqfxeb6mqaPmtrhUmErtLxphZ+cuCbKckKzEtN3IVyHxFPldBdyjm0L1+vPZchL3h1BHqTxmgKvpS\nQmJPVfQl6k8ZoOHUERpOHclYrclPclC++KkYlckOP3a6vXarjeW91/dZg9oFUZUqE/lW1woLsZpJ\nptcdWOguWaAKSSVfiUmx8Ko64+cvH5absDuibRnzoncj1/Qft5/lkfoDP6y2qjX5SQ7Klx9Wn5fW\nBj92+LEz2WsHK/zDqVYFplyTlLK5BvOtrlVsnLDLzMonlqRCUklWYlquFLJrZJAUOwwTtLB7jp45\ndxxIXkfMnm7x1VOs7Mz/59hznj1V/CQH5cv1DW+HcdL2lslkhx87IxXjiyotIUD1GJBbVapsnYtc\nq2sVE697ZakqJJVkJaZCsdRJTKkIs8AXStj97nOQwh505ap8BmcLK1pJyR/FdniKjSYx+SSMHnwh\nL94g97VYwu6sM9/RN8OQXFXoJKRiCXu5C3AYKHtxh4ULsdgiX6wbwitZ5Zb2c9MuE7SwXzH0Ite9\n9jjdM2P0Vzdyw6qLuLftNF/LOlw/fg/nml1cs7KLfzwywA7Z6plg5OAl8PlWQPKLyCR18jCm8i3I\nnHcSk6mcxJz6IPJK6jbpKJSwq3CXBiruLorlxS/VzeK1b06yitOnuTs6yqc23svERGXeFY6yEfbr\nX72fWmPZsGpmjOtftfps+xV4J8HoXau6GY8IX+ho4yev7YBxFgl8Km853wpIfmk8ECG2Zje0D2BW\nPoscusiznVn5LDSkb5OKIIVdxbw00d4ySRQiuSEM20qFV7JKpoQYP157NqGY6157PC7scRvMHNe9\n9rjvdXxw5mn2VlVyoLoKRDhQXcVLVZWLEoyShd39Pd8KSJlwhnY2lZPQ9pL1krTtJet7En7apCIo\nYQ/D9ankjop7Cgp1YTvrXeqbJtUTSaqklFQJMUELO0D3zFhW072owPC5jvaEaZ/paE+ZeOTGEd18\nKyBlWr+DWfks7iQm63siftp4EaSwK6WNhmUy4L7IcwnZhP0mGZxpoju6WOBzTYjJ5eVpf3UjqzyE\nvL+60fc6nq+qinvtQNx731O10B8808vL/vlmVlUstt9vBSQ3qbYV98gj9o9kJGZ55kfOjsfV/bRJ\nRsMwSjIq7llQqhd9uh+l7x7cnhBzh9QJMYXqXXLDqosSYu4AU1LJDatSx5mTe8p8snM1MLeo3X/t\nXA1T/nqlfGPqMr5Uf1dCcelsqj352UaiRx6fmhBX99PGjQq74oWKe5mTnKySy/CxDrl2eXRemubT\nW+ZYFUhy5o8Ix6oAn+U0UyUHPTi8Bf/PEBmoH1zwyB0iMWt6Nm0IvjeMCvvyouyTmJY7QSUsFSLW\nng/Z9HFfbgUtCtHFUYW9dNAkJmVJySTsQfRjd6/Dqw+6Vx/1Bw9vzWl/0pFv//NcKGQykgp7cGix\nDiVUBOG1p8Ppx75qZowIC/3YrxhKMWCYj3U4fdCvmraKLyQUsLDnf3X87kALWDi4+58XkqAGnUuH\nCnuwaLEOZUkodsatQxD92L3W4e6Dnm8BC7/k0/88HUGPIuoHFfZg0WIdyrIjU0gmiH7sqdo6fdBT\n9UX3U8AiG3Ltf+7FUgp5MirswaPFOpQlYSlfpGYiVX/1bPqxp2rr9EFP1Rc9yAIWKfufZ+m9F0vQ\nQbNOC4UW61DKkhtWXcSUJL67z9SPPZm/qXorkyQWqHD3Qf967aWL5vspYJEN6fqfZ6KYXrqDinrh\n0GIdSlmSbz/28Vea471iUo3YeGf0N6gZkKwKWGSNz/7nbsIylroKe2HRYh0utJ97YVku/dv99mcP\nU1/2sAg6qKgvR7SfexkTll4y+VKoakuO+PqpNJXN+sKEirqSUdxFZA3wfaALK+B4kzHmhqQ224E7\ngFfsSbcZY74crKnlS76JEV7FOO4fOiswcfNDpiSm5Pl/U/VWzzHURSaJ1j3M9OQlGGMlEGXjtbuF\nuFiifFnbc7z71If52uoonzk8za2vXBJYbdJsRT1MSTdKsPi5K+aAPzPGnAFcCHxCRM7waPeIMeZs\n+0+FPUDySYxwinF0R0eJiFWM49Pr7+GytucCsy+Th50piclrvjtByU1VdBeRigGqoruA3IW9WDjn\n47auCDtrotzWFQnkfOTaCyZMSTdKsGS8M4wxR4wxO+3PY8AeYHk895cA+SZGeBXjqKmY45q1DwVo\nZXoyJTFlSlByEJmksno/IlBZvZ+GV0/6tiEMwg7W+RirjnFHQz1GhNsb6hmvjuV8PvLp2hi2pBsl\nWLJ6CyUi64BzgCc9Zl8kIrtF5F4ROTPF8h8TkR0ismMm5nOovjIn38SIVMU4Uk3PlXTee6YkpkwJ\nSg6Wt24dCzHZdUEMC53Vo9zY0kwMawTLGMKNLc1ZnY+gCr6ELelGCRbf4i4iDcCtwJ8YY5KvxJ3A\nWmPMFuCbwO1e6zDG3GSMOd8Yc351pDZXm8uGIBIjBmeaspqeD6kEPl0C0vgrzSkTkNzTF7x2/wlE\nYRJ1hz3zzdzRUM9sxBL32Yjlvb84nz60FXQFrzAm3SjB4kvcRaQKS9j/xRhzW/J8Y8yoMWbc/nwP\nUCUi7cntlOzIJTEiuafMdw9u5+R84nvzk/OVfPfg9iBMXMT4K83xPwevJKZMCUjJRTKqoruQRd12\nU3vvYRR2gD+Pro977Q4xhC9E1ydMc4t5IXq+hDHpRgkWP71lBPhHYI8x5u9StOkGBowxRkRej/Wj\nMRSopWVIEIkRycU43L1lKPDPryPwP+YNTNfVpU1AgtQJSgBV80ch6i+BKKzCDtDbEGM+kijusxHh\nYOU0XUvYfTGMSTdKsGRMYhKRNwGPAL8GnLvrC8BaAGPMjSLySeDjWD1rpoA/NcakHfJPk5gKQzZ9\n3P12hSxUeT2/lFqPmExoH3QlHwJLYjLGPApJz5GL23wL+JZ/8xQlM9lmnYZZ2FXQlaVGM1RDzvb1\nL3P1BTvpaJjg6Hg9Nz91Lg+9vD7zgj5o3Tvjy3tvPBDJ23u/fvwePjjzNBUY5hF+WH0e1ze8Pe02\n/RJWUVdBV4qJinuI2b7+Za7b9jg1VfMAdDVOcN02K9oVlMD7JR+Bv378Hj40syP++FeJ4UMzO2Cc\nRQJf6t66CroSFsIz2pKyiKsv2BkXdoeaqnmuvmBnUezJdXCuD848vSiuJ/b0fNYfBmEvdK8WRckV\n9dxDTEfDRFbTwRKbQg4closHX7FoDPSF6bn+YGj1IkVJj4p7iDk6Xk9X42IhPzpeH9g2/Mbd3WQr\n8PMIlR4CP2/Svqf3JB9RV3FWygkNy4SYm586l5OzFQnTTs5WcPNT5xbJogUaD0Tif5n4YfV5JPe4\nNQZumT4vq22qsCuKf9RzDzHOS9NC9ZZxyMV7d5NJ4P+WK6muhfdGF3rL3DJ9Hl+dutK3ffmgwq6U\nI1qJaRmSS8x9Kcd2zwb11hUlEb9JTBqWUYBw9Dxxk+9ojirsSrmjYRkPhs5YR83QKPUDx+PTJrpW\ncLKtibYXeotnmAeeSU7P5ea95xueSccVVbt9Fa8uhxBMIRPTFMVBPXcPaoZG6du2lYmuFYAl7H3b\ntlIzFK5BlZwkp67GCSKykOS0ff3LOa+zEB78FVW7+VL9XayqGCEisKpihC/V38UVVbsD3XapCHvQ\n50xRvFBx96B+4Dg9j+yib9tWjm7ZSN+2rfQ8sivBkw8DhUpyClrgr629n1qZTZhWK7NcW3t/PPxS\nDsIO4UtMU5YvKu4pqB84Tuu+QxzbsoHWfYdCJ+yQPskpX7ELsoJRd2Qkq+nZUGqZobkkpilKLqi4\np2CiawXDm9bQvvsAw5vWxEM0YSJVMlPQSU75inx/zLvKUL7VoEpJ1B2W4pwpCqi4e+LE2Hse2UXH\n7r9OZKsAAAZnSURBVP3xEE3YBD5TklOQ4ucOn/gRfHe7f3z5kkCrQZWat+4mzIlpyvJCe8t4cLKt\nKSHG7sTgT7Y1hSo84yfJqZBjzfj16NNWg8qSUhV1h6VKTFMUTWIqAwo5kNhSUeqirihBEVglJqX0\nKfRIkYVkqUV9+/qXeecbn+FvTq3js69Mcucvz8naq9Z+7EoY0Jh7mVBqnm8x4upOH/RbV1exsybK\nraursu6Drv3YlbCg4l5GlIrAF8vOqy/YyViN4Y6GeowItzfUM15jsuqDrv3YlbCg4l5mhFngi90L\npqNhghtbmonZdaNiCDe2NGfVB137sSthQWPuZYgjoGGIw4fpx+bFkw3c0VDPbMQS99mI5b2/5/Bs\nhiUXWIoCK4riB/Xcy5hiCmuxvXQv/kLWxr12hxjCX7DW9zq0H7sSFtRzL3OW0osPm5gn09tM3Gt3\nmI0Ir7RAl891aD92JSyouCtAovAGJfRhF/Nkur59QSDreejl9SrmStFRcVcWkUqUvUS/1ARcUcoF\nFXfFNyrkilI66AtVRVGUZYiKu6IoyjJExV1RFGUZouKuKIqyDFFxVxRFWYaouCuKoixDVNwVRVGW\nISruiqIoy5CMSUwisgb4PtbwGga4yRhzQ1IbAW4A3g5MAlcbY3QA6yVAq/4oiuKFnwzVOeDPjDE7\nRaQReFpE/tMY84KrzRXA6+y/NwDfsf8rBcSp+uMUh3Cq/gAq8IpS5mQMyxhjjjheuDFmDNgDJA8y\n8tvA943FE0CLiKwM3FolAa36oyhKKrIaW0ZE1gHnAE8mzeoBDrm+H7anHUla/mPAx+yv0/f1ffO5\nbLZfJNqBY8U2wot3/GvkPO854zx95JtPL601vgnt8UxC7QwWtTM4TvHTyLe4i0gDcCvwJ8aY0Vws\nMsbcBNxkr2+HMeb8XNazlKidwaJ2BovaGSylYqcffPWWEZEqLGH/F2PMbR5N+oA1ru+r7WmKoihK\nEcgo7nZPmH8E9hhj/i5FszuBD4vFhcCIMeZIiraKoihKgfETlrkY+BDwaxF51p72BbAKSxpjbgTu\nweoGuR+rK+Qf+ljvTVlbWxzUzmBRO4NF7QyWUrEzI2KMKbYNiqIoSsBohqqiKMoyRMVdURRlGbIk\n4i4iFSLyjIjc7TFPROQbIrJfRHaLyLlLYZMXGezcLiIjIvKs/ffFItnYKyK/tm3Y4TE/FMfTh51h\nOZ4tInKLiLwoIntE5I1J88NyPDPZWfTjKSKbXdt/VkRGReRPktoU/Xj6tLPoxzNflqpA9nVYma1N\nHvPCNHRBOjsBHjHGXLmE9qTiLcaYVIkWYTqe6eyEcBzPG4D7jDHvFZFqoC5pfliOZyY7ocjH0xiz\nFzgbLEcJqzv0T5KaFf14+rQTwnF95kzBPXcRWQ28A/heiiahGLrAh52lQiiOZykgIs3Am7G6+mKM\nmTHGnEhqVvTj6dPOsHEZcMAY82rS9KIfzyRS2VnyLEVY5u+BzwCxFPNTDV2w1GSyE+Ai+1HyXhE5\nc4nsSsYAPxeRp+3hHJIJy/HMZCcU/3ieChwF/skOx31PROqT2oThePqxE4p/PN28H/hXj+lhOJ5u\nUtkJ4TqeWVNQcReRK4FBY0xYxzkBfNu5E1hrjNkCfBO4fUmMW8ybjDFnYz3efkJE3lwkOzKRyc4w\nHM9K4FzgO8aYc4AJ4HNFsCMTfuwMw/EEwA4bXQX8uFg2+CGDnaE5nrlSaM/9YuAqEekFfgRcKiI/\nSGoThqELMtppjBk1xozbn+8BqkSkfYntxBjTZ/8fxIoTvj6pSRiOZ0Y7Q3I8DwOHjTHOQHi3YImo\nmzAcz4x2huR4OlwB7DTGDHjMC8PxdEhpZ8iOZ04UVNyNMZ83xqw2xqzDevx5wBjz+0nNij50gR87\nRaRbRMT+/HqsYze0lHaKSL1YY+pjP5ZfDiSPrFn04+nHzjAcT2NMP3BIRDbbky4DXkhqVvTj6cfO\nMBxPFx8gdaij6MfTRUo7Q3Y8c2KpesskICJ/DHkNXbAkJNn5XuDjIjIHTAHvN0uf3tsF/MS+5iqB\nHxpj7gvh8fRjZxiOJ8B/A/7FfkR/GfjDEB5PP3aG4njaP+a/CfyRa1rojqcPO0NxPPNBhx9QFEVZ\nhmiGqqIoyjJExV1RFGUZouKuKIqyDFFxVxRFWYaouCuKoixDVNwVRVGWISruiqIoy5D/C0FKeAeM\nJlWdAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ax=plt.gca()\n", "ax.contourf(coords[:,:,0],coords[:,:,1],np.amax(gmm_pdf,axis=2))\n", "ax.plot(X[0:50,0],X[0:50,1],'x',X[50:100,0],X[50:100,1],'o',X[100:150,0],X[100:150,1],'^')\n", "plt.title('K-means GMM of all three classes, scatter plot overlaid')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Expectation Maximization (EM)\n", "Did you notice: K-means can be used to calculate the mixture weights and covariance matrices, but it doesn't use them in the optimization? The total distance, $D$, only depends on the means.\n", "\n", "Expectation-Maximization (EM) uses a training criterion that uses all of the GMM parameters: Maximum Likelihood.\n", "\n", "The \"Log Likelihood of the Data given the Model\" is\n", "$${\\mathcal L} = \\ln p(X|Y,\\Lambda)$$\n", "\n", "where $$X=[\\vec{x}_1,\\vec{x}_2,\\ldots]$$\n", "$$Y=[y_1,y_2,\\ldots]$$\n", "and the parameters are $$\\Lambda=\\left\\{c_{00},\\ldots,\\Sigma_{21}\\right\\}$$\n", "\n", "If we assume that the observations are independent, given the parameters, then\n", "$${\\mathcal L}=\\ln \\prod_{n=0}^{N-1} p_{X|Y,\\Lambda}(\\vec{x}_n|y_n,\\Lambda)$$\n", "\n", "\n", "$${\\mathcal L} = \\ln \\prod_{n=0}^{N-1} p_{X|Y}(\\vec{x}_n|y_n)$$\n", "\n", "where $y_n$ is the true class of the n'th datum, and of course\n", "$${\\mathcal L} = \\sum_{n=0}^{N-1} \\ln p_{X|Y}(\\vec{x}_n|y_n)$$\n", "\n", "So in other words,\n", "$${\\mathcal L} = \\sum_{n=0}^{N-1} \\ln\\left(\\sum_{k=0}^{K-1}c_{yk}{\\mathcal N}(\\vec{x}_n;\\vec\\mu_{yk},\\Sigma_{yk})\\right)$$\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The EM algorithm finds the maximum likelihood model parameters. Suppose we define the set of all model parameters to be\n", "$$\\Lambda = \\left\\{c_{00},\\ldots,c_{21},\\vec\\mu_{00},\\ldots,\\vec\\mu_{21},\\Sigma_{00},\\ldots,\\Sigma_{21}\\right\\}$$\n", "\n", "A \"maximum likelihood\" training algorithm is defined to be one that chooses the parameters in order to maximize the likelihood:\n", "$$\\Lambda^* = \\arg\\max {\\mathcal L}(\\Lambda)$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Why is the log useful?\n", "The log is useful because it allows us to differentiate ${\\mathcal L}$ with respect to any particular parameter,\n", "and wind up with a derivative that has separate terms for each of the $N$ different data. \n", "\n", "Let me show you what I mean. Suppose we tried to differentiate like this:\n", "$$\\frac{\\partial}{\\partial\\vec\\mu_{00}}\\prod_{n=0}^{N-1}p_{X|Y}(\\vec{x}_n|y_n)$$\n", "\n", "... we would get a product form, with all of the terms for all different $\\vec{x}_n$ multiplied together. On the other hand, suppose we do this:\n", "$$\\frac{\\partial}{\\partial\\vec\\mu_{00}}\\sum_{n=0}^{N-1}\\ln p_{X|Y}(\\vec{x}_n|y_n)$$\n", "\n", "... we get the sum of individual derivatives, one for each $\\vec{x}_n$:\n", "$$\\frac{\\partial{\\mathcal L}}{\\partial\\vec\\mu_{00}} = \\sum_{n=0}^{N-1}\\frac{\\partial}{\\partial\\vec\\mu_{00}}\\ln p_{X|Y}(\\vec{x}_n|y_n)$$\n", "\n", "OK, sure, but that last derivative is zero, unless the token $n$ is one that comes from class $y_n=0$. So let's get rid of all of the other training tokens now, and focus only on the ones that come from class $y_n=0$. For example, in the iris dataset, this would be the first $N_0=50$ tokens. In general, let's say we're only summing the $N_0$ tokens for which $y_n=0$:\n", "$$\\frac{\\partial{\\mathcal L}}{\\partial\\vec\\mu_{00}}=\\sum_{n=0}^{N_0-1}\\frac{\\partial}{\\partial\\vec\\mu_{00}}\\ln p_{X|Y}(\\vec{x}_n|y_n=0)$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Let's calculate that derivative\n", "Let's calculate the derivative of the log-likelihood with respect to $\\vec\\mu_{00}$. Continuing the equation above, we get\n", "$$\\frac{\\partial{\\mathcal L}}{\\partial\\vec\\mu_{00}}=\\sum_{n=0}^{N_0-1}\\frac{\\partial}{\\partial\\vec\\mu_{00}}\\ln\\left(\\sum_{k=0}^{K-1}c_{0k}\n", "{\\mathcal N}(\\vec{x}_n;\\vec\\mu_{0k},\\Sigma_{0k})\\right)$$\n", "\n", "The derivative of $\\ln(x)$ is $\\frac{1}{x}$!! So\n", "$$\\frac{\\partial{\\mathcal L}}{\\partial\\vec\\mu_{00}}=\\sum_{n=0}^{N_0-1}\\left(\\frac{1}\n", "{\\sum_{k=0}^{K-1}c_{0k}{\\mathcal N}(\\vec{x}_n;\\vec\\mu_{0k},\\Sigma_{0k})}\\right)\n", "c_{00}\\frac{\\partial \\mathcal N(\\vec{x}_n;\\vec\\mu_{00},\\Sigma_{00})}{\\partial\\vec\\mu_{00}}$$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The derivative of a Gaussian is actually not too hard to calculate, if we just expand it out:\n", "$$\\frac{\\partial{\\mathcal N}(\\vec{x};\\vec\\mu,\\Sigma)}{\\partial\\vec\\mu}=\n", "\\frac{1}{(2\\pi)^{D/2}|\\Sigma|^{1/2}}\\frac{\\partial}{\\partial\\vec\\mu}e^{-\\frac{1}{2}d_\\Sigma^2(\\vec{x},\\vec\\mu)}$$\n", "\n", "The derivative of $e^x$ is $e^x$!! So\n", "$$\\frac{\\partial{\\mathcal N}(\\vec{x};\\vec\\mu,\\Sigma)}{\\partial\\vec\\mu}=\n", "\\frac{1}{(2\\pi)^{D/2}|\\Sigma|^{1/2}}e^{-\\frac{1}{2}d_\\Sigma^2(\\vec{x},\\vec\\mu)} \\frac{\\partial\\left(-\\frac{1}{2}d_\\Sigma^2(\\vec{x},\\vec\\mu)\\right)}{\\partial\\vec\\mu}={\\mathcal N}(\\vec{x};\\vec\\mu,\\Sigma) \\left(-\\frac{1}{2}\\frac{\\partial d_\\Sigma^2(\\vec{x},\\vec\\mu)}{\\partial\\vec\\mu_{00}}\\right)$$\n", "\n", "Remember that the Mahalanobis distance is\n", "$$d_\\Sigma^2(\\vec{x},\\vec\\mu)=(\\vec{x}-\\vec\\mu)^T\\Sigma^{-1}(\\vec{x}-\\vec\\mu)$$\n", "\n", "You might never have seen the derivative of something like this before, but you've certainly seen the scalar version:\n", "$$\\frac{\\partial}{\\partial\\mu_d}\\left(\\frac{(x_d-\\mu_d)^2}{\\sigma_d^2}\\right)=2\\frac{\\mu_d-x_d}{\\sigma_d^2}$$\n", "\n", "The vector derivative is just the same thing:\n", "$$\\frac{\\partial}{\\partial\\vec\\mu}(\\vec{x}-\\vec\\mu)^T\\Sigma^{-1}(\\vec{x}-\\vec\\mu)=\n", "2\\Sigma^{-1}(\\vec\\mu-\\vec{x}))$$\n", "\n", "So we get the very cool result that the derivative of a Gaussian is scaled by the Gaussian:\n", "$$\\frac{\\partial{\\mathcal N}(\\vec{x};\\vec\\mu,\\Sigma)}{\\partial\\vec\\mu}=\n", "{\\mathcal N}(\\vec{x};\\vec\\mu,\\Sigma)\\Sigma^{-1} (\\vec{x}-\\vec\\mu)$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Putting that all together, we have\n", "$$\\frac{\\partial{\\mathcal L}}{\\partial\\vec\\mu_{00}}=\n", "2\\sum_{n=0}^{N_0-1}\\left(\\frac{c_{00}{\\mathcal N}(\\vec{x}_n;\\vec\\mu_{00},\\Sigma_{00})}\n", "{\\sum_{k=0}^{K-1}c_{0k}{\\mathcal N}(\\vec{x}_{n};\\vec\\mu_{0k},\\Sigma_{0k})}\\right) \\Sigma_{00}^{-1}\\left(\\vec{x}_n-\\vec\\mu_{00}\\right)$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The gamma probability\n", "The part in parentheses, above, shows up over and over again in algorithms related to the EM algorithm. It is called the class-posterior, or more colloquially, the \"gamma probability\".\n", "\n", "Let's parse what it means. First, remember that the mixture weights sum to 1. So the mixture weight can be interpreted as the probability of choosing the 0'th Gaussian:\n", "$$c_0 = p_K(k=0)$$\n", "\n", "OK, then second, the normal distribution is the likelihood of $\\vec{x}$, given that we have chosen the 0'th Gaussian:\n", "$${\\mathcal N}(\\vec{x};\\vec\\mu_0,\\Sigma_0) = p_{X|K}(\\vec{x}|k=0)$$\n", "\n", "Then the summation in the denominator is kind of like a marginal pdf:\n", "$$\\sum_k c_k {\\mathcal N}(\\vec{x};\\vec\\mu_0,\\Sigma_0)=\\sum_k p_K(0)p_{X|K}(\\vec{x}|0) = p_X(\\vec{x})$$\n", "\n", "So the whole term in parentheses is like the posterior probability of choosing the 0'th Gaussian, given knowledge of the datum $\\vec{x}$:\n", "$$\\frac{c_0{\\mathcal N}(\\vec{x};\\vec\\mu_0,\\Sigma_0)}{\\sum_k c_k{\\mathcal N}(\\vec{x};\\vec\\mu_k,\\Sigma_k)}\n", "= \\frac{p_K(0)p_{X|K}(\\vec{x}|0)}{\\sum_k p_K(k)p_{X|K}(\\vec{x}|k)} = p_{K|X}(0|\\vec{x})$$\n", "\n", "This is called the \"gamma probability\" because most papers use the letter $\\gamma$ for it:\n", "$$\\gamma_0(n) = p_{K|X}(0|\\vec{x}_n) = \\frac{c_0{\\mathcal N}(\\vec{x}_n;\\vec\\mu_0,\\Sigma_0)}{\\sum_{\\ell=0}^{K-1}c_\\ell\n", "{\\mathcal N}(\\vec{x}_n;\\vec\\mu_{\\ell},\\Sigma_\\ell)}$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The maximum likelihood estimate of mu\n", "OK, from above we have\n", "$$\\frac{\\partial{\\mathcal L}}{\\partial\\vec\\mu_{00}}=\\sum_{n=0}^{N-1}\\gamma_{0}(n)\\Sigma_{00}^{-1}\\left(\\vec{x}_n-\\vec\\mu_{00}\\right)$$\n", "\n", "The maximum value of $\\vec\\mu_{00}$ is achieved for \n", "$$\\frac{\\partial{\\mathcal L}}{\\partial\\vec\\mu_{00}}=0$$\n", "\n", "Solving, we find that\n", "$$\\vec\\mu_{00}\\left(\\sum_{n=0}^{N-1}\\gamma_{0}(n)\\right) = \\sum_{n=0}^{N-1}\\gamma_{0}(n)\\vec{x}_n$$\n", "\n", "or\n", "$$\\vec\\mu_{00}=\\frac{\\sum_{n=0}^{N-1}\\gamma_0(n)\\vec{x}_n}{\\sum_{n=0}^{N-1}\\gamma_0(n)}$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### EM is just a soft-decision version of K-means\n", "Notice that EM turns out to be just a soft version of K-means. In K-means, we either assign a token to cluster $k(n)=0$, or we don't. Basically, that's like setting $\\gamma_0(n)\\in\\left\\{0,1\\right\\}$ --- a binary decision. Then the number of tokens in the 0'th cluster is\n", "$$N_{00} = \\sum_{n=0}^{N_0-1}\\gamma_0(n)$$\n", "\n", "and the average in the 0'th cluster is\n", "$$\\vec\\mu_{00}=\\frac{1}{N_{00}}\\sum_{n=0}^{N_0-1} \\gamma_0(n)\\vec{x}_n$$\n", "\n", "The only difference between K-means and EM is that, for EM, we have soft decisions instead of hard decisions. Because Gaussians are never exactly zero, the EM algorithm has\n", "$$0 < \\gamma_0(n)< 1$$\n", "\n", "... Each training token is not allocated to just one cluster. Instead, each training token is given a soft assignment to the different clusters, $\\gamma_k(n)$ is the degree to which token $n$ is allocated to cluster $k$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ... but EM is not closed form\n", "Notice that the EM re-estimation equation for $\\vec\\mu_{00}$ depends on $\\gamma_0(n)$. But $\\gamma_0(n)$, in turn, depends on $\\vec\\mu_{00}$. So we have an iterative two-step solution, very similar to K-means:\n", "* The E-step: calculate $\\gamma_k(n)$ for every training token $n$, and for every Gaussian cluster $k$\n", "* The M-step: re-estimate $\\vec\\mu_{yk}$, $\\Sigma_{yk}$, and $c_{yk}$ for every class $y$ and for every Gaussian cluster $k$\n", "\n", "The two steps, above, are iterated until the log-likelihood, ${\\mathcal L}$, stops changing." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Formulas for the M-step\n", "I gave you, already, the formula for re-estimating $\\vec\\mu_{00}$. The formulas for $\\Sigma_{00}$ and $c_{00}$ are similar. They are:\n", "$$c_{yk} = \\frac{\\sum_{n=0}^{N_y-1} \\gamma_k(n)}{N_y}$$\n", "\n", "and\n", "$$\\vec\\mu_{yk} = \\frac{\\sum_{n=0}^{N_y-1}\\gamma_k(n)\\vec{x}_n}{\\sum_{n=0}^{N_y-1}\\gamma_k(n)}$$\n", "\n", "and\n", "$$\\Sigma_{yk} = \\frac{\\sum_{n=0}^{N_y-1}\\gamma_k(n)(\\vec{x}_n-\\vec\\mu_{yk})(\\vec{x}_n-\\vec\\mu_{yk})^T}{\\sum_{n=0}^{N_y-1}\\gamma_k(n)}$$\n", "\n", "all three are basically a kind of weighted average, where the weighting factor is the gamma-probability calculated in the E-step." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Formula for the E-step\n", "$$\\gamma_k(n) = \\frac{c_{yk}{\\mathcal N}(\\vec{x}_n;\\vec\\mu_{yk},\\Sigma_{yk})}{\\sum_{\\ell=0}^{K-1}c_{y\\ell}\n", "{\\mathcal N}(\\vec{x}_n;\\vec\\mu_{y\\ell},\\Sigma_{y\\ell})}$$" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.1" } }, "nbformat": 4, "nbformat_minor": 2 }