{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ECE 417 Lecture 12: Mixture Gaussian Models\n", "## Mark Hasegawa-Johnson, October 10, 2017\n", "This file is distributed under a CC-BY license. You may freely re-use or re-distribute the whole or any part. If you re-distribute a non-trivial portion of it, give me credit." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Outline of Today's lecture\n", "* Jupyter notebook\n", "* Discriminant-based classifier; Bayesian classifier\n", "* Downloading some example data\n", "* Likelihood models: Gaussian\n", "* Gaussian classifier\n", "* Likelihood models: Mixture Gaussian\n", "* Mixture Gaussian classifier" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Jupyter Notebook\n", "* A Jupyter notebook is a document with two types of cells: code cells, and markdown cells. \n", "* Code cells contain python code. When you click \"play\" (or hit Shift-Enter), the code runs. Results are shown immediately after the cell. Results might include printed text, or matplotlib plots.\n", "* Markdown cells, like this one, contain text. Text is formatted using wiki-style markdown.\n", "* The whole notebook is available on the course web page. I strongly encourage you to download it, edit the code snippets, and re-run them, in order to get a better understanding of the lecture material." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first thing to do is to load all of the standard python libraries that we'll need." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import scipy.stats as stats\n", "import requests\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Discriminant-based classifiers: all classifiers can be written this way\n", "* A classifier reads in a sample, $\\vec{x}$, and tries to determine what label to give it. Let's assume that $\\vec{x}$ is a\n", "$D$-dimensional real vector, so we can write $$\\vec{x}\\in\\Re^D$$.\n", "* Usually the label is an integer, $y$. For example, if there are $C$ different classes, then we can say that $$y\\in\\left\\{0,1,\\ldots,C-1\\right\\}$$.\n", "* Any classifier can be summarized by a set of discriminant function $g_y(\\vec{x})$. The The discriminant function $g_y(\\vec{x})$ takes a datum, and produces a real-valued score.\n", "* Given a particular input $\\vec{x}$, the classifier tries each discriminant function, one after the other,\n", "and outputs whichever label has the best score.\n", "$$\\hat{y}=\\arg\\max_y g_y(\\vec{x})$$\n", "where the notation $\\hat{y}$ means \"the value of y chosen by the classifier.\"\n", "* So, for example, suppose that there are only two classes: $y=0$ and $y=1$. Then the classifier would have the \n", "following decision rule:\n", "$$\\hat{y}=\\left\\{\\begin{array}{ll} 1 & g_1(\\vec{x}) > g_0(\\vec{x})\\\\\n", " 0 & \\mbox{otherwise}\\end{array}\\right.$$\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Bayesian classifiers: If the discriminant is a monotonic function of p(y|x)\n", "A Bayesian classifier (also known as a MAP classifier, also known as a minimum-probability-of-error or MPE classifier) is one\n", "for which the discriminant function is the posterior probability of $y$ given $\\vec{x}$:\n", "$$\\mbox{one possibility:}~~~g_y(\\vec{x}) = p_{Y|X}(y|\\vec{x})$$\n", "... or any monotonic function of that. \n", "\n", "For example, from the definition of conditional probability, we get\n", "$$\\hat{y}=\\arg\\max_y p_{Y|X}(y|\\vec{x}) = \\arg\\max_y\\frac{p_{X,Y}(\\vec{x},y)}{p_X(\\vec{x})}\n", "=\\arg\\max_y p_{X,Y}(\\vec{x},y)$$\n", "so instead of using the posterior probability as the discriminant function, we could use the joint probability\n", "\n", "$$\\mbox{another possibility that gives the same result:}~~~g_y(\\vec{x})=p_{X,Y}(\\vec{x},y)$$\n", "\n", "On the other hand, logarithm is a monotonic function, so\n", "$$\\hat{y}=\\arg\\max_y p_{X,Y}(\\vec{x},y) = \\arg\\max_y \\ln p_{X,Y}(\\vec{x},y)$$\n", "so we could use\n", "$$\\mbox{yet another possibility that gives the same result:}~~~g_y(\\vec{x})=\\ln p_{X,Y}(\\vec{x},y)$$\n", "\n", "It's particularly useful to divide that into the prior probability $p_Y(y)$, and the likelihood $p_{X|Y}(\\vec{x}|y)$. The reason it's useful: because $p_Y(y)$ is just a lookup table, and $p_{X|Y}(\\vec{x}|y)$ is often much, much easier to learn from data than is $p_{Y|X}(y|\\vec{x})$. Thus we often want to use this discriminant:\n", "\n", "$$\\mbox{This one usually is easiest to compute and use:}~~~g_y(\\vec{x})=\\ln p_Y(y) + \\ln p_{X|Y}(\\vec{x}|y)$$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# Bayesian classifiers: terminology\n", "You need to know these four terms. If $\\vec{x}$ is the observation and $y$ is the class label, then\n", "* Posterior: $p_{Y|X}(y|\\vec{x})$\n", "* Likelihood: $p_{X|Y}(\\vec{x}|y)$\n", "* Prior: $p_Y(y)$\n", "* Evidence: $p_X(\\vec{x})$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Downloading some example data\n", "So let's talk about how to estimate the likelihood function. First, let's get some data. Here I'll download the \"Iris\" dataset, which is the one that Fisher used in the paper where he invented linear discriminant analysis." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['5.1', '3.5', '1.4', '0.2', 'Iris-setosa'],\n", " ['4.9', '3.0', '1.4', '0.2', 'Iris-setosa'],\n", " ['4.7', '3.2', '1.3', '0.2', 'Iris-setosa'],\n", " ['4.6', '3.1', '1.5', '0.2', 'Iris-setosa'],\n", " ['5.0', '3.6', '1.4', '0.2', 'Iris-setosa'],\n", " ['5.4', '3.9', '1.7', '0.4', 'Iris-setosa'],\n", " ['4.6', '3.4', '1.4', '0.3', 'Iris-setosa'],\n", " ['5.0', '3.4', '1.5', '0.2', 'Iris-setosa'],\n", " ['4.4', '2.9', '1.4', '0.2', 'Iris-setosa'],\n", " ['4.9', '3.1', '1.5', '0.1', 'Iris-setosa'],\n", " ['5.4', '3.7', '1.5', '0.2', 'Iris-setosa'],\n", " ['4.8', '3.4', '1.6', '0.2', 'Iris-setosa'],\n", " ['4.8', '3.0', '1.4', '0.1', 'Iris-setosa'],\n", " ['4.3', '3.0', '1.1', '0.1', 'Iris-setosa'],\n", " ['5.8', '4.0', '1.2', '0.2', 'Iris-setosa'],\n", " ['5.7', '4.4', '1.5', '0.4', 'Iris-setosa'],\n", " ['5.4', '3.9', '1.3', '0.4', 'Iris-setosa'],\n", " ['5.1', '3.5', '1.4', '0.3', 'Iris-setosa'],\n", " ['5.7', '3.8', '1.7', '0.3', 'Iris-setosa'],\n", " ['5.1', '3.8', '1.5', '0.3', 'Iris-setosa'],\n", " ['5.4', '3.4', '1.7', '0.2', 'Iris-setosa'],\n", " ['5.1', '3.7', '1.5', '0.4', 'Iris-setosa'],\n", " ['4.6', '3.6', '1.0', '0.2', 'Iris-setosa'],\n", " ['5.1', '3.3', '1.7', '0.5', 'Iris-setosa'],\n", " ['4.8', '3.4', '1.9', '0.2', 'Iris-setosa'],\n", " ['5.0', '3.0', '1.6', '0.2', 'Iris-setosa'],\n", " ['5.0', '3.4', '1.6', '0.4', 'Iris-setosa'],\n", " ['5.2', '3.5', '1.5', '0.2', 'Iris-setosa'],\n", " ['5.2', '3.4', '1.4', '0.2', 'Iris-setosa'],\n", " ['4.7', '3.2', '1.6', '0.2', 'Iris-setosa'],\n", " ['4.8', '3.1', '1.6', '0.2', 'Iris-setosa'],\n", " ['5.4', '3.4', '1.5', '0.4', 'Iris-setosa'],\n", " ['5.2', '4.1', '1.5', '0.1', 'Iris-setosa'],\n", " ['5.5', '4.2', '1.4', '0.2', 'Iris-setosa'],\n", " ['4.9', '3.1', '1.5', '0.1', 'Iris-setosa'],\n", " ['5.0', '3.2', '1.2', '0.2', 'Iris-setosa'],\n", " ['5.5', '3.5', '1.3', '0.2', 'Iris-setosa'],\n", " ['4.9', '3.1', '1.5', '0.1', 'Iris-setosa'],\n", " ['4.4', '3.0', '1.3', '0.2', 'Iris-setosa'],\n", " ['5.1', '3.4', '1.5', '0.2', 'Iris-setosa'],\n", " ['5.0', '3.5', '1.3', '0.3', 'Iris-setosa'],\n", " ['4.5', '2.3', '1.3', '0.3', 'Iris-setosa'],\n", " ['4.4', '3.2', '1.3', '0.2', 'Iris-setosa'],\n", " ['5.0', '3.5', '1.6', '0.6', 'Iris-setosa'],\n", " ['5.1', '3.8', '1.9', '0.4', 'Iris-setosa'],\n", " ['4.8', '3.0', '1.4', '0.3', 'Iris-setosa'],\n", " ['5.1', '3.8', '1.6', '0.2', 'Iris-setosa'],\n", " ['4.6', '3.2', '1.4', '0.2', 'Iris-setosa'],\n", " ['5.3', '3.7', '1.5', '0.2', 'Iris-setosa'],\n", " ['5.0', '3.3', '1.4', '0.2', 'Iris-setosa'],\n", " ['7.0', '3.2', '4.7', '1.4', 'Iris-versicolor'],\n", " ['6.4', '3.2', '4.5', '1.5', 'Iris-versicolor'],\n", " ['6.9', '3.1', '4.9', '1.5', 'Iris-versicolor'],\n", " ['5.5', '2.3', '4.0', '1.3', 'Iris-versicolor'],\n", " ['6.5', '2.8', '4.6', '1.5', 'Iris-versicolor'],\n", " ['5.7', '2.8', '4.5', '1.3', 'Iris-versicolor'],\n", " ['6.3', '3.3', '4.7', '1.6', 'Iris-versicolor'],\n", " ['4.9', '2.4', '3.3', '1.0', 'Iris-versicolor'],\n", " ['6.6', '2.9', '4.6', '1.3', 'Iris-versicolor'],\n", " ['5.2', '2.7', '3.9', '1.4', 'Iris-versicolor'],\n", " ['5.0', '2.0', '3.5', '1.0', 'Iris-versicolor'],\n", " ['5.9', '3.0', '4.2', '1.5', 'Iris-versicolor'],\n", " ['6.0', '2.2', '4.0', '1.0', 'Iris-versicolor'],\n", " ['6.1', '2.9', '4.7', '1.4', 'Iris-versicolor'],\n", " ['5.6', '2.9', '3.6', '1.3', 'Iris-versicolor'],\n", " ['6.7', '3.1', '4.4', '1.4', 'Iris-versicolor'],\n", " ['5.6', '3.0', '4.5', '1.5', 'Iris-versicolor'],\n", " ['5.8', '2.7', '4.1', '1.0', 'Iris-versicolor'],\n", " ['6.2', '2.2', '4.5', '1.5', 'Iris-versicolor'],\n", " ['5.6', '2.5', '3.9', '1.1', 'Iris-versicolor'],\n", " ['5.9', '3.2', '4.8', '1.8', 'Iris-versicolor'],\n", " ['6.1', '2.8', '4.0', '1.3', 'Iris-versicolor'],\n", " ['6.3', '2.5', '4.9', '1.5', 'Iris-versicolor'],\n", " ['6.1', '2.8', '4.7', '1.2', 'Iris-versicolor'],\n", " ['6.4', '2.9', '4.3', '1.3', 'Iris-versicolor'],\n", " ['6.6', '3.0', '4.4', '1.4', 'Iris-versicolor'],\n", " ['6.8', '2.8', '4.8', '1.4', 'Iris-versicolor'],\n", " ['6.7', '3.0', '5.0', '1.7', 'Iris-versicolor'],\n", " ['6.0', '2.9', '4.5', '1.5', 'Iris-versicolor'],\n", " ['5.7', '2.6', '3.5', '1.0', 'Iris-versicolor'],\n", " ['5.5', '2.4', '3.8', '1.1', 'Iris-versicolor'],\n", " ['5.5', '2.4', '3.7', '1.0', 'Iris-versicolor'],\n", " ['5.8', '2.7', '3.9', '1.2', 'Iris-versicolor'],\n", " ['6.0', '2.7', '5.1', '1.6', 'Iris-versicolor'],\n", " ['5.4', '3.0', '4.5', '1.5', 'Iris-versicolor'],\n", " ['6.0', '3.4', '4.5', '1.6', 'Iris-versicolor'],\n", " ['6.7', '3.1', '4.7', '1.5', 'Iris-versicolor'],\n", " ['6.3', '2.3', '4.4', '1.3', 'Iris-versicolor'],\n", " ['5.6', '3.0', '4.1', '1.3', 'Iris-versicolor'],\n", " ['5.5', '2.5', '4.0', '1.3', 'Iris-versicolor'],\n", " ['5.5', '2.6', '4.4', '1.2', 'Iris-versicolor'],\n", " ['6.1', '3.0', '4.6', '1.4', 'Iris-versicolor'],\n", " ['5.8', '2.6', '4.0', '1.2', 'Iris-versicolor'],\n", " ['5.0', '2.3', '3.3', '1.0', 'Iris-versicolor'],\n", " ['5.6', '2.7', '4.2', '1.3', 'Iris-versicolor'],\n", " ['5.7', '3.0', '4.2', '1.2', 'Iris-versicolor'],\n", " ['5.7', '2.9', '4.2', '1.3', 'Iris-versicolor'],\n", " ['6.2', '2.9', '4.3', '1.3', 'Iris-versicolor'],\n", " ['5.1', '2.5', '3.0', '1.1', 'Iris-versicolor'],\n", " ['5.7', '2.8', '4.1', '1.3', 'Iris-versicolor'],\n", " ['6.3', '3.3', '6.0', '2.5', 'Iris-virginica'],\n", " ['5.8', '2.7', '5.1', '1.9', 'Iris-virginica'],\n", " ['7.1', '3.0', '5.9', '2.1', 'Iris-virginica'],\n", " ['6.3', '2.9', '5.6', '1.8', 'Iris-virginica'],\n", " ['6.5', '3.0', '5.8', '2.2', 'Iris-virginica'],\n", " ['7.6', '3.0', '6.6', '2.1', 'Iris-virginica'],\n", " ['4.9', '2.5', '4.5', '1.7', 'Iris-virginica'],\n", " ['7.3', '2.9', '6.3', '1.8', 'Iris-virginica'],\n", " ['6.7', '2.5', '5.8', '1.8', 'Iris-virginica'],\n", " ['7.2', '3.6', '6.1', '2.5', 'Iris-virginica'],\n", " ['6.5', '3.2', '5.1', '2.0', 'Iris-virginica'],\n", " ['6.4', '2.7', '5.3', '1.9', 'Iris-virginica'],\n", " ['6.8', '3.0', '5.5', '2.1', 'Iris-virginica'],\n", " ['5.7', '2.5', '5.0', '2.0', 'Iris-virginica'],\n", " ['5.8', '2.8', '5.1', '2.4', 'Iris-virginica'],\n", " ['6.4', '3.2', '5.3', '2.3', 'Iris-virginica'],\n", " ['6.5', '3.0', '5.5', '1.8', 'Iris-virginica'],\n", " ['7.7', '3.8', '6.7', '2.2', 'Iris-virginica'],\n", " ['7.7', '2.6', '6.9', '2.3', 'Iris-virginica'],\n", " ['6.0', '2.2', '5.0', '1.5', 'Iris-virginica'],\n", " ['6.9', '3.2', '5.7', '2.3', 'Iris-virginica'],\n", " ['5.6', '2.8', '4.9', '2.0', 'Iris-virginica'],\n", " ['7.7', '2.8', '6.7', '2.0', 'Iris-virginica'],\n", " ['6.3', '2.7', '4.9', '1.8', 'Iris-virginica'],\n", " ['6.7', '3.3', '5.7', '2.1', 'Iris-virginica'],\n", " ['7.2', '3.2', '6.0', '1.8', 'Iris-virginica'],\n", " ['6.2', '2.8', '4.8', '1.8', 'Iris-virginica'],\n", " ['6.1', '3.0', '4.9', '1.8', 'Iris-virginica'],\n", " ['6.4', '2.8', '5.6', '2.1', 'Iris-virginica'],\n", " ['7.2', '3.0', '5.8', '1.6', 'Iris-virginica'],\n", " ['7.4', '2.8', '6.1', '1.9', 'Iris-virginica'],\n", " ['7.9', '3.8', '6.4', '2.0', 'Iris-virginica'],\n", " ['6.4', '2.8', '5.6', '2.2', 'Iris-virginica'],\n", " ['6.3', '2.8', '5.1', '1.5', 'Iris-virginica'],\n", " ['6.1', '2.6', '5.6', '1.4', 'Iris-virginica'],\n", " ['7.7', '3.0', '6.1', '2.3', 'Iris-virginica'],\n", " ['6.3', '3.4', '5.6', '2.4', 'Iris-virginica'],\n", " ['6.4', '3.1', '5.5', '1.8', 'Iris-virginica'],\n", " ['6.0', '3.0', '4.8', '1.8', 'Iris-virginica'],\n", " ['6.9', '3.1', '5.4', '2.1', 'Iris-virginica'],\n", " ['6.7', '3.1', '5.6', '2.4', 'Iris-virginica'],\n", " ['6.9', '3.1', '5.1', '2.3', 'Iris-virginica'],\n", " ['5.8', '2.7', '5.1', '1.9', 'Iris-virginica'],\n", " ['6.8', '3.2', '5.9', '2.3', 'Iris-virginica'],\n", " ['6.7', '3.3', '5.7', '2.5', 'Iris-virginica'],\n", " ['6.7', '3.0', '5.2', '2.3', 'Iris-virginica'],\n", " ['6.3', '2.5', '5.0', '1.9', 'Iris-virginica'],\n", " ['6.5', '3.0', '5.2', '2.0', 'Iris-virginica'],\n", " ['6.2', '3.4', '5.4', '2.3', 'Iris-virginica'],\n", " ['5.9', '3.0', '5.1', '1.8', 'Iris-virginica'],\n", " [''],\n", " ['']]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Download data from the UCI repository\n", "r=requests.get('http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data')\n", "# Split at every newline; split each line at commas\n", "dataset = [ x.split(',') for x in r.text.split('\\n') ]\n", "dataset\n" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 63, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEWCAYAAAB1xKBvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH3ZJREFUeJzt3Xu4XVV97vHvaxKuIiiJqIQQKqkaKESaBkSqUJEmgI2e\nqg1FtCqmWDhHK9XGeve053Bqa60FjalGwGqorWBTCXJRD1gtmEAjIVw0jVESwIT7zQKBt3/MEZks\n9mXsZK+sBfv9PM969pzjMudvrb33+q15WWPINhEREcN5Rq8DiIiIp4YkjIiIqJKEERERVZIwIiKi\nShJGRERUScKIiIgqSRjxJJI+KukftuP+LGn/inZTS9vxW7GPre47zHbfKennku6XtOcI+54t6c/L\n8pGS1m9lDHtJukLSfZL+urLPOklHb83+tlXt7zv6TxLGGCXp9yWtKG90t0q6SNIRvY7rqUTSBOCT\nwDG2n2n7jh6FMh+4HXiW7dM7K9uJqRuSAMaOJIwxSNJ7gE8B/wfYC5gCnAX8Ti/jegraC9gJWN3j\nOPYFrne+hRtdloQxxkjaHfg4cKrt820/YPsR29+w/b5B+vyTpNsk3VNOfRzQqjtW0vXldMgGSX9S\nyidK+oakuyXdKem7kob9e5N0nKT/kHSvpJslfXSAZm+TdEs5MvqTVt9nSFog6T8l3SHpq5KeM8h+\n/kDS2hL3TySdOEi7HSV9quzvlrK8o6RfBW4qze6W9O2RvnYjIelwScvLdpZLOryUnw28BXhfOVo8\nuqPffODEVv2/tqpnSLq2bPMfJe3U6ne8pJXl9/d9SQcNEtcVZfGHZfu/V8rfIWlN+d0vlfSCQfof\nUX7PR5b1F0u6tPS7SdIbW23PlnSWpAvL7+0qSS8sdZL0N5I2lr+dVZIOHMlrHBVs5zGGHsBsYDMw\nfog2HwX+obX+NmA3YEeaI5OVrbpbgd8sy88GDinL/xdYCEwoj98ENMj+DOxflo8Efo3mw8xBwM+B\n15a6qaXtEmDX0m4TcHSpfxdwJTC5xPo5YElH3/Gl773Ai0rd84EDBont42WbzwUmAd8H/nfnNod4\nLYd67c4G/rz1vNcPso3nAHcBJ5X4Tyjre3ZuZ5D+T6oH1gE/AF5Qtn8DcEqpeymwETgUGEeTkNYB\nOw73+yvrv0VziuyQ8rz/Driisz3N3+LNwKxSvmtZf2t5ni8t25neeh53ALNK/ZeB80rdbwNXA3sA\nAl4CPL/X/29Pt0eOMMaePYHbbW+u7WB7se37bD9Ek0wOLkcqAI8A0yU9y/Zdtq9plT8f2NfNEcx3\nXf6zh9nX/7e9yvZjtq+lSQ6v7Gj2MTdHRquAL9K8gQKcAnzA9vpWrK/XwBe6HwMOlLSz7VttD3Za\n6UTg47Y32t4EfIzmjbvKMK9dreOAH9v+ku3NtpcANwKvGeF2On3a9i227wT+FZhRyucDn7N9le1H\nbZ8DPAQcVrndE4HFtq8pz/v9wMskTW21eQNNQp9j+wel7Hhgne0vluf5H8DXStstLrD9g/L3++VW\nzI/QJOYX03wwucH2rbUvRNRJwhh77gAmDvIm+iSSxkk6o5zmuZfmkybAxPLzd4FjgZ9KulzSy0r5\nJ4A1wCXl1M+Cyv0dKuk7kjZJuocmCUzsaHZza/mnNJ+SoTmXf0E5jXI3zafmR2muNfyS7QeA3yvb\nvrWc4njxICG9oOxjoP0N91yGe+1qdcawJY69R7idTre1lh8EnlmW9wVO3/I6ltdyHyqfNx3x2r6f\n5u+uHe+7ga/avq5Vti9waMd+TwSeN1zMtr8NnElzLW6jpEWSnlUZb1RKwhh7/p3m0+JrK9v/PjAX\nOBrYneY0DDSH/dhebnsuzSmbrwNfLeX32T7d9q/QXEx/j6RXVezvK8BSYB/bu9Oc1lJHm31ay1OA\nW8ryzTSfWPdoPXayvaFzJ7Yvtv1qmqOgG4G/HySeW2jeyAba33CGfO1GoDOGLXE86XkNYqQXw28G\n/qLjddylHNnUeEK8knalObJtx/sG4LWS3tWx38s79vtM2++s2antT9v+dWA68KvAeyvjjUpJGGOM\n7XuADwNnSXqtpF0kTZA0R9JfDtBlN5oEcwewC82dVQBI2kHSiZJ2t/0IzXWBx0rd8ZL2lyTgHppP\n+o9VhLgbcKft/5I0i+ZNt9OHStwH0Jzv/sdSvhD4C0n7lhgmSZrb2VnN9xbmljeyh4D7h4htCfDB\nsq2JNK9d7XdUBn3tRmgZ8KtqboUeXy4sTwe+Udn/58CvjGB/fw+cUo72JGlXNTcj7Fa5/SXAWyXN\nkLQjzfO+yva6VptbgFcB75K0JSF8g+Z5nlT+JidI+g1JLxku4NLuUDW3Oj8A/Bd1f28xAkkYY5Dt\nvwbeA3yQ5qLxzcBpNEcInc6lOb2wAbie5gJw20nAunLK5RSaUwgA04DLaN6M/x34jO3vVIT3R8DH\nJd1H8+b81QHaXE5zuutbwF/ZvqSU/y3N0cklpf+VNBduOz2D5vnfAtxJc41ksE+xfw6sAK4FVgHX\nlLIaw712Vdx8v+N44HSa5PM+4Hjbt1du4gs015nuljTQ77hzfyuAd9Cc4rmL5rX+gyG6fBQ4p2z/\njbYvAz5Ec/3hVuCFwLwB9vMzmqSxQNLJtu8Djiltb6E5/fT/aC6cD+dZNInuLprX/A6a06IxilRx\nHTIiIiJHGBERUScJIyIiqiRhRERElSSMiIioMqpDPffaxIkTPXXq1F6HERHxlHH11VffbntSTdun\nVcKYOnUqK1as6HUYERFPGZI6RxEYVE5JRURElSSMiIiokoQRERFVkjAiIqJKEkZERFRJwoiIiCpd\nSxiS9ikT4VwvaXXHuPdb2kjSp9XM/XutpENadbPLnL5raiffiYiI7unmEcZm4HTb02mmdjxV0vSO\nNnNohsGeRjMt5GehmamMZuasOTTj/p8wQN+IiNiOupYwyjzJ15Tl+2imy+ycUnIucK4bVwJ7SHo+\nzSTva2yvtf0wcF5pGxERPbJdvuldJn9/KXBVR9XePHF+5vWlbKDygSbCQdJ8mqMTpkyZMirxRsTg\npi64sNchALDujOOYuuDCrv/sZ+1Yt4euX/SW9Eyambfebfve0d6+7UW2Z9qeOWlS1XAoERGxFbp6\nhFHm1/0a8GXb5w/QZAOwT2t9cimbMEh5RET0SDfvkhLNXMI32P7kIM2WAm8ud0sdBtxj+1ZgOTBN\n0n6SdqCZ43dpt2KNiIjhdfMI4+XAScAqSStL2Z8BUwBsLwSWAcfSTDL/IPDWUrdZ0mnAxcA4YLHt\n1V2MNSIihtG1hGH73wAN08bAqYPULaNJKBER0QfyTe+IiKiShBEREVWSMCIiokoSRkREVEnCiIiI\nKkkYERFRJQkjIiKqJGFERESVJIyIiKiShBEREVWSMCIiokoSRkREVEnCiIiIKkkYERFRJQkjIiKq\nJGFERESVrk2gJGkxcDyw0faBA9S/FzixFcdLgEm275S0DrgPeBTYbHtmt+KMiIg63TzCOBuYPVil\n7U/YnmF7BvB+4HLbd7aaHFXqkywiIvpA1xKG7SuAO4dt2DgBWNKtWCIiYtv1/BqGpF1ojkS+1io2\ncJmkqyXN701kERHR1rVrGCPwGuB7HaejjrC9QdJzgUsl3ViOWJ6kJJT5AFOmTOl+tBERY1TPjzCA\neXScjrK9ofzcCFwAzBqss+1Ftmfanjlp0qSuBhoRMZb1NGFI2h14JfAvrbJdJe22ZRk4BriuNxFG\nRMQW3bytdglwJDBR0nrgI8AEANsLS7PXAZfYfqDVdS/gAklb4vuK7W92K86IiKjTtYRh+4SKNmfT\n3H7bLlsLHNydqCIiYmv1wzWMiIh4CkjCiIiIKkkYERFRJQkjIiKqJGFERESVJIyIiKiShBEREVWS\nMCIiokoSRkREVEnCiIiIKkkYERFRJQkjIiKqJGFERESVJIyIiKiShBEREVWSMCIiokoSRkREVOla\nwpC0WNJGSQPOxy3pSEn3SFpZHh9u1c2WdJOkNZIWdCvGiIio180jjLOB2cO0+a7tGeXxcQBJ44Cz\ngDnAdOAESdO7GGdERFToWsKwfQVw51Z0nQWssb3W9sPAecDcUQ0uIiJGrNfXMA6XdK2kiyQdUMr2\nBm5utVlfygYkab6kFZJWbNq0qZuxRkSMab1MGNcAU2wfBPwd8PWt2YjtRbZn2p45adKkUQ0wIiIe\n17OEYfte2/eX5WXABEkTgQ3APq2mk0tZRET0UM8ShqTnSVJZnlViuQNYDkyTtJ+kHYB5wNJexRkR\nEY3x3dqwpCXAkcBESeuBjwATAGwvBF4PvFPSZuAXwDzbBjZLOg24GBgHLLa9ultxRkREna4lDNsn\nDFN/JnDmIHXLgGXdiCsiIrZOr++SioiIp4gkjIiIqJKEERERVZIwIiKiShJGRERUScKIiIgqSRgR\nEVElCSMiIqokYURERJUkjIiIqJKEERERVZIwIiKiShJGRERUScKIiIgqSRgREVElCSMiIqokYURE\nRJVBZ9yTtArwQFWAbR801IYlLQaOBzbaPnCA+hOBPy3buw94p+0flrp1pexRYLPtmVXPJiIiumao\nKVqP38Ztn00zBeu5g9T/BHil7bskzQEWAYe26o+yffs2xhAREaNk0IRh+6dbliXtC0yzfZmknYfq\n1+p/haSpQ9R/v7V6JTC5JuCIiOiNYa9hSHoH8M/A50rRZODroxzH24GLWusGLpN0taT5w8Q3X9IK\nSSs2bdo0ymFFRMQWNRe9TwVeDtwLYPvHwHNHKwBJR9EkjD9tFR9hewYwBzhV0isG6297ke2ZtmdO\nmjRptMKKiIgONQnjIdsPb1mRNJ6BL4aPmKSDgM8Dc23fsaXc9obycyNwATBrNPYXERFbryZhXC7p\nz4CdJb0a+CfgX7d1x5KmAOcDJ9n+Uat8V0m7bVkGjgGu29b9RUTEthn24jWwgOaU0SrgD4FlNEcF\nQ5K0BDgSmChpPfARYAKA7YXAh4E9gc9Igsdvn90LuKCUjQe+YvubI3pWEREx6mrudnpM0jnAVTSn\nom6yPewpKdsnDFN/MnDyAOVrgYOH235ERGxfwyYMSccBC4H/pPmS3X6S/tD2RUP3jIiIp5OaU1J/\nTfMlujUAkl4IXMgTb4ONiIinuZqL3vdtSRbFWpphOyIiYgwZaiyp/1EWV0haBnyV5hrGG4Dl2yG2\niIjoI0OdknpNa/nnwCvL8iZg565FFBERfWmosaTeuj0DiYiI/lZzl9RONN/DOADYaUu57bd1Ma6I\niOgzNRe9vwQ8D/ht4HKawQdz0TsiYoypSRj72/4Q8IDtc4DjeOK8FRERMQbUJIxHys+7JR0I7M4o\njlYbERFPDTVf3Fsk6dnAh4ClwDNpxoGKiIgxpGYsqS0DDV4O/Ep3w4mIiH411Bf33jNUR9ufHP1w\nIiKiXw11hLHbdosiIiL63lBf3PvY9gwkIiL6W81dUhEREUkYERFRp2sJQ9JiSRslDTgftxqflrRG\n0rWSDmnVzZZ0U6lb0K0YIyKi3rAJQ9K7JD2rvMF/QdI1ko6p2PbZwOwh6ucA08pjPvDZsr9xwFml\nfjpwgqTpFfuLiIguqjnCeJvte4FjgGcDJwFnDNfJ9hXAnUM0mQuc68aVwB6Sng/MAtbYXmv7YeC8\n0jYiInqo5pveKj+PBb5ke7UkDdWh0t7Aza319aVsoPJBx66SNJ/mCIUpU6aMQljRL6YuuHDI+nVn\nHMfUBRd2/Wc/G8vPPba/miOMqyVdQpMwLpa0G/BYd8OqZ3uR7Zm2Z06aNKnX4UREPG3VHGG8HZgB\nrLX9oKTnAKMxudIGYJ/W+uRSNmGQ8oiI6KGaI4yXATfZvlvSm4APAveMwr6XAm8uF9MPA+6xfSvN\nfOHTJO0naQdgXmkbERE9VHOE8VngYEkHA6cDnwfO5fE5vgckaQlwJDBR0nrgIzRHD9heCCyjOc21\nBniQctRie7Ok04CLgXHAYturR/zMIiJiVNUkjM22LWkucKbtL0h6+3CdbJ8wTL2BUwepW0aTUCIi\nok/UJIz7JL0feBPwCknPoBwpRETE2FFzDeP3gIeAt9u+jeYi9Ce6GlVERPSdmgmUbgM+2Vr/Gc01\njIiIGENqhgY5TNJySfdLeljSo5JG4y6piIh4Cqk5JXUmcALwY2Bn4GTgM90MKiIi+k/VaLW21wDj\nbD9q+4sMPahgREQ8DdXcJfVg+QLdSkl/CdxK5tGIiBhzat74T6L5At1pwAM0w3b8bjeDioiI/lNz\nl9RPy+IvgMzzHRExRg2aMCStAjxYve2DuhJRRET0paGOMI7fblFERETfGyphTAD2sv29dqGklwO3\ndTWqiIjoO0Nd9P4UcO8A5feWuoiIGEOGShh72V7VWVjKpnYtooiI6EtDJYw9hqjbebQDiYiI/jZU\nwlgh6R2dhZJOBq7uXkgREdGPhrro/W7gAkkn8niCmAnsALyu24FFRER/GTRh2P45cLiko4ADS/GF\ntr9du3FJs4G/pfmm+Odtn9FR/17gxFYsLwEm2b5T0jrgPuBRmln/ZtbuNyIiRl/NN72/A3xnpBuW\nNA44C3g1sB5YLmmp7etb2/4EZTImSa8B/tj2na3NHGX79pHuOyIiRl83BxGcBayxvdb2w8B5wNwh\n2p8ALOliPBERsQ26mTD2Bm5ura8vZU8iaReaIdO/1io2cJmkqyXNH2wnkuZLWiFpxaZNm0Yh7IiI\nGEi/DFP+GuB7HaejjrA9A5gDnCrpFQN1tL3I9kzbMydNmrQ9Yo2IGJO6mTA20AyFvsXkUjaQeXSc\njrK9ofzcCFxAc4orIiJ6pJsJYzkwTdJ+ZQKmecDSzkaSdgdeCfxLq2xXSbttWQaOAa7rYqwRETGM\nmhn3tortzZJOAy6mua12se3Vkk4p9QtL09cBl9h+oNV9L5rvgGyJ8Su2v9mtWCMiYnhdSxgAtpcB\nyzrKFnasnw2c3VG2Fji4m7FFRMTI9MtF74iI6HNJGBERUSUJIyIiqiRhRERElSSMiIiokoQRERFV\nkjAiIqJKEkZERFRJwoiIiCpJGBERUSUJIyIiqiRhRERElSSMiIiokoQRERFVkjAiIqJKEkZERFRJ\nwoiIiCpdTRiSZku6SdIaSQsGqD9S0j2SVpbHh2v7RkTE9tW1KVoljQPOAl4NrAeWS1pq+/qOpt+1\nffxW9o2IiO2km0cYs4A1ttfafhg4D5i7HfpGREQXdDNh7A3c3FpfX8o6HS7pWkkXSTpghH2RNF/S\nCkkrNm3aNBpxR0TEAHp90fsaYIrtg4C/A74+0g3YXmR7pu2ZkyZNGvUAIyKi0c2EsQHYp7U+uZT9\nku17bd9flpcBEyRNrOkbERHbVzcTxnJgmqT9JO0AzAOWthtIep4kleVZJZ47avpGRMT21bW7pGxv\nlnQacDEwDlhse7WkU0r9QuD1wDslbQZ+AcyzbWDAvt2KNSIihte1hAG/PM20rKNsYWv5TODM2r4R\nEdE7vb7oHRERTxFJGBERUSUJIyIiqiRhRERElSSMiIiokoQRERFVkjAiIqJKEkZERFRJwoiIiCpJ\nGBERUSUJIyIiqiRhRERElSSMiIiokoQRERFVkjAiIqJKEkZERFRJwoiIiCpdTRiSZku6SdIaSQsG\nqD9R0rWSVkn6vqSDW3XrSvlKSSu6GWdERAyva1O0ShoHnAW8GlgPLJe01Pb1rWY/AV5p+y5Jc4BF\nwKGt+qNs396tGCMiol43jzBmAWtsr7X9MHAeMLfdwPb3bd9VVq8EJncxnoiI2AbdTBh7Aze31teX\nssG8HbiotW7gMklXS5o/WCdJ8yWtkLRi06ZN2xRwREQMrmunpEZC0lE0CeOIVvERtjdIei5wqaQb\nbV/R2df2IppTWcycOdPbJeCIiDGom0cYG4B9WuuTS9kTSDoI+Dww1/YdW8ptbyg/NwIX0JziioiI\nHulmwlgOTJO0n6QdgHnA0nYDSVOA84GTbP+oVb6rpN22LAPHANd1MdaIiBhG105J2d4s6TTgYmAc\nsNj2akmnlPqFwIeBPYHPSALYbHsmsBdwQSkbD3zF9je7FWtERAyvq9cwbC8DlnWULWwtnwycPEC/\ntcDBneUREdE7+aZ3RERUScKIiIgqSRgREVElCSMiIqokYURERJUkjIiIqJKEERERVZIwIiKiShJG\nRERUScKIiIgqSRgREVElCSMiIqokYURERJUkjIiIqJKEERERVZIwIiKiShJGRERU6WrCkDRb0k2S\n1khaMEC9JH261F8r6ZDavhERsX11LWFIGgecBcwBpgMnSJre0WwOMK085gOfHUHfiIjYjrp5hDEL\nWGN7re2HgfOAuR1t5gLnunElsIek51f2jYiI7Ui2u7Nh6fXAbNsnl/WTgENtn9Zq8w3gDNv/Vta/\nBfwpMHW4vq1tzKc5OgF4EXDTNoQ9Ebh9G/p3W7/HB4lxNPR7fND/MfZ7fNA/Me5re1JNw/HdjqTb\nbC8CFo3GtiStsD1zNLbVDf0eHyTG0dDv8UH/x9jv8cFTI8ZO3UwYG4B9WuuTS1lNmwkVfSMiYjvq\n5jWM5cA0SftJ2gGYByztaLMUeHO5W+ow4B7bt1b2jYiI7ahrRxi2N0s6DbgYGAcstr1a0imlfiGw\nDDgWWAM8CLx1qL7dirVlVE5tdVG/xweJcTT0e3zQ/zH2e3zw1IjxCbp20TsiIp5e8k3viIiokoQR\nERFVkjDoz2FIJO0j6TuSrpe0WtK7SvlzJF0q6cfl57N7HOc4Sf9RvlPTj/HtIemfJd0o6QZJL+un\nGCX9cfn9XidpiaSdeh2fpMWSNkq6rlU2aEyS3l/+d26S9Ns9jPET5fd8raQLJO3RbzG26k6XZEkT\nexnjSI35hNHHw5BsBk63PR04DDi1xLUA+JbtacC3ynovvQu4obXeb/H9LfBN2y8GDqaJtS9ilLQ3\n8L+AmbYPpLnBY14fxHc2MLujbMCYyt/kPOCA0ucz5X+qFzFeChxo+yDgR8D7+zBGJO0DHAP8rFXW\nqxhHZMwnDPp0GBLbt9q+pizfR/NGtzdNbOeUZucAr+1NhCBpMnAc8PlWcT/FtzvwCuALALYftn03\nfRQjzZ2KO0saD+wC3EKP47N9BXBnR/FgMc0FzrP9kO2f0NzxOKsXMdq+xPbmsnolzfe3+irG4m+A\n9wHtO456EuNIJWE0b8I3t9bXl7K+IWkq8FLgKmCv8l0VgNuAvXoUFsCnaP7wH2uV9VN8+wGbgC+W\n02afl7QrfRKj7Q3AX9F80ryV5ntIl/RLfB0Gi6lf/3/eBlxUlvsmRklzgQ22f9hR1TcxDiUJo89J\neibwNeDdtu9t17m5J7on90VLOh7YaPvqwdr0Mr5iPHAI8FnbLwUeoOP0To9fw2fTfLLcD3gBsKuk\nN7Xb9MFr+CT9GFObpA/QnNL9cq9jaZO0C/BnwId7HcvWSsKoG8KkJyRNoEkWX7Z9fin+uZoRfSk/\nN/YovJcDvyNpHc1pvN+S9A99FB80n9LW276qrP8zTQLplxiPBn5ie5PtR4DzgcP7KL62wWLqq/8f\nSX8AHA+c6Me/ZNYvMb6Q5sPBD8v/zWTgGknPo39iHFISRp8OQyJJNOfeb7D9yVbVUuAtZfktwL9s\n79gAbL/f9mTbU2les2/bflO/xAdg+zbgZkkvKkWvAq6nf2L8GXCYpF3K7/tVNNeq+iW+tsFiWgrM\nk7SjpP1o5rb5QQ/iQ9JsmlOkv2P7wVZVX8Roe5Xt59qeWv5v1gOHlL/TvohxWLbH/INmeJIfAf8J\nfKDX8ZSYjqA57L8WWFkexwJ70tyl8mPgMuA5fRDrkcA3ynJfxQfMAFaU1/HrwLP7KUbgY8CNwHXA\nl4Adex0fsITmmsojNG9qbx8qJuAD5X/nJmBOD2NcQ3MdYMv/y8J+i7Gjfh0wsZcxjvSRoUEiIqJK\nTklFRESVJIyIiKiShBEREVWSMCIiokoSRkREVEnCiKclSXtKWlket0na0FrfYYD2+0ta2YtYt4ak\nkyV9qtdxxNjStSlaI3rJ9h0038FA0keB+23/VU+D6iOSxtl+tNdxxFNLjjBizJH0vjL/xHWS/ucA\n9fuXwQoPkTRe0icl/aDMs3ByaXO0pG9JOr/MX3DuIPv6N0lnlP43STq8lD/hCEHSNyUdUfZ3d9nn\nakkXSzpU0uWS1ko6trX5fUv5jyV9sLWtt5T9rZT0GUnPaG33U5KupQ9HQo3+l4QRY4qkQ4ETgd8A\nXgb8kaRfa9W/BPgn4M1uhpefTzPI4qzS51RJU0rzQ4DTaOZReYmkwwbbben/XuoGntsduMj2AcDD\nwEdphg15A/DxVrtZNMOMzwB+X9IMSQcCrwMOtz2D5izCvNZ2r7B9kO1/r4gj4glySirGmiOAr9n+\nBYCkrwO/CVxCM2T3BcBrbd9Y2h9Dkwzab7rTyvKVtm8p21kJTKWZh6HTloEjry5thvML25eW5VU0\nw55vlrSqo//Ftu9qPY8jaP6nfwNY0QxPxc48Pmz2w+X5RWyVJIyIx91NM4HR4TTjOwEI+CPb32o3\nlHQ08FCr6FEG/396aIA2m3niEf5OreWHW8uPtfo/1rGPznF9XOJdbPtDHfGOp0lEGQsotlpOScVY\n813gdZJ2LnONzC1l0LwxzwVOlvTGUnYxzWmr8QCSXiRp51GIYx3wUjWmAr++Fds4Rs2c5bvQxP09\nmoEB36gyV3S5W2zKUBuJqJUjjBhTbP9A0hKaYe2hmVxplaT9S/39aiaHulTSA8DngCnAynKKZyOj\nM4Xv5TTzHdwArKYZXXWkltMMM/4C4BzbKwEkfQy4TNIzaEZKPYXmyClim2S02oiIqJJTUhERUSUJ\nIyIiqiRhRERElSSMiIiokoQRERFVkjAiIqJKEkZERFT5b4cLAYf9TUwiAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Get a dictionary from labels t indices, and back again\n", "label2class = { 'Iris-setosa':0, 'Iris-versicolor':1, 'Iris-virginica':2 }\n", "class2label = { 0:'Iris-setosa', 1:'Iris-versicolor', 2:'Iris-virginica' }\n", "# Read out a list of the class labels, convert to integers\n", "Y = [ label2class[x[4]] for x in dataset if len(x)==5 ]\n", "# Plot the class labels of each token\n", "plt.bar(range(0,len(Y)),Y)\n", "plt.title('Class labels of all of the tokens')\n", "plt.xlabel('Token number')\n", "plt.ylabel('Class label')" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(150, 4)" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a numpy arrays for each data subset\n", "X = np.array([ x[0:4] for x in dataset if len(x)==5 ], dtype='float64')\n", "X.shape" ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 71, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEWCAYAAACe8xtsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcHHWd//HXJweHBJJAIgIJZEkQBYwggRAOZcOKcijr\nLuIBYlhXxBVRo8sG8KHIKu4qyyWr8wNRZAHdrHLJpSgQIoFAAiFcATMYSDgnAXIIxiHz+f1R3056\nerq7qmeqqqt73s/Hox8zXVX9rU9VV/en6/jU19wdERGRRgxpdgAiItJ6lDxERKRhSh4iItIwJQ8R\nEWmYkoeIiDRMyUNERBqm5JEhM3Mzm5TDfMzMfmpmr5rZ/Qlfc4WZfTvDmHJZ9kaY2aFmtqLZcTTC\nzA4xsydTbC+1993Mtjezu81srZn9l5mdaWY/TqPtojCzCWFbHhae32pmn252XCVpbx+NaHryMLOD\nzWyema02s1fM7B4z22+Abc4wsz9UDMv0y3IgqsXboIOB9wPj3H3/DNqvy8zuMrN/zqr9wczd57r7\n7v15bdbvO3AysBLYxt2/6u7nunu/toO4bajyS7xZ3P0Id/9ZM2MoN5DtY6Ca+kaY2TbATcDngdnA\nZsAhwPpmxlWNmQ1z9zebHUcNuwDL3P3PzQ6kPwq+bqW2XYDHPUGlsd7jNuTuTXsAU4DXYqb5LPAE\nsBZ4HHhPGD4L6Cwb/pEw/J3AX4ANwDrgNaJfSN3AX8OwX4dpdwR+BXQBfwJOK5vv2cAvgauANcA/\nV4ntCqADuD3EMQfYpWy8A5PC/yOBK8O8ngG+TrTn1yfeGuthR+BG4BVgKfDZMPwzFa//VsXrqrYf\nYv9v4OYQ+3xgYtnr3hGW6xXgSeC4GnF9J7T9l9D+JWXLfgrwx/Ae/DdgYdwM4B7gAmAV8O0w/J/C\ne/0q8JuKdZkonjDttsBPgedDW9eH4YcCK8qmq7oNhXGTwvu5mujX9f+G4RbifjlsF48Ae4VxmwPn\nAc8CL4VtY8swbgzRD6XXwjLMBYYk+IxUxrwM+BqwOMT2v8AWVV6X9ft+Bb0/U39H9Jm5KoyfELaB\nz4T1cTewBdHnaVVYDw8A21NjG6qY37OhvXXhMY3oc7RvGH98GL9n2efi+rL35cKwPTwf/t+8xnIN\nDe/hSuBp4Auh3WFh/F2E7wJ6b8evhekPDMOXh23k02Vt19s+DgVWAF8Nr3sBOKnstUcSbaNrgeeA\nr9XYPt4ZYnwNeAz4cMV7VvW9p852XXPbHMiX/0AfwDZhQ/oZcAQwumL8R8OK2i8s3CTCF0oYtyPR\nF/DHgD8DO5S9qX+osrF/u+z5EGAh8A2iPZ5dw5v/gTD+bKIPx9+Habes8QFaC7w3bBgXlc+X3snj\nSuAGYGuiD9ZTwGdqxVtlXncDPyT6AO5NlISmJ3l9nfWxCtifaA/0auAXYdxWRBv/SWHcPkQfpj1q\ntH8XFck1LPtNwChg5xDvB8vieRP4Ymh/S+AYoqT4zjDs68C8fsZzM9GX6mhgOPC+Gh+0etvQz4Gz\nwrgtgIPD8A+E7WYU0Tb5zrLXXECU4LcN7/Ovge+Gcd8l+rIYHh6HEJJpzPteGfMy4P4Q97ZEyfaU\nJr3vV9D7M3U2fZPHlaHdLYHPhXXyFqIv6X2JDnlBlW2oYl6l9oaVDbsS+Gr4/1KiHwKfLxv3lfD/\nOcB9wFuBscA84N9rzOcUYAkwPqzfO6mfPN4M62so8G2ixPDfRN8HhxN9P4xIsH0cGto6J2wfRwKv\nE74TiZLJIeH/0Wz6Eb1x+wivWwqcSfSdNj3Mf/cE733N7brmexK38Wb9CEFeQZR13wwrd/sw7jfA\nlxK2swg4JuZDU76hTwWerZjmDOCnZR+Eu2PmeUVp5YfnI4h+QY0Pz50o4Q0l+oW2R9m0nwPuqhVv\nxXzGh3a3Lhv2XeCKhK+vtT5+XPb8SGBJ+P9jwNyK6f8f8M0a7d9F9eRxcNnz2cCssngq1/2thGQa\nng8h+vDs0kg8wA5ADxU/RLzig5ZgG7qS6AtpXMU004kS/wGU7TkQfeD+TO9f8dOAP4X/zyH68TCp\nwc9Hr5iJkscJZc+/B3Q06X2/gvjksWvZ+H8i+uKenGQbqhhfaq88eXwGuDH8/wTwz2z6MnyGTV+w\nncCRZa/7ANFh3mrzuYOyZEyUAOoljz+WTfuuMO32ZcNWEf3Yi9s+DgXeqFi+l4EDwv/PEn1nbFNr\n+yD6QfIivbfLnwNnJ3jvq27X9R5NP2Hu7k+4+wx3HwfsRfSL6sIwejzRG9+HmZ1oZovM7DUzey28\ndkwDs94F2LH0+tDGmUS70SXLE7SzcRp3X0e0u79jxTRjiH4VPFM27Blgp4Sx7gi84u5r+/n6Wl4s\n+/91ouQH0bqZWrFujgfellL70Hfd7gJcVDa/V4g+cDs1GM94onX1alxwMdvQ6WH+95vZY2b2TwDu\nfgdwCdGvy5fN7NJw7m4s0S/qhWXt3RaGA3yf6Ffhb83saTObFRdfHfXW60Ben9b7Xq78ff4foh+E\nvzCz583se2Y2fABtzwEOMbMdiH6gzQYOMrMJRIeJF4XpdqTvZ6/yM0rZtMsrpq3npbL/3wBw98ph\nI4jfPgBWee/zQuXvzT8Sfdk/Y2ZzzGxardjdvaci/vLviarvfZ3tuqamJ49y7r6EKDvuFQYtByZW\nTmdmuwCXAacC27n7KOBRog87RNm/T/MVz5cTZf1RZY+t3f3IOq+pZnxZXCOIdkmfr5hmJdEhsF3K\nhu1MdEguyXyeB7Y1s61rvD5OkuUotxyYU7FuRrj751Nqv9prlgOfq5jnlu4+r8F4lhOtq1H1Zh63\nDbn7i+7+WXffkegX3w9Llx67+8Xuvi+wB/B24F+J3uM3iI65l2Ic6e6lD+daj65I2hX4MDDTzA5r\neK01Juv3vaEY3L3b3b/l7nsQnRs4GjgxYax9xrv7UqIvwC8SHSVYQ/TleDLRHlfpS/R5+n72Kj+j\nJS9Q9pkO06ah7vYRx90fcPdjiA69XU+UKCs9D4w3s/Lv9cTfEzW265qamjzM7B1m9lUzGxeejwc+\nQXR8EuDHwNfMbN9QyzApfOi3ItqYusLrTmJTwoHo18A4M9usYtiuZc/vB9aa2b+Z2ZZmNtTM9urH\nZcJHhsuNNwP+HbjP3Xv9qnb3DURv9nfMbOuwDDOJTh7Wirf89cuJdve/a2ZbmNlkol32q6pNX0Xd\n9qu4CXi7mX3KzIaHx35m9s467e9aY1xSHcAZZrYngJmNNLOPNhqPu79AdAjsh2Y2Okz73irzq7sN\nmdlHS9sl0Ul3B3rCfKeGX8x/JjrJ2xO+qC4DLjCzt4Y2djKzD4T/jw7brxGd6N5AdHitdBn5Ff1c\nb/Vk/b43xMz+1szeZWZDiU7KdhPWAfHbUFeYtnKaOUQ/AOaE53dVPIfo0M3XzWysmY0hOs9Z67Mz\nGzjNzMaZ2WiiiyoGLG77qMfMNjOz481spLt3E627niqTzidKpqeH9+5Q4EPALxLMo+p2Xe81zd7z\nWEt07mG+mf2ZKGk8SnTFAe7+f0RXYlwTpr0e2NbdHwf+C7iXaKN7F9FVDyV3EF1p8KKZrQzDLgf2\nCLuM14cv9KOJjkf+ieiXwY+JdncbcQ3wTaLDLPsCJ9SY7otEb8rTwB/C635SJ95KnyA67vs8cB3R\ncejfJYwxSfsbhcNjhwMfD/N7EfhPopOA1VwEHGtRkeLFCWOqnOd1YR6/MLM1RNvBEf2M51NEX0xL\niI4bf7nK/OK2of2Itst1ROfhvuTuTxNd5HEZUUJ5huiY9vfDa/6N6NDUfWEZfgeUrsHfLTxfF+b5\nQ3e/M4wbXzHvtGT9vjfqbURXMK4hOkcxh+hQFsRsQ+7+OtF3wT3hM3xAGDWH6OTz3TWeQ3QiewHR\nFWqPAA+GYdVcRnRo7eEw3bWNL2ZN9baPOJ8CloXXnUJ0OLEXd/8rUbI4guj77IfAieGITpx623VV\npUsnpR/Cr8UV7v71ZscirSnsFTxMdBK5u9nxiCTV1CJBkcEu/FpM5bCQSJ6afdhKRERakA5biYhI\nwzLf8whXMT1kZjdVGXeoRTdEXBQe38g6HhERGbg8znl8iejKiloFJ3Pd/eikjY0ZM8YnTJiQRlwi\nIoPGwoULV7r72Pgpk8k0eYTr5I8iusRuZhptTpgwgQULFqTRlIjIoGFmcdXyDcn6sNWFRLd5qFds\ncqCZLbaok5U9q01gZieb2QIzW9DV1ZVJoCIiklxmycPMjgZedveFdSZ7ENjZ3ScDPyAqAuzD3S91\n9ynuPmXs2NT2ukREpJ+y3PM4CPiwmS0jKo+fbma9bgng7mvCzQRx91uA4eH2ASIiUmCZJQ93P8Pd\nx7n7BKLbHdzh7r1u3WFmbwv3+sHM9g/xrMoqJhERSUfuFeZmdgqAu3cAxwKfN7M3ie44+XFX4YmI\nSOG1XJHglClTXFdbCUDHnE4mjxvJgRM3Hemc17mSxStWc8r7+tzJX2RQM7OF7j4lrfZ0exJpWZPH\njeTUax5iXmd0w9h5nSs59ZqHmDyu0Rsji0ijdGNEaVkHThzDJZ/ch1OveYgTpu7MVfOf5ZJP7tNr\nT0REsqE9D2lpB04cwwlTd+biO5ZywtSdlThEcqLkIS1tXudKrpr/LKdNn8RV85/deAhLRLKl5CEt\nq3SO45JP7sPMw3ffeAhLCUQke0oe0rIWr1jd6xxH6RzI4hWrmxyZSPvTpboiIoOALtUVEZGmU/IQ\nEZGGKXmIiEjDlDxERKRhSh4iItIwJQ8REWmYkoeIiDRMyUNERBqm5CEiIg1T8pCm6JjT2eceVPM6\nV9Ixp7NJEYlII5Q8pCnUkZNIa1NnUNIU6shJpLVpz0OaRh05ibQuJQ9pGnXkJNK6lDykKdSRk0hr\nU/KQplBHTiKtTZ1BiYgMAuoMSjKnGgwRiaPkIX2oBkNE4qjOQ/pQDYaIxNGeh1SlGgwRqUfJQ6pS\nDYaI1KPkIX2oBkNE4ih5SB+qwRCROKrzEBEZBFquzsPMhprZQ2Z2U5VxZmYXm9lSM1tsZu/JOh4Z\nPFSvIpKdPA5bfQl4osa4I4DdwuNk4Ec5xCODhOpVRLKTafIws3HAUcCPa0xyDHClR+4DRpnZDlnG\nJINHeb3K+b99cuNFALrsWGTgst7zuBA4HeipMX4nYHnZ8xVhWC9mdrKZLTCzBV1dXelHKW1L9Soi\n2cgseZjZ0cDL7r5woG25+6XuPsXdp4wdOzaF6GSwUL2KSDay3PM4CPiwmS0DfgFMN7OrKqZ5Dhhf\n9nxcGCYyYKpXEclOZsnD3c9w93HuPgH4OHCHu59QMdmNwInhqqsDgNXu/kJWMcngonoVkezkfmNE\nMzsFwN07gFuAI4GlwOvASXnHI+3rlPdN7DPswIljdN5DJAW5JA93vwu4K/zfUTbcgS/kEYPkZ8ZP\n7+egSdvx2UM2fXlfNreTe5au4oqT9m9iZCKSFt2eRFJ30KTtOPfmJVw2NyrGu2xuJ+fevISDJm3X\n5MhEJC3qz0NSV9rjOPfmJdz+2Es8sOxVzjzqHb32RESktWnPQzLx2UMmst+E0dy/7FX2mzBaiUOk\nzSh5SCYum9vJA8teZf8Jo3lg2asbD2GJSHvQYStJXekcR+lQVek5oD0QkTah5CGpu2fpql7nOEp/\n71m6SslDpE2oPw8RkUGg5frzEBGR9qPkMchMP+8uzrpuca9hZ123mOnn3dWcgAZAnT1Jq+h6vYsZ\nt81g5Rv9v69aGm2kScljkJk2cVuunr98YwI567rFXD1/OdMmbtvkyBqnzp6kVXQs7uDBlx6k4+GO\n+IkzbCNNOucxCJUSxo4jt+D51X/h+Knj+c5HJjc7rH4pJYwTpu7MVfOfVWdPUjhdr3dxxLVHsH7D\nejYfujm3/eNtjNmysW00jTZ0zkMG7Dsfmbwxcew4couWTRygzp6k+DoWd9DjUX94Pd7Trz2HNNpI\nm5LHIHTWdYs3Jo7nV/+lzzmQVqLOnqTIul7v4oalN9Dd0w1Ad0831y+9vqHzFmm0kQUlj0GmdMjq\n+KnjmXfGYRw/dXyvcyCtRJ09SdGV7zGUNLrnkEYbWVCR4CBzb+crvc5xlP7e2/lKM8Pql3qdPenw\nlRTBwy8/vHGPoaS7p5tFLy/KtY0s6IS5iMggoBPmMiBp1EbEtaH6C5H2p+QxyKRRGxHXhuovRNqf\nDlsNQmnURsS1ofoLkWLRYSsZsDRqI+LaUP2FSHtT8hiE0qiNiGtD9Rci7U3JY5BJozYirg3VX4i0\nPyWPQaZebURabaQxDxEpNp0wFxEZBHTCvKCKUttQlDhE8lC0Pi4GEyWPlBSltqEocYjkoWh9XAwm\nOmyVoqLUNhQlDpEspdHHxWCiw1YFVpTahqLEIZKlIvZxMZgoeaSoKLUNRYlDJCtF7eNiMFHySElR\nahuKEodIlorax8VgouSRkqLUNhQlDpEsFbWPi8GkXyfMzexWdz8ig3hiFfmEuYhIUeV2wtzM3lPj\nsS+wd4JAtzCz+83sYTN7zMy+VWWaQ81stZktCo9vDHB5Br0ZP72fy+b2rum4bG4nM356f6LxkE+t\niOpRRFpbvcNWDwDnAf9V8TgPGJWg7fXAdHd/N1Gy+aCZHVBlurnuvnd4nNNQ9NLHQZO249ybl2xM\nEJfN7eTcm5dw0KTtEo2HfGpFVI8i0trq9WH+BPA5d/9j5QgzWx7XsEfHw9aFp8PDo7WKSlrQZw+Z\nCMC5Ny/h9sde4oFlr3LmUe/YODxuPGw6T5JlrUge8xCR7NTb8zi7zvgvJmnczIaa2SLgZeB2d59f\nZbIDzWyxmd1qZnvWaOdkM1tgZgu6urqSzHpQ++whE9lvwmjuX/Yq+00Y3SsxJBkP+dSKqB5FpHXV\nTB7u/kt3f7LGuOuTNO7uG9x9b2AcsL+Z7VUxyYPAzu4+GfgBULVdd7/U3ae4+5SxY8cmmfWgdtnc\nTh5Y9ir7TxjNA8terXqOo954yKdWRPUoIq2r3mGr1Lj7a2Z2J/BB4NGy4WvK/r/FzH5oZmPcXd8i\n/VQ6h1E6FFV6DvR6Xms89K4VOXDiGA6YuF2v52nIYx4ikp3M6jzMbKyZjQr/bwm8H1hSMc3bzMzC\n//uHeFZlFdNgcM/SVX3OcZx51Du4Z+mqROMhn1oR1aOItLbMboxoZpOBnwFDiZLCbHc/x8xOAXD3\nDjM7Ffg88CbwBjDT3efVa1d1HiIijUu7ziPRYSszOxCYUD69u19Z7zXuvhjYp8rwjrL/LwEuSRir\niIgUROxhKzP7H6LajoOB/cIjtezVLtIoektSwDfQNpLEOdBlSWM5CmXxbLhgLzh7VPR38ex+NZOk\n4yJ1biStIsk5jynAQe7+L+7+xfA4LevAWk0aRW9JCvgG2kaSOAe6LGksR2Esng2/Pg1WLwc8+vvr\n0/qVQJJ0XKTOjaRVxJ7zMLP/A05z9xfyCam+Ip/zSKMTptIX7X7hMtrKAr402kgS50CXJY3lKIQL\n9gqJo8LI8fCVR/sOryFJx0Xq3Eiy1IzOoMYAj5vZb8zsxtIjrQDaSRpFb0kK+AbaRpI4B7osaSxH\nIaxe0djwGpJ0XKTOjaSVJEkeZwN/D5xL73tcSYU0it6SFPANtI0kcQ50WdJYjkIYOa6x4VUk6bhI\nnRtJq4lNHu4+h6g+Y+vweCIMkzJpdMJUXsA3+5QDOfOod/Q6d5BGG0niHOiypLEchXHYN2D4lr2H\nDd8yGp5Qko6L1LmRtJokV1sdB9wPfBQ4DphvZsdmHVirSaPoLUkB30DbSBLnQJcljeUojMnHwYcu\njs5xYNHfD10cDU8oScdF6txIWk2SE+YPA+9395fD87HA78Kt1nNX5BPmIiJF1YwT5kNKiSNYlfB1\n0gRxNRrqhKnAFs+m68K9mNGxGysv7H89yUAtWbWEaddM46lXnmrK/KU1JEkCt4UrrWaY2QzgZuCW\nbMOS/oqr0VAnTAUV6kk6hq7jwS02p2Poun7XkwzUrLmzWNe9jtPvPj33eUvrSHRvKzP7R+Cg8HSu\nu1+XaVR16LBVvLgajTTqUSRlF+xF17rnOGLcjqwfMoTNe3q4bcXzjBmxU0P1JAO1ZNUSPnrTRzc+\n/9WHfsXbt317bvOX7DTjsBXu/it3nxkeTUsckkxcjYY6YSqg1SvoGDWSHgyAHoyOUSMbricZqFlz\nZ/V6rr0PqaVm8jCzP4S/a81sTdljrZmtqfU6ab64Gg11wlQ8XaPGccOIregeEiWP7iHG9SO2YuWo\n5PUkA7Vk1RI6V/c+99W5ulPnPqSqej0JHhz+bu3u25Q9tnb3bfILURoRV6ORRj2KpK9j0hR6oq5t\nNuoxo2NSfvcgrdzrKNHeh1STpM5jopltHv4/1MxOK3XyJMUTV6OhTpiK6eGedXRXJI9uMxb1rMst\nhhXrqh8iqzVcBrckdR6LiO6sO4HoKqsbgD3d/cjMo6tCJ8xFRBrXjBPmPe7+JvAR4Afu/q/ADmkF\nUARp1D7EtZFXHxeq4+iHlPrryFyCOpC8+gOJm09sHDn2kSLZSJI8us3sE8CngZvCsOHZhZS/NGof\n4trIq48L1XE0KMX+OjKVsA4kr/5A4uZTd3zOfaRINpIcttoDOAW4191/bmZ/Axzn7v+ZR4CVsjps\nlUbtQ1wbefVxoTqOBqTUX0fmEtSB5NUfSNx8YuPIsY8U2ST3w1bu/ri7n+buPw/P/9SsxJGlNGof\n4trIq48L1XE0IKX+OjKXoA4kr/5A4uYTG0eOfaRIdpJcbXWQmd1uZk+Z2dNm9iczezqP4PKURu1D\nXBt59XGhOo4GpNBfRx7i6kDy6g8kbj6J4sipjxTJVpJzHpcD5wMHA/sRXXm1X5ZB5S2N2oe4NvLq\n40J1HA1Kob+OPMTVgeTVH0jcfBLFkVMfKZKtJMljtbvf6u4vu/uq0iPzyHKURu1DXBt59XGhOo4G\npdBfRx7i6kDy6g8kbj6J4sipjxTJVpIT5v8BDAWuBdaXhrv7g9mGVp3qPEREGteMOo+pRIeqyvsw\nPy+tANpFHrUi08+7i7OuW9xr/FnXLWb6eXf1L2hpOZnXNdw0k65vj4lqSb49Bm6a2Xgbi2ez5KK9\nmPaTPXnqogLXzciAJLna6m+rPKbnEVwryaNWZNrEbbl6/vKNCeSs6xZz9fzlTJu4bcpLI0WVaV3D\nTTNhweV0jBwR1ZKMHAELLm8sgYQajllbdrNuiHH6lt3FrJuRAUty2Gp7or2OHd39iFD3Mc3dL88j\nwEpFPmyVR61IKWHsOHILnl/9F46fOp7vfGRy2osiBZR5XcO3tqVriPetJekx+OYrydq4YC+WvP4C\nH91pBzADd3713Au8/S07FKtuZhBqxmGrK4DfADuG508BX04rgHaSR63Idz4yeWPi2HHkFkocg0jm\ndQ2+oXotiW9I3sbqFcwa23ubPX3smOLVzciAJUkeY9x9NtADEO5z1cDWNHjkUSty1nWLNyaO51f/\npc85EGlPedQ1dA0bXr2WZFjyuxEtGT2Ozs2GR3sdAGZ0bjacp0YXq25GBi5J8vizmW0HOICZHQDo\nus8KedSKlA5ZHT91PPPOOIzjp47vdQ5E2lcedQ0dk/bbuNexcR4YHZOSl3XN2n5s1eGn1xgurWtY\ngmlmAjcCE83sHmAscGymUbWgevUVSQ9fxbVxb+crvc5xlP7e25nweLS0rDzqGh5+y1Z0r6+oJRli\nLHrLVonbWPHm2k17HSVm0XBpK7EnzAHMbBiwO2DAk+7eHfOSzBT5hLmISFGlfcI8ds/DzIYCRxJ1\nBjUMONzMcPfzY163BXA3sHl43S/d/ZsV0xhwUWj/dWBGs4oPRUQkuSTnPH4NzAC2A7Yue8RZD0x3\n93cDewMfDOdLyh0B7BYeJwM/ShZ2ckmK94rSgVJch1GttCypdPZz00z41rZw9sjob7V6gzTmk0Yn\nSwnayMO9z93Lu698N/Ofn199grj1lXA58uiEKck8itAZVKvEmbYkyWOcu/+Du3/T3b9VesS9yCOl\nDpiHh0flMbJjgCvDtPcBo8ws1V4KkxTvFaUDpbgOo1pmWdLo7CcUrG28TNQ39C1YS2M+aXSylLCN\nPHzt7q/R4z3MnFMj0dZbXw0sRx6dMCWZRxE6g2qVONOWJHncamaH96dxMxsa+kB/Gbjd3St/Du0E\nlPcKsyIMS03ppPOp1zzE+b99cuPVTOUnsZNMk4fSzRLPvXkJx3XM23gX3tLNFFtmWX5/DnS/0XtY\n9xvR8KQWXhE/PI35/P4cunrWc8OIrXALl6b2rO/VRukyWcerXx6boI083Pvcvaz56xoA1vx1Td+9\nj7j1lXA5YtdHCpLMI4842iXOLCRJHvcB15nZG2a2xszWmtmaJI27+wZ33xsYB+xvZnv1J0gzO9nM\nFpjZgq6uroZfn6R4rygdKMV1GNUSy5JGZz+1CtPKh6cxnzQ6WUrQRh6+dvfXej3vs/cRt74SLkce\nnTAlmUcROoNqlTizkCR5nA9MA97i7tu4+9buvk0jM3H314A7gQ9WjHoOGF/2fFwYVvn6S919irtP\nGTu28evFkxTvFaUDpbgOo1piWdLoYMmGxg9Po1OhFDpZimsjD+V7HSV99j5i1leS5cilWDHJOi9A\nZ1CtEmdWkiSP5cCjnuSa3jJmNtbMRoX/twTeDyypmOxG4ESLHEDUd8gLjcwnTpLivaJ0oBTXYVTL\nLEsaHSztOyN+eBqdCqXQyVJcG3mo3Oso6bX3EbO+kixHLsWKSdZ5ATqDapU4s5IkeTwN3GVmZ5jZ\nzNIjwet2AO40s8XAA0TnPG4ys1PM7JQwzS2h/aXAZcC/9GMZ6krSOVJROlCK6zCqZZYljQ6Wjj4f\npnxm056GDY2eH112hXganQql0MlSXBt5WPvX6kV4vYbHrK8ky5FLsWKSdV6AzqBaJc6sJLmr7jer\nDU9yxVUWVCQoItK43O+qW355biOX6raSwtRGSG9JajjSqPPII46EcdatB0ippiXz9VUg7VhfURQ1\nk4eZXRj+/trMbqx85Bdi9gpRGyG9JanhSKPOI484GoizZj1AijUtma6vgmnH+oqiqHnYysz2dfeF\nZva+auPnjd20AAAQ30lEQVTdfU6mkdWQ1WGrNDpykhRdsFf4kqswcvymToWSTFOEOBLGWbezpzSW\nNY/1VSCZd57VYnI7bOXuC8PfOcDjwOPuPqf0SCuAomh6bYT0lqSGI406jzziSBhn3XqAlGpaBtxG\nC2nX+oqiqHvOw8zONrOVwJPAU2bWZWYNXG/ZOppeGyG9JanhSKOeJI84ErQRWw+QxrLmsb4Kop3r\nK4qi3jmPmcBBwH7uvq27jwamAgeZ2VfyCjAPhaiNkN6S1HCkUU+SRxwJ2oitB0hjWfNYXwXRzvUV\nRVFvz+NTwCfc/U+lAe7+NHACcGLWgeWpELUR0luSGo406knyiCNBG7H1AGksax7rqyDaub6iKOqd\nMH/U3avei6reuKypzkNEpHF51nn8tZ/jRFITe51+gj4/crnWP6U44qZZsmoJ066ZxlOvPNV3ZF41\nHG1WK1KUvklaTb3k8e5wF93Kx1rgXXkFKINb3ev0k/T5EddGGlKMI26aWXNnsa57HafffXrvEXnV\ncLRhrUhR+iZpNfUu1R0a7qJb+dja3YfnGaQMTrH9ICTo8yOXvhRSiiNumiWrltC5OrrrQefqzt57\nH2n0bZJEXvPJSVH6JmlFSW6MKNIUsdfpJ+jzI5dr/VOKI26aWXNn9Xrea+8jrxqONqsVKUrfJK1I\nyUMKKdF1+jF9fuR2rX8KccRNU77XUdJr7yOvGo42qhUpSt8krUrJQwop0XX6MX1+5HatfwpxxE1T\nuddRsnHvI68ajjaqFSlK3yStSslDCinRdfoxfX7kdq1/CnHETbNiXfXDQhuH51XD0Ua1IkXpm6RV\nxfbnUTSq8xARaVzu/XnIIFWEa/nTiOGmmSw5961M+8mePHXuW6vWX+QSR0Jx9QDtWC8grUnJQ/oq\nwrX8acQQ6i9mjRnNuiHG6WNGV62/yDyOBsTVA7RjvYC0JiUP6asI1/KnEcPCK1gyfBidmw0HMzo3\nG85Tw4fVrsvIKo6E4uoB2rVeQFqTkof0VYRr+dOIwTcwa2zvfllOHzumdl1GVnEkFFcP0K71AtKa\nlDykryJcy59CDEs223zjXgewae9js81zjSOJuHqAdq4XkNak5CF9FeFa/hRimDV+16rDT68xPKs4\nkoirB2jnegFpTUoe0lcRruVPIYYV9uamvY4Ss2h4jnEkEVcP0M71AtKaVOchIjIIqM5D2kca9RNx\nbRSoRkMGr3bcNpQ8pDnSqJ+Ia6NgNRoyeLXjtqHkIc2RRv1EXBsFqtGQwatdtw0lD2mONOon4too\nUI2GDF7tum0oeUhzpFE/EddGQWo0ZPBq521DyUOaI436ibg2ClKjIYNXO28bSh7SHGnUT8S1UZAa\nDRm82nnbUJ2HiMggoDoPERFpusySh5mNN7M7zexxM3vMzL5UZZpDzWy1mS0Kj9brCLkJBlxwVISO\nnpLEkTDOtinAWjybrgv3YkbHbqy8sHnvS9usT8lUlnsebwJfdfc9gAOAL5jZHlWmm+vue4dHjh1G\ntK4BFRwVoaOnJHE0EGdbFGCF5e0Yuo4Ht9icjqHrmvO+0CbrUzKXWfJw9xfc/cHw/1rgCWCnrOY3\nWAy44KgIHT0liSNhnG1TgPX7c+jqWc8NI7bCzbh+xFas7Fmf+/vSNutTMpfLOQ8zmwDsA8yvMvpA\nM1tsZrea2Z41Xn+ymS0wswVdXV0ZRlp8Ay44KkJHT0niSBhn2xRgrV5Bx6iR9BDdBbgHo2PUyNzf\nl7ZZn5K5zJOHmY0AfgV82d3XVIx+ENjZ3ScDPwCur9aGu1/q7lPcfcrYsWOzDbjAUik4KkJHT0ni\nSBBnOxVgdY0axw0jtqJ7SJQ8uoeEvY9R+b0v7bQ+JXuZJg8zG06UOK5292srx7v7GndfF/6/BRhu\nZmMqp5NIKgVHRejoKUkcCeJspwKsjklT6Knoe6THjI5JqV1ZGR9DG61PyV6WV1sZcDnwhLufX2Oa\nt4XpMLP9Qzyrsoqp1aVScFSEjp6SxJEgznYqwHq4Zx3dFcmj24xFPevyi6GN1qdkL7MiQTM7GJgL\nPAKUfs6cCewM4O4dZnYq8HmiK7PeAGa6+7x67apIUESkcWkXCQ5Lq6FK7v4HwGKmuQS4JKsY2tLi\n2dEVOKtXRMf/D/tG/nsNablpJiy8AnwD2FDYdwYcXXUnVUQKJrPkIRko1T6ULmEt1T5A6yWQm2bC\ngss3PfcNm54rgYgUnm5P0kqKUqORhoVXNDZcRApFyaOVFKVGIw2+obHhIlIoSh6tpCg1GmmwoY0N\nF5FCUfJoJUWp0UjDvjMaGy4ihaLk0UqKUqORhqPPhymf2bSnYUOj5zpZLtIS1BmUiMggoM6gmqBj\nTifzOnvf32de50o65nQ2KaIYRemvI06rxJkXrQ9pIUoeCUweN5JTr3loYwKZ17mSU695iMnjRjY5\nsiqK0l9HnFaJMy9aH9JilDwSOHDiGC755D6ces1DnP/bJzn1moe45JP7cODEAt7DsVVqQVolzrxo\nfUiLUfJI6MCJYzhh6s5cfMdSTpi6czETB7ROLUirxJkXrQ9pMUoeCc3rXMlV85/ltOmTuGr+s33O\ngRRGq9SCtEqcedH6kBaj5JFA6RzHJZ/ch5mH777xEFYhE0ir1IK0Spx50fqQFqPkkcDiFat7neMo\nnQNZvGJ1kyOrolVqQVolzrxofUiLUZ2HiMggoDoPkbwtnk3XhXsxo2M3Vl7Yz/oL1XBIm1HyEKkn\n1F90DF3Hg1tsTsfQdY3XX6iGQ9qQkodIPb8/h66e9dwwYivcjOtHbMXKnvWN1V+ohkPakJKHSD2r\nV9AxaiQ9oUflHoyOUSMbq79QDYe0ISUPkTq6Ro3jhhFb0T0kSh7dQ8Lex6gG6i9UwyFtSMlDpI6O\nSVPoMes1rMeMjkkNXLSiGg5pQ0oeInU83LOO7ork0W3Gop51yRtRDYe0IdV5iIgMAqrzEBGRplPy\nEBGRhil5iIhIw5Q8RESkYUoeIiLSMCUPERFpmJKHiIg0TMlDREQapuQhIiINyyx5mNl4M7vTzB43\ns8fM7EtVpjEzu9jMlprZYjN7T1bxDCrqeEhEMjYsw7bfBL7q7g+a2dbAQjO73d0fL5vmCGC38JgK\n/Cj8lf4qdTxU6j+i1PEQ6F5KIpKazPY83P0Fd38w/L8WeALYqWKyY4ArPXIfMMrMdsgqpkFBHQ+J\nSA5yOedhZhOAfYD5FaN2ApaXPV9B3wSDmZ1sZgvMbEFXV1dWYbYHdTwkIjnIPHmY2QjgV8CX3X1N\nf9pw90vdfYq7Txk7dmy6AbYbdTwkIjnINHmY2XCixHG1u19bZZLngPFlz8eFYdJf6nhIRHKQ5dVW\nBlwOPOHu59eY7EbgxHDV1QHAand/IauYBgV1PCQiOcjyaquDgE8Bj5jZojDsTGBnAHfvAG4BjgSW\nAq8DJ2UYz+Ax+TglCxHJVGbJw93/AFjMNA58IasYREQkG6owFxGRhil5iIhIw5Q8RESkYUoeIiLS\nMIvOWbcOM+sCnmliCGOAlU2cfyNaJVbFma5WiRNaJ9Z2iHMXd0+tyrrlkkezmdkCd5/S7DiSaJVY\nFWe6WiVOaJ1YFWdfOmwlIiINU/IQEZGGKXk07tJmB9CAVolVcaarVeKE1olVcVbQOQ8REWmY9jxE\nRKRhSh4iItIwJY86zGyomT1kZjdVGXeoma02s0Xh0ZQOM8xsmZk9EmJYUGW8mdnFZrbUzBab2Xua\nEWeIJS7WoqzTUWb2SzNbYmZPmNm0ivGFWKcJ4izK+ty9LIZFZrbGzL5cMU3T12nCOIuyTr9iZo+Z\n2aNm9nMz26JifObrM8tbsreDLxH1vb5NjfFz3f3oHOOp5W/dvVZh0BHAbuExFfhR+Nss9WKFYqzT\ni4Db3P1YM9sMeEvF+KKs07g4oQDr092fBPaG6AcZUYdv11VM1vR1mjBOaPI6NbOdgNOAPdz9DTOb\nDXwcuKJssszXp/Y8ajCzccBRwI+bHcsAHQNc6ZH7gFFmtkOzgyoqMxsJvJeoIzPc/a/u/lrFZE1f\npwnjLKLDgE53r7xLRNPXaYVacRbFMGBLMxtG9KPh+Yrxma9PJY/aLgROB3rqTHNg2CW81cz2zCmu\nSg78zswWmtnJVcbvBCwve74iDGuGuFih+ev0b4Au4KfhkOWPzWyrimmKsE6TxAnNX5+VPg78vMrw\nIqzTcrXihCavU3d/DjgPeBZ4gagH1t9WTJb5+lTyqMLMjgZedveFdSZ7ENjZ3ScDPwCuzyW4vg52\n972JdlO/YGbvbVIcScTFWoR1Ogx4D/Ajd98H+DMwqwlxxEkSZxHW50bh0NqHgf9rZhxxYuJs+jo1\ns9FEexZ/A+wIbGVmJ+Qdh5JHdQcBHzazZcAvgOlmdlX5BO6+xt3Xhf9vAYab2Zi8Aw2/QnD3l4mO\nz+5fMclzwPiy5+PCsNzFxVqQdboCWOHu88PzXxJ9SZcrwjqNjbMg67PcEcCD7v5SlXFFWKclNeMs\nyDr9O+BP7t7l7t3AtcCBFdNkvj6VPKpw9zPcfZy7TyDafb3D3XtldjN7m5lZ+H9/onW5Ks84zWwr\nM9u69D9wOPBoxWQ3AieGqy8OINrFfSHPOEvxxcVahHXq7i8Cy81s9zDoMODxismavk6TxFmE9Vnh\nE9Q+FNT0dVqmZpwFWafPAgeY2VtCLIcRXdhTLvP1qautGmBmpwC4ewdwLPB5M3sTeAP4uOdfrr89\ncF3YlocB17j7bRVx3gIcCSwFXgdOyjnGRmItwjoF+CJwdTh88TRwUkHXaVycRVmfpR8M7wc+Vzas\ncOs0QZxNX6fuPt/Mfkl0CO1N4CHg0rzXp25PIiIiDdNhKxERaZiSh4iINEzJQ0REGqbkISIiDVPy\nEBGRhil5SNsxsw0W3fH0MTN72My+amZDwrgpZnZxk+Kal1I725rZ7Wb2x/B3dBrtijRCl+pK2zGz\nde4+Ivz/VuAa4B53/2ZzI0uHmX0PeMXd/8PMZgGj3f3fmh2XDC7a85C2Fm6FcjJwaqi2PdRC/yxm\ndraZ/czM5prZM2b2D2b2PYv6HLnNzIaH6fY1szkW3dDxNxbuTmpmd5nZf5rZ/Wb2lJkdEobvGYYt\nsugGeruF4evCXzOz71vUF8MjZvaxMPzQ0Gapj46rS9XMFY4Bfhb+/xnw99mtQZHqlDyk7bn708BQ\n4K1VRk8EphPdCO8q4E53fxdR9fBRIYH8ADjW3fcFfgJ8p+z1w9x9f+DLQGnP5hTgonATyClE96Eq\n9w9E/Ua8m+g+Rd+3TbfL3ie0tQewK9F91iptX3ariReJqvdFcqXbk8hgd6u7d5vZI0QJ5rYw/BFg\nArA7sBdwe9gJGEp0G+ySa8PfhWF6gHuBsyzqE+Zad/9jxTwPBn7u7huAl8xsDrAfsAa4391XAJjZ\notDmH2oF7+5uZjr2LLnTnoe0PTPbFdgAvFxl9HoAd+8BusvuU9RD9OPKgMfcfe/weJe7H175+tD+\nsNDWNUR7Mm8At5jZ9AbCXV/2/8Y2K7xUduhshxrLJZIpJQ9pa2Y2FugALunnDeyeBMZa6B/czIZb\nTAdAIVk97e4XAzcAkysmmQt8zMyGhvjeC9zfQEw3Ap8O/386zEMkVzpsJe1oy3DIZzjRXUf/Bzi/\nPw25+1/N7FjgYou6fh1G1MvkY3VedhzwKTPrJjoncW7F+OuAacDDRL0rnu7uL5rZOxKG9R/AbDP7\nDPBMmJ9IrnSproiINEyHrUREpGFKHiIi0jAlDxERaZiSh4iINEzJQ0REGqbkISIiDVPyEBGRhv1/\nRermbRHmsewAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plot a scatter plot of the three classes, so we can see how well they separate in the first two dimensions\n", "plt.plot(X[0:50,0],X[0:50,1],'x',X[50:100,0],X[50:100,1],'o',\n", " X[100:150,0],X[100:150,1],'^')\n", "plt.title('Scatter plot of the three classes, in the first two dimensions')\n", "plt.xlabel('Dimension 0')\n", "plt.ylabel('Dimension 1')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Likelihood models: Gaussian\n", "Now let's build a Gaussian model of each of those classes, and see how well it fits." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[,\n", " ,\n", " ]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEVFJREFUeJzt3X2MZXV9x/H3p7troYqscacWWXCtVtGQddeOQNVoxaBo\nY9GWaOIDkWoIqY+pTWhMakr9p/2ntYToZKOpNGqajeITFVuqVEOQpQPsDl2gZvEB8aE7q4JuNXRh\nv/3jXnC8DnPPzN479+4v71cymXvO+c7cT4ZfPnM499ydVBWSpLb82qQDSJJGz3KXpAZZ7pLUIMtd\nkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNWjjpJ54y5YttW3btkk9vSQdl2655ZZDVTUzbG5i5b5t\n2zbm5+cn9fSSdFxK8u0uc16WkaQGWe6S1CDLXZIaZLlLUoMsd0lq0HFT7nNfuZsb7z70S/tuvPsQ\nc1+5e0KJJGl6HTflvn3rybz9E7c9UvA33n2It3/iNrZvPXnCySRp+kzsPvfVev7TtnDl63fy9k/c\nxhvPPp2P7bmHK1+/k+c/bcuko0nS1DluztyhV/BvPPt0rvjyAd549ukWuyQ9iuOq3G+8+xAf23MP\n7zz36Xxszz2/cg1ektQztNyTnJDk5iT7kuxPcvkKs89L8mCSC0cb8xfX2K98/U7+7GXPfOQSjQUv\nSb+qy5n7A8C5VfUcYAdwfpJzBoeSbAD+Fvi30UbsWbj3/l+6xv7wNfiFe+8fx9NJ0nFt6AuqVVXA\n4f7mpv5HLTP6DuBTwPNGlm6JS1/8tF/Z9/ynbfG6uyQto9M19yQbkuwFDgLXVdWegeOnAq8BPjT6\niJKk1epU7lX1UFXtALYCZyU5c2DkA8BlVXV0pe+T5JIk80nmFxcX15ZYkjTUqu6Wqar7gOuB8wcO\nzQL/nORbwIXAB5O8epmv31VVs1U1OzMz9N+alySt0dBr7klmgCNVdV+SE4Hz6L1w+oiqeuqS+Y8C\n11TVZ0acVZLUUZd3qJ4CXNW/G+bXgN1VdU2SSwGqam6cASVJq9flbpkFYOcy+5ct9ap687HHkiQd\ni+PqHaqSpG4sd0lqkOUuSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwl\nqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGDS33JCckuTnJ\nviT7k1y+zMwFSRaS7E0yn+SF44krSepiY4eZB4Bzq+pwkk3ADUmuraqblsx8CfhcVVWS7cBu4Iwx\n5JUkdTC03KuqgMP9zU39jxqYObxk87GDxyVJ66vTNfckG5LsBQ4C11XVnmVmXpPkLuBfgD95lO9z\nSf+yzfzi4uKx5JYkraBTuVfVQ1W1A9gKnJXkzGVmPl1VZwCvBt7/KN9nV1XNVtXszMzMseSWJK1g\nVXfLVNV9wPXA+SvMfBX47SRbjjGbJGmNutwtM5Nkc//xicB5wF0DM09Pkv7j5wK/Dvxw9HElSV10\nuVvmFOCqJBvo/TLYXVXXJLkUoKrmgD8GLkpyBPg58Lr+C7GSpAnIpDp4dna25ufnJ/LcknS8SnJL\nVc0Om/MdqpLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMs\nd0lqkOUuSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNGlruSU5IcnOSfUn2\nJ7l8mZk3JFlIcnuSG5M8ZzxxJUldbOww8wBwblUdTrIJuCHJtVV105KZbwIvrqofJ3kFsAs4ewx5\nJUkdDC33qirgcH9zU/+jBmZuXLJ5E7B1VAElSavX6Zp7kg1J9gIHgeuqas8K428Brh1FOEnS2nQq\n96p6qKp20DsjPyvJmcvNJXkJvXK/7FGOX5JkPsn84uLiWjNLkoZY1d0yVXUfcD1w/uCxJNuBDwMX\nVNUPH+Xrd1XVbFXNzszMrCWvJKmDLnfLzCTZ3H98InAecNfAzOnA1cCbqurr4wgqSequy90ypwBX\nJdlA75fB7qq6JsmlAFU1B7wPeCLwwSQAD1bV7JgyS5KG6HK3zAKwc5n9c0sevxV462ijSZLWyneo\nSlKDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrsk\nNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ2y3CWpQZa7JDXIcpekBg0t9yQnJLk5yb4k+5Nc\nvszMGUm+luSBJH8+nqiSpK42dph5ADi3qg4n2QTckOTaqrppycyPgHcCrx5HSEnS6gw9c6+ew/3N\nTf2PGpg5WFX/CRwZfURJ0mp1uuaeZEOSvcBB4Lqq2rOWJ0tySZL5JPOLi4tr+RaSpA46lXtVPVRV\nO4CtwFlJzlzLk1XVrqqararZmZmZtXwLSVIHq7pbpqruA64Hzh9PHEnSKHS5W2Ymyeb+4xOB84C7\nxh1MkrR2Xe6WOQW4KskGer8MdlfVNUkuBaiquSS/BcwDjweOJnk38Oyq+sm4gkuSHt3Qcq+qBWDn\nMvvnljz+Ab3r8ZKkKeA7VCWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUGW\nuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIaNLTc\nk5yQ5OYk+5LsT3L5MjNJckWSA0kWkjx3PHElSV1s7DDzAHBuVR1Osgm4Icm1VXXTkplXAL/T/zgb\n+FD/syRpAoaeuVfP4f7mpv5HDYxdAPxTf/YmYHOSU0YbVZLUVadr7kk2JNkLHASuq6o9AyOnAt9Z\nsn1vf58kaQI6lXtVPVRVO4CtwFlJzlzLkyW5JMl8kvnFxcW1fAtJUgerulumqu4DrgfOHzj0XeC0\nJdtb+/sGv35XVc1W1ezMzMxqs0qSOupyt8xMks39xycC5wF3DYx9Driof9fMOcD9VfX9kaeVJHXS\n5W6ZU4Crkmyg98tgd1Vdk+RSgKqaA74AvBI4APwMuHhMeSVJHQwt96paAHYus39uyeMC3jbaaJKk\ntfIdqpK0jhZ/tsibv/hmDv380Fifx3KXpHU0tzDHrf9zK3P75oYPHwPLXZLWyeLPFvnsgc9SFJ85\n8Jmxnr1b7pK0TuYW5jhaRwE4WkfHevZuuUvSOnj4rP3I0SMAHDl6ZKxn75a7JK2DpWftDxvn2bvl\nLknrYN/BfY+ctT/syNEj7D24dyzP1+VNTJKkY/TJP/zkuj6fZ+6S1CDLXZIaZLlLUoMsd0lqkOUu\nSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUFDyz3JaUmuT3JHkv1J\n3rXMzBOSfDrJQpKbk5w5nriSpC66nLk/CLynqp4NnAO8LcmzB2beC+ytqu3ARcA/jDamJGk1hpZ7\nVX2/qm7tP/4pcCdw6sDYs4Ev92fuArYledKIs0qSOlrVNfck24CdwJ6BQ/uAP+rPnAU8Bdi6zNdf\nkmQ+yfzi4uJa8kqSOuhc7kkeB3wKeHdV/WTg8N8Am5PsBd4B3AY8NPg9qmpXVc1W1ezMzMwxxJYk\nraTT31BNsolesX+8qq4ePN4v+4v7swG+CXxjhDklSavQ5W6ZAB8B7qyqv3uUmc1JHtPffCvw1WXO\n7iVJ66TLmfsLgDcBt/cvu0Dv7pjTAapqDngWcFWSAvYDbxlDVklSR0PLvapuADJk5mvAM0YVSpJ0\nbHyHqiQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIa\nZLlLUoMsd0lqkOUuSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDhpZ7ktOSXJ/kjiT7k7xr\nmZmTk3w+yb7+zMXjiStJ6mJjh5kHgfdU1a1JTgJuSXJdVd2xZOZtwB1V9aokM8B/J/l4Vf3fOEJL\nklY29My9qr5fVbf2H/8UuBM4dXAMOClJgMcBP6L3S0GSNAFdztwfkWQbsBPYM3DoSuBzwPeAk4DX\nVdXREeSTJK1B5xdUkzwO+BTw7qr6ycDhlwN7gScDO4Arkzx+me9xSZL5JPOLi4vHEFuStJJO5Z5k\nE71i/3hVXb3MyMXA1dVzAPgmcMbgUFXtqqrZqpqdmZk5ltySpBV0uVsmwEeAO6vq7x5l7B7gpf35\nJwHPBL4xqpCSpNXpcs39BcCbgNuT7O3vey9wOkBVzQHvBz6a5HYgwGVVdWgMeSVJHQwt96q6gV5h\nrzTzPeBlowolSTo2vkNVkhpkuUuTsLAb/v5M+KvNvc8LuyedSI1Z1X3ukkZgYTd8/p1w5Oe97fu/\n09sG2P7ayeVSUzxzl9bbl/76F8X+sCM/7+2XRsRyl9bb/feubr+0Bpa7tN5O3rq6/dIaWO7Senvp\n+2DTib+8b9OJvf3SiFju0nrb/lp41RVw8mlAep9fdYUvpmqkvFtGmoTtr7XMNVaeuUtSgyx3SWqQ\n5S5JDbLcJalBlrskNShVNZknThaBb6/hS7cA0/pvxU9rtmnNBdObbVpzwfRmm9ZcML3Z1pLrKVU1\n9E/ZTazc1yrJfFXNTjrHcqY127TmgunNNq25YHqzTWsumN5s48zlZRlJapDlLkkNOh7LfdekA6xg\nWrNNay6Y3mzTmgumN9u05oLpzTa2XMfdNXdJ0nDH45m7JGmIqS33JN9KcnuSvUnmlzmeJFckOZBk\nIclzpyjbG/qZbk9yY5LnTEOuJXPPS/JgkgvXI1fXbEl+v398f5KvTEOuJCcn+XySff1cF69Trs1J\nPpnkriR3Jvm9geOTXP/Dsk1q/a+Ya8ncJNb/0GwjX/9VNZUfwLeALSscfyVwLRDgHGDPFGV7PvCE\n/uNXrFe2Ybn6MxuALwNfAC6cop/ZZuAO4PT+9m9OSa73An/bfzwD/Ah4zDrkugp4a//xY4DNA8cn\nuf6HZZvU+l8xV3//pNb/sJ/ZyNf/1J65d3AB8E/VcxOwOckpkw4FUFU3VtWP+5s3AdP0J3beAXwK\nODjpIANeD1xdVfcAVNW05CvgpCQBHkev3B8c5xMmORl4EfARgKr6v6q6b2BsIuu/S7ZJrP+OPzOY\nwPrvmG3k63+ay72Af09yS5JLljl+KvCdJdv39veth2HZlnoLvTOs9bBiriSnAq8BPrROeZYa9jN7\nBvCEJP/Rn7loSnJdCTwL+B5wO/Cuqjo65kxPBRaBf0xyW5IPJ3nswMyk1n+XbEut1/ofmmuC67/L\nz2zk63+ay/2FVbWD3v/WvS3JiyYdaIlO2ZK8hN7ivmxKcn0AuGwdymk5w7JtBH4X+APg5cBfJnnG\nFOR6ObAXeDKwA7gyyePHnGkj8FzgQ1W1E/hf4C/G/Jxddc62zuu/S65Jrf8u2Ua+/qe23Kvqu/3P\nB4FPA2cNjHwXOG3J9tb+vmnIRpLtwIeBC6rqh1OSaxb45yTfAi4EPpjk1VOS7V7gX6vqf6vqEPBV\nYOwvxHXIdTG9/12uqjoAfBM4Y8yx7gXurao9/e1P0iuHpSa1/rtkm8T675JrUuu/S7aRr/+pLPck\nj01y0sOPgZcB/zUw9jngov5dA+cA91fV96chW5LTgauBN1XV18edqWuuqnpqVW2rqm30FtifVtVn\npiEb8FnghUk2JvkN4GzgzinIdQ/w0v7Mk4BnAt8YZ66q+gHwnSTP7O96Kb0X25aayPrvkm0S679L\nrkmt/47/PUe+/qf1b6g+Cfh07zUsNgKfqKovJrkUoKrm6L3a/UrgAPAzemdY05LtfcAT6Z0ZADxY\n4/9Hi7rkmpSh2arqziRfBBaAo8CHq2qwaNc9F/B+4KNJbqd3Z8pl/TOrcXsH8PEkj6H3y+TiKVn/\nXbJNYv13yTVJK2Ybx/r3HaqS1KCpvCwjSTo2lrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUu\nSQ36f7tuy7m1DTOIAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Compute the mean of each class. axis=0 means to compute the average row vector \n", "mu0 = np.mean(X[0:50,:],axis=0)\n", "mu1 = np.mean(X[50:100,:],axis=0)\n", "mu2 = np.mean(X[100:150,:],axis=0)\n", "plt.plot(mu0[0],mu0[1],'x',mu1[0],mu1[1],'o',mu2[0],mu2[1],'^')" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[array([[ 0.12424898, 0.10029796, 0.01613878, 0.01054694],\n", " [ 0.10029796, 0.14517959, 0.01168163, 0.01143673],\n", " [ 0.01613878, 0.01168163, 0.03010612, 0.00569796],\n", " [ 0.01054694, 0.01143673, 0.00569796, 0.01149388]]),\n", " array([[ 0.26643265, 0.08518367, 0.18289796, 0.05577959],\n", " [ 0.08518367, 0.09846939, 0.08265306, 0.04120408],\n", " [ 0.18289796, 0.08265306, 0.22081633, 0.07310204],\n", " [ 0.05577959, 0.04120408, 0.07310204, 0.03910612]]),\n", " array([[ 0.40434286, 0.09376327, 0.3032898 , 0.04909388],\n", " [ 0.09376327, 0.10400408, 0.07137959, 0.04762857],\n", " [ 0.3032898 , 0.07137959, 0.30458776, 0.04882449],\n", " [ 0.04909388, 0.04762857, 0.04882449, 0.07543265]])]" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Compute the covariance of each class.\n", "Sig0 = np.cov(X[0:50,:],rowvar=False)\n", "Sig1 = np.cov(X[50:100,:],rowvar=False)\n", "Sig2 = np.cov(X[100:150,:],rowvar=False)\n", "[ Sig0, Sig1, Sig2 ]" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(400, 250, 2)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a coordinate system on which we can calculate the Gaussian pdf\n", "x0coords, x1coords = np.mgrid[4:8:0.01,2:4.5:0.01]\n", "# Combine x0coords and x1coords into an array that we can use in the plt.contourf function\n", "# ... this syntax is copied from the scipy.stats.multivariate_normal web page\n", "coords = np.empty(x0coords.shape + (2,))\n", "coords[:,:,0] = x0coords\n", "coords[:,:,1] = x1coords\n", "coords.shape" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Remember the multivariate normal is \n", "$$p_{X|Y}(\\vec{x}|y) = \\frac{1}{(2\\pi)^{D/2}|\\Sigma_y|^{1/2}}e^{-\\frac{1}{2}(\\vec{x}-\\vec\\mu_y)^T\\Sigma_y^{-1}(\\vec{x}-\\vec\\mu_y)}$$" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHP1JREFUeJzt3X/0XHV95/HnK5CECIHIj8qPkKQLFqzWAocCBbQpioUQ\nwNNFBUUKWxvxuCuuuGy1lRVXt7anpVhRKVtXRShWsVLEQMUiKmcLFrIRFggraCDBIBBJQgATAu/9\n436+eHOd+c6dmTszd+68Hud8z3fm3s/c+5479/uaz3zunftVRGBmZs0yY9QFmJlZ9RzuZmYN5HA3\nM2sgh7uZWQM53M3MGsjhbmbWQA73BpL0YUlXDmldx0j6oaTNkt5Yov0iSSFpx2HUN2ySFqRtscOo\na+lGN/uMpFskvaPNPEn6nKQnJX2/2ip/aV2fl/TRQa5jnE1UuEt6q6Q70h/fOkk3SDq2guWO7U4m\nabWk1/exiI8Al0bELhFx7QCWPy1JcyVdnNbztKSHJV0j6chBrXM6EfFw2hbPj2L9NXAscDwwPyKO\nGHUxVZJ0iKQ7JT2Tfh8y6pqmMzHhLul9wCXA/wBeBiwAPgWcMsq6OhmDHu5C4J5RrFjSbOBm4DeA\npcCuwCuALwEnjqImYyGwOiKe7vaBdd7XJc0C/gm4Engp8AXgn9L0eoqIxv8AuwGbgTdN02Y2Wfj/\nJP1cAsxO8xYDa4HzgceAdcA5ad4y4Dlga1rH19P0VwC3ABvIwu+U3LpuAd6Ru382cGvufgDvBn4I\n/LhFrYtSm2Wp1nXA+3PzPwxcmbt/SqphQ1r3K9L0LwIvAM+m2i9os23+CHgA+BlwHbBvmv5g4fGz\nC4/7peXnav8D4GHgCeBPco+ZAfxxWvZ64MvA7m3qekd67jt3eP0/AawBNgF3Aq/Jzfs88NHc/cXA\n2tz9/wo8AjwF3A+8Lk0/ArgjLfOnwMWF12bHdP8c4L70+B8B7yyuixb7VZvncQvwUeB/p+35dWAP\n4KpUx78Bi3Ltj07TNqbfR+fm/SrwnVTXTcClhX3mqLSeDcAPgMXt9t/c9D8Efg48n+q7aLr9p8y+\nntocm6tlDXB28bUjC9zrgceBJ9Pt+YW/sR+l5/tj4G1p+oFpO2wk2xf/oU0Nb0j7gXLTHgZOGFWu\ndfoZeQFDeZJwArBt6g+uTZuPALcBvwLslXam/57mLU6P/wgwE1gCPAO8tLiTpfsz0878QWAWcFza\nqQ5K87f746B1uN8E7A7MaVHrotTmamBnsp7r48Dr0/wPT/2hAr8GPE32UXkmWcA+AMxK81dPPa7N\ndjku7fSHkb0BfhL4bm5+p8dvNz9X+/8E5gC/CWzhF28456XXYX5a398CV7dZ9peAz5d4/c8kC8Ed\nyYL0UWCnNq/dYlK4AweRhcm+udoPSLf/FXh7ur0LcFTh+U2F+0nAAYCA30n7zWFl9qsWz+OW9Nod\nQNZhuRf4f8Dr03O7Avhcars7Wci9Pc07I93fI1f/xWkbv5Zs/5zaZ/Yje2NdQvZme3y6v1er/bdQ\n49lsvy932n867esLU21npG20B3BI8bVL0/898BJgLvAV4No0b2eyN7+pv799gFem21cDf5Ke507A\nsW2e138GbihM+zpw/qhyrdPPpAzL7AE8ERHbpmnzNuAjEfFYRDwOXET2hzHluTT/uYhYTtYzOajN\nso4i+4P/eERsjYibyXoSZ3RR859FxM8i4tlp2lwUEU9HxN3A59os/y3ANyLipoh4DvhLslA9umQd\nbwP+V0SsiIgtwAeA35a0qOwTmab2ZyPiB2Q9w99M088l68mvTev7MHBam4/se5IFNfDimOgGSZsk\n3T81PSKujIj1EbEtIv6KLGTavXZ5z6e2vy5pZkSsjogH07zngAMl7RkRmyPitlYLiIhvRMSDkfkO\n8E3gNbkm3exXkIX3gxGxEbgBeDAivpX27a8Ah6Z2JwE/jIgvpud9NbAKOFnSAuC3gA9FxJaI+C5Z\nUE05E1geEcsj4oWIuInsU8qSEtusqMz+M92+/lbgWxFxddpG6yNiZbFRmv7ViHgmIp4CPkb2Zjrl\nBeBVkuZExLqImBpKfI7sDWTfiPh5RNza5nnsQta7z9tE9kZSS5MS7uuBPTuM6e0LPJS7/1Ca9uIy\nCm8Oz5C94O2WtSYiXigsb7/yJbOmyzbFevO1vPi8Uk1ruqil+PjNZNuzm+fSyqO52/ltuRD4Wgrp\nDWRDGs+THScpWk/WC5uqbWVEzAN+nyyUAZD0fkn3SdqYlrkb2RvDtCLiAeC9ZG8wj0n6kqSpbfyH\nZJ+KVkn6N0lLWy1D0omSbpP0s7TuJYV1d7NfQTYENOXZFvenHlvcn+EX++C+wJOx/bh4vu1C4E1T\nr0Gq+1hy27oLZfaf6fb1/cmG6KYl6SWS/lbSQ5I2Ad8F5knaIT3Pt5B1HNZJ+oakg9NDLyD7VPV9\nSfdI+g9tVrGZ7JhO3m5knypqaVLC/V/JPvpPd6reT8h26ikL0rQyipfW/Amwv6T89l1ANmYH2TDJ\nS3Lz9i6xzFb2Lyy/Vb3bPS9JSo+bqqXTeoqP35nsk9AjbR+xvW4vO7oGODEi5uV+doqIVuv7F+AN\nqaaWJL2G7A/4zWTDHfPIemBKTaZ9LSLi7yPiWLJtEMCfp+k/jIgzyIbx/hy4plhHOuD7VbJPSy9L\n616eW/cgFfdn+MU+uA54aaHeBbnba4AvFl6DnSPi4/3W0Wb/mW4fWUM2DNXJ+WSfeI6MiF3Jhpog\nbeuI+OeIOJ7sDWoV2bAgEfFoRPxRROwLvBP4tKQDWyz/HuDV6e9nyqsZ0ckEZUxEuKePsBcCn5L0\nxvQuPzP1qv4iNbsa+FNJe0naM7Uve674T4F/l7t/O1kP7IK0nsXAyWRjxAArgd9PdRxI1gvsxYfS\nMl5JduDuH1q0+TJwkqTXSZpJ9kewheyYQqvai64GzklDHrPJzja6PSJWl6yx0/KLLgM+JmkhQHo9\nTm3T9gqyoPqapFdJ2kHSTsDhuTZzyca1Hwd2lHQh2/fAVgJLJO0uaW+ynjpp3QdJOi4975+T9Yxf\nSPPOlLRX+iS0IT0k/0kNsuMts9O6t0k6kezA3DAsB34tnf67o6S3AL8OXB8RD5ENs1wkaVY6Hfjk\n3GOvJBu++b2pbSppsaT5PdTR7/5zFfB6SW9Oz2OPNqcgziV7fTZI2h34b1MzJL1M0qnpjWULWS98\n6nV8U+55PUn2RlN8HSE7zvA88B5JsyW9J7W9ueTzGLqJCHeANNb6PuBPyf7Y1gD/EZg6N/ujZDv8\nXcDdwIo0rYzPko3LbpB0bURsJftjOZHsYNKngbMiYlVq/9dkZ9f8lOyUqqt6fFrfITvA9i/AX0bE\nN4sNIuJ+sjHUT6ZaTgZOTjUC/BnZm9oGSe9v8fhvAR8i64GuI+tFnd5FjdMuv4VPkJ1R8U1JT5Ed\nXG15znpE/Bz4XbIDi98gGwO9n2w8+c2p2T8DN5IdeHyILKTzwwBfJBvzX002Hp5/g5wNfJxsuz1K\n1kv/QJp3AnCPpM2p5tOLY8Zp7Pc9ZG+wT5KNH19XYhv0LSLWk50eej7ZMMgFwNKIeCI1eSvZdv0Z\nWRBekXvsGuBUshMCpv5W/gs95EW/+09EPEw2lHV+qnUlvzg+k3cJ2bGkJ8j2mRtz82aQ/e3/JC3j\nd4B3pXm/BdyeXsfrgPMi4kct6thK9sn/LLI387OBN+b+jmpHEf5nHeMmHYz6MTCzw0FiM5tQE9Nz\nNzObJA53M7MG8rCMmVkDueduZtZAI7tQz6wZc2LODrX9cpeZWS1t2vb4ExGxV6d2Iwv3OTvM5eg9\n3zSq1ZuZjaUbH/108ZvHLXlYxsysgRzuZmYN5HA3M2sgh7uZWQM53M3MGsjhbmbWQA53M7MGcrib\nmTWQw93MrIEc7mZmDeRwNzNrIIe7mVkDOdzNzBrI4W5m1kAOdzOzBnK4m5k1kMPdzKyBHO5mZg3k\ncDczayCHu5lZAznczcwayOFuZtZADnczswZyuJuZNZDD3cysgRzuZmYNtOOoC7Dm2nrw/J4eN2vV\n2oorMZs8DnerTK9h3m45Dnmz3jncrS9VBXq7ZTvgzXrjcLeeDDLUW63HIW/WHYe7dWVYoW5m/fHZ\nMlbK1oPnjzTY/aZi1h2Hu01r1KGeV5c6zMaBh2WsJQep2Xhzz922U6eeeit1rs2sThzu9iIHp1lz\neFjGHOpmDeSe+4RzsJs1U+meu6QdgDuARyJiaWGegE8AS4BngLMjYkWVhVq1BhXqGw+Yvd393R7c\nMpD1mNn0uhmWOQ+4D9i1xbwTgZennyOBz6TfVkNVBHsxxMu2qyLsfVkCs85Khbuk+cBJwMeA97Vo\ncipwRUQEcJukeZL2iYh11ZVq/eo31MsGeplluEdvNlhlx9wvAS4AXmgzfz9gTe7+2jRtO5KWSbpD\n0h1bX3i2q0KtP/0E+8YDZlcS7MVlmtngdAx3SUuBxyLizn5XFhGXR8ThEXH4rBlz+l2cldRrsA8i\n1IvLN7PBKDMscwxwiqQlwE7ArpKujIgzc20eAfbP3Z+fptkI9RPqZjbeOvbcI+IDETE/IhYBpwM3\nF4Id4DrgLGWOAjZ6vH20egn2QffU263TzKrX85eYJJ0LEBGXAcvJToN8gOxUyHMqqc560m2wO2DN\nmqercI+IW4Bb0u3LctMDeHeVhVn3xjXUNx4w22fPmFXM31BtiHEN9l74HHezznxtmQboJth7DfWn\nFqp027kPRU/rMLPqONzH3CCDvZtALz7OAW82Wg73MVY22LsJ9V4DvdVyHPBmo+Mx9zFVdbA/tVCV\nBXt+mWY2Gg73MVRlsA8i1IvLN7Ph87DMGKk61MeRz5QxK8c99zHhYDezbjjcx0BVwT7oIZjp1mtm\nw+VhmZqrMtjL2rJga6l2sx+eVXqZZjZcDvcaqyLYy4Z62UAvPmaYAe/xdrPyHO41VSbYq+it9xLq\nxce7B29WPx5zr6FhBPuWBVv7Dvb8ssysXtxzr5l+g71MqNdNmStCekjGrDvuudfIIIO9yp56u+Wb\nWX043Gti0MFuZpPF4V4DdQr2RfMfZ9H8x7t6zKB5SMasex5zH7FBBXvZUG8X5Pnpq9fuVWpZvfB/\nYDIbDId7zQ0i2LvtmU+17zXkfelfs+FzuI9Qp157u2DvdRimbsMtZXhIxqw3DvcRqTrYhxHqi+Y/\nXukQjYdkzAbHB1RHYByDfRTcazfrnXvuQzasYK9LqHu83Ww03HMfoiYHe7fXl+k0JONeu1l/HO5D\n0uRgN7P68bDMENQp2I/fe1XL6Tc9enDHx3ar3ZCMD6SaDZ7Dvaa6CfZ+Qr04fxAh3y0PyZj1z+E+\nYL322lvpJdg7hXoVfD13s/pxuA9QlcMx3Qb7MEJ9Or0OybjXblYNH1AdkCYG+yCvMWNm1XLPfQBG\nFeyj6K23GpJxr91s9Nxzr9gkBXuVHOxm1XK418A4BHurIZkqe+1mVi2He4V66bWPQ7APmnvtZtVz\nuFekqlMexznY3Ws3q4+O4S5pJ0nfl/QDSfdIuqhFm8WSNkpamX4uHEy59TSI67JPqUOwlx2S6YV7\n7WaDUeZsmS3AcRGxWdJM4FZJN0TEbYV234uIpdWXON76PYDayqCCvZ9vp7rXblYvHcM9IgLYnO7O\nTD++jmtS5n+gFvUzzj7soZh+eu0+9dFsdEqd5y5pB+BO4EDgUxFxe4tmR0u6C3gEeH9E3NNiOcuA\nZQA7zdil56LrYtgHUMsG+2m7rnjx9jWbDiv1mLK6OUPGzEan1AHViHg+Ig4B5gNHSHpVockKYEFE\nvBr4JHBtm+VcHhGHR8Ths2bM6afu2qv6AGqZYD9t1xXbBfvUtDJaDcn0841U99rNRqurs2UiYgPw\nbeCEwvRNEbE53V4OzJS0Z2VV1tB0vfaqD6CWDfZBq6rX7mA3G7wyZ8vsJWleuj0HOB5YVWiztySl\n20ek5a6vvtx6GOQ4ey/6DfZh99rNbPDKjLnvA3whjbvPAL4cEddLOhcgIi4DTgPeJWkb8CxwejoQ\n2zjDPp+9U6+9U7B3GnMve4ZM2V67h2PM6qHM2TJ3AYe2mH5Z7valwKXVljZ+hj0cM6ihmDK9dg/H\nmNWbv6HahToNx5QJ9l567YM89dHMhsfhXlLdhmOGxcMxZuPJ4V6BOg7HVNVrL/JwjNl4cLiXUKfh\nmDI8HGNmDvc+1W04xmfHmBk43DuqqtdeNN3/QO1Vr5caKPba/WUls/HncJ/GIK8d00o/vfYywV71\nOLuHY8zqy/8gu0dlh2NaqbLXXra3PuxxdvfazUbL4d7GsA+idtNr73b4pZ9g93CM2XjysEwP6tJr\nL6PqA6gwfa/dwW5WDw73Fsbt1Md22gW7x9nNms/hXlDVN1G7MYhvo3YT7B5nN2seh3sF+vnvSoPQ\nb7D7fHaz8edwzxlFr71qgwj2ThzsZvXjcO9TnXrtgwp2j7ObjR+HezLuvfZRBbt77Wb15PPc+1Dm\nMgODNt2pjg52s8nlnjvV9tqHefpjFcHejoPdbLy55z4EVY+3d/piUjfB7gOoZs008T33XnvtVQ7J\nlP0WaZm2VQS7D6CajT/33CvUz5DMTY8e3PbLTGXDfxjB7l672XiY6HCvQ689r5sefF67ywk42M0m\n18QPy4w7B7uZtTKx4d7LxcHqxsFuZu1M9LDMdOr8paXpruroYDczmNCeez+99nbj7dMdTC1zid2y\nHOxmVoZ77mOi0xuET3c0szyHewuDGJJZvXavnr7M1GuoQ3/B7l672XibuHAf5YHUsgFfdhjHwW5m\n7UxcuPejivPbqxp/7+VyAg52s8kxUQdUy/Taex2S6eaiXP2Y/fAsB7uZdTRR4T7uehmGAQe72STy\nsMwY6PSpwMFuZkUTE+7j+I3UfkIdHOxmk6zjsIyknSR9X9IPJN0j6aIWbSTpbyQ9IOkuSYcNptzB\n6vcUyKrG3acbV5/iYDez6ZTpuW8BjouIzZJmArdKuiEibsu1ORF4efo5EvhM+j1xZj88q+dL/5Z9\nc3Cwm1knHcM9IgLYnO7OTD/FdDkVuCK1vU3SPEn7RMS6SqsdE1MhXSbku+ntVxHq4GA3mwSlxtwl\n7QDcCRwIfCoibi802Q9Yk7u/Nk3bLtwlLQOWAew0Y5ceS+5eVePtcx+Krs51r/L0SAe7mXWj1KmQ\nEfF8RBwCzAeOkPSqXlYWEZdHxOERcfisGXN6WcTEmftQONjNrGtdneceERuAbwMnFGY9Auyfuz8/\nTbM+lPnn1Q52M2ulzNkye0mal27PAY4Hiv/s8zrgrHTWzFHAxkkdb69C2d66g93M2ikz5r4P8IU0\n7j4D+HJEXC/pXICIuAxYDiwBHgCeAc4ZUL1dq/r89m7H3btddhndXK7XwW42mcqcLXMXcGiL6Zfl\nbgfw7mpLq6+qA75sqIOHYcysnIn5hmrV+g34bgId3Fs3s+443PswFdBlQ77bQJ/iYDezbjncK9Br\naHfS7b/Cc7Cb2RRf8jenTv9X1MFuZv1wz71mHOpmVgWHe0308qnBwW5m7TR6WKaXc9xHMTTjYDez\nqjU63Hs1rIDv5lumeQ52M+uk0cMys1at7fkbqrs9uKXvf97Rbrm9cqibWVnuuU+jyh58r730KQ52\nM+tGo3vuVeinB1/Vm4OD3cy65XAvoVVI5wN/UGP0DnUz65XDvUeDPujqYDezfjjca8ahbmZVcLjX\nhEPdzKrU+LNlxiE0x6FGMxsv7rmPkEPdzAal8T13qF+Izlq1tnY1mVmzuOc+RA50MxuWiei5w2iD\n1T11Mxu2iQl3GH7AO9TNbFQmKtxhOAHvUDezUZvIMfd+rhY53TLNzOpiIsMdtg/jXoLeYW5mdTax\n4Z7noDazppm4MXczs0ngcDczayCHu5lZAznczcwayOFuZtZADnczswZyuJuZNZDD3cysgRzuZmYN\n5HA3M2ugjuEuaX9J35Z0r6R7JJ3Xos1iSRslrUw/Fw6mXDMzK6PMtWW2AedHxApJc4E7Jd0UEfcW\n2n0vIpZWX6KZmXWrY889ItZFxIp0+yngPmC/QRdmZma962rMXdIi4FDg9hazj5Z0l6QbJL2yzeOX\nSbpD0h1bX3i262LNzKyc0pf8lbQL8FXgvRGxqTB7BbAgIjZLWgJcC7y8uIyIuBy4HGC3mb8SPVdt\nZmbTKtVzlzSTLNivioh/LM6PiE0RsTndXg7MlLRnpZWamVlpZc6WEfBZ4L6IuLhNm71TOyQdkZa7\nvspCzcysvDLDMscAbwfulrQyTfsgsAAgIi4DTgPeJWkb8CxwekR42MXMbEQ6hntE3AqoQ5tLgUur\nKsrMzPrjb6iamTWQw93MrIEc7mZmDeRwNzNrIIe7mVkDOdzNzBrI4W5m1kAOdzOzBnK4m5k1kMPd\nzKyBHO5mZg3kcDczayCHu5lZAznczcwayOFuZtZADnczswZyuJuZNZDD3cysgRzuZmYN5HA3M2sg\nh7uZWQM53M3MGsjhbmbWQA53M7MGcribmTWQw93MrIEc7mZmDeRwNzNrIIe7mVkDOdzNzBrI4W5m\n1kAOdzOzBnK4m5k1kMPdzKyBHO5mZg3UMdwl7S/p25LulXSPpPNatJGkv5H0gKS7JB02mHLNzKyM\nHUu02QacHxErJM0F7pR0U0Tcm2tzIvDy9HMk8Jn028zMRqBjzz0i1kXEinT7KeA+YL9Cs1OBKyJz\nGzBP0j6VV2tmZqWU6bm/SNIi4FDg9sKs/YA1uftr07R1hccvA5alu1tufPTT/7eb9Y/InsAToy6i\nBNdZLddZLddZnYVlGpUOd0m7AF8F3hsRm3qpKCIuBy5Py7sjIg7vZTnD5Dqr5Tqr5TqrNS51llHq\nbBlJM8mC/aqI+McWTR4B9s/dn5+mmZnZCJQ5W0bAZ4H7IuLiNs2uA85KZ80cBWyMiHVt2pqZ2YCV\nGZY5Bng7cLeklWnaB4EFABFxGbAcWAI8ADwDnFNiuZd3Xe1ouM5quc5quc5qjUudHSkiRl2DmZlV\nzN9QNTNrIIe7mVkDDSXcJe0g6f9Iur7FvNpcuqBDnYslbZS0Mv1cOKIaV0u6O9VwR4v5tdieJeqs\ny/acJ+kaSask3Sfptwvz67I9O9U58u0p6aDc+ldK2iTpvYU2I9+eJesc+fbsV1dfYurDeWTfbN21\nxbw6XbpgujoBvhcRS4dYTzu/GxHtvmhRp+05XZ1Qj+35CeDGiDhN0izgJYX5ddmeneqEEW/PiLgf\nOASyjhLZ6dBfKzQb+fYsWSfUY//s2cB77pLmAycBf9emSS0uXVCiznFRi+05DiTtBryW7FRfImJr\nRGwoNBv59ixZZ928DngwIh4qTB/59ixoV+fYG8awzCXABcALbea3u3TBsHWqE+Do9FHyBkmvHFJd\nRQF8S9Kd6XIORXXZnp3qhNFvz18FHgc+l4bj/k7SzoU2ddieZeqE0W/PvNOBq1tMr8P2zGtXJ9Rr\ne3ZtoOEuaSnwWETcOcj19KtknSuABRHxauCTwLVDKe6XHRsRh5B9vH23pNeOqI5OOtVZh+25I3AY\n8JmIOBR4GvjjEdTRSZk667A9AUjDRqcAXxlVDWV0qLM227NXg+65HwOcImk18CXgOElXFtrU4dIF\nHeuMiE0RsTndXg7MlLTnkOskIh5Jvx8jGyc8otCkDtuzY5012Z5rgbURMXUhvGvIQjSvDtuzY501\n2Z5TTgRWRMRPW8yrw/ac0rbOmm3Pngw03CPiAxExPyIWkX38uTkiziw0G/mlC8rUKWlvSUq3jyDb\nduuHWaeknZVdU5/0sfwNQPHKmiPfnmXqrMP2jIhHgTWSDkqTXgfcW2g28u1Zps46bM+cM2g/1DHy\n7ZnTts6abc+eDOtsme1IOhf6unTBUBTqPA14l6RtwLPA6TH8r/e+DPha2ud2BP4+Im6s4fYsU2cd\ntifAfwKuSh/RfwScU8PtWabOWmzP9GZ+PPDO3LTabc8SddZie/bDlx8wM2sgf0PVzKyBHO5mZg3k\ncDczayCHu5lZAznczcwayOFuZtZADnczswb6/wWh4FCr/JK7AAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Calculate the Gaussian pdfs at the same set of points\n", "pdf0 = stats.multivariate_normal(mu0[0:2],Sig0[0:2,0:2]).pdf(coords)\n", "pdf1 = stats.multivariate_normal(mu1[0:2],Sig1[0:2,0:2]).pdf(coords)\n", "pdf2 = stats.multivariate_normal(mu2[0:2],Sig2[0:2,0:2]).pdf(coords)\n", "plt.contourf(x0coords,x1coords,pdf0)\n", "plt.title('Contour plot of the Gaussian model for class 0')" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH3xJREFUeJzt3X2wZHV95/H3Z4Z5QBkgKyjCMIwBFIm64BowuDFEQ1YG\nxGwtW6Lxic1mglEXK1js6m4ZQ7CiVYmrRpFMNCIPSlDBIItW2NJJJBYoIEERdHlmRp6VYRCc4eG7\nf5xz8UzTffv06XP6/M7pz6vq1tzb3bf7d093v+fXvz73XEUEZmbWL0vaHoCZmdXPcTcz6yHH3cys\nhxx3M7MectzNzHrIcTcz6yHHvWWSHpb0q22PY1KSTpd0v6S7S17+A5LObXA8t0n6naauf8xtnyXp\n9Pzz35T0o2nHVdxektbkj5Ol+dcbJf3Xusa/yBjeJunyRc7/j5LuzMd2aNPjKdzu70v6x1ndXlc5\n7ovIn5jbJe0xcPr3JIWktdPeRkTsEhG3THs9syRpDXAKcHBE7DXk/CMlbWrw9p+KaWoi4lsR8YKa\nr/OO/HHyRJ3XW4O/BN6Zj+17TdyApLX5c22nhdMi4ryI+N0mbq9PHPfxbgXesPCFpBcDz2hvOElY\nAzwQEfe2PZAqiqGwqewHXN/2IGw4x328c4C3FL5+K3B28QKSjsln8w/lL1M/UDjv9ZJulbRr/vXR\nku6WtGf+dUg6IP/8LElnSPpa/lL3XyTtJemjkn4m6cbiy9/i9xa+f2F54EhJmySdKuleSXdJ+j1J\n6yT9WNJPJb1v1A8taTdJZ0u6T9Ltkv6XpCX5EsNlwN75GM8a+L5nAl8rnP+wpL3zs5fn17lV0vWS\nXlb4vr0lfTm/vVsl/bcR41oP/D5wan7dXy2cfYik6yRtkfT3klYObIv/ni8jfTY//VhJ10p6UNK3\nJb1k0vEMGd/IVy2SXphf1xsm/JmfNnsF9ssfH1sl/WPx1aWk4/Lt+2C+hPPCgTFszM+7XtJxhfOe\nJeni/HH8HWD/EeNZIelhYCnwr5Juzk8v83g8pfB4PLFw2Z0l/VX+WNsi6XJJOwP/nF/kwfz+/g0N\nLBdJOkLSd/Pv+66kIwrnbZT058O2laSVks6V9EC+Pb4r6TnDfuZOigh/jPgAbgN+B/gR8EKyB/Mm\nshlLAGvzyx0JvJjsP8uXAPcAv1e4nvOAs4BnAT8Bji2cF8AB+ednAfcD/w5YCXyD7JXDW/LbPh34\n5rDvLXz/6YUxPQ68H1gG/CFwH/B5YBXwa8CjwPNG/OxnA/+QX3Yt8GPgDwrXvWmR7fa084EPAL8A\n1uU/y18AV+TnLQGuzse6HPhV4BbgP4y4/qd+zoH76jvA3sC/AW4AThrYFh8GVgA7A4cC9wKH5+N5\na34dK6YZz+DPzi8fQy8F7li478fdRr69zs0/X5vf1zvlX28Ebgaen/8sG4EP5ec9H/g5cFR+v58K\n3JTfxrL88/flX78K2Aq8IP/e84ELgGcCLwI2A5cvcj8PPv7KPB5Py8exDngE+JX8/E/mP8c++f1x\nRH5f7PCz55d928K48vv6Z8CbgZ3IXmX/DHhWiW31R8BXyV6JLyV73u3adnfq+vDMvZyF2ftRZNHY\nXDwzIjZGxPcj4smIuA74AvBbhYu8g+yJtBH4akRcsshtXRQRV0fEL4CLgF9ExNmRrbf+PVmUynoM\n+GBEPEb2xN0D+FhEbI2I64EfAv928JuUvXF3AvDe/LK3AX9F9gSaxuURcWn+s5xTuO1fB/aMiNMi\nYntk70H8bT6GSXw8In4SET8le9IeUjjvSeBPI2JbRDwKrAf+JiKujIgnIuJzwDbg5TWOZ8FvAhcD\nbync99Pexmcj4sf5z3JB4Wd9PfB/IuKy/H7/S7KoHZH/bLuQxW17RHwDuAR4Q36f/yfg/RHx84j4\nAfC5ij/vKI8Bp0XEYxFxKfAw8AJJS4D/ApwcEZvz++PbEbGtxHUeA/y/iDgnIh6PiC8ANwKvLVxm\n1LZ6jGzCdUB+m1dHxEM1/ayt89pjOeeQvTx8HgNLMgCSDgc+RDbbWU424/jiwvkR8aCkLwJ/QvYE\nWsw9hc8fHfL1LhOM+4H45Ztwj464/mHXtwfZ7Or2wmm3k82qplHcs+YRYGW+1LAf2TLOg4XzlwLf\nmvL69y58fV/+H+aC/YC3SnpX4bTl+fc8UdN4FpwE/FNEbBy4/WluY/BnXbgf96Zwv0XEk5LuJLvv\nHgfujIgnC9+7cL/uSdaDOwfOq9MDEfH4kHHvQfZK9eYK17nDz5sbfKyO2lbnAPsC50vaHTgX+J/5\nf4qd55l7CRFxO9nyyDrgwiEX+TzZzGzfiNgNOBPQwpmSDiGbmXwB+HiNQ3uEHd/cfdqeKxXdTzar\n2a9w2hoGXrEsYtJDjd4J3BoRuxc+VkXEupquf9j33En2qqZ4m8/IZ36Tjmeck4A1kv73wO3XeRsL\nfkLhfpMksoBtzs/bN58pL1i4X+8ji/++A+dNourj8X6yJbtha/zj7usdft5cqcdq/grizyLiYLJX\nNsey4/trnea4l/cHwKsi4udDzlsF/DQifiHpMOCNC2fkb+qdS7bOeSKwj6Q/rmlM1wJvlLRU0mvY\ncSmosny2fwHwQUmrJO1H9qqj7H7q9wDPkrRbyct/B9iav+G5c/7zvEjSry9y/dP+bsDfAidJOlyZ\nZyp7Y3xVhfGMsxV4DfBKSR/KT6v7NhZcABwj6dWSlpHtsroN+DZwJVmAT5W0TNKRZMsX5+f3+YXA\nByQ9Q9LBZO9DTKLS4zF/JfF3wEfyN5mX5m+criD7T+dJRt/flwLPl/RGSTtJej1wMNly06Ik/bak\nF+dLUg+RTWieHPNtneG4lxQRN0fEVSPO/mPgNElbyd4gu6Bw3l+QvRT+VL6G+CbgdEkH1jCsk8me\nnA+S7UHylRquc8G7yN6YuwW4nOzVyd+V+caIuJHsVcot+V4Ie4+5/BNks6ZDyF4h3Q98Ghj1n8Nn\ngIPz6670M+f35R8CnyB7A+4msjfqqoynzO09SPaezdGS/ryJ28hv50dkj7G/zq/ztcBr8zX27fnX\nR+fnnUH2PsCN+be/k2zJ4m6yN0M/O+HNT/N4fA/wfeC7wE/J3vxeEhGPAB8E/iW/v18+8PM+QLYd\nTwEeIHsD+diIuL/Ebe4FfIks7DcA/0S2VNMLivAf6zAz6xvP3M3MeshxNzPrIcfdzKyHHHczsx5q\n7ZeYli/ZOXZeuqqtmzcz66SHHr/v/ojYc9zlWov7zktXccQe/7mtmzcz66Sv331Gqd8c9rKMmVkP\nOe5mZj3kuJuZ9ZDjbmbWQ467mVkPOe5mZj3kuJuZ9ZDjbmbWQ467mVkPOe5mZj3kuJuZ9ZDjbmbW\nQ467mVkPOe5mZj3kuJuZ9ZDjbmbWQ467mVkPOe5mZj3kuJuZ9ZDjbmbWQ467mVkPOe5mZj3kuJuZ\n9ZDjbmbWQ467mVkPOe5mZj20U9sDsP7aftDqSt+3/MZNNY/EbP447labqjEfdT2OvFl1jrtNpa6g\nj7puB96sGsfdKmky6sNux5E3m4zjbhOZVdTNbDreW8ZK2X7Q6lbD7v9UzCbjuNui2o56USrjMOsC\nL8vYUA6pWbd55m47SGmmPkzKYzNLieNuT3E4zfrDyzLmqJv1kGfuc85hN+un0jN3SUuBq4DNEXHs\nwHkCPgasAx4B3hYR19Q5UKtXU1Hfsv+KHb7e7eZtjdyOmS1ukmWZk4EbgF2HnHc0cGD+cTjwqfxf\nS1AdYR+MeNnL1RF7H5bAbLxScZe0GjgG+CDwJ0Mu8jrg7IgI4ApJu0t6bkTcVd9QbVrTRr1s0Mtc\nh2f0Zs0qu+b+UeBU4MkR5+8D3Fn4elN+2g4krZd0laSrtj/56EQDtelME/Yt+6+oJeyD12lmzRkb\nd0nHAvdGxNXT3lhEbIiIl0XEy5Yv2Xnaq7OSqoa9iagPXr+ZNaPMsswrgOMkrQNWArtKOjci3lS4\nzGZg38LXq/PTrEXTRN3Mum3szD0i3hsRqyNiLXAC8I2BsANcDLxFmZcDW7ze3q4qYW96pj7qNs2s\nfpV/iUnSSQARcSZwKdlukDeR7Qp5Yi2js0omDbsDa9Y/E8U9IjYCG/PPzyycHsA76hyYTa6rUd+y\n/wrvPWNWM/+Gak90NexVeB93s/F8bJkemCTsVaO+dT+Vvuyq26PSbZhZfRz3jmsy7JMEffD7HHiz\ndjnuHVY27JNEvWrQh12PA2/WHq+5d1TdYd+6n2oLe/E6zawdjnsH1Rn2JqI+eP1mNntelumQuqPe\nRd5Txqwcz9w7wmE3s0k47h1QV9ibXoJZ7HbNbLa8LJO4OsNe1rY120tdbsUdy0tfp5nNluOesDrC\nXjbqZYM++D2zDLzX283Kc9wTVSbsdczWq0R98Ps9gzdLj9fcEzSLsG9bs33qsBevy8zS4pl7YqYN\ne5mop6bMESG9JGM2Gc/cE9Jk2OucqY+6fjNLh+OeiKbDbmbzxXFPQEphX7v6Ptauvm+i72mal2TM\nJuc195Y1FfayUR8V8uLpt23as9R1VeG/wGTWDMc9cU2EfdKZ+cLlq0beh/41mz3HvUXjZu2jwl51\nGSa15ZYyvCRjVo3j3pK6wz6LqK9dfV+tSzRekjFrjt9QbUEXw94Gz9rNqvPMfcZmFfZUou71drN2\neOY+Q30O+6THlxm3JONZu9l0HPcZ6XPYzSw9XpaZgZTCftReNw49/bK7Dxr7vZMatSTjN1LNmue4\nJ2qSsE8T9cHzm4j8pLwkYzY9x71hVWftw1QJ+7io18HHczdLj+PeoDqXYyYN+yyivpiqSzKetZvV\nw2+oNqSPYW/yGDNmVi/P3BvQVtjbmK0PW5LxrN2sfZ6512yewl4nh92sXo57AroQ9mFLMnXO2s2s\nXo57jarM2rsQ9qZ51m5WP8e9JnXt8tjlsHvWbpaOsXGXtFLSdyT9q6TrJf3ZkMscKWmLpGvzj/c3\nM9w0NXFc9gUphL3skkwVnrWbNaPM3jLbgFdFxMOSlgGXS/paRFwxcLlvRcSx9Q+x26Z9A3WYpsI+\nzW+netZulpaxcY+IAB7Ov1yWf/g4rrkyfwN10DTr7LNeiplm1u5dH83aU2o/d0lLgauBA4BPRsSV\nQy52hKTrgM3AeyLi+iHXsx5YD7ByyS6VB52KWb+BWjbsx+96zVOff+mhl5b6nrIm2UPGzNpTKu4R\n8QRwiKTdgYskvSgiflC4yDXAmnzpZh3wFeDAIdezAdgAsNuyZ/e6CHW/gVom7MWoF08rE/hhSzLT\n/EaqZ+39V+VV67T8uClvot9QjYgHJX0TeA3wg8LpDxU+v1TSGZL2iIj76xtqWhZ7YNf9BmrVsNet\nrlm7n6Dd0Ea8xxk1Jj+mnm5s3CXtCTyWh31n4CjgwwOX2Qu4JyJC0mFke+E80MSAU9DkOnsV04Z9\n1rN2S0uKEZ/U4M/g2JebuT8X+Fy+7r4EuCAiLpF0EkBEnAkcD7xd0uPAo8AJ+RuxvTPr/dnHzdrH\nhX3ckkzZPWTKztq9HJO2PoS8jOLPOa+PuTJ7y1wHHDrk9DMLn38C+ES9Q+ueWS/HNLUUU2bW7uWY\n9M1LyMeZ19D7qJATSGk5pkzYq8zam9z10ZrlmI+3sI3mIfKOe0mpLcfMipdj0uagVzMPkXfca5Di\nckxds/ZBXo5pl2Nerz5H3nEvIaXlmDK8HNMvDnrz+hh5x31KqS3HeO+YfnDQ27H9oNW9ecw67mPU\nNWsftNjfQK2q6qEGBmft/mWldjjoaehL4B33RTR57Jhhppm1N3mIAR/xsTkOepr6EHjHvaKyyzHD\n1DlrLztbn/U6e9efGE1z1NPX9cA77iPM+k3USWbtky6/TBN2L8fUx0Hvni4H3nGvIJVZexl1v4EK\ni8/au/pEaJKjbm1w3Ifo2q6Po4wKu9fZm9fFoE8zaZlE1x5DXZ29O+4D6vpN1Ek08duok4Td6+zT\n61LMZxXxSW4/9eB3MfCOew2m+etKTZg27N6ffXEOef2K40w99F3huBe0MWuvWxNhH6fvYXfMZ2vh\nZ3Dkp+O4TymlWXtTYZ/HJ5mD3r7UIt+1pRnHPdf1WXtbYe/Sg32crgQ99cdi3VKLfFc47lMoc5iB\npi22q6PDvriuxBzmL+jDbNl/hQM/Acedemfts9z9sY6wj9LnsDvq3eXAl+e4z0Dd6+3jfjFpkrDP\nyxuoDnp/OPDlzH3cq87a61ySuezug0rv6z6LsPflidOloIOjbvWa+7jXaZolmcUCX/YQArMIexdm\n7V2KuoNejWfv48113FOYtReVjfigUYcTmKewdyno4Khb8+Y67n0w72FPJeqO9ex59r64uY17KlGY\nxjyHvc37zyG3LpjbuI+T8hN4saM69j3ss456yo+DKupYUqyyh5XN3lzGfZpAjHpyLPZm6m2b9qxt\nd8h5Dfsso96XoDf13lDxeucp9Ck9H8qYy7h30bhjsPd1d8dZRN0xn/425ynyXeG4D9HEk73q7L1q\n1GG6sLc9S2k66g56vbbuJwc+MXMX9zbfiCsb+DJ/KQn6GXZHfbxUgj6oz4Fve7JTxdzFfRp1PKnK\nhnucKocTSDnsTUa9D0GHdKNe1OfAd81cxb1MQKqGYMUdy2dy0LAqs3WYz7A76jbP5iruXde3sDcR\n9b4EHRz1cWa1U0AXl2TAce+EcYfq7VrYuxr1umM76n5z1NPR1bDDHMW9i7+ROk3UYT7C3mTUm47s\nsP3FHfZyurIrb5vGxl3SSuCfgRX55b8UEX86cBkBHwPWAY8Ab4uIa+ofbrOmDUVd6+5l/qhG18Le\nhai3GVZHPT1dnrVDuZn7NuBVEfGwpGXA5ZK+FhFXFC5zNHBg/nE48Kn837kzTeDL/qWkLoU99ag7\nqt3jWXs5Y+MeEQE8nH+5LP8YrMvrgLPzy14haXdJz42Iu2odbUcsRLpM5Cf503d1RB26GfY6o+6g\nN6cvu0F2fdYOJdfcJS0FrgYOAD4ZEVcOXGQf4M7C15vy03aIu6T1wHqAlUt2qTjkydUVmVW3x0Rh\nmCTcZW57MQ774hz0fpjFrL0PYYeScY+IJ4BDJO0OXCTpRRHxg0lvLCI2ABsAdlv27H78F9+wMjOh\nlMLuqM+vpmftDvtkJtpbJiIelPRN4DVAMe6bgX0LX6/OT7MpzGvYHXWz6ZXZW2ZP4LE87DsDRwEf\nHrjYxcA7JZ1P9kbqlnldb69DnVGH5sOe0mzdUW+HZ+3pKTNzfy7wuXzdfQlwQURcIukkgIg4E7iU\nbDfIm8h2hTyxofFOrO69NSZdd5/0usvoY9gd9e5y2NNUZm+Z64BDh5x+ZuHzAN5R79DSVXfgJ3ly\neBlmR7OI+mJ7PdX5pnkXOezpmpvfUK3btIGf9EmR0mwd6gl7ilGf9HcUBi8/T7F32NPmuE9h0l8Z\nr/pkSCnsfZut130kz4Xr63PkZ7Evu8M+Pce9Bk092Cd9gPc97ClHfdj19zHwDnt3OO4Fu928LZlD\nxvYx7G3P1mdxvP3B2+tL4PsSdZiPsIPjnpzUog7thr2LUe8bh72bHPdEVHlwO+yLc9SnM6vjxHgZ\nphm9jnuVMLWxNOOwP900YXfUp9OnqMN8hh16HveqZhX4qg9uh300h726WR7R0WFvXq/jvvzGTZUj\n1VTgp3lQd+mIjlW2naM+e7M+RO8sj8U+z2GHnsd9WnUGftoHtcM+msM+mbaOue7Z+mw57mNME/i6\nHswO+2gOezlt/hENz9bb4biXMOzBWYxYUw/erv2NU4d9R23u457CX0Sa9Z/Dc9h35LhX1PQD12Ff\nXOphn4UUAj6KZ+vtc9wTM+sHqsPejGlm7SlHexzP1tPhuCeijQdp24cUqMJhT5Ojnp7ex32a3SFn\npasP1Fnvx96FsE+ji2F31NPV+7inrM0Halv/4fX9LyZVnbV3KeyzDvoCh30ycxH31GbvbT9Ivc7e\njL6H3VHvlrmIeypSeJB2cZ29C/oa9raCDmk8X7psbuLe5uzdD9L+ztr7+uZpm1EHP2fqMDdxh9kH\nPrUHqGft9epb2NsOOqT3nOmyuYo7zCbwKT5Au3qUx1T1JewpBB3SfM503dzFHZoJvB+c82HaQwq0\nHfZUYr7Az5vmzGXcYccHVZXQd+lB6eWY6XU56qkFHbr1/OmquY17kR9ozen6kkwdB//q4x/BqMrP\ntdlx3Hsupf37q1pxx/KZ7zFT1xEdmwx76iFf4KC3w3G3seZpSSalqHcl3qM46u1y3HusD7P2BU3O\n3us+7nrVsHc95gsc9TQ47tYZCxGuI/JN/CGNaWbrXQ+7g54ex72n6pq1p7gkUwxzmdA3/ReRpl2C\n6XLYHfV0Oe7WaW3+KTuYz9m6g94NjrtZBfM2W3fQu8dx76GU3khddXt0fl/3onmKuoPebY672Rh1\n7avehbA76P3huFvjujp7n5eoO+j9NDbukvYFzgaeAwSwISI+NnCZI4F/AG7NT7owIk6rd6jWZV0K\nfN+j7pjPhzIz98eBUyLiGkmrgKslXRYRPxy43Lci4tj6h2h9kXLg6zxMQGpRd8zn09i4R8RdwF35\n51sl3QDsAwzG3RJQ95upu928rdZ93VMKfN3HfUkl6o65wYRr7pLWAocCVw45+whJ1wGbgfdExPVD\nvn89sB5g5ZJdJh2r9cRCVGcd+aYO4uW/M2opKh13SbsAXwbeHREPDZx9DbAmIh6WtA74CnDg4HVE\nxAZgA8Buy56dzp+jsVYUY1t36GdxmN1ZR90ht0mUirukZWRhPy8iLhw8vxj7iLhU0hmS9oiI++sb\nqrWl7qWZYcbFeDD+ff3jFw641aXM3jICPgPcEBEfGXGZvYB7IiIkHQYsAR6odaQ219r+83TQXNQd\ndGtCmZn7K4A3A9+XdG1+2vuANQARcSZwPPB2SY8DjwInRET7z0arzSxm76ly1K2Lyuwtczmw6IJo\nRHwC+ERdg7Lqmvjj3wvmKfBeerGu82+o2kT6HPhZvEHqsNusOO42sT4FflZ7vDjqNmuOu1WyEMUu\nRn6WuzA66tYWx72Hmlx3H9SFyLfxS0aOurXNcbdapBR5/8aomePeW7OcvRcNhrXp2KdwPBcH3VLk\nuFujysR38D+AFIJdhqNuKXPce6yt2fukuhLzBY66dYHj3nNdCXzqHHTrmiVtD8Ca5zBVt/zGTd5+\n1kmeuc8Jz+DLc8ytDxz3OeLAj+agW9847nPGgf8lB936zHGfQwtRm8fIO+g2Lxz3OTYPkXfMbV45\n7taryDvmZhnH3Z5SDGNXQu+Ymw3nuNtQg9FMIfYOuVl5jruVMiysdQff8Tarj+NulTnGZuny4QfM\nzHrIcTcz6yHH3cyshxx3M7MectzNzHrIcTcz6yHH3cyshxx3M7MectzNzHrIcTcz6yHH3cyshxx3\nM7MectzNzHrIcTcz6yHH3cysh8bGXdK+kr4p6YeSrpd08pDLSNLHJd0k6TpJL21muGZmVkaZP9bx\nOHBKRFwjaRVwtaTLIuKHhcscDRyYfxwOfCr/18zMWjB25h4Rd0XENfnnW4EbgH0GLvY64OzIXAHs\nLum5tY/WzMxKmejP7ElaCxwKXDlw1j7AnYWvN+Wn3TXw/euB9fmX275+9xk/mOT2W7IHcH/bgyjB\n46yXx1kvj7M++5W5UOm4S9oF+DLw7oh4qMqIImIDsCG/vqsi4mVVrmeWPM56eZz18jjr1ZVxllFq\nbxlJy8jCfl5EXDjkIpuBfQtfr85PMzOzFpTZW0bAZ4AbIuIjIy52MfCWfK+ZlwNbIuKuEZc1M7OG\nlVmWeQXwZuD7kq7NT3sfsAYgIs4ELgXWATcBjwAnlrjeDROPth0eZ708znp5nPXqyjjHUkS0PQYz\nM6uZf0PVzKyHHHczsx6aSdwlLZX0PUmXDDkvmUMXjBnnkZK2SLo2/3h/S2O8TdL38zFcNeT8JLZn\niXGmsj13l/QlSTdKukHSbwycn8r2HDfO1renpBcUbv9aSQ9JevfAZVrfniXH2fr2nNZEv8Q0hZPJ\nfrN11yHnpXTogsXGCfCtiDh2huMZ5bcjYtQvWqS0PRcbJ6SxPT8GfD0ijpe0HHjGwPmpbM9x44SW\nt2dE/Ag4BLKJEtnu0BcNXKz17VlynJDG47OyxmfuklYDxwCfHnGRJA5dUGKcXZHE9uwCSbsBryTb\n1ZeI2B4RDw5crPXtWXKcqXk1cHNE3D5weuvbc8CocXbeLJZlPgqcCjw54vxRhy6YtXHjBDgifyn5\nNUm/NqNxDQrg/0q6Oj+cw6BUtue4cUL72/N5wH3AZ/PluE9LeubAZVLYnmXGCe1vz6ITgC8MOT2F\n7Vk0apyQ1vacWKNxl3QscG9EXN3k7Uyr5DivAdZExEuAvwa+MpPBPd2/j4hDyF7evkPSK1saxzjj\nxpnC9twJeCnwqYg4FPg58D9aGMc4ZcaZwvYEIF82Og74YltjKGPMOJPZnlU1PXN/BXCcpNuA84FX\nSTp34DIpHLpg7Dgj4qGIeDj//FJgmaQ9ZjxOImJz/u+9ZOuEhw1cJIXtOXaciWzPTcCmiFg4EN6X\nyCJalML2HDvORLbngqOBayLiniHnpbA9F4wcZ2Lbs5JG4x4R742I1RGxluzlzzci4k0DF2v90AVl\nxilpL0nKPz+MbNs9MMtxSnqmsmPqk78s/11g8MiarW/PMuNMYXtGxN3AnZJekJ/0auCHAxdrfXuW\nGWcK27PgDYxe6mh9exaMHGdi27OSWe0tswNJJ8FUhy6YiYFxHg+8XdLjwKPACTH7X+99DnBR/pjb\nCfh8RHw9we1ZZpwpbE+AdwHn5S/RbwFOTHB7lhlnEtsz/8/8KOCPCqcltz1LjDOJ7TkNH37AzKyH\n/BuqZmY95LibmfWQ425m1kOOu5lZDznuZmY95LibmfWQ425m1kP/Hw0Ze8z4E7ZhAAAAAElFTkSu\nQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Show contour plot of the maximum of all three classes\n", "maxpdf = np.maximum(pdf0,np.maximum(pdf1,pdf2))\n", "plt.contourf(x0coords,x1coords,maxpdf)\n", "plt.title('Maximum of the three likelihood functions')" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# Gaussian classifier\n", "A Gaussian classifier is just a Bayesian classifier, with a Gaussian pdf.\n", "\n", "Remember that a Bayesian classifier means that\n", "$$\\hat{y} = \\arg\\max_y \\ln p_Y(y)+\\ln p_{X|Y}(\\vec{x}|y)$$\n", "\n", "A Gaussian classifier is one in which $p_{X|Y}(\\vec{x}|y)$ is Gaussian:\n", "$$p_{X|Y}(\\vec{x}|y)={\\mathcal N}(\\vec{x};\\vec{\\mu}_y,\\Sigma_y)$$\n", "\n", "So the classifier is\n", "$$\\hat{y} = \\arg\\max_y \\ln p_Y(y) + \\ln{\\mathcal N}(\\vec{x};\\vec\\mu_y,\\Sigma_y)$$\n", "\n", "Suppose that all of the classes have equal priors, $p_Y(y)=\\frac{1}{3}$ for the cases of three classes. Then the Gaussian classifier is just \n", "$$\\hat{y} = \\arg\\max_y {\\mathcal N}(\\vec{x};\\vec{\\mu}_y,\\Sigma_y)$$" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHFRJREFUeJzt3X+0HGWd5/H3h5CEKAFUUGJuLswkQBREYBExoMuGRYUg\nuCw66AgrMy4i7ohHVxxZBkF3FBfHA8ooZlAhBoZFBIZFGOTnCIPAhhhACMxiJIQQIKDkh4RcAt/9\no54bm6b73up7u7uqqz+vc+653V3V1d9+uvrTTz1dXaWIwMzMqmWLogswM7P2c7ibmVWQw93MrIIc\n7mZmFeRwNzOrIIe7mVkF9V24SzpD0sIOLv8BSQely5L0I0m/l3S3pHdLerhTj10Vkg6S9HjRdQBI\n2llSSNqyoMf/uKTbi3jsZiQdIOn/SVov6YNdfNxC3j+SdpO0RNI6SZ/p9uOPVSXDXdJHJS1KK98q\nSddJOrAbjx0Ru0fErenqgcAhwEBE7BcRt0XEbt2oYzSSbpX0ibIubxx1FBbGjToOZWmXNvsKcF5E\nbB0RV3XqQdLrOGv4eoHvn1OAWyJiakR8u4DHH5PKhbukzwHnAF8D3gQMAn8PHFFAOTsBj0bEH8a7\noKJ6jmYN7AQ8UHQRXdSbzzciKvMHbAusBz40wjxnAAtrrv8EeBJYA/wC2L1m2mHAg8A6YCXw39Pt\n2wPXAM8BvwNuA7ZI0x4F/iPwl8ALwEuppjOBg4DHa5b/ZuCnwGrgt8Bn6uq8HFgIrAU+0eT5Lkj3\nXw6cVlNH/fPcGQhgS+BvU10vpNrOS/ME8BlgGfAMcPZ4ljfCazAptdvbam57I/A8sMNwOwGfB54G\nVgHH18w7D/hVapcVwBk10x5Lda1Pf+/Ksd7cCnwduDst85+A19c/z5rX7OpU/yPAf023vx8YAl5M\nj3vvCO08G7ghLeNh4MM1tbwhLX9tquerwO0j1N7y+ttgGTOBm4Fn0+t+MbBdk3l/A7wMbEjPaTJp\nnW/0Hqtpv/+SXptngP9RM+8E4NS03HXAPcCM9FwC+EN6nD/j1e+ft6TX7jmy8D2iZtqFZJ26n6Xl\n3gXMHKEdj0jLeC4t8y3p9pvrXsNd6+73erJ19QPp+tZpvTiu8DwsuoC2PpnsDbZp+I3YZJ7NK166\n/hfA1LSSngMsqZm2Cnh3uvw6YJ90+evA+cDE9PduQGna5hUd+HjtG7N25STbaroHOJ0s7P6ULFTf\nV1Pni8AH07xTGjyXBWRBNDW9if4N+Msmz3P4TTYcUrdS94GRpt+SVtjBtLxPjHV5o7xW3wW+UXP9\nZOD/1LTTJrLN/4lkIfU88Lqa6W9L7bIn8BTwwUZ15azlVrLw2wN4LdkHbn04DT/PX6TatwL2Ivtg\nnduojRq1S1r+CuB4sg/GvckC761p+qXAZWm+PVJdI4V7y+tvg2XMIhs+nEz24foL4JwRHvNRXhnm\n9dc3t0NN+/0DMAV4O7CRP4bnF4D7gd0ApelvqFkfZzV5/0wkC9FTyd4/c8lCfLc0/UKyD6v9Ujtf\nDFza5PnsSvYhckha7ilp2ZPyrNvAe8k+YN+Ynufl7cq08fxVbVjmDcAzEbEp7x0i4ocRsS4iNpKt\nlG+XtG2a/CLwVknbRMTvI2Jxze3TgJ0i4sXIxgJbPUjPO4AdIuIrETEUEcvIVoxjaub5ZURcFREv\nR8SG2jtLmpDm/VKq/1Hg74BjW6yj3jci4ncR8RhZWHxknMtr5iLgI5KUrh8L/Lhm+ovAV1L7XkvW\na9oNICJujYj7U7vcB/wj8O/HWc+PI+LXkQ2h/Q3w4dTGm0maARwAfDEiXoiIJcAFwHEtPM7hZEN1\nP4qITRHxK7IPkw+lx/vPwOkR8YeI+DVZOzU1xvW3fhmPRMQNEbExIlYD32L87VnvzIjYEBH3km3V\nvD3d/gngtIh4ODL3RsSzOZa3P1kv+az0/rmZbGu6dn29MiLuTnlwMdmHcSN/BvwstcGLwDfJPojm\n5HliEfFzsi2om8g6Ip/Mc79Oq1q4Pwtsn3d8WtIESWdJ+o2ktWQ9EMiGXSB7ox0GLJf0L5LelW4/\nm+yT/eeSlkn66zHUuhPwZknPDf+R9ULeVDPPihHuvz1ZL2N5zW3LgeljqKVW7WMuJxuGaLuIuIus\nN36QpNlkvcera2Z5tu5D+nmyNzOS3inpFkmrJa0BTuSPr9lY1T/viQ2W+WbgdxGxrm7eVtp8J+Cd\nda/7nwM7kvWat2xQS0PjWH/rl/MmSZdKWpmWs5Dxt2e9J2sub34tyYZgfjOG5b0ZWBERL9fcVv9a\nNHvMRsva3M5pmSto7XWdT7aldWHOD6eOq1q4/5Jsky/v7lkfBY4kGyPflmwTErLNQyLi/0bEkWSb\nW1eRbS6Tekqfj4g/JRur+5ykg1usdQXw24jYruZvakQcVjPPSFsDz5D1zHaquW2QbDMess3M19RM\n27Hu/s2WPaNueU+Mc3kjuQj4GFmv/fKIeCHn/S4h+yCYERHbkg2RDW8BjPUwp/XP+0WyNq71BPB6\nSVPr5h1u80aPXX/bCuBf6l73rSPiU2RDPJsa1NLMmNbfBr6W6nxbRGxD9pqoybyNjLZujGQF2Zh/\nq54AZkiqzbDa16LVZW1+H6WtyRl5l5W2uOaTDZOeVLuHT5EqFe4RsYZsDPvvJX1Q0mskTZR0qKT/\n1eAuU8k+DJ4lWzm/NjxB0iRJfy5p27SptpbsiyQkHS5pVloJ1pB94fLyq5Y+sruBdZK+KGlK6oXt\nIekdOZ/rS2Rv1r+VNFXSTsDnyHpdAEuA90gaTJvpX6pbxFNk4/z1viDpdWkI4mTgf491eWk3wDNG\neBoLgf9EFiYLRpiv3lSyHvQLkvYjC7lhq8lei8211OweufMIy/yYpLdKeg3ZWP/lqY03i4gVwB3A\n1yVtJWlPsi/Oh9v8KWDnusCpb5drgF0lHZvWzYmS3iHpLenxrgDOSOvuW8m+iBypHVpef5ssZz2w\nRtJ0snHwViwBjknPZV/g6BbuewHwVUm7KLOnpDekac3WUci+IH0eOCU97kHAB8i+s2jVZcA8SQdL\nmkj2Rf5Gstc6j1PJPhz/gmyrfkH9kF4RKhXuABHxd2QhdxrZG30F8N/Iei71FpBtjq0k26vgzrrp\nxwKPpk3VE8k2nwF2AW4ke0P8EvhuRNzSYp0vkY2/7kW2p8wzZCv6tiPdr85fkfWalgG3k/Vof5iW\nfwNZMN9H9sXtNXX3PRc4WtkPrGr33f2nNP8Ssj0NfjCO5c0A/rVZ8SksF5O9MW7L/azhJOArktaR\nfZhv7pFGxPNke6n8axr22D/VMfw6N/Njsi/hniT7srTZj1U+QtZDfgK4EvhyRNyYpv0k/X9W0vD4\n9ivaJQ3pvJfs+5In0uN9g+zLTMjW1a3T7RcCPxqh5rGuv/XOBPYh66j8jOwDphV/Q9b7/n1a1iUt\n3PdbZK/fz8k+gH5ANt4N2XcIF6XX8cO1d4qIIbIwP5TsvfNdsj1UHmqxdiLiYbIOxnfSsj5AtvfL\n0Gj3lfTvyPLmuPSe/gbZ+jyWodq2Gt7DwwxJAewSEY+0YVkDwGURMeKXUpJ+CDwREaeN9zFHeIzT\ngNUR8f0m028l27vjgk7VYNZt/mGMdUREPM4oexukYZKjyHYH7GQt/7OTyzcro8oNy1hvkPRV4NfA\n2RHx26LrMasaD8uYmVWQe+5mZhVU2Jj7pC2mxJQJU0ef0cwKsWm7yfzJtKeKLsPqPHD/i89ExA6j\nzVdYuE+ZMJU523+oqIc3sxGsnjeTi7/8Tdr/Q1Ubr9mDq5r+armW95Yxs1c45+4riy7B2sBj7ma2\nmYO9OhzuZgY42KvG4W5mDvYKcrib9TkHezU53M36mIO9uhzuZn3KwV5tDnezPuRgrz6Hu1mfcbD3\nB4e7WR858MZcP260CnC4m/WJA29cztHbLB59RqsEh7tZH3Cw9x+Hu1nFbVg4xcHehxzuZhW2YeEU\nvr9rK+ertqrwUSHNKmrpWYNcv+u5RZdhBXHP3ayClp41yPVzHez9zOFuVjEOdgOHu1mlLDtploPd\nAIe7WWUsO2kWV3/87KLLsJJwuJtVgIPd6jnczXrc6nkzHez2Kg53sx62et5MLv7yN4suw0rI4W7W\no4ZmDzjYrSn/iMmsB/mwvTYa99zNeoyD3fJwuJv1EAe75eVwN+sRDnZrhcPdrAc42K1VDnezknOw\n21g43M1KzMFuY+VwNyspn8zaxsPhblZCPuepjZfD3axkHOzWDg53sxLxyaytXRzuZiXhk1lbOznc\nzUrAwW7t5gOHmRVs6VmDXL+rT41n7eWeu1mBfDJr65Tc4S5pgqRfSbqmwTRJ+rakRyTdJ2mf9pZp\nVj0OduukVnruJwNLm0w7FNgl/Z0AfG+cdZlV2rKTZjnYraNyhbukAWAecEGTWY4EFkTmTmA7SdPa\nVKNZpfhk1tYNeXvu5wCnAC83mT4dWFFz/fF02ytIOkHSIkmLhl7e0FKhZlXgk1lbt4wa7pIOB56O\niHvG+2ARMT8i9o2IfSdtMWW8izPrKT6ZtXVTnp77AcARkh4FLgXmSlpYN89KYEbN9YF0m5nhYLfu\nGzXcI+JLETEQETsDxwA3R8TH6ma7Gjgu7TWzP7AmIla1v1yz3jM0e8DBbl035h8xSToRICLOB64F\nDgMeAZ4Hjm9LdWY9bmj2AN9d8J2iy7A+1FK4R8StwK3p8vk1twfw6XYWZtbrHOxWJP9C1axDHOxW\nJIe7WQf49HhWNIe7WZs52K0MHO5mbeRgt7JwuJu1iYPdysThbjZOS88adLBb6fhkHWbjcOCNyzln\nGwe7lY977mZjdOCNy30yaysth7vZGGxYOMXBbqXmcDdrkU9mbb3AY+5mLfDJrK1XuOdulpPPeWq9\nxOFuloOD3XqNw91sFD6ZtfUih7vZCHwya+tVDnezJhzs1ssc7mYNrJ4308FuPc3hblbHJ7O2KnC4\nm9XwyaytKhzuZonPeWpV4nA3w8Fu1eNwN8Mns7bqcbhb3/OJNqyKHO7W1xzsVlUOd+tbDnarMoe7\n9SUHu1Wdw936joPd+oHD3fqKg936hcPd+oaD3fqJw936goPd+o3D3SrPwW79yOFulXbgjcuLLsGs\nEA53q6wDb1zO0dssLroMs0I43K2SNiyc4mC3vuZwt8rZsHAK39/1kqLLMCvUqOEuaStJd0u6V9ID\nks5sMM9BktZIWpL+Tu9MuWYjc7CbZbbMMc9GYG5ErJc0Ebhd0nURcWfdfLdFxOHtL9Esn6VnDXL9\nrucWXYZZKYwa7hERwPp0dWL6i04WZdaqpWcNcv1cB7vZsFxj7pImSFoCPA3cEBF3NZhtjqT7JF0n\nafcmyzlB0iJJi4Ze3jCOss3+yMFu9mq5wj0iXoqIvYABYD9Je9TNshgYjIg9ge8AVzVZzvyI2Dci\n9p20xZTx1G0GwLKTZjnYzRpoaW+ZiHgOuAV4f93tayNifbp8LTBR0vZtq9KsgWUnzeLqj59ddBlm\npZRnb5kdJG2XLk8BDgEeqptnR0lKl/dLy322/eWaZVbPm+lgNxtBnr1lpgEXSZpAFtqXRcQ1kk4E\niIjzgaOBT0naBGwAjklfxJq13ep5M7n4y98sugyzUsuzt8x9wN4Nbj+/5vJ5wHntLc3s1YZmD7Bu\nJxVdhlnp5em5m5XG43O3AuCIC78w5mVMm7PSP3SyynO4W08Ymj2wOdjHa9Ud0znijld+OHj83qrG\n4W6lt+ykWR1/jNotgeOOugnABx6znuZwt1LrRrDXW3DFwdl/DmbanJUAHsaxnuNwt9IqItjrrbpj\nOgCf5KOAQ956hw/5a6VUhmCvteqO6ay6Yzrvu/lkLl+7T9HlmI3K4W6lU7ZgrzX5sUksuOJg3nfz\nyUWXYjYih7uVSpmDvdbkxyaNa3dMs07zmLuVQjt3deymIy78AhsHh3zwMisd99ytcKvnzezJYB82\n+bFJHou30nG4W6FWz5tZicMJDI/Ff/LfPlp0KWaAw90KVJVgr7XqjukOeCsFh7sVosoHAHPAWxk4\n3K3revXL01Y44K1oDnfrqn4I9mGr7pjuL1mtMN4V0rqmV/Zhb6cFVxwMR/kgZNZ97rlbV/RjsA8b\nPhCZWTc53K3j+jnYh3n83brN4W4d5WDP+AtW6zaHu3WMg/2V/AWrdZPD3TrCwd7YDU/OLroE6xMO\nd2urodkDDvYRuPdu3eJwt7ZZdtKsvtmHfTzce7ducLhbW7i3nt/wqfvMOsnhbuO2et7MokvoOT6T\nk3Waw93GpYpHduyGyY9NKroEqziHu42Zg3183Hu3TnK425g42MfPvXfrJIe7tWRo9oCDvY3ce7dO\n8VEhLbd+OlyvWa9zz91yc7C3n4dmrFMc7paL92PvHB9QzDrB4W6jcrCb9R6Hu43IwW7Wm0YNd0lb\nSbpb0r2SHpB0ZoN5JOnbkh6RdJ8kHxmpAhzs3eGDiVkn5NlbZiMwNyLWS5oI3C7puoi4s2aeQ4Fd\n0t87ge+l/9aDvFeMWe8bNdwjIoD16erE9Bd1sx0JLEjz3ilpO0nTImJVW6u1jnNv3awaco25S5og\naQnwNHBDRNxVN8t0YEXN9cfTbfXLOUHSIkmLhl7eMNaarUMc7MX5h0XvLroEq5hc4R4RL0XEXsAA\nsJ+kPcbyYBExPyL2jYh9J20xZSyLsA5xsBdr8mOTPO5ubdXS3jIR8RxwC/D+ukkrgRk11wfSbdYD\nHOxm1ZNnb5kdJG2XLk8BDgEeqpvtauC4tNfM/sAaj7f3Bge7WTXl2VtmGnCRpAlkHwaXRcQ1kk4E\niIjzgWuBw4BHgOeB4ztUr7WRT7JhVl159pa5D9i7we3n11wO4NPtLc06ZWj2AGtmTvaRHc0qzL9Q\n7TPD+7A72MtnwRUHF12CVYjDvY/4x0lm/cPh3icc7Gb9xeHeBxzsZv3H4V5xDnaz/uRwrzgHu1l/\n8jlUK8o9drP+5nCvIP/q1Mw8LFMxDnYzA4d7pTjYzWyYw70iHOxmVsvhXgEOdjOr53DvcQ52M2vE\ne8v0KO/qaGYjcbj3oNXzZvqojmY2Ig/L9BgHu5nl4XDvIQ52M8vL4d4jhmYPONjNLDeHew/wl6dm\n1iqHe8k52M1sLBzuJeZgN7Ox8q6QJeUfJ5nZeLjnXkIOdjMbL4d7yTjY+9dxR91UdAlWIQ73EnGw\nm1m7ONxLwsFuZu3kcC8BB7uZtZv3limQd3U0s05xuBfEvXUz6yQPyxRg9byZRZdgZhXncO8yH9nR\nGpk2ZyVHb7O46DKsQhzuXeRgt2YO2fGhokuwinG4d4mD3cy6yeHeBQ52M+s2h3uH+SQbZlaEUcNd\n0gxJt0h6UNIDkk5uMM9BktZIWpL+Tu9Mub1l9byZ3o/dcvGXqdZuefZz3wR8PiIWS5oK3CPphoh4\nsG6+2yLi8PaX2Ju8H7vlNW3OyqJLsAoateceEasiYnG6vA5YCkzvdGG9zMFuZkVracxd0s7A3sBd\nDSbPkXSfpOsk7d7k/idIWiRp0dDLG1outhc42M2sDHIffkDS1sBPgc9GxNq6yYuBwYhYL+kw4Cpg\nl/plRMR8YD7AthPfGGOuuqQc7GZWFrl67pImkgX7xRFxRf30iFgbEevT5WuBiZK2b2ulJedgt7HY\nODjE93e9pOgyrILy7C0j4AfA0oj4VpN5dkzzIWm/tNxn21lomTnYzaxs8gzLHAAcC9wvaUm67VRg\nECAizgeOBj4laROwATgmIio37FLPP04ys7IaNdwj4nZgxASLiPOA89pVVC9wb93Mysy/UB0DB7u1\ny/Vzzy26BKsoh3uLHOxm1gsc7i3wSTbMrFc43HPyl6fWbj7sgHWSwz0HB7t1gvdvt05yuI/CwW5m\nvSj34Qf6zdDsAdbMnOxgN7Oe5HBvYGj2gI/Dbh113FE3FV2CVZyHZeo42K0bfHIO6zSHex0Hu5lV\ngcO9hn+gZGZV4XBPHOzWLR5vt25wuONgN7Pq6ftwd7BbN20cHPKXqdYVfbsrpEPdirDzwOqiS7A+\n0Zc9dwe7mVVd34W7g92K4vOlWjf1Vbg72M2sX/RNuDvYzayf9EW4+yQbVgY+pZ51U+XD3YfstTLY\nODhUdAnWZyod7g52M+tXldzP3Ud2tLLxkIx1W+V67g52M7OKhbuD3crI4+1WhMqEu4PdyspDMlaE\nyoS7g93M7I8qEe7+gZKV1bQ5K4suwfpUz4e7g93K7JAdHyq6BOtTPR3uDnYzs8Z6Ntwd7FZ20+as\n9Ik5rDA99yMmh7qZ2eh6qufuYLde4mO3W5F6Jtwd7NZL/MMlK1pPhLuD3XqNf7hkRRs13CXNkHSL\npAclPSDp5AbzSNK3JT0i6T5J+7SrQAe79Rr32q0M8nyhugn4fEQsljQVuEfSDRHxYM08hwK7pL93\nAt9L/8fFwW5mNjaj9twjYlVELE6X1wFLgel1sx0JLIjMncB2kqaNp7Ch2QPjubtZYTwkY2WgiMg/\ns7Qz8Atgj4hYW3P7NcBZEXF7un4T8MWIWFR3/xOAE9LVPYBfj6f4LtkeeKboInJwne3lOtvLdbbP\nThGxw2gz5d7PXdLWwE+Bz9YGeysiYj4wPy1vUUTsO5bldJPrbC/X2V6us716pc48cu0tI2kiWbBf\nHBFXNJhlJTCj5vpAus3MzAqQZ28ZAT8AlkbEt5rMdjVwXNprZn9gTUSsamOdZmbWgjzDMgcAxwL3\nS1qSbjsVGASIiPOBa4HDgEeA54Hjcyx3fsvVFsN1tpfrbC/X2V69UueoWvpC1czMekNP/ELVzMxa\n43A3M6ugroS7pAmSfpX2h6+f1rFDF7RqlDoPkrRG0pL0d3pBNT4q6f5Uw6IG00vRnjnqLEt7bifp\nckkPSVoq6V1108vSnqPVWXh7Stqt5vGXSFor6bN18xTenjnrLLw9x6tbx3M/meyXrds0mNaRQxeM\n0Uh1AtwWEYd3sZ5m/kNENPuhRZnac6Q6oRzteS7wzxFxtKRJwGvqppelPUerEwpuz4h4GNgLso4S\n2e7QV9bNVnh75qwTyrF+jlnHe+6SBoB5wAVNZmn7oQvGIkedvaIU7dkLJG0LvIdsV18iYiginqub\nrfD2zFln2RwM/CYiltfdXnh71mlWZ8/rxrDMOcApwMtNpk8HVtRcf5xXH7umG0arE2BO2pS8TtLu\nXaqrXgA3SronHc6hXlnac7Q6ofj2/BNgNfCjNBx3gaTX1s1ThvbMUycU3561jgH+scHtZWjPWs3q\nhHK1Z8s6Gu6SDgeejoh7Ovk445WzzsXAYETsCXwHuKorxb3agRGxF9nm7aclvaegOkYzWp1laM8t\ngX2A70XE3sAfgL8uoI7R5KmzDO0JQBo2OgL4SVE15DFKnaVpz7HqdM/9AOAISY8ClwJzJS2sm6cM\nhy4Ytc6IWBsR69Pla4GJkrbvcp1ExMr0/2myccL96mYpQ3uOWmdJ2vNx4PGIuCtdv5wsRGuVoT1H\nrbMk7TnsUGBxRDzVYFoZ2nNY0zpL1p5j0tFwj4gvRcRAROxMtvlzc0R8rG62wg9dkKdOSTtKUrq8\nH1nbPdvNOiW9Vtkx9Umb5e/l1UfWLLw989RZhvaMiCeBFZJ2SzcdDDxYN1vh7ZmnzjK0Z42P0Hyo\no/D2rNG0zpK155h0a2+ZV5B0Iozr0AVdUVfn0cCnJG0CNgDHRPd/3vsm4Mq0zm0JXBIR/1zC9sxT\nZxnaE+CvgIvTJvoy4PgStmeeOkvRnunD/BDgkzW3la49c9RZivYcDx9+wMysgvwLVTOzCnK4m5lV\nkMPdzKyCHO5mZhXkcDczqyCHu5lZBTnczcwq6P8DjhznGqOVPusAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Find the (x0,x1) coordinates for which class 0 is the best choice\n", "yhat_is_0 = (pdf0 == maxpdf)\n", "yhat_is_1 = (pdf1 == maxpdf)\n", "yhat_is_2 = (pdf2 == maxpdf)\n", "yhat = yhat_is_1 + 2*yhat_is_2\n", "# Now let's plot that\n", "plt.contourf(x0coords,x1coords,yhat)\n", "plt.title('Classifier output, yhat, plotted as a function of x')" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Check our results: Gaussian borders must be quadratic\n", "\n", "Remember that the Gaussian pdf is given by \n", "$$\\hat{y} = \\arg\\max p_{X|Y}(\\vec{x}|y) = \\arg\\max \\ln p_{X|Y}(\\vec{x}|y)$$\n", "\n", "But the logarithm of a Gaussian is a quadratic function of $\\vec{x}$:\n", "$$\\ln p_{X|Y}(\\vec{x}|y) = -d_\\Sigma^2(\\vec{x},\\vec{\\mu}_y) + \\mbox{constant}$$\n", "\n", "So the boundary between any two classes, in a Gaussian classifier, needs to be either a straight line, or a quadratic.\n", "\n", "For example, the boundary between class 0 and class 1 is the set of points such that\n", "$$(\\vec{x}-\\vec\\mu_0)^T\\Sigma_0^{-1}(\\vec{x}-\\vec\\mu_0) = (\\vec{x}-\\vec\\mu_1)^T\\Sigma_1^{-1}(\\vec{x}-\\vec\\mu_1)+\\mbox{constant}$$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# Likelihood models: Gaussian mixture model (GMM)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "So, notice that pdf2 is looking elliptical here, even though the data distribution wasn't really very elliptical. Let's analyze that a little more carefully, by drawing a scatter plot of the same data on top of the countour plot." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Mark\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:2: MatplotlibDeprecationWarning: pyplot.hold is deprecated.\n", " Future behavior will be consistent with the long-time default:\n", " plot commands add elements without first clearing the\n", " Axes and/or Figure.\n", " \n", "C:\\Users\\Mark\\Anaconda3\\lib\\site-packages\\matplotlib\\__init__.py:917: UserWarning: axes.hold is deprecated. Please remove it from your matplotlibrc and/or style files.\n", " warnings.warn(self.msg_depr_set % key)\n", "C:\\Users\\Mark\\Anaconda3\\lib\\site-packages\\matplotlib\\rcsetup.py:152: UserWarning: axes.hold is deprecated, will be removed in 3.0\n", " warnings.warn(\"axes.hold is deprecated, will be removed in 3.0\")\n" ] }, { "data": { "text/plain": [ "[]" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHh9JREFUeJzt3XuMXNV9B/Dvb3eMvTgYx4a1XT9Cgi1QktYmWCTFpCU8\n0kBoUB6VyFNKK7lEJEqq0KhJq6Km6j/9o6IpTZCbpAqFNO8AooSAiQmhqnG8gG2wAXvBG7PGrHdh\nd+x9emZ+/WPmru9c35n7OvcxZ74faeXduWfunL1ef/fnM+fcI6oKIiKyS0/eHSAiIvMY7kREFmK4\nExFZiOFORGQhhjsRkYUY7kREFmK4ExFZiOFORGQhhjsRkYVKeb3wWb192ldaktfLExF1pPLcyKiq\nnh/ULrdw7ystweWrPpHXyxMRdaSHhm4fCtOOwzJERBZiuBMRWYjhTkRkIYY7EZGFGO5ERBZiuBMR\nWYjhTkRkIYY7EZGFGO5ERBZiuBMRWYjhTkRkIYY7EZGFGO5ERBZiuBMRWYjhTkRkIYY7EZGFGO5E\nRBZiuBMRWYjhTkRkIYY7EZGFGO5ERBZiuBMRWYjhTkRkIYY7EZGFGO5ERBZiuBMRWYjhTkRkIYY7\nEZGFGO5ERBZiuBMRWYjhTkRkIYY7EZGFGO5ERBZiuBMRWYjhTkRkIYY7EZGFGO5ERBYKHe4i0isi\nT4vIAz7HRES+ISKHRGSviLzLbDeJiCiKKJX7FwEcaHHsOgAbGh9bAXwrYb+IiCiBUOEuImsAfBDA\nt1s0uRHAXVq3E8BSEVllqI9ERBRR2Mr9dgBfAVBrcXw1gCOur19pPEZERDkIDHcRuQHAiKoOJH0x\nEdkqIrtFZPdcdTrp6YiIqIUwlfsWAB8SkcMAfgDgKhG529NmGMBa19drGo81UdVtqrpZVTef1dsX\ns8tERBQkMNxV9auqukZVLwBwE4BfqeqnPM3uB/CZxqyZ9wCYUNVXzXeXiIjCKMV9oojcDACqeieA\nBwFcD+AQgCkAnzXSOyIiiiVSuKvqYwAea3x+p+txBXCLyY4REVF8XKFKRGQhhjsRkYUY7kREFmK4\nExFZiOFORGQhhjsRkYUY7kREFmK4ExFZiOFORGQhhjsRkYUY7kREhsz0TeLJ636M2b7JXJ7vxnAn\nIjJkcONOvLFiGIc2PpnL890Y7kREBsz0TWJ4w35AgOEN+yNX30mf78VwJyIyYHDjTigUAKCoRa6+\nkz7fi+FORJSQU3Vrqb7NtJZqkarvpM/3w3AnIkrIXXU7olTfSZ/vh+FORJTQeP+x+arboaUaxvvD\n7Taa9Pl+Ym+zR0REdVvu/2Suz/fDyp2IyEIMdyIiCzHciYgsxHAnIrIQw52IyEIMdyIiCzHciYgs\nxHAnIrIQw52IyEIMdyIqPJObWHQLhjsRFZ7JTSy6BcOdiArN9CYW3YLhTkSFZnoTi27Bu0ISUS5O\nXLo6sM1cqYzhiw5Ae5o3sVi/591YOL047S52NIY7ESUWJqjjONK//cxNLHoUB655Fhce/Ujb554z\nMJxKnzoFw52IAqUV3kHKi4egPdWmx7SnivLiocDntupzt4Q+w52I5uUV4q1ccvCvjJ/T+z3aGvaB\n4S4iiwA8DmBho/1PVPU2T5srAdwH4OXGQz9T1a+b7SoRmVS0IM+L+zrYFPRhKvdZAFep6kkRWQDg\nCRH5haru9LT7jareYL6LRJQUgzwcm4I+MNxVVQGcbHy5oPGhrZ9BRHljmCfX6UEfasxdRHoBDABY\nD+DfVdVvounlIrIXwDCAW1X1OXPdJKJ2GObpcq5vJ4V8qHBX1SqATSKyFMDPReSdqvqsq8lTANY1\nhm6uB3AvgA3e84jIVgBbAWBR7zmJO0/UrRjm+eikkI80W0ZVx0VkB4APAHjW9XjZ9fmDIvJNETlP\nVUc9z98GYBsAnLtwBYd2iCJgoBdHJ4R84O0HROT8RsUOEekDcC2A5z1tVoqIND6/rHHeMfPdJeou\nJy5dPf9BxVPkv5cwlfsqAN9rjLv3APiRqj4gIjcDgKreCeBjAD4nIhUA0wBuarwRS0QRFDksyF9R\nq/gws2X2ArjE5/E7XZ/fAeAOs10j6g6dHOhjb+81fs7l+6vBjQroxKWrCxXwXKFKlJOih3oawZ30\ndYse/EUKeIY7UQpm+iax58oHsemx65vuXugE+lypjBfWfRMX/e5TOKuyJK9uAsgvxOPw62vRAr8o\nAc9wJ0qBe+egdac+fcbxI/3bUV58GEf6twfe3dCkTgrysNzfU1GCvggBz3AnMqxp56CLDmDl8+Wm\n6nyuVMbIst8CohhZthtrR65JpXq3MciDFCno8w547sREZNCJS1fj+WuehTb+ZSlqONK/vamN+x7l\nfsfjGnt7b9NHt+v268BwJzLAmYvuVOXOPci1p4qRZbsxV6qv8ws6HhXDPFie1yfPN80Z7kQJeBcY\n+e4c5KrOg44HYXUeX17XLK+A55g7UUTt/rEG7RwUZ2chhrhZY2/vzX08PguS10LScxeu0MtXfSKX\n1yaKI8sKrAiBPnPxTGrnXvT8otTOHVbWAW/qzdWHhm4fUNXNQe1YuRMFSCPU6/Pc726a555HoE9c\n8BKODf8HVq7eioWLVmX2uq1+cWQZ+s71trWKZ7gTtZBmpe6e57506Z+l9jpufoF6fOhHUJ3F8dd+\niDVv+VIm/WjH3cesgt7WYRqGO5FH2sMvc6UyXlten+f+2vLdeFP1T1CC+XnuQcMqszNHUTk1AgCo\nnBrB7MyrmVbvQfIIeptwtgxRQxa31h17ey8OXvxo0zz30Z5HjJx75uKZpo8gx1/7kefrHxrpRxrC\nfk9xZTEklvWsGVbu1PWyCHRHBWVMyC5AGsMAUsUEduE8XBu5ek8Sdu6qfb5vBazevZzvOY1K3rbh\nGVbu1LXSrtT95lWP9jzsO889bPUepTJvx1u1n368uNW7W5pVvC1YuVNXSjvUW5mWw6erdodUMY2X\nWz4njSCrnHo90uNFlEYVb1P1znCnrpJXqDveWr011LnSrkwvWP/1pq+vWD/o+moQST1x6MLE5whr\n5uIZvuHqg+FOXSHvUA8r7VBvDvHsXifLsKc6hjtZL61gNxXqrQK9Uinj+LEf4PyVH0epdE7k4yaC\nfGpmCk/s2YH3bnof+haeHbuNuy9xgj7oezVZvdsyNMM3VMlaab1hauoGVEFvjI6/vgOzM0MYf/1X\noY5fsX6w6cOEfYNPY+SNY9h36OlEbdzi9C/oWgB8k9WL4U7W6fRQB+qV6uSJAQCKyRMDqFROtDw+\nfXI3Ll27L3G/vKZmpvDS8EEAwODwQUzPTsVq00rYkA+6FuSP4U5W6fRQd4y/vgPOTf1UtalivWL9\nIPqq90JQmz8etmqOYt/g00198HuNMG2CBAV8u2vhxer9NIY7WSHNaj2pqPPST1eqzrhvdb46v2L9\n4Hy1XNN6uNe0FrlqDhLmNUz2o1XA+12LTq3es95yj+FOHa+o1XrcxUbuStXhrord1bLfcRPCvIbp\nfvgFfKtr0a56pzrOlqGOVeRKPYkFehCnK9W6mtZwfLx+u4DR8ZH5atnvuAlhXiONflyxfrBpNs3s\nzO/gvRZAtfE4tcNwp45UxGBPGurzlev6D7dt98Et7Y+38/mVj4Zr99ElQIt73dxxLHk/wlq97gup\nv4atGO7UcUwHe2FC3bCwQZ7kvHccu9r4+b3Ve9bSmOOe9Xg7wHCnDmJbtR4U6mEWB7nFCfPRqQr+\n7tfH8U9/fD6Wnx09Dj6/8lG8ODaLv3hwFO9/9wfx5iXLI5+D0sFwp45QtGBPM9Qd7sVBl71ji2+b\npNX5d/eMY89rs/junnH89R+eF+sctz1+HKeqFTyx5zH86Xs/mqg/NsqjagcY7tQBbBmGiTL84l0c\n9PvrL5mv3k0Nt4xOVfDAoUkogAcOTeLPNy6NXL2/ODaLlycqAICJyXG8UR7r6OrdhtsOODgVkgqt\nSMGe5D7qUcfV/RYHfX7lo0bH0b+7Z3z+NWqq+O6e8cjnuO3x401fH3juASN9s0VeVTvAcKcCK1qw\nxxHnPip+i4OGjr6AsalKrD74car2U42ZjKdq9eo9ymu4q3bHSxMVvFEeM9bPqHjr39MY7lQ4pleb\nJl2QlFW17vBbHBS3sm7FXbXHfQ1v1e54Ys9jSbqWG9NDMnlW7QDH3KlgbKnWEzk5eMbioFM1YO/I\nbLLzuuwbmZ2v2uO+xtGT/lX+yenOuzWAjcT72zsr5y5coZev+kQur03FxGBPb256lpLOfY87xz3J\nkEwnVe0PDd0+oKqbg9oFDsuIyCIR2SUie0TkORH5B582IiLfEJFDIrJXRN4Vt+PUnbIM9grKGOq5\nAxWUfY+fXD+CV1/Z1vLmVJVK2fd4kmCP82bpruEpXP69w/jtcPwbho1OVXDzL15tOdYedNyEqZkp\nPPzk/7S94Vira25C2GCfK5Wx723fxFzJ/+fGabO//3bM9k2a6l5sYcbcZwFcpaobAWwC8AEReY+n\nzXUANjQ+tgL4ltFektWyrthHex7GtLyM0Z5Hzjg2c/FM5E0ygOTBHsff/vo4agp87df+Y99huOe5\nxzlugns+f6uqPejvJIs3Uo/0b0d58WEc6d/ets0bK4ZxaOOTqfcnSGC4a93JxpcLGh/esZwbAdzV\naLsTwFIRWWW2q2SjrIO9gjImZBcgignZ1VS9z1w8E2mTjMkTA/O34Y0rbrDvGp7Cibn6P8MTcxqr\nevfOc/dW50HH/UQdkvHO5/erzIP+TrIYjpkrlTGy7LeAKEaW7fat3udKZYws3QUIMLxhf+7Ve6jZ\nMiLSKyLPABgB8Iiqen8trQZwxPX1K43HvOfZKiK7RWT3XHU6bp/JEnmMsY/2PAxt1CaK2nz17oyx\nB20M4T4uqMW+vW3SOet/66nW41TvQfPcTcyDD+KeGVStwbcyj7JZRxRRxtmP9G9v+rnxq96Pyb1N\nbfKu3kOFu6pWVXUTgDUALhORd8Z5MVXdpqqbVXXzWb19cU5Blsgj2E9X7Y1/1FLFhOzCyfX1YAza\nGMJ7PO7mFEnfNHVX7Y6o1XvQPPc48+DjVu2nZwaduRFH0N9JFsMxTtWuPfU+aE/1jOp9wf4XMbxh\nP7TU2B2rVMu9eo80z11VxwHsAPABz6FhAGtdX69pPEaUurCzYtxVu0OlNl8JBm0MEbSJRla8Vbsj\nSvUeNM/dxDz4IK02+3BX5u3+TrKaHeOu2uf74KneBzfu9G2TZ/UeZrbM+SKytPF5H4BrATzvaXY/\ngM80Zs28B8CEqr5qvLdkBdMLlMKalsOnq/Z5pzd+CNoYwu941M0pTEx1PDnnP3251eN+gua5R50H\nH2f6o99mH96NOFr9nZyaOIK4ok57LC8emq/aHdpTRXnxEID6tMfx/mPzVft8m1IN4/35xWDgPHcR\n+QMA3wPQi/ovgx+p6tdF5GYAUNU7RUQA3IF6RT8F4LOqurvdeTnPvTsVZS57HvPYbZjD7ifJvPY4\nc9q7ZT57K2HnuQeuUFXVvQAu8Xn8TtfnCuCWqJ2k7lKUYI8rrU01OhmDvbh4bxnKRJGCPenOSXF4\nq/YsFgelrUjBHrQwrduCHWC4U5fJ7X4xHlksDkrLHceuLlSwA+0XpnVjsAMMd8pAkar2IoizOKgo\nkoZ6GsHebmFatwY7wHCnlBUp2ItyM7AsFgeZlke1DoQbY2+1MK2bgx1guBNlysQmGVkr8l0eWy1M\nO+fFN2K9ZiudFuwAw51SVKSqPU/ucMxicZApJqr1tG/f67cwDep/e4C4OjHYAW7WQV0ijxkyfkxs\nkpG2pJU6kN092f0WprkXGCXVqcEOMNwpJaza/f3XjWavi0l5hjoQbw77W6u3AuD4uh+GO1EG7jh2\ndSFXqJoIdEceOygBDPZWOOZOxsWt2sPsdBMkaDFLVvx2F3IHaZhFTC+OzeLqe4ZwcMzskI0zlm4q\n2LMYW/ezfH/V6C5KQPJgn+mbxJPX/Tj3e7kDDHcqkFY73UQZkvFbzJLHeLt7dyE3J1TDLGK67fHj\nmDyluO3x+DsteV/XdKWeJNSzvJVAmF2UTFTsgxt3FmYnJg7LkFFJqnb3TjdrR67BWZUlkc7RtJgF\nu3AerkUJ0c5hgnd3od9ffwn6Fp7ddPy+g9Pzi5j+fONSLD+7+Z/ii2OzeHmiXtW/NFHBwbFZbFi+\nMHQfTIa4V5JxdSD7YZigny1TwzAzfZMY3rB/fiem9XvejYXTi42cOw6GOxWC3043Fx79SKRz+C1m\nWVn7aOK+PXHowkgLmdz3KXfu937ZO7b4Hj9VE/zV/72p6fjnVz56RrV+2+PH8f0Pr2l6LM0A99Np\noe5o97NlcnzdfU93517u79h5lbHzR8Vwp9y12ukmSvXuu5ilUb0ven5JZkMz3t2FnN2anOo96DgA\n/MMLm/DyxOGm8740UcE/vrgJb16yPJPvw5E00IF83zBt97O1/Mkz92uNy6navTsx5Vm9c8ydjIk7\nJBNmp5sgvrssuZaiJxU25FrtLuSMvQcdB4D/3fuYfx/2+D+ehiTj6Y6k4+pA8pkwvj9btSqOyX2J\nzutVxJ2YWLlT7oJ2ugnDd5clqWIaLxvoYV2Y4Rm/3YXcuzUFHQeAyWn/ivJki8dNMVGlA2b2NTU1\nvdH3ZyuFHZI6ciemtHAnJvuYXrjkMLGAyeSwjE2bdpgKdKBYoe5ly9x1wOBOTEQ2WPT8ImMBH/UN\n1qIxGeiAmVAHGOymMdzJKjMYxlDvv+Mt1VuwCOkt9XcCMm7IT81M4Yk9O/DeTe9rmiYZtU1YpgMd\nYKgXHcOdrHK0924oZnC09x68rfqVpmMmq3eHOzSjTpd0Fjm5p0FGbRPUJ9NMBTrAUE8bw50Kb/n+\naqhx9xkMYw6vAQLM6THMYPiM6j2NgHeEDfqgRU5h23hfM02dEOoAg92N4U7WONp7t+frM6t3IN2A\nd7QL3dGR+9oucgLqVXu1MfmiWgN+OfASzuu/MbX++jEZ6ABDPWsMd7KCu2oH0LZ6B7IJeD+VShmT\nJwagrkVMB185hKneD6NUOme+zfArhwA4YVjF5IkBLF121XybtJgOdIChnhcuYqKOEBQQ3qr99OP3\ntHxOGkEWZPz1Hb6LmMZf/1WkNiY5i43SqNTTHFdnsLfHyp2scApjp6t2hwCndLTt85xAy6qKn535\nHU5X5I5q4/HwbZJK6xdbmlW6g6EeDsOdOka7N1Yvqv5zonNnFfKr133BSJuo0v5fCkO9eBju1FHC\nzpyJK0zIVyplHD/2A5y/8uOpj4HHlcWQUxaBDnRWqM/0TWLPlQ9i02PX53q7X4Bj7mRQVv8IswiV\nduPQ46/vwOzMUGpj4FG5+5rG2LlXmmPpbp04rs7NOogSSruCd3OHZQVlTC4YAKCZzWDx60fWsqrS\ngc6q1N24WQeRIVkGvGO052FoTQEBtKY4+dLjvhuCRBm7zzO022GgR8PNOshq5wwMp3Z3SD9ZBny7\nDUG82/kVNbCDZBnogB2hDnCzDqJUZDUGnPaGIHlxrl/WlbotwQ5wsw7qEllX7w4nnNKq5LPYECQL\nWVfnbjYFulsRN+tguFMq8gp4IL2Qf2v1VqPny0qeYQ7YG+huW+7/ZN5dOENguIvIWgB3AVgBQAFs\nU9V/9bS5EsB9wHwJ8zNV/brZrhJFk3YlX1R5h7mjG0K9yMJU7hUAX1bVp0TkHAADIvKIqu73tPuN\nqt5gvotUpIURUaRRvc+Vynhh3d246HefwlmVJcFPQHPYFSXoKyhjuOcurK595ow3Y6NIGuRxrmc7\nDPTiCHxDVVVfVdWnGp+fAHAASHGLGzpDkRZGRGX6H/uR/u0oLz6MI/3bYz3f/eZhnhXuaM/DmJaX\nI70Z6+27if4nvZ7A6TdHGezFEmnMXUQuAHAJAL+UuVxE9gIYBnCrqj6XuHdUuIURcZiq4OdKZYws\n+y0gipFlu7F25JrE1WargEyzwj89pVKbplJm/csm6fVkmBdb6HAXkTcB+CmAL6lq2XP4KQDrVPWk\niFwP4F4AG3zOsRXAVgBY1FvMe3IUTdEWRsRlIuCP9G9vuhZH+rfjwqMfMdG9M6QZtIO/90tgWX0h\nFLSGk+O/TO37aCfO9WSgd45Q89xFZAHqwX6Pqv7Me1xVy6p6svH5gwAWiMh5Pu22qepmVd18Vm9f\nwq7br9XCiNm+yZx7Fk+S/7o7Vab21ENXe6oYWbYbcyVvnVFsRfk+ovSDwy6dKTDcRUQAfAfAAVX9\nlxZtVjbaQUQua5x3zGRHu1ERF0aYECck3FWmw6k2O0lRvo+gfjDQO1+YYZktAD4NYJ+IPNN47GsA\n1gGAqt4J4GMAPiciFQDTAG5S71YyFFkRF0aY4oRG2KGa8uKh+SrToT1VlBcPGe9bmoryfbTqx6Qc\nYqBbQvLK4HMXrtDLV30il9em4slrwVM3Y4h3poeGbh9Q1c1B7bhClQohaiVP0THMuwvDnQqFIW8O\nw7y7MdypkNzBxKAPxiAnL4Y7FR6DvhmDnMJguFNH8Qab7WHPIKe4GO7U0fzCr9MCnwFOaWC4k3Xa\nhWXWwc/gprww3KmrMGypW3APVSIiCzHciYgsxHAnIrIQw52IyEIMdyIiCzHciYgsxHAnIrIQw52I\nyEIMdyIiCzHciYgsxHAn8pjpm8ST1/0Ys32TeXeFKDaGO5HH4MadeGPFMA5tfDLvrhDFxnAncpnp\nm8Twhv2AAMMb9rN6p47FcCdyGdy4EwoFAChqrN6pYzHciRqcql1LNQCAlmqs3qljMdyJGtxVu4PV\nO3UqhjtRw3j/sfmq3aGlGsb7X82pR0TxcScmooYt938y7y4QGcPKnYjIQgx3IiILMdyJiCzEcCci\nshDDnYjIQgx3IiILMdyJiCzEcCcishDDnYjIQoHhLiJrRWSHiOwXkedE5Is+bUREviEih0Rkr4i8\nK53uEhFRGGFuP1AB8GVVfUpEzgEwICKPqOp+V5vrAGxofLwbwLcafxIRUQ4CK3dVfVVVn2p8fgLA\nAQCrPc1uBHCX1u0EsFREVhnvLRERhRJpzF1ELgBwCQDvPVBXAzji+voVnPkLgIiIMhL6rpAi8iYA\nPwXwJVUtx3kxEdkKYGvjy9mHhm5/Ns55MnYegNG8OxEC+2kW+2kW+2nOW8I0ChXuIrIA9WC/R1V/\n5tNkGMBa19drGo81UdVtALY1zrlbVTeHef08sZ9msZ9msZ9mdUo/wwgzW0YAfAfAAVX9lxbN7gfw\nmcasmfcAmFBV7nBARJSTMJX7FgCfBrBPRJ5pPPY1AOsAQFXvBPAggOsBHAIwBeCz5rtKRERhBYa7\nqj4BQALaKIBbIr72tojt88J+msV+msV+mtUp/Qwk9VwmIiKb8PYDREQWyiTcRaRXRJ4WkQd8jhXm\n1gUB/bxSRCZE5JnGx9/n1MfDIrKv0YfdPscLcT1D9LMo13OpiPxERJ4XkQMi8oee40W5nkH9zP16\nishFrtd/RkTKIvIlT5vcr2fIfuZ+PZMKPc89oS+ivrJ1ic+xIt26oF0/AeA3qnpDhv1p5X2q2mou\nbpGuZ7t+AsW4nv8K4CFV/ZiInAXgbM/xolzPoH4COV9PVX0BwCagXiihPh36555muV/PkP0EivHz\nGVvqlbuIrAHwQQDfbtGkELcuCNHPTlGI69kJRORcAH+E+lRfqOqcqo57muV+PUP2s2iuBjCoqkOe\nx3O/nh6t+tnxshiWuR3AVwDUWhwvyq0LgvoJAJc3/iv5CxF5R0b98lIA20VkoLHi16so1zOon0D+\n1/OtAI4D+M/GcNy3RWSxp00RrmeYfgL5X0+3mwD8t8/jRbiebq36CRTrekaWariLyA0ARlR1IM3X\nSSpkP58CsE5V/wDAvwG4N5POnekKVd2E+n9vbxGRP8qpH0GC+lmE61kC8C4A31LVSwBMAvibHPoR\nJEw/i3A9AQCNYaMPAfhxXn0II6CfhbmecaVduW8B8CEROQzgBwCuEpG7PW1C3bogZYH9VNWyqp5s\nfP4ggAUicl7G/YSqDjf+HEF9nPAyT5MiXM/Afhbker4C4BVVdW6E9xPUQ9StCNczsJ8FuZ6O6wA8\npaqv+RwrwvV0tOxnwa5nLKmGu6p+VVXXqOoFqP/351eq+ilPs9xvXRCmnyKyUkSk8fllqF+7sSz7\nKSKLpX5PfTT+W/5+AN6br+V+PcP0swjXU1WPATgiIhc1HroawH5Ps9yvZ5h+FuF6unwcrYc6cr+e\nLi37WbDrGUtWs2WaiMjNQPFvXeDp58cAfE5EKgCmAdyk2a8AWwHg542fuRKA76vqQwW8nmH6WYTr\nCQBfAHBP47/oLwH4bAGvZ5h+FuJ6Nn6ZXwvgL12PFe56huhnIa5nElyhSkRkIa5QJSKyEMOdiMhC\nDHciIgsx3ImILMRwJyKyEMOdiMhCDHciIgsx3ImILPT/IFnKp+a3C80AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.contourf(x0coords,x1coords,pdf2)\n", "plt.hold(True)\n", "plt.plot(X[100:150,0],X[100:150,1],'g^')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# GMM definition\n", "A GMM models the likelihood of each class as the weighted sum of $K$ different Gaussians, like this:\n", "$$p_{X|Y}(\\vec{x}|y) = \\sum_{k=1}^K c_k {\\mathcal N}(\\vec{x};\\vec\\mu_{yk},\\Sigma_{yk})$$\n", "\n", "Notice that we've increased the number of parameters by a factor of $K$. A Gaussian model has just one $D\\times D$ covariance matrix per class. A GMM has $K$ of them. There are two possible solutions to avoid parameter overload:\n", "* Make $K$ small, or\n", "* Make the covariance matrix diagonal\n", "In this lecture I'll use the first solution, and just set $K=2$ Gaussians per class. In the real world, though, people are more likely to use the second solution --- to set $K$ to a pretty big number, and then to use diagonal covariance matrices." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Constraints on the mixture weights\n", "Here are some facts you know about pdfs in general:\n", "* They must be non-negative, i.e., $p_X(\\vec{x})\\ge 0$ for all $\\vec{x}$.\n", "* They must integrate to 1.0, i.e., $$\\int p_X(\\vec{x})d\\vec{x}=1$$\n", "\n", "The Gaussian is a valid pdf. That means that these things are true for a Gaussian:\n", "$${\\mathcal N}(\\vec{x};\\vec\\mu,\\Sigma)\\ge 0,~~~\\mbox{and}~~~\\int {\\mathcal N}(\\vec{x};\\vec\\mu,\\Sigma)d\\vec{x}=1$$\n", "\n", "Since these things are already true for each of the Gaussians separately, making them true for the GMM is simple. We just have to choose the right mixture weights, $c_k$. In fact, the rules are easy:\n", "* The mixture weights have to be non-negative, $c_k\\ge 0$\n", "* The mixture weights have to sum to one: $$\\sum_{k=1}^K c_k = 1$$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Constraints on the covariance matrices\n", "The covariance matrix has to be positive semi-definite, meaning that it has to have all non-negative eigenvalues.\n", "\n", "In fact, a positive-semidefinite covariance matrix is kind of hard to deal with --- it's better if you keep it positive definite, instead. This can be done by finding the eigenvalues, then if there are any zero-valued eigenvalues, just add a small constant to each one. Usually people add a constant somewhere between 0.01 and 0.001, then check to see if their results make sense." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Learning a GMM from data\n", "OK, the title of this slide is a lie. I'm not going to tell you, yet, how to learn a GMM from data. I'll tell you about that in the next lecture.\n", "\n", "For now, let's just look at the scatter plot for class $y=2$, the Iris-virginica class. Notice that you could draw an ellipse around all the data points with $x_0<6.8$, and you could draw another ellipse around all the data points with $x_0>6.8$. So let's arbitrarily divide the data into those two halves, and find the mean and covariance of each of those two half-datasets.\n", "\n", "WARNING: this is not the way you should ever train a GMM in practice. You'll learn a much better method in the next lecture." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8XGV97/HPtwnILZJAdqkhCbGhgIgJkQ2JCEpB5NZT\nS2tPK0QPHIHiAYUWe6jYWvFCK7VUaNQU8AB9JZGD3FRoVXqECEQSNhLDJdEmEkKEwA5yCVaISX7n\nj2dNmExm9qzZe3Zm1trf9+u1X3vWWs8867cu+zfPftaaZykiMDOzcvmNTgdgZmbt5+RuZlZCTu5m\nZiXk5G5mVkJO7mZmJeTkbmZWQk7ubSYpJO2/A9YjSddJekHSkhbfu1rSe4YrtnaSdL2kz3X7uiUd\nKGmppA2SPpbzPYM+V6qP4VDOhU6S9JikY3KWHdI5O5TzSNKnJc2rmj5V0lOSXpE0Y7AxDbeOJHdJ\nR0laJOklSb+QdL+kw4dY5xmS7quZ17HE0Ey9eFt0FHA8MDEijmhTWG0n6RRJ90l6UdI6SddKGjNM\n67pH0lnDUXcO/xu4OyLGRMRVO/jcK8S5UCsi3hoR9wy1HknHSFrbhpDy+iJwfkTsEREPD/fKJJ2Z\nNQRulpQ7Z+/w5C7pjcAdwD8DewH7ApcCr+3oWJqRNLrTMQxgP2B1RPyy04E0sSfwOWAC8BbS8f6H\njkY0PPYDHuvguotwLgBd/3eVxw471pL+G3A5cCLpb+fLud8cETv0B+gFXmxS5mxgObABeBx4ezb/\nr4BVVfNPzea/BXgV2Ay8ArwInAP8GtiYzft2VnYCcAvQDzwBfKxqvZ8GbgbmAS8DZ9WJ7XpgLnBX\nFsdCYL+q5QHsn73eE/jXbF1PAn9N+kDdLt4G+2EC8C3gF8BK4Oxs/odr3n9pi/txNfCe7PURwA+z\nffYMMAfYOVsm4J+A57L98QhwSLbs5KzODcDPgY/nPP5/CDwywPIZwI+yev8vcCPwuWzZOFLDoB94\nIXs9MVv2+Wx/vJrtkznZ/CuBp7L4HwKObuFcvb6y7mz694Cl2b5aBEzL5n+/Zt11z7069QdwLvCf\nWZ1fBpQtm5rV+zywHpgPjK1672rgPXnOhRx1XZwdww3AT4Dj6tSxc7btH82mRwH3A5/KuS9XZ+tZ\nRmrIja45D3cFbsiO63LSf0Jra97/8ez9L2Xnxi7A7sCvgC3Z9r8CTGhwLL8M3Jlt52JgatXyhucJ\nKS/MA96Q1R/AL4FVDbZ1oLqOAPqyZc8CVwywz44C1gAzsuk9SPnm07n2ed4TvV0/wBuzk+wG4CRg\nXM3yP85OtMNJyWV/suSZLZtASpB/ku3gN2XLzgDua/LH+RvZzv5UdrL+NvAz4ISqg/hr4A+ysrs2\nOEk2AO/KDvaV1etl2+T+r8A3gTHAFOCnwIcbxVtnXT8AvpKdxIeSktqxed7fZD+u5vU/qsOAWaQ/\ntimkP6wLs2UnZPtrbFbHW6r29zOVk5aUdN+e8/h/CbixwbKdSR+Cfw7sBLw/Ox6V5L438EfAbtk+\n/QZwe9X776HmAxmYnb1vNHARsA7YpeqPp2FDo/r8IX3oPAfMJCW2/5HtxzfUWzc1516D+oP0ATUW\nmJwd3xOzZfuTulreAPRk58KXqt5bfQybnQsN6wIOJCWiCdn0FKqSXk09h5CS71uATwIPAKNyHvfV\npA+HSWR/VzXb8PekxDUOmEhK4rXJfQnp738v0nl6brbsmOqyAxzL50nJdTTpA+7GquUDnSefBubV\n+xtvsK6B6voh8MHs9R7ArAZ1CLgAeGvN/N1Ijdx9mu7zPAem3T/ZyXE9sBbYRGqd7pMt+y5wQc56\nlgLva3SCs31ynwmsqSnzCeC6qoP4gxwnSfVJsQep1TSp+sCTEsBG4OCqsn8G3JPzD3JSVu+Yqnl/\nB1yf8/0N9yNVf1R1ll0I3Ja9Ppb0gTQL+I2acmuy7XljC8f9eFJyOKDB8ncBT5O1XrN5i2iQJEkf\neC9UTd9Dnf+2at7zAjA9Z7xbzx/gq8Bna5b/BHh3vXXXnnsN6g/gqKrpm4C/alD2D4CH6x3DZufC\nQHVl5+pzpP8Cdsrx3ouy7X4B+J0W1rka+J+NzkOqGlnZ9Flsn9xnV01fDszNXh9DvuR+bdX0ycCK\nPOcJLSb3JnX9gNQNPT7v+wf705ELqhGxPCLOiIiJpNbABFKLDlJSW1XvfZI+lN2R8KKkF7P3jm9h\n1fsBEyrvz+q4BNinqsxTOerZWiYiXiF1m0yoKTOe1Pp8smrek6R+szwmAL+IiA2DfH/D/VhN0gGS\n7sgudr4MXEa2TyPi+6Rumi8Dz0m6OrtmAqkFfTLwpKSFkt7RZD2zgAXA+yPipw2KTQB+HtlfQWbr\n/pO0m6R/kfRkFusPgLGSRg2w3o9LWp5dvH+R1FXWyjlTsR9wUc25M4ntj3ur1lW9/i9SYwFJ+0i6\nUdLPs22dN8i4B6wrIlaSPtA/TTrGN0oaaJtuIO2Lf4uI/2wxlIH+tibULK9Xtu6+akHD97fxPGlW\n14eBA4AVkh6U9HuDWUceHb8VMiJWkD5VD8lmPUXqI9yGpP2Aa4Dzgb0jYizwKOnfF0ifpttVXzP9\nFPBERIyt+hkTEScP8J56JlXFtQfp38Sna8qsJ3Up7Fc1bzKpqyTPep4G9qq5s6T6/c3U3Y91fBVY\nQWqFvZH0YVfZp0TEVRFxGHAw6aT8y2z+gxHxPuA3gdtJrc66stvFvkVquf2/AWJ5BthXkqrmTa56\nfRGpG2FmFuu7KquohFuz3qNJfbf/ndT9N5bUX1tdf15PAZ+vOXd2i4ivNyif5zwayGVZHW/LtnU2\ng4u7aV0RsSAijiKdqwF8YYC6vkLqSjpB0lEtxjHQPnmG1B1TMalRwRbrbaqd50mzuiLiPyPiA6S/\nmy8AN0vafSjxN9KJu2UOknSRpInZ9CTgA6T+O4BrgY9LOiy7f3f/LLHvTjqI/dn7zuT1DwRIFycm\nStq5Zt5vV00vATZIuljSrpJGSTpkELdhnpzdzrkz8FnggYjYpqUREZtJCe/zksZk2/AXpFZTo3ir\n3/8UqUvi7yTtImka6VN/Xr3ydTTaj7XGkC7uvCLpIOAjlQWSDpc0U9JOpOsbrwJbJO0s6XRJe0bE\nr7P3b6kXhKRDgO+QLsR9u0nMPyR1031M0k6S/pDUR1od66+AFyXtBfxtzftrj/eYrL5+YLSkT5Gu\n+QzGNcC52f6QpN2VbvNsdFtnbSytGkO6ePeSpH3JPlTbXZfS/fnHSnoD6fhWLk5uR9IHSddozgA+\nBtyQNW4qtyMOJcneBHxC0rgsxvNbeO+zwN6S9hzkutt5ngxYl6TZknoiYgvpIjo02N9D1YmW+wZS\n3/diSb8kJfVHSa0yIuIbpDsfFmRlbwf2iojHgX8kJYBngbeRrtZXfJ90e9I6SeuzeV8DDs7+jb49\nS7i/R+qrfYLUur6W9G9TKxaQEssvSCf77AblPkpKij8D7sve938GiLfWB0gXuJ4GbgP+NiL+I0+A\njfZjnaIfB07LylxDuguh4o3ZvBdI3SPP8/ptjB8EVmf/5p8LnN4glItIF/G+pvSlj1ck1b2NLCI2\nku6mOYO0b/8EuLWqyJdId1WsJ50336mp4krg/Upf5rmKdN3hO6TrBk+SktfWD2FJR0t6pUHctbH1\nke4+mkPaHyuzOBvZ5tzLs44alwJvJ7X67mTb/dDOut5Aupi5ntRt8Zuk61DbkDSZtP8/FBGvRMQC\n0l0f/5QVmURqjAzWZ0jX4J4A/oN011qu26Oz//6/Dvws29+tdpUNeJ60ua4Tgcey8+5K4E8j4leD\nXNeAKrddWU6SriddvPnrTsdi1i0kXQt8IyK+26b6PkJKfO9uR30jUdG/TGBmXSAihvTNYElvInVj\n/RD4HdJ/fHPaENqI5eRuZt1gZ+BfgDeT+qJvJF28tUFyt4yZWQl1/FZIMzNrv451y4wfPz6mTJnS\nqdWbmRXSQw89tD4iepqV61hynzJlCn19fZ1avZlZIUl6snkpd8uYmZWSk7uZWQk5uZuZlZCTu5lZ\nCTm5m5mVUNPkno1IuETSj5WeVn5pnTJ7Svp2VZkzhydcM7MdaMM6uO4k2PBsd9dZR56W+2ukR7tN\nJ42meKLSgxeqnQc8npU5BvjHRkPZmpkVxsLLYc0DsHCgIe67oM46mib3SCrDou6U/dSOWRDAmOwh\nC3uQhmvd1M5Azcx2qA3rYOl8iC3pdzta2sNRZwO5+tyzh1osJT1r8a6IWFxTZA7puahPA4+Qnt25\n3QD0ks6R1Cepr7+/f4ihm5kNo4WXpyQM6Xc7WtrDUWcDuZJ7RGyOiENJj8E6Inu6TrUTSA+rnkDq\nupmj15+1WV3P1RHRGxG9PT1Nvz1rZtYZlRb25o1pevPGobe0h6POAbR0t0xEvAjcTXqaSLUzgVuz\nLpyVpKepHNSeEM3MdrDqFnbFUFvaw1HnAPLcLdMjaWz2elfgeNIDlautAY7LyuxDeojxz9obqpnZ\nDrJ2yest7IrNG9P8bqpzAHkGDnsT6UG4o0gfBjdFxB2SzgWIiLmkh0RfL+kR0lO+L46IRs8FNTPr\nbufeV4w6B9A0uUfEMmBGnflzq14/Dby3vaGZmdlg+RuqZmYl5ORuZlZCTu5mZiXk5G5mVkJO7mZm\nJeTkbmZWQk7uZmYl5ORuZlZCTu5mZiXk5G5mVkJO7mZmJeTkbmZWQk7uZmYl5ORuZlZCTu5mZiXk\n5G5mVkJ5HrO3i6Qlkn4s6TFJlzYod4ykpVmZhe0P1cwKY8M6uO6kYXv4szWXp+X+GnBsREwHDgVO\nlDSrukD2jNWvAL8fEW8F/rjtkZpZcSy8HNY8MGwPf7bmmib3SF7JJnfKfqKm2GnArRGxJnvPc22N\n0syKY8M6WDofYkv67dZ7R+Tqc5c0StJS4DngrohYXFPkAGCcpHskPSTpQw3qOUdSn6S+/v7+oUVu\nZt1p4eUpsUP67dZ7R+RK7hGxOSIOBSYCR0g6pKbIaOAw4BTgBOBvJB1Qp56rI6I3Inp7enqGGLqZ\ndZ1Kq33zxjS9eaNb7x3S0t0yEfEicDdwYs2itcB3I+KXEbEe+AEwvT0hmllhVLfaK9x674g8d8v0\nZBdMkbQrcDywoqbYN4GjJI2WtBswE1je7mDNrMutXfJ6q71i88Y033ao0TnKvAm4QdIo0ofBTRFx\nh6RzASJibkQsl/QdYBmwBbg2Ih4dtqjNrDude1+nI7CMImpvfNkxent7o6+vryPrNjMrKkkPRURv\ns3L+hqqZWQk5uZuZlZCTu5lZCTm5m5mVkJO7mVkJObmbmZWQk7uZWQk5uZuZlZCTu5lZCTm5m5mV\nkJO7mVkJObmbmZWQk7uZWQk5uZuZlZCTu5lZCTm5m5mVUJ7H7O0iaYmkH0t6TNKlA5Q9XNImSe9v\nb5hm25q7cBWLVq3fZt6iVeuZu3DVoMqZlU2elvtrwLERMR04FDhR0qzaQtlj+L4AfK+9IZptb9rE\nPTl/wcNbE/eiVes5f8HDTJu456DKmZVN02eoRnoO3yvZ5E7ZT71n830UuAU4vG3RmTVw5NTxzDlt\nBucveJjZMyczb/Ea5pw2gyOnjh9UObOyydXnLmmUpKXAc8BdEbG4Zvm+wKnAV5vUc46kPkl9/f39\ng43ZDEiJe/bMyVz1/ZXMnjm5YcLOW86sTHIl94jYHBGHAhOBIyQdUlPkS8DFEbGlST1XR0RvRPT2\n9PQMLmKzzKJV65m3eA0fO3Z/5i1es13feqvlzMqkabdMtYh4UdLdwInAo1WLeoEbJQGMB06WtCki\nbm9bpGZVKn3nlS6WWVP33ma61XJmZZPnbpkeSWOz17sCxwMrqstExJsjYkpETAFuBv6XE7sNp2Vr\nX9omQVf61petfWlQ5czKRul66QAFpGnADcAo0ofBTRHxGUnnAkTE3Jry1wN3RMTNA9Xb29sbfX19\nQwjdzGzkkfRQRPQ2K5fnbpllwIw68+fWKU5EnJEnQDMzGz7+hqqZWQk5uZuZlZCTu5lZCTm5W6kV\nYWyZM65bwjX3bhvPNfeu4ozrlnQoIisDJ3crtSKMLfPO/ffmsjtXbE3w19y7isvuXME799+7w5FZ\nkbX0JSazoinC2DJnHz0VgMvuXMFdjz3Lg6tf4JJTDto632ww3HK30ivC2DJnHz2Vw6eMY8nqFzh8\nyjgndhsyJ3crvSKMLXPNvat4cPULHDFlHA+ufmG7PnizVrlbxkqtCGPLVPrYK10xlWnALXgbNLfc\nrdSKMLbM/Suf36aP/eyjp3LJKQdx/8rnOxyZFVnTsWWGi8eWMTNrXd6xZdxyNzMrISd3M7MScnI3\nMyshJ3czsxLK8ySmXSQtkfRjSY9JurROmdMlLZP0iKRFkqYPT7hmwyfvODSdGq+m5fVuWAfXnQQb\nnh3WuLpu3Qbka7m/BhwbEdOBQ4ETJc2qKfME8O6IeBvwWeDq9oZpNvzyjkPTqfFqWl7vwsthzQOw\n8AvDGlfXrduAFm+FlLQbcB/wkYhY3KDMOODRiNh3oLp8K6R1o0rCbDYOTd5ynYqPDevgyumw6VUY\nvQtcsAzG7DPs8XV83SNAW2+FlDRK0lLgOeCuRok982Hg3xvUc46kPkl9/f39eVZttkPlHYemU+PV\n5F7vwsshtqTXsWXHtqA7uW7bKldyj4jNEXEoMBE4QtIh9cpJ+l1Scr+4QT1XR0RvRPT29PQMNmaz\nYZN3HJpOjVeTa70b1sHS+bB5Y5revDFN74j+706u27bR0t0yEfEicDdwYu0ySdOAa4H3RYS/N22F\nUz0OzV+898CtQwXXu4iZp1yn4tum5Vyxo1rQnVy3bSPP3TI9ksZmr3cFjgdW1JSZDNwKfDAifjoc\ngZoNt7zj0HRqvJrc61275PWWc8XmjWn+cOvkum0bTS+oZi3yG4BRpA+DmyLiM5LOBYiIuZKuBf4I\neDJ726ZmHf6+oGpm1rq8F1SbDvkbEcuAGXXmz616fRZwVqtBmpnZ8PA3VM3MSsjJ3cyshJzczcxK\nyMndzKyEnNzNMoUZECyvZ5bB302CdY8OrR4rJCd3s0xhBgTL69az4bWX4ZYPtyFKK5qmt0KajRSV\nLwXt6AHBhmW9zyyD/uy7hv0rUuv9t+qOGmIl5Za7WZWuHxAsr1vP3nbarfcRx8ndrEpXDwiWV3Wr\nvaLSercRw8ndLNP1A4LlVdtqr3DrfURxcjfLdP2AYHm9sLq1+VZKLT2JqZ08cJiZWeva+iQmMzMr\nFid3M7MScnI3MyshJ3czsxLK85i9XSQtkfRjSY9JurROGUm6StJKScskvX14wrVuk3dclHaX66Rj\nv3gPn7xt2TbzPnnbMo794j2Dqq9Q+2bDOrjupOYPvM5brlPKsh0DyNNyfw04NiKmA4cCJ0qaVVPm\nJOB3sp9zgK+2NUrrWnnHRWl3uU56x9S9mL/4qa0J/pO3LWP+4qd4x9S9BlVfofbNwsthzQPNH3id\nt1ynlGU7BtDSrZCSdgPuAz4SEYur5v8LcE9EfD2b/glwTEQ806gu3wpZHpUk02xclHaX66RKQp+w\n5y48/dKrnD5zEp8/ddqg6yvEvtmwDq6cDptehdG7wAXLYMw+gy/XKQXfjrbeCilplKSlwHPAXdWJ\nPbMv8FTV9NpsXm0950jqk9TX39+fZ9VWAHnHRWl3uU76/KnTtib2CXvuMqTEDgXZNwsvh9iSXseW\nxq3ZvOU6pSzb0USu5B4RmyPiUGAicISkQQ0vFxFXR0RvRPT29PQMpgrrQnnHRWl3uU765G3Ltib2\np196dbs++FZ1/b7ZsA6WzofNG9P05o1purYvOm+5TinLduTQ0t0yEfEicDdwYs2inwOTqqYnZvOs\n5PKOi9Lucp1U6ZI5feYkFn3iOE6fOWmbPvhWFWLfVLdiK+q1ZvOW65SybEcOee6W6ZE0Nnu9K3A8\nUDPkHN8CPpTdNTMLeGmg/nYrj7zjorS7XCf9cNUvtulj//yp0zh95iR+uOoXg6qvEPtm7ZLXW7EV\nmzem+YMp1yll2Y4cml5QlTQNuAEYRfowuCkiPiPpXICImCtJwBxSi/6/gDMjYsCrpb6gambWurwX\nVJs+iSkilgEz6syfW/U6gPNaDdLMzIaHv6FqZlZCTu5mZiXk5G5mVkJO7lZqXTEeizVW4LFbup2T\nu5VaV4zHYo0VeOyWbufkbqVWuRf8/AUPc8X3frL1S0DdOKTBiFP5FmhsKdy3P4vAyd1Krwhj1YxI\nBR+7pds5uVvpFWGsmhGnBGO3dDsndyu1IoxVMyKVYOyWbufkbqVWhLFqRqQSjN3S7Vp6WEc7eWwZ\nM7PWtfVhHWZmVixO7mZmJeTkbmZWQk7uZmYl5ORuZlZCeR6zN0nS3ZIel/SYpAvqlNlT0rcl/Tgr\nc+bwhGv1DMfgWN0+4Fbe+FrZjjOuW8I19247/5p7V3HGdb49r+M6NcBYgQc2y9Ny3wRcFBEHA7OA\n8yQdXFPmPODxiJgOHAP8o6Sd2xqpNTQcg2N1+4BbeeNrZTveuf/eXHbniq0J/pp7V3HZnSt45/57\nD/PWWFOdGmCswAObtXyfu6RvAnMi4q6qeZ8AJpGS/BTgLuCAiNqvoL3O97m3VyVpzZ45mXmL17Rl\ncKzhqLOd8sbXynZUEvrhU8bx4OoXuOSUgzj76KnDvSk2kA3r4MrpsOlVGL0LXLAMxuxT3vU2MSz3\nuUuaQnqe6uKaRXOAtwBPA48AF9RL7JLOkdQnqa+/v7+VVVsTwzE4VrcPuJU3vla24+yjp3L4lHEs\nWf0Ch08Z58TeDTo1wFjBBzbLndwl7QHcAlwYES/XLD4BWApMAA4F5kh6Y20dEXF1RPRGRG9PT88Q\nwrZawzE4VrcPuJU3vla245p7V/Hg6hc4Imu51/bB2w7WqQHGSjCwWa7kLmknUmKfHxG31ilyJnBr\nJCuBJ4CD2hemDWQ4Bsfq9gG38sbXynZUumQuOeUgbjr3SC455aBt+uCtAzo1wFgJBjbLc7eMgK8B\nyyPiigbF1gDHZeX3AQ4EftauIG1gwzE4VrcPuJU3vla24/6Vz2/Tx3720VO55JSDuH/l88O8NdZQ\npwYYK8HAZk0vqEo6CriX1Jde+Si7BJgMEBFzJU0ArgfeBAj4+4iYN1C9vqBqZta6vBdURzcrEBH3\nkRL2QGWeBt6bPzwzMxtO/oaqmVkJObmbmZWQk7uZWQk5uVtdZRlnpZWxZbp9PB2zVji5W11lGWel\nlbFlun08HbNW+Bmq1lBZxllpZWyZbh9Px8zPULUhK8s4K62MLdPt4+mY5eXkbg2VZZyVVsaW6fbx\ndMzyavolJhuZqsdZOfvoqVungUK14KvHljly6nhmTd17m+nBljXrdm65W11lGWellbFlun08HbNW\n+IKqmVmB+IKqmdkI5uRuZlZCTu5mZiXk5G5mVkJ5nsQ0SdLdkh6X9JikCxqUO0bS0qzMwvaHaiOB\nx3exrrJhHVx3UqGenVqRp+W+CbgoIg4GZgHnSTq4uoCkscBXgN+PiLcCf9z2SG1E8Pgu1lUWXg5r\nHijUs1Mrmib3iHgmIn6Uvd4ALAf2rSl2GukB2Wuycs+1O1AbGSr3lp+/4GGu+N5P/CUi65wN62Dp\n/PRg7KXzC9d6b6nPXdIUYAawuGbRAcA4SfdIekjShxq8/xxJfZL6+vv7BxOvjQAe38W6wsLLU2KH\n9LtgrffcyV3SHsAtwIUR8XLN4tHAYcApwAnA30g6oLaOiLg6Inojorenp2cIYVuZeXwX67hKq33z\nxjS9eWPhWu+5kruknUiJfX5E3FqnyFrguxHxy4hYD/wAmN6+MG2kqB7f5S/ee+DWLhoneNuhqlvt\nFQVrvee5W0bA14DlEXFFg2LfBI6SNFrSbsBMUt+8WUs8vot1hbVLXm+1V2zemOYXRNOxZSQdBdwL\nPAJUPsouASYDRMTcrNxfAmdmZa6NiC8NVK/HljEza13esWWaDvkbEfcBylHuH4B/yBeemZkNJ39D\n1cyshJzczcxKyMndzKyEnNzNzEqoMMndA0q1x0jbj0XY3iLEaMVTmOTuAaXaY6TtxyJsbxFitOIp\n1DNUKyf97JmTmbd4jQeUGqSRth+LsL1FiNG6QymfoeoBpdpjpO3HImxvEWK0YilUcveAUu0x0vZj\nEba3CDFawURER34OO+ywaMX9K/tjxme+F/ev7K87bfmMtP1YhO0tQozWPYC+yJFjC9Ny94BS7THS\n9mMRtrcIMVrxFOqCqpnZSFfKC6pmZpaPk7uZWQk5uZuZlZCTu5lZCeV5zN4kSXdLelzSY5IuGKDs\n4ZI2SXp/e8M0M7NWNH0SE7AJuCgifiRpDPCQpLsi4vHqQpJGAV8AvjcMcZqZWQuattwj4pmI+FH2\negPpwdf71in6UeAW4Lm2RmhmZi1rqc9d0hRgBrC4Zv6+wKnAV5u8/xxJfZL6+vv7W4vUzMxyy53c\nJe1BaplfGBEv1yz+EnBxRGwZqI6IuDoieiOit6enp/Vozcwslzx97kjaiZTY50fErXWK9AI3SgIY\nD5wsaVNE3N62SM3MLLemyV0pY38NWB4RV9QrExFvrip/PXCHE7uZWefkabm/E/gg8Iikpdm8S4DJ\nABExd5hiMzOzQWqa3CPiPkB5K4yIM4YSkJmZDZ2/oWpmVkJO7mZmJeTkbmZWQk7uZmYl5ORuZlZC\nTu5mZiXk5G5mVkJO7mZmJeTkbmZWQk7uZmYl5ORuZlZCTu5mZiXk5G5mVkJO7mZmJeTkbmZWQk7u\nZmYl1DS5S5ok6W5Jj0t6TNIFdcqcLmmZpEckLZI0fXjCtaKau3AVi1at32beolXrmbtwVYciMiu3\nPC33TcBFEXEwMAs4T9LBNWWeAN4dEW8DPgtc3d4wreimTdyT8xc8vDXBL1q1nvMXPMy0iXt2ODKz\ncsrzmL1ngGey1xskLQf2BR6vKrOo6i0PABPbHKcV3JFTxzPntBmcv+BhZs+czLzFa5hz2gyOnDq+\n06GZlVJLfe6SpgAzgMUDFPsw8O8N3n+OpD5Jff39/a2s2krgyKnjmT1zMld9fyWzZ052YjcbRrmT\nu6Q9gFtTLAmBAAAEhElEQVSACyPi5QZlfpeU3C+utzwiro6I3ojo7enpGUy8VmCLVq1n3uI1fOzY\n/Zm3eM12ffBm1j5Nu2UAJO1ESuzzI+LWBmWmAdcCJ0XE8+0L0cqg0sde6YqZNXXvbabNrL3y3C0j\n4GvA8oi4okGZycCtwAcj4qftDdHKYNnal7ZJ5JU++GVrX+pwZGblpIgYuIB0FHAv8AiwJZt9CTAZ\nICLmSroW+CPgyWz5pojoHaje3t7e6OvrG0LoZmYjj6SHmuVXyHe3zH2AmpQ5Czgrf3hmZjac/A1V\nM7MScnI3MyshJ3czsxJycjczK6Gmd8sM24qlfl6/u2Y4jQfK8G2ZsmwHeFu6UVm2A8q/LftFRNNv\ngXYsue8okvry3DbU7cqyHeBt6UZl2Q7wtlS4W8bMrISc3M3MSmgkJPeyjC1flu0Ab0s3Kst2gLcF\nGAF97mZmI9FIaLmbmY04Tu5mZiVUiuQuaXX2cO6lkrYbalLJVZJWZg/yfnsn4swjx7YcI+mlbPlS\nSZ/qRJx5SBor6WZJKyQtl/SOmuWFOC45tqMQx0TSgVUxLpX0sqQLa8oU5Zjk2ZZCHBcASX8u6TFJ\nj0r6uqRdapa3flwiovA/wGpg/ADLTyY9+k+kh3wv7nTMQ9iWY4A7Oh1nzm25ATgre70zMLaIxyXH\ndhTmmFTFPApYR/pCTOGOSc5tKcRxIT2T+glg12z6JuCMoR6XUrTcc3gf8K+RPACMlfSmTgdVZpL2\nBN5FetALEbExIl6sKdb1xyXndhTRccCqiKj9lnjXH5M6Gm1LkYwGdpU0GtgNeLpmecvHpSzJPYD/\nkPSQpHPqLN8XeKpqem02rxs12xaAI7N/zf5d0lt3ZHAteDPQD1wn6WFJ10ravaZMEY5Lnu2AYhyT\nan8KfL3O/CIck1qNtgUKcFwi4ufAF4E1wDPASxHxvZpiLR+XsiT3oyLiUOAk4DxJ7+p0QEPQbFt+\nBEyOiGnAPwO37+gAcxoNvB34akTMAH4J/FVnQxqUPNtRlGMCgKSdgd8HvtHpWIaqybYU4rhIGkdq\nmb8ZmADsLmn2UOstRXLPPvmIiOeA24Ajaor8HJhUNT0xm9d1mm1LRLwcEa9kr/8N2ElSNz5hei2w\nNiIWZ9M3k5JktSIcl6bbUaBjUnES8KOIeLbOsiIck2oNt6VAx+U9wBMR0R8RvyY9j/rImjItH5fC\nJ3dJu0saU3kNvBd4tKbYt4APZVecZ5H+7XlmB4faVJ5tkfRbkpS9PoJ0DJ/f0bE2ExHrgKckHZjN\nOg54vKZY1x+XPNtRlGNS5QM07sbo+mNSo+G2FOi4rAFmSdoti/c4YHlNmZaPS9NnqBbAPsBt2TEc\nDSyIiO9IOhfSA7yBfyNdbV4J/BdwZodibSbPtrwf+IikTcCvgD+N7HJ6F/ooMD/71/lnwJkFPS7N\ntqMwxyRrNBwP/FnVvCIekzzbUojjEhGLJd1M6kbaBDwMXD3U4+LhB8zMSqjw3TJmZrY9J3czsxJy\ncjczKyEndzOzEnJyNzMrISd3M7MScnI3Myuh/w+Xt3oUYmIOhQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Arbitrarily split the class-2 data into two halves, at the threshold x0=6.8\n", "class2 = X[100:150,:]\n", "class2_left = class2[class2[:,0] <= 6.8,:]\n", "class2_right = class2[class2[:,0] > 6.8,:]\n", "plt.plot(class2_left[:,0],class2_left[:,1],'x',class2_right[:,0],class2_right[:,1],'^')\n", "plt.title('Scatter plot of class 2 data: left half as x, right half as ^')" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[[0.7, 0.3],\n", " array([ 6.25714286, 2.90857143]),\n", " array([ 7.36 , 3.12666667])]" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Now we'll estimate the mixture weights, mean, and covariance of the Gaussians on the left half and right half\n", "c_2_0 = class2_left.shape[0]/class2.shape[0]\n", "c_2_1 = class2_right.shape[0]/class2.shape[0]\n", "mu_2_0 = np.mean(class2_left[:,0:2],axis=0)\n", "mu_2_1 = np.mean(class2_right[:,0:2],axis=0)\n", "Sig_2_0 = np.cov(class2_left[:,0:2],rowvar=False)\n", "Sig_2_1 = np.cov(class2_right[:,0:2],rowvar=False)\n", "# Just as a check, print the mixture weights, and the two mean vectors\n", "[ [c_2_0, c_2_1], mu_2_0, mu_2_1 ]" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEICAYAAAC55kg0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XvYJGV95vHvPTDACAgRUA7DISusRNEFokiAGIKSBUTJ\nZomLRhE8ENTLxWsxHtA14qpZs7vGA8FxhF1ARYIoBg0a8YCCBgiMCHIwATnMwCAHMzMM4CDMb/94\nntaemj5Ud1d3V3fdn+t6r3mnqrrq6erq313PU9X9KiIwM7NmWjDtBpiZ2fQ4BMzMGswhYGbWYA4B\nM7MGcwiYmTWYQ8DMrMHmOgQkfV3Sa6e4/d0krZW0ybTaMGmSDpW0ouSy75f0uSG3c7Ckf837949L\nLL+HpJC06TDbG6Bdl0t6wzi3MYhxvAcknSbprEk8ftRtjSofM3tOa/uTUGkISDpO0tWSHpF0f/79\nzZJU5XbKiogjI+Lcqtcr6YR8cPxNYfoxefo5eft3R8RWEfFkiXWeI+mDVbe1zzYjv06btk1bmKfV\n/QMkHwDOyPv3K8WZku6U9JIptGtiygTuON4DEfHhiBg66AZ5/KjbahpJL5V0paRVku6TdJakrXs9\nprIQkHQq8HHgfwE7As8ATgYOBjarajs1cjvwisKZ5WuBf5lGY0Y4w/034Mi2/x+Zp9Xd7sBN025E\nXSmZeE9/3D0t62sb4IPAzsDvALuQanJ3ETHyT97wI8B/7rPcS4EfAWuA5cD72+YdCqwoLH8n8JL8\n+wHAtfmxPwc+mqdvAXwOeAhYBfwz8Iw873LgDfn3ZwLfycs9CHwe2LawrbcDNwCrgb8DtujyPE4A\nrgS+Abw0T3sacF/e4efkaXsAAWya568AXpbnbQXcBhwPnAT8CngcWAt8NS8TwJ5t2z0H+GD7/gLe\nmbf72Tz9aOD6vC9+CDyvx+sRwHuBL7ZNuwh4Tzo0fj1tZ+AS4Be5zW9sm7cot+vfgJuBv2h/HfNj\nvwQ8ANwB/Ne2ee8HPtejfW/M2/tF3v7OefrtwHrgsby/Ni887rOF+e9oey1eC9ydj4H3tD1mAfCu\nvO6HgAuBp/Vo2zF5P6/JjzliiGPuncA9wMPAT4EX9zrWO7Th0MK+vhz4EPCD/Nz3LLRnT+B7pOP7\nQeDvuqz3hfmY2qRt2n8Cbii+bm379fV5v34/Tz8euCs/9//Ohu/lTo/v9rpscIwAh5CO61WkGnJC\nv9pSon5tApyWX8eHgeuAXYvvwV7boHcdOgH4WV73HcCfdWjDjsCjwHZt0/YnvW8WDlKPC+v9E+DG\nnssMu/LCho4AngA27bPcocBzSW+45+UD/I87HdB5WvuB80/Aa/LvWwEH5t//HPgq8JT8Yv4u8NQO\nb8g9gcOBzYEdgO8DHyts6xpS0XoacAtwcpfncQIpBF5FfiMBbwY+TUrhjUIg//+PSG+upwOfAS5q\nW+c55ALfNq1fCDwBfCQ/p0XAfsD9pDfxJqQ31p0UimRh/fvk12Fb4Lfy7/uwYQh8HziTdKDvmw/M\nw/K8/wlckffZrsBPWq9jfp2vA95H6g3+O9Kb4T92eoMX2nYYqSDsn5/fJ8kFpnhsdHn8BvPbXovP\n5H31H4B1wO/k+acAVwGL8/Y+DXyhy7oPIBXSw/Nz3AXYe5BjDngWqZDs3Na+Z/Y61ru8n4ohcDfw\nHNKJx8JCe75ACvgF+bU8pMf+ux04vO3/XwTeVXzd2vbrecCWed8+mxS+h+TX/X+TTnJ6hUC316V9\n2d1JhfSV+bltB+zbr7aUqF9/AdyYXxPlNmxXfA/22gZd6lDeJ2uAZ+XldgKe06UdlwJvavv/3wCf\nzL8fQgqXbj8dX0vgY8AFvZ5/Vd3F7YEHI+KJ1gRJP8zjUo9JehFARFweETdGxPqIuIF0UP5ByW38\nCthT0vYRsTYirmqbvh3phXoyIq6LiDXFB0fEbRFxWUSsi4gHgI922PYnIuLeiPgF6QXdt0+bLgYO\nlbQN6cznvF4LR8Q3SW+mbwNHkQ6cUawH/jI/p8dIPYpPR8TVeV+cS3pDHdhjHb8kPdf/kn8uydMA\nkLQraUjvnRHxy4i4HjiL9HwBXgF8KCJ+ERHLgU+0rfsFwA4R8YGIeDwifkZ6sx9X4rn9GfB/I2JZ\nRKwD3g38nqQ9Sjy2l9Mj4rGI+DHwY9IbHtLQ5XsiYkXe3vuBY7sMb7w+t+2yfCzfExG3Fhfqc8w9\nSQqHZ0taGBF3RsTteV63Y72McyLipoh4IiJ+VZj3K1Ih3Tm/llf2WM8XSMWWPKZ8VJ7Wzfsj4pF8\nHB5L6s1eGRGPk04Cok+7u70u7V4FfCsivhARv4qIh/LxOGpteQPw3oj4aSQ/joiHigv12UavOrQe\n2EfSoohYGRHdhjHPBV4NkG8meSWpV0vel9v2+NnotZR0OOlE8H29nnxVIfAQsH37GyYiDoqIbfO8\nBblRL5T0XUkPSFpNeuNtX3Ibrwf+PXCrpH+WdHSe/lngH4ELJN0r6a8lLSw+WNIzJF0g6R5Ja0hd\nt+K272v7/VHSWVhX+YD/B9KQynYR8YMSz2Mp6Uz7nE4H2oAeiIhftv1/d+DUHL6rJK0inZ3v3Gc9\n55GKeqcg2xn4RUQ83DbtLtLZb2v+8sK89vbsXGjPaaTrRf3s3L6uiFhLOpZ26fqIcrq9xrsDF7e1\n8xZSoe7U1l1JZ8o99TrmIuI24G2ksLk/L9d6nbod62Us7zHvHaQz3Wsk3STpdbmdp+W7rNZKWpKX\nPR/4E0mbk4YUlkXEXZ1Xu9F2NzgmIuJR0mvXS5n3Xtf9PmJtKft69tpGxzoUEY+QTq5OBlZK+gdJ\ne3fZxN+TTgp+m9SDXB0R15R8DsW2Hkh6DY+NiJ7XKasKgX8inXEe02e580lnmrtGxDbAEtJBCema\nwlNaC+Yk3KH1/4j414h4JWko5SPARZK2zGcEp0fEs4GDSGPirbPUdh8mnY08NyKeSkrcKu5aOg84\nlfQG7yk/p6X5MW8u3HrW6UzpUdr2CWncsF3xMctJZ+XtZwhPiYheZ3CQhnN2IhW84hnFvcDTCncY\n7EYaywZYSXoTtc9rb88dhfZsHRFH9WlPa7u7t/4jaUvSmdY9XR+xoX5nnkXLgSMLbd0iIjptbzlp\nvL+fnsdcRJwfEYeQnmeQjuuux3rJ59H1eUfEfRHxxojYmdQLPVPSnpHuwNkq/5ycl72ZFMJHks7A\nzx9guytJw2oASFpEeu1G1Wu/96oto6y31DZ61aGI+MeIOJz0HruV1BveSD6hu5B0nLyG3AsAkPT7\nbUHd6ef325bdL7fzdRHx7X5PqpIQiIhVwOmkg+pYSVtLWiBpX9KYWMvWpLPKX0o6gHRwtfwLsEW+\nxWkh6ex689ZMSa+WtENErCeNgQGsl/SHkp6bC+waUrdsfYdmbk0ap1wtaRfSOGAVvkdK7U+WWPY0\n0pvldaQLyOfpN58h+DlpzLzd9cCrJG0i6Qj6d28/A5ycz1gkacu8P3veIhYRAbwMeHn+vX3ectKF\nuL+StIWk55HOVFuhdyHwbkm/JWkx8Na2h18DPCzpnZIW5eexj6QX9HkekLraJ0raN5+Nfhi4OiLu\nLPFY6Lw/e1kCfEjS7gCSdpDU7aTm7Ny2F+fjfJcuZ3ddjzlJz5J0WH5uvyRdyF2f53U81gd4Lh1J\n+tP8GkG6kB991ns+6VrJi0jDmGVdBLxM0kGSNiP1dqo44fo88BJJr5C0qaTtco2B3rWldcvwCV3W\nexbwPyTtld83z5PUKbS6bqNbHcq9wWNyiK8jHQ+99vl5pGuOL6ctBCLiirag7vRzRW7HPqQbVt4a\nEV/tsZ1fq+wWsoj4a+C/kbqcP88/nybdAfHDvNibgQ9Iepg0TnVh2+NX5/lnkc72HiHd/dJyBHCT\npLWkW1GPy8MxO5IOujWkLvz3aNt5bU4nXWRcTRrC+fLITzq1OyLi25GuI3Ql6XdJ++f4SJ8b+Ajp\nTfiuvMjZpK7gKkmt+95PIRXnVaQx8o3uhy+05VrSHTVnkN7kt5EOqDLP46YeY5WvJF3Au5d0HeQv\nI+Jbed7ppDPGO4BvsuGB+yTpjGjfPP9B0uu7TYn2fIt0V8mXSGeWz6TctYSWvwLem/fn20ss/3HS\n2dM38/F5FekCe6e2XQOcSLpwt5p0zO3eYdFex9zmpIvqD/KbmwXened1O9ZH9QLg6rzeS4BTIl2n\n6aY15v2diHiw7EbycfRW4ALSa7eWdMPCumEbntd7N+naxKmkO8au5zfXDrrWlhxE25Fe004+mpf/\nJqmOnE26SF3UdRt0r0MLSO/7e3Ob/wB4U4/n+ANSSPQbfuvmVNIIytltvYSet1KrcOJnZlYpSVuR\nTmT2iog7prD9Q4C35CG22pP0HeD8iJjIJ6UdAmZWOUkvI90FJ+D/kHpV+xeHG21Deaj0MtJ1h4f7\nLV+Fuf7uIDObmmNIQyD3AnuRhrQcAD1IOhf4FvC2SQUAuCdgZtZo7gmYmTXY1L7sabMFi2LRJj3v\nXDQzs4I1TzzwYETs0H/JcqYWAos22ZqDtv/TaW3ezGwmfeO+M4e5dbQrDweZmTWYQ8DMrMEcAmZm\nDeYQMDNrMIeAmVmDOQTMzBrMIWBm1mAOATOzBnMImJk1mEPAzKzBHAJmZg3mEDAzazCHgJlZgzkE\nzMwazCFgZtZgDgEzswZzCJiZNZhDwMyswRwCZmYN5hAwM2swh4CZWYM5BMzMGswhYGbWYA4BM7MG\ncwiYmTWYQ8DMrMEcAmZmDeYQMDNrMIeAmVmDOQTMzBrMIWBm1mAOATOzBnMImJk1mEPAzKzBHAJm\nZg3mEDAzazCHgJlZgzkEzMwarHQISNpE0o8kfa3DPEn6hKTbJN0gaf9qm2lmZuMwSE/gFOCWLvOO\nBPbKPycBnxqxXWZmNgGlQkDSYuClwFldFjkGOC+Sq4BtJe1UURvNzGxMyvYEPga8A1jfZf4uwPK2\n/6/I0zYg6SRJ10q69vH1jw3UUDMzq17fEJB0NHB/RFw36sYiYmlEPD8inr/ZgkWjrs7MzEZUpidw\nMPBySXcCFwCHSfpcYZl7gF3b/r84TzMzsxrrGwIR8e6IWBwRewDHAd+JiFcXFrsEOD7fJXQgsDoi\nVlbfXDMzq9Kmwz5Q0skAEbEEuBQ4CrgNeBQ4sZLWmZnZWA0UAhFxOXB5/n1J2/QA3lJlw8zMbPz8\niWEzswZzCJiZNZhDwMyswRwCZmYN5hAwM2swh4CZWYM5BMzMGswhYGbWYA4BM7MGcwiYmTWYQ8DM\nrMEcAmZmDeYQMDNrMIeAmVmDOQTMzBrMIWBm1mAOATOzBnMImJk1mEPAzKzBHAJmZg3mEDAzazCH\ngJlZgzkEzMwazCFgZtZgDgEzswbbdNoNMDOr0uN7L+67zGa3rphAS2aDQ8DMZk6ZQj/M45sYDg4B\nM6u9UYv+MNtpSiA4BMysdiZV9Mu2YZ4DwSFgZlNXh6LfS6t98xgGDgEzm4q6F/5O5jEMHAJmNjGz\nWPg7macwcAiY2djMS9Hv5vG9F898EDgEzKwy8170O5n1IHAImNnAmljse5nlIOgbApK2AL4PbJ6X\nvygi/rKwzKHA3wN35ElfjogPVNtUM5s0F/vyZvU6QZmewDrgsIhYK2khcKWkr0fEVYXlroiIo6tv\nopmNgwv8eMxar6BvCEREAGvzfxfmnxhno8ysGi700zFLQVDqmoCkTYDrgD2Bv42IqzssdpCkG4B7\ngLdHxE0d1nMScBLAFgu2GrrRZrYhF3sbVqkQiIgngX0lbQtcLGmfiPhJ2yLLgN3ykNFRwFeAvTqs\nZymwFGCbhU93b8JsCC74s2FWegMD3R0UEaskfRc4AvhJ2/Q1bb9fKulMSdtHxIPVNdWseVzwZ9ss\nBEHfPyojaYfcA0DSIuBw4NbCMjtKUv79gLzeh6pvrtn8e3zvxb/+sdlX99exTE9gJ+DcfF1gAXBh\nRHxN0skAEbEEOBZ4k6QngMeA4/IFZTPro+5FwuZbmbuDbgD26zB9SdvvZwBnVNs0s/nlwt8sdR4W\n8ieGzSbEhX90q5+5eSXr2eb2dZWsZx44BMzGzMV/OFUV/H7rnlQg1LU34BAwGwMX/sGNs+iX2W5T\newcOAbMK1aH4z8KQybQKfi+TCIM69gYcAmYVmGTxn1QBrWOhnoTVz9y8Ub0Ch4DZCMZd/JtaiKdt\nnEFQt96AQ8BsQOMs/C769dGUHoFDwGwAVQeAi369NSEIHAJmJVRZ/F34Z8s4gqBOQ0IOAbMeqir+\nLvyzbZ57BA4Bsw5c/K0pHAJmbaoo/lUX/od3V6Xra9n6Ln/H4yDmtTfgEDDLRg2AKor/uAp+mW05\nFCarLtcFHALWeNMu/pMs/L20t8OB0Nk89gYcAtZoowTAsMW/LkW/FwdCczgErJFc/MtrtdthkMxb\nb8AhYI0zbAAMU/xntfB38vDuchDMIYeANcYkz/6rKP7rdnt8pMdvfvdmI7ehyL2C+eMQsEaYxNn/\nKIV/1IJfZp1VhoJ7BfPDIWBzb5gAGHfxH0fRL7vNqsLAQTAfHAI2t+p29j+Nwt9JlWHQ1CCYp4vD\nDgGbS3U6+69L8S9at9vjDgJzCNj8GWcAlC3+dS38RVUFgc0uh4DNlUEDoC7Ff4/FDwz1uJY7V+ww\n9GOrCAL3BmaXQ8DmwrTP/gct/qMW/V7rGyYQ3CNoLoeAzbxxBUDVxb/qwt9vO4OGgYOgmRZMuwFm\no5iFANhj8QMTC4Dids36cU/AZta0xv8HKf7TtsfiB0a6XmCdzcvtoeAQsBk1jgCo4uy/DoW/aJAg\n8JBQ8zgEbOZMIwDKnP0PEwCH73jrQMtfdt/eA2/DrBeHgM2UOgbAIMV/0KLf7fGDhoGHhawbh4DN\njEkHQJ2Kf7f11aVn4M8IzC6HgM2EqgNgUmf/VRf/TuuvSxA0RVUXhevw94XBIWAzoE4BUJfib1aV\nvp8TkLSFpGsk/VjSTZJO77CMJH1C0m2SbpC0/3iaa00zawFw+I63TjwAqtzeMHcGeShotpXpCawD\nDouItZIWAldK+npEXNW2zJHAXvnnhcCn8r9mQ5ulAGjqmX/TAmCePh/Q0rcnEMna/N+F+af4yh8D\nnJeXvQrYVtJO1TbVmsQBYPOsLtcDoOQ1AUmbANcBewJ/GxFXFxbZBVje9v8VedrKwnpOAk4C2GLB\nVkM22WxDsxgAxz512UbTLlozW6Oo7gXMh1IhEBFPAvtK2ha4WNI+EfGTQTcWEUuBpQDbLHx6s44g\nK22QXsCgfwC+aNgAGKb4dyr8neZPKwwGuR7QtACYZwN9gVxErAK+CxxRmHUPsGvb/xfnaWYDGfZP\nQnYz7OcAqgyAY5+6rG8AFJevWpUfFGtiAFTZC6jTUBCUuztoh9wDQNIi4HCg+C64BDg+3yV0ILA6\nIlZiNoBJXgeYRAAMWvynyd8X1N28DgO1lBkO2gk4N18XWABcGBFfk3QyQEQsAS4FjgJuAx4FThxT\ne82A2QiAedS0XkDVAVC3XgCUCIGIuAHYr8P0JW2/B/CWaptmTVLldYCmBUC/Twz3Gwoq2wtoWgA0\nhf+ojE1d1dcBhlHVV0DPWg/AAdDdvA8DtTgEbKaMqxfQS9lewDQCYNReQBkOgGrUcSgIHAI2ZfMy\nDFRlAFR1i2gVw0AOgGrUNQDAXyBnU1SHYaBZNsq3hzoANtaU4Z8i9wRsJrgXsKFRhoEcABsbZwDU\nuRcA7gnYlLgXsLF5DIAyxXXUT32Poqln/+0cAlZ70+gFTNokAqCMUQNgmKLa6THjDoZJFf+69wLA\nIWBTMAu9gEl9M+ggF4HH/XmAUQKg6qJaXF8VoTDps/5ZCABwCFjNzUIv4KI1+w98XWDQO4DqGgCT\nKqzdttPt+PAwT3kOAZuoWegFDKNsEAxz+2cdA6AuRbYu7SialV4AOASsxqZ5wXAY4/gKaAfA7Jml\nAACHgE3QJL8meh7U7S4gF//+Zi0AwCFgc2rYr4iogzIfAptkALj4lzOLAQAOAaupaQ8FXXbf3hP/\n28GjFn+ofvjHATD/HAI2EfN6QbgqdQsAF//BzGovAPy1ETajJnE9YJTv5hlkGw6A2TbLAQDuCZj1\nNI5hoUHCZdLfBOoAKG/Wi3+LQ8DGruq/HVyFO1fsUPoDY62iPUoYDNqrKPP1Dz77n555CQBwCJiV\n1l7I+wXCKENJPvuvt3kKAHAI2Jza/O7N+t4mOkhvoGgc1wuqOPuH8gHg4j+4eQsAcAiYTd2kiz84\nAAY1j8W/xSFgY1X3W0NbBXjSXys9yFc+++x/uuY5AMAhYDNq67ui722iZYaEWkYZGhrEtIo/jD8A\nisWy7icA/cx78W9xCJhl4+gVDPNHXsoUf5ju2X+ZAtltmbqHQ1OKf4tDwObaIL2BlvbCPWggjPKX\nvaou/lBtAFRVHOvaY2ha8W9xCNjMKjMkBMMFQcuof66xjDoX/0kUxvZtTDoQmlr42zkErBFGCYJx\nKFv4WyYdANMqjp22W2UwuOhvzCFgtbPN7etKf2q4bG8A6hEELv6DK9OmVlDUsf115xCwRmkV4UmG\nwaCFH1z8BzXr7Z8mh4DNvEF6Ay3thbnqQBim6LdM+vv+XTzNIWC1NMiQEAwXBC2dina/YBil0BdN\n4+/8uvhbi0PA5sYoQVBUZZHvZlp/5N0BYO0cAjZWm926Yui7OwbtDUC1QTAOwxR+cPG38XEI2Nxp\nFdq6hMGwhR9c/G38HAJWa8P0BlqmGQajFH5w8bfJ6fs3hiXtKum7km6WdJOkUzosc6ik1ZKuzz/v\nG09zbRaNWoxGLYhb3xW//hmX9m2MeubvALBJKtMTeAI4NSKWSdoauE7SZRFxc2G5KyLi6OqbaDZa\nj6BdtwJdprcwzhCZpa95sPnSNwQiYiWwMv/+sKRbgF2AYgiYjVVVQdDJOAt8N3X8cjdrnr7DQe0k\n7QHsB1zdYfZBkm6Q9HVJz+ny+JMkXSvp2sfXPzZwY212VVWk5uGPolQ15ANpvzoAbBSlLwxL2gr4\nEvC2iFhTmL0M2C0i1ko6CvgKsFdxHRGxFFgKsM3Cp0/+1MvmQquAjqtXMA5Vh5cLv1WlVAhIWkgK\ngM9HxJeL89tDISIulXSmpO0j4sHqmmqzbpTPDHRS9zCY1h9zMRtE3xCQJOBs4JaI+GiXZXYEfh4R\nIekA0jDTQ5W21OZC1UEA9QmDcQ5VufjbuJTpCRwMvAa4UdL1edppwG4AEbEEOBZ4k6QngMeA4yLC\nwz3W0TiCADYuwuMMhUldm3Dxt3Erc3fQlUDP++ci4gzgjKoaZVaFXoW6X0BM+wK0i79Nij8xbFMx\nrt5AWdMu8p248Ns0OARsalpFry5/aHxaXPxtmhwCNnXT7hVMgwu/1YVDwGqhCUHgwm915BCw2pjH\n4SEXfqs7h4DVzqyHgQu/zRKHgNXWrISBi77NMoeA1V57ka1DILjo2zxxCNhM6VSAxxUMLvbWBA4B\nm3ku1mbDG+jvCZiZ2XxxCJiZNZhDwMyswRwCZmYN5hAwM2swh4CZWYM5BMzMGswhYGbWYA4BM7MG\ncwiYmTWYQ8DMrMEcAmZmDeYQMDNrMIeAmVmDOQTMzBrMIWBm1mAOATOzBnMImJk1mEPAzKzBHAJm\nZg3mEDAzazCHgJlZgzkEzMwazCFgZtZgDgEzswbrGwKSdpX0XUk3S7pJ0ikdlpGkT0i6TdINkvYf\nT3PNzKxKm5ZY5gng1IhYJmlr4DpJl0XEzW3LHAnslX9eCHwq/2tmZjXWtycQESsjYln+/WHgFmCX\nwmLHAOdFchWwraSdKm+tmZlVqkxP4Nck7QHsB1xdmLULsLzt/yvytJWFx58EnJT/u+4b9535k0G2\nPyXbAw9OuxEluJ3Vcjur5XZWZ/cqV1Y6BCRtBXwJeFtErBlmYxGxFFia13dtRDx/mPVMkttZLbez\nWm5ntWalnVUqdXeQpIWkAPh8RHy5wyL3ALu2/X9xnmZmZjVW5u4gAWcDt0TER7ssdglwfL5L6EBg\ndUSs7LKsmZnVRJnhoIOB1wA3Sro+TzsN2A0gIpYAlwJHAbcBjwInlljv0oFbOx1uZ7Xczmq5ndWa\nlXZWRhEx7TaYmdmU+BPDZmYN5hAwM2uwiYSApE0k/UjS1zrMq81XTvRp56GSVku6Pv+8b0ptvFPS\njbkN13aYX4v9WaKdddmf20q6SNKtkm6R9HuF+XXZn/3aOfX9KelZbdu/XtIaSW8rLDP1/VmynVPf\nn5My0IfFRnAK6ZPGT+0wr05fOdGrnQBXRMTRE2xPN38YEd0+0FKn/dmrnVCP/flx4BsRcaykzYCn\nFObXZX/2aydMeX9GxE+BfSGdUJFuE7+4sNjU92fJdkI9js+xG3tPQNJi4KXAWV0WqcVXTpRo56yo\nxf6cBZK2AV5EugWaiHg8IlYVFpv6/izZzrp5MXB7RNxVmD71/VnQrZ2NMYnhoI8B7wDWd5nf7Ssn\nJq1fOwEOyl3Yr0t6zoTaVRTAtyRdl7+Go6gu+7NfO2H6+/O3gQeA/5eHAc+StGVhmTrszzLthOnv\nz3bHAV/oML0O+7Ndt3ZCvfbn2Iw1BCQdDdwfEdeNczujKtnOZcBuEfE84JPAVybSuI0dEhH7krrV\nb5H0oim1o59+7azD/twU2B/4VETsBzwCvGsK7einTDvrsD8ByMNVLwe+OK02lNGnnbXZn+M27p7A\nwcDLJd0JXAAcJulzhWXq8JUTfdsZEWsiYm3+/VJgoaTtJ9xOIuKe/O/9pHHMAwqL1GF/9m1nTfbn\nCmBFRLS+EPEiUrFtV4f92bedNdmfLUcCyyLi5x3m1WF/tnRtZ83251iNNQQi4t0RsTgi9iB1u74T\nEa8uLDb1r5wo005JO0pS/v0A0r57aJLtlLSl0t90IA8H/BFQ/CbWqe/PMu2sw/6MiPuA5ZKelSe9\nGLi5sNjU92eZdtZhf7Z5Jd2HWKa+P9t0bWfN9udYTeruoA1IOhlG+sqJiSi081jgTZKeAB4DjovJ\nf9z6GcDF+djcFDg/Ir5Rw/1Zpp112J8AbwU+n4cGfgacWMP9WaadtdifOfQPB/68bVrt9meJdtZi\nf06Cvza2OtUbAAAAMUlEQVTCzKzB/IlhM7MGcwiYmTWYQ8DMrMEcAmZmDeYQMDNrMIeAmVmDOQTM\nzBrs/wMVO995UVnQoQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Now let's calculate the GMM model of class 2\n", "gaussian_2_0 = stats.multivariate_normal(mu_2_0,Sig_2_0).pdf(coords)\n", "gaussian_2_1 = stats.multivariate_normal(mu_2_1,Sig_2_1).pdf(coords)\n", "gmm_pdf2 = c_2_0*gaussian_2_0 + c_2_1*gaussian_2_1\n", "# And let's plot its contour plot\n", "plt.contourf(x0coords,x1coords,gmm_pdf2)\n", "plt.title('Gaussian Mixture Model of the class Iris-virginica, class y=2')" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAF2VJREFUeJzt3V/MZVV5x/HfwzAzTGUcLkAhvPxpCHESTQNkghYaYzE0\ngkRvMKGJmnIzxZBGExtTvbBp73pj1BKYTDSNRFtj8U8NAZppqqlcQAPjFBFoAu0gQxgRE2egkpki\nTy/ec+zhcM571t57/dvrfD/JG973nH32Xmc5/s5z1l57bXN3AQDackbpBgAA4iPcAaBBhDsANIhw\nB4AGEe4A0CDCHQAaRLgDQIMIdwBoEOEOAA06s9SBd5yxy3dt213q8AAwSidf+8VL7n7equ2Khfuu\nbbt1zbkfKXV4ABilB47f+WzIdgzLAECDCHcAaBDhDgANItwBoEGEOwA0iHAHgAYR7gDQIMIdABpE\nuANAgwh3AGgQ4Q4ADSLcAaBBhDsANIhwB4AGEe4A0CDCHQAaRLgDQIMIdwBoEOEOAA0i3AGgQYQ7\nADSIcAeABhHuANAgwh0AGkS4A0CDCHcAaBDhDgANItwBoEGEOwA0iHAHgAYR7gDQIMIdABpEuANA\ngwh3AGgQ4Q4ADSLcAaBBhDsANCg43M1sm5n92MzuXfCcmdmXzexpM3vMzK6K20wAQBddKvdPSnpy\nyXM3SLp88rNf0l0D2wUAGCAo3M1sQ9IHJX1lySYflnS3b3pI0jlmdkGkNgIAOgqt3L8o6TOSXl/y\n/IWSnpv5+9jkMQBAASvD3cxukvSiuz869GBmtt/MHjGzR06//urQ3QEAlgip3K+V9CEzOyrpm5Ku\nM7Ovz23zvKSLZv7emDz2Bu5+0N33ufu+HWfs6tlkAMAqK8Pd3T/r7hvufqmkWyT9q7t/dG6z70v6\n+GTWzHsknXD3F+I3FwAQ4sy+LzSz2yTJ3Q9Iuk/SjZKelvRrSbdGaR0AoJdO4e7uP5T0w8nvB2Ye\nd0m3x2wYAKA/rlAFgAYR7gDQIMIdABpEuANAgwh3AGgQ4Q4ADSLcAaBBhDsANIhwB4AGEe4A0CDC\nHQAa1HvhMACo0em9G9H2teOpY9H2lRvhDmC0YgZ5yP7HFPaEO4BRSR3ooceuPegJdwDVKxnoy9Qe\n9IQ7gGrVGOqLTNtZU8gT7gCqM5ZQn1dTyDMVEkA1Tu/dGG2wz6rhPVC5AyiuhjCMrXQVT+UOoJhW\nKvWtlHp/hDuAIloP9VklPsQIdwBZrUO1vkzO9024A8hmXUN9Vq4+4IQqgCxyBvuJy3b2et2eZ05F\nbslip/duJD/RSrgDSCp1qPcN8tB9pQr81AFPuANIJlWwxwz0LseKHfQpA54xdwBJpAj2E5ftzBrs\nOY6f6gOQyh1AdDEDq2SYLzNtU6xKPkUFT+UOIKpYwV66Sg8Rs42xK3jCHUA0MYN9TGoMeMIdQBQx\ngmkM1foytbWbcAcwWKxgH7sYH06xqndOqAIYZGgYxQ71ly+x3q/d/axHacOJy3YOOtka4wQr4Q6g\ntxqCfUiYb7WvoUFfOuAJdwBFDAn2mIEecoy+QT804IdYOeZuZmeZ2b+b2X+Y2U/N7K8WbPM+Mzth\nZkcmP59P01wAtRhStfcN9pcvsSzBHvO4Qz7EhvRxSOV+StJ17v6KmW2X9KCZ3e/uD81t9yN3v6l3\nSwCMRu5gLxHoi7x8ifWq4ktU8Csrd9/0yuTP7ZOfOGcdAIxOzmAvValvpW+b+lbwffs7aCqkmW0z\nsyOSXpR0yN0fXrDZNWb2mJndb2bv7NUaAM3qE+w1q719QSdU3f03kq4ws3MkfdfM3uXuj89scljS\nxZOhmxslfU/S5fP7MbP9kvZL0llnnD248QDSLTy1aKZG32O1FuxTXYdp+g7P9Jk502m2jLv/ysx+\nIOkDkh6fefzkzO/3mdmdZnauu7809/qDkg5K0p7tb2NoB+gg912MSiwlMDTUT118uvdrd/5sR6/X\n5Qr4rlaGu5mdJ+l/J8G+S9L1kv5mbpvzJf3c3d3MrtbmcM8vUzQYWAet3I4uR7APCfRl++ka9H1P\ntHbRtXoPqdwvkPQ1M9umzdD+lrvfa2a3SZK7H5B0s6RPmNlrkl6VdIu7U5kDHbQS6FMpgz1WoK/a\nf5eQ7xLwOar3leHu7o9JunLB4wdmfr9D0h1xmwa0rbUwn5Uq2FOH+qLjpQr41LhCFcio5UDvo+Zg\nnz9uaMiHBnyf6v303g3peNi2hDuQwTqFemjVHhrspUJ9XtcqvjTCHUhknQJ9qtVgnwoN+JTVeyjC\nHYhsHUO9i1zBfunGL5Y+d/TYeb33GzvgUyHcgQhKBXrXi4NSztCIuS57n2DfKsxXbds17GMO0aSq\n3gl3YIAcoR4zNJftK9eiViFVe9dg7xLqq/bRJeRDAr5k9U64Az2kCvVSt5qbPW6foA9pd+xgjxHq\ny/Y5ZNimFoQ70EGKUK/t3qHT9uReojY02FOE+qJjhAR8rOo9xdAM4Q4EiBnqtYX5MqEhH6NqrynY\nZ48VK+BLINyBLaxjqM8bWsnHWuExZ7DPHnOsQzRB67kD6+b03o0owX7isp2//Rm7Re8hxvsKqdpL\nBHuXY696DyWWMKZyB+YMDfUUQT4kHGLO1ug6NhxjOKZksM+2IXUFH3vcnXAHJmoI9RQV3vw+h4b9\nNIRyfBsZEuzXn//Umx47dHzvkOZsqbaxd8Ida69kqJf4uj57zL5Bn+Mkap9gXxToy57vGvRDq/fc\nc94Jd6y1nDd7nqrpFnLTttSyTG1fq0J9q9ekrOZLItyxlnKHek2BvkjukI9ZtfcJ9vnXhwb8mGbP\nEO5YO7lu8izVH+rzYg0dDHnfocE+NNTn9xWjgq9p3J1wx9rIVa2XvMnz1JCAST02HOP9xQz22X2W\nHqKJOWOGcMdayFGtl77B87J99gn6Ugte1TDtcZWxDM0Q7mhen2Af882dtzpejvuBph6KSlG1z+67\ndPUeC+GOZqWu1sdw/895pceEh/ZDymDPIec3IpYfQJP6Vuuh87e73E2olmCf6tKenCeEaxmSGfsH\nyBSVO5qTchgmxy3iuoRc37Hf0hV8H62Ebi6EO5rSNdhjhnrqW8OFvD72nYSkbkMJY5v62TLCHU1I\nVa2P4e5Bi/YfGvI5K/it+qmWIZmWEO4YvTEEe+7w6hLyYxyiwWqEO0at1DAMS9Xm1cp4OwuHAQFS\nBHsroT4rRsCXuqhp3bCeO9ZejcHeNdSHVKOxL7RhaKY9hDtGJ3ew17SC4fx+WlzNEHEQ7hiV2MGe\no1pPfbm81O6a5CW00peEO0ajpmAvHeqLjtVKKCEOlh/AKOQM9lVLBqwK9uvPf6rK2R1DTvSWPpla\n0wfXVsNbNZ23oHJH9XIH+zIhoV4S1ftwKfsv9wck4Y6qtRzsN7/18JbP33Pyqs77xHjFnAYpBYS7\nmZ0l6d8k7Zxsf4+7/+XcNibpS5JulPRrSX/i7lv/ywVWGEOwdwn1VWG+bHtCftOh43uTfjsKqdrH\nNOMopHI/Jek6d3/FzLZLetDM7nf3h2a2uUHS5ZOfd0u6a/JfoJcagj1Wtd411Je9PiTkxzo0c/TY\nedVd/NVVTePtUkC4u7tLemXy5/bJz/zg0Ycl3T3Z9iEzO8fMLnD3F6K2Fmuh9mDPFeo5bRVMpU+m\nzkpVvaeu2kv0YdBsGTPbZmZHJL0o6ZC7Pzy3yYWSnpv5+9jksfn97DezR8zskdOvv9q3zcBv1Rjs\nN7/1cJJgH9OHxSK1VbZTNXzTiT3eLgWGu7v/xt2vkLQh6Woze1efg7n7QXff5+77dpyxq88u0Lgu\nVXutwV7SVkFV+3hxaPtihnHovla1rcYPrk6zZdz9V2b2A0kfkPT4zFPPS7po5u+NyWNAsDEHe+lQ\nT6XrcMLuZz3LDTumodx3iCZntV5qWGtl5W5m55nZOZPfd0m6XtJ8j35f0sdt03sknWC8HV2UDvat\ntB7sNVWdXb9ddA3pQ8f3dn5N6qo9xZCMFFa5XyDpa2a2TZsfBt9y93vN7DZJcvcDku7T5jTIp7U5\nFfLWJK0FBugz3bGVYO8zJJOi4tz5sx3Rbxi+KKxjzRoaGuyx+3DHU8eCtw2ZLfOYpCsXPH5g5neX\ndHvwUYEZuar2ZVoJ9j5hVlPVPhVjWmSOYI8hVdUusbYMCis9HNM3REoEe9+LmXJX7ateG/KBUvrk\nb8jxc1ftXRHuaF7Kuey1aKVqr0HpD5ZlugzJSIQ7Cio9HLNMjcMxY6naQ9VavYceM0bVnnJIRiLc\nUQjDMfHkrNr3PHMqKJRifUAcPXZetpCPFey1INzRrBTTHkuIXbXXstRAl5BMGfBdPkBC2pyiau86\nJCMR7iig9HDMmBaoWhXsy6r2FMMxXQMpJOS6BnzMkO+6v1jBngvruaNJLVyslCLYlwVU12Df88yp\noBuPpzB9f30/pPt8QMQcislRtUuEOzKjag/TN9i3UmKsOGQ5gr4XNs2H9KL/bWNU+qH9VsNJ1FmE\nO1CZIcGeYpw9RyDFuHI1xbh8zGDvo2/VLjHmjoxKz5AZgzEFe2johwZfbbNQYgd7zqpdItyBYClv\nd3fPyatGFexddQn40iHfpQ0pg31I1S4xLINMut5dKYUax9tDPzBqDfZUJ1ZTLDAWcswuapoZswjh\njurkvBq1q3tOXhVl1kyMUJfqqNhDA77rWu/T95Aj5FMGe4mqXSLckUHOqj1HEAwJ+C5DO33vqlTL\nRUrLjt/1wzlVyPcd/hlDsEuEO9DLNKRXhXyfcfq+1bo0PNj7jrN3GZ7pe7em2ffWN+iHjuenDvaY\nCHdUpdSFMX3FPsk6xmDvY+jt+ErN2Q/Vty9jVe0S4Y7EajiRGurQ8b3F1pYZEupSHcHe9eTqtF0l\nz6GE6DqUVbpin2IqJNZGyEUuOW+cPD3e0Gq9hmAfsq/S5wG2krNtMat2icodeJMcFXzIh0jq+3em\nqjD7TI8cOkwTW59QH9KfsYNdItyBhVIEfJdvBWMN9tn99wl4qewwTd9KvbZglwh3NGbVxS9dbr48\nDeO+Id9niCd1qEv5xoT7XuBUIuSHDL/UMsY+j3BHMrWeTO0S8FKecfhcN2Tus9zskP8dh1zBOvt+\nUgR9jPH0ocGeqmqXCHesqa4Bn7Idq5S++0/JgJ+af399wj72ydGag10i3NGg0HVJSgV87bd0k+IH\nT+w1aErOsIkxDJM62CWmQmLN5br58vRYYw32GGFU69h0F2MJdonKHSMU++4+Q2/btmq/XcS8T2fs\n27kNHZ6Ryt6eb4hYH0y5gl0i3NGwrsvGhty2LfS1XbSwdngX0/aNJeTHGOwS4Y7GDVkXPPWQTZf1\nUWoJ9hjV+1TtIR9zGCl3sEuEO0aqyxWNJW78sJVaQl3qFzoxA16qK+RTnBcoEewS4Y7KpFo6NueN\nH7Y6fqja1wyPHfBS2ZBPdbK3VLBLhDvWTK6Qr/VGEFK8wEkR8NIb31fKoE89e6dksEuEO0au5I0f\nlu2vj1w3gYgdOKkCfmrRe+0T+DmnYZYO9SnCHdXpsy742G78IOVdeTBl4KQO+Hk1z5evJdglLmJC\nQjn/oe9+1qteF3xWn7bueeZUlcGe8xi1q60PVlbuZnaRpLslvV2SSzro7l+a2+Z9kv5J0n9PHvqO\nu/913KZinQxZUbCmdcGnWlpKdtXxal0wLpXaQn0qZFjmNUmfdvfDZrZb0qNmdsjdn5jb7kfuflP8\nJmJdjWnJ2GVt6Kv2RalWHXtdAr7WYJcCwt3dX5D0wuT3l83sSUkXSpoPd+BNWl4ydtmxhhhzqM9q\nPeBr6eetdDqhamaXSrpS0sMLnr7GzB6T9LykP3f3nw5uHaA0S8ZK/QO/tqVjpTrDpsVhmhr7eZng\ncDezsyV9W9Kn3P3k3NOHJV3s7q+Y2Y2Svifp8gX72C9pvySddcbZvRuN9ZNiwanSJ2DHumZJVy2E\nfO19vIi5r/4HbmbbJd0r6Z/d/QsB2x+VtM/dX1q2zZ7tb/Nrzv1Ih6ZizGL9H7uGS9SHGPt6JTGM\nKeRr7OMHjt/5qLvvW7VdyGwZk/RVSU8uC3YzO1/Sz93dzexqbU6x/GXHNqNhscZga1qHpAtC/f+N\noZIfex9LYcMy10r6mKSfmNmRyWOfk3SxJLn7AUk3S/qEmb0m6VVJt3jIVwKgp9pDvqUFqFKZfT81\nBH1r/RsyW+ZBSVueeXL3OyTdEatRaFNri03Na3HxqVxKBH3r/cryA8gqx2JTUvqwz3EJfOvhs8yi\n9x3j38y69SfhjuxyzIFeFr6hoV9y/ZJ1C6EQ9El3hDuKKHWRS62LThFeiI2Fw1AMgbbZB/QDUqBy\nR1GtX6a+CGGOHAh3FDeGec8xEOrIiXBHNVqs4gl0lEK4oyq1XdjSFWGOWhDuqNYYhmsIc9SKcEf1\naqrmCXOMBeGOUUl19WLIcYAxIdwxegQx8GZcxAQADSLcAaBBhDsANIhwB4AGEe4A0CDCHQAaRLgD\nQIMIdwBoEOEOAA0i3AGgQYQ7ADSIcAeABhHuANAgwh0AGkS4A0CDCHcAaBDhDgANItwBoEGEOwA0\niHAHgAYR7gDQIMIdABpEuANAg1aGu5ldZGY/MLMnzOynZvbJBduYmX3ZzJ42s8fM7Ko0zQUAhDgz\nYJvXJH3a3Q+b2W5Jj5rZIXd/YmabGyRdPvl5t6S7Jv8FABSwsnJ39xfc/fDk95clPSnpwrnNPizp\nbt/0kKRzzOyC6K0FAATpNOZuZpdKulLSw3NPXSjpuZm/j+nNHwAAgExChmUkSWZ2tqRvS/qUu5/s\nczAz2y9p/+TPUw8cv/PxPvvJ7FxJL5VuRADaGRftjIt2xnNJyEZB4W5m27UZ7N9w9+8s2OR5SRfN\n/L0xeewN3P2gpIOTfT7i7vtCjl8S7YyLdsZFO+MaSztDhMyWMUlflfSku39hyWbfl/TxyayZ90g6\n4e4vRGwnAKCDkMr9Wkkfk/QTMzsyeexzki6WJHc/IOk+STdKelrSryXdGr+pAIBQK8Pd3R+UZCu2\ncUm3dzz2wY7bl0I746KdcdHOuMbSzpVsM5cBAC1h+QEAaFCWcDezbWb2YzO7d8Fz1SxdsKKd7zOz\nE2Z2ZPLz+UJtPGpmP5m04ZEFz1fRnwHtrKU/zzGze8zsKTN70sx+f+75WvpzVTuL96eZvWPm+EfM\n7KSZfWpum+L9GdjO4v05VPA894E+qc0rW9+64Lmali7Yqp2S9CN3vylje5b5Q3dfNhe3pv7cqp1S\nHf35JUkPuPvNZrZD0u/MPV9Lf65qp1S4P939PyVdIW0WStqcDv3duc2K92dgO6U6/n32lrxyN7MN\nSR+U9JUlm1SxdEFAO8eiiv4cAzPbI+m92pzqK3c/7e6/mtuseH8GtrM275f0jLs/O/d48f6cs6yd\no5djWOaLkj4j6fUlz9eydMGqdkrSNZOvkveb2TsztWueS/oXM3t0csXvvFr6c1U7pfL9+buSfiHp\n7ybDcV8xs7fMbVNDf4a0Uyrfn7NukfQPCx6voT9nLWunVFd/dpY03M3sJkkvuvujKY8zVGA7D0u6\n2N1/T9LfSvpelsa92R+4+xXa/Hp7u5m9t1A7VlnVzhr680xJV0m6y92vlPQ/kv6iQDtWCWlnDf0p\nSZoMG31I0j+WakOIFe2spj/7Sl25XyvpQ2Z2VNI3JV1nZl+f2yZo6YLEVrbT3U+6+yuT3++TtN3M\nzs3cTrn785P/vqjNccKr5zapoT9XtrOS/jwm6Zi7TxfCu0ebITqrhv5c2c5K+nPqBkmH3f3nC56r\noT+nlrazsv7sJWm4u/tn3X3D3S/V5teff3X3j85tVnzpgpB2mtn5ZmaT36/WZt/9Mmc7zewttrmm\nviZfy/9I0vzia8X7M6SdNfSnux+X9JyZvWPy0PslPTG3WfH+DGlnDf0544+1fKijeH/OWNrOyvqz\nl1yzZd7AzG6T6l+6YK6dN0v6hJm9JulVSbd4/ivA3i7pu5N/c2dK+nt3f6DC/gxpZw39KUl/Jukb\nk6/o/yXp1gr7M6SdVfTn5MP8ekl/OvNYdf0Z0M4q+nMIrlAFgAZxhSoANIhwB4AGEe4A0CDCHQAa\nRLgDQIMIdwBoEOEOAA0i3AGgQf8HmEXaq75TL68AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "maxgmm = np.maximum(gaussian_2_0,gaussian_2_1)\n", "plt.contourf(x0coords,x1coords,maxgmm)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# GMM Classifier\n", "A GMM classifier is a Bayesian classifier in which one or more of the classes has a GMM likelihood function.\n", "\n", "So the classification function is\n", "$$\\hat{y}=\\arg\\max_y p_Y(y)p_{X|Y}(\\vec{x},y)$$\n", "\n", "where $p_{X|Y}(\\vec{x}|y)$ is a GMM for at least one of the classes." ] }, { "cell_type": "code", "execution_count": 129, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 129, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAEICAYAAAA+16EyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu4XGV99vHvDSQhclRACdkJqQkYBRWRIo1gaRBRgmAt\nKp4QT4jYvljPUqvi64VUWgW1CqgtRkRUCr68iCJyUDAChggoBBUjJIQoAU1CMLDB/PrH82ycTGb2\nzN57ZtZaM/fnunJlz8yaNb91vGc961lrFBGYmZlVzRZFF2BmZjYeDjAzM6skB5iZmVWSA8zMzCrJ\nAWZmZpXkADMzs0qacIBJ+qik8zpRTJPx3ybp4Py3JP23pD9KulHSQZJ+2YXPnClpvaQtuzDu50v6\ndR7/yzo9/lE+tyvzajwkXSPpLUXXUVaSzpX08aLrqKq8bT21BHV0dd84EZJOlvSlouuYqLYCTNJr\nJC3OK8YqSd+VdGC3iwOIiL0i4pr88EDgUGAoIvaPiGsj4mkT/QxJd0l6Yc1nLo+IbSPizxMddwMf\nAz6Xx//tLowfAEkhac7I407NqzY/e3LeeH8t6aE8f/9L0qxefH6nSPpnSb+TtC7XP6UENU2TdImk\ne/MynlVwPZMlfVjSL/OyXpn3Dy8qqqa8bS3rxWcVuW+sq+PJkr6e14u1kn4s6XnNho+IUyNiXF8i\nJR0j6Ya8vO/Lf58oSfn1c/O6eVTd+z6dnz8uPz4uP/503XBH5efPbVVLywCT9C7gDOBU4CnATOA/\ngSPbm9yO2h24KyIeKuCzO2V34Laii+iyC0nrx2uAHYBnA4uBQ4osaiwkHQZ8gFTz7sBTgVMKLSrZ\nCHwP+IdOj1jSVuN424XAUcCxwBOBvwLOBBZ0sLRSKtm+cVvgp8BzgScBXwG+I2nbTn6IpHeTlu/p\nwK6k6T4BeD4wuWbQX5HWiZH3bQW8EvhN3Sh/A7yybt17Q35/axHR9B9p57MeeMUow3wUOK/m8beA\n3wFrgR8Be9W8djhwO/AgsBJ4T35+Z+BSYA3wB+BaYIv82l3AC4E3Aw8Df841nQIcDNxTM/4ZwEXA\nauAB0pEOwGzgqvzc/cDXgB3za18l7RQ25PG+D5gFBLBVHmY34JJc253AW+um/5vAwjxdtwH7NZlX\nv6n7rCkj09doftbU8QZgea79X2qG3RI4OY/3QeCmPA9+lN/3UP6cVzWYV08Hrsnz/DbgyJrXziVt\niN/J470BmD3aulLz3hfm6ZsxyjDXAG9ptWzy6+/P68qDwC+BQ/Lz+5NCcR3we+BTNe85AFiUp+0W\n4OCa144DluXx/RZ4bZMazwdOrXk8H/jdKNN0JrAi13MTcFC76wjwHGBJfu0bwAXAx1vM563yMp7V\nYrgP1KwftwN/Xzcvfgx8Os//j+fn3wQsBf4IXA7s3mJZD02gho+y6f5jFptuew2XFzAH+CFpP3M/\n8I2acQQwJ/+9APhZXi4rgI82+KyG21fZ941t1LgOeG6r2oCtgfPyOrCGFIRPaTLNDwH/0OJzzwX+\nnbRdPjE/dwTwXeA64LiaZXsd6QvZgvzck/I8Oh04t+U0tijkxcBjIytTmwvpTcB2pJ3zGcDNNa+t\nIm/YpG9r++a/PwGcBUzK/w4ClF+7i7yDH5ngmvEdTN4pk3bmt5A2xm3yQjmwZmU/NNe0S155zqgZ\nz+Of0WQj+hHw+TzOfUgBOb9m+h/OK+CWeVquH2V+1X9W/ePaFWukji8CU0lHMo8AT8+vvxf4OfA0\nQPn1neo34gbzahIpiE8mfWuaT9pwnlazAj5AComtSKFyQc24LgU+0GT6TgN+2GK9uoa/BFjTZZOn\nawWwW838mJ3//gnw+vz3tsAB+e/pufbDSS0Mh+bHu+T1Yl3NdE6jZidSV+MtwKtqHu+U5+lOTYZ/\nXR5mK+DdpI1w61brSJ7/dwP/nJfL0cCjdC7AXkH6ArYF6YvMQ8C0mu3pMeCf8vimko6m7iR9wdkK\n+BCwaJRlfU3LnczoNXyUJgE22vICvg78Sx7n49t6/bpPWu+fmYd7Fmmn+rJ2tq+y7xtb1LdPXud2\naFUb8Dbg/wNPyOvnc4HtxzPNNfuPjwPnAG/Pz30TeDWNA+w15C8gwInA2fn9LQOsVRPiTsD9EfFY\ni+EeFxH/FREPRsQjeSY9W9IO+eVHgWdI2j4i/hgRS2qen0b6pvdopPM10e5nZvuTNpL3RsRDEfFw\nRFyXa7ozIq6IiEciYjXwKeBv2xmppBmkw+P353HeDHyJmsNjUqheFumc2VdJG0InnRIRGyLiFtKO\ndWT8bwE+FBG/jOSWiHigjfEdQNrpnxYRwxFxFSmUXl0zzMURcWNe9l8jbRAARMQREXFak3HvRNoY\n29Ji2fyZtLE/Q9KkiLgrIkaaIB4F5kjaOSLWR8T1+fnXAZfl5bExIq4gHakdnl/fCOwtaWpErIqI\nZs2525K+KY9Yl//frsl0nBcRD0TEYxHxH7nu2nOOzdaRA0g7pjPyun8h6RtwR0TEtyLi3jwvvgH8\nmrStjLg3Ij6b695Aag76REQszcv+VGAfSbs3GP3OpKAGQNKTJK3J52AeHkMNo2m2vB4lNe3uVrut\nN5j+ayLi5/mzbyUFX/2232z7Gk1p942StietY6dExNrRhq35jJ1Iof/niLgpItY1GG5n6qZZ0qK8\nzDdIekHd8AuBYyXtSJrnzc75XwwcnOfFsfl9bWkVYA8AO7fbNi5pS0mnSfqNpHWkowtIEw6p3f5w\n4G5JP5T0N/n500nf+r4vaZmkD7Q7ATVmAHc3WqEkPUXSBfkE8zrS4fLOm42hsd2AP0TEgzXP3U36\npj/idzV//wnYepznE5qpH/9Iu/YMNm9TbsduwIqI2FjzXKtparct/QHSBteW0ZZNRNwJvJO0sd+X\nh9stv/XNwJ7AHZJ+KumI/PzuwCvyRrVG0hpS559pkc6dvoq0k14l6TuS5jYpbT2wfc3jkR3Ngw2G\nRdJ7JC3NO+81efjadazZOrIbsLJup3R3k5rGTNKxkm6umRd719W1ou4tuwNn1gz/B9LR/XQ2t8my\njog/RMSOpG/wj3d4aaOGhlosr/flum5U6qn8pibT/zxJV0taLWltHlf9Z49nXS/lvlHSVNLR1PUR\n8Yl2aiOF3eXABbkTyCclTWow3GbTHBHz8jJ/gLo8yV8qdiEdKV+avyBtJj//HdLR/k4R8eM2624Z\nYD8hHVK32937NaQmiBeSNuBZ+XnlQn8aEUcBTyal8Tfz8w9GxLsj4qmkE6DvkjTWE/4rgJlNVqhT\nSU0Fz4yI7Unf0lXz+mjfaO4FniSp9pv3TFI7dSc8RDp0H7HrGN67gnQOaazuBWZIql3+nZqmHwD7\nSxpqc/hRl01EnB8RB5J2rAH8W37+1xHxatK69G/AhZK2Ic2Tr0bEjjX/thk5YoyIyyPiUNKO9w5S\n81Ejt7HpN/FnA79vdIQr6SDSDvWVpDb/HUlHb6oftoFVwPSRHlzZzDbe11I+avoi8I+kHcOOwC8Y\nfd1fAbytbv5NjYhFDT7iSuCvR1vWbdQw6vrfbHlFxO8i4q0RsRupCezzqul1W+N80vnrGRGxA6k5\nrp3l0krp9o1KvWS/DdxDmidtyUd2p0TEM4B5pPNVxzYYdGSaj2rwWjPnkZrUWx1VLczDjemyg1ED\nLB9+fhj4T0kvk/QESZMkvUTSJxu8ZTvSBD5AWilPHXlBqbvtayXtEBGPkppkNubXjpA0J2/Ea0lN\nRxs3G/vobiTtDE6TtI2krSU9v6au9cBaSdNJ545q/Z7Uy6zRPFhB6hDwiTzOZ5G+/Xfq+o6bgWPy\nfN2PdA6kXV8C/q+kPZQ8S9JO+bWm00TqlPEn4H35cw8GXkrqPDAhEfED4ArgYknPlbSVpO0kndDk\nW3LTZSPpaZLm5w3zYVKHgZF15nWSdslHkWvyWzaSlstLJR2Wv/VuLelgSUP5aO+oHHSP5M9ttp4t\nBN4s6RmSngj8K6ltv5HtSOcGVgNbSfowmx69jeYn+b3/Jy+Ll9OieU3S1vzlCGdKftzINqSAWp3f\n90bS0c9ozgI+KGmv/J4dJL2i0YAR8X3gauDb+Uhncv7mfsAYargZeIHStZc7AB+smc6my0vSK2qC\n84/5Mxoty+1ILSgPS9qfFCRtyetNwy+3Zds35vl+IWkbeUNd60qr6fw7Sc9Uuu51HalJcbP3R8Qa\nUue5z0s6Om/XW0jah7ScG/kM6Tz0j1qU8cM83GfbrRva6Eaf2/PfRTq8W036hvaPNG7PXEhq/lhJ\n6lFzfd3rrwfuyofQJwCvzc/vQfrmvp60QX8+Iq4ey4TkcwsvJXUKWE76FvKq/PIpwL6kFeA7pJ6K\ntT4BfEipieM9DUb/atI3pntJ7bUfyTvqTvhX0lHUH3Od54/hvZ8ifVP7PmnF+zLpZDSkZrev5Gl6\nZe2bImKYNK9eQup59Xng2Ii4o50PVbrW5eRRBjkauIzUo24t6Rv3fqRlXG+0ZTOF1FHgflIzz5P5\nyw7uxcBtktaTegAek89jrCB9QzyZv6yv7yWt61uQ1uV7SU1jfwu8vdEERMT3gE+SdtB3k3rAfaTJ\n9F5O6kn1qzzsw2zeNNdQXhYvJ53Q/gNpna1fP+uN9GKFdFTSrGnmduA/SNvU70mdGUZtnomIi0lH\ntBfk7fQXpPWkmb8nnT89j/RF4rek7fqwdmqIdI7yG8CtpN6bl9aMe7Tl9dfADXn5XwKcFI2v/ToR\n+JikB0mB883Rpr/ODNKX14ZKtm8cOXJ6EbBG6bq09bl1oJVdSeG3jtT79IekZsVG0/zJPM3vIy3P\n35M6XbyfBvMqNytf2eq8XSRXRsQf2qj3cWoxXjOzgaR0p4pvRcTlRddijTnAzMysknwzXzMzqyQH\nmJmZVZIDzMzMKqmTF9sWZvIWU2Pqlg1vkGBmXTbj6WtaD2Slc9vPH70/InYpuo6J6IsAm7rldszb\nueGlKmbWRWfceDHt39TGymTuzFUdu+NLUdyEaGbjMjy33ZutmHWHA8zMxmx47hCfXzimmyaYdZwD\nzMzGzOFlZeAAM7MxcdOhlYUDzMza5qZDKxMHmJm17TfHeZdh5eG10czasnrBbC6ff2bRZZg9zgFm\nZi0Nzx3iax/596LLMNuEA8zMWlo7e0rrgcx6zAFmZi356MvKyAFmZqNaduKcokswa8gBZmZNLTtx\nDpccd3rRZZg15AAzs6aOffmVRZdg1pQDzMwaWnraTI7efknRZZg15QAzs80sO3GOr/my0nOAmdlm\nps1bWXQJZi31xQ9amlnnLD1tJpfv6aMvKz8fgZnZJtx0aFXhADOzx204b2rRJZi1zQFmZkAKr7P3\nPL/oMsza5gAzMwAO3fWOokswGxMHmJmx4bypvubLKscBZjbglp04x02HVkkOMLMB59tFWVU5wMwG\n2IE/uNtNh1ZZDjCzAeXzXlZ1DjCzAeXzXlZ1DjCzAXTgD+4uugSzCXOAmQ0Yn/eyfuEAMxsg/o0v\n6ycOMLMB4hv1Wj9xgJkNCJ/3sn7jADMbAD7vZf3IAWY2ABxe1o8cYGZ97owbLy66BLOucICZ9TGf\n97J+5gAz61M+72X9zgFm1od8n0MbBD0PMElbSvqZpEsbvCZJn5F0p6RbJe3b6/rMqm7DeVN9n0Mb\nCEUcgZ0ELG3y2kuAPfK/44Ev9Koos37h8LJB0dMAkzQELAC+1GSQo4CFkVwP7ChpWs8KNKs4d9qw\nQdLrI7AzgPcBG5u8Ph1YUfP4nvzcZiQdL2mxpMXDGzd0tkqzCnKnDRs0PQswSUcA90XETZ0YX0Sc\nExH7RcR+k7eY2olRmlWWO23YIOrlEdjzgSMl3QVcAMyXdF7dMCuBGTWPh/JzZtaEO23YoOpZgEXE\nByNiKCJmAccAV0XE6+oGuwQ4NvdGPABYGxGrelWjWdUsPW2mw8sG1lZFFyDpBICIOAu4DDgcuBP4\nE/DGAkszK7Wlp830z6PYQCskwCLiGuCa/PdZNc8H8I4iajKrEoeXme/EYVY5y06c4/AywwFmVinL\nTpzDJcedXnQZZqXgADOriNULZju8zGo4wMwqYHjuEPcf9GjRZZiVigPMrOSG5w5xz/yteet+1xZd\nilmpOMDMSu6e+VvzyMxh32nDrE7h14GZWWOrF8zmwd1VdBlmpeUAMyuhZSfOKboEs9JzE6JZyTi8\nzNrjADMrEYeXWfscYGYlMVp4+c4bZptzgJmVgI+8zMbOnTjMCuSehmbj5yMws4I4vMwmxgFmVoCx\nhNexL7+yy9WYVZMDzKzHxnrk5TtwmDXmADPrITcbmnWOA8ysR4bnDjm8zDrIAWbWAyN3lB8rn/8y\na87d6M26zM2GZt3hADPrIl+gbNY9bkI065KJhte0eSvdA9FsFA4wsy7wkZdZ9znAzDqsU+F16K53\ndGQ8Zv3KAWbWQZ0KLzcfmrXmThxmHTDebvJmNn4OMLMJ8vkus2K4CdFsAroVXmfveX5XxmvWTxxg\nZuPUrfB6ZOZwV8Zr1m8cYGbjsHrB7KJLMBt4DjCzMer2raFmDa3u2rjN+okDzGwMenFfQ5//MmuP\neyGatWF47hBrZ0/xTXnNSsQBZtaCr/EyKyc3IZqNotfh5d//MmtfzwJM0taSbpR0i6TbJJ3SYJiD\nJa2VdHP+9+Fe1WdWr4gjL98+yqx9vWxCfASYHxHrJU0CrpP03Yi4vm64ayPiiB7WZdaQmw3Nyq1n\nARYRAazPDyflf9GrzzcbC98eyqz8enoOTNKWkm4G7gOuiIgbGgw2T9Ktkr4raa9RxnW8pMWSFg9v\n3NC1mm2wDM8dKiy8fP7LbGx62gsxIv4M7CNpR+BiSXtHxC9qBlkCzMzNjIcD3wb2aDKuc4BzAHaY\n9GQfydmE+ajLrFoK6YUYEWuAq4EX1z2/LiLW578vAyZJ2rmAEm3AOLzMqqeXvRB3yUdeSJoKHArc\nUTfMrpKU/94/1/dAr2q0weTwMqumXjYhTgO+ImlLUjB9MyIulXQCQEScBRwNvF3SY8AG4Jjc+cOs\nK8oSXv4FZrOx62UvxFuB5zR4/qyavz8HfK5XNdlgK0t4ARy66x2tBzKzTfhOHDaQyhReZjY+vhei\nDRTf19CsfzjAbGD04qdQzKx33IRoA6Hs4eUOHGZj5wCzvjc8d6jU4TVt3sqiSzCrJAeY9TWf8zLr\nXw4w61sOL7P+5gCzvuTwMut/DjDrOw4vs8HgbvTWV6p4gfLZe55fdAlmleQjMOsbVQyvR2YOF12C\nWWU5wKwvVDG8zGxiHGBWeQ4vs8HkALNKc3iZDS534rBKck9DM3OAWeX4qMvMwE2IVjGrF8wuugQz\nKwkHmFVG2e8ob2a95QCzSnB4mVk9B5iVnsPLzBpxgFmpObzMrBn3QrRSGp47BODwMrOmHGBWSr7G\ny8xacROilY6v8zKzdjjArFQcXmbWLgeYlYbDy8zGwgFmpeDwMrOxcoBZ4RxeZjYe7oVohfEd5c1s\nIhxgVggfdZnZRLkJ0XrO4WVmneAAs55yeJlZpzjArGccXmbWSQ4w6wn/EKWZdVrPAkzS1pJulHSL\npNskndJgGEn6jKQ7Jd0qad9e1Wfd4zvKm1k39PII7BFgfkQ8G9gHeLGkA+qGeQmwR/53PPCFHtZn\nHTY8d8jh1cKU5ZN5269eU3QZZpXUs270ERHA+vxwUv4XdYMdBSzMw14vaUdJ0yJiVa/qtM7wNV5m\n1m09PQcmaUtJNwP3AVdExA11g0wHVtQ8vic/12hcx0taLGnx8MYN3SnYxsXhZWa90NMAi4g/R8Q+\nwBCwv6S9JzCucyJiv4jYb/IWUztXpE2Iw8vMeqWQXogRsQa4Gnhx3UsrgRk1j4fyc1YBDi8z66Ve\n9kLcRdKO+e+pwKHAHXWDXQIcm3sjHgCs9fmv6nB4mVkv9fIIbBpwtaRbgZ+SzoFdKukESSfkYS4D\nlgF3Al8ETuxhfTYBvkh5/FYtms6F63zFiNlY9bIX4q3Acxo8f1bN3wG8o1c12cS52dDMiuK70du4\n+ajLzIrkW0nZuDi8OuuLiw8qugSzynGA2Zg5vDpvyvLJPg9mNkYOMBsTh5eZlYUDzNrm8OquhRcd\nUnQJZpXiALO2OLzMrGwcYDaq4blDDq8e8nkws/Y5wKyp1Qtm+xqvHnMzoln7HGDWkH/Hy8zKzgFm\nmxmeO+TwKpB/4NKsPQ4w24RvDVW8u+7ZpegSzCrBAWaPc3iVw5Tlk30UZtYGB5gBDq+y8VGYWWsO\nMHN4lZCPwsxa893oB5yv8Sqvu+7ZBfYsugqz8vIR2ABzeJWbb/BrNjoH2IByeFWDf2bFrDkH2ABy\neFXHlOWTOeyqk4ouw6yUHGADxuFVPVOWTy66BLNScoANEIdXdR157nuLLsGsdBxgA8B3lO8Pbko0\n25QDrM/5jvL9w9eGmW3KAdbHfEf5/rNq0fSiSzArDQdYn3J49S+fDzNLHGB9yOHV/w676iSfE7OB\n5wDrMw6vwTBl+WRfI2YDzwHWR/xDlIPHHTtskDnA+oTvKD+4Vi2a7hCzgeQA6wPLTpzj8BpwqxZN\n58hz3+ub/9pAcYBVnC9QtloLLzrER2M2MBxgFebwskZGjsYcZNbv/IOWFeXwslZWLZrOkYveyyMz\nh5k1tJqz9zy/6JLMOspHYBXk8LKxmLJ8MqsWTeewq07yOTLrKz4CqxiHl43XlOWTWbj8EBZyCACX\nHHd6wRWZTYwDrEIcXtZJtbekmjZvJYfuegdHb7+kwIrMxqZnASZpBrAQeAoQwDkRcWbdMAcD/w/4\nbX7qooj4WK9qLCvfXcO6bdWi6SxkOgs5hGNffqWDzCqhl0dgjwHvjoglkrYDbpJ0RUTcXjfctRFx\nRA/rKjUfdVmvLbwoNTNOm7fSHT+s1HrWiSMiVkXEkvz3g8BSwL8NMQqHlxVppOOHu+NbWRXSC1HS\nLOA5wA0NXp4n6VZJ35W01yjjOF7SYkmLhzdu6FKlxVm9YHbRJZg93oPRIWZl1PMAk7Qt8D/AOyNi\nXd3LS4CZEfEs4LPAt5uNJyLOiYj9ImK/yVtM7V7BBfA5Lysbh5iVUU8DTNIkUnh9LSIuqn89ItZF\nxPr892XAJEk797LGojm8rKwcYlY2PQswSQK+DCyNiE81GWbXPByS9s/1PdCrGovm8LKyW7XIp62t\nPHrZC/H5wOuBn0u6OT93MjATICLOAo4G3i7pMWADcExERA9rLMTw3CHWzp7i8LJKeNuvXuPeiVYK\nPQuwiLgOGHUPHRGfAz7Xm4rKwb/jZVWzatF02LPoKsx8L8RCObzMzMbPAVYgh5dV1WFXnVR0CWYO\nsKL4ImWrsinLJxddgpkDrAgOLzOziXOA9ZjDy8ysMxxgPeTwsn7iH8e0ojnAemD1gtkOL+s7X1x8\nUNEl2IDzD1p2mYPLzKw7fATWRQ4vM7PucYB1icPLzKy7HGBd4PAyM+s+B1iH+YcobVC8db9riy7B\nBpwDrIP8cyg2SI7efknRJdiAc4B1iMPLzKy33I1+gnxHeTOzYvgIbAIcXmZmxXGAjZPDy8ysWA6w\ncXB42aCbNm9l0SWYOcDGw+Flg+7sPc8vugQzB9hY+SJlM7NycICNgcPLzKw8HGBtcniZmZWLrwNr\nwcFltqlLjju96BLMAB+BjcrhZWZWXg6wJhxeZmbl5gBrwOFl1tgjM4eLLsHscQ6wOg4vs+b8EypW\nJg6wGg4vM7PqcIBlDi8zs2pxgOFfUTZrx7R5K/0jllYqAx9g/iFKM7NqGtgLmX1HebOxOXTXO4ou\nwWwTA3kE5vAyG5tHZg67+dBKZ+ACzOFlZtYfBirAHF5mZv2jZwEmaYakqyXdLuk2SSc1GEaSPiPp\nTkm3Stq3kzU4vMzG5/L5ZxZdgtlmetmJ4zHg3RGxRNJ2wE2SroiI22uGeQmwR/73POAL+f8J83Ve\nZmb9pWdHYBGxKiKW5L8fBJYC0+sGOwpYGMn1wI6Spk30sx1eZmb9RxHR+w+VZgE/AvaOiHU1z18K\nnBYR1+XHVwLvj4jFDcZxPHB8frg38Isul90JOwP3F11EG1xn51ShRnCdnVaFOnePiF2KLmIien4d\nmKRtgf8B3lkbXmMVEecA5+RxLo6I/TpUYte4zs6qQp1VqBFcZ6dVpc6q62kvREmTSOH1tYi4qMEg\nK4EZNY+H8nNmZmab6GUvRAFfBpZGxKeaDHYJcGzujXgAsDYiVvWqRjMzq45eNiE+H3g98HNJN+fn\nTgZmAkTEWcBlwOHAncCfgDe2Oe5zOltq17jOzqpCnVWoEVxnp1WlzkorpBOHmZnZRA3UnTjMzKx/\nOMDMzKySKhVgkraU9LN8vVj9a129DdVYtKjzYElrJd2c/324oBrvkvTzXEOj6+xKMT/bqLMs83NH\nSRdKukPSUkl/U/d6WeZnqzoLn5+Snlbz+TdLWifpnXXDFD4/26yz8PnZz6r2e2Anke7gsX2D17p2\nG6pxGK1OgGsj4oge1tPM30VEs4styzQ/R6sTyjE/zwS+FxFHS5oMPKHu9bLMz1Z1QsHzMyJ+CewD\n6csg6VKai+sGK3x+tlknlGP97EuVOQKTNAQsAL7UZJCu3IZqrNqosypKMT+rQNIOwAtIl4kQEcMR\nsaZusMLnZ5t1ls0hwG8i4u665wufn3Wa1WldVJkAA84A3gdsbPL6dGBFzeN72Pxei73Qqk6AebnZ\n47uS9upRXfUC+IGkm/JtueqVZX62qhOKn59/BawG/js3HX9J0jZ1w5RhfrZTJxQ/P2sdA3y9wfNl\nmJ+1mtUJ5ZqffaUSASbpCOC+iLip6FpG02adS4CZEfEs4LPAt3tS3OYOjIh9SE0x75D0goLqaKVV\nnWWYn1sB+wJfiIjnAA8BHyigjlbaqbMM8xOA3MR5JPCtompoR4s6SzM/+1ElAox0EfSRku4CLgDm\nSzqvbpgy3IaqZZ0RsS4i1ue/LwMmSdq5x3USESvz//eR2u33rxukDPOzZZ0lmZ/3APdExA358YWk\noKhVhvnZss6SzM8RLwGWRMTvG7xWhvk5ommdJZuffacSARYRH4yIoYiYRTpUvyoiXlc3WOG3oWqn\nTkm7SlLDupizAAAA1klEQVT+e3/SMnigl3VK2kbpN9nITUgvYvO7+Rc+P9upswzzMyJ+B6yQ9LT8\n1CHA7XWDFT4/26mzDPOzxqtp3ixX+Pys0bTOks3PvlO1XoibkHQCTPg2VF1XV+fRwNslPQZsAI6J\n3t8O5SnAxXm72go4PyK+V8L52U6dZZifAP8EfC03Jy0D3ljC+dlOnaWYn/kLy6HA22qeK938bKPO\nUszPfuVbSZmZWSVVognRzMysngPMzMwqyQFmZmaV5AAzM7NKcoCZmVklOcDMzKySHGBmZlZJ/wu2\n6rV4Tjk6nAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Let's try creating a classifier.\n", "# We'll use Gaussian models for classes 0 and 1, \n", "# and a GMM for class 2\n", "maxpdf = np.maximum(pdf0,np.maximum(pdf1,gmm_pdf2))\n", "yhat_using_gmm = (pdf1==maxpdf) + 2*(gmm_pdf2==maxpdf)\n", "plt.contourf(x0coords,x1coords,yhat_using_gmm)\n", "plt.title('Classification function: Classes 0 and 1 are Gaussian, Class 2 is GMM')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Check our results: GMM boundaries don't need to be quadratic!!\n", "Remember that in a Gaussian classifier, the border between any two classes needs to be quadratic or linear. In a GMM classifier, that's no longer true. Roughly speaking, the border can have as many wiggles as the GMM has Gaussians." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.1" } }, "nbformat": 4, "nbformat_minor": 2 }