nbs/02-Linear-algebra-with-Numpy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear algebra with Numpy\n",
    "\n",
    "It is possible to do symbolic linear algebrea with [Sympy](http://www.sympy.org/en/index.html) but for numeric computations [Numpy](http://www.numpy.org/) is a high performance library that should be used. \n",
    "\n",
    "Here is how it is described: \n",
    "\n",
    "> NumPy is the fundamental package for scientific computing with Python. It contains among other things: [...]\n",
    " useful linear algebra, Fourier transform, and random number capabilities.\n",
    "\n",
    "In this section we will see how to:\n",
    "\n",
    "- Manipulate matrices;\n",
    "- Solve Matrix equations;\n",
    "- Calculate Matrix inverse and determinants.\n",
    "\n",
    "## Manipulating matrices\n",
    "\n",
    "It is straightforward to create a Matrix using Numpy. Let us consider the following as a examples:\n",
    "\n",
    "$$\n",
    "A = \\begin{pmatrix}\n",
    "5 & 6 & 2\\\\\n",
    "4 & 7 & 19\\\\\n",
    "0 & 3 & 12\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "B = \\begin{pmatrix}\n",
    "14 & -2 & 12\\\\\n",
    "4 & 4 & 5\\\\\n",
    "5 & 5 & 1\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "\n",
    "First, similarly to Sympy, we need to import Numpy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can define $A$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "A = np.matrix([[5, 6, 2],\n",
    "               [4, 7, 19],\n",
    "               [0, 3, 12]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[ 5,  6,  2],\n",
       "        [ 4,  7, 19],\n",
       "        [ 0,  3, 12]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "B = np.matrix([[14, -2, 12],\n",
    "               [4, 4, 5],\n",
    "               [5, 5, 1]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[14, -2, 12],\n",
       "        [ 4,  4,  5],\n",
       "        [ 5,  5,  1]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can obtain the following straightforwardly:\n",
    "\n",
    "- $5A$ (or any other scalar multiple of $A$);\n",
    "- $A ^ 3$ (or any other exponent of $A$);\n",
    "- $A + B$;\n",
    "- $A - B$;\n",
    "- $AB$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[25, 30, 10],\n",
       "        [20, 35, 95],\n",
       "        [ 0, 15, 60]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "5 * A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[ 557, 1284, 3356],\n",
       "        [ 760, 2305, 6994],\n",
       "        [ 288, 1074, 3519]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A ** 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[19,  4, 14],\n",
       "        [ 8, 11, 24],\n",
       "        [ 5,  8, 13]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A + B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[ -9,   8, -10],\n",
       "        [  0,   3,  14],\n",
       "        [ -5,  -2,  11]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A - B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[104,  24,  92],\n",
       "        [179, 115, 102],\n",
       "        [ 72,  72,  27]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A * B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**EXERCISE** Compute $A ^ 2 - 2 A + 3$ with:\n",
    "\n",
    "$$A = \n",
    "\\begin{pmatrix}\n",
    "1 & -1\\\\\n",
    "2 & 1\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "---\n",
    "\n",
    "## Solving Matrix equations\n",
    "\n",
    "We can use Numpy to (efficiently) solve large systems of equations of the form:\n",
    "\n",
    "$$Ax=b$$\n",
    "\n",
    "Let us illustrate that with:\n",
    "\n",
    "$$\n",
    "A = \\begin{pmatrix}\n",
    "5 & 6 & 2\\\\\n",
    "4 & 7 & 19\\\\\n",
    "0 & 3 & 12\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "b = \\begin{pmatrix}\n",
    "-1\\\\\n",
    "2\\\\\n",
    "1 \n",
    "\\end{pmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "A = np.matrix([[5, 6, 2],\n",
    "               [4, 7, 19],\n",
    "               [0, 3, 12]])\n",
    "b = np.matrix([[-1], [2], [1]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the `linalg.solve` command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[ 0.45736434],\n",
       "        [-0.62790698],\n",
       "        [ 0.24031008]])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.linalg.solve(A, b)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can verify our result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[-1.],\n",
       "        [ 2.],\n",
       "        [ 1.]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A * x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**EXERCISE** Compute the solutions to the matrix equation $Bx=b$ (using the $B$ defined earlier).\n",
    "\n",
    "---\n",
    "\n",
    "## Matrix inversion and determinants\n",
    "\n",
    "Computing the inverse of a matrix is straightforward:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[-0.20930233,  0.51162791, -0.7751938 ],\n",
       "        [ 0.37209302, -0.46511628,  0.6744186 ],\n",
       "        [-0.09302326,  0.11627907, -0.08527132]])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Ainv = np.linalg.inv(A)\n",
    "Ainv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can verify that $A^{-1}A=\\mathbb{1}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[  1.00000000e+00,   2.77555756e-17,   3.05311332e-16],\n",
       "        [ -2.08166817e-16,   1.00000000e+00,  -2.08166817e-16],\n",
       "        [  5.55111512e-17,  -5.55111512e-17,   1.00000000e+00]])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A * Ainv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above might not look like the identity matrix but if you look closer you see that the diagonals are all `1` and the off diagonals are a **very** small number (which from a computer's point of view is `0`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To calculate the determinant:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-128.99999999999997"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.linalg.det(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**EXERCISE** Compute the inverse and determinant of $B$ (defined previously).\n",
    "\n",
    "---\n",
    "\n",
    "## Summary\n",
    "\n",
    "In this section we have seen how to using Numpy:\n",
    "\n",
    "- Manipulate matrices;\n",
    "- Solve linear systems;\n",
    "- Compute Matrix inverses and determinants.\n",
    "\n",
    "This again just touches on the capabilities of Numpy.\n",
    "\n",
    "Let us take a look at [Pandas](03 - Data analysis with Pandas.ipynb) for data analysis."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}