{ "metadata": { "name": "", "signature": "sha256:db714a93f6656fae684befbdbec6a96c12e405e659ee9bd2e8bcc2ae76bdd595" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Introduction to Numpy" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Python lists:\n", "\n", "* are very flexible\n", "* don't require uniform numerical types\n", "* are very easy to modify (inserting or appending objects).\n", "\n", "However, flexibility often comes at the cost of performance, and lists are not the ideal object for numerical calculations." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "This is where **Numpy** comes in. Numpy is a Python module that defines a powerful n-dimensional array object that uses C and Fortran code behind the scenes to provide high performance." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "The downside of Numpy arrays is that they have a more rigid structure, and require a single numerical type (e.g. floating point values), but for a lot of scientific work, this is exactly what is needed." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "The Numpy module is imported with:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import numpy" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Although in the rest of this course, and in many packages, the following convention is used:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import numpy as np" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 3 }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is because Numpy is so often used that it is shorter to type ``np`` than ``numpy``." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Creating Numpy arrays" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "The easiest way to create an array is from a Python list, using the ``array`` function:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "a = np.array([10, 20, 30, 40])" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "a" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Numpy arrays have several attributes that give useful information about the array:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "a.ndim # number of dimensions" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "a.shape # shape of the array" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "a.dtype # numerical type" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "*Note: Numpy arrays actually support more than just one integer type and one floating point type - they support signed and unsigned 8-, 16-, 32-, and 64-bit integers, and 16-, 32-, and 64-bit floating point values.*" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "There are several other ways to create arrays. For example, there is an ``arange`` function that can be used similarly to the built-in Python ``range`` function, with the exception that it can take floating-point input:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "np.arange(10)" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "np.arange(3, 12, 2)" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "np.arange(1.2, 4.4, 0.1)" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Another useful function is ``linspace``, which can be used to create linearly spaced values between and including limits:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "np.linspace(11., 12., 11)" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 3, "text": [ "array([ 11. , 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8,\n", " 11.9, 12. ])" ] } ], "prompt_number": 3 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "and a similar function can be used to create logarithmically spaced values between and including limits:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "np.logspace(1., 4., 7)" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Finally, the ``zeros`` and ``ones`` functions can be used to create arrays intially set to ``0`` and ``1`` respectively:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "np.zeros(10)" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "np.ones(5)" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Combining arrays" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Numpy arrays can be combined numerically using the standard ``+-*/**`` operators:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x = np.array([1,2,3])\n", "y = np.array([4,5,6])" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "x + 2 * y" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "x ** y" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Note that this differs from lists:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x = [1,2,3]\n", "y = [4,5,6]" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "x + 2 * y" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Exercise 1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create an array which contains 11 values logarithmically spaced between $10^{-20}$ and $10^{-10}$." ] }, { "cell_type": "code", "collapsed": false, "input": [ "# your solution here" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create an array which contains the value 2 repeated 10 times (hint: there are two quick ways to do that \u2013 one is my multiplying a list, one my multiplying an array)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# your solution here" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Try using ``np.empty(10)`` and compare the results to ``np.zeros(10)``. Also compare to what the people next to you got for np.empty. What do you think is going on?" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# your solution here" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create an array containing 5 times the value 0, as a 32-bit floating point array (hint: have a look at the docs for np.zeros; the type you'd be looking for is called float32)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# your solution here" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Accessing and Slicing Arrays" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Similarly to lists, items in arrays can be accessed individually:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x = np.array([9,8,7])" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "x[0]" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "x[1]" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "and arrays can also be **sliced** by specifiying the start and end of the slice (where the last element is exclusive):" ] }, { "cell_type": "code", "collapsed": false, "input": [ "y = np.arange(10)" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "y[0:5]" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "optionally specifying a step:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "y[0:10:2]" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "As for lists, the start, end, and step are all optional, and default to ``0``, ``len(array)``, and ``1`` respectively:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "y[:5]" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "y[::2]" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Exercise 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Given an array ``x`` with 10 elements, find the array ``dx`` containing 9 values where ``dx[i] = x[i+1] - x[i]``. Do this without loops!" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# your solution here" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Multi-dimensional arrays" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Numpy can be used for multi-dimensional arrays:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x = np.array([[1.,2.],[3.,4.]])" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "x.ndim" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "x.shape" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "y = np.ones([3,2,3]) # ones takes the shape of the array, not the values" ], "language": "python", "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "y" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "y.shape" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Multi-dimensional arrays can be sliced differently along different dimensions:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "z = np.ones([6,6,6])" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "z[::3, 1:4, :]" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Functions" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "In addition to an array class, Numpy contains a number of **vectorized** functions, which means functions that can act on all the elements of an array, typically much faster than could be achieved by looping over the array." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "For example:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "theta = np.linspace(0., 2. * np.pi, 10)" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "theta" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "np.sin(theta)" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Another useful package is the ``np.random`` sub-package, which can be used to genenerate random numbers fast:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# uniform distribution between 0 and 1\n", "np.random.random(10)" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "# 10 values from a gaussian distribution with mean 3 and sigma 1\n", "np.random.normal(3., 1., 10)" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Another very useful function in Numpy is [numpy.loadtxt](http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html) which makes it easy to read in data from column-based data. For example, given the following file:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%cat data/columns.txt" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "1995.00274 0.944444\r\n", "1995.00548 -1.61111\r\n", "1995.00821 -3.55556\r\n", "1995.01095 -9.83333\r\n", "1995.01369 -10.2222\r\n", "1995.01643 -9.5\r\n", "1995.01916 -10.2222\r\n", "1995.02190 -6.61111\r\n", "1995.02464 -2.94444\r\n", "1995.02738 1.55556\r\n", "1995.03012 0.277778\r\n", "1995.03285 -1.44444\r\n", "1995.03559 -3.61111\r\n" ] } ], "prompt_number": 1 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "We can either read it in using a single multi-dimensional array:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "data = np.loadtxt('data/columns.txt')\n", "data" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 4, "text": [ "array([[ 1.99500274e+03, 9.44444000e-01],\n", " [ 1.99500548e+03, -1.61111000e+00],\n", " [ 1.99500821e+03, -3.55556000e+00],\n", " [ 1.99501095e+03, -9.83333000e+00],\n", " [ 1.99501369e+03, -1.02222000e+01],\n", " [ 1.99501643e+03, -9.50000000e+00],\n", " [ 1.99501916e+03, -1.02222000e+01],\n", " [ 1.99502190e+03, -6.61111000e+00],\n", " [ 1.99502464e+03, -2.94444000e+00],\n", " [ 1.99502738e+03, 1.55556000e+00],\n", " [ 1.99503012e+03, 2.77778000e-01],\n", " [ 1.99503285e+03, -1.44444000e+00],\n", " [ 1.99503559e+03, -3.61111000e+00]])" ] } ], "prompt_number": 4 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Or we can read the individual columns:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "date, temperature = np.loadtxt('data/columns.txt', unpack=True)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 5 }, { "cell_type": "code", "collapsed": false, "input": [ "date" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 6, "text": [ "array([ 1995.00274, 1995.00548, 1995.00821, 1995.01095, 1995.01369,\n", " 1995.01643, 1995.01916, 1995.0219 , 1995.02464, 1995.02738,\n", " 1995.03012, 1995.03285, 1995.03559])" ] } ], "prompt_number": 6 }, { "cell_type": "code", "collapsed": false, "input": [ "temperature" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are additional options to skip header rows, ignore comments, and read only certain columns. See the documentation for more details." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Masking" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "The index notation ``[...]`` is not limited to single element indexing, or multiple element slicing, but one can also pass a discrete list/array of indices:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x = np.array([1,6,4,7,9,3,1,5,6,7,3,4,4,3])\n", "x[[1,2,4,3,3,2]]" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which is returning a new array composed of elements 1, 2, 4, etc from the original array." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Alternatively, one can also pass a boolean array of ``True/False`` values, called a **mask**, indicating which items to keep:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x[np.array([True, False, False, True, True, True, False, False, True, True, True, False, False, True])]" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Now this doesn't look very useful because it is very verbose, but now consider that carrying out a comparison with the array will return such a boolean array:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x > 3.4" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "It is therefore possible to extract subsets from an array using the following simple notation:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x[x > 3.4]" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Conditions can be combined:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x[(x > 3.4) & (x < 5.5)]" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Of course, the boolean **mask** can be derived from a different array to ``x`` as long as it is the right size:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "x = np.linspace(-1., 1., 14)\n", "y = np.array([1,6,4,7,9,3,1,5,6,7,3,4,4,3])" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "y[(x > -0.5) & (x < 0.4)]" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Since the mask itself is an array, it can be stored in a variable and used as a mask for different arrays:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "keep = (x > -0.5) & (x < 0.4)\n", "x_new = x[keep]\n", "y_new = y[keep]" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "x_new" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "y_new" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "A mask can also appear on the left hand side of an assignment:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "y[y > 5] = 0." ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "y" ], "language": "python", "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Exercise 3" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The [data/munich_temperatures_average_with_bad_data.txt](data/munich_temperatures_average_with_bad_data.txt) data file gives the temperature in Munich every day for several years:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "!head data/munich_temperatures_average_with_bad_data.txt # shows the 10 first lines of a file" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Read in the file using ``np.loadtxt``. The data contains bad values, which you can identify by looking at the minimum and maximum values of the array. Use masking to get rid of the bad temperature values." ] }, { "cell_type": "code", "collapsed": false, "input": [ "\n", "# your solution here\n" ], "language": "python", "metadata": {}, "outputs": [] } ], "metadata": {} } ] }