Rootfinding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Jupyter notebooks\n",
    "\n",
    "This is a [Jupyter](http://jupyter.org/) notebook using Python.  You can install Jupyter locally to edit and interact with this notebook.\n",
    "\n",
    "# Rootfinding\n",
    "\n",
    "Rootfinding is the process of solving $$f(x) = 0$$ for $x$.  The standard assumption is that $f : R \\to R$ is _continuous_.  We are interested in developing general-purpose algorithms---those that can use $f(x)$ as a black box without needing to look inside.  When we implement our rootfinding algorithm in software, the user will pass a function or program to compute $f(x)$.  Rootfinding methods for differentiable functions may also use the derivative $f'(x)$.\n",
    "\n",
    "Some questions immediately arise:\n",
    "* **Existence.** When does this equation have at least one solution?\n",
    "* **Uniqueness.** When is the solution unique?\n",
    "\n",
    "Let's consider some test functions, defined here along with their derivatives which we'll use later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width);\n",
       "        canvas.attr('height', height);\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'];\n",
       "    var y0 = fig.canvas.height - msg['y0'];\n",
       "    var x1 = msg['x1'];\n",
       "    var y1 = fig.canvas.height - msg['y1'];\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x;\n",
       "    var y = canvas_pos.y;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        event.shiftKey = false;\n",
       "        // Send a \"J\" for go to next cell\n",
       "        event.which = 74;\n",
       "        event.keyCode = 74;\n",
       "        manager.command_mode();\n",
       "        manager.handle_keydown(event);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4XuydB7wcVfn3n929e3dv7+khFRQiWIBQlKIIERT9A0lQBMSCiFgSlRZCCIGE+saAgAaVFlFSSRCVIBhJAgIhomCoqaTf3svW9zmzu/fuvXd3Z+bOljkzv81nsnNnzjl7nnLO851nmoPwgQagAWgAGoAGoAFoABqwlQYctpIWwkID0AA0AA1AA9AANAANEAAQTgANQAPQADQADUAD0IDNNAAAtJnBIS40AA1AA9AANAANQAMAQPgANAANQAPQADQADUADNtMAANBmBoe40AA0AA1AA9AANAANAADhA9AANAANQAPQADQADdhMAwBAmxkc4kID0AA0AA1AA9AANAAAhA9AA9AANAANQAPQADRgMw0AAG1mcIgLDUAD0AA0AA1AA9AAABA+AA1AA9AANAANQAPQgM00AAC0mcEhLjQADUAD0AA0AA1AAwBA+AA0AA1AA9AANAANQAM20wAA0GYGh7jQADQADUAD0AA0AA0AAOED0AA0AA1AA9AANAAN2EwDAECbGRziQgPQADQADUAD0AA0AACED0AD0AA0AA1AA9AANGAzDQAAbWZwiAsNQAPQADQADUAD0AAAED4ADUAD0AA0AA1AA9CAzTQAALSZwSEuNAANQAPQADQADUADAED4ADQADUAD0AA0AA1AAzbTAADQZgaHuNAANAANQAPQADQADQAA4QPQADQADUAD0AA0AA3YTAMAQJsZHOJCA9AANAANQAPQADQAAIQPQAPQADQADUAD0AA0YDMNAABtZnCICw1AA9AANAANQAPQAAAQPgANQAPQADQADUAD0IDNNAAAtJnBIS40AA1AA9AANAANQAMAQPgANAANQAPQADQADUADNtMAANBmBoe40AA0AA1AA9AANAANAADhA9AANAANQAPQADQADdhMAwBAmxkc4kID0AA0AA1AA9AANAAAhA9AA9AANAANQAPQADRgMw0AAG1mcIgLDUAD0AA0AA1AA9AAABA+AA1AA9AANAANQAPQgM00AAC0mcEhLjQADUAD0AA0AA1AAwBA+AA0AA1AA9AANAANQAM20wAA0GYGh7jQADQADUAD0AA0AA0AAOED0AA0AA1AA9AANAAN2EwDAECbGRziQgPQADQADUAD0AA0AACED0AD0AA0AA1AA9AANGAzDQAAbWZwiAsNQAPQADQADUAD0AAAED4ADUAD0AA0AA1AA9CAzTRgKwCcMWPGbrZvFy/dvIT5c8eqVatW8vbJ/PfjvFTz0ux0Oq9Yvnz5uzbzBYgLDUAD0AA0AA1AAzbRgN0AcCfb9WsrV658O96+DIAv8t+P8fZlM2fOvIjB8Hpen6rFB7j8tBUrVqzXUtZKZSC3laypLgvsra4jK5WAva1kTXVZYG91HVmxhN0AcFcoFPra6tWr34oZk+Gvhtc/nDJlSuX8+fNDYjtvO8jlPsvlBDCm/EyfPn0xZxF/plbOavsht9UsCj9PpAH4OfzcDhqAn9vByoNltB0AsgpaeHHw53WXy3Ujg94RvDzJGb+j46DwNZEFZLD7p5pbYOCoacha+2Fva9lTTRrYW01D1toPe1vLnmrS2NXeMb3YCgAvvPDCMWvWrNnHGT4XA95ChsBj+Xq/mxkA/8gA+PE4AHyd91+nBoBcxvGLX/ziN/fee++1ao5mtf1XXXXVoqVLl86xmlxq8kBuNQ1Zaz/sbS17qkkDe6tpyFr7OX7fw/H7B8wCYWtJpk0aWwFgvEoYAkfw3+/n5eVNCgQCO/gUcIWWU8DiWgkGxmmirXHjxhWx83xfm6pRChqABqABaAAagAbMpAGGwIf37NnTIfrECaH1drqm3zYAeP755xd6vV43Z/rEKWBxnZ+4bu98/vvzvP4PPgJ4nA3/OK9P5+3XabkJhDOApVy25dChQ+KWYjP5dMb7UlJSQm1tbRn/HbP9AOQ2m0Uy2x/YO7P6NVvrsLfZLJK5/nDMpxEjRB6Iyni9NXO/ZN6WbQOADHYT2AyrBeTzIuQWN3j8lEHvo4svvvgozuo9xn9XCaDj9W/zDSDb1MwWA8CDBw/aDgBLS0uptdV+YwZyq40Ka+2Hva1lTzVpYG81DVlnvwDAkSNHAgCtY9LsSgIABABm1+Ny92sIjLnTfS5+GfbOhdZz95t2tDcAMJIJw2eIGgAAAgCH6DrSVbNjgBBGgtzSuaqhDsPehtQnVWUAIADQkMMCAAGAhhxIosoIjBIZKw1dhb3ToESJmrCjvQGAAEBDQxQACAA05EASVbZjgEAGEONboiFqqKt2HN8AQACgoUEDAESAMORAElW2Y4AAAGJ8SzRENXVV3OUswGfgh1+KQMFgUFMbshUST+hI9MQKACAA0JAvAwARIAw5kESVAYASGSsNXYW906BEEzZhR7smkxkACAA0NEQBgABAQw4kUWU7Bg5kADG+JRqimrpqx3EMAEzuGrgLWNOwSVwIAIgAYcB9pKpqx8ABAMT4lmqQauisHccxABAAqGFo6C8CAESA0O81ctawY+AAAGJ8yzlak/fajuMYAAgAzMg4BgAiQGTEsUzYqB0DBwAQ49uEQ9FQl2Qbx4FAgGbNmkUbNmwgsb5161YqLi7WpQMAIABQl8NoLQwARIDQ6iuyl5MtcKRL35A7XZqUox2r21s2+datW0cLFiygTZs2UWFhIfl8Ppo/fz79+c9/Jr/fTyeffDItWrSIRo0aldTBAIAAwIzMPgBAAGBGHMuEjcoWONKlQsidLk3K0Y7V7S2bfEuWLKGNGzfSmjVrFAeaM2cObdmyhR5//HEqKytT/n7vvfdo/fr1AMAhDDHcBDIEpcWqAAABgAbcR6qqsgWOdCkXcqdLk3K0Y3V7yyTf7Nmzae3atRQKhcjr9dJxxx1H//nPf+ihhx6is88+W3GoxsZG+sxnPkMrV66kE088MaGTIQOIDGBGZh8AIAAwI45lwkZlChzpVB/kTqc2zd+W1e2tJp94aDJ1d6XXUN6ChA+f1vIjixcvps2bNysZwHfeeYemTZtG//73v6mmpqa3+mmnnUbf+c536Nvf/jYAUItS48ogA6hTYfHFAYAAQAPuI1VVtcAhlTA6Ogu5dSjLAkWtbm81+cJdnRT6ydfTaknn/U+Ro6BwSG3GA+Drr79OF110EW3fvp08Hk9ve1/5ylfonHPOoZ/85CcAQJ1aBgDqVBgAMKIBtYnEgFpNXRVym9o8ae8c7J12lZq6QavbW00+ZABN7Z5p7xwA0IBKkQFEBtCA+0hVVS1wSCWMjs5Cbh3KskBRq9tbNvniM4A9PT00ZcoU+vWvf93vGsDjjz+eVqxYgWsAhzD+AIBDUFqsCgAQAGjAfaSqKlvgSJdyIXe6NClHO1a3t2zyxQOg8KCbbrqJ3njjDXrkkUeUu4DF3++//z4999xzSR0MN4EkH3sAQAPzEgAQAGjAfaSqKlvgSJdyIXe6NClHO1a3t2zyDQRA8RxA8VxAcXdw7DmAd955J40cORIAOIQhBgAcgtKQAcQ1gAbcRsqqsgWOdCkZcqdLk3K0Y3V7W12+RF6GDCAygBmZfZABRAYwI45lwkbtGDiEGSC3CZ0xg12yur2tLh8AUN/gQAZQn776lQYAAgANuI9UVe0YOACAGN9SDVINnbXjOEYGEBlADUNDfxEAIAKEfq+Rs4YdAwcAEONbztGavNd2HMcAQABgRsYxABABIiOOZcJG7Rg4AIAY3yYcioa6ZMdxDAAEABoaNMkqAwARIDLiWCZsNH4SdQS7qaDlX9RZ+XkT9jS9XbJjwAT4Wndes6M/AwABgOmNCtHWAIDWnSiTOYwdJ9CBQOBp+w+V1K6l+knzMzKuzNQo7G0ma2S+L1a3t9XlS+QhAEAAYEZmDgAgADAjjmXCRuMn0ZLa1eRlCKybdJsJe5reLtkxYCIDaN15zY7+DAAEAKY3KiADiMdjZMSjzNto/CRatftucgZaqG7yQvN2OE09s2PABAACANM0fEzRDAAQAJgRR0QG0LoTJU4B99dAbBJ1+hupes89ys7aSYuIHNZ+khQAMCNTp2kbtbq9rS4fTgHrG1rWnr316UJ3aQAgAFC300haIRY4vC1bqLD5JcrzNzAA3s4A6JJUIm3dtmPARAbQuvOabP4cCARo1qxZtGHDBhLrW7dupeLiYm2DN1oKGUBkAHU5jNbCAEDrTpTIACbOAJYeeopCrgIqbHmVaicuIHK6tQ4XKcvJFjDTpWTInS5Nmqsd2ey6bt065d2/mzZtosLCQnrkkUdozZo19N5771FFRQVt2bJFVcEAQACgqpMMpQAAEAA4FL+RsY4yiba0UNXuRdQ27EIqP/gE1U2cT2GnR0ZxNPdZtoCpWTCVgpA7XZo0Vzuy2XXJkiW0ceNGBfrE569//StfdeKgDz/8kJYtWwYANOheOAVsQIEAQACgAfeRqqoIHB11H1LlvgepfvyNVLNrAdVNmEdhzgZa+SNbwEyXLSB3ujRprnZksuvs2bNp7dq1FAqFyOv10tSpUxXoE58VK1bQPffcAwA06F4AQAMKBAACAA24j1RVReDwf/Q3yu98n1pGXk7DdtzMAHgTA6C+63GkEpo7K1PATKduIXc6tWmettTsyjGNugKhtHa4IM+pZO2G8lm8eDFt3ry5NwMYawMAOBRtDq4zNKuk57elbwUACACU3ok1CiACh+P9B8jvHUed5acxAN6kZAJDeaUaW5CzmFrAlFMq9V5DbnUdyVhCza6d/iB9Y8WHaRXtTzOPpEL30G4WAwCm1RSDGgMAGtAvABAAaMB9pKpaWlJE+f+9lppHXUkB72iq2T6HGsZdRyF3uVRy6O2sWsDU254s5SG3LJbS1081uyIDqE+fspcGABqwIAAQAGjAfaSqWuZqJPeHD1D9hLn86BcnA+BcBsCfMQBWSiWH3s6qBUy97clSHnLLYil9/ZTNrsgA6rOv3tIAQL0aiysPAAQAGnAfqapWdP6Lgi07qHXkpUq/a3bMo8axP6Zgfo1UcujtrGwBU698ycpD7nRp0lztyGbXgQAYDAaV5wGuWrWKxB3C4vpA8fF4kj+NAI+BSe6DAEAD4xMACAA04D5SVa06/Dh1eo6irvJTlH5X75xPTaOvpqBnuFRy6O2sbAFTr3wAwP4asLq9ZZNvIACKv8USu6lEnLIW66+++iqNHj06oTsDAAGA6ZoX+7UDAAQAZsSxzNZoyM+PfbmNM34/4ozfsCgALqDm0Xw9oGek2Xqb1v7IFjDTJTzkTpcmzdWOHe0KAAQAZmQUAgABgBlxLJM16u7cQeV1K6nuiOt73/1btWshPw7mW3xDyBiT9Ta93bFjwBQahNzp9SOztGZHuwIAAYAZGX8AQABgRhzLZI0WNTxPXkc7NVRe2Nuzql13MAB+kwHwCJP1Nr3dsWPABABad16zoz8DAAGA6Y0K0dYAgNadKJM5jB0n0Ip9vyYafjo1uaf0AeDuu6h1+MXkLxifkbFllkbtaG8AoHXnNTv6MwAQAJiReAIAtO5ECQCMaMAR6uYbPm6jnk/cSq3deb1qqdxzL7XVXED+wkkZGVtmadSOARMAaN15zY7+DAAEAGYkngAArTtRAgBZA3yHXenh5eQMtFLomF9Qa2ufvSv3LGYAPJ8B8MiMjC2zNGrHgAkAtO68Zkd/BgACADMSTwCA1p0oAYBERQ0vkLft39Q45moqqRzdHwA/WkLtVV8iX9HHMzK2zNKoHQMmANC685od/RkACADMSDwBAFp3orQ7AHra3qSSumf4WX8/UJ71N3ASrdj7K+qoPIsB8JiMjC2zNGrHgAkAtO68Zkd/BgACADMSTwCA1p0o7QyA7q5dVH7gUb7L91LyFR6lqGIwAD5InRVnUE/xJzIytszSqB0DJgDQuvOaHf0ZAAgAzEg8AQBad6K0KwC6fPUk7vptrzqHustO6lXDIADkMp1ln6WekuMyMrbM0qgdAyYA0Lrzmmz+LF77NmvWLNqwYYPyCritW7dScXGxrukBAAgA1OUwWgsDAK07UdoRAF3+Bio78LhyXV979Xn9VDBwEi3f9zB1lZ3IAPhprcNFynKyBcx0KRlyp0uT5mpHNruuW7eOFixYQJs2bSK3200333wzvfzyy1RbW6uclTjrrLPoxhtvpLKysqSKBgACADMyCgGAAMCMOFaWG3UEu6mwaQMVtrxCXaUnMPydz89/caYGwP2/o26Gv+7S47Pc2+z+nGwBM13agdzp0qS52pHNrkuWLKGNGzfSmjVrqLOzk+6//36aPn06TZo0ierr6+lHP/oRlZSU0O9+9zsA4BBczTGEOqgS1QAAEAAo9WAIh8jbuoWKG/9OgfzhDH5fSfpu34GBo+zAI9RTdCyfJj5RahWodV62gKkmj9b9kFurpuQqJ5NdZ8+eTWvXrqVQiOcpr5emTp1Ky5Yt66fw9evXkyj3zjvvAACH4IoAwCEoLVYlBoB/+ctfyOFwUH5+vpKmFt+x9djf4jsvr+9BugZ+1hRVZZpI0qkwK8jtCLaTp30bZ/xe5Wf9+fh6vy/zad+je9/zm0hfgwHwMQbAo/tdJ5hOPZulLSvYeyi6hNxD0Zr568hm18WLF9PmzZuVDGCijzj9u337dlq5ciUAcAjuBwAcgtIGAuCTTz5JHR0d5Pf7yefzKUtsXXwzKCpVnE5nLyDGg2EiWBT748vE1gVExm8XbebiI9tEki4dySq3I9jB0PcOedvfInfXTvJ7x/L1e5/kU76cwXOoH5gMAsCDy8hXMIm6yk9Nl2pN2Y6s9jaqTMhtVIPmrK9mVxGr+F6LtH5E3kMkSIbySQWAAvrENYEiS/jxjyd/HimuAUyu+aFZZSiWtGAdLaeAIwMq0A8IBwKigMR4YEy1LvaJ9mIfl8ulAKHaEgPHgeUGbo/9Lb5TDVq1icSC5lZEkkJu9jlnoJnc3Xt4+UhZ8noO8und0Xzd3nGcufsEhdzlukw0UO7Sg0/ye4DHMQB+Tlc7shWWwt4ZUCrkzoBSTdCkml39/jA9t6YlrT390oVlHJ+GhhrJAPAPf/gD3XXXXfTYY4/R8cenvg4ZAAgATKtDxxrTAoCZ+GFxTUQMBGPwGA+NAhAFZMbAM75MovX4cvH9jZ22jv+OrRcWFipF44Exti9+W6p1Aa+yfdQm0KzKw6DnCHVQnq+OXL5a/q7l7zr+PkROzvgFPCM503dEdBmvG/riZRkEgIf+pABlZ8XpWRU52z9mKntnUXjInUVlZ/Gn1OwqQwbwwQcfpN/+9rf0xz/+kY45Rv1B9ABAAKDqEJsxY8ZkLvQ4L9W8NPOp1SuWL1/+bqqKuQJAVWGGWCA+WxmDwmTfAt7a29t7ITMeIuPriPX4vwW8xj7i9HUsgylAMQaLYttAcIztT/YdqxO/f+C2oZ6GSAVCQ1S1ejW+QcMR7GSQayMXL85Au/JOXpHZcwWaeGkmp58XvoYvmFdGQXcN38gxTFmCvPgZzsiZr/47GksMAkB+R3DAPYw6Kz+vsQU5i6kFTDmlUu815FbXkYwlZLPrwAzgwoULlesBOTbT5MkiZKt/AIAAQFUvYQB8kQs9xtcVLJs5c+ZFDEPX8/pULQB4YMcOCvONHw4JM1qqiklSYKgTSTAYVIBw4BLLaCbal6p8ovZi28R3/EcAZwwQY3AYD4kD1weWEX+Lh5CKvsbvS7gufosTnHmOALmdQf72K7DmCHUri5MfvRJZ7+J1XkKdUeDr6F13UJhCTi+FXCW8FFMor4Rhr0LJ5AXzIkvIXUFhp2eoZtRcb6C9Sw6vYuisYAA8S3MbMhYcqp/LKGtODnRMpiir21s2+eIBcP/+/XTSSSf13mApXEckLcSBvXhQ9KhRoxJ6k7iD+IMPPlBijhIbxGVU3V083wbpnK/9n6hTxm3Y75EWLPjQTsybbNAa7Q7DXw238eGUKVMq58+fr6SoeNtBzlZ9dvXq1TuTtR/LAN41517q6uqhoFCngEAO/goMOnkR39H1yDa+acPFV8X22ye29ZXvrRsro9SJ7ef16DPacmk8ceOKOM2cjo8hORJUjm1Sbr4JBRioGNjEEo5AmCsc4MVHecT7eD1P/E2R7WKbixcBbso3W9XN63kOAXEMc44wuaJQ5+Ztbmeod8l3hSi2uF1cLnp/TpA9qifg4MVJPUEndfN3d8AVWYIu3pZHnQE3dfF3VzCfF7HuoU5egg43T3DcEPuAk/3BwY06HPwtfExZoutcxsk+ovytrItykTLKdq4jtgkIFouyPdqm0q7ToUykQnexb96kzBDiq4hP+Xd18aQZ/fs4/1+px1FM291nRLb11o2UF9v4V3rLK/vFNqV+33bl7+i+gfXiy8bKCZXGfq9vnYdHbxuOyG8ov2XIsxTXli1gpmM8plvu2E1wfDyjfKJfvSu9f0d3Dvw7JlP/epG/BraVbFu8XqL35PVTVWxbSUkxtbW1D1JjojqJBRggY7oMYrCd2EgYNryc2jvaDLYmV/XujnZa8ZuHOAbwHM7Q5+azKx6ex4sLC+gbdy4RwgAA5TJpent78cUXf4Zh70nO+PGzMCIfBsDXRBZw1apV/0z2azEApOdfIAoEeTIKMy6If6HoNx9thGN/89EH/wso+6Lryj5RJvJ3ZD//i24P8vbI3/H7uZSyPUQBBpEgO7P4FiUCDCTit0U/lAgoPvFBMLoe+YpE8thqtHDvIUG/7bFyUUXEAnVsYu8Ls3w01n9ajfu7/z6R3erbGb9PfV2p29uXyHrkd2Prcdt4oCf8hGOtRL5jvxr/Hd9aWCkfIQsRDMR6336xra+d2D6xjU/kRn5eKRyLgLE+RYNY/D6lWP/9/ery7vxwHvGDhvgfP3KIEdXF//L4Tt68MH8rfw28M5wzidyI8CO+Han3n4+BuJO6lL9jn4EW7O8QEU2L3pXmCw90ULuv7w7ieEsIseOt1Nd+n+X62yURrKlviyBr/CduS9SJYyVie+Jr9K331evXYhKGNI6WfX3W35b2GtpLJh4mcV47qAA/2IoKwl7FB8W6+N8d9cXBduFjMTFX9c5zkVlQ+B4/M0FZOC+TvBNZ2OPsGUnO9qPI1XEkOXzDlAOVhB/h28oYjSzinyP6txhpYvxG5mD+P9F2MRqV7ZG5WtRVRmjcNmJd9bUlchKirejcHt0nyseXiazHlRnQhth/6eVfpYICdxa0aZ6fqOID3/ytb3CHxLyoHJZGOidO0ZzzRbEGADSPubLfkyQA+DoPsOu0AOC7/7qDwsEexbGcHPQdYc6u8PiOuJr4m7eLv8Q+scb7e7cpdcTforQo1beutNPvbxHaRTnO2HCwF+XFOl9J109pkXAvgJSDNENhZJ2X3vVA3DojY2w7A4IoJ6bpMEOoUoe3CWQV38o+3q7sVyYssT2CuyHlX+SUawQi+ha+J1X5OywyWQp9OqPr0W/l72h2lL9FOfF3WGRAxaKs50X/Fs8U4L+VX4qWFe3HtRv5PW5D2R75Vn5b2SZ0FQG5oXwK+Kixq7Orv76ViVgRPCJ/7Du2TQCjAo1KTOCMJM89IUdk4b9d/O1m3+CkofKdJxYuE1tXvrlu5Luv3xGLiIWUJWJ18T34I35XSC68RnyLdkSbIksW5M51c7CILZ18pNzFSwcfLYv1ziDfVCT283cPbxPLUdUfcqxx0If145Xsm2jPzd8lfC1nqdPN324q5aXYxVAgToez/sXiFplG/tca6KEmXzcvXdTk76a2QDflczkvl/eyzb2cJffwt5/9K8DXjfo4k+vnRRzw5Cv73bw/TynnZn8Q+7uDfurhTK/op/hbfCJeGMs4cj+5rivaF7Eu+tQvlCtGimPw2EGT2KxkOeMypbyujDLuk3IgxnoU35FwHUWDqC8ofeb+ij6LxS1kY912cz97lG/WKy9Cl0IeoTPxLfrawzoQMnVx2U5efFyuJM9DlWJxe/nbS+W8HoF8vkFM6Iu/ffzdGvRRE+u60c/6FkvAp9hW/JYo18OLkEH8rtB7gdCp8u2K2NCV32tLZZ31XsQ6j9lV+I7oT+T3uE3xLXTQL2UmTtOJLC37AI9B4QOKL/CSzzIK3cQ+PcKOSv9YLzzndLM9xbewZ2Qbw6JYhB4ENPJ+P+8X2wO8iAEWPVTj3wxFZx7ROh+uKPtiPsFW6v07Yi2fp4O6vW3U4+XLMDgzX9BdzBkjMUriD236DvuUaSTqZL0HGbFtsVkw+oNipo5NO7E+xOrHH6sr+xK1ofQ7osc+v44I03coGplgevuilIz+Lcpx14/91O18CUtVghnCupvc7h7679t/VPxcGSOsC7HuZH/+6uk3CMEBgNY1v7pkek4B8/WB0zhbOE20WllZmb906dJrnn3gZ9TT3aMcwYnTfcqZR7HwuvIdBQDlbw6aYtKMbON1XhHbe+vE1+utz5mkaLlIe5EsVKx9nm/5dKO4oUI8ukUEDQ5s4m/lOjTOCUX/zhM3V4jAIoKgeHyMCIB8dOQWR0hiHy/5Yp2PjMS3J4/3Rdd7t/M2pY1oPTf/Hf9RbiQJ8iICi/It5BN64b8V+cV6RP6ITOLINiJLTAfKka0iY+RbBNSIrGJdBFmhw4hexXeQdSoCTpCne1EuAkFiG3/zjKnoSwHQvkxdBAwjIKjcHKIs4rRp5HmNYpuAI3E6MXKqVAQthif+Fmfk88Tf4tsltocjC1O/OCXctx75Oz+63a18RwKI6LcvyKeFlYX4dLCDFz5VzOudfg7MvC6+u/ixDF1+hjBe71b+ZjjjpdvP2TxWqALkDGTKNwsf4uApMgVB/g4xNIi/lUUEZf4OxraJ7AD/7eH4VsL/Fee7qNTL8MbZgbKCfKoo9FBFES+FXv4uoCIPB34PZx3FUbMIK6zv+JtqfHx9TQePgQNNLbS/oVFZ9jY00MHGJmrr7KCOnh7e30md3d1K30ZXVtCE4cNoPC8Thg+nUZWVXK6LGtvbqIlvLmpsa6O2rk7y8qUGhVWmL0AAACAASURBVB4P/7ZX+fbwI49a+ZVQLWLhZ2828+mdTm67iK/zKSssorKiyFJSUBCBMGWsRTIx4p+P9Sb6KpaeAMMD606xc3S8xL4VbIxGd+Vb8Z2IryptRqEgn4FX9CmfYcnD6+JvxV+ibYpvoacuvlyijWXv6OmmdtZTN1+H5M1nwOJ+CtlKuP9FLJ/oj9inLNH+lXg9bJMCpaxYClknzV3tVN/ewkszNXa0UEt3m+KTBdyml/sj+iTWy73FVF5YyvVLqMxbwr9TzGM7n/24D7ribSlsEwjxEwQYrHx8faqPr0/187c/3MXbuvnQj+0X7mH5u4U2eUzxpRPRMSG+xWUPYszEvsW4GbRNjKn4MuKghMGTwpyZEgvnER1hvoFJWcTffH21si+2LfrN5cQ2RmZeIvOQctCrHKD6le/IEjtI7Ts3o4wHMU4ih7fK/2JeFnOTOI/S5uiiDgffSc8Hx0pZUSZq94gvRX4tBrpK9k45wI/MYxEIEwf3yv+RA79oL5UDfuEf4qBPOXyPJAYiB2eRUrFDfaVsLDGgfEf2OMVlHtHakb8j876C2Mp6ZN4XehXryhkCXvedcDw18Bxgp0996166ecmlg0Qu4LHw1JI36aqrrnqwsbFRuZ6Jx+76FStWrLeLfoaWBrGgdhgC/8GD8nE2/uO8Pp1FvE7rTSDrn32fLzAVBNOnmLjVyMZBG+I2JdinpuJ+B9hxhTUntuKOJAf91gCv6P0zbrtHXAPoF2NGya8p06/Iy7l4povm3PqtK7k3ZV80NyeupYvVjdsuUC3WXqwdZZsADuVboF3kqFbJ3fD2KMpFj+77H633Hm0PEFKZqGNGUapE2ooE+dh69HQOg5WApXCE4DnbywGGASLMAVp8hzirIp6eGuIMC/n4QIC/Q0I3/B3mv0Oc6QpzlkvZxlkqAWZKWwwK4fhrRZWMZ+Qa0XDvdZ8ioka383ekfOSawPjtSlu920QZEWX7ykb2Ra8fTbKetAw7VQFDVRcDjHItIgef8oKdHMQC1OA7JpJdVS68i2Zio38L4O7b1gfZ0Yv4otAd2963n+NipJ4SQ6Pb49Mn/VIvMUeOWjoKa9HUSNTqsTaSDJQUg0a8Z7SNYbT3w/6n3MwjricNcdaf153KOm9j6BDrYp+DAcrJfyvrnKFSyivfvDCUOBjY+60zdET+Ft/iauLYMnhyiIyEyOiJZbp7Rw0DQ3SkCOWxL0ez79HtAlKU0cMHTWHlLAL7CH+LAwhx4KCMCpFp5kyfn2FZGRbRlGbkmlouogyL6LoYL8qRligXHTf87RAHdtGj33DsaE0ZOzx+xb7I0SB/i7p81MN/i3VxYT4fpUS2K+u8nS+vcYjy4vmnyhiMlBHfSnneFrnWlQ+Aud8u8QxTt4dvgvco3w53Pjl5Ed/EfxNvJ7GeJxaGR+Vb1BVL5IyDcq2tcvZBAHzEryPXqsb7Yx/oxawU/60cgAh1RQ9ixbpycCpEUywcWVc8QnxH15WDQ1742K938YnriMV28c2L2C/WY+V8bM/edaX9vt9Qfov7LbZde96xVO5Kz7XbajHKLPuLeO46dOjQoO6IA7UjjzxSbEcG0CzGylU/+DTwUZxBeYx/X+THW3j923wDyLZU/bHaY2D06B4XxyfWVgQSowFKHGmLwKV8RwJWZD3Rtkgg66sbA86+YBdpo38A7LdNSX9GfycKq0p7sTq926LBV0T2AduUaB9fPpqeFacIgwy8sfLFn+Kg6uFTuS/zBfO96eheUohL6cZvi9JEFLAj6bTogZNCGdn7OLycLSnkDEkhB/0CXheL2FbAGRRed3gYKDz9vx2cJXXm92W8w5yRDXEUFt/9FwYcPiBUtnFUF+sikivbOLqHxboCNiLiR8uI7Qr09JVT1uO3R/crWlKgJPYdD9ExGI8CsQL94mAgdplEdL13u4CcaDtxBwYuziCKjFhv3djBRPSGpL6DkMjNRvF/Jzo46d3WewATO9iJu1mu9wCo/zYBd/E300UOksTNdfE30/G6SDUm2xa7EU/UTfGRaV5TznoILo6eVVEuQeB18bdwub716Hbe/6lx/Jio7o7sDTQT/FKns4D+8NouJVMtsuTKJSsiW+520tVnfQIAaAIbSdkFAKD97pyXKUCkc1ANlLuo4QW+TqqF2oZdlLafiWVeo+dZI2AZzcz2blP+Fpsj2aa+/aIbkW1OftexK9jI11bycxOV5yjyd0hs4+cphqOP2VFO6uXxY3YKORlWwIuXM6j8yB3+VtYdHmXxFJbzKWYGPUd+dBt/8ynJMN+d3ZvdjGlgYKq872IuQVgRmIrQWxS6YlnLGMDF9sUBXW/ms29bOp5nqWY0+LmahuTcb0e79rgK6I9b9vQCsTiwEYAsIHDhBZ8BAMrpyrnvNQAQAJh7L8xODwYGjsLGDeTy11Hb8JnZ6UCCXxEPxs7r3kfungNKX1z+en4TSn3fw7HFcxLFQ7KV79LIOj9TMZzHz1Tk5ypqeX6iHQOmUDXkzplbZ/SH7WhXPAg6uUvFXdWVUb+zZOMAQACgJR07gVCDALDpJeX9wq0jvp4dFfAp4jwGvfzODxn49irgJzJ9wfwa5a0nQX4rSTC/mt9OUs3rlXxKMj1vQbFjwAQAWndes6M/AwABgBkJUgBA606UyRzGjhNoIiAoaNqkgFjriEsyMraURvlGivyuHeTpeJfyO97jmyN6yFdwpPJu44B3DL+LeJSmLJ6RDsLeRrQnX12r29vq8iXyOAAgADAjMxEAEACYEccyYaMDJ9GC5pfJ3bWLWkcOfryC0e47A21U2LyRvC2v8w2qRdRTdLSy+AvG87VzfQ+eNvo7WurbMWAiA2jdeU02fxavb5s1a5byqjexvnXrVuV1nHo+AEAAoB5/0VwWAGjdiRIZwP4aGASALf/irNyH1DLqcs3jRa2guKavsGkjFbS+zpm+ydRReSZn+cbG3Tyh1kL698sWMNOlAcidLk2aqx3Z7Lpu3TpasGABbdq0iQr5dZQ33HCDAoMtLS0k3vE7depUmjdvHo0ZMyapogGAAMCMjEIAIAAwI45lwkYHTqIiO+fp2MYA+G3jveVn6BU3rGfw20K+wiMZ/M5STu+a4SNbwEyXziB3ujRprnZks+uSJUto48aNtGbNGkWRH374IY0ePVqBQfFu8rvvvpteffVV+tvf/gYAHIKr4SaQISgtVgUACAA04D5SVR0EgK1vkLftv9Q8+ruG5HDw2yXKDjzObYSpveZrDH4jDbWX7sqyBcx0yQ+506VJc7Ujk11nz55Na9euVd5wFMv2LVu2rFehnfw2oHvvvZeefPJJev/99wGAQ3A1AOAQlAYAxGMiDLiNlFUHA+C/GQC3MgBeOWR5HMFOKj/wKD+SpYBaRvC1hGm6c3fIHUpQUaaACbmNa8Dq9laTTzyL08evLUznJ5/fGjXUZ1cuXryYNm/e3JsBFP164okn6I477lDe0OPmB5bffPPN9J3vfAcAOASjAQCHoDQAIADQgNtIWXVg4PBw9q+ArwNsHvODIcnjCLQz/P2eQvzIlpYR38j6zR1aO60WMLW2I1s5yC2bxbT1V82uPfxebX6/vbbGNJbid+2Sh99zPZRPIgCMtVNbW0tPPfUUnXDCCXTqqacCAIegYADgEJQGAAQAGnAbKasOAsD2t/mGjU3UNPaHuuVxBlqofP/vlev8WofPUN69ataPWsA0a7+N9gtyG9WgOeur2VWGDGC8ZhsaGuikk06i1157jaqqxFtcB39wE0hyXwQAGhinuAYQ1wAacB+pqg6cRPPbt1FR0wYGwB/pksMR7KLKvQ/wzR4Tqa3mgsjr1Ez8UQuYJu66oa5BbkPqM21l2eyaKgMolHzw4EE68cQTlZtAjj32WACgTs8DAOpUWHxxACAA0ID7SFV1EADyg5mL+M7dpiN+qksOcdrY2/pvahpztenhTwgmW8DUZYwUhSF3ujRprnZks2s8ADY2NtILL7xA55xzDpWXl9P+/ftp7ty59O677yp3CotrDZEB1OdvAEB9+upXGgAIADTgPlJVHQyAH/CjW/5CjUfM1iVHxd4Hqav0ROoum6qrXq4KyxYw06UnyJ0uTZqrHdnsGg+ATU1NdPXVV9O2bdtIXKtYVlZGJ598Mv385z+n8ePHJ1U0TgEn90EAoIHxCQAEABpwH6mqDpxE3Z3bqaRuLTWO+4VmOVw9h6ly3wNUP34Ov+GjQHO9XBaULWCmS1eQO12aNFc7drQrABAAmJFRCAAEAGbEsUzY6CAA5NfAlR5eSQ3jr9Pc26L6v5KL3/bROuLrmuvkuqAdA6bQOeTOtedl5vftaFcAIAAwI6MJAAgAzIhjmbDRgZNoXtceKjv8JwbAG7T1Nhykqt138l2/M8nPb/uQ5WPHgAkAtO68Zkd/BgACADMSbwCA1p0okzmMHSfQRECQ172Xyg4uo4YJczSNrfyOd/mU8TpqGMcZQ5Pf+RsvEOytybyWKWR1e1tdvkSOCAAEAGZkggIAAgAz4lgmbHRQBrDnAD/L7xGqnzhXU29LDz5Jwfwa6qg6R1N5sxSyY8BEBtC685od/RkACADMSDwBAFp3okQGsL8GBk6irp5DVLF/KQPgLapjyxHsoOpdd/Adw7MYAqtVy5upgB0DJgDQuvOaHf0ZAAgAzEhMAQBad6IEAKoAoK+WxCNd6ifdqjq2CppfIU/7W0N+bZzqD2SwgB0DJgDQuvOaHf0ZAAgAzEiIAABad6IEAKoBYD2/0eM+qpt0m+rYqtj7K+oqO5m6+fl/sn3sGDABgNad1+zozwBAAGBG4g4A0LoTJQAwNQA6/U1Utedeqpu8MOXYcvUc5Gf//Zrq+WaRsNObkXGYyUbtGDABgNad1+zozwBAAGBGYgQA0LoTJQBQBQADLVTNj3WpnbSI7+pN/jz54rpnyRnq4Me/XJyRMZjpRu0YMAGA1p3X7OjPAEAAYEbiBADQuhMlADA1ADoCbVSzexED4O0MgK7E6uJn/1XvvoNahn+Dn/03KSNjMNON2jFgAgCtO6/J5s+BQIBmzZpFGzZsILG+detWKi4u1jXsAYAAQF0Oo7UwANC6EyUAUAUA+c7eml23U+3EBUROd0J1ufhGkUq+/q9uIt8oItGz/+KFkS1gap271MpBbjUNyblfNruuW7eOFixYQJs2baLCwsJepQeDQTr//PPprbfeopdffpnGjRuX1CAAQABgRkYrABAAmBHHMmGjAydRR6ibanbeynB3S9Jr+/L4+r/y/b/lR8XMM6FE2rokW8DUJpV6KcitriMZS8hm1yVLltDGjRtpzZo1/dT9y1/+UskGvvTSS7R582YA4BCdMfnFO0Ns0E7VAIAAQLv4+6DAEfLRsJ23UN2Emyns6jsyj9dH5G0hT/DbQm6SVk2yBcx0KRpyp0uT5mpHJrvOnj2b1q5dS6FQiLxeL02dOpWWLVtGb7/9Nv3gBz+g3//+9/TFL34RGUADLgYANKA8ACAA0ID7SFV1UOAIB2jYjpupbjzf3ZtXklAW3e8LNqFGZAqY6VQf5E6nNs3Tlqpdw2FyhHvS2uGww5PyRrFUP7Z48WIlwxfLAPp8PjrvvPNo3rx5NHHiRDr55JMBgAasBQA0oDwAIADQgPtIVXUwAIYYAG+i+vE3UCivLKEs7s6dVFq7mhrGXyuVrPGdVQ2Y0kqWuuOQ25qGVbNr7NKOdEqf6jIRtd8ZCIALFy6klpYWuvvuu2nfvn10yimn4BSwmhJT7AcAGlAeABAAaMB9pKqaKHDUbJ9DDeOupZC7IgkAfkgldX+mxnE/k0pWACCRGihIa1CVjltdblX5TJwB3LJlC/34xz+mF198kYqKimjv3r106qmnAgANDEYAoAHlAQABgAbcR6qqiQFwbsr3++Z3vEfFDeu5zE+lkhUACACU1mEtBrjxGUCx/tBDD1FBQYEipbg2UGQDy8vL6corr6Sf/jTxPIO7gJM7BQDQwEgHAAIADbiPVFUTTaLVO26hprHXUDB/WEJZ8tu3UVHTP7jMj6WSFQAIAJTWYS0MgB0dHdTW1tYr4YEDB+irX/2qcqPI0UcfrWQFE30AgADAjIxnACAAMCOOZcJGEwIgPwamafRVFPSMSNhjT9tbVNi8mQHwhyaUSFuXVE+ZaWtGulKQWzqTaeqwbHYdeA1gvJC4BlCTyVMWQgbQgA4BgABAA+4jVdXEAHg7NY/+DgU8o5IA4JtU0LKFmsd8XypZkQFEBlBah7VYBjAddkAGEBnAdPjRoDYAgADAjDiWCRtNNIlW7VpELSMvo4B3bMIee1vfIG/bfxgSv2dCibR1SbaMiTap1EtBbnUdyVjCjnYFAAIAMzJWAYAAwIw4lgkbTQiAu+9U3vMbKEj8GiZvy2vk6XiHWkZ924QSaeuSHQOm0Azk1uYfspWyo10BgADAjIxTACAAMCOOZcJGEwPg3dQ6fAb5CyYk7HFB8yuU37Wds4SXm1AibV2yY8AEAFp3XrOjPwMAAYDaZnudpQCA1p0ok7mCHSfQZEBQuedeaqu5gPyFkxIDYNMmcnd/RK0jv6lzZJmnOOxtHltkoydWt7fV5UvkIwBAAGBG5g4AIAAwI45lwkYTTaKVe35J7dVfJl/RUQl7XNj0T8rrOUStI75uQom0dcmOARMZQOvOa3b0ZwAgAFDbbK+zFADQuhMlMoD9NZBoEq346D7qqJrGAPjxxADY+CK5/I3UxqeJZf3YMWACAK07r9nRnwGAAMCMxB8AoHUnSgCgBgDc+yvqqDiLfMXHJFRXUcPz5Ay2U9uwCzMy/rLRqB0DJgDQuvOaHf0ZAAgAzEisAABad6IEAGoBwAeps+J06ik+NjEA1j9HjnAPtdd8LSPjLxuN2jFgAgCtO6/Z0Z8BgADAjMQKAKB1J0oAoDoAlu/7DXWVnUI9JZ9MqK7iumeV7e01X8nI+MtGo3YMmABA685rsvlzIBCgWbNm0YYNG0isb926lYqLi3UNfQAgAFCXw2gtDAC07kQJANQCgA8zAJ7IAPjpJAC4jsKOfOqoPlfrkDJdOdkCZroUCLnTpUlztSObXdetW0cLFiygTZs2UWFhIc2ePZuefvpp8ng8xPGXHA4HXXHFFXTjjTcmVTQAEACYkVEIAAQAZsSxTNhookm0fP/vqZuzf92lJyTscUnt0xRyFfGNIueYUCJtXZItYGqTSr0U5FbXkYwlZLPrkiVLaOPGjbRmzRpF3QIAg8Eg3X///ZrVDwAEAGp2Fj0FAYAAQD3+InPZRJNo2YFHqadoCnWXTU0MgIdXUdBdQZ2VZ0krumwBM12Khtzp0qS52pHJrgL21q5dS6FQiLxeL02dOpWqq6sBgGl0KUca27JdUwBAAKBdnD4xAD7Oj4D5GJ8GPjmhGkoPLadA/nAGwDOlVZNMATOdSobc6dSmedqSza6LFy+mzZs398sAPv88P13A6VReV3jaaafRddddR5WVlUmVjAwgMoAZGYEAQABgRhzLhI0mmkRLD/5BeQ1cV/lnkwDgH8nvGUtdFaeZUCJtXZItYGqTSr0U5FbXkYwl1OwqrqsLhLrTKlqe06tcqzeUz0AA/N///kcjRoxQMoF79uyhG264gTo6OuiZZ54BAA5BwUOzyhB+yIpVAIAAQCv6dSKZEgLgodSAV3ZwGfkKJiYFRBl0pxYwZZBhKH2E3EPRmvnrqNnVH+yiNe99P62CXPjxh8ntKhhSmwMBcGAj+/bto5NPPlm5SWTChMTvJEcGEBnAITmfWiUAIABQzUessj8xAD5FAc9IfhbgGQnFLDvwGF8jeDRfI3iStGpQC5jSCqbScchtTcuq2dXsGcCBVtm/f78CgC+99BJNnDgxodEAgADAjIxmACAAMCOOZcJGE02iJYdX8E0eNXyN3+cT9jhyl/BxfJfwiSaUSFuX1AKmtlbkKwW55bOZlh7LZtf4DGBPTw+98MILdPrpp1NJSQnt3btXefxLc3MzPfts5JmjiT4AQACglrGhuwwAEACo22kkrZAYAFczAJYxAH4xMQDue1h5REx36WcklZqUC81bW+Hn0hpQZ8etbm/Z5IsHwK6uLvrmN79J77//Pvn9fuXGj89//vP085//XLkmEACo09m5OK4B1K+z3hoAQARGA+4jVdWEAKjynL+Kfb+mzrLP8oOij5NK1vjOyhYw06VoyJ0uTZqrHTvaFRlAZAAzMgoBgADAjDiWCRtNNIkW14k3fXj4TR9fStjjir0P8PWBZ/K7gj9hQom0dcmOAVNoBnJr8w/ZStnRrgBAAGBGxikAEACYEccyYaOJAfDPfA7BRe3V5yUGwI/uU94C4uMbQWT92DFgAgCtO6/Z0Z8BgDYHwOnTp9/CzyG6htWwT5z2ZnD736pVqy6LqsUxY8YM8V4Z8cLSEJe7b8WKFQ9qCVgAQOtOlMnsb8cJNBkQFNX/lQdTkNprzk+orso9ixkOv8IAeJSW4WTKMrC3Kc2SsU5Z3d5Wly+RYwAAAYC3sArKGPp+NlAVDIeXM/RdtnLlyrMvueSSCr649E1+yvi5y5cvf1dtlgEAAgDVfMQq+xNNokX1zzEA9jAAfi0JAN5DbTUXkr9wkrRqsGPARAbQuvOaHf0ZAAgAvIWhrpwze7MHqoKzf88yAD7B+1aIfTNnzryL3z3Yw7A4Ty1qAQCtO1EiA9hfAwkBsIFfyRTsoLZhFyRUV9XuO6l1+Nf5bSHj1YaSaffbMWACAK07r9nRnwGAAEBxCvh7rIZ6XhoY3G5nwPunUAsD4FsMfFeuXr36tejfV/P+k3j/FWpRCQBo3YkSAKgOgIWNL5LL30Rtw6cnBsBdi6hl5GUU8I5VG0qm3W/HgAkAtO68Zkd/BgBaHAAZ4l5hEScPEFM84ibMy6cDgYA/Ly+vgU/zBvmU76kMg0+7XK4Tnnrqqb0DAZD3/5DrTAUApo7JdpxIEBj7B8bCxg0MgHUMgDMTOkv1zgXUPPp7/LaQUaYFPLWOwc/VNGSt/Va3t9XlS+SNAECLA6DeKYih7zmus5SB8Gm9p4D5FPE0zhhOE7/JD6LMX7p06TXt7e16uyB9+fz8fPL5fNLLoVcAyN2nMdehv5Ozay/5J3wnoRo9//k5+T72cwoXyAuAsLfeESJ3eavbmxMf1NTUJLeRdPa+oqKCgsFgwlrFxcV01VVXPdjY2KgEM75UbD1fDbZe509IW9wWD4JmyBvNsLdfWOmiiy46ko28kVdP423bGei+xadyL50yZcq0Dz74oEzcBMKA92U+JbxNzao4BWzdUyXJbG/HI+hkmc+C5s3k7tpDrSO/mVBdNTtupsaxP6VgfvKn9KuNsVzvh71zbYHs/r7V7W11+ZAB1DdebAGAfFr3MT7tK95HJQ4DArwsEtk/oar58+c7t23bdh+vioeZhXi5j/c9oEWNAEAAoBY/sUKZRIGjoPkVyu/aoVznl+hTs30ONYy7lkLuCmlVYMeAmQz4pTWijo5b3d6yyceXb9GsWbNow4YNJNa3bt1KImun54NTwMm1ZQsA1OMsesoCAAGAevxF5rIJAbDlVcrveI9aRl0xWLRwiIbtuInqx99IobxSaUWXLWCmS9GQO12aNFc7stl13bp1tGDBAtq0aRMVFhYqynzhhRdoyZIlxGfsyOPx0CmnnEIPP/xwUkUDAAGAGRmFAEAAYEYcy4SNJppEvS1byNPxNgNggmsAQ34atnMe1U2YS2FXkQkl0tYl2QKmNqnUS0FudR3JWEI2uwrQ27hxI61Zs0ZR97PPPktz5syhX/7yl3TGGWcQx2B655136JOf/CQAcAgOiQzgEJQWqwIABAAacB+pqiYEwNat5G17U7nTd+DHEeqmmp23MgDewgDolUrW+M7KFjDTpWjInS5Nmqsdmew6e/ZsWrt2LfE1+eT1emnq1KlK1u/qq6+mK664QrNikQFEBlCzs+gpCAAEAOrxF5nLJppEPQx/BZwFbB7z/cEAyA+Irtl1O9VOXMC31rmlFV2mgJlOJUPudGrTPG2p2pUzag4GrnR+wk4nv4B1aLmmxYsX0+bNm5UM4Pbt2+nMM8+kefPmkXhvQ21tLU2ePJmuvfZa5TRwsg8AEACYTn/ubQsACADMiGOZsNHEAPgWA+ArDIA/GNRjZ6CVqnffQbWTbufJ32VCibR1STVgamtGulKQWzqTaeqwml0d/LiUkW+/o6ktrYUOHnsMnwUY2hwQD4BbtmyhCy64gI488kh65JFHaOzYsfSHP/yBFi5cqNwkMmbMmIRdAgACALX6qq5yAEAAoC6HkbhwQgBs/x8VNr1ETWOvGQyA/IaQqj33UN2khUM++jeDutQCphn6mIk+QO5MaDX3bara1cQZwHfffZfOPvtsuvPOO+nSSy/tVebpp59OV155JV12WeKnEQAAAYAZGXkAQABgRhzLhI0mmkTzO96hIn4dXNPYHw/qsctXT5V772MAvM2E0mjvkmrA1N6UVCUht1Tm0txZ2ewanwEULx74xCc+oZwCBgBqNnnKgkM7MZ+e35a+FQAgAFB6J9YoQGIAfI+KG9ZT4xE/TQCAh6li70NUP+lWjb9gzmKyBcx0aRFyp0uT5mpHNrvGA6DQJD+3V7kr+NFHH1VO+T755JO0aNEievHFF2n06NEJlY0MIDKAGRmFAEAAYEYcy4SNJgTAzg+ouO4v1Dhu9qAe5/UcoPL9v6P6ifNMKI32LskWMLVLlrok5E6XJs3Vjmx2HQiA4pVuAvhWrVqlPBj6Yx/7mPJYmBNOOCGpogGAAMCMjEIAIAAwI45lwkYTTaLuzh1UUvc0A+AvBgNg914qO/gENUy4yYTSaO+SbAFTu2QAwEQasLq9rS6fHpvy28Fo5MiRokoZr9svmLHgOAVsYMYEANpvzNhxAhVDJCEAdu2i0sMrqWH8dYNGkbtrN+97ivfdYGCE5b4q7J17G2SzB1a3t9XlAwDq1NIODwAAIABJREFUGy0AQH366lcaAAgANOA+UlVNFDjyuvZQ2aE/cpbvxsEAyNnB0to1DIDXSiXnwM7aMWAmA36pDamx81a3t9XlAwBqdPRoMQCgPn0BAKMasONEgsDYH/jzuvfxad7HE57mzVeuD3yWTw//zMAIy31V+HnubZDNHljd3laXDwCob7QAAPXpCwAIAKTWVmQ+hRtEbvT4Pd/ocfOgUZTf8S4VNTxPTQnuEDYw5LJe1Y4BEwc61h3fdvRn3ASSfNoEABoIKTgFbN2JMplb2HECTQYErp5DVLH/NwyA8wcDYPs2KmrawM8I/JGBEZb7qrB37m2QzR5Y3d5Wlw8ZQH2jBQCoT1/IACIDiAxg1Adcvjp+2POv+GHP/L7fAR9P21tU2PIyNY252sAIy31VOwZMZACte2BrR39GBhAZwIxEEmQArTtRIgPYXwOJJlGXv4Eq9/yS6ibz+34HAeCb/J7gLfye4O9nZOxlq1E7BkwAoHXnNTv6MwAQAJiReAEAtO5ECQBUB0Cnv5nf93s3A+CiQerytr5B3rb/UvPo72Zk7GWrUTsGTACgdee1kpISEs+/G/hxuVwkHrJsxQ/HaWpraxskGp4DiOcAGvJ3AKB1J0oAoAYADLRS9e47qHbSQp5JnP0qeFteIw/fCNIy6gpDYyzXlQGAubZAdn8f9s6uvnP5awBAAKAh/wMAAgANOZBElRMFRkegnWp2L2QAvI1nkrx+0hQ0v0L5XTuoZeRlEkk5uKsAAqnNp7vzsLdulUlbAQAIADTkvABAAKAhB5KockIADHZRza4FVDvxViJnfn8AbNpE7p691DriEomkBADGNAAQktptdXfejvYGAAIAdQ+U+AoAQACgIQeSqHJCAAz1UM3O+VQ34RYKu7z9pCls+ic/J/AQA+DXJZISAAgALMVd/lKPWO2dBwACALV7S4KSAEAAoCEHkqhywgxByE/Dds5jAJzLAFjUHwAbXySXv4nahk+XSEoAIAAQACj1gNXReQAgAFCHuwwuCgAEABpyIIkqJwTAcJCG7ZhL9ePnUCivpJ804i0gzmA7tQ27UCIpAYAAQACg1ANWR+cBgABAHe4CAIzXgB2vGRHyQ+44L+DHKwzbMYcB8AYGwLL+AFj/N3KEfdRe8zVDYyzXlWHvXFsgu78Pe2dX37n8NQAgANCQ/yEDiAygIQeSqHKywFiz/SZqGPdzCrkr+0lTXPes8nd7zVckkhIZQGQAkQGUesDq6DwAEACow12QAUQGEBnAgaOgZsfN1Dj2pxTMrx4AgOso7MinjupzDY2xXFdGRijXFsju78Pe2dV3Ln8NAAgANOR/yAAiA2jIgSSqnCwwVu+4hZrGXsMAOKyfNCW1ayjkKqaOqnMkkhIZQGQAkQGUesDq6DwAEACow12QAUQGEBnAgaOgeucCahp9JQU9I/sD4OFVFHRXUGflWYbGWK4rIyOUawtk9/dh7+zqO5e/BgAEABryP2QAkQE05EASVU6aAdx1OzWP/DYFvKP7SVN66CkKeEZQZ8WZEkmJDCAygMgASj1gdXQeAAgA1OEuyAAiA4gM4MBRULVrkfK6t4B3bH8APPgk+b1HUFfFaYbGWK4rIyOUawtk9/dh7+zqO5e/BgAEABryP2QAkQE05EASVU4WGKt230Wtwy8mf8H4ftKUHVxGvoJJ1FV+qkRSIgOIDCAygFIPWB2dBwACAHW4CzKAyAAiAzgoA7j7HgbAixgAJ/YHwAOPUk/RMdRddpKhMZbrysgI5doC2f192Du7+s7lrwEAAYCG/A8ZQGQADTmQRJWTBcbKPf+P2vhhz/7Cyf2kKd//e+ou+SR1l54gkZTIACIDiAyg1ANWR+cBgABAHe6CDCAygMgADhwFlR8tofaqc8lX9LH+ALjvYeoqO5F6Sj5taIzlujIyQrm2QHZ/H/bOrr5z+WsAQACgIf9DBhAZQEMOJFHlZIGx4qP7+Vl/ZzMAHt1Pmoq9D1Fn+ecYAI+TSEpkAJEBRAZQ6gGro/MAQACgDndBBhAZQGQAB46Cir0PUEfF58lXPGUAACbebmjA5aAyMkI5UHoOfxL2zqHys/zTAEAAoCGXQwYQGUBDDiRR5aQZQJHp40e99BQf20+ayo/u41PD0zgz+HGJpEQGEBlAZAClHrA6Og8ABADqcBdkAJEBRAZw4Cgo37eUr/U7iU/1fqo/AO5ZTO01XyFf4VGGxliuKyMjlGsLZPf3Ye/s6juXvwYABAAa8j9kAJEBNORAElVOFhjL9/+W7/Y9nu/2/Uw/aarE42GG8eNhCvs/HkYikZWuAghks5ix/sLexvQnU20AIADQkL8CAAGAhhxIosrJATDx416qdt/Jzwf8+qAHREskMgCwFeNbNn8dan/tCL4AQADgUMeLUg8AiABhyIEkqpwsQJQpD3yewg98nto/A7hrIb8i7vJBr4iTSGQAIABQNncdcn8BgA77BTP2FseQPQYVAYAIELYZBUkB8OAT/Mq3I/mVb6f000X1zgXUPPp7FPCMklpHdgyMOPVtPxawo58jAwgANBSckAHERGnIgSSqnCxAlB78A5/mncAA+Nn+ALjjFmoa+0MK5g+XSMrBXbVjYAQAYl6TetBq7DwAEACo0VUSFwMAYqI05EASVU4KgIf+SH7PGOqqOL2fNDXb51LjEbMYAKslkhIAGNMAwFdqt9XdeTvaGwAIANQ9UOIrAAABgIYcSKLKyQFwOZ/mHc7PAjyzT5pwmGp23EQN466lkLtCIikBgABAPAdQ6gGro/MAQACgDncZXBQACAA05EASVU4GgCWHV1LQXUWdlV+IA8AgDdsxl+rH30ihvFKJpAQAAgABgFIPWB2dBwACAHW4CwAwXgN2PGUg5Ifc/cdBSe1qCrlKlfcB935Cfhq2cx7VTZhLYVeRoTGW68qwd64tkN3fh72zq+9c/hoAEABoyP+QAUQG0JADSVQ5WWAsrl3LkFfAADitVxpHsJtqdt1KdRNvobDTK5GUyAAiA4gMoNQDVkfnAYAAQB3uggwgMoDIAA4cBcV1z1DY4aaO6nPjALCdAXAh1U5cQOR0Gxpjua6MjFCuLZDd34e9s6vvXP4aABAAaMj/kAFEBtCQA0lUOWkGsO5ZRQrx3t/Yxxloperdd1DtpIU8wzglkhIZQGQAkQGUesDq6DwAEACow12QAUQGEBnAgaOgqP5v5Aj7GQC/2geA/iaq2nMP1U1eZGh8maEyMkJmsEL2+gB7Z0/Xuf4lACAA0JAPIgOIDKAhB5KocrLAWNSwnhzBLmof9n+90rh89VS59z6qm3SbRBIm7iqAQHoT6hIA9talLqkLAwABgIYcGAAIADTkQBJVTg6AfydnsI3ahl3YB4A9h6li36+pftJ8iSQEACLDjwy/9ANWhwAAQACgDnfBKWAECASIgaOgsPFFcvEp37bh03t35fUcoPL9v6P6ifMMjS8zVEZGyAxWyF4fYO/s6TrXvwQABAAa8kFkAJEBNORAElVOFhgLG/9Jef7D1Dr84j4A7N5LZQeXUcOEORJJiAwgDvBwgCf9gNUhAADQQgA4c+bM8xjIbmX7H8vfD61atepncb7gmDFjxv38t3hWRYgNf9+KFSsejO5PtS+lOwEAAYA65hupiyYDwIKmjeTu2U+tI77RK5+7azeVHl5ODeOvl1pm0XlkhKQ3oS4BYG9d6pK6MADQQgDIgDeZvbGAoWwGfxfHA+D06dMvZ2NftnLlyrMvueSSCr/f/6bT6Tx3+fLl76bap+bdAEAAoJqPWGV/UgBs3kzurj3UOvKbfQDYuYNK6tZQI78LWPYPgEB2C+rrP+ytT18ylwYAWggAY47IQHcLr5fFAyDD4bNs7Cc467dClONs4V2hUKiHy8xLtU/NuQGAAEA1H7HK/uQA+Arld22nlpGX94qa3/EBFdf/hQFwtvTiAwikN6EuAWBvXeqSujAA0D4A+BYD35WrV69+TXgsQ9/VDG8nMQBewetJ96l5NwAQAKjmI1bZnywwelteI0/Hu9Qy6oo4AHyXihqep6Yjfiq9+AAC6U2oSwDYW5e6pC4MAJQIABnUXmFvE6d54z8O/iPMy6f59O5+sSNJBrAf5HGZH3LRqYkAMH6fmncDAAGAaj5ilf1JAbB1C3na3qaW0d/pFdXT/j8qbPonNY39kfTiAwikN6EuAWBvXeqSujAAUCIA1OppmT4FzKePp3E2cZroT2VlZf7SpUuvaW9v19o9y5TLz88nn89nGXm0CgK5+2vK2fAquRpeI/9Rfdk+Z+MblFf3T/J97Bda1WracrC3aU2TkY7B3hlRq2kbLS4upquuuurBxsZGJZjxvQHr+Uqx9abtcJo7JjJolvoIAGQjlrMRey9AYmj7FmfrLp0yZcq0Dz74oEzcBMJCn8dZw3cS7WPA+zKfLt6mphhkAJEBVPMRq+xPlhnxtP2HCvg0cPOYq3pF9bb+m7ytb/C270svPjJC0ptQlwCwty51SV0YGUALZQD5FPEX2Bsf56WEFwG2LQxoP+TTvM/Onz/fuW3btvsE9PES4uU+hr8HhPem2qfm3QBAAKCaj1hlf3IAfIsKW16mpjFXxwHgFvK2vUXNo78rvfgAAulNqEsA2FuXuqQuDAC0EADmwhMBgADAXPhdLn4zWWDMb99GRU0b+l3vl+jGkFz0OR2/CSBIhxblaQP2lsdWRnsKAAQAGvIhACAA0JADSVQ5KQDyHcBFDX/nO35/0itNQfPL/GiYnfxomMskkjBxVwEE0ptQlwCwty51SV0YAAgANOTAAEAAoCEHkqhycgB8n4ob/kaNR8zqA0Dl7SD7+O0gl0gkIQAwXgMAIeldV5cAdrQ3ABAAqGuQDCwMAAQAGnIgiSonCxDuzu381o9n+KHPfW9eVN4P7OP3A4/oez+wRKL266odA6NQAOSW1WOH1m872hsACAAc2miJ1gIAAgANOZBElZMD4E4GwNX9XvtW2PgCufzN1DZ8ukQSIgOIDCDAV/oBq0MAACAAUIe7DC4KAAQAGnIgiSonBcCu3VR6eDk1jL++VxrxFhBnsIPahl0gkYQAQAAgAFD6AatDAAAgAFCHuwAAESAQIAaOgrzuj6js4JPUMOHGPgCs/xs5wn5qr/mqofFlhsp2PDWGU8A4sDXD2Mt0HwCAAEBDPoYMICZKQw4kUeVkIJTXvZ/KDz5G9RNu6pWmuO5ZZb295isSSYgMIA7wcIAn/YDVIQAAEACow12QAUSAQIAYlAHsOUjl+39L9RPnxQHgOgo7PNRR/SVD48sMlZEBNIMVstcH2Dt7us71LwEAAYCGfBAZQGQADTmQRJWTBUYX3+1bsffXVD9pfq80JbVrKOQqpo6qcySSEBlAHODhAE/6AatDAAAgAFCHuyADiACBADFwFLh8dVS591dUN2lBHwAeXklBdyV1Vp5laHyZoTIyQmawQvb6AHtnT9e5/iUAIADQkA8iA4gMoCEHkqhyssDo9DdS1Z7/R3WTF/ZKU3roKQp4RlJnxRkSSYgMIA7wcIAn/YDVIQAAEACow12QAUSAQIAYOAqc/Ly/6j13Ue2kRTybOJTdpXxXsN97BHVVnGZofJmhMjJCZrBC9voAe2dP17n+JQAgANCQDyIDiAygIQeSqHLSDGCglap338EAyBlAh1ORqOzgE+QrmExd5adKJCEygDjAwwGe9ANWhwAAQACgDndBBhABAgFi4Chw8AOfa3bdTrUT+RpApzsCgAcepZ6iY6i77CRD48sMlZERMoMVstcH2Dt7us71LwEAAYCGfBAZQGQADTmQRJWTBUZHsIsBcAHVTZxPYadHkah8/++ou+RT1F16gkQSIgOIAzwc4Ek/YHUIAAAEAOpwF2QAESAQIAZlAEM9VLNzPtVNmEdhV0EEAPctpa6yqdRT8mlD48sMlZERMoMVstcH2Dt7us71LwEAAYCGfBAZQGQADTmQRJWTBsZwgIbtuJkB8CYGwGJFooq9D1Fn+ecYAI+TSEJkAHGAhwM86QesDgEAgABAHe6CDCACBALEoFEQDjEA3kT142+kUF5pFAB/RR0VXyBf8RRD48sMlZERMoMVstcH2Dt7us71LwEAAYCGfBAZQGQADTmQRJWTZwDDDIBzqH7c9RRylysSVX50H7VXTSNf0cclkhAZQBzg4QBP+gGrQwAAIABQh7sgA4gAgQCRaMDUbJ9LDeN+xgBYGQHAPYupreZ88hceaWh8maEyMkJmsEL2+gB7Z0/Xuf4lACAA0JAPIgOIDKAhB5KocqrAWLNjHjWO/TEF82sUiap2302tw6YzAE6USEJkAHGAhwM86QesDgEAgABAHe6CDCACBAJEogFTvWM+NY25moKe4VEAvJNahn+DAgXjDI0vM1RGRsgMVsheH2Dv7Ok6178EAAQAGvJBZACRATTkQBJVThUYq3cuoObRVyrv/xWf6l0LqXnktyjgHSORhMgA4gAPB3jSD1gdAgAAAYA63AUZQAQIBIiEGUAF+K5g4BsdAcABQGhokOW4MjJCOTZAln8e9s6ywnP4cwBAAKAh90MGEBlAQw4kUeVUgbFq1x3UMvKbDIBHRABwxy3UNPYaviZwmEQSIgOIAzwc4Ek/YHUIAAAEAOpwF2QAESAQIBINmKrdd1Hr8IvJXzBe2S3uCm48YhYDYLWh8WWGysgImcEK2esD7J09Xef6lwCAAEBDPogMIDKAhhxIosqpAmPlnnv4sS8X8l2/k4jCg58LKJGYg7oKIJDZevr7Dnvr15msNQCAAEBDvgsABAAaciCJKqcGwLjn/oWD/GDoufxmkDn8ZpASiSTEKWBk+JHhl37A6hAAAAgA1OEuOAWMAIEAkWjAVH60hN/8cS6/+eNjRCEfDdt5C78beC6/G7jI0PgyQ2VkhMxghez1AfbOnq5z/UsAQACgIR9EBhAZQEMOJFHlVIGx4qP7qaPqiwyAx5Aj2EU1uxZQ3cT5FHZ6JJIQGUAc4OEAT/oBq0MAACAAUIe7IAOIAIEAkWjAVOx9kDorzqCe4k8wALYzAC6k2okLiJxuQ+PLDJWRETKDFbLXB9g7e7rO9S8BAAGAhnwQGUBkAA05kESVU2YA9/2aOss+Sz0lx5Ez0ELVu++k2kkLeXZxSiQhMoA4wMMBnvQDVocAAEAAoA53QQYQAQIBItGAKd+3lLrKpjIAfpqc/kaq2nMv1U1eZGhsmaUyMkJmsUR2+gF7Z0fPZvgVACAA0JAfIgOIDKAhB5KocqrAWL7/d9TN8Nddejy5fHVUufd+qpt0m0TSJe8qgMASZtQsBOytWVXSFwQAAgANOTEAEABoyIEkqpwqMJbtf4Szf8cyAJ5Irp5DVLH/N1TPN4FY4QMgsIIVtcsAe2vXlewlAYAAQEM+DAAEABpyIIkqpwTAA49RT9HR1F12EuX1HKDy/b9nALxZIumQARyoAYCQJdxXsxB2tDcAEACoeYAkKggABAAaciCJKqcEwIPLyFcwibrKT6W87o+o7OAfqGHCHImkAwACACMasCMI2VVuACAA0FCQAgACAA05kESVUwXG0oNP8nuAxzEAfo7cXbup9PByahh/vUTSAQABgADA1lZ7zecAQACgoSAFALTXhGHXI2U1uUsP/YkCntH8LMDTyd25g0rqnqbGcb8wNLbMUhkZIbNYIjv9gL2zo2cz/AoAEABoyA8BgABAQw4kUeWUGUDO+AXcw6mz8kzK73ifiuv/ygA4WyLpkAFEBhAZQGQALTFl6RLCoas0CvfTAAAQAGiXIZEKAEsOr6Kgu4IB8CwGwHepqOHv1HTETyyhGmSELGFGzULA3ppVJX1BZACRATTkxABAAKAhB5KockoArF1DIVcJvw/4bPK0/48Km16iprHXSCQdMoDIACIDiAygJaYsXUIgA6hLXf0LAwABgAbcR6qqqQCwuHYthV0FDIDTyNP2Xypo+Rc1j/mBVPIl6ywyQpYwo2YhYG/NqpK+IDKAyAAacmIAIADQkANJVDklANY9Q2FHHnVUn0fe1n+Tt20rNY++UiLpkAFEBhAZQGQALTFl6RICGUBd6kIGMKYBHCkbcBwJq6YEwPq/EIXD1F7zFfK2bCFv+1sMgN+VUMrBXYafW8KMmoWAvTWrSvqCyAAiA2jIiZEBRAbQkANJVDlVYCyq/xs5wn4GwK/y6d9X+UaQ96hl1BUSSYcMIDKAyAAiA2iJKUuXEMgA6lIXMoDIAJaS3SZKYfOUANjwPDmDHdQ27AIqaH6ZHwa9i1pHXmpgZJmnKjJC5rFFNnoCe2dDy+b4DWQAkQE05InIACIDaMiBJKqcGgBfYABsYQC8iAqaNpK7Zx+1jrhEIumQAUQGEBlAux3YAgABgIaCFAAQAGjIgSSqnAoACxv/QS5/A7UNn0GFjRsoz19LrcMvlkg6ACAAEAAIALTElKVLCJwC1qUunALGKWCcAh44ZMRz//J6DnLW7+sMgC8wDDYzDE43MLLMUxWnBM1ji2z0BPbOhpbN8RvIACIDaMgTkQFEBtCQA0lUOVVgLGjaxKd99yqnfYsa1vPp4E7lekArfAAEVrCidhlgb+26kr0kABAAaMiHAYAAQEMOJFHllAAYd+NHEb8H2BEOKHcEW+EDILCCFbXLAHtr15XsJQGAAEBDPgwABAAaciCJKqcEQH7zR37Hh/zol8upuO7PPKs4qb36yxJJl7yrAAJLmFGzELC3ZlVJXxAACAA05MQAQACgIQeSqHKqwOhteZ08HdsYAL9NymvhnF5+K8iXJJIOADhQAwAhS7ivZiHsaG8AIABQ8wBJVBAACAA05EASVU4JgK1v8Ovf/qu8/aOkdg2FXCX8XuCzJZIOAAgAjGjAjiBkV7kBgBYCwJkzZ57HQHYrO/Ox/P3QqlWrfhab1KZPn34LG/sa/nsfLw7e/z/ef1l0v2PGjBn38/q5vIS43H0rVqx4UEv0AgACALX4iRXKpAbAvvf/lhxeQUF3NXVWfsEKYgMILGFF7UIAALXrSvaSAEALASBD3GR2yAKGshn8XTwQAHlbWfy2ODi8nB3hspUrV559ySWXVPj9/jedTue5y5cvf1fNwQGAAEA1H7HK/lSB0dP2H34F3GvUPOYqKj30Jwp4RlFnxRmWEB1AYAkzahYC9tasKukLAgAtBIDx2b6BsCcygAx15ZzZmz3Qaxkcn2VHeIL3rRD7OJN4VygU6mFYnKfm4QBAAKCaj1hlf0oAbH+bCvlRME1jf0ilB58kf8E46ir/nCVEBxBYwoyahYC9NatK+oIAQBsBIBv7e+yx9bw0MLjdzoD3T+HBDIBvMfBduXr16teif1/N+0/i/VeoeTgAEACo5iNW2Z8qMOa3b6Oipg0MgD+isgNPkK/wSAbAUywhOoDAEmbULATsrVlV0hcEAEoEgAxqr7DHidO88R/xJpMwL5/mU7j7xQ6R7eOvfqd7L7jggmF5eXkNXCbI+09lwz/tcrlOeOqpp/YOBEDe/0OuPxUAmHp8Y6KUfv7TJUBKAOx4lx8A/XdqOuInDICPUk/RFOoum6qrfbMWhp+b1TKZ6RfsnRm9mrFVAKBEAKjVgRIB4MC6DH3P8balDIRP6z0FzKeIp3HGcJpos7KyMn/p0qXXtLe3a+2eZcrl5+eTz+ezjDxaBYHcgzXlbH2H8vatJt8xN5P7w/soVDmVglXWyADC3lpHhjXKwd7WsKNWKYqLi+mqq656sLGxUQlmfKnYer4abL3W+rKXs9y7gBNd78eQNzqWIbzooouOZCNvZMOdxtu2M9B9i0/lXjplypRpH3zwQZm4CYQB78t8SnibmnFxChingNV8xCr7U2VG3J3bqaRuHTWO+zmV71tKXZz96yn5tCVER0bIEmbULATsrVlV0hdEBtBCGUCGPPHcicd5KeFFgG0LA9oP+VTuswyFj7GxP8PbgrwEeFkksn/Cg+fPn+/ctm3bfbx6Hi8hXu7jfQ9o8W4AIABQi59YoUxKAOzaSaWHV1HD+OuoYu9DfAfwadRTfKwVxMZjYCxhRe1CAAC160r2kgBACwFgLpwRAAgAzIXf5eI3UwXGvK49VHb4TwyANzAA/oo6Kr5AvuIpuehm2n8TQJB2lZq6Qdjb1OZJa+cAgABAQw4FAAQAGnIgiSqnBMDuvVR2cBk1TJhDlR8tofaqL5Gv6OMSSZe8qwACS5hRsxCwt2ZVSV8QAAgANOTEAEAAoCEHkqhyagDcT+V892/9xLlUuef/UVvN18hfOPCGfYmEjesqgEBOuw2117D3UDUnXz0AIADQkNcCAAGAhhxIosqpAqOr5xBV7F/KAHgLVe2+m1qHT+eHQU+USDpkAAdqACBkCffVLIQd7Q0ABABqHiCJCgIAAYCGHEiiyikB0FfL1/49SPWTbqWqXXdQy4hLKMBvA7HCx46BUdgNclvBe7XLYEd7AwABgNpHSIKSAEAAoCEHkqhyagCsp8q991HdpNuoetdCah75LQp4x0gkHTKAyABGNGBHELKr3ABAAKChIAUABAAaciCJKqcKjE5/E1XtuZfqJi+k6p23UtPo71PQM1Ii6QCAAEAAYGurveZzACAA0FCQAgDaa8Kw65GymtzOQAtV776Taictopqdt1AjvxM4mD/M0NgyS2VkhMxiiez0A/bOjp7N8CsAQACgIT8EAAIADTmQRJVTBUZHoI1qdi9iALydanYwAI6bTUF3lUTSIQOIDCAygMgAWmLK0iWE5V4Fp0t6g4UBgABAgy4kTfWUABjsoJpdt1PtxFtpGGcA68ddTyF3uTSypeooMkKWMKNmIWBvzaqSviAygMgAGnJiACAA0JADSVQ5NQB2MwDeSnUT5iogWD9+DoXyxBsZ5f8ACOS3oR4JYG892pK7LAAQAGjIgwGAAEBDDiRR5ZSBMeTrzfxV77mLQfBmCrsKJZIOp4BxChingHEK2BJTli4hcApYl7r6FwYAAgANuI/c8B41AAAfLklEQVRUVVMCYDhAw3bcTA1H/IyqPlpMdRPnU9jpkUq+ZJ1FRsgSZtQsBOytWVXSF0QGEBlAQ04MAAQAGnIgiSqnBsAQA+BNfPfvj/l5gL/im0Fu45klTyLpkAFEBhAZQGQALTFl6RICGUBd6kIGMKYBHCkbcBwJq6rZu2b7HGrm5/+JV8LVTlrIAOiUUMrBXVaT2xJCJhACclvVsonlsqO9kQFEBtDQKEcGEBlAQw4kUWW1AFGzfS41j7qCyg88qjwQ2iofNbmtIicygMgAIgNo1dGcXC5kAA3YHAAIADTgPlJVVQOhmh3z+B3AX6eyQ0/xK+EWSCVbqs6qyW0ZQQcIArmtallkAGMaQAYQGUBDoxwACAA05EASVVYDAvEKuLaa/6OSuqepnm8CscpHTW6ryIkMIDKAyABadTQjA5gRywIAAYAZcSwTNqoGQtU7b6P26nOpuP45BsC5JpRgaF1Sk3torZq/FuQ2v43S2UM72hsZQGQADY0hACAA0JADSVRZLUBU7VpEnRVnUmHTP6lhwhyJJEvdVTW5LSMoTgErGoC9rerRg+UCAAIADXk7ABAAaMiBJKqsFhirdt9JXWWnUEHLq9Qw/nqJJAMAJtKAmr0tY2CAr23BFwAIADQ0jwEAAYCGHEiiympAULX7buou+Qx52v9DjeN+IZFkAEAAYJ8G1PzcMo4N8CUAIADQ0HgGAAIADTmQRJXVAmPlnnupp2gKeTrfo8YjZkskGQAQAAgAVBvflhnQcYIAAAGAhvwaAAgANORAElVWCxCVe35JvsKJ5O7aQ01H/EQiyQCAAEAAoNr4tsyABgD2MyWeA2jAswGAAEAD7iNVVbUAUfnRfeT3jKY832FqGnuNVLKl6qya3JYRFKcEFQ3A3lb16MFyIQOIDKAhbwcAAgANOZBEldUCYwW/AzjoriZnoIWax/xAIsmQAUQGEBlAtfFtmQGNDCAygOlyZgAgADBdvmT2dtQCRMXeBynkKiFHuIffCXyl2cXR3D81uTU3JFlByC2ZwQx21472RgYQGUBDwwYACAA05EASVVYLEOX7fsOzSR6FyUkto78jkWTIACIDiAyg2vi2zIBGBhAZwHQ5MwAQAJguXzJ7O2oBonzfw+SgIIWchdQy6ltmF0dz/9Tk1tyQZAUht2QGM9hdO9obGUBkAA0NGwAgANCQA0lUWS1AlO//HTmCnXwdYCW1jrxUIsmQAUQGEBlAtfFtmQGNDCAygOlyZgAgADBdvmT2dtQCRNmBR8nlb6AA3wncOuIbZhdHc//U5NbckGQFIbdkBjPYXTvaGxlAZAANDRsAIADQkANJVFktQJQdeJzyeg6Qn58F2Dr8YokkQwYQGUBkANXGt2UGNDKAyACmy5kBgADAdPmS2dtRCxBlB5fxQ6B3KW8DaRt+kdnF0dw/Nbk1NyRZQcgtmcEMdteO9kYGEBlAQ8MGAAgANORAElVWCxClh/5Ino53qavkeGof9n8SSYYMIDKAyACqjW/LDGhkAJEBTJczAwABgOnyJbO3oxYgSg89Rd72/1Jn2SnUXvNVs4ujuX9qcmtuSLKCkFsygxnsrh3tjQwgMoCGhg0AEABoyIEkqqwWIEoOr6CCtjeps/w0aq8+TyLJkAFEBhAZQLXxbZkBjQwgMoDpcmYAIAAwXb5k9nbUAkTJ4dUMgG9QR/kZ1FH9JbOLo7l/anJrbkiygpBbMoMZ7K4d7Y0MIDKAhoYNABAAaMiBJKqsFiBKap+mgtbXqaPiC9RRdbZEkiEDiAwgMoBq49syAxoZQGQA0+XMAEAAYLp8yeztqAWI4rp1VNjyKrVXnkOdlZ83uzia+6cmt+aGJCsIuSUzmMHu2tHeyAAiA2ho2AAAAYCGHEiiymoBorjuzwyAr1B71Zeos+IMiSRDBhAZQGQA1ca3ZQY0MoDIAKbLmQGAAMB0+ZLZ21ELEMX1f6XC5k3UVv1l6ir/nNnF0dw/Nbk1NyRZQcgtmcEMdteO9kYGEBlAQ8MGAAgANORAElVWCxBF9c9RUfNLDIBfZQA8RSLJkAFEBhAZQLXxbZkBjQwgMoDpcmYAIAAwXb5k9nbUAkRRw/NU1LSBWmsuoO6yqWYXR3P/1OTW3JBkBSG3ZAYz2F072hsZQGQADQ0bACAA0JADSVRZLUAUNr5AxY0vUuuw6dRderxEkiEDiAwgMoBq49syAxoZQGQA0+XMAEAAYLp8yeztqAWIwsYNDIDPU8vwi6mn5FNmF0dz/9Tk1tyQZAUht2QGM9hdO9obGUBkAA0NGwAgANCQA0lUWS1AFDa9RMUNz1HLiEuop/hYiSRDBhAZQGQA1ca3ZQY0MoDIAKbLmQGAAMB0+ZLZ21ELEAVNm6ik4a/UPOIy8hUfY3ZxNPdPTW7NDUlWEHJLZjCD3bWjvZEBRAbQ0LABAAIADTmQRJXVAkRB8ytUUv9nah55BfmKPiaRZMgAIgOIDKDa+LbMgEYGEBnAdDkzABAAmC5fMns7agGigN8CUsJvA2ka9V3yF042uzia+6cmt+aGJCsIuSUzmMHu2tHeyAAiA2ho2AAAAYCGHEiiymoBwtvyOpXWPU1No79P/oIJEkmGDCAygMgAqo1vywxoZACRAUyXMwMAAYDp8iWzt6MWILytW6m0dhU1jv4BBQrGmV0czf1Tk1tzQ5IVhNySGcxgd+1ob2QAkQE0NGwAgABAQw4kUWW1AOFpe5PKDq+gxjE/ooB3tESSIQOIDCAygGrj2zIDGhlAZADT5cwAQABgunzJ7O2oBQhP238ZAJ+ihrE/paBnhNnF0dw/Nbk1NyRZQcgtmcEMdteO9kYGEBlAQ8MGAAgANORAElVWCxCe9v9R2aEnqeGI2RTMHyaRZMgAIgOIDKDa+LbMgEYGEBnAdDkzABAAmC5fMns7agEiv/0dKj+0jOrHXUshd6XZxdHcPzW5NTckWUHILZnBDHbXjvZGBhAZQEPDBgAIADTkQBJVVgsQ+R3vUfnBxxkAr2cALJdIMmQAkQFEBlBtfFtmQCMDaM0M4PTp03/MRP99li7EC7NZ+J5Vq1Y9GZOW98/l/VeIfbwsX7ly5Vwt+1I5PgAQAGjFiXEoQJDf+QGVH3iU6sbPoXBeiWXUYsfAKIwHuS3jwpoEsaO9kQG0UAZw5syZn+/u7n7jmWeeabvwwgvHuFyuN9nzpzLo7eJ9pzOsPVhUVHRiR0eHAMSX+e95DIh/S7VPbeQAAAGAaj5ilf1qAcLduYMqDvyO6ibMo7CrwCpiA4QsY0ltgqj5ubZW5CtlR7kBgBYCwIFDbsaMGW+zga9ZsWLFRs7+PcD7P2Lgu1uU431X89cpDIeXp9qnNowBgABANR+xyn61AOHu2kUV+x+muonzKez0WEVsAKBlLKlNEDU/19aKfKXsKDcA0KIAyFm9LzKcPcrD8CiGvC4Gvmd4/UleXy6GJkPfuWz86/nvM1PtUxvGAEAAoJqPWGW/WoDI69pDlft/Q7WTbuNZJc8qYgMALWNJbYKo+bm2VuQrZUe5AYASASCD2is8rAa+ZNTB28Q1fZ9mmNsfze4dy99/4eVi3vav6LZ+AMiAeB7D23WJADB+n9owBgACANV8xCr71QJEXvdeqtz3EAPgIp5VxLC0xkdNbmtIOVgKyG1VyyaWy472BgBKBIBahiND4jFR+Psuw90/YnXSeQqYAXFaKBSaJtqurKzMX7p06TXt7e1aumepMvn5+eTz+SwlkxZhIHdiLTk691L+e3dTz2d+pUWN0pSBvaUxVVo6CnunRY3SNFJcXExXXXXVg42NjUowczqd6/mysfXSCGCwo5Y5VL/44ouPZjD7K+vj+wx/f4/XC4PhGfy3uA5wKsNaiI2+OXYTSKJ9fGRwCzuBaCvlBxlAZADVfMQq+9UyBK6eQ0oGsG7SAquIrMihJrelhI0TBnJb1bKJ5bKjvZEBtFAGkEHueXbt43nZw0vs1LC4zk+BwehjYL7Nq+KU8VMJHgOTcF+qaQAACAC0S5hQDRDhAHnb/kPdpSdYSiWqcltK2j5hILdFDZtELDvaGwBoIQDMxXAFAAIAc+F3ufhNOwYIZAAxvnMx1nLxm3Yc3wBAAKChsQYARID4/+2dC4xdVRWG7cMUmDZVfFAVLYI2AmkCVWqIitRiSIG29A2NhSKx2DaESGoRQnhI1GAbscqr+GAKaei70gAFGtHwFEJCAlYFBCxSRRQjtoWppa3/JmfIcJ2Zs69rCnft9U1yMvex9529/rX+s76559xzTQXkaHLEBgEA4m9HFjUtNaK/AUAA0GQaAJAGYSogR5MjNggAEH87sqhpqRH9DQACgCbTAIA0CFMBOZocsUEAgPjbkUVNS43obwAQADSZBgCkQZgKyNHkiA0CAMTfjixqWmpEfwOAAKDJNAAgDcJUQI4mR2wQACD+dmRR01Ij+hsABABNpgEAaRCmAnI0OWKDAADxtyOLmpYa0d8AIABoMg0ASIMwFZCjyREbBACIvx1Z1LTUiP4GAAFAk2kAQBqEqYAcTY7YIABA/O3IoqalRvQ3AAgAmkwDANIgTAXkaHLEBgEA4m9HFjUtNaK/AUAA0GQaAJAGYSogR5MjNggAEH87sqhpqRH9DQACgCbTAIA0CFMBOZocsUEAgPjbkUVNS43obwAQADSZBgCkQZgKyNHkiA0CAMTfjixqWmpEfwOAAKDJNAAgDcJUQI4mR2wQACD+dmRR01Ij+hsABABNpgEAaRCmAnI0OWKDAADxtyOLmpYa0d8AIABoMg0ASIMwFZCjyREbBACIvx1Z1LTUiP4GAAFAk2kAQBqEqYAcTY7YIABA/O3IoqalRvQ3AAgAmkwDANIgTAXkaHLEBgEA4m9HFjUtNaK/AUAA0GQaAJAGYSogR5MjNggAEH87sqhpqRH9DQACgCbTAIA0CFMBOZocsUEAgPjbkUVNS43obwAQADSZBgCkQZgKyNHkiA0CAMTfjixqWmpEfwOAAKDJNAAgDcJUQI4mR2wQACD+dmRR01Ij+hsABABNpgEAaRCmAnI0OWKDAADxtyOLmpYa0d8AIABoMg0ASIMwFZCjyREbBACIvx1Z1LTUiP4GAAFAk2kAQBqEqYAcTY7YIABA/O3IoqalRvQ3AAgAmkwDANIgTAXkaHLEBgEA4m9HFjUtNaK/AUAA0GQaAJAGYSogR5MjNggAEH87sqhpqRH9DQACgCbTAIA0CFMBOZocsUEAgPjbkUVNS43obwAQADSZBgCkQZgKyNHkiA0CAMTfjixqWmpEfwOAAKDJNAAgDcJUQI4mR2wQACD+dmRR01Ij+hsABABNpgEAaRCmAnI0OWKDAADxtyOLmpYa0d8AIABoMg0ASIMwFZCjyREbBACIvx1Z1LTUiP4GAAFAk2kAQBqEqYAcTY7YIABA/O3IoqalRvQ3AAgAmkwDANIgTAXkaHLEBgEA4m9HFjUtNaK/AUAA0GQaAJAGYSogR5MjNggAEH87sqhpqRH9DQACgCbTAIA0CFMBOZocsUEAgPjbkUVNS43obwAQADSZBgCkQZgKyNHkiA0CAMTfjixqWmpEfwOAAKDJNAAgDcJUQI4mR2wQACD+dmRR01Ij+hsABABNpgEAaRCmAnI0OWKDAADxtyOLmpYa0d8AIABoMg0ASIMwFZCjyREbBACIvx1Z1LTUiP4GAAFAk2kAQBqEqYAcTY7YIABA/O3IoqalRvQ3AAgAmkwDANIgTAXkaHLEBgEA4m9HFjUtNaK/AUAA0GQaAJAGYSogR5MjNggAEH87sqhpqRH9DQACgCbTAIA0CFMBOZocsUEAgPjbkUVNS43obwAQADSZBgCkQZgKyNHkiA0CAMTfjixqWmpEfwOAAKDJNAAgDcJUQI4mR2wQACD+dmRR01Ij+hsABABNpgEAaRCmAnI0OWKDAADxtyOLmpYa0d8AIABoMg0ASIMwFZCjyREbBACIvx1Z1LTUiP4GAAFAk2kAQBqEqYAcTY7YIABA/O3IoqalRvQ3AAgAmkwDANIgTAXkaHLEBgEA4m9HFjUtNaK/AUAA0GQaAJAGYSogR5MjNggAEH87sqhpqRH9DQACgCbTAIA0CFMBOZocsUEAgPjbkUVNS43obwAQADSZBgCkQZgKyNHkiA0CAMTfjixqWmpEfwOAAKDJNAAgDcJUQI4mR2wQACD+dmRR01Ij+hsABABNpgEAaRCmAnI0OWKDAADxtyOLmpYa0d8AIABoMg0ASIMwFZCjyREbBACIvx1Z1LTUiP4GAAsCwKlTp56rhM6RC/ZoE5vtXbRmzZrlyRXTp08/U/eX6Oaz2vpp++fq1avHdjpGcy/W3NlpnraVeu7iHDcBgDSInDopYUzEBgEA4u8SvJsTQ0R/A4AFAaAgb0xHR8ejGzZs2DZ58uSDBwwY8JgKf7Rg7rkKACfq9uRGM+i54wRy17S1tR2zY8eOBI8P6P4lgseNdcYBAGkQdTVSyvMRGwQAiL9L8W9dHBH9DQAWBICNBT5t2rQnlOD5q1aturcCwFMFgJMax+ndv6v12PMCvu+n5zRvrn4dq7Fn1JkGAKRB1NVIKc9HbBAAIP4uxb91cUT0NwBYKAAK+E4QnN2ooh8hkHutAsBFuv+Ctle1/VCPr6mAb4N+L9f9lem+gHCcCuMC3T++zjQAIA2irkZKeT5igwAA8Xcp/q2LI6K/AUBHAKh35h5UEX+ioZDT+XzpvL2jBWxbK6Abqd+3a5uhxx6qHjtQh3hfbW9v75gyZcqn+vfvf3diPT3/iF73LQAoWDxJYLewGQB88cUX00mHdR4r6vkhQ4a8a9u2bUXFlBMMceeoVM4Y8l1OLnMiId85KpUxJgHgsGHDUjBDdTvefzsKPAFUMT+CuSMq+DtbAHdPT4EJ8q7Xc0/q8PBVzR4C1twT9+zZc2J67eHDh7ctXrw4ffCEHxRAARRAARRAAWcKLFiw4IYtW7bsSMvWm0N3iQvuchbC/73cYgBwxowZhwvM7pAScwR/m7oqctppp314xYoVf0mPnX766Qe9/vrr9+kduzk67+/XgsYv6uF0HuDo7du37xk8ePD9+m/gUhVBeq1ef/Qa/VQ81wsCv1k3trTnzznnnO8uXbr0otLiqouHuOsUKut58l1WPuuiId91CpX1vPr3IvXvr6vnxzqEV6WxGAAUyKXDup/WtkVb56HhdC7fJr3L9x0leKIe/0+CfIHbdYK/pZ2lXF0G5izdT0WwIvcyMGm+5v5Ar3V+Wbaoj4a46zUqaQT5Limb9bGQ73qNShpBvkvKZn4sxQBgfsh9OxLj9K2erf5q5LvVM9S36yPffatnq78a+W71DPXt+qLmu1NFANBYT+mcwEjnDHTKRdzGwnE2nXw7S5hxueTbKKCz6eTbWcL6aLkAYB8JycugAAqgAAqgAAqggBcFAEAvmWKdKIACKIACKIACKNBHCgCAfSQkL4MCKIACKIACKIACXhQAAL1kinWiAAqgAAqgAAqgQB8pAAA2KaQ+NXSuLimTLv68R5uuKLN3kS4Ds7y7l9GlaT6gx2/Sdpi2Do2dr7H3NfknW2J49Q0pl2sxIxXHtb1d+kZx/0njXksxVxp9T+NXt0QgTS6iybjTN9Us0/Z+bf/SRUVnr1y58vdN/slWGd5PefyRFjMu1bpqfok+7HRND3XuOt+KMytvGne24r9AW9pvpgvNz9Mlo3a3SsKaXUdO3NV1Ujfqtf9Qxb1X10s9duPGjTub/XutMF777yWq5Qlay3BdN/aotWvXPt5DTReV65y4S8t1yuu4ceMG6dq+K3Tz8KonvaT98jztl59pzLs0OkW1kb4ydoC2VBez5e/trVC3+2oNAGCTygoIxnR0dDy6YcOGbZMnTz54wIABj+klRqtQnuumoH6mgtqi576tr6D7jApvvcYc4rFpVM1if8HfNMUwuAYAn9WYiYrziSblbbnhTcb9SwXQrrhvVp1MkVbpOpSjWy6ojAVpZ3iGaneW1v/lmTNnvnfXrl2PqX7HdQe00sh1vrX+2rzpYvKH7N69+wFJd5Q0+bvm3Krbd+r2dRlytuSQnLgrKLhKcY5qySCaXJTi+bz22c8ql/cLAE/tDgALzXVt3KXlugsAjlH93pnua788X/vl9DWwY7qWzqxZs9rU159RTXxBNfG0tPhxAkaNW9hkibkaDgAa06VCeUKNcr7eHbm38aX03DZ968hh69evfyk9p/u/0a+LevuaOuNy9vl0gcGl+iNDawDwORlpYk//Xe/zRe6DP1AXd/Vu79NHHnnkgZdddll6dzjl+6/S4XPSIQGSqx+t/TbV9U2q61XVjvNKxbJTeb+kmzp3m+/cvKlxLFDjOFTenZfiVz2Mkz4X6v5xrhJbLTY37goKfqg4j/YYZ09rVlw91mxpue6qQW9xl5rrhvjTl0WonFcf2vD4VN3/qh4/KT1efbPY3br/0ZLqvjEWANCQXe0oTlBTuFEvMUKFkg55vvkjMx2oO1v1+P6dD6pprFTT2KjH2g1/9h2dWgdCaXFpJ6Nfr2hTuP0e0X/cF95yyy3/eEcXbvzjdXFrhzFKgLRcuU2HGt74kQ4Pp3cB01cOGv/82z5da39c8XxN8PpwFctcxfJZxTK7cTGe852bN3k9HQ7fKh6+MsVfNYjk5UPe9uT0wR/MjbuCgg36k09rS4e70zvcbt/17OLN3gCwqFx3LZcMACwu113jl4+Xab/2cuMbGNLlfO3fPqnH51b7u9S3/61/6Ad1/kPfB7ZruZcAABtSokJ4UA+lc4K6/nR+tdzR2vltrQpkpH7fnnqBHnuom6aYAPAFPXdA53MqvlUqsjtaEQBz464DoRRrOjS+bt26F/SaAxRv+hq+kYr55Jarfi2or+LuoaE+ovgXtiIA9ha3Do+NErTf0RUAlff0ztfo7gDQU74bazA3b40AKP2O0GslL5cEgP9Tr4pz8M6dO/ulU150+yMpZm1XKO41rejn3DXVvAP4FgD0nusmALDIXHfpvxdpf3xyW1vb2Pb29nR++ps/AGCucxiXoCHt/BP8nd3b4dxuDgGnd1PSYaN0ArnLnxwAbDDWMN1/UjEPdRlwtei6uHMPqXnRoJlDwJ7znZu30g4L5sbdWK+a9y010Q/pH4HzvNRyd+vkEHD3H35p8HIRuU4xVf6drn9mxqZ/Zrqp63QIOPXz9KG3zh6fzvH9mOc6r1s77wDWKdTwfHXoJ/0XPEfFsam36Sq6n+tdlC3aWV6uecfo9jqNd/khkM44EwjpwwDv0aGwb3QX+/jx4w/Yb7/93i1t0iHgZKTz9WuC7h/fpNQtNbwu7irWe/Ru5zJps0xxpx3KQsXt8kMgqt0z1ei/okMgJz711FND04dAVL8n65Dw5q6JKSHfylVt3jTm44r7Pp3TO0rn9KYPgfxC+twlb1/bUoXaxGIy4x6mGv6bXnbvhAkThgwaNCh9IvinrXgUo4nQ3zhNpafzlEvMdac2NXGXmuvUg2ZqG9vZl7oBwMF67I/qbcfpg25PSSc+BNKMoaKMVWHcrVjTiaRbtHUeGk6f9tyk59Ljl+v2KUmPSZMmfXDgwIE362ZqHjt7+rCIB+0U25e0znSJkyFV3K+oAc5TA7xNsDBeO9Pxuj2n2nmu1Zj+1bj0AYjzpMnzHuLsZseQFXeaJ8gfIR3adfN92l7R7bMagcmLBjrvpf/mzZuXaL3ppOj0oZYlyuHVaf2l5bsxbwoxXf7hd4rzJ8rhranGU9yq7XRpkAu1pcs//Up+nqtxbi8DkxN39anJdF7ULm0DFXP6XNAVXuq4cZ2K5/p0GFCPH6TtZW3blMMRpec6J+7Scl15Np228Gdt6bIv6Z2/1LM7lPNj5ed0WbN0nv4NaWx1GZjFupl612/1buGZ3b1b6LX2u1s37wCWlE1iQQEUQAEUQAEUQIEMBQDADJEYggIogAIogAIogAIlKQAAlpRNYkEBFEABFEABFECBDAUAwAyRGIICKIACKIACKIACJSkAAJaUTWJBARRAARRAARRAgQwFAMAMkRiCAiiAAiiAAiiAAiUpAACWlE1iQQEUQAEUQAEUQIEMBQDADJEYggIogAIogAIogAIlKQAAlpRNYkEBFEABFEABFECBDAUAwAyRGIICKIACKIACKIACJSkAAJaUTWJBARRAARRAARRAgQwFAMAMkRiCAiiAAiiAAiiAAiUpAACWlE1iQQEUQAEUQAEUQIEMBQDADJEYggIogAIogAIogAIlKQAAlpRNYkEBFEABFEABFECBDAUAwAyRGIICKIACKIACKIACJSkAAJaUTWJBARRAARRAARRAgQwFAMAMkRiCAiiAAiiAAiiAAiUpAACWlE1iQQEUQAEUQAEUQIEMBQDADJEYggIogAIogAIogAIlKQAAlpRNYkEBFEABFEABFECBDAUAwAyRGIICKIACKIACKIACJSkAAJaUTWJBARRAARRAARRAgQwFAMAMkRiCAiiAAiiAAiiAAiUpAACWlE1iQQEUQAEUQAEUQIEMBQDADJEYggIogAIogAIogAIlKQAAlpRNYkEBFEABFEABFECBDAUAwAyRGIICKIACKIACKIACJSkAAJaUTWJBARRAARRAARRAgQwFAMAMkRiCAiiAAiiAAiiAAiUpAACWlE1iQQEUQAEUQAEUQIEMBf4L1mKG3FWSgLUAAAAASUVORK5CYII=\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib notebook\n",
    "from matplotlib import pyplot\n",
    "import numpy\n",
    "\n",
    "tests = []\n",
    "\n",
    "@tests.append\n",
    "def f0(x):\n",
    "    return x*x - 2, 2*x\n",
    "\n",
    "@tests.append\n",
    "def f1(x):\n",
    "    return numpy.cos(x) - x, -numpy.sin(x) - 1\n",
    "\n",
    "@tests.append\n",
    "def f2(x):\n",
    "    return numpy.exp(-numpy.abs(x)) + numpy.sin(x), numpy.exp(-numpy.abs(x))*(-numpy.sign(x)) + numpy.cos(x)\n",
    "\n",
    "@tests.append\n",
    "def f3(x):\n",
    "    return x*x - x + 0.25, 2*x - 1\n",
    "\n",
    "@tests.append\n",
    "def f4(x):\n",
    "    return numpy.tan(x+2), numpy.cos(x+2)**(-2)\n",
    "    return numpy.exp(-x*x), numpy.exp(-x*x)*(-2*x)\n",
    "\n",
    "@tests.append\n",
    "def f5(x):\n",
    "    return (x <= 1)*1.0, 0*x\n",
    "\n",
    "@tests.append\n",
    "def f6(x):\n",
    "    return x*numpy.sin(5/x), numpy.sin(5/x) - numpy.cos(5/x)*5/x\n",
    "\n",
    "x = numpy.linspace(-2,2,100)\n",
    "pyplot.plot(x, 0*x, color='k')\n",
    "for f in tests:\n",
    "    pyplot.plot(x, f(x)[0], label=f.__name__)\n",
    "pyplot.legend(loc='upper right')\n",
    "pyplot.style.use('ggplot')\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Which of these functions have at least one root?\n",
    "* Which have more than one root?\n",
    "* Can we determine these properties merely by evaluating $f(x)$ for some values of $x$?\n",
    "\n",
    "## Bisection\n",
    "\n",
    "Bisection is a rootfinding technique that starts with an interval $[a,b]$ containing a root and does not require derivatives."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def hasroot(f, a, b):\n",
    "    return f(a)[0]*f(b)[0] < 0\n",
    "    \n",
    "def bisect(f, a, b, verbose=False):\n",
    "    mid = (a + b)/2.\n",
    "    if b-a < 1e-5:\n",
    "        return mid\n",
    "    if verbose:\n",
    "        print('bisect', mid)\n",
    "    if hasroot(f, a, mid):\n",
    "        return bisect(f, a, mid, verbose)\n",
    "    else:\n",
    "        return bisect(f, mid, b, verbose)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Notice that we need to define `hasroot` above.\n",
    "\n",
    "Let's try running it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.4142112731933594"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bisect(tests[0], 0, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.2891797357704746e-06"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy.sqrt(2) - bisect(tests[0], 0, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get about 5 digits of accuracy.  Why?  How fast did we get there?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bisect 1.0\n",
      "bisect 1.5\n",
      "bisect 1.25\n",
      "bisect 1.375\n",
      "bisect 1.4375\n",
      "bisect 1.40625\n",
      "bisect 1.421875\n",
      "bisect 1.4140625\n",
      "bisect 1.41796875\n",
      "bisect 1.416015625\n",
      "bisect 1.4150390625\n",
      "bisect 1.41455078125\n",
      "bisect 1.414306640625\n",
      "bisect 1.4141845703125\n",
      "bisect 1.41424560546875\n",
      "bisect 1.414215087890625\n",
      "bisect 1.4141998291015625\n",
      "bisect 1.4142074584960938\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.4142112731933594"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bisect(tests[0], 0, 2, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can you find any problems with this implementation?  List them below:\n",
    "\n",
    "* No error checking\n",
    "* Can \"converge\" to nonsense\n",
    "* Recursion is bad in Python\n",
    "* Doesn't handle ill-defined points (function value not available)\n",
    "* Evaluates $f(x)$ more than necessary (multiple times at the same point)\n",
    "\n",
    "Let's try running it on the rest of the test problem set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f0 -1.4142136573791504\n",
      "f1 0.7390824317932131\n",
      "f2 -0.588533115386963\n",
      "f3 2.0999960899353027\n",
      "f4 2.0999960899353027\n",
      "f5 2.0999960899353027\n",
      "f6 -1.5915507316589355\n"
     ]
    }
   ],
   "source": [
    "for f in tests:\n",
    "    print(f.__name__, bisect(f, -2, 2.1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's going wrong here?  How can we improve the implementation and what are fundamental limitations of the algorithm?\n",
    "\n",
    "### Convergence rate\n",
    "\n",
    "Let's quantitatively revisit the convergence rate.  A convergent rootfinding algorithm produces a sequence of approximations $x_i$ such that $$\\lim_{i \\to \\infty} x_i \\to x_*$$ where $f(x_*) = 0$.  For analysis, it is convenient to define the errors $e_i = x_i - x_*$. We say that an algorithm is **linearly convergent** if $$\\lim_{i \\to \\infty} |e_{i+1}| / |e_i| = \\rho < 1.$$  A smaller convergence factor $\\rho$ represents faster convergence.\n",
    "\n",
    "What is $\\rho$ for bisection?\n",
    "\n",
    "### Remarks on bisection\n",
    "\n",
    "* Specifying an interval is often inconvenient\n",
    "* An interval in which the function changes sign guarantees convergence (robustness)\n",
    "* No derivative information is required\n",
    "* Roots of even degree are problematic\n",
    "* The solution error is directly available\n",
    "* The convergence rate is modest -- one iteration per bit of accuracy\n",
    "\n",
    "## Newton-Raphson Method\n",
    "\n",
    "Much of numerical analysis reduces to [Taylor series](https://en.wikipedia.org/wiki/Taylor_series), the approximation\n",
    "$$ f(x) = f(x_0) + f'(x_0) (x-x_0) + f''(x_0) (x - x_0)^2 / 2 + \\dotsb $$\n",
    "centered on some reference point $x_0$.\n",
    "In numerical computation, it is exceedingly rare to look beyond the first-order approximation\n",
    "$$ \\tilde f_{x_0}(x) = f(x_0) + f'(x_0)(x - x_0) . $$\n",
    "Since $\\tilde f_{x_0}(x)$ is a linear function, we can explicitly compute the unique solution of $\\tilde f_{x_0}(x) = 0$ as\n",
    "$$ x = x_0 - f(x_0) / f'(x_0) . $$\n",
    "This is Newton's Method (aka Newton-Raphson or Newton-Raphson-Simpson) for finding the roots of differentiable functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f0 (1.4142135623730951, 4.440892098500626e-16, 5)\n",
      "f1 (0.73908513321516067, 0.0, 4)\n",
      "f2 (-6.2850492733825867, -1.4268100589909238e-16, 4)\n",
      "f3 (0.5000009536743164, 9.094947017729282e-13, 19)\n",
      "f4 (1.1415926535897931, -1.2246467991473532e-16, 3)\n",
      "f5 (1, nan, 0)\n",
      "f6 (0.53051647697298443, 1.9490859162596874e-16, 6)\n"
     ]
    }
   ],
   "source": [
    "def newton(f, x, verbose=False):\n",
    "    for i in range(100):\n",
    "        fx, dfx = f(x)\n",
    "        if verbose:\n",
    "            print(f.__name__, i, x, fx)\n",
    "        if numpy.abs(fx) < 1e-12:\n",
    "            return x, fx, i\n",
    "        try:\n",
    "            x -= fx / dfx\n",
    "        except ZeroDivisionError:\n",
    "            return x, numpy.NaN, i\n",
    "\n",
    "for f in tests:\n",
    "    print(f.__name__, newton(f, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Oops, how can we fix this?\n",
    "* This output is kinda hard to read, so let's make it cleaner."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f0 -1.414213562373 1.33e-15  8\n",
      "f1  0.739085133215 0.00e+00  5\n",
      "f2 -0.588532743981 5.55e-13  4\n",
      "f3  0.499999427800 3.27e-13 20\n",
      "f4 -5.294229332623 6.72e-13  3\n",
      "f5 -0.100000000000      nan  0\n",
      "f6 -0.099471839432 -1.95e-16  3\n"
     ]
    }
   ],
   "source": [
    "for f in tests:\n",
    "    print(f.__name__, '{0:15.12f} {1:8.2e} {2:2d}'.format(*newton(f, -0.1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Did we solve all of these equations?\n",
    "* How can the iteration break down?\n",
    "* Does choosing a different initial guess lead to different solutions?\n",
    "* How is this convergence test different from the one we used for bisection?\n",
    "* Is the convergence rate similar for all test equations?\n",
    "\n",
    "## Convergence of Newton-type algorithms\n",
    "\n",
    "We would like to know sharp conditions on when Newton-type algorithms converge, and if so, how fast.  This theory will build on that for a general _Fixed Point Iteration_ $x_{i+1} = g(x_i)$ where $g$ is a continuously differentiable function.  Suppose that there exists a fixed point $r = g(r)$.  By the mean value theorem, we have that\n",
    "$$ x_{i+1} - r = g(x_i) - g(r) = g'(c_i) (x_i - r) $$\n",
    "for some $c$ with $|c - r| < |x_i - r|$.\n",
    "In other words, $|e_{i+1}| = |g'(c_i)| |e_i|$, which converges to zero if $|g'(c_i)| < 1$.\n",
    "If $|g'(r)| < 1$ then for any $\\epsilon > 0$ there is a neighborhood of $r$ such that $|g'(c)| < |g'(r)| + \\epsilon$ for all $c$ in that neighborhood.\n",
    "Consequently, we have:\n",
    "\n",
    "#### Theorem (Sauer 1.6): Linear Convergence of Fixed Point Iteration\n",
    "\n",
    "If $g$ is continuously differentiable, $r = g(r)$, and $|g'(r)| < 1$ then the fixed point iteration $x_{i+1} = g(x_i)$ is locally linearly convergent with rate $|g'(r)|$.\n",
    "\n",
    "#### Observations\n",
    "\n",
    "* A rootfinding problem $f(x) = 0$ can be converted to a fixed point problem $x = x - f(x) =: g(x)$ but there is no guarantee that $g'(r) = 1 - f'(r)$ will have magnitude less than 1.\n",
    "* Problem-specific algebraic manipulation can be used to make $|g'(r)|$ small.\n",
    "* $x = x - h(x)f(x)$ is also a valid formulation for any $h(x)$ bounded away from $0$.\n",
    "* Can we choose $h(x)$ such that $ 1 - h'(x)f(x) - h(x)f'(x) = 0$ when $f(x) = 0$?\n",
    "\n",
    "In other words,\n",
    "$$ x_{i+1} = x_i - ??? . $$\n",
    "\n",
    "* It turns out that Newton's method has _locally quadratic_ convergence to simple roots, $\\lim_{i \\to \\infty} |e_{i+1}|/|e_i^2| < \\infty$.\n",
    "* \"The number of correct digits doubles each iteration.\"\n",
    "* Now that we know how to make a good guess accurate, the effort lies in getting a good guess.\n",
    "\n",
    "#### Culture: fast inverse square root\n",
    "\n",
    "The following code appeared literally (including comments) in the Quake III Arena source code (late 1990s).\n",
    "\n",
    "```C\n",
    "float Q_rsqrt( float number )\n",
    "{\n",
    "\tlong i;\n",
    "\tfloat x2, y;\n",
    "\tconst float threehalfs = 1.5F;\n",
    "\n",
    "\tx2 = number * 0.5F;\n",
    "\ty  = number;\n",
    "\ti  = * ( long * ) &y;                       // evil floating point bit level hacking\n",
    "\ti  = 0x5f3759df - ( i >> 1 );               // what the fuck? \n",
    "\ty  = * ( float * ) &i;\n",
    "\ty  = y * ( threehalfs - ( x2 * y * y ) );   // 1st iteration\n",
    "//\ty  = y * ( threehalfs - ( x2 * y * y ) );   // 2nd iteration, this can be removed\n",
    "\n",
    "\treturn y;\n",
    "}\n",
    "```\n",
    "\n",
    "We now have [vector instructions](https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=sqrt&expand=2989,1224,4470) for approximate inverse square root.\n",
    "More at https://en.wikipedia.org/wiki/Fast_inverse_square_root\n",
    "\n",
    "## Conditioning\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.899096947850202, 6.650454451607741e-13, 24)"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def fquartic(x):\n",
    "    return (x - 0.9)**4, 4*(x - 0.9)**3\n",
    "\n",
    "newton(fquartic, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We only get three digits correct despite a very small residual (and it takes many iterations to get there).\n",
    "Difficulty computing zeros of polynomials can also arise when all the roots are simple.  For example, the Wilkinson polynomial\n",
    "$$ \\prod_{i=1}^{20} (x - i) = \\sum_{i=0}^{20} a_i x^i $$\n",
    "has roots at each of the positive integers up to 20, but the roots are extremely sensitive to perturbations of the coefficients $a_i$, as shown in this figure from Trefethen and Bau (1999).\n",
    "![Ill conditioning of roots of Wilkinson's polynomial](figures/TB-Wilkinson.png)\n",
    "\n",
    "Numerical difficulties in which \"correct\" algorithms produce unreliable solutions almost always stem from lack of *stability* and/or *ill conditioning*.\n",
    "\n",
    "### Absolute condition number\n",
    "Consider a function $f: X \\to Y$ and define the *absolute condition number*\n",
    "$$ \\hat\\kappa = \\lim_{\\delta \\to 0} \\max_{|\\delta x| < \\delta} \\frac{|f(x + \\delta x) - f(x)|}{|\\delta x|} = \\max_{\\delta x} \\frac{|\\delta f|}{|\\delta x|}. $$\n",
    "If $f$ is differentiable, then $\\hat\\kappa = |f'(x)|$.\n",
    "\n",
    "### Floating point arithmetic\n",
    "Floating point arithmetic $x \\circledast y := \\text{float}(x * y)$ is exact within a relative accuracy $\\epsilon_{\\text{machine}}$.  Formally,\n",
    "$$ x \\circledast y = (x * y) (1 + \\epsilon) $$\n",
    "for some $|\\epsilon| \\le \\epsilon_{\\text{machine}}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.00000000000000002776'"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "format((.2 - 1/3) + 2/15, '.20f')\n",
    "# format((.2 - 1/3) + (1/3 - 0.2), '.20f')\n",
    "# format((1 + 1e-12) - 1, '.20f')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "eps = 1\n",
    "while 1 + eps > 1:\n",
    "    eps /= 2\n",
    "eps_machine = eps  # We call this \"machine epsilon\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8.8900582341031727e-17"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy.log(1 + 1e-12) - numpy.log1p(1e-12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8.8900582341076178e-05"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(numpy.log(1 + 1e-12) - numpy.log1p(1e-12)) / numpy.log1p(1e-12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1.22464680e-16,  -3.39606540e-11,  -2.23936276e-06,\n",
       "        -2.36209053e-01])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = numpy.array([1,1e5,1e10,1e15])\n",
    "numpy.sin(numpy.pi*x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0.00000000e+00,  -1.11022302e-16,   0.00000000e+00,\n",
       "         0.00000000e+00])"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy.sin(x)**2 + numpy.cos(x)**2 - 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([1209489.8070879749, 756547.02744709956], 1.0000000000070415)"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[numpy.tan((3.14159+eps)/2) for eps in [1e-6,1e-8]], 1/numpy.cos(3.14159)**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Relative condition number\n",
    "\n",
    "Given the relative nature of floating point arithmetic, it is more useful to discuss **relative condition number**,\n",
    "$$ \\kappa = \\max_{\\delta x} \\frac{|\\delta f|/|f|}{|\\delta x|/|x|}\n",
    "= \\max_{\\delta x} \\Big[ \\frac{|\\delta f|/|\\delta x|}{|f| / |x|} \\Big] $$\n",
    "or, if $f$ is differentiable,\n",
    "$$ \\kappa = \\max_{\\delta x} |f'(x)| \\frac{|x|}{|f|} . $$\n",
    "\n",
    "How does a condition number get big?\n",
    "\n",
    "#### Take-home message\n",
    "\n",
    "The relative accuracy of the best-case algorithm will not be reliably better than $\\epsilon_{\\text{machine}}$ times the condition number.\n",
    "$$ \\max_{\\delta x} \\frac{|\\delta f|}{|f|} \\ge \\kappa \\cdot \\epsilon_{\\text{machine}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numerical differentiation\n",
    "\n",
    "Suppose we want to apply Newton's method to a function that we know how to evaluate, but don't have code to differentiate.  This is often because it's difficult/error-prone to write or because the interface by which we call it does not support derivatives.  (Commercial packages often fall in this category.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.2977936220414108e-09"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def diff(f, x, epsilon=1e-5):\n",
    "    return (f(x + epsilon) - f(x)) / epsilon\n",
    "\n",
    "diff(numpy.sin, 0.7, 1e-8) - numpy.cos(0.7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7.0935189069309956e-06"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = .5\n",
    "diff(numpy.tan, x) - 1/numpy.cos(x)**2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1e-14, -1271.3873432741966),\n",
       " (1e-12, 140.14843387738802),\n",
       " (1e-10, 0.32726590032689273),\n",
       " (1e-08, 19.793431113474071),\n",
       " (1e-06, 1982.7670766180381),\n",
       " (0.0001, 226466.63879947271)]"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 3.14/2\n",
    "[(eps, diff(numpy.tan, x, eps) - 1/numpy.cos(x)**2) for eps in [1e-14, 1e-12, 1e-10, 1e-8, 1e-6, 1e-4]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1e-14, -0.0001),\n",
       " (1e-12, -0.0001),\n",
       " (1e-10, -1.1182158029987482e-05),\n",
       " (1e-08, 8.8900582340963692e-09),\n",
       " (1e-06, 8.2740370951168637e-12),\n",
       " (0.0001, -9.4895312988856409e-12),\n",
       " (0.01, -5.0168102921151377e-11)]"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 1e4\n",
    "[(eps, diff(numpy.log, x, eps) - 1/x) for eps in [1e-14, 1e-12, 1e-10, 1e-8, 1e-6, 1e-4, 1e-2]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Automatically choosing a suitable $\\epsilon$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1e-14, -0.00079927783736910563),\n",
       " (1e-12, -2.2121721121481919e-05),\n",
       " (1e-10, 8.2640370990816336e-08),\n",
       " (1e-08, -4.9752407749181771e-09),\n",
       " (1e-06, -9.9996991109740918e-07),\n",
       " (0.0001, -9.9986668776530507e-05),\n",
       " (0.01, -0.0098686351910135528)]"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def diff_wp(f, x, eps=1e-8):\n",
    "    \"\"\"Numerical derivative with Walker and Pernice (1998) choice of step\"\"\"\n",
    "    h = eps * (1 + abs(x))\n",
    "    return (f(x+h) - f(x)) / h\n",
    "\n",
    "x = 1\n",
    "[(eps, diff_wp(numpy.log, x, eps) - 1/x) for eps in [1e-14, 1e-12, 1e-10, 1e-8, 1e-6, 1e-4, 1e-2]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1e-14, -0.11098307828251563),\n",
       " (1e-12, -0.00085996655070630368),\n",
       " (1e-10, -0.0050162259285571054),\n",
       " (1e-08, -0.50001677127511357),\n",
       " (1e-06, -49.674080904416769),\n",
       " (0.0001, -3068.7213347666539),\n",
       " (0.01, -9538.5241953963505)]"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 1e-4\n",
    "[(eps, diff_wp(numpy.log, x, eps) - 1/x) for eps in [1e-14, 1e-12, 1e-10, 1e-8, 1e-6, 1e-4, 1e-2]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This algorithm is imperfect, leaving some scaling responsibility to the user.\n",
    "It is the default in PETSc's \"matrix-free\" Newton-type solvers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tinkering in class (2016-09-06)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3.5/site-packages/ipykernel/__main__.py:1: RuntimeWarning: invalid value encountered in log\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy.log(-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3.5/site-packages/ipykernel/__main__.py:1: RuntimeWarning: invalid value encountered in sqrt\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = numpy.sqrt(-1)\n",
    "1/numpy.inf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "x = numpy.linspace(0,3,100)\n",
    "pyplot.plot(x, numpy.sqrt(x + 1e-1))\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.41162296288324979"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy.tan(1e100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.033025094829895622"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy.tan(1e100*(1 + 2*eps_machine))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-1.942668892225729e+84"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1e100 - 1e100*(1 + 2*eps_machine)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## In class 2016-09-14\n",
    "\n",
    "We are looking for a function for which the linear fit (`newton` solver) is more reliable than a quadratic fit (`cubic` solver)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width);\n",
       "        canvas.attr('height', height);\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'];\n",
       "    var y0 = fig.canvas.height - msg['y0'];\n",
       "    var x1 = msg['x1'];\n",
       "    var y1 = fig.canvas.height - msg['y1'];\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x;\n",
       "    var y = canvas_pos.y;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        event.shiftKey = false;\n",
       "        // Send a \"J\" for go to next cell\n",
       "        event.which = 74;\n",
       "        event.keyCode = 74;\n",
       "        manager.command_mode();\n",
       "        manager.handle_keydown(event);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAgAElEQVR4Xu3dCZgcZbm38acqIQSyAGGRRUQUlUVQ4HNBRUUUREVlSQA5KMLRKIgIsomiARQPgiAgHqJyDCACSQSNikYUFBQPuHAAERQBWWQ1ISQBEkiqvqd6kpjAZLqmn/870z19z3X1lcBUv9P1q7f7vVM93Z0ZXwgggAACCCCAAAJdJZB11d6yswgggAACCCCAAAJGADIJEEAAAQQQQACBLhMgALvsgLO7CCCAAAIIIIAAAcgcQAABBBBAAAEEukyAAOyyA87uIoAAAggggAACBCBzAAEEEEAAAQQQ6DIBArDLDji7iwACCCCAAAIIEIDMAQQQQAABBBBAoMsECMAuO+DsLgIIIIAAAgggQAAyBxBAAAEEEEAAgS4TIAC77ICzuwgggAACCCCAAAHIHEAAAQQQQAABBLpMgADssgPO7iKAAAIIIIAAAgQgcwABBBBAAAEEEOgyAQKwyw44u4sAAggggAACCBCAzAEEEEAAAQQQQKDLBAjALjvg7C4CCCCAAAIIIEAAMgcQQAABBBBAAIEuEyAAu+yAs7sIIIAAAggggAAByBxAAAEEEEAAAQS6TIAA7LIDzu4igAACCCCAAAIEIHMAAQQQQAABBBDoMgECsMsOOLuLAAIIIIAAAggQgMwBBBBAAAEEEECgywQIwC474OwuAggggAACCCBAADIHEEAAAQQQQACBLhMgALvsgLO7CCCAAAIIIIAAAcgcQAABBBBAAAEEukyAAOyyA87uIoAAAggggAACBCBzAAEEEEAAAQQQ6DIBArDLDji7iwACCCCAAAIIEIDMAQQQQAABBBBAoMsECMAuO+DsLgIIIIAAAgggQAAyBxBAAAEEEEAAgS4TIAC77ICzuwgggAACCCCAAAHIHEAAAQQQQAABBLpMgADssgPO7iKAAAIIIIAAAgQgcwABBBBAAAEEEOgyAQKwyw44u4sAAggggAACCBCAzAEEEEAAAQQQQKDLBAjALjvg7C4CCCCAAAIIIEAAMgcQQAABBBBAAIEuEyAAlxzw8ePHj/C/ftUvu/rlab/cPG3atA922XxgdxFAAAEEEECgCwQIwCUHecKECWcWRZFPnz798Op/7bHHHutdccUVjzabA369XadOnTqz2XZ8v7kAls2N6myBYx2lettgWc+pzlZY1lFqvg2OzY3qbtHtlgSgz5Tdd9999ZEjRz7kf93Iz/rNrzt5qu323nvvMzwaj+zPddi2dwEsNTMDR40j92+dI5Y6S+7fWKoECECX9Kd/t/Y/ZvjlUr+83S9P+eVEj8Grm0FzZ2wmVP/7WNa36mtLHDWORIvOEUudJfdvLFUCBKBL7rPPPtv5079/KMvyAD+bd7EH4av9f1/lly09Ah9bGbZvnx111FHnnX766UerDkg3jzNx4sRTJk+efHw3Gyj2HUeFYs8YWGKpE9CMxJzUOFaj+Pp9mq/fH8uyrNSN2jkjEYB+rPz3/dYePnz4wx571QtBGhPBI/BG/+O4554FrH5nwGOxeqGIbbLJJqN88ny0cw43txQBBBBAAAEElgp4BH7z3nvvfbL67zzPZ3bT7/QTgEtmgQffz/yM3ll+BvCn/vdN/X/fsHjx4lddfvnl1e8G9vrl24/1bzzx8MMPm/+de1RQYMyYMTZv3rzgKFwdR90cwBJLnYBmJOakxtHP+tn6669fDbaG/32uZtTOGoUA/HcAVtF3vl/W8ctinxAn+r8EftDX4VwagA899BABKJj3Y8eOtblzu/J+KND79xA46jixxFInoBmJOalxrAJwgw02IAA1nN03CgGoPeY8sGk8cdQ4VqNgiaVOQDMSc1LjSACacQYwMJcIwABeL1flgU3jiaPGkQDUOWKps+T+rbEkAAnA0EwiAEN8z7syD2waTxw1jkSLzhFLnSX3b40lAUgAhmYSARjiIwC1fMtGY4HQwWKJpU5AMxJzUuNIABKAoZlEAIb4CEAtHwGYwJPFVoeKpcYSR40jAUgAhmYSARjiIwC1fARgAk8WWx0qlhpLHDWOBCABGJpJBGCIjwDU8hGACTxZbHWoWGoscdQ4EoAEYGgmEYAhPgJQy0cAJvBksdWhYqmxxFHjSAASgKGZRACG+AhALR8BmMCTxVaHiqXGEkeNIwFIAIZmEgEY4iMAtXwEYAJPFlsdKpYaSxw1jgQgARiaSQRgiI8A1PIRgAk8WWx1qFhqLHHUOBKABGBoJhGAIT4CUMtHACbwZLHVoWKpscRR40gAEoChmUQAhvgIQC0fAZjAk8VWh4qlxhJHjSMBSACGZhIBGOIjALV8BGACTxZbHSqWGkscNY4EIAEYmkkEYIiPANTyEYAJPFlsdahYaixx1DgSgARgaCYRgCE+AlDLRwAm8GSx1aFiqbHEUeNIABKAoZlEAIb4CEAtHwGYwJPFVoeKpcYSR40jAUgAhmYSARjiIwC1fARgAk8WWx0qlhpLHDWOBCABGJpJBGCIjwDU8hGACTxZbHWoWGoscdQ4EoAEYGgmEYAhPgJQy0cAJvBksdWhYqmxxFHjSAASgKGZRACG+AhALR8BmMCTxVaHiqXGEkeNIwFIAIZmEgEY4iMAtXwEYAJPFlsdKpYaSxw1jgQgARiaSQRgiI8A1PIRgAk8WWx1qFhqLHHUOBKABGBoJhGAIT4CUMtHACbwZLHVoWKpscRR40gAEoChmUQAhvgIQC0fAZjAk8VWh4qlxhJHjSMBSACGZhIBGOIjALV8BGACTxZbHSqWGkscNY4EIAEYmkkEYIiPANTyEYAJPFlsdahYaixx1DgSgARgaCYRgCE+AlDLRwAm8GSx1aFiqbHEUeNIABKAoZlEAIb4CEAtHwGYwJPFVoeKpcYSR40jAUgAhmYSARjiIwC1fARgAk8WWx0qlhpLHDWOBCABGJpJBGCIjwDU8hGACTxZbHWoWGoscdQ4EoAEYGgmEYAhPgJQy0cAJvBksdWhYqmxxFHjSAASgKGZRACG+AhALR8BmMCTxVaHiqXGEkeNIwFIAIZmEgEY4iMAtXwEYAJPFlsdKpYaSxw1jgQgARiaSQRgiI8A1PIRgAk8WWx1qFhqLHHUOBKABGBoJhGAIT4CUMtHACbwZLHVoWKpscRR40gAEoChmUQAhvgIQC0fAZjAk8VWh4qlxhJHjSMBSACGZhIBGOIjALV8BGACTxZbHSqWGkscNY4EIAEYmkkEYIiPANTyEYAJPFlsdahYaixx1DgSgARgaCYRgCE+AlDLRwAm8GSx1aFiqbHEUeNIABKAoZlEAIb4CEAtHwGYwJPFVoeKpcYSR40jAUgAhmYSARjiIwC1fARgAk8WWx0qlhpLHDWOBCABGJpJBGCIjwDU8hGACTxZbHWoWGosO8GxLBZblg/T7HCiUQhAAjA0tQjAEB8BqOUjABN4dsJim2C3kwyJpYa13R3LObOtOGuS5QcebtkmL9XsdIJRCEACMDStCMAQHwGo5SMAE3i2+2KbYJeTDYmlhradHcs5s6w4/XOWveQVlh14WFufBSQACcDQPZIADPERgFo+AjCBZzsvtgl2N+mQWGp429WxfLyKv89attkWln3oE20df9WRIAAJwNA9kgAM8RGAWj4CMIFnuy62CXY1+ZBYaojb0bGc/S8rvurx97ItLftgdeYv1+xswlEIQAIwNL0IwBAfAajlIwATeLbjYptgNwdkSCw1zO3m2Ii/04+37BVbW3bAoR0Rf5wB7JmLmWZKducoBKD2uLfbA5t27wZuNBx11lhiqRPQjNROc7Kc/VjP076bb2PZfxzSMfFHABKA4XsjARgmXGGAdnpg0+7ZwI6Go84bSyx1ApqR2mVOlrM8/qqnfTsw/ghAAjB8byQAw4QEoJawMVq7LBAJdm3Ah8RSR46lxrIdHMtZj/ac+dtyW8v2/1hHnflbehT4HUCeAg7dIwnAEN/zrtwOD2zaPRqc0XDUuWOJpU5AM9Jgz8nysYetOOMEy7by+PtAZ8YfZwA5Axi+NxKAYULOAGoJOQMo9hzsxVa8O4M6HJYa/sF0LO/8ixXfOMWy173FsgkHd+SZP84A/nse8iKQwH2SAAzg9XLVwXxg0+7J4I6Go84fSyx1ApqRBmtOFr/9pZXfO8+y8R+2/K3v0uzMII7CU8A8BRyafgRgiO95Vx6sBzbtXgz+aDjqjgGWWOoENCMN9Jwsi8LKyy+08rqZlk881n/v79WaHRnkUQhAAjA0BQnAEB8BqOVbNtpALxCJdqMthsVSdxiw1FgOpGO54Gkrzj/D7MH7LT/MP+Jt/RdqdqINRiEACcDnTcO99977wz4xzvdvvH/atGkz+pqnBKD2XjyQD2zaW95eo+GoOx5YYqkT0Iw0UHOy8TYvX/+i2ajRln/8OMtGjdHsQJuMQgASgCtMxfHjx7/I/8clS/7nqQTgwN5TB+qBbWD3auB/Go46cyyx1AloRhqIOVnedUfPiz1e/TrL9pto2fDhmhvfRqMQgATg8tMx8wD8eVEUx+R57ue87UwCcGDvrQPxwDawezQ4Pw1HnTuWWOoENCOlnpPF/15j5Xf/27I9DrDsbe+xKpSG4hcBSAAum9f+1O+nfUKM8ug7yUPwGgJw4O/yqR/YBn6PBucn4qhzxxJLnYBmpFRzspw/18qLz7PyjpstP+hIy7beXnOD23QUApAAbExND74t/Y/q9/7e5AG4mAAcnHtsqge2wdmbwfupOOrsscRSJ6AZKcWcLG/+vRUXfd3sxS+z/IOHWjZ2Lc2NbeNRCEACsDE9J0yY8DF/QccJ/teFfqnOd6/vlyf8/31h+vTpk5efw77trv408a7V/xs3btyIyZMnHzp//vw2nuadc9NGjBhhzzzzTOfc4Da9pTjqDgyWWOoENCMp52T51JP21IXn2rM3/NpWO/CTNuLNuwzZp3x70x89erRNnDjx3NmzZzcWHv/1r5lTp06dqTlS7T/K0HxyP+jOGcAgYItXT/Ev2xZvSkdfDUfd4cMSS52AZiTVnCxvv9mKKWebvWBDyz3+snHram5gh4zCGUDOAPY6VT0Ar/ZvfI0XgQzsPVn1wDawt7r9fhqOumOCJZY6Ac1I0TlZLlzob+x8gZX+yR7Z3h+y7C27ddVZv6VHgQAkAEP3SN4HMMT3vCtHH9i0t6ZzR8NRd+ywxFInoBmp1Tnp65XZrX+w4jL/dfexa1j+4cMtW29DzY3qwFEIQAIwNG0JwBAfAajlWzZaqwtEopvT0cNiqTt8WGosW3Es7/yLFf5xbvbwA5a9Z1/LdvKzfvkwzQ3q0FEIQAIwNHUJwBAfAajlIwATeLay2Ca4GUNiSCw1h7E/juUD91hxxXfN/vpny3Z5v1/eZ9nI1TU3pMNHIQAJwNAUJgBDfASglo8ATODZn8U2wY8fUkNiqTmcdRzLxx62csb3rPzj9f47fu+07F3jLRuzhuYGDJFRCEACMDSVCcAQHwGo5SMAE3jWWWwT/NghOSSWmsPal2M5Z7aVV06z8jdXWfaaHS17736Wrb2e5gcPsVEIQAIwNKUJwBAfAajlIwATeBItOlQsNZbPdSyLwuwv/2fFdT83u+VGs1dub/n7/WPcNqo+2p6vlQkQgARg6N5BAIb4CEAtHwGYwJNo0aFiqbFc6lg+PsvfyuUXjbN99uwzlr1xZ8ve5G/kvN4Gmh80xEchAAnA0BQnAEN8BKCWjwBM4Em06FCxjFuWixfbanffbk/O/IHZn/9ktsU2lu/oH0y1zWssGz48/gO6aAQCkAAMTXcCMMRHAGr5CMAEnkSLDhXL1iyrj2uzO2620oOv9Pfxy/Lc7A3V2b538Pt9rZE2rkUAEoCB6WNGAIb4CEAtHwGYwJNo0aFiWc+y8Tt9995l5W0efLfdZHb3Hf5xbRtZttV2lr1yOxv72jfavCefqjcYW61UgAAkAEN3DwIwxEcAavkIwASeRIsOFcveLcsFT5v9814r77/H7M7brPQXdJg/1Wtbvqon+rbadoXP6cVRMycJQAIwNJMIwBAfAajlIwATeLLY6lC73bLxUWyP/8vs/n9Y9QbN5f13N/5ujz1kNnqs2cYvsewlL29En23qfw7r/ZM6ut1RNSMJQAIwNJcIwBAfAajlIwATeLLY6lCHumUj8ObNMZv1mNnsx6yc9Wjj76X/3Rp/98vTfrZvfX8694Uv7gm+jTf1Pze1bI21akMPdcfaEMENCUACMDSFCMAQHwGo5SMAE3iy2OpQO9Gy9LdXsXlP+GVu48+y8fellzlWzvW/z/fvzfXwqy7V9tUnboxb18zfgDlr/Lnukj/9DZnXf6Flq64aQu1Ex9AOJ7oyAUgAhqYWARjiIwC1fARgAk8WWx1qu1hWb6ViTzze83TsHH8vvcafsz3gPPCqmKsuS6NvoZ+xyzKzUaM97Nb0iz9V64GXLfv7mpaN9eAb7Zc1/PsefNmIWOA1E28Xx2a3s92/TwASgKE5SgCG+AhALR8BmMCTxVaHOpCWjRdWPPSAlQ/ea/bg/Vb+65Ge4PM3T27En/nTtWP9ade11m5csjXHLQu8xmfmVr+TV0VdFXyjxqz09/F0OvVHGkjH+req87YkAAnA0KwlAEN8BKCWjwBM4Mliq0NNYVkWfjbvAX8Frb+K1h68z4PvvsYrahu/bzd6jNmGL7LML7aufzrGWutY1gi+dfxs3Vod+8bJKRx1R7lzRiIACcDQbCUAQ3wEoJaPAEzgyWKrQ1VYNt4jz8/qlXfc6pdbzP52m5/M8/+30SY9obf0z+rvY/3p2erp2yH2pXAcYiQt7Q4BSAC2NHGWXokADPERgFo+AjCBJ4utDrUVy8Yrax95sCf2/FL+9daeF1q8bCvLNt/aL9v0vIo27/0tU3S3vn1GasWxfW59+9wSApAADM1GAjDERwBq+QjABJ4stjrU/lhWv7NX/u81Vv7uGn9LFf/dvc22sOwVS4LvxS/r2KdvFZr9cVT8vKE6BgFIAIbmNgEY4iMAtXwEYAJPFlsdajPL8umnrPzT9VZef7XZXf7xZ/4JGPkb3ma29f9L/spa3V6mH6mZY/pbMDR+AgFIAIZmMgEY4iMAtXwEYAJPFlsdam+WjRdx3O5P7f7uaitv+l3P++TtsJNlr32Lv72Kv60KXzxOJpoDBCABGJpaBGCIjwc2LR8BmMCTANShLm9ZvcFyed3PrZx5eeNzb7PXvbUn/KpPyOCrTwHmpGaCEIAEYGgmEYAhPgJQy0cAJvBksdWhVpZPzPqXlb+5ysorpzfeXDnffT+zV7+urd5nT7fHaUZiTmpcCUACMDSTCMAQHwGo5SMAE3iy2GpQy2eftVX/cJ09fflF/w6/bV/vr97NNT+gi0ZhTmoONgFIAIZmEgEY4iMAtXwEYAJPFtsYahV+5W97zvjl1Rszv3uC2bY7EH4BVuZkAG+5qxKABGBoJhGAIT4CUMtHACbwZLFtDbV6/77yhl9becWFZiNX96d697Wxb32nzZs/v7UBuRb3b/EcIAAJwNCUIgBDfASglo8FIoEnAdh/1HL2Y1Z897/N7v27ZRMOtuw1OzbO+GHZf8veroGjxpEAJABDM4kADPERgFo+AjCBJ4ttfdTqY9rKX//MyssvsGz7N1g23uPPX+ix9AvL+pZ9bYmjxpEAJABDM4kADPERgFo+FtoEniy29VDLh/9pxYXnND61I//goZZtuS3373p0/d6KOdlvsl6vQAASgKGZRACG+FggtHwEYAJPFtu+UUt/D7/y5z+w8seXWrbjLpa9/z8sG7lar1fCUjNBcdQ4EoAEYGgmEYAhPgJQy0cAJvBksV05annf3VZc4Gf9/E2d8w8dZtlLN+/zCGCpmaA4ahwJQAIwNJMIwBAfAajlIwATeLLY9o5aVJ/icem3LHvH+yx79z6WrbJKU30smxLV2gDHWkxNNyIACcCmk6SvDQjAEB8BqOUjABN4stiuiFou8vf1u+x8K/1NnfOPHmPZFq+qrY5lbSrOpGqo+hyFACQAQ9OMAAzxEYBaPgIwgSfR8m/Ucu7jVpx3qtmCpy0/5HjL1nlBv8Sx7BfXSjfGUeNIABKAoZlEAIb4CEAtHwGYwJPFtge1/MedVnzjy5a9bEvLPui/77fqqv3WxrLfZL1eAUeNIwFIAIZmEgEY4iMAtXwEYAJPFluz4vqrrfzeeZbtvp9lu7zfqoWzlS8sW1F7/nVw1DgSgARgaCYRgCE+AlDLRwAm8OzmxbZctMjK6d+x8nfX+O/7HW3ZVs9/b7/+kHezZX+cmm2LYzOhet8nAAnAejNlJVsRgCE+AlDLRwAm8OzWxbacP7fn9/38z8bv+623QVi3Wy3DcM8ZAEeNKAFIAIZmEgEY4iMAtXwEYALPblxsGy/2OOPzZv4ij/w/P73SN3buL3c3WvbXqM72ONZRar4NAUgANp8lfWxBAIb4CEAtHwGYwLPbFtvSP8qtOPMEy160mWUHfcqyYcNkqt1mKYPjDGASSgKQAAxNLAIwxEcAavkIwASe3RQt5b8eseKrn7Ns820sO+AQy3Jd/FWHppssE0xF7t9iVAKQAAxNKQIwxEcAavlYIBJ4dku0lI882BN/r36dZft+xOMvl2t2i6UcjjOASUgJQAIwNLEIwBAfAajlIwATeHZDtJT/vK/nad/X72TZXh9q+W1emvF3g2UzA8X3cVQoevz42xltsEHjxU1r+N/nakbtrFFae0OnztrHZLeWANTS8sCm8cRR41iNMtQty/vu8vj7gmU7vdvf52/fZPHXDZa6Wdf3SEN9Tg6UIwHIGcDQXCMAQ3ycAdTycQYwgedQXmzLu/9qxVmTLNttb8vfuVcCvRWHHMqWyfGW+wE4arQJQAIwNJMIwBAfAajlIwATeA7Vxbb8221WnHOSZXscYPnb3pNA7vlDDlXLAcEjAOXMBCABGJpUBGCIjwDU8hGACTyHYrQ0nvY9/bOWjT/I8h13SaDW+5BD0XLA8AhAOTUBSACGJhUBGOIjALV8BGACz6EWLeVjD1vxX8c0PtM333XPBGIrH3KoWQ4oHgEo5yYACcDQpCIAQ3wEoJaPAEzgOZSipZw7x4pTj7Vsm9dYNuHgpC/46O1QDCXLBFOt9pA41qbqc0MCkAAMzSQCMMRHAGr5CMAEnkNlsS0XPN3zPn/rrm9Z9fFuCd7nrxn/ULFstp+pv4+jRpgAJABDM4kADPERgFo+AjCB51BYbMtFi6w494tmzz5r+eH+qt9VVkkg1XzIoWDZfC/Tb4GjxpgAJABDM4kADPERgFo+AjCBZ6cvtv4YZeV3vmbl/f+w/OhTLFt9VAKlekN2umW9vUy/FY4aYwKQAAzNJAIwxEcAavkIwASenb7YFpdfaOWN11p+3FcsW3NcAqH6Q3a6Zf09TbsljhpfApAADM0kAjDERwBq+QjABJ6dvNgWv/yxlT++xPJjT7Vs/Rcm0OnfkJ1s2b89Tbs1jhpfApAADM0kAjDERwBq+QjABJ6dutiWf/iNFVPOtvwIf7Pnl26eQKb/Q3aqZf/3NO01cNT4EoAEYGMm7bbbbquOHj36Uv/rFn552i+P5nl+yGWXXXZXX1ONANTcEZeOwgObxhNHjWM1Sidaln+/3T/f9/OWf9Tf7+9Vr9FhBEfqRMvgLie5Oo4aVgKQAFw+AHeaNm3az6r/MWHChEM97vb2/96JANTc2eqMwgNbHaXm2+DY3KjuFp1mWc6ZbcUXj7DMP9s3f/t76+7mgGzXaZYDgtLCD8GxBbRerkIAEoC9zqTx48dv79/w/pv2EgJQc2erMwoPbHWUmm+DY3Ojult0kmW56Nmej3hbdwPLDvrUgL/RczPTTrJsti+D+X0cNfoEIAHY60zyM4AXFEUxa/r06UcSgJo7W51ReGCro9R8GxybG9XdopMsi4v/28q7/9rzoo8Rq9bdxQHbrpMsBwylhR+EYwtonAHsFS3TUA6dUTz+jvenf989atSonadMmbKAABy4Y8sDm8YaR41jNUqnWBbX/dzKyy+w/HNnWrb2ejoA4UidYinc5SRD4ahh5QwgZwBXmEkef0d5/E1YuHDhzjNmzJjX2zTzbXb1s4O7Vt8bN27ciMmTJx86f/58zYzs8lFGjBhhzzzzTJcrxHcfx7jh0hE6wXKRv+hj3klH2OhjTrFVXrmdbufFI3WCpXiXkwyHo47VX/xpEydOPHf27NmNhcdf/Dlz6tSpM3U/ob1H4gzgkuPjv/dXPd37Ab/s7L/790Sdw8argOso1d+Gf9nWt+prSxw1jtUo7W5Zzn3cipOPtOwd77V8lz10O55gpHa3TLDLSYbEUcPKGUDOADZmksffRv7H/X6p3valOvNXhfECD8Ed+ppqBKDmjrh0FB7YNJ44ahzbPQAbn/F7xuf8Ez7WtuwjR7Xdiz6eexSYl5p5iaPGkQAkAEMziQAM8T3vyjywaTxx1Di2ewAWl37Lyr/e2vMxb6uO1O10opGYlxpYHDWOBCABGJpJBGCIjwDU8i0bjQVCB9uulsX1V1t52bf9RR9n+Nu+rK/b4YQjtatlwl1OMjSOGlYCkAAMzSQCMMRHAGr5CMAEnu242Jb33mXFaZ+x/OOfsWyrbRPsdZoh29EyzZ6mHRVHjS8BSACGZhIBGOIjALV8BGACz3ZbbMunn/IXffibPL/x7Za/e0KCPU43ZLtZptvTtCPjqPElAAnA0EwiAEN8BKCWjwBM4Nlui23x7a9aOXeO5Z860bI8T7DH6YZsN8t0e5p2ZBw1vgQgARiaSQRgiI8A1PIRgAk822mxLX53jZVTz7f8C2f7K3/HJdjbtEO2k2XaPU07Oo4aXwKQAAzNJAIwxEcAavkIwASe7bLYlo8+6E/9HmF59XYv27wmwZ6mH7JdLNPvadqfgKPGlwAkAEMziQAM8RGAWj4CMIFnOyy25aJnrfivYy3bbAvL9/1Igr0cmCHbwXJg9jTtT8FR40sAEoChmUQAhsS1iLkAACAASURBVPgIQC0fAZjAsx0W22L6FCtvu8ny40+3bJVVEuzlwAzZDpYDs6dpfwqOGl8CkAAMzSQCMMRHAGr5CMAEnoO92JZ/ucmKb3zZ8s9+1bINNk6whwM35GBbDtyepv1JOGp8CUACMDSTCMAQHwGo5SMAE3gO5mJbznvCihM/adn79rd8x10S7N3ADjmYlgO7p2l/Go4aXwKQAAzNJAIwxEcAavkIwASeg7XY+mOLFeecbDZihOUT/ff/surjyTv7a7AsO1vt+bceR80RJQAJwNBMIgBDfASglo8ATOA5WItt8YsZVl71Q8s/f5Zlo0Yn2LOBH3KwLAd+T9P+RBw1vgQgARiaSQRgiI8A1PIRgAk8B2OxLe+724pTj7X8CH+z5822TLBXgzPkYFgOzp6m/ak4anwJQAIwNJMIwBAfAajlIwATeA70YlsuXGjFF4+w7DU7Wv7e/RLs0eANOdCWg7enaX8yjhpfApAADM0kAjDERwBq+QjABJ4DvdgWl33bynv+ZvnRX7Zs2LAEezR4Qw605eDtadqfjKPGlwAkAEMziQAM8RGAWj4CMIHnQC625d/+bMVZJ1p+wtcsW3+jBHszuEMOpOXg7mnan46jxpcAJABDM4kADPERgFo+AjCB50AttuWCp6046XDL3vZuy9/+vgR7MvhDDpTl4O9p2luAo8aXACQAQzOJAAzxEYBaPgIwgedALbbF986z8oF/WH7UKZbleYI9GfwhB8py8Pc07S3AUeNLABKAoZlEAIb4CEAtHwGYwHMgFtvy9putOPdLPW/5st4GCfaiPYYcCMv22NO0twJHjS8BSACGZhIBGOIjALV8BGACz9SLbfn0Uz2f9rHrHpbv9O4Ee9A+Q6a2bJ89TXtLcNT4EoAEYGgmEYAhPgJQy0cAJvBMvdgWF51r5aMP+Xv+nTRkn/pdelhSWyY4/G05JI6aw0IAEoChmUQAhvgIQC0fAZjAM+ViW/75T1ZMPtXyL5xt2TovSHDr22vIlJbttadpbw2OGl8CkAAMzSQCMMRHAGr5CMAEnqkW2/Kp+VZ84TDLdt/X8jfvmuCWt9+QqSzbb0/T3iIcNb4EIAEYmkkEYIiPANTyEYAJPFMttsV3zrLyidmWHz7JqoWoG75SWXaD3fL7iKPmiBOABGBoJhGAIT4CUMtHACbwTLHYljf/3orzz7B80jmWjVsnwa1uzyFTWLbnnqa9VThqfAlAAjA0kwjAEB8BqOUjABN4qhfb8sl5/tTvJyzb44OWv3HnBLe4fYdUW7bvnqa9ZThqfAlAAjA0kwjAEB8BqOUjABN4qhfb4n/OtHL+PMsPO6FrnvpdeljUlgkOd0cMiaPmMBGABGBoJhGAIT4CUMtHACbwVC625W039bzqd9LXu+qpXwJQOzGVc1J7yzprNAKQAAzNWAIwxEcAavkIwASeqsW2XLig56nfd+5p+VvfleCWtv+QKsv239O0txBHjS8BSACGZhIBGOIjALV8BGACT9ViW0w938p7/mb50V8e8m/4vLLDoLJMcJg7akgcNYeLACQAQzOJAAzxEYBaPgIwgadisS3vudOK0z9j+efOtGyDjRPcys4YUmHZGXua9lbiqPElAAnA0EwiAEN8BKCWjwBM4BldbMtFi6z40pGWbf8Gy9+zb4Jb2DlDRi07Z0/T3lIcNb4EIAEYmkkEYIiPANTyEYAJPKOLbXHlNCtv+LXlJ/jZv+GrJLiFnTNk1LJz9jTtLcVR40sAEoChmUQAhvgIQC0fAZjAM7LYlg//04qTP2X5kSdb9tLNE9y6zhoyYtlZe5r21uKo8SUACcDQTCIAQ3wEoJaPAEzg2epiWxaFFV/9nGUbb2r5vh9JcMs6b8hWLTtvT9PeYhw1vgQgARiaSQRgiI8A1PIRgAk8W11si2tnWvmTqZaf6B/3NnL1BLes84Zs1bLz9jTtLcZR40sAEoChmUQAhvgIQC0fAZjAs5XFtpwzy4rPf8Lyjxxl2dbbJ7hVnTlkK5aduadpbzWOGl8CkAAMzSQCMMRHAGr5CMAEnq0stov/29/rb/gID8BPJ7hFnTtkK5adu7fpbjmOGlsCkAAMzSQCMMRHAGr5CMAEnv1dbMs/XW/FRedaftI3LBuzRoJb1LlD9teyc/c07S3HUeNLABKAoZlEAIb4CEAtHwGYwLM/i2351JP+1O+hlu35Qcvf8LYEt6azh+yPZWfvadpbj6PGlwAkAEMziQAM8RGAWj4CMIFnfxbb4nuTrXz4AcuPOMmqxYWvFQX6Y4ndygVw1MwOApAADM0kAjDERwBq+QjABJ51F9uej3s73vLPn2XZCzZMcEs6f8i6lp2/p2n3AEeNLwFIAIZmEgEY4iMAtXwEYALPOottuXhxz8e9bbuD5bt398e99XUI6lgmOIRDbkgcNYeUACQAQzOJAAzxEYBaPgIwgWedxba46odWXvszP/t3tmWrdPfHvRGACSbhc4asMyfT34rO/wkEIAEYmsUEYIiPANTyEYAJPJsttuXsx3re8+8Tn7Vs820S3IKhM2Qzy6Gzp2n3BEeNLwFIAIZmEgEY4iMAtXwEYALPZovt4nNPsWy11Sw/6IgEP31oDdnMcmjtbbq9wVFjSwASgKGZRACG+AhALR8BmMCzr8W2/L8brJhytuUn855/degJlzpKzbfBsblRnS0IQAKwzjxZ6TYEYIiPANTyEYAJPFe22JYLnrbiC/6ef7vvZ/mb3pHgJw+9IQkXzTHFUeNIABKAoZlEAIb4CEAtHwGYwHNli20x9Xwr/3Gn5Uf7x77xnn+15AmXWkxNN8KxKVGtDQhAArDWRFnZRgRgiI8A1PIRgAk8e1tsy/vutuLUYyz/7BmWbfiiBD91aA5JuGiOK44aRwKQAAzNJAIwxEcAavkIwASez11sy8Lf8++/jrVsi1dZvscBCX7i0B2ScNEcWxw1jgQgARiaSQRgiI8A1PIRgAk8n7vYFtdcaeVVP7B80jmWjVg1wU8cukMSLppji6PGkQAkAEMziQAM8RGAWj4CMIHn8ottOWe2v+ffoZZ/9GjLXrldgp82tIckXDTHF0eNIwFIAIZmEgEY4iMAtXwEYALP5Rfb4punNX5CFYB89V+AcOm/WW/XwFHjSAASgKGZRACG+AhALR8BmMBz6WJb3naTFZNPtfwkf8+/Nccl+ElDf0jCRXOMcdQ4EoAE4LKZNH78+M38Py7wyzp+mZPn+YGXXXbZ7X1NNQJQc0dcOgoPbBpPHDWO1SiV5ROz/mXFpMMs23l3y9/2Ht3gXTYS81JzwHHUOBKABODyAfhL/48p06ZNu2jChAl7edwd639/LQGoubPVGYUHtjpKzbfBsblR3S0qyznfPc/KW/5g+fGnWZYPq3tVtnuOAPNSMyVw1DgSgARgYyb52b91/Y87t9pqq3GTJk0qlvy/h4qieOP3v//9u1c23TgDqLkjcgYQR62AbrRR85+wucce3POGzy9+mW7gLhyJcNEcdBw1jgQgAdiYSfvss892HnsX+xm/LZZOLY/CG6qzgNOnT/8VAai5wzUbhQe2ZkL1vo9jPadmW/n93/JzTrLF66xv+QcmNtuc7zcRYF5qpgiOGkcCkADsKwBv9AXgGAJQc2erMwoPbHWUmm+DY3OjOlsUN/zabPp3LDvxXMtWH1XnKmzThwDzUjM9cNQ4EoAEYGMm9ecpYP/9wF39bOGu1fXGjRs3YvLkyYfOnz9fMyO7fJQRI0bYM8880+UK8d3HMW5YPDnP5h7xQRt78Kcsf91b4gMygjEvNZMAR41jNcro0aNt4sSJ586ePbux8PiLP2dOnTp1pu4ntPdIWXvfvIG7dR6BV/u/CC7wg3+B/31v/8nH8CKQgfOvfhL/stV44xh3LL7nL/x45EFb8/Nn2rx58+IDMgL3b9Ec4P6tgeQMIGcAl80k/z3Al/uZvSn+P9b2yxP+9w/7C0Bu62uq8SIQzR1x6Sg8sGk8cYw5lvfcacXpn7H882fbGi/b3ObOnRsbkGs3BJiXmomAo8aRACQAQzOJAAzxPe/KPLBpPHFs3bEsFlvxpaMse9VrLX/vfkRL65Tcv4V2yw/F/VsDSwASgKGZRACG+FggtHzLRmOBaB22+OWPrbz6R5ZPOseyVUYQgK1Tcv8W2hGAekwCkAAMzSoCMMTHAqHlIwCDnuWcWVaccIjlHz/Osi235WnLoOdzr84/TDSgOGocCUACMDSTCMAQHwGo5SMAg57F5K9ULwO0/CNHYRm07O3qhIsGFUeNIwFIAIZmEgEY4iMAtXxES8Cz/PMfrfjm6Zaf5O/5t+Y4LAOWK7sq4aJBxVHjSAASgKGZRACG+AhALR/R0qJnuXChFZM+Ydmue1j+1netMAqLbYuovVwNS40ljhpHApAADM0kAjDERwBq+QjAFj2Lyy+08o5bLD/uK5b5U8DLf7HYtohKAOrgnjMSc1JDSwASgKGZRACG+AhALR8B2IJn+c/7rDjlSI+/0yzbeFPmZAuGda9CuNSV6ns7HDWOBCABGJpJBGCIj8VWy0cA9tOzLAorTjvespe83PLxB/V6bRbbfqL2sTmWGkscNY4EIAEYmkkEYIiPANTyEYD99Cyu+7mVP77M8hO/btnI1QjAfvr1d3PCpb9ivW+Po8aRACQAQzOJAAzxEYBaPgKwH57l3Dk97/l30Kcan/qxsi8W236gNtkUS40ljhpHApAADM0kAjDERwBq+QjAfngW559p5cKnbdghx/d5LRbbfqASgDqsPkZiTmqYCUACMDSTCMAQHwGo5SMAa3qWt99sxTdO8ad+/T3/xq1DANZ0i25GuEQFe66Po8aRACQAQzOJAAzxEYBaPgKwhmf57LNWnPhJy976Tsvf/r6m12CxbUpUewMsa1PxjxINVZ+jEIAEYGiaEYAhPgJQy0cA1vAsZlxi5c03WH78Vy0bNqzpNYiWpkS1N8CyNhUBqKEiAJs4ZgPgPGR/BAGoPbQsEBpPHHt3LB/+pxUnH2750V+27MUvq4WNZS2mWhthWYup6UY4NiWqtQFnADkDWGuirGwjAjDExxlALR9nAPvw9PuqFWecYNmGL7J8v4/WlmexrU3VdEMsmxLV2gDHWkxNNyIACcCmk6SvDQjAEB8BqOUjAPvwLK6/2sorLrT8pG9YttrqteVZbGtTNd0Qy6ZEtTbAsRZT040IQAKw6SQhAENE/boyD2z94lrpxjiuSNN4z7/PH2r5hw6zbNvX9wsZy35x9bkxlhpLHDWOBCABGJpJnAEM8XEGUMvHGcCVeBbfOt3KRYts2MeP67c4i22/yfiHiY6s15GYkxpgApAADM0kAjDERwBq+QjAXjzLW35vxbfP8Kd+/T3/1hzXb3EW236TEYA6MgIwoSUBSACGphcBGOIjALV8BOBzPMsFT1nxhU9Y9p59Ld9xl5a0CcCW2AgXHRuPk4ksCUACMDS1CMAQHw9sWj4C8DmexSXftPKf91r+6S9a9WDfyhcB2Ipa79fBUmOJo8aRACQAQzOJAAzxEYBaPgJwOc/yrjsab/uSf/4sy16wYcvSLLYt03H/1tGtMBJzUgNLABKAoZlEAIb4WCC0fATgEoFykX/c20mfsmyHnSzfbe+QMottiI9w0fFx/xZbEoAEYGhKEYAhPgJQy8cCsUSg+NGlVv7pd5Z/1j/ubfjwkDIBGOIjAHV83L/FlgQgARiaUgRgiI8A1PKxQLhA+eB9VnzpyH593Ftfh4EA1E1SLDWWOGocCUACMDSTCMAQHwGo5ev6ACyLwoqvHGfZSze3fPxBEl0WWwljYxAsNZY4ahwJQAIwNJMIwBAfAajl6/oALK650sqfX2H5pHMsW3WkRJfFVsJIAOoYCWmRJQFIAIamEgEY4iMAtXxdHYDl7Mca7/mX+6d9ZFtuK5MlAGWUhIuIkjmpgSQACcDQTCIAQ3wEoJavawPQ74dWnHOyZaPHWn7Qp6SqLLY6Tiw1ljhqHAlAAjA0kwjAEB8BqOXr2gAsfvsLK6/4ruUnft2yUaOlqiy2Ok4sNZY4ahwJQAIwNJMIwBAfAajl68oAbDz1O+kwy//z05Zt8xq5KIutjhRLjSWOGkcCkAAMzSQCMMRHAGr5ui4AG0/9fm2SZWuNs/zAw5NostjqWLHUWOKocSQACcDQTCIAQ3wEoJav6wKwuPZnVv54as+rflcflUSTxVbHiqXGEkeNIwFIAIZmEgEY4iMAtXxdFYDlvx6x4sRPWv4xf9XvVrpX/T73kLDY6iYplhpLHDWOBCABGJpJBGCIjwDU8nVNADbe8PmMEyx7wUaWH3BIIsWeYVlsdbxYaixx1DgSgARgaCYRgCE+AlDL1zUBWFz9Y3/D5x/4U79nWzZy9USKBKAalnDRiOKocSQACcDQTCIAQ3wEoJavKwKwfPRBK076lOWf+Jxlm2+TSPDfw7LY6oix1FjiqHEkAAnA0EwiAEN8BKCWb8gHYFkstuK0z1q28aaWf2BiIr0Vh2Wx1TFjqbHEUeNIABKAoZlEAIb4CEAt35APwOKqH1p5zU8s/4I/9Sv6rN9mh4DFtplQ/e9jWd+qry1x1DgSgARgaCYRgCE+AlDLN6QDsHz4AStOPsLyw/19/16+VSK55w/LYqujxlJjiaPGkQAkAEMziQAM8RGAWr4hG4DlYn/q9yv+di8v2dzyfQ5OpNb7sCy2Om4sNZY4ahwJQAIwNJMIwBAfAajlG7IBWMz4npW//43lJ5xp2YhVE6kRgKlhCReNMI4aRwKQAAzNJAIwxEcAavmGZACWf7vNirMmWX7cVxov/hjoLxZbnTiWGkscNY4EIAEYmkkEYIiPANTyDbkALJ+c72/58knLdtnD8p13T6TV97Astjp2LDWWOGocCUACMDSTCMAQHwGo5RtSAej3LSvOO9Xs2WcsP8w/9SPLEmkRgAMFS7hopHHUOBKABGBoJhGAIT4CUMs3pAKwuHamlTMu8bd8OcuyMWskkmo+LIttc6O6W2BZV4p/lGik+h6FACQAQ/OMAAzxEYBaviETgOWD91lxylGWH+pv+rzFqxIp1RuWaKnnVGcrLOsoNd8Gx+ZGdbYgAAnAOvNkpdsQgCE+AlDLNyQCsPSnfKv4y165veV7fSiRUP1hWWzrWzXbEstmQvW+j2M9p2ZbEYAEYLM50uf3CcAQHwGo5RsSAVhc+i0r77rD8mNPtWz48ERC9Ydlsa1v1WxLLJsJ1fs+jvWcmm1FABKAzeYIARgS6t+VeWDrn9fKtu5Ux/Lm31vx7dP9/f6+Ztl6G2gwgqN0qmVwt5NcHUsNK44aRwKQAAzNJM4Ahvg4A6jl6+gzgOWcWVaceLhl/kkf+et3SiTT/2FZbPtvNtT+YaIT0IzEnNQ4EoAEYGgmEYAhPgJQy9exAVgWhRVf+4Jla6xl+cFHJlJpbVgW29bcersWlhpLHDWOBCABGJpJBGCIjwDU8nVsABY/mWrlb3/R89TvaqsnUmltWBbb1twIQJ3bc0diTmpsCUACMDSTCMAQHwGo5evIACz//MfGGz7nR3/Zsk1emkik9WFZbFu3I1x0dsuPxJzUuBKABKDtvffeh/lE+KhPqcIv3nTladOnT7+4zhQjAOso1d+GB7b6Vn1t2SmO5aMPWfGlT1u270cs36F9fu+PxVYzDwlAHNMIaEYlAAlAmzBhwk4LFiz4w4wZM+btueeeLxw2bNhNPr1eO23atHuaTTMCsJlQ/77fKeHSv70a+K07wbFcuMCKLx9t2ebbWO4B2K5fnWDZrnYEYJojw5zUuBKABODzZtL48eNv9Ylx6NSpU69tNs0IwGZC/fs+D2z981rZ1u3uWH3Ob/mt06184nHLjzipLd7vr1MtNTNmYEZp93k5MArxn4Jj3LAagQAkAFeYSX428O2+OH3H/+fL/Qzg082mGQHYTKh/3+eBrX9enRotxcwrrPzljyz/3FctG7uWZqcTjcKc1MFiqbHEUeNIAHZBAPoZvet9umz2nCmT+X+XftnWQ++f1fd8u639j5/4ZR//f7+rM8UIwDpK9bfhga2+VV9btrNj+Zf/s+LcL1l+1CmWbfoyzQ4nHKWdLRPudpKhsdSw4qhxJAC7IADrTBWPvy2XxN/BHn9X93UdP0u4a1EUu1bbjBs3bsTkyZMPnT9/fp0fwzZNBEaMGGHPPPMMTkGBdnVc7C/6mHf8RFtt/4/bqjvtFtzLgbl6u1oOzN5rfwqWGk8cNY7VKKNHj7aJEyeeO3v27MbCk+f5TP/1r5m6n9DeI1Vnwrr6a5999tnCg+5KR/iox99V/cHgDGB/tJpvy79smxvV2aIdHcuFC6049RjLXrqF5ft/rM5utMU27WjZFjAt3AgsW0Dr5So4ahw5A8gZwOqp35/7dNreL/f6ZelTw8fWiUECUHNHXDoKD2waz3ZzbLzo4/wzrJz1qOWf/qK/6GMVzY4OwCjtZjkAu5zsR2CpocVR40gAEoChmUQAhvied2Ue2DSe7eZY/GKGlT+73D/p48zGx7110le7WXaS3XNvK5aao4ejxpEAJABDM4kADPERgFq+ZaO10wJR3nyjFd88zfIjT/anfzdPtMfphm0ny3R7OTAjY6lxxlHjSAASgKGZRACG+AhALV/bBWB51x1WnPl5yz98uGXbvzHR3qYdlsVW54ulxhJHjSMBSACGZhIBGOIjALV8bRWA5UMP+Is+jrXsvftZ/rb3JNrT9MOy2OqMsdRY4qhxJAAJwNBMIgBDfASglq9tArCcM8uK//L4e91bLN/jgER7OTDDstjqnLHUWOKocSQACcDQTCIAQ3wEoJavLQKwfOpJK077jGUveqllB36y8XFLnfzFYqs7elhqLHHUOBKABGBoJhGAIT4CUMs36AFYPvusFWdNMltlhOWHfratP+O3Lj2LbV2p5tth2dyozhY41lFqvg0BSAA2nyV9bEEAhvgIQC3foAZgWRRWfut0K//1iH/M25csW3Vkor0b2GFZbHXeWGoscdQ4EoAEYGgmEYAhPgJQyzdoAdh4o+fLvm3lrX+0/LhTLRuzRqI9G/hhWWx15lhqLHHUOBKABGBoJhGAIT4CUMs3aAFY/Oz7VvqbPefHevytu36ivRqcYVlsde5Yaixx1DgSgARgaCYRgCE+AlDLNygBWFw708rp3+l52tdf+DHUvlhsdUcUS40ljhpHApAADM0kAjDERwBq+QY8AItf/sjKH15s+SdOsOzlWyXam8EdlsVW54+lxhJHjSMBSACGZhIBGOIjALV8AxqAxU/9aV9/6jf/1CTLNn15oj0Z/GFZbHXHAEuNJY4aRwKQAAzNJAIwxEcAavkGJAAbL/iYcYmVv7rS8iNO8qd9X5JoL9pjWBZb3XHAUmOJo8aRACQAQzOJAAzxEYBavuQB2Ii/6VOsvOFXlh95smUbvijRHrTPsCy2umOBpcYSR40jAUgAhmYSARjiIwC1fEkDsPE+f5d808pbbvT4+6JlL9gw0a1vr2FZbHXHA0uNJY4aRwKQAAzNJAIwxEcAavmSBWBZLLbywq9b+bfbes78rfOCRLe8/YZlsdUdEyw1ljhqHAlAAjA0kwjAEB8BqOVLEoDlokVW/s+ZVt5/T0/8rbV2olvdnsOy2OqOC5YaSxw1jgQgARiaSQRgiI8A1PLJA7Bc8LQV3/6q2axHe17wMXbNRLe4fYdlsdUdGyw1ljhqHAlAAjA0kwjAEB8BqOWTBmD56ENWnPslM4++/GPHWjZqTKJb297Dstjqjg+WGkscNY4EIAEYmkkEYIiPANTyyQKw/POfrPjW6Za9YWfL9j7QsmHDEt3S9h+WxVZ3jLDUWOKocSQACcDQTCIAQ3wEoJYvHICNt3mZebmVP77Msv0/bvkOOyW6hZ0zLIut7lhhqbHEUeNIABKAoZlEAIb4CEAtXygAy4ULrLzgHCvvut3yQ463bJPNEt26zhqWxVZ3vLDUWOKocSQACcDQTCIAQ3wEoJav5QAsH3vYim982Wz11S2f6L/v14Uv9ljZoWCx1U1SLDWWOGocCUACMDSTCMAQHwGo5WspAMvbb7Zi8lcse+2bLZtwsGXDhye6VZ05LIut7rhhqbHEUeNIABKAoZlEAIb4CEAtX78CsHz2WSt/cpmVV/3Qsv0+avmb3pHo1nT2sCy2uuOHpcYSR40jAUgAhmYSARjiIwC1fLUDsPz77Vb47/vZqiMt/9Bhlm28aaJb0vnDstjqjiGWGkscNY4EIAEYmkkEYIiPANTyNQ3A6o2dyysusvI3V1n23v0se/v7uvotXurws9jWUaq3DZb1nJpthWMzoXrfJwAJwHozZSVbEYAhPgJQy9dnAJa33WTFReearb2u5Qd8wrL1N0r004fWsCy2uuOJpcYSR40jAUgAhmYSARjiIwC1fL0GYPnkPCun/o+Vf7resr38TZ3fvKtleZ7oJw+9YVlsdccUS40ljhpHApAADM0kAjDERwBq+VYIwCfmzLHy99d5/J1v5u/pl//Hxy0bt26inzh0h2Wx1R1bLDWWOGocCUACMDSTCMAQHwGo5WuMVhaFrXb7TfbkZf9jtvBpy/b8UM9bvGRZgp829IdksdUdYyw1ljhqHAlAAjA0kwjAEB8BKOSrws/8ad7iR5da5p/qYe/cy7I3vt2yVVYR/pTuG4rFVnfMsdRY4qhxJAAJwNBMIgBDfASggG/58LOnn7LsXXvbGrvtafOe9gjkKyzAYhsmXDYAlhpLHDWOBCABGJpJBGCIjwAM8JWLF5v93/82zvjZk/M9/MZb5m/mXJ3xY4EIwD7nqlhiqRPQjMSc1DgSgARgaCYRgCE+ArAFvvKBe6y8/morb/i1mX9sW1Y91fumXVZ4qpcFogXYlVwFSyx1ApqRmJMaRwKQAAzNJAIwxEcA1uQr5z7uwXdtI/zs0Qct224Hy3Z4m9nmW/tbugzDsaZjK5ux2LaigWm1DQAAGPJJREFU1vt1sNRY4qhxJAAJwNBMIgBDfIRLH3zlMwvNbvm9FVX0/eX/zDbbohF92fYefyNX7xOeBUI3L7HEUiegGYk5qXEkAAnA0EwiAEN8BOByAuWiZ83+caeVd9zil1vN7rrDzN+3L9thp57L2uvVxmaBqE3VdEMsmxLV3gDL2lT8A09D1ecoBCABGJpmBGCIr6sDsCz8RRz33b0k+G4x+/vtZiNWtewVW/tTu9tY5hdbb4OW3r+PhVY3L7HEUiegGYk5qXEkAAnA0EwiAEN8XRGAPkfMnnjc7MH7rPTLsj//ea/f+/wj2V7xykbsNYJvwxe1FHzPhWSB0M1LLLHUCWhGYk5qHAlAAjA0kwjAEN+QCcDG07dV5D3+Lysfn9X40x572Moq8h6839+mZZ7ZOi8w22gTyzbcuCf0NtzE7IX+3728iCOqygIRFfz39bHEUiegGYk5qXEkAAnA0EwiAEN8bR2AjffZe3Ku2bzq8oSV1Z/ze/5u8/2/H5/dE3pzPPjmzvF98Y9aW3Oc2VprNy7Z2lXwVaH3IrMNNrZs1ZFarD5GY4HQUWOJpU5AMxJzUuNIABKAoZlEAIb4BjwAy4X+ytrHH+s5W9eIOo+5uUuCrvpznofckuCzp+b7B+v607cjVzMbs4bZ6LGNS1b9fYz/fU2PvLXWWRJ8/ufYNS0b9vy3ZNEK1RuNBaKeU52tsKyjVG8bLOs5NdsKx2ZC9b5PABKA9WbKSrYiAEN88gBsBN7DD5jNesTK2R56sx5b9qfNerTnDN4qI8zWWKsn6vzSCLqx1d/XbIRdtuRPG90Telm1fYd9sUDoDhiWWOoENCMxJzWOBCABGJpJBGCIr+UAXPbCivv9UzH8kzGs+tMv9siDZqv5Gbvq9+3GredPw67b81Yq1Vuo+J9W/XcVfZk/XTuEv1ggdAcXSyx1ApqRmJMaRwKQAAzNJAIwxFc7ABu/j3fPX6287SYr7/5rI/gaZ/PW29CyF77YbONNLfOLbfySxu/hDfXAa6bOAtFMqP73saxv1WxLLJsJ1fs+jvWcmm1FABKAzeZIn98nAEN8fQZgWT19e9ufGhe73d8nz3+/LttyW7OXbdkTex5+A/nCCu2eph2NBULniyWWOgHNSMxJjSMBSACGZhIBGOJb4crVWb7V7/u7PXnjdVb+2aPPP/PWXrq5ZVttZ9krt2uc3ctyf988vpoKsEA0Jaq9AZa1qZpuiGVTolob4FiLqelGBCAB2HSS9LUBARjia1y5/Je/YOM3V1n521/4U7ceeFtv34i+xqdhrD4q/gO6cAQWCN1BxxJLnYBmJOakxpEAJABDM4kAbI2vXLTI7OYbrbhupln1ubceffmOu9jYN7zV5s1/srVBudYyARYI3WTAEkudgGYk5qTGkQAkAEMziQDsH1/pT+uW1/nZvut/2Xg7lsyjL3vjzpb5e+pVXzyw9c9zZVvjqHFkTuocsdRZcv/WWBKABGBoJhGA9fiqj0QrrrjIrHpBx6tea/mbd/WneF/1vN/p44GtnmezrXBsJlT/+1jWt2q2JZbNhOp9H8d6Ts22IgAJwGZzpM/vE4B98zV+v2/G96z8w297zva9a7xl1Zswr+SLB7bQdFx2ZRw1jpy10jliqbPk/q2xJAAJwNBMIgB75yv9s3HLK6dZee1My7Z/o2Xv3c+ydddvas0DW1OiWhvgWIup1kZY1mKqtRGWtZiaboRjU6JaGxCABGCtibKyjQjAFWXKp5+y8uc/sPKqH/pTvFtb/v7/6Hmj5ppfPLDVhGqyGY4aR85a6Ryx1Fly/9ZYEoAE4LKZtMcee6w3fPhwf8dhu37atGl71pliBGCPUlkstvKaK6388WVm67/Q8r0+aNlmW9YhXGEbHtj6TdbrFXDUOBItOkcsdZbcvzWWBCABuGwmjR8//nKfELM86tYmAOvfwapX9hb/8zWzeXMt3/c/zV7p7+PX4mft8sBW372vLXHUOBItOkcsdZbcvzWWBCAB2JhJEyZMOMj/qE5Z3eoB+D4CsPkdrCwKK3/9Uyu/f4Flb3qHZXv4Wb9VV21+xT624IEtxLfsyjhqHIkWnSOWOkvu3xpLApAAND/z5x8sa5fOnz//zWPGjNmXAGx+56o+p7e44Gz/uLaHLD/wk5b5p3YovnhgUyjyfooaxZ5RmJM6TSw1ljhqHAnALghAD7zrfbps9pwpk/l/+8fPLt5u2LBhF/rfj/Ozfjf6tgf639/LGcDe72Aex/4mzldbedm3el7dO+Fgy1ZbXXNvZLHFUSagG4jFFkudgGYk5qTGkQDsggDsa6rsv//+Y5955pm7fJt5S7Yb43+u5pffeQS+o7fr+tPFuxZF4e9kbDZu3LgRkydPPtTPHmpmZBuPUsyZZU996wxb9PfbbdTEo22V7XaQ39oRI0aYHw/5uN02II66I44lljoBzUjMSY1jNcro0aNt4sSJ586ePbux8OR5PnPq1Kn+GaXd8VWdCeNriYDH3Yd4Cvj506G89Y/+Qo8zLNvi1ZZ9YKJlo8cmmTP8y1bDiqPGsRoFSyx1ApqRmJMaR84AdvkZwOdOIwJwRZHGU76/nGHlDy627D8+bvnrd9Lc81YyCg9sGl4cNY4EoM4RS50l92+NJQFIAIZm0lB+H8By0bNWfm+ylTffaPmhn7XsJa8IWdW5Mg9sdZSab4Njc6O6W2BZV6r5dlg2N6qzBY51lJpvQwASgM1nSR9bDNUALOfPteK8U82enG/5YZ+zbNy6Iae6V+aBra5U39vhqHHkrJXOEUudJfdvjSUBSACGZtJQDMDyoQes+PrJZhtuYvnBR1g2snpNzMB88cCmccZR40i06Byx1Fly/9ZYEoAEYGgmDbUALP9ykxWTv2LZW95p2fsPsCzPQz79vTIPbP0V6317HDWORIvOEUudJfdvjSUBSACGZtJQCsCi+izf6d+xbH9/sccb3hZyafXKPLC1Krfi9XDUOBItOkcsdZbcvzWWBCABGJpJQyEAGx/pNvV8K2+81vJDPmPZZtUn4g3OFw9sGnccNY5Ei84RS50l92+NJQFIAIZmUqcHYCP+Lj7Pqqd+8yNPtmzd9UMe0SvzwBYV7Lk+jhpHLHWOWOosuX9rLAlAAjA0kzo5ABvx991vWHnHLZZ/+kuWrT0wr/TtC5wHttB0XHZlHDWORIvOEUudJfdvjSUBSACGZlKnBuAK8XeUx98Avc1LM2we2JoJ1fs+jvWc6myFZR2lettgWc+p2VY4NhOq930CkACsN1NWslUnBmAj/i4618q/3mr5Uad4/K0TMlBemQc2jSaOGkfOWukcsdRZcv/WWBKABGBoJnVaADbi78JzrLzzLz3xt9baof1XX5kHNo0ojhpHokXniKXOkvu3xpIAJABDM6mTArAsFlt5wdet/PvtHn/+tG+bxR8LRGgqrnBlFggsdQK6kZiXGkscNY4EIAEYmkmdEoCN+JviZ/7u/qvH3xctW7O9zvwtPQg8sIWm47Ir46hx5B8lOkcsdZbcvzWWBCABGJpJnRCAjfj7ztlW/uNv/mrf9o0/FojQVOQMoI4PSywTCWiGJQA1jgQgARiaSR0RgPf+3YopZ1t++CQ/8zcutL+pr8wDm0YYR40j/yjROWKps+T+rbEkAAnA0EzqhACsdrA6C5jlw0L7OhBX5oFNo4yjxpFo0TliqbPk/q2xJAAJwNBM6pQADO3kAF6ZBzYNNo4aR6JF54ilzpL7t8aSACQAQzOJAAzxPe/KPLBpPHHUOBItOkcsdZbcvzWWBCABGJpJBGCIjwDU8i0bjQVCB4slljoBzUjMSY0jAUgAhmYSARjiIwC1fARgAk8WWx0qlhpLHDWOBCABGJpJBGCIjwDU8hGACTxZbHWoWGoscdQ4EoAEYGgmEYAhPgJQy0cAJvBksdWhYqmxxFHjSAASgKGZRACG+AhALR8BmMCTxVaHiqXGEkeNIwFIAIZmEgEY4iMAtXwEYAJPFlsdKpYaSxw1jgQgARiaSQRgiI8A1PIRgAk8WWx1qFhqLHHUOBKABGBoJhGAIT4CUMtHACbwZLHVoWKpscRR40gAEoChmUQAhvgIQC0fAZjAk8VWh4qlxhJHjSMBSACGZhIBGOIjALV8BGACTxZbHSqWGkscNY4EIAEYmkkEYIiPANTyEYAJPFlsdahYaixx1DgSgARgaCYRgCE+AlDLRwAm8GSx1aFiqbHEUeNIABKAoZlEAIb4CEAtHwGYwJPFVoeKpcYSR40jAUgAhmYSARjiIwC1fARgAk8WWx0qlhpLHDWOBCABGJpJBGCIjwDU8hGACTxZbHWoWGoscdQ4EoAEYGgmEYAhPgJQy0cAJvBksdWhYqmxxFHjSAASgKGZRACG+AhALR8BmMCTxVaHiqXGEkeNIwFIAIZmEgEY4iMAtXwEYAJPFlsdKpYaSxw1jgQgARiaSQRgiI8A1PIRgAk8WWx1qFhqLHHUOBKABGBoJhGAIT4CUMtHACbwZLHVoWKpscRR40gAEoChmUQAhvgIQC0fAZjAk8VWh4qlxhJHjSMBSACGZhIBGOIjALV8BGACTxZbHSqWGkscNY4EIAEYmkkEYIiPANTyEYAJPFlsdahYaixx1DgSgARgaCYRgCE+AlDLRwAm8GSx1aFiqbHEUeNIABKAoZlEAIb4CEAtHwGYwJPFVoeKpcYSR40jAUgAhmYSARjiIwC1fARgAk8WWx0qlhpLHDWOBCABGJpJBGCIjwDU8hGACTxZbHWoWGoscdQ4EoAEYGgmEYAhPgJQy0cAJvBksdWhYqmxxFHjSAASgKGZRACG+AhALR8BmMCTxVaHiqXGEkeNIwFIAIZmEgEY4iMAtXwEYAJPFlsdKpYaSxw1jgQgARiaSQRgiI8A1PIRgAk8WWx1qFhqLHHUOBKABGBoJhGAIT4CUMtHACbwZLHVoWKpscRR40gAEoChmUQAhvgIQC0fAZjAk8VWh4qlxhJHjSMBSACGZhIBGOIjALV8BGACTxZbHSqWGkscNY4EIAEYmkkEYIiPANTyEYAJPFlsdahYaixx1DgSgARgaCYRgCE+AlDLRwAm8GSx1aFiqbHEUeNIABKAoZlEAIb4CEAtHwGYwJPFVoeKpcYSR40jAUgAhmYSARjiIwC1fARgAk8WWx0qlhpLHDWOBCAB2JhJEyZM2Mtj7gv+18wvpV/eM23atPuaTTMCsJlQ/77PA1v/vFa2NY4ax2oULLHUCWhGYk5qHAlAAtD22muvbfM8v3j48OE7XXLJJY8ccMABo4YNG7Z4ypQpC5pNMwKwmVD/vs8DW/+8CECNV1+jMCd1xlhqLHHUOBKABKDtvffeF/l0+vX06dO/3d9pRQD2V6zv7Xlg03jiqHHkDKDOEUudJfdvjSUBSADa+PHj/+gT4UqPuR19Wo3xP3/iMVg9HVw9FdznFwHYTKh/3+eBrX9enAHUeHEGML0jAagz5nFSY0kAdkEAeuBd79Nls+dMmaW/67et//+f+uVev+y1YMGC4SNHjpzhYXe5R+A3mk2zpQH48MMPm/+92eZ8v4nAmDFjbN68eTgFBXAMAi53dSyx1AloRmJOahyrAFx//fWrwdbwv8/VjNpZo1Qh1NVfHog/coDv+4s+plQQ/pTwIT4ZXu///cHeYPwFI7sWRbFr9b1NNtlk1Omnn/7RrgZk5xFAAAEEEOhQgaOOOuqb995775PVzffXA8ycOnXqzA7dlX7f7K4PQA+6/Tzodvczfvt7DOZVDPrZvOv8v7/aTNO3y3zynOcReHSzbfl+c4GJEyeeMnny5OObb8kWfQngqJsfWGKpE9CMxJzUOFaj+Pp9mq/fH/OTPl35FF7XB6DPgczP+n3FJ8C7/e+L/M/rHn300cN/9atfLaozzfy6Z3gsHllnW7bpWwBLzQzBUeNYjYIlljoBzUjMSY0j9+8u+B1A3VTpfSTujDphLDWWOGocWSB0jljqLLl/Y6kS4AxgULL6ncBu+p2BIFefV8dSo4ujxrEaBUssdQKakZiTGkfu35wB1M0kRkIAAQQQQAABBDpEgDOAHXKguJkIIIAAAggggIBKgABUSTIOAggggAACCCDQIQIEYIccKG4mAggggAACCCCgEiAARZL+i7l7+fsCVh8ht/RTRt7jbyZ9n2j4rhtmjz32WG/48OG3+I5f7457dh1AcIf9lYKH+VsaVW9SXvjFp2Z5mr9d0cXBYbvm6v6eoNWnB13gl3X8MsffIPbAyy677PauARDt6G677bbq6NGjL/XhtvDL03551C0Pccu7RD+i64bx+/aH/b59vu/4+/2xcUbXAQh22O/fI3yY6r1+qw91qOblzSv78AfBj2vbIQhAwaHZa6+9tvUHtYs9WHa65JJLHjnggANGDRs2bPGUKVMWCIbvyiH8Dnq5P8jN8nBZmwDs/xTwf5Ds5B9t+IcZM2bM23PPPV/o8/EmH+W1bnlP/0frvmv4/Pul7/UU97poyT/ujvW/v7b7JGJ7vCQAd3K7n1UjueWhfp/e2/97p9jI3Xltn5cv8j2/ZMnen0oAtjYPfB6e6R8Akfs/ig+vRqhOOFxxxRWPtjZa516LABQcO/8X2UU+zK99Mn1bMFzXD+F3zoMcYUu/3OqLxfsIwPiU8IXjVg/qQ/0ti66Njza0R3CrdX0P79xqq63GTZo0qTqDav7/HvIF443f//737x7ae59279xxe/8Jfpee9pK0P2lIjp653899Hh7jJxzO8D08kwDs/3HefffdVx85cuRDfs2N3G9+/0cYOtcgAAXH0u+Uf/TF9UqPlR19uDH+5088Bqung7vy42UipG65qV//0vnz57/ZP/R8XwIwotlzXQ/qt7vjd/yvL/cHvOrpDr76ENhnn32280X2YreqnrZsfPm8vMENj/X79a/Aa13A5+IFbjuLT0/qv6GfaPi0rzOjfF6e5PPxGgKw/4ZL7stb+5/VU+fVrya83S9P+eVEd726tRE791oEYI1j53e2632z6neClv9a+rt+2/r//Klf7vXLXv6023D/18UMXywu9we5b9QYvqs26cty8eLF2/lTlRc6yHF+Z7zRtz3Q//5ezgA+f4o0m5Nu9s/lHux+4n/fx//f77pqsrW4sysJwBv9Pn0MAdgiql/N4+94N3z3qFGjdubXY/rn6Pf36hmR6vf+3uT348UEYP/8lt96yf37Dz4XD6h+L9otX+3fv8ovW7rtY62P3HnXJAAFx8wn0I98mO/75JlSDef/UjvE/6X2+m78pdII5/777z/2mWeeqX45fN6Sccb4n6v55Xdu+Y7I2N143SWLRhV/B3fjv25bPeY8Bdyq3Mqv5/F3lC+4ExYuXLhz9Xup+p8wtEd0v4+53wm+lwv9Uq3b6/vlieqFhx4xk4f23mv3zn/fb23/ff2H/TGxeiFI41k6v8/f6H9UJx666iwgASiYW37n3M+f1tjd74j7+0TKqxj0O+Z1/t/Vq4z4alHAXT/EU8Ct4fm/crfwOXmlX/uj/qBW/euWr34I+P34av9H3AX+O5MX+N/39qsew4tA+gG43Kbud6T/5wf8srMbPtHaKFxreQHOAMbmg/v9zNeWs3yN/umSXzu6wZ+BetXll19e/W5g13wRgJpDnflZv6/4gvFuH26R/3ndo48+evivfvWrRZrhu3MUArD14179srhfu/qF++pXE5b+ukL1SlZisAarB/TLPaCn+KZr++UJ//uH/QUgt9W4KpusGH8b+X/e75elZ/arubjA5+EOQLUuUP0Dxa/9NV4E0prhkuirnlKv3uZpsa/ZJ/o/9n7Q2midey0CsHOPHbccAQQQQAABBBBoSYAAbImNKyGAAAIIIIAAAp0rQAB27rHjliOAAAIIIIAAAi0JEIAtsXElBBBAAAEEEECgcwUIwM49dtxyBBBAAAEEEECgJQECsCU2roQAAggggAACCHSuAAHYuceOW44AAggggAACCLQkQAC2xMaVEEAAAQQQQACBzhUgADv32HHLEUAAAQQQQACBlgQIwJbYuBICCCCAAAIIINC5AgRg5x47bjkCCCCAAAIIINCSAAHYEhtXQgABBBBAAAEEOleAAOzcY8ctRwABBBBAAAEEWhIgAFti40oIIIAAAggggEDnChCAnXvsuOUIIIAAAggggEBLAgRgS2xcCQEEEEAAAQQQ6FwBArBzjx23HAEEEEAAAQQQaEmAAGyJjSshgAACCCCAAAKdK0AAdu6x45YjgAACCCCAAAItCRCALbFxJQQQQAABBBBAoHMFCMDOPXbccgQQQAABBBBAoCUBArAlNq6EAAIIIIAAAgh0rgAB2LnHjluOAAIIIIAAAgi0JEAAtsTGlRBAAAEEEEAAgc4VIAA799hxyxFAAAEEEEAAgZYECMCW2LgSAggggAACCCDQuQIEYOceO245AggggAACCCDQkgAB2BIbV0IAAQQQQAABBDpXgADs3GPHLUcAAQQQQAABBFoSIABbYuNKCCCAAAIIIIBA5woQgJ177LjlCCCAAAIIIIBASwL/H0s/w+2ZDYtCAAAAAElFTkSuQmCC\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fb75d557128>]"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = numpy.linspace(-5,5)\n",
    "pyplot.figure()\n",
    "f = numpy.sin(x) + 1.1*x\n",
    "pyplot.plot(x, f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Accuracy of numerical differentiation\n",
    "\n",
    "##### Discretization error\n",
    "The `diff` and `diff_wp` functions use a \"forward difference\" formula: $\\tilde f'(x) := (f(x+h) - f(x))/h$.\n",
    "Using the Taylor expansion of $f(x+h)$, we compute the discretization error\n",
    "$$ \\begin{split} \\frac{f(x+h) - f(x)}{h} - f'(x) &= \\frac{f(x) + f'(x) h + f''(x) h^2/2 + O(h^3) - f(x)}{h} - f'(x) \\\\\n",
    "&= \\frac{f'(x) h + f''(x) h^2/2 + O(h^3)}{h} - f'(x) \\\\\n",
    "&= f''(x) h/2 + O(h^2)\n",
    "\\end{split} . $$\n",
    "\n",
    "This is the *discretization error* caused by choosing a finite (not infinitesimal) differencing parameter $h$, and the leading order term depends linearly on $h$.\n",
    "\n",
    "#### Rounding error\n",
    "We have an additional source of error, *rounding error*, which comes from not being able to compute $f(x)$ or $f(x+h)$ exactly, nor subtract them exactly.  Suppose that we can, however, compute these functions with a relative error on the order of $\\epsilon_{\\text{machine}}$.  This leads to\n",
    "$$ \\begin{split}\n",
    "\\tilde f(x) &= f(x)(1 + \\epsilon_1) \\\\\n",
    "\\tilde f(x \\oplus h) &= \\tilde f((x+h)(1 + \\epsilon_2)) \\\\\n",
    "&= f((x + h)(1 + \\epsilon_2))(1 + \\epsilon_3) \\\\\n",
    "&= [f(x+h) + f'(x+h)(x+h)\\epsilon_2 + O(\\epsilon_2^2)](1 + \\epsilon_3) \\\\\n",
    "&= f(x+h)(1 + \\epsilon_3) + f'(x+h)x\\epsilon_2 + O(\\epsilon_{\\text{machine}}^2 + \\epsilon_{\\text{machine}} h)\n",
    "\\end{split}\n",
    "$$\n",
    "\n",
    "where each $\\epsilon_i$ is an independent relative error on the order of $\\epsilon_{\\text{machine}}$ and we have used a Taylor expansion at $x+h$ to approximate $f(x \\oplus h)$.\n",
    "We thus write the rounding error in the forward difference approximation as\n",
    "$$ \\begin{split}\n",
    "\\left\\lvert \\frac{\\tilde f(x+h) \\ominus \\tilde f(x)}{h} - \\frac{f(x+h) - f(x)}{h} \\right\\rvert &=\n",
    "  \\left\\lvert \\frac{f(x+h)(1 + \\epsilon_3) + f'(x+h)x\\epsilon_2 + O(\\epsilon_{\\text{machine}}^2 + \\epsilon_{\\text{machine}} h) - f(x)(1 + \\epsilon_1) - f(x+h) + f(x)}{h} \\right\\rvert \\\\\n",
    "  &\\le \\frac{|f(x+h)\\epsilon_3| + |f'(x+h)x\\epsilon_2| + |f(x)\\epsilon_1| + O(\\epsilon_{\\text{machine}}^2 + \\epsilon_{\\text{machine}}h)}{h} \\\\\n",
    "  &\\le \\frac{(2 \\max_{[x,x+h]} |f| + \\max_{[x,x+h]} |f' x| \\epsilon_{\\text{machine}} + O(\\epsilon_{\\text{machine}}^2 + \\epsilon_{\\text{machine}} h)}{h} \\\\\n",
    "  &= (2\\max|f| + \\max|f'x|) \\frac{\\epsilon_{\\text{machine}}}{h} + O(\\epsilon_{\\text{machine}}) \\\\\n",
    "\\end{split} $$\n",
    "where we have assumed that $h \\ge \\epsilon_{\\text{machine}}$.\n",
    "This error becomes large (relative to $f'$ -- we are concerned with relative error after all)\n",
    "* $f$ is large compared to $f'$\n",
    "* $x$ is large\n",
    "* $h$ is too small\n",
    "\n",
    "#### Total error and optimal $h$\n",
    "\n",
    "Suppose we would like to choose $h$ to minimize the combined discretization and rounding error,\n",
    "$$ h^* = \\arg\\min_h | f''(x) h/2 | + (2\\max|f| + \\max|f'x|) \\frac{\\epsilon_{\\text{machine}}}{h} $$\n",
    "(dropping the higher order terms), which we can compute by differentiating with respect to $h$ and setting the result equal to zero\n",
    "$$ |f''|/2 - (2\\max|f| + \\max|f'x|) \\frac{\\epsilon_{\\text{machine}}}{h^2} = 0 $$\n",
    "which can be rearranged as\n",
    "$$ h^* = \\sqrt{\\frac{4\\max|f| + 2\\max|f'x|}{|f''|}} \\sqrt{\\epsilon_{\\text{machine}}} .$$\n",
    "Of course this formula is of little use for computing $h$ because all this is to compute $f'$, which we obviously don't know yet, much less $f''$.\n",
    "However, it does have value:\n",
    "* It explains why `1e-8` (i.e., $\\sqrt{\\epsilon_{\\text{machine}}}$) was empirically found to be about optimal for well-behaved functions.\n",
    "* It explains why even for the best behaved functions, our best attainable accuracy with forward differencing is $\\sqrt{\\epsilon_{\\text{machine}}}$.\n",
    "* If we have some special knowledge about the class of functions we need to differentiate, we might have bounds on these quantities and thus an ability to use this formula to improve accuracy.  Alternatively, we could run a parameter sweep to empirically choose a suitable $h$, though we would have to re-tune in response to parameter changes in the class of functions.\n",
    "* If someone claims to have a simple and robust rule for computing $h$ then this formula tells us how to build a function that breaks their rule.  There are no silver bullets.\n",
    "* If our numerical differentiation routine produces a poor approximation for some function that we run into in the wild, this helps us explain what happened and how to fix it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Centered difference\n",
    "\n",
    "Instead of the forward difference approximation\n",
    "$$ \\frac{f(x+h) - f(x)}{h} $$\n",
    "we could use the centered difference formula,\n",
    "$$ \\frac{f(x+h) - f(x-h)}{2h} . $$\n",
    "(One way to derive this formula is to average a forward and backward difference.  We will learn a more general method later in the course when we do interpolation.)\n",
    "We can compute the discretization error by Taylor expansion,\n",
    "$$ \\frac{f(x) + f'(x)h + f''(x)h^2/2 + f'''(x)h^3/6 - f(x) + f'(x)h - f''(x)h^2/2 + f'''(x) h^3/6 + O(h^4)}{2h}\n",
    "= f'(x) + f'''(x)h^2/6 + O(h^3) $$\n",
    "showing that the leading error term is of order $h^2$, versus order $h$ for forward differences.\n",
    "A similar computation including rounding error will find that the optimal $h$ is now of order $\\sqrt[3]{\\epsilon_{\\text{machine}}}$ so the best attainable accuracy is $\\epsilon_{\\text{machine}}^{2/3}$.\n",
    "This accuracy improvement (versus $\\sqrt{\\epsilon_{\\text{machine}}}$) is significant, but we'll also see that it is twice as expensive when computing derivatives of multi-variate functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stability\n",
    "\n",
    "We use the notation $\\tilde f(x)$ to mean a numerical algorithm for approximating $f(x)$.  Additionally, $\\tilde x = x (1 + \\epsilon)$ is some \"good\" approximation of the exact input $x$.\n",
    "\n",
    "### (Forward) Stability\n",
    "**\"nearly the right answer to nearly the right question\"**\n",
    "$$ \\frac{\\lvert \\tilde f(x) - f(\\tilde x) \\rvert}{| f(\\tilde x) |} \\in O(\\epsilon_{\\text{machine}}) $$\n",
    "for some $\\tilde x$ that is close to $x$\n",
    "\n",
    "### Backward Stability\n",
    "**\"exactly the right answer to nearly the right question\"**\n",
    "$$ \\tilde f(x) = f(\\tilde x) $$\n",
    "for some $\\tilde x$ that is close to $x$\n",
    "\n",
    "* Every backward stable algorithm is stable.\n",
    "* Not every stable algorithm is backward stable.\n",
    "\n",
    "#### Example: $\\tilde f(x) = \\text{float}(x) + 1$\n",
    "The algorithm computes\n",
    "$$\\tilde f(x) = \\text{float}(x) \\oplus 1 = [x(1+\\epsilon_1) + 1](1 + \\epsilon_2) = (x + 1 + x\\epsilon_1)(1 + \\epsilon_2) $$\n",
    "and we can express any $\\tilde x = x(1 + \\epsilon_3)$.\n",
    "To see if if the algorithm is stable, we compute\n",
    "$$ \\frac{\\tilde f(x) - f(\\tilde x)}{|f(\\tilde x)|} = \\frac{(x + 1 + x\\epsilon_1)(1 + \\epsilon_2) - [x(1+ \\epsilon_3) + 1]}{\\tilde x + 1} = \\frac{(x + 1)\\epsilon_2 + x(\\epsilon_1 - \\epsilon_3) + O(\\epsilon^2)}{x + 1 + x\\epsilon_3} . $$\n",
    "If we can choose $\\epsilon_3$ to make this small, then the method will be (forward) stable, and if we can make this expression exactly zero, then we'll have backward stability.\n",
    "Trying for the latter, we solve for $\\epsilon_3$ by setting the numerator equal to zero,\n",
    "$$ \\epsilon_3 = \\frac{x + 1}{x}\\epsilon_2 + \\epsilon_1 + O(\\epsilon^2)/x $$\n",
    "which is small so long as $|x| \\gg 0$, but the first term blows up as $x \\to 0$.\n",
    "In other words, the fact that $\\epsilon_2$ can produce a large error relative to the input causes this algorithm to not be backward stable.\n",
    "In contrast, this $x\\to 0$ case is not a problem for forward stability because $\\epsilon_3 = \\epsilon_1$ yields error on the order of $\\epsilon_2$.\n",
    "\n",
    "#### Example: $\\tilde f(x,y) = \\text{float}(x) \\oplus \\text{float}(y)$\n",
    "\n",
    "Now we are interested in\n",
    "$$ \\frac{\\tilde f(x,y) - f(\\tilde x,\\tilde y)}{f(\\tilde x,\\tilde y)} $$\n",
    "and we can vary both $\\tilde x$ and $\\tilde y$.  If we choose $y=1$, then the ability to vary $\\tilde y$ is powerful enough to ensure backward stability.\n",
    "\n",
    "### Accuracy of backward stable algorithms (Theorem)\n",
    "\n",
    "A backward stable algorithm for computing $f(x)$ has relative accuracy\n",
    "$$ \\left\\lvert \\frac{\\tilde f(x) - f(x)}{f(x)} \\right\\rvert \\in O(\\kappa(f) \\epsilon_{\\text{machine}}) . $$\n",
    "This is a rewording of a statement made earlier -- backward stability is the best case."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}