update

Julia Tutorial of Data Analysis in Oceanography.ipynb

@@ -57,17 +57,17 @@
    "lastKernelId": null
   },
   "kernelspec": {
-   "display_name": "Julia 1.7.2",
+   "display_name": "Julia 1.9.0",
    "language": "julia",
-   "name": "julia-1.7"
+   "name": "julia-1.9"
   },
   "language_info": {
    "file_extension": ".jl",
    "mimetype": "application/julia",
    "name": "julia",
-   "version": "1.7.2"
+   "version": "1.9.0"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 1
+ "nbformat_minor": 4
 }

JuliaExercise.ipynb

@@ -1305,15 +1305,15 @@
    "lastKernelId": null
   },
   "kernelspec": {
-   "display_name": "Julia 1.7.2",
+   "display_name": "Julia 1.8.0",
    "language": "julia",
-   "name": "julia-1.7"
+   "name": "julia-1.8"
   },
   "language_info": {
    "file_extension": ".jl",
    "mimetype": "application/julia",
    "name": "julia",
-   "version": "1.7.2"
+   "version": "1.8.0"
   }
  },
  "nbformat": 4,

JuliaIntroArray.ipynb

@@ -0,0 +1 @@
+{"metadata":{"kernelspec":{"language":"julia","display_name":"Julia 1.9.0","name":"julia-1.9"},"language_info":{"file_extension":".jl","version":"1.9.0","name":"julia","mimetype":"application/julia"}},"nbformat_minor":5,"nbformat":4,"cells":[{"metadata":{},"cell_type":"markdown","id":"3cfe26f7-ba42-48b9-b1ca-b58133d979d8","source":["# Introduction to array programming in Julia\n","\n","\n","## Getting started\n","\n","\n","* Start the computer and boot into **Debian** (not Windows)\n","* Open Firefox and log in to the internet with your ULiège credentials\n","* Open a terminal (Application -> Utilities -> Terminal). This is how the terminal icon looks like: ![icon](img/gnome-terminal.svg)\n","* Launch Jupyter from the terminal with the command (please pay attention to uppercase versus lowercase; after typing the command you need hit the `⏎ Enter` key):\n","\n","```bash\n","jupyter notebook \n","``` \n","\n","* This should open a new window in your webbrowser. Click on `New ▼` and then `Julia 1.7` (or a higher version)\n","* Make sure that the kernel is set to `1.7`.\n","\n","\n","## What is Julia?\n","Julia is a programming language for scientific computing among others.\n","\n","It can use to:\n","   * data processing, numerical modelling\n","   * data visulization (e.g. modules `PyPlot`)\n","   * parallel processing\n","   * statistics\n","   * optimization\n","   * machine learning\n","   * ...many more\n","\n","Julia is freely available at https://julialang.org/downloads/.\n","The functionality of julia can be extended using packages.\n","\n","To load a package (e.g. `Images`) one needs to load it with the command `using`:\n","\n","```julia\n","using Images\n","```\n","\n","If the package is not installed you will get an error message:\n","\n","```\n","ERROR: ArgumentError: Package Images not found in current path.\n","- Run `import Pkg; Pkg.add(\"Images\")` to install the Images package.\n","```\n","\n","As shown in the error message, the package can be installed with:\n","\n","```julia\n","import Pkg; Pkg.add(\"Images\" )\n","```\n","\n","\n","# Julia exercise\n","\n","This lecture aims to teach the basis of array programming in julia by examples. The focus is on arrays, because:\n","\n","* Many oceanographic datasets are multidimensional arrays. For example:\n","     * satellite sea-surface temperature data are typicall 3d arrays where the dimensions are longitude, latitude and time.\n","     * output of numerical models are typicall 4d arrays where the dimensions are longitude, latitude, depth and time.\n","* The prupose of this lecture is to learn manipulating an array of data. To make it more visual, every students is asked to take a picture and transfer the picture to this computing (for example by sending yourself an email).\n","* Download the picture. Per default, Firefox places all downloaded in the folder `/home/students/Downloads`.\n","* Take note of the full path of the downloaded image, for example `/home/students/Downloads/IMG_abc_xyz.jpg` (with `.jpg`, the file extension)"]},{"metadata":{},"cell_type":"markdown","id":"bae783ec-87c2-49df-a1ec-8666d7808b8d","source":["Load the necessary modules (if some module is not installed, see the instructions above how to install it). Execute a cell by hitting `Shift-Enter`. `[*]` means that a cell is currently still running."]},{"metadata":{},"outputs":[],"cell_type":"code","id":"c1602965-6616-4b53-ba55-47dfc6c43534","source":[],"execution_count":35},{"metadata":{},"cell_type":"markdown","id":"a92c7ea0-2d38-4f89-8d0f-5e3479b15d22","source":["The variable `filename` contains the full path of the image"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"0861a046-dcd3-4945-9bb8-c3112761284a","source":[],"execution_count":69},{"metadata":{},"cell_type":"markdown","id":"60fb67e6-6051-45d7-9f45-21d2e011b442","source":["Load the file and show the size of the image (in pixels)"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"592d0a22-5266-43bb-905d-81ed35a3e186","source":[],"execution_count":37},{"metadata":{},"cell_type":"markdown","id":"370883b2-5738-47a6-af09-a1febb29f9ee","source":["An image is a matrix of red, green, blue (RGB) values. Every value is an unsigned integer of 8 bits (`UInt8`)."]},{"metadata":{},"outputs":[],"cell_type":"code","id":"24d5b225-1483-450b-a980-8aa9a33d90f8","source":[],"execution_count":38},{"metadata":{},"cell_type":"markdown","id":"8a599670-e949-477c-87c7-56362d7bb810","source":["The width of the image"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"7995f558-8557-4de8-af6f-db64bb765ef6","source":[],"execution_count":39},{"metadata":{},"cell_type":"markdown","id":"ba6f5d4b-cff7-4cc8-aa7a-a8c6944f108c","source":["The high of the image"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"6c4d7de0-9ce4-4357-97a1-7ba572012c70","source":[],"execution_count":40},{"metadata":{},"cell_type":"markdown","id":"155799d8-ab9c-410d-a5c1-52237c3e2ca1","source":["Resize the image to a width of 500 pixels keeping the aspect ratio:"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"46614c87-91df-4a96-9638-44303c0b5d89","source":[],"execution_count":41},{"metadata":{},"cell_type":"markdown","id":"16ad4cb7-caaf-436b-9ece-99a510486066","source":["<div class=\"alert alert-block alert-info\"><b>Oceanographic example:</b> datasets do not have often the same spatial resolution and one need to interpolate them on a common grid. This can be done using e.g. the module <code>Interpolations</code>.</div>"]},{"metadata":{},"cell_type":"markdown","id":"c0709da4-df20-4b4b-b006-4c9ae97218a2","source":["Instead of working with an 2D matrix of colors (RGB), it will be easier to work with an 3D array where the last dimensions is the 3 channels"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"e56a9e5e-d5c8-4786-891e-af33c4e89209","source":[],"execution_count":42},{"metadata":{},"cell_type":"markdown","id":"dc3578cf-2bd9-4597-b663-23c85a47e6a1","source":["This image values can be visualized with `imshow`. Notice the orientation of the axes. The 1st dimension is the y-axis and the 2nd dimension of the x-axis and the origin is in the top-left corner. This is a common convention for image processing, but for oceanographic data, the first dimension (resp. 2nd dimension) is typically the longitude (latitude) mapped on the x-axis and the origin is in the lower-left corner."]},{"metadata":{},"outputs":[],"cell_type":"code","id":"9c5b7e69-597e-4759-8b8b-06a6c7e65c0c","source":[],"execution_count":43},{"metadata":{},"cell_type":"markdown","id":"0d0e3e52-9162-4a90-8161-c45c116bb68a","source":["Get the color of the pixel (20,20):"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"2950fca4-96c4-4c17-83de-db5fc2f09d5b","source":[],"execution_count":44},{"metadata":{},"cell_type":"markdown","id":"f9bdffd5-2d65-4a2e-b102-1beb11b3fbf4","source":["<div class=\"alert alert-block alert-info\"><b>Oceanographic data:</b> When validating a model with in situ data, one need to extract the model data at a specific location.</div>"]},{"metadata":{},"cell_type":"markdown","id":"beadc85e-3ebd-4b0b-a5a9-d5e196b61866","source":["Add a red pixel at the location (20,20). We will from now on make always a copy of the image to preserve the original."]},{"metadata":{},"outputs":[],"cell_type":"code","id":"fd2de5ab-9d0a-4c21-a86c-3d0841a7c9f8","source":[],"execution_count":45},{"metadata":{},"cell_type":"markdown","id":"0f6de948-6b95-4805-ac84-7bdd1701d939","source":["Show a subset of the image"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"d4395930-8784-4a55-8bb2-eda2ef327631","source":[],"execution_count":46},{"metadata":{},"cell_type":"markdown","id":"5c648556-82eb-4667-8fcd-8fe21dbf99ad","source":["<div class=\"alert alert-block alert-info\"><b>Oceanographic example:</b> some dataset cover often a larger zone and one need to subset it to the domain of interest.</div>"]},{"metadata":{},"cell_type":"markdown","id":"e88a3e1b-aae1-43b3-ac1e-31c9194f3b96","source":["Set the color red to maximum value (1)"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"e7ad7177-49af-4faa-a4cf-729b467e4dd9","source":[],"execution_count":67},{"metadata":{},"cell_type":"markdown","id":"2958341a-8a3c-495d-aecb-52510db738de","source":["Add a red box from the pixel (1,1) to (20,20)"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"b7a1a409-30e1-4b2d-aff2-4a94c50d6ace","source":[],"execution_count":48},{"metadata":{},"cell_type":"markdown","id":"a01ebb9d-fa18-451a-ae34-49eb4cb58b3e","source":["Invert the color"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"7f7d09f3-a571-499d-9f64-1bcd2725ab4b","source":[],"execution_count":49},{"metadata":{},"cell_type":"markdown","id":"33d4b370-9e05-48a0-93f5-f9a1b53c451d","source":["Invert the color of a single channel"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"32b73991-f05d-4555-a222-a4629bdf3634","source":[],"execution_count":50},{"metadata":{},"cell_type":"markdown","id":"dd2eb61c-669e-4e26-b3a9-586c8ea11fb5","source":["Show a single channel"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"bdd1a04d-9968-4f11-9370-5ec66f39dd93","source":[],"execution_count":51},{"metadata":{},"cell_type":"markdown","id":"4e94696f-4b44-4a0d-afb0-8c7ed269479a","source":["Recombine an image from different channels"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"3fc711ba-7e24-43c5-9ab5-efcd6a44b961","source":[],"execution_count":52},{"metadata":{},"cell_type":"markdown","id":"63d9cdbb-99a0-4ef1-b23e-067a05b9a8c0","source":["Recombine them in a different order"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"b6c2cef1-6281-474b-abc0-87ca425ebe89","source":[],"execution_count":53},{"metadata":{},"cell_type":"markdown","id":"5504f923-fa1c-4f72-a4d7-c374494cdc08","source":["Extract part of the image where the red channel is larger than 0.7 (or any other threshold)."]},{"metadata":{},"outputs":[],"cell_type":"code","id":"b320ebd2-1b66-4d39-b3e6-672eeb0abadb","source":[],"execution_count":54},{"metadata":{},"cell_type":"markdown","id":"9564a6da-07a8-40f5-9e59-900f4a905c27","source":["<div class=\"alert alert-block alert-info\"><b>Oceanographic example:</b> find the area where the temperature is higher than a given threshold.</div>"]},{"metadata":{},"cell_type":"markdown","id":"2cf52900-d69c-4ddc-ad88-b905f3226a8a","source":["Set the red channel to zero where it is larger than 0.7."]},{"metadata":{},"outputs":[],"cell_type":"code","id":"8e59202b-b6e2-440d-bd76-cd2cd76e934a","source":[],"execution_count":55},{"metadata":{},"cell_type":"markdown","id":"fb3c1232-f1e6-4f75-af22-6bc5bde6b41d","source":["Transpose an image"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"55003708-d67c-4184-930b-427b35cbd17e","source":[],"execution_count":56},{"metadata":{},"cell_type":"markdown","id":"1d5d86e0-abcb-49bb-8681-d317fb03f5f2","source":["Reverse a single dimension"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"c7f882f9-b388-4168-a491-4d51aa56cbd1","source":[],"execution_count":57},{"metadata":{},"cell_type":"markdown","id":"c1f4fe7d-9248-4117-88b6-80ad3a27d3f3","source":["Looping throught all elements of an array: compute the average of the red channel"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"a6eac723-d294-4776-b825-2474dcabed3b","source":[],"execution_count":58},{"metadata":{},"cell_type":"markdown","id":"1a4619ad-08ad-4fc5-a39e-0d70708cba92","source":["Of course, computing the average is already implemented in julia (in the module `Statistics`)"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"e2784453-a9c7-4391-b3df-764cfeecb3fb","source":[],"execution_count":59},{"metadata":{},"cell_type":"markdown","id":"62fdf81b-debb-41ef-b31b-b6f6e3ea7201","source":["<div class=\"alert alert-block alert-info\"><b>Oceanographic example:</b> compute the average temperature</div>"]},{"metadata":{},"cell_type":"markdown","id":"aaabcc84-54ed-4ca8-93f1-8725f760c2c2","source":["Compute the finite difference along the 1nd dimension of the red channel (as an appoximation of the gradient)\n","\n","$$\n","{G_x}_{i,j} = I_{i+1,j} - I_{i,j}\n","$$\n","\n","where `I` is a 2D array, `G_x` the corresponding gradient and $i$ and $j$ are the indices."]},{"metadata":{},"outputs":[],"cell_type":"code","id":"80623cba-5cb8-4ea9-85f5-f325978db25b","source":[],"execution_count":70},{"metadata":{},"outputs":[],"cell_type":"code","id":"85467331-a3e2-4e38-8000-87d171734d8e","source":[],"execution_count":71},{"metadata":{},"cell_type":"markdown","id":"15778178-bdb1-4919-9832-239571f36d31","source":["<div class=\"alert alert-block alert-info\"><b>Oceanographic example:</b> detect temperature fronts.</div>"]},{"metadata":{},"cell_type":"markdown","id":"2ebde01a-5d38-4100-bcc0-8927f75eaad5","source":["Compute the (finite difference) Laplacian as:\n","\n","$$\n","L_{i,j} = \\frac{I_{i+1,j} + I_{i-1,j} + I_{i,j+1} + I_{i,j-1}}{4} - I_{i,j}\n","$$\n"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"6a9b3687-cf6f-4885-b1cc-f24ea96ac749","source":[],"execution_count":63},{"metadata":{},"outputs":[],"cell_type":"code","id":"80931f7d-4004-4e0c-9669-c4b608f489eb","source":[],"execution_count":64},{"metadata":{},"cell_type":"markdown","id":"3d507218-1fd2-4502-a98e-d98943a41c8a","source":["<div class=\"alert alert-block alert-info\"><b>Oceanographic example:</b> tracer diffusion is implemented using the Laplacian operator.</div>"]},{"metadata":{},"cell_type":"markdown","id":"65a457bd-bde9-4554-8764-19da5ffae9ed","source":["Implement the [Sobel operator](https://en.wikipedia.org/wiki/Sobel_operator) and apply it to the red channel. Mathemacially `G` is defined as:\n","\n","$$\\begin{align}\n","{G_x}_{i,j} &= (I_{i-1,j-1} + 2\\, I_{i-1,j} + I_{i-1,j+1}) - (I_{i+1,j-1} + 2 \\, I_{i+1,j} + I_{i+1,j+1}) \\\\\n","{G_y}_{i,j} &= (I_{i-1,j-1} + 2\\, I_{i,j-1} + I_{i+1,j-1}) - (I_{i-1,j+1} + 2 \\, I_{i,j+1} + I_{i+1,j+1}) \\\\\n","G_{i,j} &= \\sqrt{{G_x^2}_{i,j} + {G_y^2}_{i,j}} \\\\\n","\\end{align}$$\n"]},{"metadata":{},"outputs":[],"cell_type":"code","id":"3ca8fe0d-47c2-4b7e-bc47-6b5cfba2a4d5","source":[],"execution_count":65},{"metadata":{},"outputs":[],"cell_type":"code","id":"9a58ad83-91f8-4568-85d4-f760b634b0cb","source":[],"execution_count":66},{"metadata":{},"cell_type":"markdown","id":"c86e7ade-d098-4802-ab96-8e3d36147694","source":["<div class=\"alert alert-block alert-info\"><b>Oceanographic example:</b> The governing equations of numerical models can be implemented using finite difference schemes</div>"]}]}

JuliaIntroArray_SOLUTION.ipynb

@@ -13,7 +13,7 @@
     "\n",
     "* Start the computer and boot into **Debian** (not Windows)\n",
     "* Open Firefox and log in to the internet with your ULiège credentials\n",
-    "* Open a terminal (Application -> Utilities -> Terminal). This is how the terminal icon looks like: ![icon](img/gnome-terminal.svg)\n",
+    "* Open a terminal (Application -> Utilities -> Terminal). This is how the terminal icon looks like: ![icon](https://raw.githubusercontent.com/Alexander-Barth/JuliaTutorial/master/img/gnome-terminal.svg)\n",
     "* Launch Jupyter from the terminal with the command (please pay attention to uppercase versus lowercase; after typing the command you need hit the `⏎ Enter` key):\n",
     "\n",
     "```bash\n",

JuliaIntroduction.ipynb

@@ -3887,5 +3887,5 @@
   }
  },
  "nbformat": 4,
- "nbformat_minor": 1
+ "nbformat_minor": 4
 }

JuliaPlotting.ipynb

@@ -411,15 +411,15 @@
    "lastKernelId": null
   },
   "kernelspec": {
-   "display_name": "Julia 1.7.2",
+   "display_name": "Julia 1.8.0",
    "language": "julia",
-   "name": "julia-1.7"
+   "name": "julia-1.8"
   },
   "language_info": {
    "file_extension": ".jl",
    "mimetype": "application/julia",
    "name": "julia",
-   "version": "1.7.2"
+   "version": "1.8.0"
   }
  },
  "nbformat": 4,