src/reading_laspy_pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reading a LAS file into a Pandas data frame via laspy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install laspy library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [laspy](https://pypi.org/project/laspy/) package implements reading and writing of the [LAS fromat](https://www.asprs.org/divisions-committees/lidar-division/laser-las-file-format-exchange-activities) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: laspy in /usr/local/python/3.10.8/lib/python3.10/site-packages (2.5.1)\n",
      "Requirement already satisfied: numpy in /home/codespace/.local/lib/python3.10/site-packages (from laspy) (1.26.1)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install \"laspy\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open LAS file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We follow the tutorial from https://pythonhosted.org/laspy/tut_part_1.html and adapt to laspy 2.0 using https://laspy.readthedocs.io/en/latest/migration.html#from-laspy-1-7-x-to-laspy-2-0-0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import laspy\n",
    "from laspy.file import File\n",
    "las = laspy.read(\"../data/test.las\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find out which attributes are stored with each record in the LAS file. This depends on the version of the LAS format and the application that created the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X\n",
      "Y\n",
      "Z\n",
      "intensity\n",
      "return_number\n",
      "number_of_returns\n",
      "scan_direction_flag\n",
      "edge_of_flight_line\n",
      "classification\n",
      "synthetic\n",
      "key_point\n",
      "withheld\n",
      "scan_angle_rank\n",
      "user_data\n",
      "point_source_id\n",
      "gps_time\n",
      "red\n",
      "green\n",
      "blue\n"
     ]
    }
   ],
   "source": [
    "pointformat = las.point_format\n",
    "for spec in las.point_format:\n",
    "    print(spec.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using pandas (https://pandas.pydata.org/) load selected attributes of the records into a table with named headers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "df = pd.DataFrame({'X': np.array(las.x), 'Y': np.array(las.y), 'Z': np.array(las.z), 'intensity': las.intensity, 'raw_classification': las.raw_classification})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check how many records were loaded and show the first 5 rows of the data frame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1339520"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>Z</th>\n",
       "      <th>intensity</th>\n",
       "      <th>raw_classification</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>555000.0625</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>120.940003</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>555000.6875</td>\n",
       "      <td>4887199.5</td>\n",
       "      <td>117.330002</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>555001.3125</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>115.339996</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>555000.1875</td>\n",
       "      <td>4887197.0</td>\n",
       "      <td>123.910004</td>\n",
       "      <td>31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>555001.9375</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>111.110001</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             X          Y           Z  intensity  raw_classification\n",
       "0  555000.0625  4887200.0  120.940003         30                   1\n",
       "1  555000.6875  4887199.5  117.330002         22                   1\n",
       "2  555001.3125  4887200.0  115.339996         10                   1\n",
       "3  555000.1875  4887197.0  123.910004         31                   1\n",
       "4  555001.9375  4887200.0  111.110001          8                   1"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Further columns can be added at a later time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>Z</th>\n",
       "      <th>intensity</th>\n",
       "      <th>raw_classification</th>\n",
       "      <th>gps_time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>555000.0625</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>120.940003</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.43750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>555000.6875</td>\n",
       "      <td>4887199.5</td>\n",
       "      <td>117.330002</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.50000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>555001.3125</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>115.339996</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.50000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>555000.1875</td>\n",
       "      <td>4887197.0</td>\n",
       "      <td>123.910004</td>\n",
       "      <td>31</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.53125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>555001.9375</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>111.110001</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.53125</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             X          Y           Z  intensity  raw_classification  \\\n",
       "0  555000.0625  4887200.0  120.940003         30                   1   \n",
       "1  555000.6875  4887199.5  117.330002         22                   1   \n",
       "2  555001.3125  4887200.0  115.339996         10                   1   \n",
       "3  555000.1875  4887197.0  123.910004         31                   1   \n",
       "4  555001.9375  4887200.0  111.110001          8                   1   \n",
       "\n",
       "       gps_time  \n",
       "0  467000.43750  \n",
       "1  467000.50000  \n",
       "2  467000.50000  \n",
       "3  467000.53125  \n",
       "4  467000.53125  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = df.assign(gps_time = las.gps_time)\n",
    "df2.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But carefull as using 'assign' copies the whole data frame to a new one. Setting with [..] should avoid this. See https://stackoverflow.com/questions/48177914/why-use-pandas-assign-rather-than-simply-initialize-new-column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>Z</th>\n",
       "      <th>intensity</th>\n",
       "      <th>raw_classification</th>\n",
       "      <th>gps_time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>555000.0625</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>120.940003</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.43750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>555000.6875</td>\n",
       "      <td>4887199.5</td>\n",
       "      <td>117.330002</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.50000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>555001.3125</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>115.339996</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.50000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>555000.1875</td>\n",
       "      <td>4887197.0</td>\n",
       "      <td>123.910004</td>\n",
       "      <td>31</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.53125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>555001.9375</td>\n",
       "      <td>4887200.0</td>\n",
       "      <td>111.110001</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>467000.53125</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             X          Y           Z  intensity  raw_classification  \\\n",
       "0  555000.0625  4887200.0  120.940003         30                   1   \n",
       "1  555000.6875  4887199.5  117.330002         22                   1   \n",
       "2  555001.3125  4887200.0  115.339996         10                   1   \n",
       "3  555000.1875  4887197.0  123.910004         31                   1   \n",
       "4  555001.9375  4887200.0  111.110001          8                   1   \n",
       "\n",
       "       gps_time  \n",
       "0  467000.43750  \n",
       "1  467000.50000  \n",
       "2  467000.50000  \n",
       "3  467000.53125  \n",
       "4  467000.53125  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['gps_time'] = las.gps_time\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can query simple statistics for all attributes (i.e. along each column)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "395.3800050094404"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Z'].max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "X                     5.543902e+05\n",
       "Y                     4.886536e+06\n",
       "Z                    -1.160000e+00\n",
       "intensity             1.000000e+00\n",
       "raw_classification    1.000000e+00\n",
       "gps_time              4.659909e+05\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.min()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The resulting series is of uniform type. Check the data type for a specific attribute (i.e. a single column)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('uint16')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['intensity'].dtype"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}