doc/source/notebooks/09_node_queries.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Querying nodes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preamble\n",
    "The code in this section is identical to the code in sections \"Introduction\" and \"Loading\" from the previous tutorial. It assumes that you have already downloaded the circuit. If not, take a look to the notebook **01_circuits** (Downloading a circuit)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import bluepysnap\n",
    "import pandas as pd\n",
    "\n",
    "circuit_path = \"sonata/circuit_sonata.json\"\n",
    "circuit = bluepysnap.Circuit(circuit_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This section presents different ways to query data from nodes and edges. Since we just went over node sets, lets put them into the spotlight and see how we can utilize them in queries.\n",
    "\n",
    "## Queries with node sets\n",
    "As we know, we can use the plain node set name as a string (e.g., `circuit.nodes.ids('VPL_TC')`) to use a node set for queries. However, this only works for the node sets integrated in the circuit.\n",
    "\n",
    "Luckily, if we want to use node sets external to the circuit config, we can do so by just passing the `NodeSet` object as a query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CircuitNodeIds([('CorticoThalamic_projections', 0),\n",
       "            ('CorticoThalamic_projections', 1),\n",
       "            ('CorticoThalamic_projections', 2),\n",
       "            ('CorticoThalamic_projections', 3),\n",
       "            ('CorticoThalamic_projections', 4),\n",
       "            ('CorticoThalamic_projections', 5),\n",
       "            ('CorticoThalamic_projections', 6),\n",
       "            ('CorticoThalamic_projections', 7),\n",
       "            ('CorticoThalamic_projections', 8),\n",
       "            ('CorticoThalamic_projections', 9),\n",
       "            ('MedialLemniscus_projections', 0),\n",
       "            ('MedialLemniscus_projections', 1),\n",
       "            ('MedialLemniscus_projections', 2),\n",
       "            ('MedialLemniscus_projections', 3),\n",
       "            ('MedialLemniscus_projections', 4),\n",
       "            ('MedialLemniscus_projections', 5),\n",
       "            ('MedialLemniscus_projections', 6),\n",
       "            ('MedialLemniscus_projections', 7),\n",
       "            ('MedialLemniscus_projections', 8),\n",
       "            ('MedialLemniscus_projections', 9),\n",
       "            (           'thalamus_neurons', 0),\n",
       "            (           'thalamus_neurons', 1),\n",
       "            (           'thalamus_neurons', 2),\n",
       "            (           'thalamus_neurons', 3),\n",
       "            (           'thalamus_neurons', 4),\n",
       "            (           'thalamus_neurons', 5),\n",
       "            (           'thalamus_neurons', 6),\n",
       "            (           'thalamus_neurons', 7),\n",
       "            (           'thalamus_neurons', 8),\n",
       "            (           'thalamus_neurons', 9)],\n",
       "           names=['population', 'node_ids'])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_sets_0_9 = bluepysnap.node_sets.NodeSets.from_dict({'nodes_0-9': {'node_id': [*range(10)]}})\n",
    "\n",
    "# circuit.nodes.ids('nodes_0-9')  # This would raise an error: BluepySnapError: Undefined node set: 'nodes_0-9'\n",
    "circuit.nodes.ids(node_sets_0_9['nodes_0-9'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying in general\n",
    "We can query data based on mtype, etype, node_id, region, layer or any of the properties the nodes / edges have:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'@dynamics:holding_current',\n",
       " '@dynamics:threshold_current',\n",
       " 'etype',\n",
       " 'layer',\n",
       " 'model_template',\n",
       " 'model_type',\n",
       " 'morph_class',\n",
       " 'morphology',\n",
       " 'mtype',\n",
       " 'orientation_w',\n",
       " 'orientation_x',\n",
       " 'orientation_y',\n",
       " 'orientation_z',\n",
       " 'region',\n",
       " 'rotation_angle_xaxis',\n",
       " 'rotation_angle_yaxis',\n",
       " 'rotation_angle_zaxis',\n",
       " 'synapse_class',\n",
       " 'x',\n",
       " 'y',\n",
       " 'z'}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "circuit.nodes.property_names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On top of these, a query can also be based on `node_id` or the `population_type`.\n",
    "\n",
    "When the query is a `dict` and there is a `list` in the query, it is (usually) considered as an \"OR\" condition, and the keys of the query are considered as an \"AND\" condition. E.g.,\n",
    "```python\n",
    "circuit.nodes.ids({                    # give me the ids of nodes that\n",
    "    'mtype': ['VPL_TC', 'VPL_IN']      # have mtype of 'VPL_TC' or 'VPL_IN' and\n",
    "    'population': 'thalamus_neurons'   # belong to a population 'thalamus_neurons'\n",
    "})  \n",
    "```\n",
    "\n",
    "Let's start with simple examples and work our way up to more complex queries.\n",
    "\n",
    "## Querying with ids\n",
    "\n",
    "### Integers as `id`s\n",
    "There are a few methods to query using ids. The most simple way is just use a single integer or a list of integers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CircuitNodeIds([('CorticoThalamic_projections', 1),\n",
      "            ('MedialLemniscus_projections', 1),\n",
      "            (           'thalamus_neurons', 1)],\n",
      "           names=['population', 'node_ids'])\n",
      "CircuitNodeIds([('CorticoThalamic_projections', 1),\n",
      "            ('MedialLemniscus_projections', 1),\n",
      "            (           'thalamus_neurons', 1)],\n",
      "           names=['population', 'node_ids'])\n"
     ]
    }
   ],
   "source": [
    "print(circuit.nodes.ids(1))\n",
    "print(circuit.nodes.ids([1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This results in any nodes in any of the populations that have nodes with the given id(s). \n",
    "\n",
    "If we want to specify a population, we can do the above query with a `dict` instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CircuitNodeIds([('thalamus_neurons', 1)],\n",
       "           names=['population', 'node_ids'])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "circuit.nodes.ids({'node_id': 1, 'population': 'thalamus_neurons'})"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using `CircuitNodeIds` / `CircuitEdgeIds`\n",
    "\n",
    "We could also use `CircuitNodeIds` (`CircuitEdgeIds` for edges) to specify the population and the node ids to consider. These objects are the same objects returned from the `ids` function and we wouldn't generally need to create them by hand.\n",
    "\n",
    "The main things you need to know is that these objects are returned from the `ids` functions and they can be directly passed on to `get` functions.\n",
    "\n",
    "For the sake of an example, lets create a `CircuitNodeIds` object. There are several methods to instantiate them, but we'll use `from_dict` here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CircuitNodeIds([('thalamus_neurons', 1)],\n",
      "           names=['population', 'node_ids'])\n"
     ]
    }
   ],
   "source": [
    "ids_from_dict = bluepysnap.circuit_ids.CircuitNodeIds.from_dict({'thalamus_neurons': [1]})\n",
    "print(ids_from_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you may see, it's exactly what was returned by the `circuit.nodes.ids` function. Let's use it to do a get query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>layer</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>population</th>\n",
       "      <th>node_ids</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>thalamus_neurons</th>\n",
       "      <th>1</th>\n",
       "      <td>Rt</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          layer\n",
       "population       node_ids      \n",
       "thalamus_neurons 1           Rt"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = circuit.nodes.get(ids_from_dict, properties=['layer'])\n",
    "pd.concat([df for _,df in result])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Resolved `ids` in `get` queries\n",
    "As mentioned, we can use the output of the ids as an argument to the `get` function:\n",
    "```python\n",
    "ids = circuit.nodes.ids({'node_id': [1, 2], 'population': 'thalamus_neurons'})\n",
    "circuit.nodes.get(ids, properties=['layer'])\n",
    "```\n",
    "but, better yet, we can just pass the query to the `get` function as the `id`s are resolved internally:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>layer</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>population</th>\n",
       "      <th>node_ids</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">thalamus_neurons</th>\n",
       "      <th>1</th>\n",
       "      <td>Rt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Rt</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          layer\n",
       "population       node_ids      \n",
       "thalamus_neurons 1           Rt\n",
       "                 2           Rt"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = circuit.nodes.get({'node_id': [1, 2], 'population': 'thalamus_neurons'}, properties=['layer'])\n",
    "pd.concat([df for _,df in result])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying with population type\n",
    "We can query nodes (or edges) based on their population type as specified in the [SONATA circuit configuration file](https://sonata-extension.readthedocs.io/en/latest/sonata_config.html).\n",
    "\n",
    "Let's find all the source nodes of projections (i.e., `virtual` nodes):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>model_template</th>\n",
       "      <th>model_type</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>population</th>\n",
       "      <th>node_ids</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">CorticoThalamic_projections</th>\n",
       "      <th>0</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">MedialLemniscus_projections</th>\n",
       "      <th>5018</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5019</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5020</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5021</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5022</th>\n",
       "      <td></td>\n",
       "      <td>virtual</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>88443 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     model_template model_type\n",
       "population                  node_ids                          \n",
       "CorticoThalamic_projections 0                          virtual\n",
       "                            1                          virtual\n",
       "                            2                          virtual\n",
       "                            3                          virtual\n",
       "                            4                          virtual\n",
       "...                                             ...        ...\n",
       "MedialLemniscus_projections 5018                       virtual\n",
       "                            5019                       virtual\n",
       "                            5020                       virtual\n",
       "                            5021                       virtual\n",
       "                            5022                       virtual\n",
       "\n",
       "[88443 rows x 2 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = circuit.nodes.get({'population_type': ['virtual']})\n",
    "pd.concat([df for _,df in result])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying with regex\n",
    "We can use regex to query data (e.g., to cover \"OR\" cases) by using the key `$regex` and specify an expression as a `str`. For example, let's search `thalamus_neurons` by `mtypes` that start with `VPL`:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mtype</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>node_ids</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5002</th>\n",
       "      <td>VPL_TC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5003</th>\n",
       "      <td>VPL_TC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5004</th>\n",
       "      <td>VPL_TC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5005</th>\n",
       "      <td>VPL_TC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5006</th>\n",
       "      <td>VPL_TC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100760</th>\n",
       "      <td>VPL_IN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100761</th>\n",
       "      <td>VPL_IN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100762</th>\n",
       "      <td>VPL_IN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100763</th>\n",
       "      <td>VPL_IN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100764</th>\n",
       "      <td>VPL_IN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>65198 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           mtype\n",
       "node_ids        \n",
       "5002      VPL_TC\n",
       "5003      VPL_TC\n",
       "5004      VPL_TC\n",
       "5005      VPL_TC\n",
       "5006      VPL_TC\n",
       "...          ...\n",
       "100760    VPL_IN\n",
       "100761    VPL_IN\n",
       "100762    VPL_IN\n",
       "100763    VPL_IN\n",
       "100764    VPL_IN\n",
       "\n",
       "[65198 rows x 1 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    " circuit.nodes['thalamus_neurons'].get({\"mtype\": {\"$regex\": \"^VPL_.*\"}}, properties=['mtype'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying ranges of values\n",
    "\n",
    "As shown above, we can query data based on discrete values of properties such as node set, layer etc. However, all properties are not made equal. \n",
    "\n",
    "When there is a `list` in the query, it is _usually_ considered an \"OR\" condition. There is one exception and that is numeric properties represented by `float`s. Whenever a property is represented by a `float`, a `list` will need to specify exactly two values: start of a range and end of a range (i.e., minimum and maximum values to consider).\n",
    "\n",
    "Let's take a look which properties in `'thalamus_neurons'` are floats:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "@dynamics:holding_current       float32\n",
       "@dynamics:threshold_current     float32\n",
       "etype                          category\n",
       "layer                            object\n",
       "model_template                   object\n",
       "model_type                       object\n",
       "morph_class                    category\n",
       "morphology                     category\n",
       "mtype                          category\n",
       "orientation_w                   float32\n",
       "orientation_x                   float32\n",
       "orientation_y                   float32\n",
       "orientation_z                   float32\n",
       "region                           object\n",
       "rotation_angle_xaxis            float32\n",
       "rotation_angle_yaxis            float32\n",
       "rotation_angle_zaxis            float32\n",
       "synapse_class                  category\n",
       "x                               float32\n",
       "y                               float32\n",
       "z                               float32\n",
       "dtype: object"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "circuit.nodes['thalamus_neurons'].property_dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's take `x`, `y`, `z` and query nodes that inside a 50x50x50 box limited by:\n",
    "* `100 <= x <= 150`\n",
    "* `500 <= y <= 550`\n",
    "* `400 <= z <= 450`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>node_ids</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>26101</th>\n",
       "      <td>109.218941</td>\n",
       "      <td>526.994873</td>\n",
       "      <td>433.528503</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27832</th>\n",
       "      <td>116.552238</td>\n",
       "      <td>519.171082</td>\n",
       "      <td>414.830933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27993</th>\n",
       "      <td>100.712425</td>\n",
       "      <td>535.644287</td>\n",
       "      <td>401.546112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28285</th>\n",
       "      <td>133.707870</td>\n",
       "      <td>531.594727</td>\n",
       "      <td>444.947723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28377</th>\n",
       "      <td>145.187637</td>\n",
       "      <td>547.493591</td>\n",
       "      <td>403.406708</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28378</th>\n",
       "      <td>127.572868</td>\n",
       "      <td>538.761536</td>\n",
       "      <td>422.776123</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28380</th>\n",
       "      <td>143.356384</td>\n",
       "      <td>525.541626</td>\n",
       "      <td>409.481781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28381</th>\n",
       "      <td>135.235031</td>\n",
       "      <td>517.312378</td>\n",
       "      <td>428.180695</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   x           y           z\n",
       "node_ids                                    \n",
       "26101     109.218941  526.994873  433.528503\n",
       "27832     116.552238  519.171082  414.830933\n",
       "27993     100.712425  535.644287  401.546112\n",
       "28285     133.707870  531.594727  444.947723\n",
       "28377     145.187637  547.493591  403.406708\n",
       "28378     127.572868  538.761536  422.776123\n",
       "28380     143.356384  525.541626  409.481781\n",
       "28381     135.235031  517.312378  428.180695"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "circuit.nodes['thalamus_neurons'].get({\n",
    "    'x': [100, 150],\n",
    "    'y': [500, 550],\n",
    "    'z': [400, 450],    \n",
    "}, properties=['x', 'y', 'z'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complex (or rather, combined) queries\n",
    "In some cases, we might want to specify our queries further, which makes them complex but not necessarily complicated.\n",
    "\n",
    "For example, the issue with compound node sets is that it is always considered an \"OR\" condition, since it's a list.\n",
    "What if you wanted to combine two node sets and make it an \"AND\" condition. For this case, there is a key `$node_set` we must combine with yet another key `$and`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CircuitNodeIds([('thalamus_neurons',     0),\n",
       "            ('thalamus_neurons',     1),\n",
       "            ('thalamus_neurons',     2),\n",
       "            ('thalamus_neurons',     3),\n",
       "            ('thalamus_neurons',     4),\n",
       "            ('thalamus_neurons',     5),\n",
       "            ('thalamus_neurons',     6),\n",
       "            ('thalamus_neurons',     7),\n",
       "            ('thalamus_neurons',     8),\n",
       "            ('thalamus_neurons',     9),\n",
       "            ...\n",
       "            ('thalamus_neurons', 91247),\n",
       "            ('thalamus_neurons', 91248),\n",
       "            ('thalamus_neurons', 91249),\n",
       "            ('thalamus_neurons', 91250),\n",
       "            ('thalamus_neurons', 91251),\n",
       "            ('thalamus_neurons', 91252),\n",
       "            ('thalamus_neurons', 91253),\n",
       "            ('thalamus_neurons', 91254),\n",
       "            ('thalamus_neurons', 91255),\n",
       "            ('thalamus_neurons', 91256)],\n",
       "           names=['population', 'node_ids'], length=35567)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "circuit.nodes.ids({\n",
    "    '$and': [  # list of queries that are considered as: AND conditions\n",
    "         {'$node_set': 'thalamus_neurons'},\n",
    "         {'$node_set': 'Rt_RC'}\n",
    "]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This, however, can not be done with node sets external to the circuit.\n",
    "\n",
    "It may also be that you want to query something as an \"OR\" condition rather than \"AND\".\n",
    "For example querying\n",
    "```python\n",
    "circuit.nodes.ids({'mtype': 'VPL_IN', 'region': 'mc2;Rt'})\n",
    "```\n",
    "will never return any ids. So how do we make it an \"OR\" case? That's right, with the help of `$or`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Rt_RC', 'mc2;Rt')\n",
      "('VPL_IN', 'mc0;VPL')\n",
      "('VPL_IN', 'mc1;VPL')\n",
      "('VPL_IN', 'mc2;VPL')\n",
      "('VPL_IN', 'mc3;VPL')\n",
      "('VPL_IN', 'mc4;VPL')\n",
      "('VPL_IN', 'mc5;VPL')\n",
      "('VPL_IN', 'mc6;VPL')\n"
     ]
    }
   ],
   "source": [
    "result = circuit.nodes['thalamus_neurons'].get({\n",
    "    '$or':[ # same as with $and, except the list is considered as OR condition\n",
    "        {'mtype': 'VPL_IN'},\n",
    "        {'region': 'mc2;Rt'}\n",
    "]}, properties=['mtype', 'region'])\n",
    "\n",
    "# Let's print all the unique mtype-region pairs:\n",
    "for pair in result.groupby(['mtype', 'region'], observed=True).groups.keys():\n",
    "    print(pair)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "In this notebook we took a deeper look into how to query data in SNAP using nodes as an example. In the next notebook we'll cover edge queries."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}