EU_EurLex_Cellar_doc_reference_querier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notebook for searching citations to or from EU legislation and / or EU case law. \n",
    "\n",
    "Users can give one or multiple CELEX IDs as input. The functions return a dictionaries with a nodes list and / or an edges list with each source - target pair as tuple.\n",
    "\n",
    "Users have the option to export the edges list to a networkx graph. Basic code is provided.\n",
    "\n",
    "Users can (but do not have to) compare the retrieved CELEX IDs with a predefined list of CELEX IDs (eg provided by experts). Code for evaluation metrics (F1, precision, recall) is provided."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# install dependencies\n",
    "\n",
    "#!pip install SPARQLWrapper"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Functions for Citation Retrieval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from SPARQLWrapper import SPARQLWrapper, JSON\n",
    "\n",
    "def get_citations(source_celex, cites_depth=1, cited_depth=1):\n",
    "    \"\"\"\n",
    "    Gets all the citations one to X steps away. Hops can be specified as either\n",
    "    the source document citing another (defined by `cites_depth`) or another document\n",
    "    citing it (`cited_depth`). Any numbers higher than 1 denote that new source document\n",
    "    citing a document of its own.\n",
    "\n",
    "    This specific implementation does not care about intermediate steps, it simply finds\n",
    "    anything X or fewer hops away without linking those together.\n",
    "    \"\"\"    \n",
    "    sparql = SPARQLWrapper('https://publications.europa.eu/webapi/rdf/sparql')\n",
    "    sparql.setReturnFormat(JSON)\n",
    "    sparql.setQuery('''\n",
    "        prefix cdm: <http://publications.europa.eu/ontology/cdm#>\n",
    "        prefix xsd: <http://www.w3.org/2001/XMLSchema#>\n",
    "\n",
    "        SELECT DISTINCT * WHERE\n",
    "        {\n",
    "        {\n",
    "            SELECT ?name2 WHERE {\n",
    "                ?doc cdm:resource_legal_id_celex \"%s\"^^xsd:string .\n",
    "                ?doc cdm:work_cites_work{1,%i} ?cited .\n",
    "                ?cited cdm:resource_legal_id_celex ?name2 .\n",
    "            }\n",
    "        } UNION {\n",
    "            SELECT ?name2 WHERE {\n",
    "                ?doc cdm:resource_legal_id_celex \"%s\"^^xsd:string .\n",
    "                ?cited cdm:work_cites_work{1,%i} ?doc .\n",
    "                ?cited cdm:resource_legal_id_celex ?name2 .\n",
    "            }\n",
    "        }\n",
    "        }''' % (source_celex, cites_depth, source_celex, cited_depth))\n",
    "    ret = sparql.queryAndConvert()\n",
    "\n",
    "    targets = set()\n",
    "    for bind in ret['results']['bindings']:\n",
    "        target = bind['name2']['value']\n",
    "        targets.add(target)\n",
    "    targets = set([el for el in list(targets) if el.startswith('3')]) #Filters the list...\n",
    "    ###... Set filters based on filtertype (eg '3'=legislation, '6'=case law).\n",
    "        \n",
    "    return targets\n",
    "\n",
    "def get_citations_multiple(sources, cites_depth=1, cited_depth=1, union=True):\n",
    "    \"\"\"\n",
    "    Gets citations coming from multiple sources (given as a list of CELEX IDs).\n",
    "    By default gets the union of all the resulting CELEXes, but of interest\n",
    "    might be the intersect instead, returning only documents that are common\n",
    "    between all the sources.\n",
    "    \"\"\"\n",
    "    results = [get_citations(source, cites_depth, cited_depth) for source in sources]\n",
    "    results.append(sources) #ensures that source nodes (ie starting points) are included in nodes list\n",
    "\n",
    "    if union:\n",
    "        return set().union(*results)    \n",
    "    else:\n",
    "        start_set = results[0]\n",
    "        if len(sources) > 1:\n",
    "            return start_set.union(*results[1:])\n",
    "        else:\n",
    "            return start_set\n",
    "\n",
    "def get_citations_structure(source, cites_depth=1, cited_depth=1, dont_repeat=set()):\n",
    "    if cites_depth > 0 and cited_depth > 0:\n",
    "        cites, nodes1 = get_citations_structure(source, cites_depth, 0, dont_repeat)\n",
    "        cited, nodes2 = get_citations_structure(source, 0, cited_depth, dont_repeat)\n",
    "        return cites.union(cited), nodes1.union(nodes2)\n",
    "\n",
    "\n",
    "    new_cites_depth = max(cites_depth - 1, 0)\n",
    "    new_cited_depth = max(cited_depth - 1, 0)\n",
    "\n",
    "    dont_repeat = dont_repeat.union({source})\n",
    "\n",
    "    links = set()\n",
    "    nodes = {source}\n",
    "    targets = get_citations(source, min(cites_depth, 1), min(cited_depth, 1))\n",
    "\n",
    "    for target in targets:\n",
    "        nodes.add(target)\n",
    "        # We're looking for citations from the source\n",
    "        if cites_depth > 0:\n",
    "            links.add((source, target))\n",
    "        # Or to the source\n",
    "        else:\n",
    "            links.add((target, source))\n",
    "\n",
    "        if new_cites_depth or new_cited_depth and target not in dont_repeat:\n",
    "            new_links, new_nodes = get_citations_structure(target, new_cites_depth, new_cited_depth)\n",
    "            links = links.union(new_links)\n",
    "            nodes = nodes.union(new_nodes)\n",
    "\n",
    "    return links, nodes\n",
    "\n",
    "def get_citations_structure_multiple(sources, cites_depth=1, cited_depth=1):\n",
    "    links = set()\n",
    "    nodes = set(sources)\n",
    "    for source in sources:\n",
    "        if source.startswith('3'):\n",
    "            new_links, new_nodes = get_citations_structure(source, cites_depth, cited_depth)\n",
    "            links = links.union(new_links)\n",
    "            nodes = nodes.union(new_nodes)\n",
    "#            nodes = set([el for el in list(nodes) if el.startswith('3')]) #Filters the list. Filtertype: '3'=legislation, '6'=case law.\n",
    "    return links, nodes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indicate the source node(s) and the cited_depth and cites_depth. In the example where C is the source node and A cites B and B cites C (A -> B -> C), cited_depth=2 means that the citations A -> B and B -> C are included. If cited_depth=1, only the citation A -> B is retrieved. Similarly, if A is the source node, a cites_detph=2 means that the citations A -> B and B -> C are included.\n",
    "\n",
    "An example for retrieving citations for one given source:\n",
    "\n",
    "    source = '32021R0664'\n",
    "    links, nodes = get_citations_structure(source, cited_depth=1, cites_depth=2)\n",
    "\n",
    "In case of multiple source nodes, the CELEX IDs can be put in a list. For example:\n",
    "\n",
    "    sources = ['32019R0945','32021R0664']\n",
    "    links, nodes = get_citations_structure_multiple(sources, cited_depth=0, cites_depth=2)\n",
    "\n",
    "(it is also possible to input one source node in the list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CSV Export"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare the edges list for csv\n",
    "edges_list_for_csv = []\n",
    "for i in links:\n",
    "    to_add = i[0][1:]+','+i[1][1:]\n",
    "    edges_list_for_csv.append(to_add)\n",
    "\n",
    "# Write to file\n",
    "with open('./output/edges_extracted/extracted_edges(network).csv', 'w', newline='') as f:  \n",
    "    for entries in edges_list_for_csv:\n",
    "        f.write(entries)\n",
    "        f.write(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build graph with NetworkX"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import\n",
    "import networkx as nx\n",
    "\n",
    "# Build complete network of docs\n",
    "links, _ = get_citations_structure_multiple(sources, cites_depth=2, cited_depth=1) #set cites_depth, cited_depth\n",
    "\n",
    "g = nx.Graph()\n",
    "g.add_edges_from(links)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# saving graph created above in gexf format\n",
    "nx.write_gexf(g, \"./networkfiles/networkxile.gexf\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluation\n",
    "\n",
    "If you want to compare the retrieved nodes with the nodes in a self-made overview.\n",
    "\n",
    "Example:\n",
    "\n",
    "docs_list = {\n",
    "    '32018R1139',\n",
    "    '32022R0868'\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create overview\n",
    "\n",
    "docs_list = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def do_stats(nodes, print_res=True):\n",
    "    \"\"\"\n",
    "    Compares the retrieved nodes with the nodes in docs_list. \n",
    "    Returns total number of nodes as well as: \n",
    "    (1) Common nodes (ie nodes that appear both in the retrieved results and in docs_list), \n",
    "    (2) Missed nodes (ie nodes in docs_list that were not retrieved), and \n",
    "    (3) Extra nodes (ie retrieved nodes not in docs_list). \n",
    "    Returns evaluation metrics (precision, recall, F1).\n",
    "    \"\"\"\n",
    "    nodes = set(nodes)\n",
    "\n",
    "    precision = len(nodes.intersection(docs_list)) / float(len(nodes))\n",
    "    recall = len(nodes.intersection(docs_list)) / float(len(docs_list))\n",
    "    f1 = 2 * (precision * recall) / (precision + recall)\n",
    "    if print_res:\n",
    "        print(f'Total nodes found in search: {len(nodes)}')\n",
    "        print(f'Precision: {precision}\\nRecall: {recall}\\nF1: {f1}')\n",
    "\n",
    "        print(f'Common nodes ({len(nodes.intersection(docs_list))}): {nodes.intersection(docs_list)}')\n",
    "        print(f'Missed nodes ({len(docs_list - nodes)}): {docs_list - nodes}')\n",
    "        print(f\"Extra nodes ({len(nodes - docs_list)}): {nodes - docs_list}\")\n",
    "    return (precision, recall, f1)\n",
    "\n",
    "do_stats(nodes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to create a matrix of depths and to calculate stats for each combination of depth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import\n",
    "import csv\n",
    "\n",
    "# Create a list to store the results\n",
    "results = []\n",
    "\n",
    "# Go through matrix of depths and calculate stats for each\n",
    "for i in range(5): #set depths\n",
    "    for j in range(5): #set depths\n",
    "        try:\n",
    "            precision, recall, f1 = do_stats(get_citations_multiple(sources, i, j), print_res=False)\n",
    "            results.append([i, j, precision, recall, f1])\n",
    "            print(f'Cites depth: {i:02d} | Cited depth: {j:02d} | Pr {precision:.2f} Re {recall:.2f} F1 {f1:.2f}')\n",
    "        except ZeroDivisionError:\n",
    "            print(\"Tried dividing by zero, skipping\")\n",
    "\n",
    "# Save results to a CSV file\n",
    "filename = './output/stats_evaluation/stats.csv'\n",
    "with open(filename, 'w', newline='') as csvfile:\n",
    "    writer = csv.writer(csvfile)\n",
    "    writer.writerow(['Cites Depth', 'Cited Depth', 'Precision', 'Recall', 'F1 Score'])\n",
    "    writer.writerows(results)\n",
    "\n",
    "print(f'Results saved to {filename}')"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "46d8675e7301772991a005dee3941691e992e064f540356d3b3c650b45d03ba5"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}