Merge pull request #290 from johannbrehmer/future

MadMiner overhaul for v0.3.0

.gitignore

@@ -173,6 +173,8 @@ examples/example3_systematics/data
 examples/example3_systematics/mg_processes
 examples/example3_systematics/models
 
+examples/example4_tth
+
 # MG / Pythia / Delphes patch
 patches/
 

docs/conf.py

@@ -24,7 +24,7 @@
 author = 'Johann Brehmer, Felix Kling, Irina Espejo, and Kyle Cranmer'
 
 # The short X.Y version
-version = '0.2.8'
+version = '0.3.0'
 # The full version, including alpha/beta/rc tags
 release = version
 

examples/tutorial_delphes/3_systematic_uncertainties.ipynb

@@ -0,0 +1,366 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# MadMiner parton-level tutorial, part 3: Systematic uncertainties\n",
+    "\n",
+    "Johann Brehmer, Felix Kling, Kyle Cranmer 2018"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this tutorial we'll explain how to add systematic uncertainties to the MadMiner workflow."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Preparations"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Before you execute this notebook, make sure you have running installations of MadGraph, Pythia, and Delphes."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
+    "\n",
+    "import logging\n",
+    "import numpy as np\n",
+    "import matplotlib\n",
+    "from matplotlib import pyplot as plt\n",
+    "%matplotlib inline\n",
+    "\n",
+    "from madminer.core import MadMiner\n",
+    "from madminer.lhe import LHEReader\n",
+    "from madminer.sampling import combine_and_shuffle\n",
+    "from madminer.sampling import SampleAugmenter\n",
+    "from madminer import sampling\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Please enter here the path to your MG5 root directory."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "mg_dir = '/Users/johannbrehmer/work/projects/madminer/MG5_aMC_v2_6_4'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "MadMiner uses the Python `logging` module to provide additional information and debugging output. You can choose how much of this output you want to see by switching the level in the following lines to `logging.DEBUG` or `logging.WARNING`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# MadMiner output\n",
+    "logging.basicConfig(\n",
+    "    format='%(asctime)-5.5s %(name)-20.20s %(levelname)-7.7s %(message)s',\n",
+    "    datefmt='%H:%M',\n",
+    "    level=logging.INFO\n",
+    ")\n",
+    "\n",
+    "# Output of all other modules (e.g. matplotlib)\n",
+    "for key in logging.Logger.manager.loggerDict:\n",
+    "    if \"madminer\" not in key:\n",
+    "        logging.getLogger(key).setLevel(logging.WARNING)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1.-2. Parameters and benchmarks"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We'll just load the MadMiner setup from the first part of this tutorial:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "miner = MadMiner()\n",
+    "miner.load('data/madminer_example.h5')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Set up systematics, save settings, run MadGraph"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This is where things become interesting: We want to model systematic uncertainties. Currently this can be done in one of two ways: based on scale variation or based on PDF variations. You can also use both simultaneously. Here we just vary the scales:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "miner.set_systematics(scale_variation=(0.5,2.), pdf_variation=None)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Again, we save our setup:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "miner.save('data/madminer_example_systematics.h5')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now it's time to run MadGraph. MadMiner will instruct MadGraph to use its built-in `systematics` tool to calculate how the event weights change under the scale variation."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "miner.run(\n",
+    "    sample_benchmark='sm',\n",
+    "    mg_directory=mg_dir,\n",
+    "    mg_process_directory='./mg_processes/signal_systematics',\n",
+    "    proc_card_file='cards/proc_card_signal.dat',\n",
+    "    param_card_template_file='cards/param_card_template.dat',\n",
+    "    run_card_file='cards/run_card_signal.dat',\n",
+    "    log_directory='logs/signal',\n",
+    "    python2_override=True,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Run smearing and extract observables"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This is just as before:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "lhe = LHEReader('data/madminer_example_systematics.h5')\n",
+    "\n",
+    "lhe.add_sample(\n",
+    "    lhe_filename='mg_processes/signal_systematics/Events/run_01/unweighted_events.lhe.gz',\n",
+    "    sampled_from_benchmark='sm',\n",
+    "    is_background=False,\n",
+    "    k_factor=1.1,\n",
+    ")\n",
+    "\n",
+    "lhe.set_smearing(\n",
+    "    pdgids=[1,2,3,4,5,6,9,22,-1,-2,-3,-4,-5,-6],   # Partons giving rise to jets\n",
+    "    energy_resolution_abs=0.,\n",
+    "    energy_resolution_rel=0.1,\n",
+    "    pt_resolution_abs=None,\n",
+    "    pt_resolution_rel=None,\n",
+    "    eta_resolution_abs=0.1,\n",
+    "    eta_resolution_rel=0.,\n",
+    "    phi_resolution_abs=0.1,\n",
+    "    phi_resolution_rel=0.,\n",
+    ")\n",
+    "\n",
+    "lhe.add_observable(\n",
+    "    'pt_j1',\n",
+    "    'j[0].pt',\n",
+    "    required=False,\n",
+    "    default=0.,\n",
+    ")\n",
+    "lhe.add_observable(\n",
+    "    'delta_phi_jj',\n",
+    "    'j[0].deltaphi(j[1]) * (-1. + 2.*float(j[0].eta > j[1].eta))',\n",
+    "    required=True,\n",
+    ")\n",
+    "lhe.add_observable(\n",
+    "    'met',\n",
+    "    'met.pt',\n",
+    "    required=True,\n",
+    ")\n",
+    "\n",
+    "lhe.add_cut('(a[0] + a[1]).m > 124.')\n",
+    "lhe.add_cut('(a[0] + a[1]).m < 126.')\n",
+    "lhe.add_cut('pt_j1 > 30.')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "lhe.analyse_samples()\n",
+    "lhe.save('data/madminer_example_systematics_with_data.h5')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### A look at distributions"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's see what our MC run produced:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "_ = plot_uncertainty(\n",
+    "    filename='data/madminer_example_systematics_with_data.h5',\n",
+    "    parameter_points=['sm', np.array([10.,0.])],\n",
+    "    line_labels=['SM', 'BSM'],\n",
+    "    uncertainties='none',\n",
+    "    n_bins=20,\n",
+    "    n_cols=3,\n",
+    "    normalize=True,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Make (unweighted) training and test samples with augmented data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sampler = SampleAugmenter('data/madminer_example_systematics_with_data.h5')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "When we generate training data, we now also have to specify the values of the nuisance parameters. The helper functions `sampling.nominal_nuisance_parameters()` and `sampling.iid_nuisance_parameters()` can be used in addition to the usual ones. The `theta0` and `theta1` return now includes values for the nuisance parameters."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "x, theta0, theta1, y, r_xz, t_xz, _ = sampler.sample_train_ratio(\n",
+    "    theta0=sampling.random_morphing_points(100, [('gaussian', 0., 15.), ('gaussian', 0., 15.)]),\n",
+    "    theta1=sampling.benchmark('sm'),\n",
+    "    nu0=sampling.iid_nuisance_parameters(\"gaussian\", 0., 1.),\n",
+    "    nu1=sampling.nominal_nuisance_parameters(),\n",
+    "    n_samples=1000,\n",
+    "    folder='./data/samples',\n",
+    "    filename='train'\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To be continued..."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 2",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.15"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}