Skip to content

Use the Allen Brain Observatory – Visual Coding on AWS

Jump to bottom
David Feng edited this page Jun 19, 2018 · 38 revisions

Allen Brain Observatory – Visual Coding on AWS

The Allen Brain Observatory – Visual Coding is the first standardized in vivo survey of physiological activity in the mouse visual cortex, featuring representations of visually evoked calcium responses from GCaMP6-expressing neurons in selected cortical layers, visual areas, and Cre lines. All of this data is now available to everyone in Amazon S3. We are excited to make the motion-corrected calcium fluorescence videos for all recording sessions available in S3 so that users can systematically test their own analysis algorithms on the entire data set without having to ship physical hard disks. Users also have access to the Neurodata Without Borders (NWB) files containing raw and baseline-corrected fluorescence traces extracted by the Allen Institute for comparison.

Experiment Organization

In this data set, we record from a single population of neurons while the mouse passively observes a battery of visual stimuli in three separate ~90 min. sessions. The three sessions consist of interleaved presentations of the following stimuli:

  • drifting gratings, natural movies (clip 1, clip 3)
  • static gratings, natural movies (clip 1), natural scenes/images
  • locally sparse noise, natural movies (clip 1, clip 2)

We call the combination of the three recordings from a single population of neurons an “experiment container.” Each session has a separate NWB file with cell-level response data, for a total of 3 NWB files in each experiment container. Learn more about the design of the experiment, the stimulus protocol of each session, and the organization of the data on our web site and in our technical whitepapers.

Access Data in S3 via Allen SDK

The Allen Brain Observatory data set is hosted on Amazon Web Services (AWS). In order to use the data set, you need to have an AWS account. You can create an AWS account by following these instructions. The instructions below walk through the steps necessary for creating a Jupyter notebook instance and using this data set.

Option 1: Create a SageMaker Jupyter Notebook Instance via the Launch Button

  1. Click on
  2. Continue clicking next until you get to the review page.
  3. On the review page, check the checkbox that allows the AWS Cloudformation to create roles and click on Create. You will be redirected to the Cloudformation page. Wait for the template to be created.
  4. You can check the status of the notebook instance here.

The URL of the notebook instance is the following: https://allen-brain-observatory.notebook.us-west-2.sagemaker.aws/tree

Option 2: Create a SageMaker Jupyter Notebook Instance via the AWS CLI

  1. Install the AWS CLI by following these instructions.
  2. Configure your machine to use your AWS account by following these instructions.
  3. Download the template from here: https://s3-us-west-2.amazonaws.com/allen-brain-observatory-templates/cloudformation/allen-brain-observatory-sagemaker.yml
  4. Run the following command in the directory where you downloaded the template and wait for the instance to be created
aws cloudformation create-stack --stack-name allen-brain-observatory --template-body file://./allen-brain-observatory-sagemaker.yml --capabilities CAPABILITY_IAM

You can check the status of the notebook instance here.

The URL of the notebook instance is the following: https://allen-brain-observatory.notebook.us-west-2.sagemaker.aws/tree

Play with the Data

Once your notebook is running, you can access frames of a video like this:

import h5py
import matplotlib.pyplot as plt
%matplotlib inline
Clone this wiki locally