Skip to content

Latest commit

 

History

History
213 lines (171 loc) · 7.54 KB

README.md

File metadata and controls

213 lines (171 loc) · 7.54 KB
Raw

JSON-stat for Eurostat

JSON-stat for Eurostat (EuroJSONstat, for short) is a JavaScript library to deal with Eurostat’s JSON-stat API (requests and responses). EuroJSONstat is part of the JSON-stat Toolkit. It is meant to complement, not replace, Eurostat’s own set of libraries (eurostat.js). While the latter addresses many issues, EuroJSONstat library is focused on JSON-stat-related ones.

For instance, Eurostat does not include roles in its datasets. EuroJSONstat offers a method to retrieve role-enriched datasets (fetchDataset) or allows you to add role information to an already retrieved Eurostat dataset (setRole). This is possible because Eurostat uses standardized dimension names.

JSON-stat does not provide a single way to describe status symbols. Instead, it offers the extension property to allow providers to customize their JSON-stat responses to their needs. Eurostat uses a standardized way to attach a label to each status symbol. EuroJSONstat offers a method to retrieve such information (getStatusLabel).

A design principle of the JSON-stat format is the strict separation of data and metadata in order to allow the use of exactly the same structure for full (data and metadata) responses, data-only responses and metadata-only responses. Unfortunately, Eurostat does not support metadata-only responses. EuroJSONstat offers several ways to avoid this limitation: fetchFullQuery and, since version 2.1, fetchEmptyDataset.

EuroJSONstat.fetchEmptyDataset("ei_cphi_m").then(js=>{
  if(js.class==="error"){
    console.log(`Error ${js.status} (${js.label})`);
  }else{
    console.log(js); //JSON-stat dataset with "value" as an empty array
  }
});

Finally, EuroJSONstat includes a function to convert a fully explicit query (see later) into a metadata-only dataset instance (getEmptyDataset).

Resources

Table of Contents

Queries

Queries are a special kind of object used in many EuroJSONstat functions as an argument (input query) but also as a return value (output query).

A query must contain a dataset property with a valid Eurostat dataset code:

{
  "dataset": "une_rt_a"
}

A query can optionally request information on one of the Eurostat-supported languages (default is English). A specific Eurostat API version can also be requested:

{
  "dataset": "une_rt_a",
  "lang": "fr",
  "version": "1.0"
}

Since version 2.0.0, EuroJSONstat uses Eurostat's API Statitics and the default API version is 1.0. Prior versions of EuroJSONstat used Eurostat's JSON Web Service (the default version was 2.1).

Queries can filter data in datasets. For instance, dataset une_rt_a for Austria in years 2017-2018 would require this query:

{
  "dataset": "une_rt_a",
  "filter": {
    "geo": ["AT"],
    "time": ["2017", "2018"]
  }
}

EuroJSONstat functions that return queries always include a class property with value "query" and can include label information. For instance, the previous implicit query (an input query that relies on some default parameters and values) can be tried to be translated into an explicit query (an equivalent query that includes all values requested explicitly or implicitly) using the fetchQuery function:

EuroJSONstat.fetchQuery(iq, false).then(eq=>{
  if(eq.class!=="error"){
    console.log(eq);
  }
});

fetchQuery returns a metadata-enriched query:

{
  "class": "query",
  "dataset": "une_rt_a",
  "filter": {
    "age": ["TOTAL", "Y25-74", "Y_LT25"],
    "unit": ["PC_ACT", "PC_POP", "THS_PER"],
    "sex": ["F", "M", "T"],
    "geo": ["AT"],
    "time": ["2017", "2018"]
  },
  "label": {
    "dataset": "Unemployment by sex and age - annual average",
    "dimension": {
      "age": "age",
      "unit": "unit",
      "sex": "sex",
      "geo": "geo",
      "time": "time"
    },
    "category": {
      "age": ["Total", "From 25 to 74 years", "Less than 25 years"],
      "unit": ["Percentage of active population", "Percentage of total population", "Thousand persons"],
      "sex": ["Females", "Males", "Total"],
      "geo": ["Austria"],
      "time": ["2018"]
    }
  },
  "lang": "en",
  "version": "1.0"
}

A fully explicit query is a query that exposes all the dimensions and categories of a non-filtered dataset.

EuroJSONstat.fetchFullQuery({
  "dataset": "une_rt_a",
  "lang": "fr"
}).then(eq=>{
  if(eq.class==="error"){
    console.log(`Error ${eq.status} (${eq.label})`);
  }else{
    console.log(eq);
  }
});

Filters

Another special kind of object are filters. The "filter" property of queries is a filter. Filters are made of parameters (properties of the filter object), usually dimension names, and a list of valid values (array), usually category names. For example:

{
  "geo": ["AT"],
  "time": ["2017", "2018"]
}

Some EuroJSONstat functions accept filters as arguments. For example, the addParamQuery can create a new query from a query and a filter:

EuroJSONstat.addParamQuery(
  //query
  {
    "dataset": "une_rt_a",
    "filter": {
      "geo": ["AT"],
      "time": ["2017", "2018"]
    }
  },
  //filter
  {
    "age": ["TOTAL"]
  }
);

Sample code

The following sample code tries to show the combined use of many functions in EuroJSONstat. Since version 2.1, a similar goal can be attained with the use of a single function: fetchEmptyDataset.

//A query
const query={
  dataset: "une_rt_a"
};

//Queries can be modified easily

//Add filter (Austria 2018) to previous query
const fquery=EuroJSONstat.addParamQuery(query, { geo: ["AT"], time: ["2018"] });
console.log(fquery);

//Translate query into Eurostat end-point
const url=EuroJSONstat.getURL(fquery);
console.log(url);

//Remove time and geo from previous filtered query
const uquery=EuroJSONstat.removeParamQuery(fquery, ["time", "geo"]);
console.log(uquery);
//To remove time, removeTimeQuery() is more powerful than generic removeParamQuery()

//Retrieve a Eurostat standardized JSON-stat dataset using a query
EuroJSONstat.fetchDataset(uquery).then(ds=>{
  if(ds.class==="dataset"){
    console.log(`"${query.dataset}" dataset has label "${ds.label}".`);

    const status=ds.Data(0).status;

    //Eurostat status meaning is easily retrieved
    console.log(`Status symbol of first observation is "${status}" which means "${EuroJSONstat.getStatusLabel(ds, status)}".`);

    //When standardized, Eurostat’s datasets are enriched with roles
    console.log(`Classification dimensions: ${ds.role.classification}`);    
  }
});

//Get a fully explicit query from previous query
//and convert it into a dataset instance
EuroJSONstat.fetchFullQuery(uquery).then(fq=>{
  if(fq.class==="query"){
    //Convert the fully explicit query into a dataset instance
    const ds=EuroJSONstat.getEmptyDataset(fq);
    console.log(`"${ds.extension.datasetId}" dataset has label "${ds.label}".`);
    console.log(`Classification dimensions: ${ds.role.classification}`);    
    console.log(`Available time periods: ${ds.Dimension("time", false)}`);    
    console.log(`Available geographical areas: ${ds.Dimension("geo", false)}`);    
  }
});