Skip to content

Searching For Requests

Panagis Alisandratos edited this page May 19, 2017 · 2 revisions

The /search/requests endpoint takes the following query-string parameters:

name type default description
query string None string to search by/for
foil_id bool False True if searching by FOIL Request ID
title bool False True if searching by Title
agency_description bool False True if searching by Agency Description
description bool False True if searching by Description
requester_name bool False True if searching by Requester Name
date_rec_from string None minimum Date Received to filter by
date_rec_to string None maximum Date Received to filter by
date_due_from string None minumum Date Due to filter by
date_due_to string None maximum Date Due to filter by
agency_ein string None Agency EIN to filter by
open bool False True if filtering by open requests
closed bool False True if filtering by closed requests
in_progress bool False True if filtering by in-progress requests
due_soon bool False True if filtering by due-soon requests
overdue bool False True if filtering by overdue requests
size int 100 number of requests to show per result page
start int 0 starting index of request result set
sort_date_received string None sort direction for Date Received
sort_date_due string None sort direction for Date Due
sort_title string None sort direction for Title
tz_name string None client timezone (e.g. America\New_York)
  • If the current user is anonymous, description will be set to False regardless of what is sent in the request
  • If the current user is not an agency user, requester_name, in_progress, due_soon, and overdue will all be set to False regardless of what is sent in the request
  • date_* parameter values must be in the python.datetime format: %m/%d/%Y
  • sort_* parameter values must be either desc or asc, any other values are ignored

When a GET request is sent to the /search/requests endpoint:

  1. Whitespace is stripped from the value for query

  2. If the stripped-value is an empty string or foil-id, title, agency_description, and description are all False, an empty result set is returned, otherwise...

  3. If foil_id yields True, "FOIL-" (if present) is stripped from query

  4. The sort argument value for elasticsearch-py.search is generated

    • e.g. if sort_date_received = asc, sort_date_due = desc, and sort_title = asc, then sort = ["date_received:asc", "date_due:desc", "title.keyword:asc"]
  5. A list of Request statuses to filter by is generated

    • e.g. if the values for open and due_soon are both True, then statuses = ["Open", "Due Soon"]
  6. A list of date ranges to filter by is generated, with dates offset to UTC based on tz_name

    • e.g. if date_received_from = 01/01/2000, date_received_to = 12/21/2012, and date_due_from = 09/23/1994 and tz_name = "Europe/London", then
    date_ranges = [
        {
            'range': {
                'date_received': {
                    'gte': '01/01/2000',
                    'lt': '12/21/2012'
                }
            }
        },
        {
            'range': {
                'date_due': {
                    'gte': '09/23/1994'
                }
            }
        },
    ]
    
  7. Using the values of query, title, description, agency_description, requester_name, and agency_ein, and the lists of statuses and date ranges, the body of the Elasticsearch Query DSL JSON dict is generated. For information on how the dict is generated see the Requests Query DSL Generator Methods section below.

  8. An elasticsearch query is executed and the results are formatted

    • Result dates are converted from UTC to the client's local time
    • Data from each result is used to render a Request result row in HTML
  9. The formatted results, along with the total and current number of hits is returned to the client

Requests Query DSL Generator Methods

foil_id

For searching by FOIL request ID only

# query = "0256", searching by foil id and 
# filtering by request statuses "Overdue" and "Closed"

{
  'query': {
    'bool': {
      'must': [
        {
          'wildcard': {
            '_uid': 'request#FOIL-*0256*'
          }
        },
        {
          'terms': {
            'status': [
              'Overdue', 
              'Closed'
            ]
          }
        }
      ]
    }
  }
}

agency_user

No search limits when compared to other users

# query = "Leeroy Jenkins", searching by requester name and 
# filtering by agency DOITT and status "In Progress"

{
  'query': {
    'bool': {
      'should': [
        {
          'bool': {
            'must': [
              {
                'match': {
                  'requester_name': 'Leeroy Jenkins'
                }
              },
              {
                "terms": {
                  "status": [
                    "In Progress"
                  ]
                }
              },
              {
                'term': {
                  'agency_ein': '0002'
                }
              }
            ]
          }
        }
      ]
    }
  } 
}

# query = "emails", searching by title and description, and 
# filtering by agency DOITT

{
  'query': {
    'bool': {
      'should': [
        {
          'bool': {
            'must': [
              {
                'match': {
                  'title': 'emails'
                }
              },
              {
                "terms": {
                  "status": [
                    "In Progress"
                  ]
                }
              },
              {
                'term': {
                  'agency_ein': '0002'
                }
              }
            ]
          }
        },
        {
          'bool': {
            'must': [
              {
                'match': {
                  'description': 'emails'
                }
              },
              {
                "terms": {
                  "status": [
                    "In Progress"
                  ]
                }
              },
              {
                'term': {
                  'agency_ein': '0002'
                }
              }
            ]
          }
        }
      ]
    }
  } 
}

anonymous_user

Limit to public title and public agency description

# query = "confidential", searching by title and agency description, and 
# filtering by status "Open"
 
{
  'query': {
    'bool': {
      'should': [
        {
          'bool': {
            'must': [
              {
                'match': {
                  'title': 'confidential'
                }
              },
              {
                'term': {
                  'title_private': false
                }
              },
              {
                'terms': {
                  'status': [
                    'Open'
                  ]
                }
              }
            ]
          }
        },
        {
          'bool': {
            'must': [
              {
                'match': {
                  'agency_description': 'confidential'
                }
              },
              {
                'term': {
                  'agency_descrption_private': false
                }
              },
              {
                'terms': {
                  'status': [
                    'Open'
                  ]
                }
              }
            ]
          }
        }
      ]
    }
  }
}

public_user

Limit to public title or private iff user is requester, public agency description, and description iff user is requester

# query = "sensitive", searching by title, agency description, and description
# Note the empty status list; no request statuses were selected

{
  'query': {
    'bool': {
      'should': [
        {
          'bool': {
            'must': [
              {
                'match': {
                  'title': 'sensitive'
                }
              },
              {
                'bool': {
                  'should': [
                    {
                      'term': {
                        'requester_id': '123-456'
                      }
                    },
                    {
                      'term': {
                        'title_private': false
                      }
                    }
                  ]
                }
              },
              {
                "terms" : {
                  "status": []
                }
              }
            ]   
          }
        },
        {
          'bool': {
            'must': [
              {
                'match': {
                  'agency_description': 'sensitive'
                }
              },
              {
                'term': {
                  'agency_description_private': false
                }
              },
              {
                "terms" : {
                  "status": []
                }
              }
            ]
          }
        },
        {
          'bool': {
            'must': [
              {
                'match': {
                  'description': 'sensitive'
                }
              },
              {
                'term': {
                  'requester_id': '123-456'
                }
              },
              {
                "terms" : {
                  "status": []
                }
              }
            ]
          }
        }
      ]
    }
  }
}

queryless

When empty search query is provided

# query = "", filtering by agency DORIS and by statuses "Open" and "Closed"

{
  'query': {
    'bool': {
      'must': [
        {
          'match_all': {}
        },
        {
          'terms': {
            'status': [
              'Open', 
              'Closed'
            ]
          }
        },
        {
          'term': {
            'agency_ein': '0860'
          }
        }
      ]
    }
  }
}

Inactive Parameters & Functionality

name type default description
highlight bool False True if including highlights in results
by_phrase bool False True if performing a match phrase query instead of the default match query

Highlights must be processed after search results have been fetched so that private and non-requester fields are not revealed to unauthorized users.