Skip to content

API

Jump to bottom
Leon Jacobs edited this page May 7, 2023 · 10 revisions

An API is available for programmatic access to the database and or to trigger screenshots. To access the API, the web interface needs to be started. This can be done with gowitness server. Use the --help flag to learn how to set an alternate listen port / address.

Note: gowitness was built with the idea that the reporting server will only be served on localhost, and as such has no authentication mechanism built in. If you want to expose the API server to a wider network then it is highly encouraged that you place authentication in front of the server. An example of how to do that using Traefik is available here.

Endpoints

An up to date reference on the API endpoints available can be found in the source code here and here. All endpoints live under the /api/ path. For example, /api/list, /api/detail/:id.

get list

  • HTTP Verb: GET
  • Endpoint: /list
  • Parameters: None

Get's all of the screenshots in the database, returned as a JSON formatted array. The ID field in the payload should be used as a reference in a following /detail/* call to fetch the appropriate details.

Example call:

curl localhost:7171/api/list

Example Response:

[
  {
    "ID": 1,
    "URL": "https://whatsapp.com",
    "FinalURL": "https://www.whatsapp.com/",
    "ResponseCode": 200,
    "Title": "Whatsapp"
  },
  {
    "ID": 2,
    "URL": "https://bing.com",
    "FinalURL": "https://www.bing.com:443/?toWww=1&redig=DB01213BD54C426785DF89E5C3DAFFCF",
    "ResponseCode": 200,
    "Title": "Bing"
  }
]

get detail

  • HTTP Verb: GET
  • Endpoint: /detail/:id
  • Parameters: :id is the id of the URL to query

Get's details about a screenshotted URL. This endpoint can be quite verbose as it includes the DOM, all of console logs, network logs, headers and TLS information that could be saved.

Example call:

curl localhost:7171/api/detail/2

Example Response:

{
  "ID": 2,
  "CreatedAt": "2022-06-10T15:48:48.411458+02:00",
  "UpdatedAt": "2022-06-10T15:48:48.558231+02:00",
  "DeletedAt": null,
  "URL": "https://bing.com",
  "FinalURL": "https://www.bing.com:443/?toWww=1&redig=DB01213BD54C426785DF89E5C3DAFFCF",
  "ResponseCode": 200,
  "ResponseReason": "200 OK",
  "Proto": "HTTP/1.1",
  "ContentLength": -1,
  "Title": "Bing",
  "Filename": "https-bing.com.png",
  "IsPDF": false,
  "PerceptionHash": "p:9bcb1df47430308f",
  "DOM": "<html></html>",
  "TLS": {
    "ID": 2,
    "CreatedAt": "2022-06-10T15:48:48.412845+02:00",
    "UpdatedAt": "2022-06-10T15:48:48.412845+02:00",
    "DeletedAt": null,
    "URLID": 2,
    "Version": 771,
    "ServerName": "www.bing.com",
    "TLSCertificates": [
      {
        "ID": 3,
        "CreatedAt": "2022-06-10T15:48:48.413079+02:00",
        "UpdatedAt": "2022-06-10T15:48:48.413079+02:00",
        "DeletedAt": null,
        "TLSID": 2,
        "Raw": null,
        "DNSNames": [
          {
            "ID": 8,
            "CreatedAt": "2022-06-10T15:48:48.413468+02:00",
            "UpdatedAt": "2022-06-10T15:48:48.413468+02:00",
            "DeletedAt": null,
            "TLSCertificateID": 3,
            "Name": "www.bing.com"
          }
        ]
      }
    ]
  },
  "Headers": [
    {
      "ID": 9,
      "CreatedAt": "2022-06-10T15:48:48.414665+02:00",
      "UpdatedAt": "2022-06-10T15:48:48.414665+02:00",
      "DeletedAt": null,
      "URLID": 2,
      "Key": "P3p",
      "Value": "CP=\"NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND\""
    },
    {
      "ID": 10,
      "CreatedAt": "2022-06-10T15:48:48.414665+02:00",
      "UpdatedAt": "2022-06-10T15:48:48.414665+02:00",
      "DeletedAt": null,
      "URLID": 2,
      "Key": "Set-Cookie",
      "Value": "SUID=M;"
    }
  ],
  "Technologies": [
    {
      "ID": 20,
      "CreatedAt": "2022-06-10T15:49:30.410426+02:00",
      "UpdatedAt": "2022-06-10T15:49:30.410426+02:00",
      "DeletedAt": null,
      "URLID": 52,
      "Value": "Google Font API"
    },
    {
      "ID": 21,
      "CreatedAt": "2022-06-10T15:49:30.410426+02:00",
      "UpdatedAt": "2022-06-10T15:49:30.410426+02:00",
      "DeletedAt": null,
      "URLID": 52,
      "Value": "React"
    }
  ],
  "Console": [
    {
      "ID": 92,
      "CreatedAt": "2022-06-10T15:49:30.410989+02:00",
      "UpdatedAt": "2022-06-10T15:49:30.410989+02:00",
      "DeletedAt": null,
      "URLID": 52,
      "Time": "0001-01-01T00:00:00Z",
      "Type": "console.error",
      "Value": "\"[GSI_LOGGER]: Check credential status returns invalid response.\""
    }
  ],
  "Network": [
    {
      "ID": 4306,
      "CreatedAt": "2022-06-10T15:49:30.412363+02:00",
      "UpdatedAt": "2022-06-10T15:49:30.412363+02:00",
      "DeletedAt": null,
      "URLID": 52,
      "RequestID": "88745.136",
      "RequestType": 0,
      "StatusCode": 0,
      "URL": "https://url.com",
      "FinalURL": "",
      "IP": "",
      "Time": "2022-06-10T06:51:21.013651999+02:00",
      "Error": "net::ERR_CONNECTION_REFUSED"
    },
  ]
}

get screenshot

  • HTTP Verb: GET
  • Endpoint: /detail/:id/screenshot
  • Parameters: :id is the id of the URL to query

Get's the raw screenshot taken. Depending on if IsPDF is true from the above /detail/:id endpoint, the response would be either a PDF or a PNG file.

Example call:

curl localhost:7171/api/detail/52/screenshot

Example Response:

Binary data to be saved to a file.

search

  • HTTP Verb: GET
  • Endpoint: /search
  • Parameters: q is the value to search for

Searches the database for data that contains the query specified in the q parameter. This endpoint can be quite verbose as it includes the DOM, all of console logs, network logs, headers and TLS information that could be saved.

Example call:

curl localhost:7171/api/search?q=bing.com

Example Response:

{
  "ID": 2,
  "CreatedAt": "2022-06-10T15:48:48.411458+02:00",
  "UpdatedAt": "2022-06-10T15:48:48.558231+02:00",
  "DeletedAt": null,
  "URL": "https://bing.com",
  "FinalURL": "https://www.bing.com:443/?toWww=1&redig=DB01213BD54C426785DF89E5C3DAFFCF",
  "ResponseCode": 200,
  "ResponseReason": "200 OK",
  "Proto": "HTTP/1.1",
  "ContentLength": -1,
  "Title": "Bing",
  "Filename": "https-bing.com.png",
  "IsPDF": false,
  "PerceptionHash": "p:9bcb1df47430308f",
  "DOM": "<html></html>",
  "TLS": {
    "ID": 2,
    "CreatedAt": "2022-06-10T15:48:48.412845+02:00",
    "UpdatedAt": "2022-06-10T15:48:48.412845+02:00",
    "DeletedAt": null,
    "URLID": 2,
    "Version": 771,
    "ServerName": "www.bing.com",
    "TLSCertificates": [
      {
        "ID": 3,
        "CreatedAt": "2022-06-10T15:48:48.413079+02:00",
        "UpdatedAt": "2022-06-10T15:48:48.413079+02:00",
        "DeletedAt": null,
        "TLSID": 2,
        "Raw": null,
        "DNSNames": [
          {
            "ID": 8,
            "CreatedAt": "2022-06-10T15:48:48.413468+02:00",
            "UpdatedAt": "2022-06-10T15:48:48.413468+02:00",
            "DeletedAt": null,
            "TLSCertificateID": 3,
            "Name": "www.bing.com"
          }
        ]
      }
    ]
  },
  "Headers": [
    {
      "ID": 9,
      "CreatedAt": "2022-06-10T15:48:48.414665+02:00",
      "UpdatedAt": "2022-06-10T15:48:48.414665+02:00",
      "DeletedAt": null,
      "URLID": 2,
      "Key": "P3p",
      "Value": "CP=\"NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND\""
    },
    {
      "ID": 10,
      "CreatedAt": "2022-06-10T15:48:48.414665+02:00",
      "UpdatedAt": "2022-06-10T15:48:48.414665+02:00",
      "DeletedAt": null,
      "URLID": 2,
      "Key": "Set-Cookie",
      "Value": "SUID=M;"
    }
  ],
  "Technologies": [
    {
      "ID": 20,
      "CreatedAt": "2022-06-10T15:49:30.410426+02:00",
      "UpdatedAt": "2022-06-10T15:49:30.410426+02:00",
      "DeletedAt": null,
      "URLID": 52,
      "Value": "Google Font API"
    },
    {
      "ID": 21,
      "CreatedAt": "2022-06-10T15:49:30.410426+02:00",
      "UpdatedAt": "2022-06-10T15:49:30.410426+02:00",
      "DeletedAt": null,
      "URLID": 52,
      "Value": "React"
    }
  ],
  "Console": [
    {
      "ID": 92,
      "CreatedAt": "2022-06-10T15:49:30.410989+02:00",
      "UpdatedAt": "2022-06-10T15:49:30.410989+02:00",
      "DeletedAt": null,
      "URLID": 52,
      "Time": "0001-01-01T00:00:00Z",
      "Type": "console.error",
      "Value": "\"[GSI_LOGGER]: Check credential status returns invalid response.\""
    }
  ],
  "Network": [
    {
      "ID": 4306,
      "CreatedAt": "2022-06-10T15:49:30.412363+02:00",
      "UpdatedAt": "2022-06-10T15:49:30.412363+02:00",
      "DeletedAt": null,
      "URLID": 52,
      "RequestID": "88745.136",
      "RequestType": 0,
      "StatusCode": 0,
      "URL": "https://url.com",
      "FinalURL": "",
      "IP": "",
      "Time": "2022-06-10T06:51:21.013651999+02:00",
      "Error": "net::ERR_CONNECTION_REFUSED"
    },
  ]
}

take a screenshot

  • HTTP Verb: POST
  • Endpoint: /screenshot
  • Parameters: A post body containing the URL to be processed and a flag if the screenshot should be returned. Optionally you may specify additional headers to send along with the screenshot request. Post body example:
{
  "url": "https://google.com",
  "oneshot": false,
  "headers" : [
    "foo: bar",
    "baz: foo"
  ]
}

Take a screenshot of the url specified. If oneshot is set to true, the screenshot will be taken and reflected back to the requestor. If oneshot is set to false, the screenshot will be processed in the background, and the results persisted to the database.

Example request:

# request a screenshot to be reflected back without persisting it to the database
curl -X POST http://localhost:7171/api/screenshot --data '{"url": "https://google.com", "oneshot": "true"}'

# request a screenshot to be processed in the background
curl -X POST http://localhost:7171/api/screenshot --data '{"url": "https://google.com", "oneshot": "false"}'

# request a screenshot to be processed in the background, with extra headers
curl -X POST http://localhost:7171/api/screenshot --data '{"url":"<url>", "oneshot": "false", "headers": ["foo: bar", "baz: foo"]}'

Example response (will only return json if oneshot was false. otherwise a png image will be returned):

{
  "status": "created"
}

Table of Contents

Table of Contents

Guides

Clone this wiki locally