Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

False positive http-crawl-non_statics Jellyseerr #1123

Open
jalapeno1083 opened this issue Sep 29, 2024 · 3 comments
Open

False positive http-crawl-non_statics Jellyseerr #1123

jalapeno1083 opened this issue Sep 29, 2024 · 3 comments

Comments

@jalapeno1083
Copy link

jalapeno1083 commented Sep 29, 2024 •
edited
Loading

Describe the bug
False positive when scrolling through Jellyseerr and loading a few different pages and scrolling down. This makes the client load many thumbnails.

edit: I just double checked. It only happens when I scroll down my request list domain.com/requests
edit2: Ok it also happens, when browsing normally. Scrolling down the request list just gets you banned faster.

To Reproduce
Install Jellyseerr and start browsing and scrolling. To get yourself banned even faster, fill your requests list and then scroll down the requests page domain.com/requests

Expected behavior

Using Jellyseerr normally like browsing, scrolling, loading thumbnails, and scrolling down the requests lists on Jellyseerr without getting banned.

Info about alert

cscli alerts inspect ################################################################################################

  • ID : 154

  • Date : 2024-09-29T14:30:09Z

  • Machine : localhost

  • Simulation : false

  • Remediation : true

  • Reason : crowdsecurity/http-crawl-non_statics

  • Events Count : 74

  • Scope:Value : Ip:XXXXX

  • Country : XXXXX

  • AS : XXXXX

  • Begin : 2024-09-29 14:29:50.845089625 +0000 UTC

  • End : 2024-09-29 14:30:08.794147427 +0000 UTC

  • UUID : 76bc859b-7c5d-4eb7-b829-c1eb5a1c6594

  • Context :
    +------------+---------------------+
    | Key | Value |
    +------------+---------------------+
    | method | GET |
    | status | 304 |
    | target_uri | /api/v1/request/585 |
    | target_uri | /api/v1/request/621 |
    | target_uri | /api/v1/request/642 |
    | target_uri | /api/v1/request/630 |
    | target_uri | /api/v1/request/633 |
    | target_uri | /api/v1/request/599 |
    | user_agent | - |
    +------------+---------------------+

  • Events :

  • Date: 2024-09-29 14:30:07 +0000 UTC
    +---------------------+-----------------------------+
    | Key | Value |
    +---------------------+-----------------------------+
    | ASNNumber | XXXXX |
    | ASNOrg | XXXXX |
    | IsInEU | false |
    | IsoCode | XXXXX |
    | SourceRange | XXXXX |
    | datasource_path | /var/log/traefik/access.log |
    | datasource_type | file |
    | http_args_len | 0 |
    | http_path | /api/v1/request/585 |
    | http_status | 304 |
    | http_user_agent | - |
    | http_verb | GET |
    | log_type | http_access-log |
    | service | http |
    | source_ip | XXXXX |
    | timestamp | 2024-09-29T14:30:07Z |
    | traefik_router_name | jellyseerr@file |
    | user | - |
    +---------------------+-----------------------------+

  • Date: 2024-09-29 14:30:07 +0000 UTC
    +---------------------+-----------------------------+
    | Key | Value |
    +---------------------+-----------------------------+
    | ASNNumber | XXXXX |
    | ASNOrg | XXXXX |
    | IsInEU | false |
    | IsoCode | XXXXX |
    | SourceRange | XXXXX |
    | datasource_path | /var/log/traefik/access.log |
    | datasource_type | file |
    | http_args_len | 0 |
    | http_path | /api/v1/request/621 |
    | http_status | 304 |
    | http_user_agent | - |
    | http_verb | GET |
    | log_type | http_access-log |
    | service | http |
    | source_ip | XXXXX |
    | timestamp | 2024-09-29T14:30:07Z |
    | traefik_router_name | jellyseerr@file |
    | user | - |
    +---------------------+-----------------------------+

  • Date: 2024-09-29 14:30:07 +0000 UTC
    +---------------------+-----------------------------+
    | Key | Value |
    +---------------------+-----------------------------+
    | ASNNumber | XXXXX |
    | ASNOrg | XXXXX |
    | IsInEU | false |
    | IsoCode | XXXXX |
    | SourceRange | XXXXX |
    | datasource_path | /var/log/traefik/access.log |
    | datasource_type | file |
    | http_args_len | 0 |
    | http_path | /api/v1/request/642 |
    | http_status | 304 |
    | http_user_agent | - |
    | http_verb | GET |
    | log_type | http_access-log |
    | service | http |
    | source_ip | XXXXX |
    | timestamp | 2024-09-29T14:30:07Z |
    | traefik_router_name | jellyseerr@file |
    | user | - |
    +---------------------+-----------------------------+

  • Date: 2024-09-29 14:30:07 +0000 UTC
    +---------------------+-----------------------------+
    | Key | Value |
    +---------------------+-----------------------------+
    | ASNNumber | XXXXX |
    | ASNOrg | XXXXX |
    | IsInEU | false |
    | IsoCode | XXXXX |
    | SourceRange | XXXXX |
    | datasource_path | /var/log/traefik/access.log |
    | datasource_type | file |
    | http_args_len | 0 |
    | http_path | /api/v1/request/630 |
    | http_status | 304 |
    | http_user_agent | - |
    | http_verb | GET |
    | log_type | http_access-log |
    | service | http |
    | source_ip | XXXXX |
    | timestamp | 2024-09-29T14:30:07Z |
    | traefik_router_name | jellyseerr@file |
    | user | - |
    +---------------------+-----------------------------+

  • Date: 2024-09-29 14:30:07 +0000 UTC
    +---------------------+-----------------------------+
    | Key | Value |
    +---------------------+-----------------------------+
    | ASNNumber | XXXXX |
    | ASNOrg | XXXXX |
    | IsInEU | false |
    | IsoCode | XXXXX |
    | SourceRange | XXXXX |
    | datasource_path | /var/log/traefik/access.log |
    | datasource_type | file |
    | http_args_len | 0 |
    | http_path | /api/v1/request/633 |
    | http_status | 304 |
    | http_user_agent | - |
    | http_verb | GET |
    | log_type | http_access-log |
    | service | http |
    | source_ip | XXXXX |
    | timestamp | 2024-09-29T14:30:07Z |
    | traefik_router_name | jellyseerr@file |
    | user | - |
    +---------------------+-----------------------------+

  • Date: 2024-09-29 14:30:07 +0000 UTC
    +---------------------+-----------------------------+
    | Key | Value |
    +---------------------+-----------------------------+
    | ASNNumber | XXXXX |
    | ASNOrg | XXXXX |
    | IsInEU | false |
    | IsoCode | XXXXX |
    | SourceRange | XXXXX |
    | datasource_path | /var/log/traefik/access.log |
    | datasource_type | file |
    | http_args_len | 0 |
    | http_path | /api/v1/request/599 |
    | http_status | 304 |
    | http_user_agent | - |
    | http_verb | GET |
    | log_type | http_access-log |
    | service | http |
    | source_ip | XXXXX |
    | timestamp | 2024-09-29T14:30:07Z |
    | traefik_router_name | jellyseerr@file |
    | user | - |
    +---------------------+-----------------------------+

Additional context

Collections in use

COLLECTIONS

Name 📦 Status Version Local Path

crowdsecurity/base-http-scenarios ✔️ enabled 1.0 /etc/crowdsec/collections/base-http-scenarios.yaml
crowdsecurity/http-cve ✔️ enabled 2.7 /etc/crowdsec/collections/http-cve.yaml
crowdsecurity/iptables ✔️ enabled 0.2 /etc/crowdsec/collections/iptables.yaml
crowdsecurity/linux ✔️ enabled 0.2 /etc/crowdsec/collections/linux.yaml
crowdsecurity/nginx ✔️ enabled 0.2 /etc/crowdsec/collections/nginx.yaml
crowdsecurity/sshd ✔️ enabled 0.5 /etc/crowdsec/collections/sshd.yaml
crowdsecurity/traefik ✔️ enabled 0.1 /etc/crowdsec/collections/traefik.yaml
crowdsecurity/whitelist-good-actors ✔️ enabled 0.1 /etc/crowdsec/collections/whitelist-good-actors.yaml
LePresidente/jellyfin ✔️ enabled 0.2 /etc/crowdsec/collections/jellyfin.yml
LePresidente/jellyseerr ✔️ enabled 0.1 /etc/crowdsec/collections/jellyseerr.yml

Happy to provide any additional logs.
@DanteMS
Copy link

DanteMS commented Oct 6, 2024 •
edited
Loading

I'm using the following whitelist for Jellyseerr:

name: overseerr-jellyseerr-whitelist
description: "Whitelist events from Overseerr and Jellyseerr"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type in ['http_access-log', 'http_error-log']"
whitelist:
  reason: "Overseerr/Jellyseerr whitelist"
  expression:
   - evt.Meta.http_status in ['200', '499'] && evt.Parsed.static_ressource == 'false' && evt.Meta.http_verb == 'GET' && evt.Meta.http_path matches '\\/api\\/v1\\/(movie|tv|request)\\/(\\d+)' # When browsing Movies, Series or Requests

Put it into /etc/crowdsec/config/parsers/s02-enrich.
Maybe someone could merge it with the Jellyseerr and the Overseerr collections?

@wacomoto
Copy link

Thanks for the information. I've just recently encountered this issue.

My set up had been working with the acquisition template looking directly at /var/log/jellyseerr/overseerr-*.log. After getting banned (http status 403 then 200) I have updated the acquisition file per the hub example and added a whitelist for Jellyseerr. The regex expression above didn't work for me and I've found only whitlisting http_status 200 to have been sufficient (so far). My whitelist for Jellyseerr on traefik:

name: crowdsecurity/jellyseerr-whitelists
description: "Whitelist false positives from Jellyseerr api"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type in ['http_access-log', 'http_error-log']"
whitelist:
  reason: "Whitelist false positive from Jellyseerr api"
  expression:
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/movie/'
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/tv/'
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/request/'

@Jgigantino31
Copy link

Jgigantino31 commented Jan 27, 2025 •
edited
Loading

I was having the same issue with Jellyseerr. Scrolling fast on the requests page would reliably cause http-crawl-non_statics to trigger. I am using nginx and having crowdsec parse nginx's access and error logs. The whitelist from @DanteMS works and I saw the whitelist count go up but I was still getting false positives. Investigating nginx's access log, I noticed Jellyseerr was also returning an HTTP status code 304 for a decent amount of requests which are not caught by the whitelist. Requests were also returned with the normal 200 and some with 499. After changing the whitelist to also include the HTTP 304 status code, all false positives stopped as all the requests are now captured in the whitelist.

name: overseerr-jellyseerr-whitelist
description: "Whitelist events from Overseerr and Jellyseerr"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type in ['http_access-log', 'http_error-log']"
whitelist:
  reason: "Overseerr/Jellyseerr whitelist"
  expression:
   - evt.Meta.http_status in ['200', '304', '499'] && evt.Parsed.static_ressource == 'false' && evt.Meta.http_verb == 'GET' && evt.Meta.http_path matches '\\/api\\/v1\\/(movie|tv|request)\\/(\\d+)' # When browsing Movies, Series or Requests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@DanteMS @jalapeno1083 @wacomoto @Jgigantino31