add proxy polling restriction, allow to query "/clients/host" #178

peekjef72 · 2024-05-26T10:32:17Z

Add in the proxy part the ability to specify which networks (or servers) can request the proxy.
This allows you to define two roles:

pollers: The pollers, generally the prometheus hosts, can scrap target through the proxy.
targets.: they can only register on the proxy and respond to requests from it.

The feature makes it possible to fill the security hole which allows in the current version two neighboring hosts, even if they cannot communicate directly, to obtain metrics from each other or the list of all hosts known by the proxy.

To enable the feature; add in the command line interface of the pushprox_proxy:

--scrape.pollers-ip 'ipV4/32,netV4#2/net,...

example: start pushproxy proxy and client on same server:

proxy listen on every IP on port 8080 (default), polling only authorized through loopback
client accessible through fqdn "localhost" connected to proxy on server ip (192.168.0.196:8080)

$ ./pushprox-proxy --scrape.pollers-ip 127.0.0.1/32 &
$ ./pushprox-client --fqdn=localhost --proxy-url=http://192.168.0.196:8080 &

Try to scrap an exporter listening on port 9321 on same server through server ip: reply "403 Forbidden" and in body "Not an authorized poller"

$ curl -v --proxy http://192.168.0.196:8080 -H "X-Prometheus-Scrape-Timeout-Seconds: 5" "http://localhost:9321"
* Rebuilt URL to: http://localhost:9321/
*   Trying 192.168.0.196...
* TCP_NODELAY set
* Connected to 192.168.0.196 (192.168.0.196) port 8080 (#0)
> GET http://localhost:9321/ HTTP/1.1
> Host: localhost:9321
> User-Agent: curl/7.61.1
> Accept: */*
> Proxy-Connection: Keep-Alive
> X-Prometheus-Scrape-Timeout-Seconds: 5
> 
ts=2024-05-26T09:53:37.089Z caller=main.go:288 level=error msg="can't reverse client address" err="lookup 196.0.168.192.in-addr.arpa. on 192.168.0.1:53: no such host"
< HTTP/1.1 403 Forbidden
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Sun, 26 May 2024 09:53:37 GMT
< Content-Length: 25
< 
Not an authorized poller
* Connection #0 to host 192.168.0.196 left intact

try to scrap httpapi_exporter through loopback ip :

$ curl -v --proxy http://127.0.0.1:8080 -H "X-Prometheus-Scrape-Timeout-Seconds: 5" "http://localhost:9321"
* Rebuilt URL to: http://localhost:9321/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET http://localhost:9321/ HTTP/1.1
> Host: localhost:9321
> User-Agent: curl/7.61.1
> Accept: */*
> Proxy-Connection: Keep-Alive
> X-Prometheus-Scrape-Timeout-Seconds: 5
> 
ts=2024-05-26T10:03:02.010Z caller=coordinator.go:122 level=info msg=DoScrape scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d url=http://localhost:9321/
ts=2024-05-26T10:03:02.010Z caller=main.go:208 level=info msg="Responded to /poll" url=http://localhost:9321/ scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d
ts=2024-05-26T10:03:02.010Z caller=main.go:206 level=info msg="Got scrape request" scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d url=http://localhost:9321/
ts=2024-05-26T10:03:02.011Z caller=coordinator.go:143 level=info msg=WaitForScrapeInstruction fqdn=localhost
ts=2024-05-26T10:03:02.011Z caller=main.go:140 level=info scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d msg="Retrieved scrape response"
ts=2024-05-26T10:03:02.012Z caller=main.go:189 level=info msg="Got /push" scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d
ts=2024-05-26T10:03:02.012Z caller=coordinator.go:175 level=info msg=ScrapeResult scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d
ts=2024-05-26T10:03:02.012Z caller=main.go:146 level=info scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d msg="Pushed scrape result"
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Date: Sun, 26 May 2024 10:03:02 GMT
< X-Prometheus-Scrape-Timeout: 4.998953
< Transfer-Encoding: chunked
< 
<html>
      <head>
        <title>Prometheus httpapi_exporter</title>
        <style type="text/css">
          body { margin: 0; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.42857143; color: #333; background-color: #fff; }
          .navbar { display: flex; background-color: #222; margin: 0; border-width: 0 0 1px; border-style: solid; border-color: #080808; }
....

Add the feature to query if a specified fqdn can be found the the fqdn list of knwown hosts
two cases:

Poller role is defined: only pollers are authorized to access the full clients lists,
poller is not defined: as usuall the full list is available.

e.g.: poller role is not defined.

can query the full list /clients
can query a specfic fqdn /client/fqdn : if not found return 404

$ curl -v http://192.168.0.196:8080/clients/
*   Trying 192.168.0.196...
* TCP_NODELAY set
* Connected to 192.168.0.196 (192.168.0.196) port 8080 (#0)
> GET /clients/ HTTP/1.1
> Host: 192.168.0.196:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 26 May 2024 10:22:05 GMT
< Content-Length: 42
< 
[{"targets":["localhost"],"labels":null}]
* Connection #0 to host 192.168.0.196 left intact

look for fqdn "localhost":

$ curl -v http://192.168.0.196:8080/clients/localhost
*   Trying 192.168.0.196...
* TCP_NODELAY set
* Connected to 192.168.0.196 (192.168.0.196) port 8080 (#0)
> GET /clients/localhost HTTP/1.1
> Host: 192.168.0.196:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 26 May 2024 10:22:36 GMT
< Content-Length: 42
< 
[{"targets":["localhost"],"labels":null}]
* Connection #0 to host 192.168.0.196 left intact

poller role is defined ( 127.0.0.1:8080 only )

$ curl -v http://127.0.0.1:8080/clients/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /clients/ HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 26 May 2024 10:27:10 GMT
< Content-Length: 42
< 
[{"targets":["localhost"],"labels":null}]
* Connection #0 to host 127.0.0.1 left intact

$ curl -v http://127.0.0.1:8080/clients/localhost
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /clients/localhost HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 26 May 2024 10:27:19 GMT
< Content-Length: 42
< 
[{"targets":["localhost"],"labels":null}]
* Connection #0 to host 127.0.0.1 left intact


$ curl -v http://127.0.0.1:8080/clients/host1
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /clients/host1 HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Sun, 26 May 2024 10:27:24 GMT
< Content-Length: 1
< 

* Connection #0 to host 127.0.0.1 left intact

SuperQ

Please remove commented out code.

Also, this needs a rebase with the change to the new logging library.

Signed-off-by: peekjef72 <[email protected]>

peekjef72 force-pushed the master branch from 5aed93b to 5302694 Compare June 1, 2024 13:54

SuperQ requested changes Nov 8, 2024

View reviewed changes

add proxy polling restriction

2ff6709

Signed-off-by: peekjef72 <[email protected]>

peekjef72 force-pushed the master branch from 5302694 to 2ff6709 Compare November 10, 2024 15:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add proxy polling restriction, allow to query "/clients/host" #178

add proxy polling restriction, allow to query "/clients/host" #178

peekjef72 commented May 26, 2024 •

edited

Loading

SuperQ left a comment

add proxy polling restriction, allow to query "/clients/host" #178

Are you sure you want to change the base?

add proxy polling restriction, allow to query "/clients/host" #178

Conversation

peekjef72 commented May 26, 2024 • edited Loading

SuperQ left a comment

Choose a reason for hiding this comment

peekjef72 commented May 26, 2024 •

edited

Loading