Block non-ckan routes from inventory-proxy

[nickumia-reisys]

User Story

In order to reduce noisy logs, the Data.gov security team wants to prevent non-ckan traffic from hitting the ckan app and causing non-helpful errors.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

• GIVEN proxy rules have been set up
  WHEN a non-ckan route is accessed from such as https://inventory.data.gov/vulnerable.php
  THEN There should not be errors in NR

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

• Inventory high error rate due to scanning/probing traffic #3635
• Get list of CKAN Routes #4042

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

[Notes or a checklist reflecting our understanding of the selected approach]

[nickumia-reisys]

Since you asked yesterday @robert-bryson, it's not necessary to whitelist each full route independently. We can still gain a lot of benefit by just whitelisting the top-level routes (i.e. the "Parent" Routes minus the '/' route). We'll want / to be available, but not /*.

Edit 1: The only routes that may be problematic are the "static files" (i.e. the webassets [js/css] and any other files that have custom routes. I don't anticipate this to be a large effort, but if anyone knows differently, please advise.

Edit 2: If you are still worried about having to maintain a bunch of code in the nginx.conf, we could look into writing a python script that generates the conf file, so we don't need to edit it directly, but whether that is worth it depends on how many lines we have. If we have 20 routes to whitelist, I think it's okay. If it's 50 (for whatever reason), maybe generating it is not such a bad idea. I think we'll have less than 20 though, so 🤷

[robert-bryson]

With this PR and a successful deploy into prod, this is done. I think this will be something of a first pass on reducing noise in New Relic. Future work to tighten the rules further may be needed.

As written this ticket targets inventory specifically. @nickumia-reisys & @FuhuXia & team as a whole, do you think this will need to be done for catalog as well?