[v6] Resolving of opendns family shield umbrella page not working

[urmet0]

Versions

Core
Version is 4902c70 (Latest: null)
Branch is development
Hash is 4902c700 (Latest: e682f69a)
Web
Version is 603f35d (Latest: null)
Branch is development
Hash is 603f35d5 (Latest: fb368323)
FTL
Version is vDev-f2c82da (Latest: null)
Branch is development
Hash is f2c82da (Latest: 80ef20e)

Platform

• OS and version: Debian 12
• Platform: Raspberry Pi

Expected behavior

Using OpenDNS family shield upstream dns with pihole v6 should redirect you to cisco umbrella page (like pihole v5) did.

Set up these upstream DNS servers (=opendns family shield dns):
208.67.222.123
208.67.220.123
When opendns (upstream) is blocking a request, it sends you to its cisco umbrella page.
You can test that (having the mentioned DNS servers set up) by opening this test-url: http://www.internetbadguys.com/

Expected is, that you get redirected to a page like that:
https://phish.opendns.com/main?url=www.internetbadguys.com&server=ams1&prefs=&tagging=&nref

With Pihole v5 this works.

Actual behavior / bug

When I try this with pihole v6, it does not work any more.

A nslookup shows the different answers of 2 different pihole (v5 vs v6).
Pihole v5 answers with the address of the umbrella-page (146.112.61.108) and pihole v6 answers with 0.0.0.0.
No matter what option I change in pihole configuration, it did not change this behavior.

1) Pihole V5
nslookup www.internetbadguys.com
Server: 192.168.1.2
Address: 192.168.1.2#53

Non-authoritative answer:
Name: www.internetbadguys.com
Address: 146.112.61.108

2) Pihole V6
nslookup www.internetbadguys.com 192.168.1.3
Server: 192.168.1.3
Address: 192.168.1.3#53

Name: www.internetbadguys.com
Address: 0.0.0.0

Steps to reproduce

Steps to reproduce the behavior:

1. Run development version of pihole via docker (image: pihole/pihole:development)
2. Configure upstream-DNS: 208.67.222.123, 208.67.220.123
3. Access www.internetbadguys.com
4. You will get an empty page in your browser, but you should get redirected to a cisco-umbrella page (maybe you have to accept certificate-issues, but after that you should see the cisco-umbrella page).

Debug Token

• URL: https://tricorder.pi-hole.net/5pmr2v1l/

Screenshots

Additional context

[DL6ER]

Yes, this is expected because the Umbrella page is a known block page. The detection of upstream IP blocking has been introduced six years ago in #372. It should be part of Pi-hole since v4.1.

It seem the mere "detection" has evolved to a more active "local blocking" at some point during development of Pi-hole v6.

Anyway ... There are now two things I could see us doing to fix this for you:

1. You could use dns.blocking.mode = "IP" and configure `dns.reply.host.force4 = "146.112.61.108"' to effectively send any blocked domain to the Umbrella page, or
2. We add a new option to control the detection of upstream blocking so Pi-hole will not intervene and present 0.0.0.0 for you but pass along the original reply from upstream.

I am pretty sure you'd want no. 2 but please sit back and think about it once more, I will do the same. I am not even sure what the default setting for such an option should be. There are good arguments for both defaulting to blocking and defaulting to just pass the IP along.

---

edit This is re responsible change in v6: #2030

[DL6ER]

Having thought about it, I think we need to make an exception here as this unintended change in behavior is probably not unwanted and has actually been added without checking for this side-effect of suppressing upstream-provided blocking pages.

Either we stop caching replies with a known blocking IP address or we store it in the cache and serve it from there to achieve the original purpose. But then we need to (considerably) enlarge the DNS blocking cache from currently 40 bytes to then 56 bytes (i.e., + 40%, numbers measured on x86_64) as it currently has no memory to store IP addresses.

[urmet0]

I only can tell, what I think about it, but I have no clue what the best solution is.

Anyway ... There are now two things I could see us doing to fix this for you:

1. You could use dns.blocking.mode = "IP" and configure `dns.reply.host.force4 = "146.112.61.108"' to effectively send any blocked domain to the Umbrella page, or
2. We add a new option to control the detection of upstream blocking so Pi-hole will not intervene and present 0.0.0.0 for you but pass along the original reply from upstream.

I tried to establish suggestion (1.), but it did not really do, what I hoped. Seems, that I get the correct IP now (tested with nslookup), but in the browser I do not get redirected to the umbrella page.

I dont think, that its a good solution to send any blocked domain to the umbrella page, because I am totally fine that pi-hole is blocking requests with 0.0.0.0.

It just would be nice, if OpenDNS Family shield is working similar in v6 as it did in v5.
But as previously said, I cant tell you, what solution is the best for pihole.

[urmet0]

Maybe now I understand, what you mean.
https://github.com/pi-hole/FTL/pull/2030 says that pihole stops asking the upstream, if the upstream once blocked it. This answers are cached.
Currently that cache does not support storing an IP-address, that pihole got as answer from the upstream dns.
I guess, pihole just blocks it the configured "default" way then.

To also store IP addresses (that the upstream gave as answer), this cache needs to get extended by at least 16 bytes to store the IP address (guess: 1 flag + IPv4 address). Then pihole would know, that it got blocked, but also has an IP to forward the request (maybe if wanted so by the user).

Defenitely sounds like a way, I would like.

What comes to my mind here:

1. Whats about IPv6 addresses in upstream answers? To store them, you would need a bigger cache then, I guess.
2. Is the cache-size if extended by that 16 bytes (or even more for IPv6) a problem? Maybe you could provide a setting for that. Turning it on will extend the cache-size for that, so you have bigger cache-sizes, but therefore the ability to get forwarded to "upstream-umbrella pages".
   Of course that would make it more complex just for my little umbrella-page request.

[DL6ER]

Over night, it became more and more apparent that this a plain regression of #2030 and needs to be fixed. No settings no whatsoever. I fixed this with a simple change: preventing FTL to short-circuit the blocked query. Usually, locally blocked queries get a reply with the locally defined blocking mode. However, the "blocked externally with an IP address" is a special case where we exactly don't want this but, instead, want the upstream response go downstream to the client because this is already blocked.

The change is in #2176. It is actually very simple and the larger part of this change goes into our tests which now check for the new behavior.

Bottom line 1: The cache does not need to get touched, we can still cache the response in a second layer of DNS cache we already have for this case.

Bottom line 2: I moved this issue into the correct repository.

[DL6ER]

The fix has been merged into the beta code. Please update and test!

[urmet0]

Thank you! Will do so in the evening.

[urmet0]

I am using the docker-image pihole/pihole:development.
This image was pulled yesterday, so it will not contain the this fix.

Is there a docker-image in place already, containing that fix?

[yubiuser]

pihole/pihole:nightly should work

[rdwebdesign]

I am using the docker-image pihole/pihole:development.

Do not use pihole/pihole:development unless you are testing/developing Pi-hole docker image itself.

If you want the image with the most recent versions of Pi-hole Core, FTL and the Web Interface, please use pihole/pihole:nightly tag. This tag is updated every day, around 5 a.m. UTC.

Is there a docker-image in place already, containing that fix?

I think the current nightly image was generated 16 hours ago and this fix was merged less than one hour later. So you will need to wait for the next nightly image.

Note:

You can also clone the docker-pi-hole repository and create your own local image with the most recent updates running build.sh.

[urmet0]

Built the image myself and tested it.
It looks good now. I get redirected to the umbrella page, like it did in pihole v5.

What I noticed while checking:
I looked into query log while testing, and it seems, that this stops to show anything at some time. Does not matter what I request (random pages found on google), it does not show that. They are all shown in the pihole.log.
Only restarting the DNS resolver one (or several) times helps to get the querylog back running again.

Is this a known issue, or should that work? If it should, maybe I can observe it and raise another ticket, when I have more information.

[rdwebdesign]

... it does not show that. They are all shown in the pihole.log;

I'm not sure if I understood your comment.

Do you see all queries in pihole.log, but not in the Query Log page?

This seems a completely different issue.
Please generate a debug log when it happens again.

[DL6ER]

@urmet0 I will close this issue ticket as the original problem was solved. If you still have any other issue (seems so), please open a new ticket for appropriate tracking. @rdwebdesign already had a question you could answer in a new ticket, too. And as always - some screenshots may be helpful for us to "see" the issue.