Reppy cache fetch error on

[mkozmelj]

Reppy return an error when trying to get robots.txt file that is not there. I try to put code in try/except block but error is still there.

[dlecocq]

@mkozmelj - can you include the code snippet you were using and the robots.txt you were trying to fetch?

If the robots.txt request ultimately returns a 404, it /should/ behave as an AllowAll (https://github.com/seomoz/reppy/blob/master/tests/test_robots.py#L196)

[PLPeeters]

I believe this behaviour is more or less fixed by #119.