Skip to content

Commit

Permalink
Initial version of the script
Browse files Browse the repository at this point in the history
  • Loading branch information
marcwrobel committed Aug 1, 2022
1 parent 69aa9b8 commit a1fc1ba
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 1 deletion.
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,7 @@
# checklinks
Helps check and fix links in a codebase.

A project codebase often contains a lot of external links. checklinks is an utility that helps checking and fixing those links.

```shell
path/to/checklinks <directory-to-check>
```
29 changes: 29 additions & 0 deletions checklinks
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/usr/bin/env bash

USER_AGENT='Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0'
RETRY=1
RETRY_DELAY=10
TIMEOUT=3 # seconds

RED='\033[0;31m'
GREEN='\033[0;32m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color

[ ! -d "$1" ] && echo "'$1' is not a directory" && exit 1

# Links are processed in a random order to reduce the risk of being blacklisted and temporarily blocked
for url in $(grep -RioEh 'https?://[^][{} "`<>),*$|\\]*[^][{} "`<>),*$|\\.:'"'"']' | grep -vE 'https?://(localhost|[0-9]+|.+:[0-9]+|example|host|somehost|nohost|link|acme.org|foo.com|application.com|[^/]+.example.com|registry.npmjs.org|apache.org/xml/features|java.sun.com/xml/ns|javax.xml.XMLConstants)' | sort | uniq | sort -R); do
# we could use --head, but it is not always supported...
status=$(curl -o /dev/null --silent --connect-timeout "$TIMEOUT" --retry $RETRY --retry-delay $RETRY_DELAY --user-agent "$USER_AGENT" --location --write-out '%{http_code}' "$url")

if [ "$status" = "200" ]; then
if [[ $url =~ "http://" ]]; then
echo -e "${BLUE}$url ($status)${NC}"
else
echo -e "${GREEN}$url ($status)${NC}"
fi
else
echo -e "${RED}$url ($status)${NC}"
fi
done

0 comments on commit a1fc1ba

Please sign in to comment.