Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Docker build #126

Merged
merged 17 commits into from
Nov 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 94 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Python application
name: Python application and Docker image CI

on:
push:
Expand Down Expand Up @@ -65,3 +65,96 @@ jobs:
path: |
dist/mail-parser-*.tar.gz
dist/mail_parser-*.whl

- name: Publish to PyPI
if: matrix.python-version == '3.10' && startsWith(github.ref, 'refs/tags/')
uses: pypa/[email protected]
with:
user: ${{ secrets.PYPI_USERNAME }}
password: ${{ secrets.PYPI_PASSWORD }}

docker:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Log in to Docker Hub
uses: docker/login-action@v3
with:
registry: docker.io
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Extract branch or tag name
id: extract_ref
run: |
if [ -n "${GITHUB_HEAD_REF}" ]; then
REF_NAME=${GITHUB_HEAD_REF}
else
REF_NAME=$(git describe --tags --exact-match 2>/dev/null || git rev-parse --abbrev-ref HEAD)
fi
echo "REF_NAME=${REF_NAME,,}" >> $GITHUB_ENV

- name: Debug REF_NAME
run: echo "REF_NAME=${{ env.REF_NAME }}"

- name: Build and push Docker image on GitHub Container Registry
run: |
cd docker
IMAGE_NAME=ghcr.io/ghcr.io/spamscope/mail-parser/mailparser
if [[ $GITHUB_REF == refs/tags/* ]]; then
TAG=${GITHUB_REF#refs/tags/}
docker build \
--label "org.opencontainers.image.source=${{ github.repositoryUrl }}" \
--label "org.opencontainers.image.description=Easy way to pass from raw mail to Python object" \
--label "org.opencontainers.image.licenses=Apache-2.0" \
--build-arg BRANCH=$TAG \
-t $IMAGE_NAME:$TAG \
-t $IMAGE_NAME:latest .
docker push $IMAGE_NAME:$TAG
docker push $IMAGE_NAME:latest
else
docker build \
--label "org.opencontainers.image.source=${{ github.repositoryUrl }}" \
--label "org.opencontainers.image.description=Easy way to pass from raw mail to Python object" \
--label "org.opencontainers.image.licenses=Apache-2.0" \
--build-arg BRANCH=${{ env.REF_NAME }} \
-t $IMAGE_NAME:develop .
docker push $IMAGE_NAME:develop
fi

- name: Build and push Docker image on Docker Hub
run: |
cd docker
IMAGE_NAME=docker.io/${{ secrets.DOCKER_USERNAME }}/spamscope-mail-parser
if [[ $GITHUB_REF == refs/tags/* ]]; then
TAG=${GITHUB_REF#refs/tags/}
docker build \
--label "org.opencontainers.image.source=${{ github.repositoryUrl }}" \
--label "org.opencontainers.image.description=Easy way to pass from raw mail to Python object" \
--label "org.opencontainers.image.licenses=Apache-2.0" \
--build-arg BRANCH=$TAG \
-t $IMAGE_NAME:$TAG \
-t $IMAGE_NAME:latest .
docker push $IMAGE_NAME:$TAG
docker push $IMAGE_NAME:latest
else
docker build \
--label "org.opencontainers.image.source=${{ github.repositoryUrl }}" \
--label "org.opencontainers.image.description=Easy way to pass from raw mail to Python object" \
--label "org.opencontainers.image.licenses=Apache-2.0" \
--build-arg BRANCH=${{ env.REF_NAME }} \
-t $IMAGE_NAME:develop .
docker push $IMAGE_NAME:develop
fi
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ clean-tests: ## remove test and coverage artifacts

clean-all: clean-tests clean-build ## remove all tests and build files

test: clean-tests ## run tests quickly with the default Python
unittest: clean-tests ## run tests quickly with the default Python
pytest

pre-commit: ## run pre-commit on all files
Expand Down
17 changes: 9 additions & 8 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
FROM python
FROM python:3.10-slim-bullseye
ENV MAIL_PARSER_PATH=/tmp/mailparser
ARG BRANCH=develop
RUN apt-get -yqq update; \
apt-get -yqq --no-install-recommends install libemail-outlook-message-perl; \
apt-get clean; \
rm -rf /var/lib/apt/lists/*; \
git clone -b $BRANCH --single-branch https://github.com/SpamScope/mail-parser.git $MAIL_PARSER_PATH; \
cd $MAIL_PARSER_PATH && python setup.py install
ENTRYPOINT ["mailparser"]
RUN apt-get -yqq update && \
apt-get -yqq --no-install-recommends install libemail-outlook-message-perl git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* && \
git clone -b ${BRANCH} --single-branch https://github.com/SpamScope/mail-parser.git ${MAIL_PARSER_PATH} && \
cd ${MAIL_PARSER_PATH} && \
python setup.py install
ENTRYPOINT ["mail-parser"]
CMD ["-h"]
18 changes: 9 additions & 9 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,33 @@
[![Build Status](https://travis-ci.org/SpamScope/mail-parser.svg?branch=develop)](https://travis-ci.org/SpamScope/mail-parser)
[![](https://images.microbadger.com/badges/image/fmantuano/spamscope-mail-parser.svg)](https://microbadger.com/images/fmantuano/spamscope-mail-parser "Get your own image badge on microbadger.com")
[![PyPI - Version](https://img.shields.io/pypi/v/mail-parser)](https://pypi.org/project/mail-parser/)
[![Coverage Status](https://coveralls.io/repos/github/SpamScope/mail-parser/badge.svg?branch=develop)](https://coveralls.io/github/SpamScope/mail-parser?branch=develop)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/mail-parser?color=blue)](https://pypistats.org/packages/mail-parser)

![SpamScope](https://raw.githubusercontent.com/SpamScope/spamscope/develop/docs/logo/spamscope.png)

# fmantuano/spamscope-mail-parser

This Dockerfile represents a Docker image that encapsulates mail-parser. The [official image](https://hub.docker.com/r/fmantuano/spamscope-mail-parser/) is on Docker Hub.
This Dockerfile represents a Docker image that encapsulates `mail-parser`. The [official image](https://hub.docker.com/r/fmantuano/spamscope-mail-parser/) is on Docker Hub.

To run this image after installing Docker, use a command like this:

```
```shell
sudo docker run -i -t --rm -v ~/mails:/mails fmantuano/spamscope-mail-parser
```

This command runs mail-parser help as default, but you can use all others options.
This command runs `mail-parser` help as default, but you can use all others options.

To share the "mails" directory between your host and the container, create a "mails" directory on your host.

There also is an example of `docker-compose`

From the `docker-compose.yml` directory, run:

```
```shell
$ sudo docker-compose up
```

The provided ```docker-compose.yml``` file is configured to:
The provided `docker-compose.yml` file is configured to:

- Mount your host's `~/mails/` folder from your source tree inside the container at `/mails/` (read-only).
- A command line test example.

See the ```docker-compose.yml``` to view and tweak the launch parameters.
See the `docker-compose.yml` to view and tweak the launch parameters.
Empty file removed report/.gitkeep
Empty file.
147 changes: 118 additions & 29 deletions src/mailparser/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@
"""

import argparse
import os
import runpy
import logging
import sys

import mailparser
Expand All @@ -31,14 +30,18 @@
safe_print,
write_attachments,
)
from mailparser.version import __version__


current = os.path.realpath(os.path.dirname(__file__))

__version__ = runpy.run_path(os.path.join(current, "version.py"))["__version__"]
log = logging.getLogger("mailparser")


def get_args():
"""
Get arguments from command line.
:return: argparse.ArgumentParser
:rtype: argparse.ArgumentParser
"""
parser = argparse.ArgumentParser(
description="Wrapper for email Python Standard Library",
epilog="It takes as input a raw mail and generates a parsed object.",
Expand Down Expand Up @@ -189,22 +192,80 @@ def get_args():


def main():
"""
Main function.
"""
args = get_args().parse_args()
log = custom_log(level=args.log_level)

log = custom_log(level=args.log_level, name="mailparser")

try:
parser = get_parser(args)
process_output(args, parser)
except Exception as e:
log.error(f"An error occurred: {e}")
sys.exit(1)


def get_parser(args):
"""
Get the correct parser based on the input source.
:param args: argparse.Namespace
:type args: argparse.Namespace
:return: MailParser
:rtype: mailparser.core.MailParser
"""
if args.file:
if args.outlook:
log.debug("Analysis Outlook mail")
parser = mailparser.parse_from_file_msg(args.file)
else:
parser = mailparser.parse_from_file(args.file)
return parse_file(args)
elif args.string:
parser = mailparser.parse_from_string(args.string)
log.debug("Start analysis by string mail")
return mailparser.parse_from_string(args.string)
elif args.stdin:
if args.outlook:
raise MailParserOutlookError("You can't use stdin with msg Outlook")
parser = mailparser.parse_from_file_obj(sys.stdin)

return parse_stdin(args)
else:
raise ValueError("No input source provided")


def parse_file(args):
"""
Parse the file based on the arguments provided.
:param args: argparse.Namespace
:type args: argparse.Namespace
:return: MailParser
:rtype: mailparser.core.MailParser
"""
log.debug("Start analysis by file mail")
if args.outlook:
log.debug("Start analysis by Outlook msg")
return mailparser.parse_from_file_msg(args.file)
else:
log.debug("Start analysis by raw mail")
return mailparser.parse_from_file(args.file)


def parse_stdin(args):
"""
Parse the stdin based on the arguments provided.
:param args: argparse.Namespace
:type args: argparse.Namespace
:return: MailParser
:rtype: mailparser.core.MailParser
"""
log.debug("Start analysis by stdin mail")
if args.outlook:
raise MailParserOutlookError("You can't use stdin with msg Outlook")
return mailparser.parse_from_file_obj(sys.stdin)


def process_output(args, parser):
"""
Process the output based on the arguments provided.
:param args: argparse.Namespace
:type args: argparse.Namespace
:param parser: MailParser
:type parser: mailparser.core.MailParser
:param log: logger
:type log: logging.Logger
"""
if args.json:
safe_print(parser.mail_json)

Expand All @@ -230,21 +291,13 @@ def main():
safe_print(parser.received_json)

if args.defects:
log.debug("Printing defects")
for i in parser.defects_categories:
safe_print(i)
print_defects(parser)

if args.senderip:
log.debug("Printing sender IP")
r = parser.get_server_ipaddress(args.senderip)
if r:
safe_print(r)
else:
safe_print("Not Found")
print_sender_ip(parser, args)

if args.attachments or args.attachments_hash:
log.debug("Printing attachments details")
print_attachments(parser.attachments, args.attachments_hash)
print_attachments_details(parser, args)

if args.mail_hash:
log.debug("Printing also mail fingerprints")
Expand All @@ -255,5 +308,41 @@ def main():
write_attachments(parser.attachments, args.attachments_path)


if __name__ == "__main__":
def print_defects(parser):
"""
Print email defects.
:param parser: MailParser
:type parser: mailparser.core.MailParser
"""
log.debug("Printing defects")
for defect in parser.defects_categories:
safe_print(defect)


def print_sender_ip(parser, args):
"""
Print sender IP address.
:param parser: MailParser
:type parser: mailparser.core.MailParser
:param args: argparse.Namespace
:type args: argparse.Namespace
"""
log.debug("Printing sender IP")
sender_ip = parser.get_server_ipaddress(args.senderip)
safe_print(sender_ip if sender_ip else "Not Found")


def print_attachments_details(parser, args):
"""
Print attachments details.
:param parser: MailParser
:type parser: mailparser.core.MailParser
:param args: argparse.Namespace
:type args: argparse.Namespace
"""
log.debug("Printing attachments details")
print_attachments(parser.attachments, args.attachments_hash)


if __name__ == "__main__": # pragma: no cover
main()
Loading