Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

recap elixir deployment #61

Merged
merged 8 commits into from
Nov 10, 2022
Merged

recap elixir deployment #61

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
7ff0854
recap elixir deployement
SimonLab Jun 3, 2020
9726d29
Update elixir-phoenix-deployement.md
SimonLab Jun 3, 2020
1dae3fb
Update elixir-phoenix-deployement.md
SimonLab Jun 3, 2020
7c9cf6b
Update elixir-phoenix-deployement.md
SimonLab Jun 3, 2020
1d9f7f5
Update elixir-phoenix-deployement.md
SimonLab Jun 3, 2020
c435b2e
Update elixir-phoenix-deployement.md
SimonLab Jun 3, 2020
9c86ef4
Update elixir-phoenix-deployement.md
SimonLab Jun 3, 2020
a5faf40
Update elixir-phoenix-deployement.md
SimonLab Jun 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 162 additions & 0 deletions elixir-phoenix-deployement.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Deploy an Elixir or Phoenix application

This markdown summarises the research done
on how to deploy a Phoenix application.


## PaaS vs VPS

In the past we have mostly used Platform as a Service (PaaS) providers
to deploy Elixir applications. The main advantage of PaaS
is it is quick to setup and have your application running
for your users.
However in the longer term using a PaaS can become costly (per unit of compute e.g. RAM/CPU).
Virtual Private Server are in contrast normally cheaper to run
but require more setup time and DevOps knowledge to maintain.

### PaaS

#### Heroku

We are used to deploy Elixir/Phoenix on Heroku
and we have already a ["how to"](https://github.com/dwyl/learn-phoenix-framework/blob/master/heroku-deployment.md) guide of this process.

However there are some limitations when using Heroku with Phoenix (see https://hexdocs.pm/phoenix/heroku.html):
![image](https://user-images.githubusercontent.com/6057298/83642784-2e1d1200-a5a7-11ea-8c97-9c7dd920469c.png)

pricing:
![image](https://user-images.githubusercontent.com/6057298/83641712-cc0fdd00-a5a5-11ea-8cbf-49981ae3747e.png)

The pricing above doesn't include the database which needs to be added to the total cost, see https://elements.heroku.com/addons/heroku-postgresql


### Gigalixir

Similar to Heroku [Gigalixir](https://www.gigalixir.com/) provides
a platform which focuses _exclusively_ on deploying Elixir/Phoenix application.
Having an exclusive focus on deploying Elixir Apps gives Gigalixir several key advantages:
1. Zero-downtime blue-green continuous deployment.
see: https://martinfowler.com/bliki/BlueGreenDeployment.html
2. No limit to concurrent connections.
3. All clustering handled transparently.
(so if you need to scale your app beyond 10k concurrent users,
you don't have to pay exponentially more for bigger "dynos"
the way you are forced to scale vertically on Heroku)

Deploying: https://elixircasts.io/deploying-with-gigalixir-%28revised%29
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


Pricing:

![image](https://user-images.githubusercontent.com/6057298/83643629-290c9280-a5a8-11ea-89da-648daec204f3.png)

Gigalixir vs Heroku:

![image](https://user-images.githubusercontent.com/6057298/83644108-d1225b80-a5a8-11ea-869a-c8c0e4a28012.png)

Like Heroku you need to add the cost for using the database:

![image](https://user-images.githubusercontent.com/6057298/83644296-09c23500-a5a9-11ea-9fdd-c50532195c95.png)

see also tiers pricing page: https://gigalixir.readthedocs.io/en/latest/tiers-pricing.html


### Render

[Render](https://render.com/) is another Paas similar to Heroku

pricing:

![image](https://user-images.githubusercontent.com/6057298/83645052-e9df4100-a5a9-11ea-8d3e-f1b18bb4cc02.png)



## Virtual Private Server

VPS allow us to manage ourself the deployement
setup. This allow us to customise the server and
the costs linked to it.


### Linode

#### Ubuntu

Linode provides and support Ubuntu:
![linode-distribution-options-screen](https://user-images.githubusercontent.com/6057298/83647811-3bd59600-a5ad-11ea-893c-b99e3df7f605.png)


From there the idea is install Erlang and Elixir on the server
and then to run the application.

- Install Erlang/Elixir with asdf:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SimonLab as previously noted dwyl/phoenix-liveview-counter-tutorial#17 (comment) please don't introduce a completely new way of managing dependencies without giving some context. This is the first mention of asdf in any file in the dwyl org.

If you want to propose using asdf https://github.com/asdf-vm/asdf as a version manager, and I have nothing against it in principal, it's as simple as opening an issue. see: dwyl/technology-stack#18
Dropping a random tool into a doc without any context is not beginner friendly and sets an undesirable example to others who may think they can add any dependency they like.

- `git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.7.8`
- Edit `~/.bashrc file` and add `. $HOME/.asdf/asdf.sh` and run `source ~/.barhrc` to access the `asdf` command
- Install required pacakges for Erlang `sudo apt install libssl-dev make automake autoconf libncurses5-dev gcc`
- Add erlang plugin to asdf: `asdf plugin-add erlang`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are using Linux as our Host OS then we 100% do not need to install Erlang/Elixir on the VM.
We can use Travis-CI which runs Ubuntu 18.04 ("Bionic") https://docs.travis-ci.com/user/reference/bionic
to build the package using distiliery and deploy it to the VM using edeliver.

And as soon as Ubuntu 20.04 Focal becomes available on Travis-CI we can upgrade! Right now only 18.04 is supported as noted in: internetarchive/openlibrary#3407 (comment)
I fear that I may not have fully explained "Continuous Integration + Continuous Deployment" in the issue ... #59 💭
Perhaps I should have been more explicit that we would be following the "12 Factor App" best practices and separating the "Build Phase" from the "Run Phase" see: https://12factor.net/build-release-run
This strict separation ensures that there is no way to modify the code on the production VM which dramatically enhances security because it almost eliminates the risk of arbitrary code execution (at least from the Erlang/Elixir perspective).

The whole point is that our Continuous Integration server (Travis-CI) does the testing and build phase and then just ships an executable binary of the application to the Linode VM. We definitely don't want to be installing (and thus having to manage/maintain) asdf then erlang and elixir on the VM if we don't absolutely need to.

We already demonstrated how to build and deploy a Phoenix App to a VM using distillery and edeliver:
https://github.com/dwyl/learn-microsoft-azure#deploy-your-phoenix-web-app-using-edeliver
When we shipped Healthlocker to Azure in 2017.
See: https://github.com/healthlocker/healthlocker/blob/master/.deliver/config and: .travis.yml#L33-L35
This worked without any issues and we spent a lot of time documenting all the scripts. e.g .deliver/version.sh

The only reason I did not specify using edeliver in the issue was because I wanted there to be a "fresh eyes" investigation into the best way of doing Phoenix deployment to a VPS in 2020. Installing the Erlang on the VPS and building the project on the production server is definitely not the way we want to do it. The "right" way of doing this is to build the release (binary) on Travis-CI (using someone else's compute time that they have generously provided for free!) and uploading it to S3 so that we have a full history of all our release objects. Then deploy the release to the VPS which does not have Erlang or anything unnecessary installed on it. By minimising the number of programs installed on the VM we minimise maintenance and simplify our lives. The moment we introduce a new tool (e.g. asdf) we need to understand the security+maintenance implications. If the asdf tool is installing any dependencies or running arbitrary shell scripts, who is responsible for ensuring that it's not doing something undesirable (malicious) like opening a TCP port or leaking credentials? Where is asdf sourcing it's binaries for erlang and elixir from? Who has control over them? Is there a security audit? Who is Trevor Brown and why should I trust them? He seems like a "nice guy" https://www.linkedin.com/in/trevor-brown-11136938/ but how do we know he is following security best practices cannot be "spearphished", pwned and infect all the servers running asdf?

P.S. sorry to be a pain about this in the in the PR review, I should have made it clearer in the issue.
12 Factor App. Don't install build dependencies on the Production Server. 👍

- Install Erlang: `asdf install erlang latest`
- Install Elixir: `asdf install elixir latest`
- Define which Elixir version to use `asdf global elxir <version>`

- Instsall Nodejs using nvm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely don't want to install node.js or any of the node build dependencies on the Production Server.
This is a massive attack vector that would allow an attacker who successfully compromised one of the thousands of dependencies in the package-lock.json file to create a postinstall script that leaks our AWS, DB or Encryption keys to a remote server.
My (security mindset) reasoning for wanting to use OpenBSD on the Production server to minimise the number of attack vectors would be completely negated by installing node.js on the VPS.

I tolerate running Node.js on my localhost because I still maintain a bunch of NPM packages,
but I definitely don't want to run Node.js on my Server unless I have to (e.g. because the Application is written in JS...)

- `wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash` (from https://github.com/nvm-sh/nvm#installing-and-updating)
- run `nvm install node`

- Clone and run the Phoenix application
- Clone the application, e.g. `git clone https://github.com/dwyl/hits.git`
- Make sure to have all the environemt variables for the application defined
- e.g. for the secret key: `mix phx.gen.secret` then `export SECRET_KEY_BASE=<secret>`
- Compile assets (see https://hexdocs.pm/phoenix/deployment.html#compiling-your-application-assets)
- `npm run deploy --prefix ./assets`
- `mix phx.digest`
- Start the server with `Mix`: `MIX_ENV=prod mix phx.server`

- Another way to run the server is to use `mix release`: https://hexdocs.pm/phoenix/releases.html

#### FreeBSD/OpenBSD

I've also been investigating how to run
an Elixir/Phoenix applicaiton on FreeBSD (and OpenBSD)

Linode provides a way to create server from image,
however the backup system won't support server running FreeBSD:
![image](https://user-images.githubusercontent.com/6057298/83650091-d931c980-a5af-11ea-8ed1-ffc693d79e41.png)

The following guide explain how to install FreeBSD on Linode:
https://www.linode.com/docs/tools-reference/custom-kernels-distros/install-freebsd-on-linode/

I've also tested the installation on one of my machine: https://github.com/SimonLab/FreeBSD-installation

The idea is then to use the FreeBSD package manager to install Elixir and Erlang:
- `pkg install erlang`
- `pkg install elixir`


### DigitalOcean

DigitalOcean provides a FreeBSD droplet:
![image](https://user-images.githubusercontent.com/6057298/83651322-472ac080-a5b1-11ea-8ce3-764cb9fe7927.png)

see https://www.digitalocean.com/products/linux-distribution/freebsd/

However OpenBSD can't be installed directly.
There are some way we could install it and investigate
if it can be used safely and without any blockers with DigitalOcean:
https://dev.to/nabbisen/custom-openbsd-droplet-on-digitalocean-4a9o

see also: https://www.digitalocean.com/community/tutorials/how-to-get-started-with-freebsd


## Current Conclusion

After reading and testing some Elixir/Phoenix/Linux/BSD installations
I can see that Linode (or similar) can be on a longer term a better tool
to manage the applications.

The simple deployement used above with Ubuntu works well,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, the instructions given above are not a "Deployment" they are a "build and run on the server". That is exactly what we don't want to do and we definitely don't want to advise other people to do it that way.

however I still have some research and testing to do especially
linked to continuous deployment without downtime.
From reading the chapter 11 "Deploy Your Application to Production" of
nelsonic marked this conversation as resolved.
Show resolved Hide resolved
Real Time Phoenix, the solutions to deploy witout downtime are based on running
applications on Elixir clusters. I'd like to learn more about this aspect
but from a MVP perspective this point might take too much time to assimilate
and I think a PaaS might be best to use at the moment.
nelsonic marked this conversation as resolved.
Show resolved Hide resolved