diff --git a/org-cyf/content/guides/deployment-flyio/_index.md b/org-cyf/content/guides/deployment-flyio/_index.md new file mode 100644 index 000000000..72bd0d807 --- /dev/null +++ b/org-cyf/content/guides/deployment-flyio/_index.md @@ -0,0 +1,10 @@ +--- +emoji: 🚀 +title: Deploying to Fly.io +description: Learn how to deploy your website to Fly.io +weight: 8 +--- + +[Fly.io](https://fly.io/) is a provider that allows you to deploy backend applications that are converted into docker containers. It also allows you to start up a small PostgreSQL database on their system. By making sure that the [frontend is served through the backend](../guides/deployment-flyio/serving-frontend) you can easily deploy your entire stack on fly.io. + +The main drawback of fly.io is that its free trial only allows you to deploy exactly two systems. For a full stack application this would be the backend (which is also serving the frontend), and the database, meaning you would only be able to deploy a single project freely. diff --git a/org-cyf/content/guides/deployment-flyio/database-access.md b/org-cyf/content/guides/deployment-flyio/database-access.md new file mode 100644 index 000000000..5ee0fb0c3 --- /dev/null +++ b/org-cyf/content/guides/deployment-flyio/database-access.md @@ -0,0 +1,40 @@ +--- +emoji: 🚀 +title: Accessing fly.io databases +description: Learn how you can access the fly.io PostgreSQL database +weight: 3 +--- + +## 🗝️ Accessing database + +If you have been following the setup guides you would have both a backend and a database system running under fly.io. + +Your database can hold data for multiple applications, so first you need to get a list of them: + +```bash +flyctl postgres db list -a YOURNAME-PROJECTNAME-db +``` + +(Make sure you use your database's name after the `-a` that you have set up before) + +On the list you will find under the NAME column three values: `postgres`, `repmgr` and finally the name of your application's datastore. It will be something like `YOURNAME_PROJECTNAME` - same as your application name but all of the dashes are replaced with underscores. + +Take a note of this name as you will need it later. + +## Uploading database + +To connect to the database you will need to use `flyctl`: + +```bash +flyctl postgres connect -a YOURNAME-PROJECTNAME-db -d YOURNAME_PROJECTNAME +``` + +Where the first value is the name of the database you set up in level 150, and the second value is the datastore name you obtained in the last section. + +The command above will start you up with a proper `psql` console where you can run commands. + +You can also pipe in SQL files. For example, if you have an `initdb.sql` file containing SQL commands to initiate a database you can do: + +```bash +flyctl postgres connect -a YOURNAME-PROJECTNAME-db -d YOURNAME_PROJECTNAME < initdb.sql +``` diff --git a/org-cyf/content/guides/deployment-flyio/serving-frontend.md b/org-cyf/content/guides/deployment-flyio/serving-frontend.md new file mode 100644 index 000000000..516bd497c --- /dev/null +++ b/org-cyf/content/guides/deployment-flyio/serving-frontend.md @@ -0,0 +1,36 @@ +--- +emoji: 🚀 +title: Serving frontend from your backend +description: Learn how to add support for serving frontend files to Express.JS backends +weight: 2 +--- + +Fly.io doesn't have a built-in CDN for serving static frontend files directly, so if you wish to deploy your frontend you need to do it through your backend. +Fortunately, Express.JS has a built-in middleware for serving your frontend through your backend. You only need to set the location of your frontend files, and it will take care of serving the contents for you. +For example, if you add the following middleware inside your `/server/app.js`: + +```js +const staticDir = path.join(__dirname, "..", "static"); +app.use(express.static(staticDir)); +``` + +Then anything under the `/static` directory will be served as-is. + +{{}} +Express.JS will not compile these files for you. If you have Javascript files that need compilation, like React JSX files you need to do that separately. +{{}} + +If you have a React application and you wish it to support React Routes you also need to make sure that every request that doesn't correspond to a real file gets routed to your main website. You can do that by adding the following code: + +```js +app.use((req, res, next) => { + if (req.method === "GET" && !req.url.startsWith("/api")) { + return res.sendFile(path.join(staticDir, "index.html")); + } + next(); +}); +``` + +This will point to any request that was not yet handled in a previous middleware, and does not start with `/api` to your `index.html` allowing React Router to handle it internally. + +You can find a full `app.js` example showing static file serving [here](https://github.com/sztupy/Full-Stack-Project-Assessment/blob/main/server/app.js). diff --git a/org-cyf/content/guides/deployment-flyio/setup/index.md b/org-cyf/content/guides/deployment-flyio/setup/index.md new file mode 100644 index 000000000..2af540008 --- /dev/null +++ b/org-cyf/content/guides/deployment-flyio/setup/index.md @@ -0,0 +1,143 @@ +--- +emoji: 🚀 +title: Setup +description: Learn how to set up fly.io +weight: 1 +--- + +## Install `flyctl` + +Fly.io relies on a command line utility to launch and deploy applications. You need to download and install it. You can find installation instructions here: https://fly.io/docs/hands-on/install-flyctl/ + +After installing you might need to close your terminal and reopen it to be able to access `flyctl` + +## Signing up + +To sign up for a fly.io account go to their sign up page at https://fly.io/app/sign-up Make sure you register using the "Sign up using GitHub", as otherwise you won't get added to the Trial package! + +![Sign up using GitHub](signup.png) + +Once signed up you need to log in locally. Type in the following on your terminal then follow the instructions: + +``` +flyctl auth login +``` + +## Application setup + +Once signed up you can now launch your application. Go to the root of your project and type + +``` +flyctl launch +``` + +Once you enter this command it will provide you with a prompt like the following: + +``` +We're about to launch your NodeJS app on Fly.io. Here's what you're getting: + +Organization: test@codeyourfuture.io (fly launch defaults to the personal org) +Name: full-stack-project-assessment (derived from your directory name) +Region: Amsterdam, Netherlands (this is the fastest region for you) +App Machines: shared-cpu-1x, 1GB RAM (most apps need about 1GB of RAM) +Postgres: (not requested) +Redis: (not requested) + +? Do you want to tweak these settings before proceeding? (y/N) +``` + +Make sure you enter `Y` on the prompt as the default settings are not going to use the free tier! + +Once you enter `Y` and press enter you will be redirected to a website where you need to fill in the details as follows: + +- Name: Use `YOURNAME-PROJECTNAME`, example `john-smith-videorec` +- Region: Pick `lhr - London`. +- VM size: Pick `shared-cpu-1x`. Anything else is not included in the free tier! +- VM memory: Pick `256Mb`. Anything else is not included in the free tier! +- Database: `Fly Postgres` +- DB Name: `YOURNAME-PROJECTNAME-db`, example: `john-smith-videorec-db` +- Configuration: `Development - Single Node`. Anything else here is not included in the free tier! +- Redis: `None` + +Once you fill in the details click "Confirm Settings" + +This will set up your database and a machine for running your backend. If everything is successful you should get something like: + +``` +Now: run 'flyctl deploy' to deploy your Node app. +``` + +Once everything is in order you can see that new files have been added by `fly launch` to your repository. These include a `Dockerfile` and a fly settings file called `fly.toml`. Make sure you commit both into your git repository, they will be needed during further deployments! + +## Application deployment + +Finally you are now ready for deployment: + +``` +flyctl deploy +``` + +This command will send your current repository to fly, build a docker image of your code then deploy that image to the Fly.io infrastructure. + +Note: if you are on the Trial Tier the deployer will return an error message saying it could only deploy your app to one machine instead of two because of the Trial Tier limitations: + +``` +Creating a second machine to increase service availability + +Error: error creating a new machine: failed to launch VM: To create more than 1 machine per app please add a payment method. +``` + +This is okay, as one machine is enough for our deployment. Actually if you are not on the Trial Tier, then the deployer will create two machines. If you don't wish to pay for both then you can decrease them to a single one by using the following command: + +``` +flyctl scale count 1 +``` + +If everything else goes well (as mentioned the error message above can be ignored) your application will be available on + +``` +https://YOURNAME-PROJECTNAME.fly.dev +``` + +Make sure to check that it works as expected! + +## Automated Deployments + +Note that fly.io doesn't have access to your GitHub account so it will not deploy your application whenever it changes. Either you need to run `fly deploy` from your computer every time you want to push a change, or you need to set up GitHub to do this for you. + +In order to do this there are two steps: You need to give GitHub access to your fly.io account, and then also need to set up a workflow that runs the deploy command every time you push changes to the `main` branch. + +For the first one you need to run the following command: + +``` +flyctl tokens create deploy -x 999999h +``` + +This will create a token that can be used by flyctl to run deployments. Make sure you save the result as you will need it later. It looks like a very long string starting with something like `FlyV1 fm2_lJPECAAAAAAAA...`. + +Next go to your GitHub repository on GitHub, and click Settings. On the left-hand side scroll down to "Secrets and variables" and select "Actions". One the page that shows up scroll down to "Repository secrets", and click "New repository secret" + +Set the Name to `FLY_API_TOKEN` and the value to the full results of the previous call. + +Now you have given GitHub access to your fly.io account. You also need to let GitHub know that you want to run a deployment every time your `main` branch changes. First create a file called `.github/workflows/fly-deploy.yml`, with the following content: + +```yaml +name: Fly Deploy +on: + push: + branches: + - main + +jobs: + fly-deploy: + name: Deploy app + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: superfly/flyctl-actions/setup-flyctl@master + - run: flyctl deploy --remote-only + env: + FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} +``` + +Once you commit this file and push it to your `main` branch, GitHub will automatically run `flyctl deploy` against whatever is in the `main` branch. diff --git a/org-cyf/content/guides/deployment-flyio/setup/signup.png b/org-cyf/content/guides/deployment-flyio/setup/signup.png new file mode 100644 index 000000000..d7d65118a Binary files /dev/null and b/org-cyf/content/guides/deployment-flyio/setup/signup.png differ diff --git a/org-cyf/content/guides/deployment/_index.md b/org-cyf/content/guides/deployment/_index.md deleted file mode 100644 index 993919eca..000000000 --- a/org-cyf/content/guides/deployment/_index.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -id: deployment -title: Deploying applications -emoji: 🖥️ ---- - -# What is deployment? - -Deploying in short is making sure the application you are developing is made available for the world wide web to see (or in case for internal systems it is at least visible to the intended audience). Without the ability to deploy the only way to show what you have done is ask others to come to your computer, and show them the work on it in person. - -{{}} -The CYF Curriculum team suggest [Netlify Deployment guide](../deployment-netlify/index.md). as a free provider to use for both your frontend and backend applications, with Supabase as a free database layer for persistence. This is because among the free tier offerings they have the least amount of limitations. - -This suggestion assumes that you are interested in a fully free option, and you can accept the limitations. If you have a deployment budget for your work and/or the limitations are not acceptable you are free to look at other choices as well. -{{}} diff --git a/org-cyf/content/guides/deployment/application_stack.md b/org-cyf/content/guides/deployment/application_stack.md deleted file mode 100644 index fcaaa8d2a..000000000 --- a/org-cyf/content/guides/deployment/application_stack.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -id: application_stack -title: The application stack -weight: 3 -emoji: 🖥️ ---- - -# The application stack - -In the previous section, we talked about the fact that most full-stack web applications have multiple layers (generally at least a frontend and a backend component and some kind of persistence layer in the form of a database). In this section, we go through these layers and talk about how they are usually deployed. - -## Frontend - -In the curriculum we teach, frontend files are considered static. What this means is that once you deploy them, they will never change until the next deployment comes. Your `index.html`, `index.js` files, your CSS files and your image assets will always contain the same content regardless of how many times they are downloaded and shown in browsers around the world. - -This makes them an ideal candidate to serve them through CDNs, Content Delivery Networks. These systems deploy your static files around the globe allowing people to access them quickly. Serving static files is fairly inexpensive, as you only need disk space and bandwidth, so a lot of companies provide this for free for small projects and you only need to pay once your user count becomes large enough. - -CloudFlare Pages and Netlify are some examples of CDN providers that have generous free tiers allowing you to host your static files with them. GitHub Pages, while not strictly considered a CDN, also allows you to host files from your repository simply. - -{{}} -Do note that while the fact that frontend files are static is true for the frameworks we teach, it is not universally true for all frameworks. Web-focused languages like PHP, and MVC frameworks like Ruby on Rails or Django, allow you to create dynamic frontend files. Some React frameworks also support server-side rendering which can make the frontend part of your application dynamic, and ineligible to be served through CDNs. Anything static, like CSS files or images however you can still serve through CDNs, and it is generally still a good idea to do so to improve your application's loading speed. -{{}} - -CDNs are however not the only way to deploy and serve your frontend. You can also add code to your backend server that will enable it to serve your static files. This allows you to use any of the backend providers (that support persistent deployments) to serve your frontend as well. The main benefit of this approach is that your frontend and backend parts remain closely tied together, you can for example easily deploy them at the same time making sure there are no version discrepancies between the two. However you would lose the benefits of CDNs, and although serving static files are not computationally too involved they still need to be handled, which will in turn slow down your backend. - -## Backend - -In contrast to the frontend, your backend systems are almost always considered dynamic. Every request they receive they need to generate a response on the fly based on the request and usually the contents of the database. This means deploying backends are much more involved, and this shows in free offerings as well - there are fewer options to host your backends for free, and the available ones have hefty limitations, forcing you to pay for an upgrade. - -To deploy your backend there are generally two wide categories of deployment mechanisms: persistent and serverless. - -### Persistent deployment - -In a persistent deployment scenario, you run your backend server all the time. This is very similar to how you usually run your backend during development on your local computer. This allows you to fully utilize the features of your backend framework, including features like backend workers. Latency - the time needed for your server to respond - is usually quite low compared to serverless deployments. - -The biggest drawback is that running a server constantly is more expensive, therefore free tiers are in very limited supply. Glitch, Render and Fly.io are some providers that still offer this, but all of them have strict limitations: the first two will for example stop your server if it doesn't receive activity for a longer time. This will make your service very slow whenever it needs to be restarted. - -Apart from the providers above persistent deployment is what people usually use VPS providers for, and the big players usually have a free tier available. AWS's EC2, Azure's Virtual Machines and Google's Compute Platform all have a one-year time-limited trial that allows you to use their slowest computer for free. However because neither of these services supports automated deployments out of the box, and it is very easy to accidentally overshoot the trial package (for example by accidentally starting up a slightly larger computer than the slowest one) we don't recommend any of these options for the projects we do in the curriculum. - -### Serverless deployment - -Serverless, sometimes called on-demand, functional or lambda deployments means that your server is not running constantly. Whenever there is a request the serverless provider will start up your backend, let it serve the request and then stop it immediately afterward. - -Benefits of this approach are that your service doesn't consume resources unless it's used. It is usually aimed for systems that are either not used constantly, or where there are periods of high usage, which needs more resources to handle. Running your backend on-demand only is also less expensive, so there are more free options available to use. - -The drawback of this approach is that your server is getting started and stopped all the time, so it needs to be developed in a way to make this less of a problem. Unfortunately, Express.JS is not designed with serverless operation in mind, but there is a plugin to allow its core functions to operate. However anything that requires your system to run constantly, including global variables or backend workers will not work and need to be replaced with an alternative. The constant starting and stopping is slow; therefore serverless functions have a higher latency than persistent servers. - -Netlify Functions, Cloudflare Pages, Supabase Functions are some examples that have free tiers available, and big players like AWS's Lambda also provide support for this in their time-limited free tier. - -## Database - -The final part of a full stack application is the persistence layer, which is usually a database, in our current curriculum likely a relational database, like Postgres. - -It's not the only kind of data store that you might need. For example, if your application needs to support file uploads and you need to store those files somewhere you would need to opt in for some kind of file storage server as well, this is outside of the scope of this guide however. - -Free Postgres databases are offered by Supabase, Fly.io and Render with some limitations. Other kinds of SQL databases are available from Cloudflare and are present inside the free tiers of AWS, Azure and GCP. These offerings do not follow Postgres' SQL standard, however, so are outside of the scope of this guide. diff --git a/org-cyf/content/guides/deployment/deployment_strategies.md b/org-cyf/content/guides/deployment/deployment_strategies.md deleted file mode 100644 index 90883add1..000000000 --- a/org-cyf/content/guides/deployment/deployment_strategies.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -id: deployment_strategies -title: Deployment strategies -weight: 4 -emoji: 🖥️ ---- - -# Deployment strategies - -In the previous section, we went through where you can deploy your application. In this guide, we will go through how to do that. - -## Manual deployment - -Manual deployments are when you need to take action yourself to get your application deployed. This can be anything from running a couple of commands to needing to press a button on a website. - -For example in the scenario where we imagined that you have both a development computer and a small server in your house, deploying your latest version manually could be following the steps below: - -1. Commit your changes and push them to the `main` branch on the development machine -2. Login onto your server computer -3. Go to the running application's directory -4. Stop the application -5. Pull the latest changes from the `main` branch -6. Run `npm i` -7. Start the application again - -Manual deployments done in the way above can be error-prone. For example, if you forget to do step 7 your application will not be running at all at the end. If your provider does not support built-in automated deployments it is usually advised to create scripts that would do all of the steps above, so any time you wish the latest version to be deployed you would just need to run the script. Scripts can fail so you would still need to make sure to check the logs and see if everything went as expected. Trying to access your website after deployment is also a good way to check that everything is in order. - -## Automated deployment - -Automated deployment or Continuous deployment is whenever your code changes, the deployment flow is initiated automatically. For example, any time your code ends up in the `main` branch of your GitHub repository deployment will commence automatically, and your code will end up on your servers. You can imagine this by someone automatically running the deployment script we've created above whenever your repository changes. - -Some frontend and backend providers have GitHub integration and will support automated deployments out of the box. For others, you either need to deploy manually by following a set of steps or automate these manual steps by creating GitHub workflows manually. - -## Pull Request checks - -While automated deployments might sound scary, for example if you make a mistake in your code automated deployments would deploy that mistake automatically. There are ways to mitigate these risks by adding checks that run after each pull request is created, blocking merging if they fail. Common checks include linting, which would make sure that your code adheres to good coding conventions. Other checks that you can include are running unit, integration and end-to-end tests for each PR making sure that the code you've done passes these requirements. - -## Snapshot builds - -Another feature that helps make automated deployments less risky is snapshot builds. These are a feature of some providers where they not only deploy your `main` branch but also deploy your other branches as well in a temporary fashion. For example, if you create a new pull request from branch `new-awesome-feature` the provider will deploy this branch separately to your production environment. This will allow you, and anyone assessing your pull request to check how your changes would look like before actually merging them. - -Not all providers support snapshot builds. For the ones that do not, a common way to mitigate this is to set up two sets of servers, one called _production_ and the other _staging_ (other common names for this second set are _pre-production_ or _sandbox_). You would then deploy to this environment first, and only after checking that it works as expected continue the deployment to _production_ one. Do note that this usually requires you to set up your environment multiple times, and this will also increase the cost of your setup. diff --git a/org-cyf/content/guides/deployment/from_computers_to_services.md b/org-cyf/content/guides/deployment/from_computers_to_services.md deleted file mode 100644 index 137fc93d4..000000000 --- a/org-cyf/content/guides/deployment/from_computers_to_services.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -id: deployment -title: From your computer to the world -weight: 2 -emoji: 🖥️ ---- - -# From your computer to the world - -While in this curriculum we are mainly going to concentrate on free deployment options to showcase your work you need to understand what "making your work available for the world wide web to see" actually means. In the following section, we will go through a couple of scenarios that are building on each other to show you an idea of why deployments are done in the way they are done. - -## In-person demos - -The easiest, but least convenient way to show your work to others, is to invite them to your computer on which you have your application running. This would assume you know each of your users and trust them enough to give them access to your computer. Also, this assumes your users are happy enough to come to your place physically. - -Unfortunately, neither of these expectations is usually safe to make, so this option is only valid in a limited set of circumstances. Having the ability to demo your work on your computer is useful in classroom settings to show others what you've done and if you have questions. This will remain a useful way during work as well, as pairing with others and demoing them your work is a tried and tested way to showcase what you have done so far to your team. - -## Enabling DMZ on router - -In the previous scenario, the biggest difficulty was that the users had to come to your machine physically in person to look at your website. We can avoid that if we keep your computer connected to your home broadband, and (among other updates that are outside of the scope of this guide) change the firewall settings on your router to allow connections from the outside world to reach your computer (this is usually called the DMZ option). You would then need to keep your machine turned on all the time, and make sure that the work that you have been doing is also running constantly. - -If everything goes well this makes your development computer visible to the world-wide-web and people would be able to see your website. However at any time you need to turn off or restart your machine, or you accidentally stop your running application, maybe to deliver updates to it, they would immediately lose access. Also, they would always see the latest development version you are working on, which might not be something that you want to show the world yet. - -Home broadband systems are also usually not that stable, and if yours is not unlimited you would also need to check the cost of extra bandwidth. Speaking of extra costs, your electricity bill will also increase by keeping your machine running 24/7. And finally, any misconfiguration in your router or computer firewall will enable keen hackers to access your internal network which can usually have serious consequences. - -## Home server - -The next idea is to outsource running your application and have a dedicated computer running your website. You could buy a second machine (this could be an inexpensive one like a Raspberry PI), and set up your router's DMZ settings to point to that instead of your development box. Let's call this second computer a **server** from now on. - -Once you install everything on it as well to mimic your desktop computer this server will then also be able to run your work the same way your development machine can run it. You would need to find a way to copy your work from the development machine to this server (we'll cover some of these in the _manual deployment_ section), but at least you are free to turn off your main computer with the knowledge that your website remains up and visible to your users. - -Because your development machine and server are now running two separate versions of your codebase, you can now also always decide which version you want to run on your server, with the version running on it is usually called **production**. Your database will also be running separately, making it less likely to accidentally update something over there instead of on your development one. - -## Co-location and VPS - -While this is a more valid scenario (and some production systems are running like this), there are still a lot of security and privacy implications of running a server at your own home. The next step would be to use a so-called co-location company, who are specializes in providing a place to store your server machines for a fee. This fee would, among others, cover the electricity, bandwidth and the security of the facility. Once you put your server over there you usually get remote access to your server to allow updates, but generally, they will make sure that the computer is powered on 24/7 and is connected to the internet. - -The next idea is to avoid the big upfront cost of buying a full computer and just rent one. Most hosting companies have an option for you to rent a dedicated server for a monthly price. Depending on budget this still has a hefty price tag, so after a while, you might realize that your project isn't actually that resource intensive and doesn't require a full server. - -That's where VPS (Virtual Private Server) providers come in. They allow you to rent _part_ of a computer running in a virtual machine. There are lots of well-known companies in this sector, like Amazon Web Services, Google Cloud Platform, Microsoft Azure, Digital Ocean, Linode or Ionos just to name a few. You might be able to get access to the slowest computer from these companies completely free on a time-limited trial basis, or they can be as inexpensive as £1-2 a month. - -## As-a-service - -Running your application stack on a VPS is already a good deployment option, but not the only one. In the examples above we have been running the frontend, backend and database part of our application on the same computer. However, these systems actually have different computational needs. Serving static frontend files for example needs bandwidth and disk space but doesn't really require a fast computer or much memory. Databases on the other hand do need a lot of memory to operate efficiently. - -Therefore if you split out your work into separate layers you would be able to deploy each layer on a system that is designed for the needs of that particular layer. For example, the database part can run on a dedicated DaaS (Database-as-a-service) instance, while the frontend can be served on fast CDNs (Content Delivery Networks). There are again plenty of companies providing these services, and fortunately, some have free tiers available to trial them out. Some of these free tiers are generous enough that they can be used to deploy and host simple applications, and we will talk about them in more detail later.