Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cleanup, local development fixes and start best practices #453

Merged
merged 4 commits into from
Nov 11, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .dockerignore
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ignoring node_modules did nothing for the build nor the local env

This file was deleted.

13 changes: 4 additions & 9 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

PORT=3000
HOST=http://localhost:3000
API_URL=https://api.test.faforever.com
Expand All @@ -10,11 +9,7 @@ OAUTH_CLIENT_ID=faf-website
OAUTH_CLIENT_SECRET=banana
SESSION_SECRET_KEY=banana
RECAPTCHA_SITE_KEY=6Lc4GmgaAAAAAPaFbnwuP_dBGG--SSy2Hi5UcqwK


TOKEN_LIFESPAN=43200 # an api token is valid for 12h
CLAN_INVITES_LIFESPAN_DAYS = 30 # 30 days
EXTRACTOR_INTERVAL= 5 # 5 minutes
PLAYER_COUNT_INTERVAL= 15 # 15 seconds


TOKEN_LIFESPAN=43200
CLAN_INVITES_LIFESPAN_DAYS=30
EXTRACTOR_INTERVAL=5
PLAYER_COUNT_INTERVAL=15
fcaps marked this conversation as resolved.
Show resolved Hide resolved
33 changes: 33 additions & 0 deletions .env.faf-stack
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# node env is set statically to "development" in the dockerfile, you could override it here.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the thinking with having both a .env.template and .env.fafstack? It seems a little redundant

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

env.example is for local development (without docker and a faf-stack)

#NODE_ENV=production

# you should not need to change the port (used by express) inside the container, just here for completion
#PORT=3000

# configs to change the ports in docker compose for the container
# only needed if your host already has 8020 binded to another service
# beware, changing the host port (8020) also needs an update in the hydra client for valid callback-urls
#WEBSITE_EXPOSED_PORT=8020
#WEBSITE_CONTAINER_PORT=3000

HOST=http://localhost:8020
API_URL=http://faf-java-api:8010
OAUTH_URL=http://faf-ory-hydra:4444

# on a local environment with docker, the internal docker-service-domain (faf-ory-hydra:4444) is not reachable for a browser
# you can omit this env and it will fallback to OAUTH_URL if you know what you are doing.
OAUTH_PUBLIC_URL=http://localhost:4444

# unsing the "production" wordpress because the faf-local-stack is just an empty instance without any news etc.
WP_URL=https://direct.faforever.com

LOBBY_API_URL=http://faf-python-server:4000

OAUTH_CLIENT_ID=faf-website
OAUTH_CLIENT_SECRET=banana
SESSION_SECRET_KEY=banana
RECAPTCHA_SITE_KEY=6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI
TOKEN_LIFESPAN=43200
CLAN_INVITES_LIFESPAN_DAYS=30
EXTRACTOR_INTERVAL=5
PLAYER_COUNT_INTERVAL=15
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should also update the faf-stack to include some of the fixes from here.

15 changes: 0 additions & 15 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,17 +1,2 @@
# Auto detect text files and perform LF normalization
* text=auto

# Custom for Visual Studio
*.cs diff=csharp

# Standard to msysgit
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
20 changes: 20 additions & 0 deletions .github/workflows/code-quality.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Website static checks

on: [pull_request]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
checks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
with:
node-version: '20'
cache: 'yarn'
- run: yarn install
# --force should be removed if all the issues are fixed
- run: ./node_modules/.bin/grunt jshint --force
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,6 @@ public/styles/css/*
.env

#Ignore link.json
link.json
link.json

public/js/*.js
16 changes: 16 additions & 0 deletions .npmcheckrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
depcheck:
ignoreMatches: [
"npm-check", # used manually to check for updates
"pug", # used in express as the template engine
"awesomplete", # used statically in grunt/concat.js

# grunt plugins used in "./grunt"
"grunt-sass",
"grunt-postcss",
"grunt-nodemon",
"grunt-contrib-watch",
"grunt-contrib-uglify-es",
"grunt-contrib-jshint",
"grunt-contrib-concat",
"grunt-concurrent"
]
29 changes: 12 additions & 17 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,27 +1,22 @@
# Use an ubuntu-image for building assets for use in a runtime image...
FROM node:lts as builder

RUN mkdir code

# Add files to /code folder
ADD . /code
FROM node:20.9-bookworm as builder
RUN apt-get update && apt-get install -y --no-install-recommends dumb-init
ENV NODE_ENV development

COPY . /code
WORKDIR /code

RUN yarn install

RUN yarn install --production=false --frozen-lockfile
RUN ./node_modules/.bin/grunt prod
RUN yarn install --production=true --ignore-optional --frozen-lockfile

# Slimmer runtime image without python/make/gcc etc.
FROM node:lts-alpine as runtime
FROM node:20.9.0-bookworm-slim as runtime
ENV NODE_ENV production

COPY --from=builder /code /code
COPY --from=builder /usr/bin/dumb-init /usr/bin/dumb-init
COPY --from=builder --chown=node:node /code /code

WORKDIR /code
USER node

# Only install runtime dependencies for the runtime image
RUN yarn --prod

CMD PORT=3000 npm start
CMD ["dumb-init", "node", "express.js"]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


EXPOSE 3000
6 changes: 6 additions & 0 deletions Dockerfile-dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
FROM node:20.9-bookworm
RUN apt-get update && apt-get install -y --no-install-recommends dumb-init
ENV NODE_ENV development
WORKDIR /code

CMD ["dumb-init", "./node_modules/.bin/grunt", "concurrent"]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multistage was actually not beneficial, so i decided to create a new dockerfile for local-dev

28 changes: 0 additions & 28 deletions Gruntfile.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,6 @@ module.exports = function(grunt) {
// Load grunt tasks automatically
require('load-grunt-tasks')(grunt);

// Time how long tasks take. Can help when optimizing build times
require('time-grunt')(grunt);

var options = {
config: {
src: './grunt/*.js'
Expand All @@ -23,34 +20,9 @@ module.exports = function(grunt) {
// Project configuration.
grunt.initConfig(configs);

// load jshint
grunt.registerTask('lint', [
'jshint'
]);

// default option to connect server
grunt.registerTask('serve', [
'jshint',
'concurrent:dev'
]);

grunt.registerTask('prod', [
'sass:dist',
'concat:js',
'uglify:dist'
]);

grunt.registerTask('server', function () {
grunt.log.warn('The `server` task has been deprecated. Use `grunt serve` to start a server.');
grunt.task.run(['serve:' + target]);
});

if(process.env.NODE_ENV === "development") {
grunt.loadNpmTasks('grunt-contrib-compass');
grunt.loadNpmTasks('grunt-contrib-uglify-es');
grunt.loadNpmTasks('grunt-sass');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-jshint');
grunt.loadNpmTasks('grunt-postcss');
}
};
1 change: 0 additions & 1 deletion Procfile
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are not using heroku somehow, right?

This file was deleted.

36 changes: 29 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,35 @@ The Main Focus for what the primary purpose of the website is. The following was
1. To Focus on Acquiring and On-boarding of new Player into FAForever (Registration of players, Documentation & Support)
2. Promote The Community (Clans, Maps, Mods, Tournaments etc.)

## Developing the Website / Setting up your local enviroment

If you want to setup your own local enviroment, there's a guide below.

https://github.com/FAForever/website/wiki/Setting-up-a-local-environment-for-the-website

For any PR/Pull Request, please make sure you detail in the comments on files/changes. Otherwise it might take longer for your change to be approved.
## Developing the Website / Setting up your local environment

### Option #1 - locally install node, yarn etc
Local installation without docker [guide](https://github.com/FAForever/website/wiki/Setting-up-a-local-environment-for-the-website)

### Option #2 - run everything in docker (linux, wsl2, mac)
Local requirements:
- docker
- docker compose

The website has dependencies to Hydra, [Lobby](https://github.com/FAForever/server), Wordpress and the [Java-API](https://github.com/FAForever/faf-java-api).
You can run those with the [local-stack](https://github.com/FAForever/faf-stack).

If you got the [local-stack](https://github.com/FAForever/faf-stack) up and running, we need to stop the "faf-website" and replace it with a development container.
````bash
cd ../faf-stack # replace path if needed
docker compose stop faf-website
````

Development-Container:
`````bash
cd ../website # replace path if needed
cp -n .env.faf-stack .env
docker compose build
docker compose run website yarn install
docker compose up
`````

this should start the express-server on http://localhost:8020/.

## Other Ways to Contribute

Expand Down
19 changes: 19 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: "3.5"

services:
website:
env_file: .env
ports:
- ${WEBSITE_EXPOSED_PORT-8020}:${WEBSITE_CONTAINER_PORT-3000}
volumes:
- .:/code
build:
dockerfile: Dockerfile-dev
context: .
networks:
- faf-stack

networks:
faf-stack:
name: faf-stack_faf
fcaps marked this conversation as resolved.
Show resolved Hide resolved
external: true
5 changes: 3 additions & 2 deletions express.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ app.locals.clanInvitations = {};
require('dotenv').config();
//Define environment variables with default values
process.env.NODE_ENV = process.env.NODE_ENV || 'production';
process.env.PORT = process.env.PORT || '4000';
process.env.PORT = process.env.PORT || '3000';
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in every .env file i could found we setting the PORT env, just because we have this fallback here.

process.env.CALLBACK = process.env.CALLBACK || 'callback';
process.env.OAUTH_URL = process.env.OAUTH_URL || 'https://hydra.faforever.com';
process.env.OAUTH_PUBLIC_URL = process.env.OAUTH_PUBLIC_URL || process.env.OAUTH_URL;
process.env.API_URL = process.env.API_URL || 'https://api.faforever.com';
process.env.OAUTH_CLIENT_ID = process.env.OAUTH_CLIENT_ID || '12345';
process.env.OAUTH_CLIENT_SECRET = process.env.OAUTH_CLIENT_SECRET || '12345';
Expand Down Expand Up @@ -227,7 +228,7 @@ app.get('/login', passport.authenticate('faforever'));
passport.use('faforever', new OidcStrategy({
issuer: process.env.OAUTH_URL + '/',
tokenURL: process.env.OAUTH_URL + '/oauth2/token',
authorizationURL: process.env.OAUTH_URL + '/oauth2/auth',
authorizationURL: process.env.OAUTH_PUBLIC_URL + '/oauth2/auth',
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a browser can not resolve the docker name aka "http://faf-ory-hydra:4444", so this is needed for docker environments, this needs also be fixed in the faf-stack

userInfoURL: process.env.OAUTH_URL + '/userinfo?schema=openid',
clientID: process.env.OAUTH_CLIENT_ID,
clientSecret: process.env.OAUTH_CLIENT_SECRET,
Expand Down
22 changes: 1 addition & 21 deletions grunt/concat.js
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

none of the removed files where used, sometimes they where just empty after #410

Original file line number Diff line number Diff line change
@@ -1,37 +1,17 @@
module.exports = {
js: {
nonull: true,
files: {
'public/js/bottom.min.js': [
'public/js/app/headroom.min.js',
'public/js/app/navigation.js'
],
'public/js/leaderboards.min.js': [
'node_modules/awesomplete/awesomplete.js',
'node_modules/moment/min/moment-with-locales.min.js',
'public/js/app/leaderboards.js'
],
'public/js/leagues.min.js': [
'node_modules/awesomplete/awesomplete.js',
'node_modules/moment/min/moment-with-locales.min.js',
'public/js/app/leagues.js'
],
'public/js/account.min.js': [
'node_modules/bootstrap-validator/dist/validator.js'
],
'public/js/blog.min.js': [
'public/js/app/blog.js'
],
'public/js/newshub.min.js': [
'public/js/app/newshub.js'
],
'public/js/report.min.js': [
'node_modules/awesomplete/awesomplete.js',
'public/js/app/report.js'
],
'public/js/browse_clans.min.js': [
'node_modules/awesomplete/awesomplete.js',
'public/js/app/browse_clans.js'
]
}
}
};
2 changes: 1 addition & 1 deletion grunt/concurrent.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module.exports = {
dev: {
tasks: ['nodemon:debug', 'concat:js', 'uglify:dev', 'watch'],
tasks: ['nodemon', 'concat', 'watch'],
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uglify is not needed on dev, and calling a taks without :, just executes all tasks

options: {
logConcurrentOutput: true
}
Expand Down
31 changes: 22 additions & 9 deletions grunt/jshint.js
Original file line number Diff line number Diff line change
@@ -1,13 +1,26 @@
module.exports = {
options: {
reporter: require('jshint-stylish'),
force: true
},
all: [
'routes/**/*.js',
'models/**/*.js'
],
server: [
'./express.js'
]
};
browser_files: {
options: {
esversion: 8,
asi: true
},
src: [
'public/js/app/**/*js',
]
},
node_files: {
options: {
node: true,
esversion: 11,
asi: true
},
src: [
'./express.js',
'scripts/**/*js',
'routes/**/*js',
]
}
};
7 changes: 0 additions & 7 deletions grunt/node-inspector.js

This file was deleted.

Loading