Skip to content

Latest commit

 

History

History
289 lines (193 loc) · 8.13 KB

SETUP.md

File metadata and controls

289 lines (193 loc) · 8.13 KB
Raw

Setting up your development environment

There are a few different options to get started:

1. Manual setup for experienced developers (i.e. using an existing development environment):

Install prerequisites

  • MySQL 8.0+
  • PHP 7.2+ (with curl, gd, intl, json, mbstring, mcrypt, mysql, xml and zip extensions)
  • nginx (or other webserver)
  • Node.js 8 or 9 (and a modern version of npm)
  • elasticsearch 5+
  • redis (not required, but you may want to use for caching and laravel's job-queue)

Clone the git repository

$ git clone https://github.com/ppy/osu-web.git

Configure .env file

# copy the example file and edit the settings, the important ones are APP_* and DB_*
$ cp .env.example .env
$ vi .env

URL rewriting

# for nginx, with root set to the `public` folder of the repo
location / {
    try_files $uri $uri/ /index.php?$query_string;
}

Consult the laravel documentation for non-nginx

Initialize database

# this script assumes you can connect passwordless as root
$ ./bin/db_setup.sh

Install packages and build assets

# will also install composer and yarn
$ ./build.sh

At this point you should be able to access the site via whatever webserver you configured.

2. Using Docker

  • First, install Docker and Docker Compose (on Windows, it's already part of Docker install).
  • Install git.
  • If using Windows, make sure it's running at least build 2004 and install Ubuntu (or another Linux distro of choice) from Windows Store. Additionally:
    • Make sure it's running WSL2 (convert it if it's still using WSL1).
    • Open Docker settings, go to Resources → WSL Integration → Enable integration with additional distros (enable for the installed distro).
  • Open terminal (or Linux console on Windows).
  • Clone this repository.
  • Run bin/docker_dev.sh. Make sure the repository folder is owned by the user executing this command (must be non-root).
  • Due to the nature of Docker (a container is killed when the command running in it finishes), the Yarn container will be run in watch mode.
  • Do note that the supplied Elasticsearch container uses a high (1+ GB) amount of RAM. Ensure that your system (or virtual machine, if running on Windows/macOS) has a necessary amount of memory allocated (at least 2 GB). If you can't (or don't want to), you can comment out the relevant elasticsearch lines in docker-compose.yml.
  • To run any of the below commands, make sure you are using the docker container: docker-compose run --rm php.
    • To run artisan commands, run using docker-compose run --rm php artisan.

Note

On Windows, the files inside Linux system can be found in Explorer from \\wsl$ location.

Default user home directory can be found inside home → <username>.

Due to difference in file permission and line endings, adjustments on git may be needed. Run these in the repository directory:

git config core.eol lf
git config core.filemode false

Docker hints

Services

There are multiple services involved:

  • php: main service for php server. Also serves as entry point for doing other stuff like testing etc
  • assets: builds assets. It sometimes behaves weirdly in which case try restarting it
  • job: runs queued job
  • schedule: runs scheduled job every 5 minutes
  • migrator: prepare database and elasticsearch (service should exit with status 0 after finishing its task)
  • notification-server: main service for notification websocket server
  • notification-server-dusk: notification server to be used by browser test
  • db: database server. Can be skipped by commenting it out and setting a different database instance
  • redis: cache and session server. Can be skipped just like db service
  • elasticsearch: search database. Can be skipped just like db service
  • nginx: proxies php and notification-server(-dusk) so they can be accessed under same host

Modifying environment (.env, .env.dusk.local) files

Sometimes a restart of notification-server and notification-server-dusk will be needed when changing those files.

Example commands

See if anything has stopped:

docker-compose ps

Start docker in background:

bin/docker_dev.sh -d
# alternatively
# docker-compose up -d

Start single docker service:

docker-compose start <servicename>

Restart single docker service:

docker-compose restart <servicename>

Direct database access

Using own mysql client, connect to port 3306 or MYSQL_EXTERNAL_PORT if set when starting up docker.

Alternatively, there's mysql client installed in php service:

docker-compose run --rm php mysql

Development

Creating your initial user

In the repository directory:

$ php artisan tinker
>>> (new App\Libraries\UserRegistration(["username" => "yourusername", "user_email" => "[email protected]", "password" => "yourpassword"]))->save();

Generating assets

Using Laravel's Mix.

# build assets (should be done automatically if using docker)
$ yarn run development

Note that if you use the bundled docker-compose setup, yarn/webpack will be already run in watch mode.

Reset the database + seeding sample data

$ php artisan migrate:fresh --seed

Run the above command to rebuild the database and seed with sample data. In order for the seeder to seed beatmaps, you must enter a valid osu! API key as the value of the OSU_API_KEY property in the .env configuration file, as the seeder obtains beatmap data from the osu! API. The key can be obtained at the "osu! API Access" page, which is currently only available on the old site.

Continuous asset generation while developing

To continuously generate assets as you make changes to files (less, coffeescript) you can run webpack in watch mode.

$ yarn run watch

Email

You can watch emails being sent from osu-web by watching the storage/logs/laravel.log file. Make sure the MAIL_DRIVER is set to log.

Use the API from osu!

To connect from osu!(lazer) via the API offered by osu-web, you need to create a special OAuth password client with:

php artisan passport:client --password

You can then change the constants in the osu! repository (./osu.Game/Online/API/APIAccess.cs).

Testing

To run test, first copy .env.testing.example to .env.testing and .env.dusk.local.example to .env.dusk.local. Make sure to set ES_INDEX_PREFIX and all the databases to something other than production.

Once the env files are set, database for testing will need to be setup:

Initializing the test database

Tests should be run against an empty database, to initialize an empty database:

APP_ENV=testing php artisan migrate:fresh --yes

or if using docker:

docker-compose run --rm -e APP_ENV=testing php artisan migrate:fresh --yes

IMPORTANT

If there are existing matching databases, they will be dropped!

PHP tests

PHP tests use PHPUnit, to run:

bin/phpunit.sh

or if using Docker:

docker-compose run --rm php test phpunit

Regular PHPUnit arguments are accepted, e.g.:

bin/phpunit.sh --filter=Route --stop-on-failure

Browser tests

Browser tests are run using Laravel Dusk:

bin/run_dusk.sh

or if using Docker:

docker-compose run --rm php test browser

Known Issues

The Dusk tests currently do not clean up completely, leaving behind test data in the database; the database should be reintialized after running a Dusk test.

Javascript tests

Javascript tests are run with Karma.

Karma is currently configured to to use Headless Chrome by default; this will require Chrome or a standalone Headless Chrome to be already installed. If you are using Docker, Headless Chrome will already be installed in the container.

yarn karma start --single-run

or if using Docker:

docker-compose run --rm php test js

Documentation

$ php artisan apidoc:generate

Documentation will be generated in the docs folder in both html and markdown formats.