Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to MongoDB #1767

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 6 additions & 11 deletions .github/workflows/deps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,15 @@ jobs:
permissions:
contents: write
services:
postgres:
image: postgres
mongodb:
image: mongo
env:
POSTGRES_PASSWORD: keepitsecret
POSTGRES_USER: testdb
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
MONGO_INITDB_ROOT_PASSWORD: keepitsecret
MONGO_INITDB_ROOT_USERNAME: testdb
ports:
- 5432:5432
- 27017:27017
env:
DATABASE_URL: postgres://testdb:keepitsecret@localhost:5432/testdb
MONGODB_URI: mongodb://testdb:keepitsecret@localhost:27017/testdb?authSource=admin
steps:
- uses: textbook/take-action@nodejs
with:
Expand Down
36 changes: 13 additions & 23 deletions .github/workflows/push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,20 +20,15 @@ jobs:
node: [20.9, 20, 22]
runs-on: ubuntu-latest
services:
postgres:
image: postgres
mongodb:
image: mongo
env:
POSTGRES_PASSWORD: keepitsecret
POSTGRES_USER: testdb
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
MONGO_INITDB_ROOT_PASSWORD: keepitsecret
MONGO_INITDB_ROOT_USERNAME: testdb
ports:
- 5432:5432
- 27017:27017
env:
DATABASE_URL: postgres://testdb:keepitsecret@localhost:5432/testdb
MONGODB_URI: mongodb://testdb:keepitsecret@localhost:27017/testdb?authSource=admin
steps:
- uses: textbook/take-action@nodejs
with:
Expand All @@ -49,18 +44,13 @@ jobs:
docker:
runs-on: ubuntu-latest
services:
postgres:
image: postgres
mongodb:
image: mongo
env:
POSTGRES_PASSWORD: keepitsecret
POSTGRES_USER: testdb
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
MONGO_INITDB_ROOT_PASSWORD: keepitsecret
MONGO_INITDB_ROOT_USERNAME: testdb
ports:
- 5432:5432
- 27017:27017
steps:
- uses: textbook/take-action@nodejs
with:
Expand All @@ -74,10 +64,10 @@ jobs:
tags: textbook/starter-kit:v2
- id: env-file
run: |
echo 'DATABASE_URL=postgres://testdb:keepitsecret@localhost:5432/testdb' >> "$ENV_FILE"
echo 'LOG_LEVEL=debug' >> "$ENV_FILE"
echo 'PORT=4321' >> "$ENV_FILE"
echo 'MONGODB_URI=mongodb://testdb:keepitsecret@localhost:27017/testdb?authSource=admin' >> "$ENV_FILE"
echo 'NODE_ENV=docker' >> "$ENV_FILE"
echo 'PORT=4321' >> "$ENV_FILE"
echo "file=$ENV_FILE" >> "$GITHUB_OUTPUT"
env:
ENV_FILE: docker.env
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

- [x] [Node] LTS support (verified working on 20.x LTS release)
- [x] [Express] server
- [x] [Postgres] database with [`pg`][node-postgres]
- [x] [MongoDB] database with [Mongoose]
- [x] Logging with [Winston] and [Morgan]
- [x] [React] client with [Vite]
- [x] Client-side routing with [React Router]
Expand Down Expand Up @@ -36,12 +36,12 @@ commonly use:
[Google App Engine]: https://cloud.google.com/appengine/?hl=en
[Heroku]: https://www.heroku.com/
[Jest]: https://jestjs.io/
[MongoDB]: https://www.mongodb.com/
[Mongoose]: https://mongoosejs.com/
[Morgan]: https://github.com/expressjs/morgan
[Node]: https://nodejs.org/en/
[node-postgres]: https://node-postgres.com/
[node-test]: https://nodejs.org/api/test.html
[Playwright]: https://playwright.dev/
[Postgres]: https://www.postgresql.org/
[Prettier]: https://prettier.io/
[React]: https://reactjs.org/
[React Router]: https://reactrouter.com/en/main
Expand Down
2 changes: 1 addition & 1 deletion api/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import express from "express";

import apiRouter from "./api.js";
import { testConnection } from "./db.js";
import config from "./utils/config.cjs";
import config from "./utils/config.js";
import {
asyncHandler,
clientRouter,
Expand Down
39 changes: 18 additions & 21 deletions api/db.js
Original file line number Diff line number Diff line change
@@ -1,34 +1,31 @@
import pg from "pg";
import mongoose from "mongoose";

import logger from "./utils/logger.js";

/** @type {pg.Pool} */
let pool;
mongoose.connection.on("error", (err) => logger.error("%O", err));
mongoose.set("debug", (collection, method, query, doc) => {
logger.debug("%O", { collection, method, query, doc });
});

/**
* @param {import("pg").ClientConfig} config
* @param {string} uri
* @param {import("mongoose").ConnectOptions} [options]
*/
export const connectDb = async (config) => {
pool = new pg.Pool(config);
pool.on("error", (err) => logger.error("%O", err));
const client = await pool.connect();
logger.info("connected to %s", client.database);
client.release();
export const connectDb = async (uri, options) => {
const client = await mongoose.connect(uri, {
bufferCommands: false,
...options,
});
logger.info("connected to %s", client.connection.name);
};

export const disconnectDb = async () => {
if (pool) {
await pool.end();
}
await mongoose.disconnect();
};

export const testConnection = async () => {
await query("SELECT 1;");
const state = mongoose.connection.readyState;
if (state !== mongoose.ConnectionStates.connected) {
throw new Error(`database connection state: ${mongoose.STATES[state]}`);
}
};

function query(...args) {
logger.debug("Postgres query: %O", args);
return pool.query.apply(pool, args);
}

export default { query };
8 changes: 6 additions & 2 deletions api/messages/messageRepository.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
import db from "../db.js";
import mongoose from "mongoose";

const MessageSchema = new mongoose.Schema({ content: String });

const Message = mongoose.model("messages", MessageSchema);

export async function getAll() {
const { rows } = await db.query("SELECT * FROM message;");
const rows = await Message.find();
return rows;
}
16 changes: 16 additions & 0 deletions api/migrate-mongo-config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";

import config from "./utils/config.js";

const __dirname = dirname(fileURLToPath(import.meta.url));

/** @type {import("migrate-mongo").config.Config} */
export default {
changelogCollectionName: "changelog",
migrationFileExtension: ".js",
migrationsDir: join(__dirname, "migrations"),
moduleSystem: "esm",
mongodb: { url: config.dbUrl },
useFileHash: false,
};
24 changes: 0 additions & 24 deletions api/migrations/1707922794590_welcome-message.cjs

This file was deleted.

21 changes: 21 additions & 0 deletions api/migrations/20240916162136-welcome-message.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/**
* Template for
* {@link https://github.com/seppevs/migrate-mongo?tab=readme-ov-file#creating-a-new-migration-script defining migrations}.
*/

/**
* @param {import("mongodb").Db} db
* @param {import("mongodb").MongoClient} client
*/
export const up = async (db) => {
await db.collection("messages").insertOne({ content: "Hello, world!" });
};

/**
*
* @param {import("mongodb").Db} db
* @param {import("mongodb").MongoClient} client
*/
export const down = async (db) => {
await db.collection("messages").drop();
};
22 changes: 0 additions & 22 deletions api/migrations/config.cjs

This file was deleted.

20 changes: 20 additions & 0 deletions api/migrations/sample-migration.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/**
* Template for
* {@link https://github.com/seppevs/migrate-mongo?tab=readme-ov-file#creating-a-new-migration-script defining migrations}.
*/

/**
* @param {import("mongodb").Db} db
* @param {import("mongodb").MongoClient} client
*/
export const up = async (db, client) => {
// TODO write your migration here.
};

/**
* @param {import("mongodb").Db} db
* @param {import("mongodb").MongoClient} client
*/
export const down = async (db, client) => {
// TODO reverse your migration here.
};
17 changes: 0 additions & 17 deletions api/migrations/template.cjs

This file was deleted.

10 changes: 5 additions & 5 deletions api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,21 @@
"type": "module",
"scripts": {
"dev": "cross-env LOG_LEVEL=debug node --inspect --watch --watch-path . server.js",
"migration": "node-pg-migrate --config-file ./migrations/config.cjs",
"test": "cross-env DATABASE_URL=http://example.com NODE_OPTIONS='--experimental-vm-modules' jest",
"migration": "migrate-mongo",
"test": "cross-env MONGODB_URI=http://example.com NODE_OPTIONS='--experimental-vm-modules' jest",
"test:watch": "npm run test -- --watch"
},
"dependencies": {
"dotenv": "^16.4.5",
"express": "^4.21.0",
"helmet": "^7.1.0",
"migrate-mongo": "^11.0.0",
"mongoose": "^8.6.2",
"morgan": "^1.10.0",
"node-pg-migrate": "^7.6.1",
"pg": "^8.12.0",
"winston": "^3.14.2"
},
"devDependencies": {
"@testcontainers/postgresql": "^10.13.1",
"@testcontainers/mongodb": "^10.13.1",
"eslint-plugin-jest": "^28.8.3",
"eslint-plugin-n": "^17.10.2",
"jest": "^29.7.0",
Expand Down
4 changes: 2 additions & 2 deletions api/server.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import app from "./app.js";
import { connectDb } from "./db.js";
import config from "./utils/config.cjs";
import config from "./utils/config.js";
import logger from "./utils/logger.js";

const { port } = config;

await connectDb(config.dbConfig);
await connectDb(config.dbUrl);

app.listen(port, () => logger.info(`listening on ${port}`));
Loading