Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker all the things! #2

Draft
wants to merge 29 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
0079e63
Dockerize all the things
xerootg Jan 10, 2023
974505d
Oops, too greedy with the zune.net rewrite
xerootg Jan 10, 2023
8a8befc
got networking up and running
xerootg Jan 11, 2023
9749fdb
get the whole stack working with the zune HD
xerootg Jan 11, 2023
a037658
get the resources into the publish output
xerootg Jan 11, 2023
2803257
lets pretend we are actually zune.net for a little bit
xerootg Jan 11, 2023
9160bb9
a couple more docker settings
xerootg Jan 11, 2023
d22276a
this doesnt work, but maybe its a starting point
xerootg Jan 11, 2023
23aa541
Somehow, this revives a bunch of the commerce endpoints. You'll have …
xerootg Feb 17, 2023
ac96eea
The RP was not needed, I had an nginx config error, [required] is not…
xerootg Feb 18, 2023
3a473d5
significant container build time improvements by building/publishing …
xerootg Feb 18, 2023
e494d33
Partially working metaservices
xerootg Apr 28, 2023
ff27ec1
A bit of reorganization now that MetaServices is running
xerootg Apr 29, 2023
b2c49d5
adding the ability to lookup the silly int64 id later if we need it
xerootg May 1, 2023
3314d8a
make runProd do important things like recreate the containers. Maybe …
xerootg May 1, 2023
05cb0a3
Handle the full Album search flow for FAI
xerootg May 2, 2023
7a56ac5
More WMIS/FAI stuff
xerootg May 4, 2023
0930bfd
Add automatic track linking, introduce amg-like IDs
xerootg May 5, 2023
9045e4c
splitting the artist controller bits used by the HD out into its own …
xerootg May 26, 2023
77e9aea
split out the HD's image controller methods. Probably should use the …
xerootg May 26, 2023
0275a76
logging is important kids. This time, it was cause i didn't have a va…
xerootg May 26, 2023
3063940
differentiate API versions a bit better
xerootg May 26, 2023
b978f58
Consolidating Image controller, fixing a strange crash in zune.exe wh…
xerootg May 30, 2023
06accc0
- Consolidation of HD specific catalog endpoints
xerootg Jun 6, 2023
8482bf3
Apparently the HD also hits the training data endpoints? COOL!
xerootg Jun 6, 2023
c2f7bfc
make things throw less
xerootg Jun 6, 2023
9b53cf8
implement caching and ratelimiting.
xerootg Jun 6, 2023
498d1cc
10m is rookie numbers.
xerootg Jun 6, 2023
4b26d86
do better. throttle musicbrainz requests or be banned.
xerootg Jun 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
##
## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore

*/DevConstants.cs

# User-specific files
*.rsuser
*.suo
Expand Down
2 changes: 1 addition & 1 deletion Zune.DB.Console/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ class Program
static async Task Main(string[] args)
{
// Set up CLI options
string connectionString = "mongodb://localhost:27017";
string connectionString = "mongodb://mongo:27017";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be pulled into IConfiguration so that it can be one of many configuration types including environment variables.

string dbName = "Zune";
var options = new OptionSet
{
Expand Down
17 changes: 17 additions & 0 deletions Zune.Net.Catalog.Image/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
Copy link
Contributor Author

@xerootg xerootg Jan 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally this should be broken into layers of dependency. I need to research the right way to do this for a .net project like this.

RUN dotnet restore

WORKDIR /source/Zune.Net.Catalog.Image
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.Catalog.Image.dll"]
2 changes: 1 addition & 1 deletion Zune.Net.Catalog.Image/appsettings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"ZuneNetContext": {
"ConnectionString": "mongodb://localhost:27017",
"ConnectionString": "mongodb://mongo:27017",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In docker, you can expose things like connection strings to a running container as an environment variables. I'm gonna probably revert this to localhost and define the runtime connection strings in docker-compose.yml

"DatabaseName": "Zune"
},
"Logging": {
Expand Down
17 changes: 17 additions & 0 deletions Zune.Net.Catalog/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.Catalog
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.Catalog.dll"]
2 changes: 1 addition & 1 deletion Zune.Net.Catalog/appsettings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"ZuneNetContext": {
"ConnectionString": "mongodb://localhost:27017",
"ConnectionString": "mongodb://mongo:27017",
"DatabaseName": "Zune"
},
"Logging": {
Expand Down
17 changes: 17 additions & 0 deletions Zune.Net.Commerce/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.Catalog
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.Catalog.dll"]
4 changes: 2 additions & 2 deletions Zune.Net.Commerce/appsettings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"ZuneNetContext": {
"ConnectionString": "mongodb://localhost:27017",
"ConnectionString": "mongodb://mongo:27017",
"DatabaseName": "Zune",
"MemberCollectionName": "Members"
},
Expand All @@ -11,7 +11,7 @@
*/
"AzureAd": {
"Instance": "https://login.microsoftonline.com/",
"Domain": "commerce.zunes.me",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I missed one.

"Domain": "commerce.zune.net",
"TenantId": "common",
"ClientId": "e7dd59f1-8ea4-49b6-8361-1b10b2551dba",

Expand Down
11 changes: 6 additions & 5 deletions Zune.Net.Inbox/Controllers/MessagingInboxController.cs
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,13 @@ public ActionResult<MessageDetails> Details(string locale, string zuneTag, strin
[HttpGet]
public IActionResult UnreadCont(string locale, string zuneTag)
{
using var ctx = new ZuneNetContext();
Member member = ctx.Members.First(m => m.ZuneTag == zuneTag);
if (member == null)
return StatusCode(StatusCodes.Status400BadRequest, $"User {zuneTag} does not exist.");
// using var ctx = new ZuneNetContext();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not compile right now. Do you have a branch where it does, that I should rebase against?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, I just haven't touched messaging in a long time. Don't worry about it

// Member member = ctx.Members.First(m => m.ZuneTag == zuneTag);
// if (member == null)
// return StatusCode(StatusCodes.Status400BadRequest, $"User {zuneTag} does not exist.");

return Content(member.Messages.Count(msg => !msg.IsRead).ToString());
// return Content(member.Messages.Count(msg => !msg.IsRead).ToString());
return Content("1");
}
}
}
17 changes: 17 additions & 0 deletions Zune.Net.Inbox/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.Inbox
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.Inbox.dll"]
2 changes: 1 addition & 1 deletion Zune.Net.Inbox/appsettings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"ZuneNetContext": {
"ConnectionString": "mongodb://localhost:27017",
"ConnectionString": "mongodb://mongo:27017",
"DatabaseName": "Zune",
"MemberCollectionName": "Members"
},
Expand Down
17 changes: 17 additions & 0 deletions Zune.Net.Login/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.Login
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.Login.dll"]
2 changes: 1 addition & 1 deletion Zune.Net.Login/appsettings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"ZuneNetContext": {
"ConnectionString": "mongodb://localhost:27017",
"ConnectionString": "mongodb://mongo:27017",
"DatabaseName": "Zune",
},
"Logging": {
Expand Down
17 changes: 17 additions & 0 deletions Zune.Net.MetaServices/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.MetaServices
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.MetaServices.dll"]
17 changes: 17 additions & 0 deletions Zune.Net.Mix/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.Mix
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.Mix.dll"]
17 changes: 17 additions & 0 deletions Zune.Net.SocialApi/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.SocialApi
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.SocialApi.dll"]
2 changes: 1 addition & 1 deletion Zune.Net.SocialApi/appsettings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"ZuneNetContext": {
"ConnectionString": "mongodb://localhost:27017",
"ConnectionString": "mongodb://mongo:27017",
"DatabaseName": "Zune",
"MemberCollectionName": "Members"
},
Expand Down
17 changes: 17 additions & 0 deletions Zune.Net.Tiles/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.Tiles
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.Tiles.dll"]
17 changes: 17 additions & 0 deletions Zune.Net.Tuners/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# https://hub.docker.com/_/microsoft-dotnet
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /source

# copy csproj and restore as distinct layers
COPY ./ .
RUN dotnet restore

WORKDIR /source/Zune.Net.Tuners
RUN dotnet publish -c release -o /app --no-restore

# final stage/image
FROM mcr.microsoft.com/dotnet/aspnet:7.0
WORKDIR /app
COPY --from=build /app ./
ENV DOTNET_EnableDiagnostics=0
ENTRYPOINT ["dotnet", "Zune.Net.Tuners.dll"]
65 changes: 65 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
version: '3.4'

services:
mongodb:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably need to see if this can start and finish booting before the rest of the containers start

image: mongo:latest
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: rootpassword
ports:
- 27017:27017
volumes:
- mongodb_data_container:/data/db

catalog:
build:
context: ./
dockerfile: Zune.Net.Catalog/Dockerfile

catalog.image:
build:
context: ./
dockerfile: Zune.Net.Catalog.Image/Dockerfile

commerce:
build:
context: ./
dockerfile: Zune.Net.Commerce/Dockerfile

inbox:
build:
context: ./
dockerfile: Zune.Net.Inbox/Dockerfile

login:
build:
context: ./
dockerfile: Zune.Net.Login/Dockerfile

metaservices:
build:
context: ./
dockerfile: Zune.Net.MetaServices/Dockerfile

mix:
build:
context: ./
dockerfile: Zune.Net.Mix/Dockerfile

social:
build:
context: ./
dockerfile: Zune.Net.SocialApi/Dockerfile

tiles:
build:
context: ./
dockerfile: Zune.Net.Tiles/Dockerfile

tuners:
build:
context: ./
dockerfile: Zune.Net.Tuners/Dockerfile

volumes:
mongodb_data_container: