NEOS-1641: Fixes stale DB Connections during Data Sync Workflow

[nickzelei]

This is a pretty large refactor that does a few things:

• Consolidates DB Connection Caching into a single connection manager
• Updates SQL Manager to use ConnectionManager under the hood (part of the connection consolidation)
• Updates the Data Sync Workflow activities to properly release their connection sessions once they are finished with them
• Moves the connection reaper out of the Sync activity and into the startup script
• Centralizes the DB pools for the data sync activity to use a single pool that is shared across all (in-process) activities
• Updates the backend to use the same connection manager logic but to operate in the auto-close mode once a session is released since everything here is transactional

[linear]

NEOS-1641 Investigate Postgres Stale Connections

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
neosync-docs	⬜️ Ignored (Inspect)	Visit Preview		Nov 27, 2024 10:08pm

[codecov]

Codecov Report

Attention: Patch coverage is 46.78899% with 290 lines in your changes missing coverage. Please review.

Project coverage is 35.29%. Comparing base (78afd79) to head (4a3b461).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/connection-manager/manager.go	75.56%	39 Missing and 4 partials ⚠️
cli/internal/cmds/neosync/sync/sync.go	0.00%	34 Missing ⚠️
...1alpha1/connection-data-service/connection-data.go	0.00%	22 Missing ⚠️
...on-manager/pool/providers/sql/sql-pool-provider.go	9.09%	20 Missing ⚠️
backend/pkg/sqlmanager/sql-manager.go	56.81%	19 Missing ⚠️
worker/internal/cmds/worker/serve/serve.go	0.00%	18 Missing ⚠️
backend/internal/cmds/mgmt/serve/connect/cmd.go	0.00%	16 Missing ⚠️
internal/benthos/benthos-builder/builders/sql.go	12.50%	14 Missing ⚠️
internal/connection-manager/session.go	33.33%	13 Missing and 1 partial ⚠️
backend/pkg/sqlmanager/sql-connection.go	0.00%	12 Missing ⚠️
... and 16 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2995      +/-   ##
==========================================
- Coverage   35.58%   35.29%   -0.29%     
==========================================
  Files         337      343       +6     
  Lines       39189    39306     +117     
==========================================
- Hits        13944    13874      -70     
- Misses      23599    23827     +228     
+ Partials     1646     1605      -41     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.