Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LdSvmTrainer slow behaviour when loading from a remote database #7380

Open
superichmann opened this issue Feb 4, 2025 · 0 comments
Open
Labels
untriaged New issue has not been triaged

Comments

@superichmann
Copy link

System Information (please complete the following information):

  • OS & Version: Windows 10
  • ML.NET Version: 4.0.0
  • .NET Version: 9

Describe the bug
When using DatabaseLoader from a remote database, the loading time and overall running time is much longer then other trainers.
As well the memory does not fills up quickly in comparison to different trainers

To Reproduce
Run LdSvmTrainer on a large dataset with hundreds of columns and millions of rows, should leave memory footprint. but it just consumes more and more data from the network sql and not fills up the memory, also takes about 100 times longer then lgbm / ff.

Run a different trainer (lgbm/ff) on the same dataset and see the memory footprint.

Expected behavior
LdSvmTrainer shoud load the entire data onto memory, similar to other ml.net trainers and then work on them.

@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged label Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
untriaged New issue has not been triaged
Projects
None yet
Development

No branches or pull requests

1 participant