Store games and metadata in SQLite database

[treymerkley]

This was mentioned on the old branch, but I want to mention it here for posterity and to hopefully get some help on it.

As it currently stands, ES scans storage every time it loads, causing massive loading times. Aloshi had the idea of using what is now standard for other media libraries, scanning only on request of the user and storing the results in an SQLite database, so loading times and memory usage are reduced.

I know a little bit about database design, but not enough to rework this. What needs to be done to get this implemented?

[Enverex]

This would catastrophically break my setup and I assume the setup of anyone that generates their gamelists outside of EmulationStation itself (mine's generated by the scraper that I use). If we do go this route, it needs to be extensively documented so that people can populate the databases outside of the ES application itself.

ES should only scan-on-demand when launched with the "--gamelist-only" option which only uses the existing gamelists and doesn't scan for new files, but I have a feeling that this switch is only half working, because if I don't mount my storage disk, the actual systems don't show any content despite the gamelists already being populated with content.

So what needs to be fixed here first is the "--gamelist-only" option being strictly obeyed, in that it shouldn't look for any content when this switch is used, it should simply read the existing gamelists and assume they're correct (this would then also further reduce the startup time, more than the option already does).

Or, as you suggested, have ES not scan for content at all at startup unless requested from the in-program menu. Negating the need for the "--gamelist-only" option at all and fixing the secondary issue that I raised.

EDIT: Looks like I actually already raised these issues on the original repo and forks years ago: Aloshi#191 and Aloshi#401 and Herdinger#39 (quite the saga...)

[treymerkley]

That looks right, but after digging through the code, I'm having a hard time figuring out what could be causing it: the default setting is not to scan, nothing seems to override that, so my guess is that I'm overlooking it.