Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DB Backend PR #12

Open
wants to merge 80 commits into
base: db-backend
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
bc2dfc9
Added SQLite3 to the project.
Aloshi Nov 26, 2014
b20b3eb
Initial work for the SQLite database replacement.
Aloshi Nov 28, 2014
02e91a3
Merge branch 'unstable' into gamelistdb
Aloshi Dec 6, 2014
6ae49e9
Refactored SystemData into SystemData + SystemManager.
Aloshi Dec 22, 2014
1449ab1
Redesigned GamelistDB to store all data in a single table.
Aloshi Dec 27, 2014
0abae19
GamelistDB now handles metadata format changes automatically (e.g. wh…
Aloshi Dec 27, 2014
c0bc2ea
GamelistDB starting to be integrated into SystemManager.
Aloshi Dec 27, 2014
2013215
Added Boost.Regex dependency.
Aloshi Dec 28, 2014
60a52e6
THE GREAT FILEDATA REWRITE OF CHRISTMAS 2014
Aloshi Dec 29, 2014
e501450
Sort support.
Aloshi Jan 8, 2015
35370a4
Fixed FileType/MetaDataListType confusions.
Aloshi Jan 8, 2015
68348ed
Added "IMPORT XML" option to the "SCRAPER" menu.
Aloshi Jan 8, 2015
a99b2e2
Fixed another metadatalisttype/filetype test.
Aloshi Jan 8, 2015
3e50413
Fixed updateExists(system) not actually working.
Aloshi Jan 8, 2015
52ed0ea
Specify FileType in FileData constructor.
Aloshi Jan 9, 2015
bb303c1
Merge branch 'unstable' into gamelistdb
Aloshi Jan 11, 2015
7150e8d
Merge branch 'master' into gamelistdb
Aloshi Jan 15, 2015
4c4fb5e
Fix build errors on Linux.
Aloshi Jan 17, 2015
c0e0f4d
Merge branch 'master' into gamelistdb
Aloshi Feb 10, 2015
44fd9d6
Merge branch 'master' into gamelistdb
Aloshi Mar 3, 2015
c52fcb1
Added automatic gamelist.xml import prompt.
Aloshi Mar 3, 2015
10a59b4
Merge branch 'master' into gamelistdb
Aloshi Mar 3, 2015
1a5b916
Added time conversion for import/export XML.
Aloshi Mar 4, 2015
7635f11
Merge branch 'master' into gamelistdb
Aloshi Mar 6, 2015
ccbf432
Prevent stripping parentheses
jgeumlek Dec 24, 2015
8ec3c0a
switch to gamelistdb
jgeumlek Dec 24, 2015
59a170d
Make DateTimeComponent match time format stored in the database
jgeumlek Dec 25, 2015
c58afe1
Clip the text list selector bar
jgeumlek Dec 25, 2015
11dce75
Pop up text list if selection overflows the boundary
jgeumlek Dec 25, 2015
5e7c53d
Revert clipping box due to bug with slide animation
jgeumlek Dec 25, 2015
44e631e
Quick/Dirty case insensitive sort.
jgeumlek Dec 25, 2015
f9264bc
Combine the two menus roughly for now. Let "start" also start games.
jgeumlek Dec 26, 2015
68924d0
Make Menus cyclic, only slightly hackish
jgeumlek Dec 26, 2015
7a2c65f
Comment out access to the commented out Jump-To-Letter feature
jgeumlek Dec 26, 2015
6b193b4
Fix saving folder metadata to database
jgeumlek Dec 26, 2015
56dcbf3
Restructure the combined contextual/global menu, including quick acce…
jgeumlek Dec 27, 2015
871f506
Make file system scans a menu action rather than automatic, shortenin…
jgeumlek Dec 27, 2015
60fd82c
Make prompt more consistent
jgeumlek Dec 27, 2015
048082a
Store invalid times as NULLs in the database
jgeumlek Dec 28, 2015
1f63858
Improve sorting options
jgeumlek Dec 28, 2015
78c33df
Update details appropriately when on a non-game entry
jgeumlek Dec 28, 2015
497ac8d
More spacing on datetime in metadata editor. Workaround for DateTimeC…
jgeumlek Dec 28, 2015
80b5fbf
Add a little feedback to the database scan
jgeumlek Dec 28, 2015
2caabc3
Make Scraper more consistent with Database menu
jgeumlek Dec 28, 2015
d39b660
Add virtual boy platform id + scraping
jgeumlek Dec 28, 2015
fccdd44
Make rating component go up by half stars
jgeumlek Dec 28, 2015
b1c8962
Add ability to remove an entry from the database without deleting the…
jgeumlek Dec 28, 2015
28087e5
Improve sorting options
jgeumlek Dec 28, 2015
a18e491
Initial support for having filters
jgeumlek Dec 29, 2015
9b7c3b6
Make game details update on list creation, like when entering a folde…
jgeumlek Dec 29, 2015
8d3ed08
minor string fixes
jgeumlek Dec 29, 2015
f9db04c
Add rough UI for creating a filter
jgeumlek Dec 29, 2015
5f0c52d
Switch fully to the SQLite date time string format
jgeumlek Dec 29, 2015
2615bbf
Remove fractional seconds to macth sqlite native date time
jgeumlek Dec 29, 2015
8ca8070
Make filtering games by year of release easier
jgeumlek Dec 29, 2015
90c42ab
Add in construction of tags table
jgeumlek Dec 29, 2015
a65ab54
Update sqlite so that we may successfully create the useful tags view
jgeumlek Dec 29, 2015
ddb71fb
Add a quick terrible UI for adding/removing tags in the metadata editor
jgeumlek Dec 29, 2015
7aa3bb1
Add some rough behavior on empty list viewing
jgeumlek Dec 29, 2015
49cb4ec
whoops
jgeumlek Dec 29, 2015
6582811
fix the addition of a folders-first setting
jgeumlek Dec 29, 2015
d9a9ab6
Continue unwinding even if we somehow get back to the root
jgeumlek Dec 29, 2015
7671d2a
Add a setting for whether folders/filters should float to the top
jgeumlek Dec 29, 2015
0ebaef2
Add secret setting "MakeCombinedSystem" to show the true power of the…
jgeumlek Dec 29, 2015
bef2083
Add default value for MakeCombinedSystem
jgeumlek Dec 29, 2015
dee6d8b
Fix some database silliness
jgeumlek Dec 29, 2015
359eb18
Refactor some more computations as queries
jgeumlek Dec 30, 2015
fd38659
Missed an ORDER BY
jgeumlek Dec 30, 2015
4f2207d
Add a random sort for fun.
jgeumlek Dec 31, 2015
0356f7d
Change the default filter name to something slightly more reasonable
jgeumlek Dec 31, 2015
08665ef
Make RatingComponent avoid introducing spurious changes
jgeumlek Dec 31, 2015
3174edc
Make single-line text edits a little more intuitive
jgeumlek Dec 31, 2015
12e65a2
Add some metadata cache. Might need tweaking on systems of limited me…
jgeumlek Dec 31, 2015
88c2f48
Make gamelists slide animation wrap around like the system select does
jgeumlek Jan 2, 2016
9ab8bfb
Make FileData cache the metadata from the database, ensure it gets cl…
jgeumlek Jan 2, 2016
a13d5a0
Hard set some values just in case an aborted slide launch left us wei…
jgeumlek Jan 2, 2016
99ef2e8
Add floating point fuzz to rating component half-star updates, and ce…
jgeumlek Jan 2, 2016
9cac5c5
Rearrange editing menu
jgeumlek Jan 4, 2016
251195a
Replace hidden MakeCombinedSystem with ability to put filters on systems
jgeumlek Feb 21, 2016
eb4de84
quick DB bug fixes
jgeumlek Mar 10, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
GamelistDB starting to be integrated into SystemManager.
  • Loading branch information
Aloshi committed Dec 27, 2014
commit c0bc2ea399a9adb8776cd16dbda47fec9bf656cf
25 changes: 9 additions & 16 deletions es-app/src/GamelistDB.cpp
Original file line number Diff line number Diff line change
@@ -337,12 +337,10 @@ void GamelistDB::addMissingFiles(const SystemData* system)
const std::string& relativeTo = system->getStartPath();
const std::vector<std::string>& extensions = system->getExtensions();

// ?1 = fileid, ?2 = filetype
std::stringstream ss;
ss << "INSERT OR IGNORE INTO files (fileid, systemid, filetype) VALUES (?1, " << system->getName() << ", ?2)";
std::string insert = ss.str();

SQLPreparedStmt stmt(mDB, insert);
// ?1 = fileid, ?2 = filetype, ?3 = systemid
SQLPreparedStmt stmt(mDB, "INSERT OR IGNORE INTO files (fileid, systemid, filetype, fileexists) VALUES (?1, ?3, ?2, 1)");
sqlite3_bind_text(stmt, 3, system->getName().c_str(), system->getName().size(), SQLITE_STATIC);

SQLTransaction transaction(mDB);

// actually start adding things
@@ -355,16 +353,11 @@ void GamelistDB::updateExists(const SystemData* system)
{
const std::string& relativeTo = system->getStartPath();

std::stringstream ss;
ss << "SELECT fileid FROM files WHERE systemid = " << system->getName();
std::string readstr = ss.str();

ss.str("");
ss << "UPDATE files SET fileexists = ?1 WHERE fileid = ?2 AND systemid = " << system->getName();
std::string updatestr = ss.str();

SQLPreparedStmt readStmt(mDB, readstr);
SQLPreparedStmt updateStmt(mDB, updatestr);
SQLPreparedStmt readStmt(mDB, "SELECT fileid FROM files WHERE systemid = ?1");
sqlite3_bind_text(readStmt, 1, system->getName().c_str(), system->getName().size(), SQLITE_STATIC);

SQLPreparedStmt updateStmt(mDB, "UPDATE files SET fileexists = ?1 WHERE fileid = ?2 AND systemid = ?3");
sqlite3_bind_text(readStmt, 3, system->getName().c_str(), system->getName().size(), SQLITE_STATIC);

SQLTransaction transaction(mDB);

9 changes: 4 additions & 5 deletions es-app/src/GamelistDB.h
Original file line number Diff line number Diff line change
@@ -19,7 +19,8 @@ boost::filesystem::path fileIDToPath(const std::string& fileID, SystemData* syst
[file ID] [system ID] [file type] [file exists] [metadata 0] [metadata 1] [metadata 2] ... etc.
The primary key for this table is the pair (file ID, system ID).

File ID is a string. File type is an int. Metadata types correspond to the MetaDataDecl vector.
File ID and system ID are strings. File type is an int. File exists is a boolean.
Metadata types correspond to the MetaDataDecl vector.

File ID is in the following format:
./path/to/game.rom - for paths relative to system root
@@ -29,12 +30,10 @@ boost::filesystem::path fileIDToPath(const std::string& fileID, SystemData* syst

File ID is ALWAYS stored with forward slashes ("generic directory separators," as boost::filesystem calls them).

File type is an enum value, where:
0 - game
1 - folder
File type corresponds to the C++ enum FileType.

File exists is a boolean indicating whether or not the file is present on the file system.
This value is set at startup.
This value is set at startup by the "updateExists" method.
*/


31 changes: 29 additions & 2 deletions es-app/src/SystemManager.cpp
Original file line number Diff line number Diff line change
@@ -122,7 +122,7 @@ void SystemManager::loadConfig()
themeFolder = system.child("theme").text().as_string(name.c_str());

// validate as best we can (make sure we're not missing required information)
if(name.empty() || path.empty() || extensions.empty() || cmd.empty())
if(!isValidSystemName(name) || path.empty() || extensions.empty() || cmd.empty())
throw ESException() << "System \"" << name << "\" is missing name, path, extension, or command!";

// convert path to generic directory seperators
@@ -142,6 +142,8 @@ void SystemManager::loadConfig()
mSystems.push_back(newSys);
}
}

updateDatabase();
}

void SystemManager::writeExampleConfig(const fs::path& path)
@@ -217,4 +219,29 @@ SystemData* SystemManager::getPrev(SystemData* system) const
return *mSystems.rbegin();
else
return *it;
}
}

bool SystemManager::isValidSystemName(const std::string& name)
{
if(name.empty())
return false;

const std::string invalid_chars = "/\\";
for(unsigned int i = 0; i < name.size(); i++)
{
for(unsigned int j = 0; j < invalid_chars.size(); j++)
if(name[i] == invalid_chars[j])
return false;
}

return true;
}

void SystemManager::updateDatabase()
{
for(auto it = mSystems.begin(); it != mSystems.end(); it++)
{
mDatabase.addMissingFiles(*it);
mDatabase.updateExists(*it);
}
}
5 changes: 4 additions & 1 deletion es-app/src/SystemManager.h
Original file line number Diff line number Diff line change
@@ -38,6 +38,9 @@ class SystemManager

// if forWrite is true, will only return ~/.emulationstation/es_systems.cfg, never /etc/emulationstation/es_systems.cfg
static boost::filesystem::path getConfigPath(bool forWrite);
static bool isValidSystemName(const std::string& name);
static std::string getDatabasePath();
void writeExampleConfig(const boost::filesystem::path& path);
static void writeExampleConfig(const boost::filesystem::path& path);

void updateDatabase();
};
23 changes: 1 addition & 22 deletions es-app/src/main.cpp
Original file line number Diff line number Diff line change
@@ -161,26 +161,6 @@ void delete_singletons_on_exit()

int main(int argc, char* argv[])
{
const char* errorMsg;

Log::open();

loadSystemConfigFile(&errorMsg);
GamelistDB test("test.db");

const std::vector<SystemData*>& systems = SystemManager::getInstance()->getSystems();
for(auto it = systems.begin(); it != systems.end(); it++)
{
SystemData* sys = *it;
test.importXML(sys, sys->getGamelistPath(false));
std::string out_path = "test_db_to_xml_";
out_path += sys->getName() + ".xml";
test.exportXML(sys, out_path);
}

Log::close();

/*
unsigned int width = 0;
unsigned int height = 0;

@@ -349,8 +329,7 @@ int main(int argc, char* argv[])
delete window.peekGui();

window.deinit();
*/


LOG(LogInfo) << "EmulationStation cleanly shutting down.";

return 0;