Clean up db file.

[vevova]

Hello Christoph.
I use hamsterdb with some tests and have question.
I dont shrink DB file after create db, records inserts,delete db and close Env.
If i do loop with records insert-delete file growth continuously.
Any comments?

Thanks, Vladimir.

[cruppstahl]

Hi Vladimir,

the db will only shrink if the pages at the end of the file are completely
empty and unused. If you keep on inserting/deleting then usually the pages
will not be empty. If you delete a lot of data (without inserting) then the
chances are increasing.

Exception: the file is currently not truncated if you use Windows and if
memory mapped I/O is enabled (this is the case if you open an existing file
instead of creating a new file). I plan to work on this limitation for one
of the next releases.

Best regards
Christoph

2015-10-27 18:25 GMT+01:00 vevova [email protected]:

Hello Christoph.
I use hamsterdb with some tests and have question.
I dont shrink DB file after create db, records inserts,delete db and close
Env.
If i do loop with records insert-delete file growth continuously.
Any comments?

Thanks, Vladimir.

—
Reply to this email directly or view it on GitHub
https://github.com/cruppstahl/hamsterdb/issues/60.

[vevova]

Thanks for answer, Christoph.

I use windows & I/O mammed disable flag.
But if i createdb - insert-delete-rec - deletedb file truncated if it size has some Kb.
If file size more then 10-20 Mb it not work, file size growth continiously.
At close moment Environment has no any DB.
Its feature windows realization?

Vladimir

[cruppstahl]

Can you send me a short test case? Then I'll have a look.

2015-10-28 9:07 GMT+01:00 vevova [email protected]:

Thanks for answer, Christoph.

I use windows & I/O mammed disable flag.
But if i createdb - insert-delete-rec - deletedb file truncated if it size
has some Kb.
If file size more then 10-20 Mb it not work, file size growth continiously.
At close moment Environment has no any DB.
Its feature windows realization?

Vladimir

—
Reply to this email directly or view it on GitHub
https://github.com/cruppstahl/hamsterdb/issues/60#issuecomment-151758245
.

[vevova]

Its modification your Sample6:

#define LOOP 200000

int
run_demo() {
uint32_t i;
hamsterdb::env env; /* hamsterdb environment object /
hamsterdb::db db; / hamsterdb database object /
hamsterdb::db db2; / hamsterdb database object /
hamsterdb::key key; / a key /
hamsterdb::key key2; / a key /
hamsterdb::record record; / a record /
char *value = "!@#$%^&()_1234567890-=q wertyuiop[asdfghjkl;zxcvbnm,./";
ham_parameter_t params[] = { /* parameters for ham_env_create_db */
{ HAM_PARAM_KEY_TYPE, HAM_TYPE_UINT32 },
{ 0, }
};

try{
    /* Create a new environment file and a database in this environment */
    try{
        env.open("test.db", HAM_DISABLE_MMAP);
    }
    catch (hamsterdb::error &e) {
        std::cerr << e.get_string()
            << std::endl;

        env.create("test.db", HAM_DISABLE_MMAP);
    }

    try{
        db = env.open_db(1);
        db2 = env.open_db(2);
    }
    catch (hamsterdb::error &e) {
        std::cerr << e.get_string()
            << std::endl;

        db = env.create_db(1, 0, &params[0]);
        db2 = env.create_db(2, 0, &params[0]);
    }

    std::cout << "======> db::insert value." << std::endl;

    for (i = 0; i < LOOP; i++) {
        key.set_size(sizeof(i));
        key.set_data(&i);

        record.set_size(strlen(value));
        record.set_data((void*)value);

        db.insert(&key, &record);
        db2.insert(&key, &record);
    }

    env.flush();

    std::cout << "======> db::find value." << std::endl;

    for (i = 0; i < LOOP; i++) {
        key.set_size(sizeof(i));
        key.set_data(&i);

        record = db.find(&key);

        /* Check if the value is ok */

    }

    db.close();
    db2.close();
    env.close();
    env.open("test.db", HAM_DISABLE_MMAP);
    db = env.open_db(1);
    db2 = env.open_db(2);

    std::cout << "======> db::erase value." << std::endl;

    /* now erase all values */

    for (i = 0; i < LOOP; i++) {
        key.set_size(sizeof(i));
        key.set_data(&i);

        db.erase(&key);

        key2.set_size(sizeof(i));
        key2.set_data(&i);

        db2.erase(&key2);
    }

    for (i = 0; i < LOOP; i++) {
        key.set_size(sizeof(i));
        key.set_data(&i);

        try {
            record = db.find(&key);
        }
        catch (hamsterdb::error &e) {
            if (e.get_errno() != HAM_KEY_NOT_FOUND) {
                std::cerr << "db::find() returned error " << e.get_string()
                    << std::endl;
                return (-1);
            }
        }
    }



}
catch (hamsterdb::error &e) {
    std::cerr << e.get_string()
        << std::endl;
    return (-1);
}

db.close();
db2.close();

env.erase_db(1);
env.erase_db(2);

env.flush();
env.close();

std::cout << "success!" << std::endl;
return (0);

}

int
main(int argc, char **argv)
{
try {
return (run_demo());
}
catch (hamsterdb::error &e) {
std::cerr << "run_demo() failed with unexpected error "
<< e.get_errno() << " ('"
<< e.get_string() << "')" << std::endl;
return (-1);
}
}

[cruppstahl]

I'm currently finishing release 2.1.12, but after that (next week) I'll
look into this. Thanks for the sample.

2015-10-28 13:58 GMT+01:00 vevova [email protected]:

Its modification your Sample6:

#define LOOP 200000

int
run_demo() {
uint32_t i;
hamsterdb::env env; /* hamsterdb environment object
/ hamsterdb::db db; / hamsterdb database object
/ hamsterdb::db db2; / hamsterdb database object
/ hamsterdb::key key; / a key
/ hamsterdb::key key2; / a key
/ hamsterdb::record record; / a record
/ char *value = "!@#$%^&()_1234567890-=q
wertyuiop[asdfghjkl;zxcvbnm,./";
ham_parameter_t params[] = { /* parameters for ham_env_create_db */
{ HAM_PARAM_KEY_TYPE, HAM_TYPE_UINT32 },
{ 0, }
};

try{
/* Create a new environment file and a database in this environment */
try{
env.open("test.db", HAM_DISABLE_MMAP);
}
catch (hamsterdb::error &e) {
std::cerr << e.get_string()
<< std::endl;

    env.create("test.db", HAM_DISABLE_MMAP);
}

try{
    db = env.open_db(1);
    db2 = env.open_db(2);
}
catch (hamsterdb::error &e) {
    std::cerr << e.get_string()
        << std::endl;

    db = env.create_db(1, 0, &params[0]);
    db2 = env.create_db(2, 0, &params[0]);
}

std::cout << "======> db::insert value." << std::endl;

for (i = 0; i < LOOP; i++) {
    key.set_size(sizeof(i));
    key.set_data(&i);

    record.set_size(strlen(value));
    record.set_data((void*)value);

    db.insert(&key, &record);
    db2.insert(&key, &record);
}

env.flush();

std::cout << "======> db::find value." << std::endl;

for (i = 0; i < LOOP; i++) {
    key.set_size(sizeof(i));
    key.set_data(&i);

    record = db.find(&key);

    /* Check if the value is ok */

}

db.close();
db2.close();
env.close();
env.open("test.db", HAM_DISABLE_MMAP);
db = env.open_db(1);
db2 = env.open_db(2);

std::cout << "======> db::erase value." << std::endl;

/* now erase all values */

for (i = 0; i < LOOP; i++) {
    key.set_size(sizeof(i));
    key.set_data(&i);

    db.erase(&key);

    key2.set_size(sizeof(i));
    key2.set_data(&i);

    db2.erase(&key2);
}

for (i = 0; i < LOOP; i++) {
    key.set_size(sizeof(i));
    key.set_data(&i);

    try {
        record = db.find(&key);
    }
    catch (hamsterdb::error &e) {
        if (e.get_errno() != HAM_KEY_NOT_FOUND) {
            std::cerr << "db::find() returned error " << e.get_string()
                << std::endl;
            return (-1);
        }
    }
}

}
catch (hamsterdb::error &e) {
std::cerr << e.get_string()
<< std::endl;
return (-1);
}

db.close();
db2.close();

env.erase_db(1);
env.erase_db(2);

env.flush();
env.close();

std::cout << "success!" << std::endl;
return (0);

}

int
main(int argc, char **argv)
{
try {
return (run_demo());
}
catch (hamsterdb::error &e) {
std::cerr << "run_demo() failed with unexpected error "
<< e.get_errno() << " ('"
<< e.get_string() << "')" << std::endl;
return (-1);
}
}

—
Reply to this email directly or view it on GitHub
https://github.com/cruppstahl/hamsterdb/issues/60#issuecomment-151836207
.

[cruppstahl]

Fixed for 2.1.12. Thanks again for the bug report!

[vevova]

Hello Christoph.
I test new version upscale db with old sample.

1. Try:
   LOOP = 50000
2. Try
   LOOP = 500
3. Try
   LOOP = 50000
   In some case on 3 or 4 or 5 try i have exception:
   on db.insert operation - ReadFile failed with OS status 38(Reached the end of file).
   Any comments?