YCSB+T: Added transactions to DB and DBWrapper and validate() to Workloa... #169

akon-dey · 2014-04-14T04:08:33Z

...d.

YCSB+T: Added an example ClosedEconomyWorkload with a simple implementation of validate and a Simple Anomaly Score.

…load. YCSB+T: Added an example ClosedEconomyWorkload with a simple implementation of validate and a Simple Anomaly Score.

Fixes MongoExceptions that were being raised previously

This branch fixes two bugs in MongoDbClient: ClassCastExceptions raised due to the incorrect handling of field values returned by database queries in read() MongoExceptions raised due to attempts to update the _id field value in update()

The return value of DB.update() was previously unchecked in ClosedEconomyWorkload.doTransactionReadModifyWrite(), leading DB.abort() not being called when a transaction failed half-way

Check DB.update() return value in ClosedEconomyWorkload.doTransactionReadModifyWrite()

Fix variable swap bug so that ClosedEconomyWorkload only moves money from higher-numbered accounts/keys to lower-numbered accounts/keys

busbey · 2015-08-06T01:40:11Z

bump on this issue. would like to see this resolved for the Sep release.

busbey · 2015-08-06T01:42:18Z

since this changes the core binding definition it'll probably take some work to get right. /cc @allanbank since the patch as is touches the Mongo code, though I've no reason to believe it will need substantially different handling than other datastores.

open quesiton: if we have commit / abort, what do we do with datastores that have no notion of transactions?

akon-dey · 2015-08-06T05:03:40Z

The default start(), commit() and abort() are defined as empty methods in DB.java. This is similar to the cleanup() and init() default implementations.

busbey · 2015-08-06T05:11:47Z

ah. and the default consistency check just says yes blindly. I see. could merge that with the new verify stuff in core workload.

thanks for the pointer!

Conflicts: core/src/main/java/com/yahoo/ycsb/Client.java core/src/main/java/com/yahoo/ycsb/workloads/CoreWorkload.java mongodb/src/main/java/com/yahoo/ycsb/db/MongoDbClient.java

…/YCSB

akon-dey · 2015-08-10T01:47:44Z

I have merged the changes but the verification code is not fully integrated yet. I'd be happy to help with it.

busbey · 2015-08-10T03:05:50Z

core/src/main/java/com/yahoo/ycsb/Client.java

+		}
+		catch (InterruptedException e)
+		{
+		  // do nothing.


this is probably wrong. we should at a minimum set the interrupted status of the thread.

busbey · 2015-08-10T03:06:42Z

please don't include additional format fixes, it makes it harder to review the substance of the change.

busbey · 2015-08-10T03:09:22Z

core/src/main/java/com/yahoo/ycsb/Client.java

+		   //and the sleep() doesn't make sense for granularities < 1 ms anyway
+		   if ( (_targetOpsPerMs>0) && (_targetOpsPerMs<=1.0) ) 
+		   {
+		      sleep(Utils.random().nextInt((int)(1.0/_targetOpsTickNs)));


this looks unrelated to the change mentioned in the PR description, could you leave it out?

Sorry about the format changes. I could merge it again if that helps.

uakfdotb · 2015-08-17T14:49:41Z

It seems like the update operations reset records to the initial value (totalcash / recordcount), if there has been a readmodifywrite performed on the record then it seems like this breaks the closed economy invariant.

…merge.

Merge latest upstream YCSB commits into YCSB+T

akon-dey · 2016-02-10T05:29:34Z

Please review the recent changes and let me know if you want me to address anything.

0xl3x1 · 2016-02-11T02:57:51Z

@akon-dey I wrote a very similar start(), commit(), abort() implementation for JdbcDBClient last year, except I added the ability to set the transaction isolation level (see https://github.com/benjaminbrent/YCSB/blob/e2aa4f4be142afc2acfd98e5c3e22bd659da8d01/jdbc/src/main/java/com/yahoo/ycsb/db/JdbcDBClient.java#L255).

It would be nice to have this in your implementation too. I could submit a pull request if you think this is worthwhile?

akon-dey · 2016-02-11T04:02:07Z

This would be useful. We should be able to set the isolation level using the workload configuration properties file. That will make it truly usable.

akon-dey · 2016-02-11T04:02:46Z

Please submit a pull request with the ability to set the workload properties.

morcelicaio · 2022-04-11T23:04:06Z

Hello @akon-dey .

Have you noticed the behavior of ycsb+t when running the load phase (in the ClosedEconomyWorkload layer) using the parameter w = 0 in the mongo db cluster?

It simply gives a NumberFormatException error on the last record (in the ClosedEconomyWorkload layer).

I'm running on Mongodb Atlas and I noticed this error doing tests.

Do you have any idea about this behavior?

akon-dey · 2022-04-12T23:17:26Z

@morcelicaio I have not experienced it. If you can send me a stack or better, a pull request, I would be happy to incorporate it.

morcelicaio · 2022-04-19T11:39:57Z

@akon-dey , I did some more tests on another computer and didn't get the same error. Maybe it could have been some point on the other machine. If I get the same error again, I'll let you know. Thanks!

morcelicaio · 2022-04-19T16:44:41Z

Hi @akon-dey and @busbey , I would like to ask another question.

Wasn't YCSB+T merged with the original YCSB?

As I understand it, in the latest available version of YCSB there is no transactional layer you added. I would like to know the status of this.

Thanks!

akon-dey · 2022-04-19T16:52:26Z

@morcelicaio I don't believe it was merged. There were some changes to the PR that had questions and hence not merged. One was code format related and the other was regarding completeness of the ClosedEconomyWorkload. The first was trivial and the second was not intended to be addressed by the code. It would be best to recreate the PR to address the first issue and merge if it looks good.

busbey · 2022-04-19T17:07:01Z

it has not yet been merged. There's also merge conflicts, so it will need to go through a rebase before review.

morcelicaio · 2022-11-08T14:01:32Z

Hello guys, @busbey, @akon-dey .

Regarding YCSB+T, he talks about evaluating the transactional part of NoSQL databases.

I'm still testing with MongoDB.
I noticed that in the YCSB+T implementation we don't have the code implemented that creates the transaction as it is in the MongoDB documentation Transaction API (https://www.mongodb.com/docs/manual/core/transactions/).

Do I need to implement this code to use YCSB+T?
Or do the start() abort() and commit() methods somehow perform these "transactions" implicitly?

Thanks

busbey · 2022-11-08T20:53:15Z

I would assume that if you don't see the mongo transaction API being called then probably the existing code is not making use of it.

morcelicaio · 2022-11-09T21:37:24Z

Just that, @busbey . So how does YCSB+T assess transactional overhead and have a layer to assess transactional consistency when the transaction code implementation is not in the YCSB+T code?

It's a question that maybe other users have as well.

busbey · 2022-11-09T22:13:17Z

as mentioned previously, this proposed change hasn't been merged yet. It also started in 2014, so it's possible it predates the mongo transactional api.

YCSB+T: Added transactions to DB and DBWrapper and validate() to Work…

ed30fdf

…load. YCSB+T: Added an example ClosedEconomyWorkload with a simple implementation of validate and a Simple Anomaly Score.

akon-dey closed this Apr 14, 2014

akon-dey reopened this Apr 14, 2014

westonplatter added the core label May 30, 2014

Benjamin and others added 8 commits December 12, 2014 10:06

Fix type cast exceptions from MongoDbClient

87c8e1f

Prevent attempts to update _id field

a1d38a1

Fixes MongoExceptions that were being raised previously

Check DB.update() return value in readModifyWrite

9997808

The return value of DB.update() was previously unchecked in ClosedEconomyWorkload.doTransactionReadModifyWrite(), leading DB.abort() not being called when a transaction failed half-way

Merge pull request #2 from benjaminbrent/bugfix-check-update-return-val

76adff1

Check DB.update() return value in ClosedEconomyWorkload.doTransactionReadModifyWrite()

Fix variable swap bug

16ebe78

Make whitespace consistent

89c6d12

Merge pull request #3 from benjaminbrent/bugfix-variableswap

d15ace0

Fix variable swap bug so that ClosedEconomyWorkload only moves money from higher-numbered accounts/keys to lower-numbered accounts/keys

0xl3x1 mentioned this pull request May 27, 2015

error when using workloade for mongodb test #184

Closed

akon-dey added 3 commits August 9, 2015 15:11

Implement start(), abort() and commit() methods.

5ce8c37

Merge branch 'master' of https://github.com/brianfrankcooper/YCSB

acb74a2

Conflicts: core/src/main/java/com/yahoo/ycsb/Client.java core/src/main/java/com/yahoo/ycsb/workloads/CoreWorkload.java mongodb/src/main/java/com/yahoo/ycsb/db/MongoDbClient.java

Finalize merge branch 'master' of https://github.com/brianfrankcooper…

2c541f3

…/YCSB

busbey reviewed Aug 10, 2015
View reviewed changes

Merge upstream brianfrankcooper/YCSB master

76174a1

Benjamin and others added 5 commits February 9, 2016 14:06

Fix incorrect merge of JdbcDBClient

06d6eeb

Update YCSB+T code to use new Generator and Status classes. Finalize …

275a01d

…merge.

Merge pull request #4 from benjaminbrent/merge-ycsbt-with-ycsb-upstream

e11f725

Merge latest upstream YCSB commits into YCSB+T

Fix checkstyle formatting errors in JdbcDBClient.java.

37ac0f5

Fix copyright message.

a66c2c3

risdenk added the improvement label Sep 28, 2016

martijndeheus mentioned this pull request Oct 29, 2020

Add transaction workload #1477

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YCSB+T: Added transactions to DB and DBWrapper and validate() to Workloa... #169

YCSB+T: Added transactions to DB and DBWrapper and validate() to Workloa... #169

akon-dey commented Apr 14, 2014

busbey commented Aug 6, 2015

busbey commented Aug 6, 2015

akon-dey commented Aug 6, 2015

busbey commented Aug 6, 2015

akon-dey commented Aug 10, 2015

busbey Aug 10, 2015

busbey commented Aug 10, 2015

busbey Aug 10, 2015

akon-dey Aug 10, 2015

uakfdotb commented Aug 17, 2015

akon-dey commented Feb 10, 2016

0xl3x1 commented Feb 11, 2016

akon-dey commented Feb 11, 2016

akon-dey commented Feb 11, 2016

morcelicaio commented Apr 11, 2022

akon-dey commented Apr 12, 2022

morcelicaio commented Apr 19, 2022

morcelicaio commented Apr 19, 2022

akon-dey commented Apr 19, 2022

busbey commented Apr 19, 2022

morcelicaio commented Nov 8, 2022

busbey commented Nov 8, 2022

morcelicaio commented Nov 9, 2022

busbey commented Nov 9, 2022

YCSB+T: Added transactions to DB and DBWrapper and validate() to Workloa... #169

Are you sure you want to change the base?

YCSB+T: Added transactions to DB and DBWrapper and validate() to Workloa... #169

Conversation

akon-dey commented Apr 14, 2014

busbey commented Aug 6, 2015

busbey commented Aug 6, 2015

akon-dey commented Aug 6, 2015

busbey commented Aug 6, 2015

akon-dey commented Aug 10, 2015

busbey Aug 10, 2015

Choose a reason for hiding this comment

busbey commented Aug 10, 2015

busbey Aug 10, 2015

Choose a reason for hiding this comment

akon-dey Aug 10, 2015

Choose a reason for hiding this comment

uakfdotb commented Aug 17, 2015

akon-dey commented Feb 10, 2016

0xl3x1 commented Feb 11, 2016

akon-dey commented Feb 11, 2016

akon-dey commented Feb 11, 2016

morcelicaio commented Apr 11, 2022

akon-dey commented Apr 12, 2022

morcelicaio commented Apr 19, 2022

morcelicaio commented Apr 19, 2022

akon-dey commented Apr 19, 2022

busbey commented Apr 19, 2022

morcelicaio commented Nov 8, 2022

busbey commented Nov 8, 2022

morcelicaio commented Nov 9, 2022

busbey commented Nov 9, 2022