cdq context tutorial

[skandragon]

Is there a tutorial on how to use contexts with cdq? I really don't need the weight of rest kit, but I do need background syncing. I am likely to ditch RestKit for a multi device sync system, and would likely use cdq there. But, I am not sure how to actually do things like have a background thread use one cdq context that, when saved, rolls up into the main context that the UI sees. Or, a way to directly modify objects in a detail view, and then either cancel or save that context and have it roll up. That sort of thing SHOULD simplify code.

I have found some for core data, but mapping them into cdq's world view is currently beyond me. Give me a OS kernel, and I'm all set -- but core data is tricky! :)

[skandragon]

Chained contexts: Bad idea or good idea?

I have added an Objective-C example of how I imagined chained contexts should work, in a real world application: https://github.com/skandragon/CoreDataMultipleContexts

I am wondering if this is the direction cdq should be moving to, where there is a single parent context, a single UI context directly below it, and then applications create temporary contexts for "real work" under that. This would allow a subview to create a temporary context, and when saved, the changes trickle up the stack to the ultimate parent, which would commit these to the store.

[kemiller]

I could swear I responded to your first question several days ago, but it's not there. Hmm. I must have been distracted. In any case, CDQ already has some support for this. You can push contexts onto a stack:

cdq.contexts.push(root_context)
cdq.contexts.push(main_context)

or implicitly create them on the stack:

cdq.contexts.new(NSPrivateQueueConcurrencyType) # private writer
cdq.contexts.new(NSMainQueueConcurrencyType) # UI context

Then in your app code, if you want to do something in an isolated "scratch pad" context, just create a new context, do your work, then pop it off the top. There's even a shortcut for that:

cdq.contexts.new(NSMainQueueConcurrencyType) do
  # your stuff
  cdq.contexts.current.save # only save the temp context
end

When you call cdq.save it will march down the stack and calls performBlock (or performBlockAndWait, as appropriate, or if you tell it to) on all of them.

We use the private-writer pattern in Temple, so that much at least is pretty well field-tested. I don't want to make it the default because it's harder to reason about and not always necessary.

Does this address what you're talking about, or is there something else?

[skandragon]

I'm a little confused about how the threading works in this case.

If I were to create the two implicit contexts, and then use the block mode inside a thread, from my read of the code that thread would get its own stack, which would not inherit from the state of the stack when it was created.

Is my read wrong, and would pushing two contexts on, then creating a thread which creates another context, and a UI view that creates another context, in effect be a tree, with those two temp contexts created from the NSMainQueueConcurrencyType context in your example?

[kemiller]

So, yes, if you're using block mode within a thread you explicitly created, it's a bit trickier as-is. You'd have to get some reference to the main queue outside the stack, pass that into the new thread, then push it, then push your temp context. That case could be better, for sure, but I don't want to make it too magical too soon, since I don't have a lot of experience with it. But I'm not sure you really want to do it like that. You probably want to just push a NSPrivateQueueConcurrencyType context and use performBlock to run whatever you need on it, and let Core Data manage the thread for you. If you're doing things with UIViews they should be on the main thread anyway. This is more or less what I mean:

cdq.contexts.push(NSPrivate...) do 
  cdq.contexts.current.performBlock( -> {
    # your background stuff
  } )
  cdq.save
end

But frankly, I don't know what the best way is. You're the pioneer. :)

[kemiller]

perhaps a better behavior for push-with-block is to actually pass the block into performBlock on the new context directly... hmm..

[skandragon]

I'm starting to believe the stack model is not really able to properly map what people end up using, which is closer to a tree. I suspect it's usually a private writer, main queue, and then multiple children from there. The stack seems too magical, to borrow your term. :)

But, as it's a stack now, that can be mimicked. In general, I suspect people do not pop the "global" contexts. That is, I push a private, and then a main queue context. I will never pop those. If we can assume that (or make it an application specific thing if they choose to pop them) I think the API can stay the same, but the concepts change.

• The first context pushed is created, and cdq.contexts.current returns this one. This is the target of cdq.save, as well. cdq.contexts.current is thread specific data. We know it's the top of the "virtual stack" because its parentContext is nil.
• When I push another, my thread's cdq.contexts.current is updated to be this new context, and its parentContext is set to be what I used to have. This builds, in effect, a virtual stack.
• When I pop, cdq.contexts.current becomes the parentContext.
• These contexts are (by default, perhaps forever) chained so when I save them all, changes propagate up and each context is saved in turn.
• Blocks pass the context in so I can decide what to do with it.

[kemiller]

So I agree it will sometimes want to be a tree. However, there's another pattern some people use, which is to share a store coordinator between threads, but have no contexts directly interacting cross-thread. That is, your worker thread talks directly to the store with its own context, and the main thread learns of any changes by subscribing to its broadcasts. This has the advantage of only updating the objects that changed, rather than all objects, which is how it works when you propagate though the parent context. That was the first case I learned about so that's how I optimized. It seems like you want it to be able to do two things:

1. Start a new thread with a blank slate, no contexts at all, only a store coordinator (maybe not even that?)
2. Start a new thread with the existing stack as baseline, but any new pushes "branch off" and don't affect any other threads.

1 is how it works now.

For 2, I'm not 100% sure I understand what you're saying in your second bullet point, but I think we're close to the same page. What I'd do there is make some variant of push, a new method or an option, which creates a new thread and starts a new stack, BUT it sets the parentContext of the lowest context in the new thread to be the top of the spawning thread. So you still have a private stack, and you can't pop or automatically save any context that is owned by the spawning thread. But changes will propagate down for the spawning thread to do whatever it needs to with them.

I can't see any reason for a tree structure within a single thread. Can you?

Passing the context to the block is a great idea.

[skandragon]

Hmm. It's good to explore this stuff in more detail. It's tricky...

In the case where a store coordinator has multiple contexts, and updates are passed around between them without any direct relationship between contexts is new to me, but I've found some discussions on it. I love learning stuff!

I think the best way to manage contexts is to ditch the idea that cdq manages them in any way, ultimately. You must have one before you can issue a query, or do other work, but how that context is created is ultimately up to the developer. When a new one is created, it might be best to be able to specify the parent explicitly. I know this is back-tracking from the ease of use cdq allows now, but perhaps each query, insert, etc. should have a context explicitly associated with it as well, or at least be able to specify the context if you don't want "the last one you made that still exists."

I can see a reason to have a tree even within a single thread, I think. A tabbed controller with several different views into data, each of which may be in "edit mode" at any given time, is possible and perhaps common. Each of these views could have its own temporary context, awaiting a save or cancel independent of the other. All of these contexts were created on the UI thread, so the second one created would change the meaning of cdq.contexts.current in the other as things stand now.

[kemiller]

My priority with CDQ is to make common cases very easy, and support advanced cases as well as possible, while leaving an "out" so you can drop down into raw Core Data for really advanced stuff, without having to abandon all of CDQ in the process.

I can see your point re: a tree structure in a single thread, but until I encounter someone actually doing that in the real world, I'm hesitant to optimize for it, since you CAN do it now. (Just keep your own context and push it onto the stack while you're using it.) The internal API would support something like a .with_context(context) method inserted into the chain, as well, which might be easier to manage in some cases than pushing onto a stack.

So you do have a concrete problem in front of you that could use a better solution, and I think we have some good candidates.