A way to enforce a order that seeds are loaded?

[marcelolx]

For example, with fixtures if on one fixture I say that it is associated with an organization, Rails magically runs the organization fixture first (I guess?), but that doesn't happen with oaken which makes it quite hard to even try to migrate an existing suite relying on fixtures because you have to organize the folders in a way that the seeds runs in the correct order.

Example

# test/fixtures/users.yml
josh:
  name: Josh

# test/fixtures/organizations.yml
acme:
  name: Acme Inc.

# test/fixtures/organization_memberships.yml
josh_at_acme:
  user: josh
  organization: acme

I know this example might be simple, and the obvious answer is, "You can do it all in a single seed file, following the right sequence", but that gets quite messy when your data model is way bigger and more interconnected than a simple user having a membership on an organization.

Another helpful example is an organization that can have multiple Plans. With fixtures, you would have both in the same folder level, but with Oaken, we might not be able to do that because the seed file that has the plans might run before the one that sets up the organization and users — now imagine this for 100 different models that are all interconnected in some way, the folder structures needed to guarantee the correct order would be painful.

[kaspth]

@marcelolx heyo 👋

For example, with fixtures if on one fixture I say that it is associated with an organization, Rails magically runs the organization fixture first (I guess?)

Rails fixtures doesn't do any ordering, but disables referential integrity (e.g. foreign_keys) and just pipes SQL to the database and thus skips callbacks and validations. It's up to apps to connect the right models via the fixture label.

but that doesn't happen with oaken which makes it quite hard to even try to migrate an existing suite relying on fixtures because you have to organize the folders in a way that the seeds runs in the correct order.

Was this after running bin/rails g oaken:convert:fixtures?

I know this example might be simple, and the obvious answer is, "You can do it all in a single seed file, following the right sequence", but that gets quite messy when your data model is way bigger and more interconnected than a simple user having a membership on an organization.

For sure, though Oaken's general idea is to have apps try to establish a logical storyline order, from bigger to smaller using their semantic sense of the app. E.g. here that'd be organizations.create then users.create then organization_memberships.create.

You're right the intention is to do a simple relationship like this inline in one seed file.

I get the idea of keeping the "interconnectedness" to try to make it easier to migrate, but given the myriad different ways that apps can organize themselves — I don't know how to maintain code that would support that.

Does it feel impossible to reorder the data set as you're migrating?

Another helpful example is an organization that can have multiple Plans. With fixtures, you would have both in the same folder level, but with Oaken, we might not be able to do that because the seed file that has the plans might run before the one that sets up the organization and users — now imagine this for 100 different models that are all interconnected in some way, the folder structures needed to guarantee the correct order would be painful.

Yeah, the idea is that you control the seed loading order in db/seeds.rb:

Oaken.prepare do
  seed :plans # Organizations depend on plans so seed those first.
  seed :organizations
end

Does this work for you? It's possible we could have our fixtures converter resolve these dependencies and generate seed calls. That might make it easier for apps to see how their object-graph is connected too.

[kaspth]

Here's another tip to help resolve the interconnectedness and get a better overview of your object graph at the same time. Basically I think an object-graph can be carved into 3-broad sections. The few roots that almost everything else connects to, then the stems that connect to the roots and then the leafs that connect to both the other two.

Sorta like this:

Oaken.prepare do
  section :roots # Potentially lots of interconnectedness and order dependencies, so be careful.
  seed :plans # Organizations depend on plans so seed those first.
  seed :organizations
  
  section :stems
  # Clearly depend on the root models having been seeded first.
  
  section :leafs
  # No-specific order dependency between these, but they may depend on seeds from earlier sections.
  # These attach to both roots and stems models.
end

section is a built-in Oaken method, supports deep-block nesting too if need be.

You could also call these sections primary/secondary/tertiary if that's more to your liking.

---

Another option, though I haven't tried this: I think it's possible to call seed from within another seed file, so you can mark dependencies from within another file. There's nothing in Oaken yet that prevents double-loading of files though.

[marcelolx]

Rails fixtures doesn't do any ordering, but disables referential integrity (e.g. foreign_keys) and just pipes SQL to the database and thus skips callbacks and validations. It's up to apps to connect the right models via the fixture label.

Oh interesting, I didn't realize that.

Was this after running bin/rails g oaken:convert:fixtures?

No, I got some confusing error when I tried to run it (sorry, I didn't save the stack trace)

For sure, though Oaken's general idea is to have apps try to establish a logical storyline order, from bigger to smaller using their semantic sense of the app. E.g. here that'd be organizations.create then users.create then organization_memberships.create.

You're right the intention is to do a simple relationship like this inline in one seed file.

I get the idea of keeping the "interconnectedness" to try to make it easier to migrate, but given the myriad different ways that apps can organize themselves — I don't know how to maintain code that would support that.

Does it feel impossible to reorder the data set as you're migrating?

Well, it doesn't feel like it is impossible, but it is a huge lift to convert all fixtures to oaken seeds in this logical storyline order — if we could live in an in-between world where we can run fixtures and oaken together, then it would be less painful since we could do it gradually, but it doesn't seem to possible to use organizations(:acme) (fixtures) once you have organizations.acme (oaken).

Yeah, the idea is that you control the seed loading order in db/seeds.rb:

Oaken.prepare do
  seed :plans # Organizations depend on plans so seed those first.
  seed :organizations
end

Does this work for you? It's possible we could have our fixtures converter resolve these dependencies and generate seed calls. That might make it easier for apps to see how their object-graph is connected too.

Interesting, I didn't know that this was possible in the Oaken.prepare block — So another question, what is really the idea of the section method? Just to describe a "section" in our seeds.rb?

Thanks for the quick reply! I think this has been helpful, I think it might be worth improving the README with some of those details about the ordering (for some, it might be obvious, but it wasn't for me 😅)

[marcelolx]

Here's another tip to help resolve the interconnectedness and get a better overview of your object graph at the same time. Basically I think an object-graph can be carved into 3-broad sections. The few roots that almost everything else connects to, then the stems that connect to the roots and then the leafs that connect to both the other two.

Sorta like this:

Oaken.prepare do
  section :roots # Potentially lots of interconnectedness and order dependencies, so be careful.
  seed :plans # Organizations depend on plans so seed those first.
  seed :organizations
  
  section :stems
  # Clearly depend on the root models having been seeded first.
  
  section :leafs
  # No-specific order dependency between these, but they may depend on seeds from earlier sections.
  # These attach to both roots and stems models.
end

section is a built-in Oaken method, supports deep-block nesting too if need be.

You could also call these sections primary/secondary/tertiary if that's more to your liking.

Another option, though I haven't tried this: I think it's possible to call seed from within another seed file, so you can mark dependencies from within another file. There's nothing in Oaken yet that prevents double-loading of files though.

Thanks, yeah, we might explore something like this