Adding various enhancements

[ainesophaur]

I would like to add some functionality to this package as I'm hoping to heavily lean on laravel-populator to setup data both in migration files, but also by invoking the populator manually. I noticed there weren't test cases in this package as compared to the knowledge base package. I wasn't sure if all my potential additions would be considered too risky without a test suite available.

I welcome all feedback as I'm targeting submitting a PR over the weekend. Out of curiosity, if I end up adding test cases to cover the current and expanded logic, are you opposed to using Pest?

Changes

Switch insert logic to use model

Switch from DB::table($this->bundle->table)->insertGetId(...) to using the model class to create the item and return the id from getKey()

New features

Ability to control the insert logic via closure

The closure would be expected to return an int|string for the inserted id

Bundle::make(Model::class)
    ->insertUsing(function(array $attributes): int|string {
        return Model::createOrFail($attributes)->getKey();
    })
    //...
;

Allow for multiple runs

Something along the lines of

Bundle::make(Model::class)
    ->unique('email', 'team_id')
    //...
;

which during InsertPipe::insert(...) do Model::updateOrCreate(..) and return the key/id from getKey()

Ability to roll back/remove records that were inserted

$populator->rollback();

Thinking the Bundle definitions would be required to provide unique columns via unique() in order to call rollback() as I would need the unique columns in order to find and delete the correct record.

Switch out the pipeline in the Processor when performing a rollback to something like

$this->data->pipeThrough([
            $this->relations(...),
            $this->defaults(...),
            $this->findAndDelete(...),
            $this->related(...),
        ]);

I could even provide delete logic via closure the same way I'm proposing the insert via closure.

Hooks

Basic hooks for before/after saving to allow further customization. The hooks would be expected to have a void return

[lukas-frey]

Hi Travis, thanks a lot for the input. I wasn't really sure anyone else besides our company was using the package. :D

Switch insert logic to use model

Switch from DB::table($this->bundle->table)->insertGetId(...) to using the model class to create the item and return the id from getKey()

We chose this approach because in older versions of Laravel, if you used the Model class queries and the the database table changed throughout the time (and so does your $fillable property), these calls would sometimes fail if you were still inserting data in the migration that was later removed from the $fillable property.

I think this no longer happens in new Laravel versions, but I'm not sure. If that's indeed the case, this change is very welcome!

New features

Ability to control the insert logic via closure

The closure would be expected to return an int|string for the inserted id

Bundle::make(Model::class)
    ->insertUsing(function(array $attributes): int|string {
        return Model::createOrFail($attributes)->getKey();
    })
    //...
;

This could be a nice addition.

Allow for multiple runs

Something along the lines of

Bundle::make(Model::class)
    ->unique('email', 'team_id')
    //...
;

which during InsertPipe::insert(...) do Model::updateOrCreate(..) and return the key/id from getKey()

Can you explain the usage of this?

We use this for example to create default accounts for admins etc. With this approach, this would cause the password to reset when run multiple times. Not to mention that running a bundle multiple times is not recommended anyways, because when the user changes their e-mail, this would result in creating another default account.

Ability to roll back/remove records that were inserted

$populator->rollback();

Thinking the Bundle definitions would be required to provide unique columns via unique() in order to call rollback() as I would need the unique columns in order to find and delete the correct record.

If you want to implement a rollback mechanism, I think you're going to need to delete any records created within the run of the Bundle, not just those marked as unique. Or am I missing something here?

Switch out the pipeline in the Processor when performing a rollback to something like

$this->data->pipeThrough([
            $this->relations(...),
            $this->defaults(...),
            $this->findAndDelete(...),
            $this->related(...),
        ]);

Make sure not to break older functionality with this, since there's currently more processors than in your example above.

I could even provide delete logic via closure the same way I'm proposing the insert via closure.

Go for it, that could be useful. 👍

Hooks

Basic hooks for before/after saving to allow further customization. The hooks would be expected to have a void return

Good idea.

I welcome all feedback as I'm targeting submitting a PR over the weekend. Out of curiosity, if I end up adding test cases to cover the current and expanded logic, are you opposed to using Pest?

Adding test cases is very welcome!

[ainesophaur]

Thank you for your feedback as it definitely gave me some points to keep in mind while Im in the planning phase. My current plan is to finish writing tests cases for the current code base before making any changes or additions. I would be happy to submit a PR with just the tests once they're completed so we can discussing the planned changes.

Allow for multiple runs

Can you explain the usage of this?

We use this for example to create default accounts for admins etc. With this approach, this would cause the password to reset when run multiple times. Not to mention that running a bundle multiple times is not recommended anyways, because when the user changes their e-mail, this would result in creating another default account.

My use case is a bit of a hybrid as I'm using the populator to populate initial data for the entire site during migrations, but I also use the populator to add "sample/seed" data for individual teams within my application. The latter case is more the need for this functionality as it'll allow users to (for example) run the seed, delete some of the data, and then run the seed again because they realized they still needed to "reference" the sample data. In the case of partial deleted data the bundle will fail to insert upon subsequent runs.

Ability to roll back/remove records that were inserted

If you want to implement a rollback mechanism, I think you're going to need to delete any records created within the run of the Bundle, not just those marked as unique. Or am I missing something here?

I was thinking of the rollback being run after the fact to which the bundle/storage would be unaware of the inserted ids. In my application I would like users to be able to clear all the sample data that was inserted by the bundle

[ainesophaur]

Do you have any feelings towards a model with a morphOne relationship to track entries? My thoughts are it'll be opt-in (so the model needs to implement a marker interface). The populated model can use a trait for the relationship to determine whether its a populated entry.

Population tracking can be disabled via any service providers boot() method by

function boot() {
   Processor::disableTracking()
}

or via the populator.php config file

<?php
return [
    'tracking' => true,
];

Model for tracking

class Population extends Model
{
    protected $guarded = [
        'populator',
        'key',
    ];

    public function populatable(): MorphTo
    {
        return $this->morphTo();
    }

    public function getMorphClass(): string
    {
        return 'population';
    }
}

return new class() extends Migration
{
    public function up(): void
    {
        Schema::create('populations', static function (Blueprint $table): void {
            $table->id();
            $table->timestamps();
            $table->string('populator');
            $table->string('key');
            $table->morphs('populatable');
            $table->unique(['populator', 'key']);
        });
    }

    public function down(): void
    {
        Schema::dropIfExists('populations');
    }
};

trait TracksPopulationPipe
{

    public function track(Collection $data): Collection
    {
        $model = $this->bundle->model;
        if($model instanceof TracksPopulatedEntries && static::hasTrackingFeature()) {
            assert($model instanceof Model);
            $processor = $this->bundle->populator->getName();
            foreach (data_get($this->bundle->populator->memory->all(), $model::class) as $key => $id) {
                Population::insert([
                    'key' => $key,
                    'populator' => $processor,
                    'populatable_id' => $id,
                    'populatable_type' => $model->getMorphClass(),
                ]);
            }
        }


        return $data;
    }

}

Trait to be used by models that want to access their tracked instance

trait HasPopulation
{
    /**
     * @return MorphOne<Population>
     */
    public function population(): MorphOne
    {
        return $this->morphOne(Population::class, 'populatable');
    }


}

Empty marker interface used to enable tracking for the model

interface TracksPopulatedEntries
{
}

ex

class TestUser extends Model implements Authenticatable, TracksPopulatedEntries
{
    use HasFactory, HasPopulation;
}

[lukas-frey]

As long as it's opt-in and you document all the new features, I'll be ok with merging it.

I'd prefer

Populator::enableTracking()

though instead.