-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixed Tautrimas comments and added basic pipeline documentation. #103
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
Pipeline | ||
======== | ||
|
||
Pipeline is used to process data with 5 events: | ||
|
||
- source | ||
- start | ||
- modify | ||
- consume | ||
- finish | ||
|
||
Pipeline starts with source event which provides all data which should be processed. | ||
Then start event is fired to indicate that items are about to come | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Then start event is fired to indicate that items are about to come_,_ |
||
after that pipeline iterates through all items from all sources by calling modify and consume events | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. after that pipeline iterates through all items from all sources, calling modify and consume events with each item [as a parameter?] |
||
with each item. After the iteration finish event is fired to notify that no more items will follow. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. After the iterations are finished, finish event is fired to notify that no more items will follow. |
||
|
||
Pipeline can have any number of listeners for each event but for functioning pipeline | ||
at least one source and consume listener should be provided. | ||
|
||
Listeners should listen to ongr.pipeline.<PipelineName>.<Target>.<Event> events. | ||
Example: | ||
|
||
.. code-block:: yaml | ||
|
||
test.import.modifier: | ||
class: %ongr_connections.import.modifier.class% | ||
tags: | ||
- { name: kernel.event_listener, event: ongr.pipeline.import.default.modify, method: onModify } | ||
.. | ||
|
||
Item skipping | ||
------------- | ||
If item for some reason should not be consumed without stopping pipeline ItemSkipException can be used. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If item for some reason should be skipped without stopping pipeline, ItemSkipException can be used. (double negatives are confusing :( ) |
||
|
||
When modifier throws `ItemSkipException` pipeline catches it and sets skipException in `ItemPipelineEvent`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When |
||
If `AbstractConsumeEventListener` is used `skip` method will called if skipException exception set otherwise `consume` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If |
||
method is called. | ||
|
||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,11 +17,30 @@ | |
use ONGR\ConnectionsBundle\Pipeline\PipelineFactory; | ||
use Symfony\Component\EventDispatcher\EventDispatcher; | ||
|
||
/** | ||
* PipelineTest class. | ||
*/ | ||
class PipelineTest extends \PHPUnit_Framework_TestCase | ||
{ | ||
/** | ||
* Pipeline data provider. | ||
* | ||
* @return array | ||
*/ | ||
public function pipelineData() | ||
{ | ||
// Modifier skips items which are string 'skip', anything else gets consumed. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Modifier skips items which are equal to 'skip', anything else gets consumed. sounds a bit better IMHO. |
||
return [ | ||
// Case #0: No data so then should be 0 consumed and skipped items. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Case #0: No data. Results should be: 0 consumed and skipped items. |
||
[[], 0, 0], | ||
// Case #1: All data should be consumed so 1 consume and 0 skips. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Case #1: All data should be consumed, so 1 consume and 0 skips. |
||
[['consume'], 1, 0], | ||
// Case #2: All data should be skipped so 0 consumes and 1 skip. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Case #2: All data should be skipped, so 0 consumes and 1 skip. |
||
[['skip'], 0, 1], | ||
// Case #3: Data with consume and skips. 1 consume and 2 skips. | ||
[['skip', 'consume', 'skip'], 1, 2], | ||
// Case #3: Data with consumes and skip. 2 consumes and 1 skip. | ||
[['consume', 'skip', 'consume'], 2, 1], | ||
]; | ||
} | ||
|
||
/** | ||
* Tests pipeline. | ||
* | ||
|
@@ -70,20 +89,4 @@ public function onModify(ItemPipelineEvent $event) | |
throw new ItemSkipException(); | ||
} | ||
} | ||
|
||
/** | ||
* Pipeline data provider. | ||
* | ||
* @return array | ||
*/ | ||
public function pipelineData() | ||
{ | ||
return [ | ||
[[], 0, 0], | ||
[['consume'], 1, 0], | ||
[['skip'], 0, 1], | ||
[['skip', 'consume', 'skip'], 1, 2], | ||
[['consume', 'skip', 'consume'], 2, 1], | ||
]; | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sentence becomes clear after reading it several times and also reading the code. Not sure if that's so great, but it's not that bad, either.