Credits

To RedHat Engineers:

  Paul Howarth,
  Denis Fateyev,

  Near the very end of the 1.8 cycle were several revisions. Each time, an
  email was sent to you stating that MCE was completed. But each morning,
  I woke up with a sense that something else was missed.

  (a) Running MCE and MCE::Shared inside an eval block.
  (b) Check for EINTR condition during sysread and syswrite.
  (c) Check the OS while running with ~ 200 workers.
  (d) Test MCE with the Wx toolkit - wxWidgets.
  (e) Test MCE and MCE::Shared by running taint mode via perl -T.
  (f) Validate DESTROY for shared Condvar and Queue objects.
  (g) Improve MCE::Shared::Handle for non-shared use, was not 100%.

  Going from 99.7% to 100.0% required so much effort. I am deeply sorry
  for the many emails sent. Although far away in miles, you were there
  in keeping me from going insane. For this I thank you.

  Best,
  Mario

--

###############################################################################
#  * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # *  #
###############################################################################

What follows are many people reporting problems, fixes or solutions.
More specifically in alphabetical order by last name.

Also, not to forget all the people mentioned in the change log.

Baumgart, Philippe

  For trying the MCE interval option and calling MCE->yield. I felt bad for
  Philippe spending many hours that it caused me to simulate the interval
  option over and over again. It turns out that there can be unnecessary
  delays from a worker taking extra time. MCE 1.815 resolves the issue by
  allowing the next available worker to run versus workers running orderly.
  Thank you, Philippe for your patience.

Berger, Joel

  The inspiration for the sixth model (MCE::Step) came from reading a blog
  by Joel, "Writing Non-Blocking Applications with Mojolicious: Part 3":

  https://blogs.perl.org/users/joel_berger/2014/01/
         writing-non-blocking-applications-with-mojolicious-part-3.html

Bouras, George

  For reporting sockets failing under the Windows environment due to a
  period of inactivity. An example is when workers execute an external
  command taking beyond 4 minutes to complete. On returning, the sockets
  have gone stale with a CPU core going 100% at that point.

  The MCE 1.601 release is attributed to the many use cases I received
  from George.

Cantrell, David (DrHyde on perlmonks.org)

  For reporting on MCE hanging with cpan -t MCE under his environment. Also,
  thank you for providing me a guest account via ssh. I was able to reproduce
  the issue. Thus, added writeable check and croak if tmp dir is not writeable.
  One cannot assume that /dev/shm is writeable by everyone. :)

Collet, Eric

  For the inspiration to the tre-agrep binary.

  I first learned of tre-agrep from Eric. He emailed me running times for a
  couple scenarios utilizing a single core. Thus increasing the need to have
  a MCE-driven wrapper to run tre-agrep in parallel.

  The bin/mce_grep script was created to show folks how one may go about
  chunking data between Perl and an external C binary efficiently without
  invoking the shell. The script supports grep, egrep, fgrep, agrep and
  tre-agrep.

Eldai, Hisham

  While working on the biofasta examples, Hisham introduced me to hg19.fa.
  A couple records inside hg19 (driven by $/ = "\n>") are 250 MiB in size.
  Memory optimizations in v1.600 came about by running against hg19.fa
  hundreds of times.

###############################################################################
#  * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # *  #
###############################################################################

Farrell, David

  David wrote a nice article about MCE (love the picture). What stood out was
  his example. I never thought about using mce_grep { ... } <$fileHandle>
  without the _f (mce_grep_f). At the time, updated MCE::Grep and MCE::Map
  to be fully aware of wantarray.

  The blog "Make your code run faster with Perl's secret turbo module" is
  located at:

  https://www.perl.com/article/61/2014/1/21/
         Make-your-code-run-faster-with-Perl-s-secret-turbo-module/

Gasper, Felipe

  For test scripts found in IO::SigGuard. I was able to get MCE::Mutex
  to fail with similar testing. The result from testing is that MCE too,
  must check for EINTR during sysread and syswrite operations. The same
  is true for MCE::Shared.

Golden, David

  For the Hash::Ordered module. I learned a lot from this *GEM* on CPAN.
  Particularly, the use of overload. I didn't know that was possible.

  For the time during collaborative efforts in making Hash::Ordered faster.
  I learned some new tricks along the way.

  Thank you for alarming me of the memory leaks with the on-demand hash
  dereferencing inside MCE::Shared::Ordhash. MCE::Shared::Object was also
  impacted. I'd never forget that dreadful day.

  MCE::Shared::{ Array, Hash, and Ordhash } are inspired by Hash::Ordered.
  In particular, the clone and iterator methods. Also, keys and values on
  taking arguments. I confess for never getting used to the "as_list"
  method name, initially. :)

Gorwits, Oliver

  For reporting on a couple issues with MCE along with a solution for each.

  Pointed me in the right direction for improving the logic for the die
  handler. Basically, eval { die 'this should not cause MCE to die' };
  This has reached 100% (all cases, threads and processes) in MCE 1.600.

Halpenny, Shawn

  For reporting an issue (bug RT#94706) with signal handling in MCE::Signal.
  Also, thank you for tip on getpgrp.

Hedden, Jerry

  From reading threads::shared. In particular the make_shared function.
  That was helpful in getting MCE::Shared working.

Hluchan, Yary

  For providing a benchmark script which I used to compare IPC performance
  between 1.6 and 1.7. For many kind words near the end of the development
  cycle for MCE 1.7 and MCE::Shared.

###############################################################################
#  * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # *  #
###############################################################################

Jacobsen, Dana

  For reporting an issue with get_ncpu on AIX. In addition, suggesting
  various solutions. Also, thank you for your time with Sandboxing with
  Perl + MCE + Inline::C at https://github.com/marioroy/mce-sandbox/.

Karasik, Dmitry

  For bug (RT#102802) and elegant solution for the die handler inside
  MCE::Core::Worker and MCE::Signal.

Kharchenko, Oleksandr

  Someone once asked Oleksandr why not use MCE. Oleksandr created 2 modules
  on CPAN, Parallel::DataPipe and Parallel::parallel_map. I used his example
  heavily in comparing IPC between pipe-driven and socket-driven. Not pleased
  with MCE's performance with chunk_size => 1, although 2 was as fast if not
  faster, I tried various things to include IO::Select. Finally, realized
  increasing the number of data channels was all that was needed (v1.500).

Kulow, Stephan

  For making the OpenSUSE package for MCE.
  https://build.opensuse.org/package/show/devel:languages:perl:CPAN-M/perl-MCE

Mabon, Philip

  For reporting on a couple issues with MCE in particular bug RT#92627.

McKeown, Benjamin

  For reporting several edge cases with MCE in particular bugs RT#105557,
  RT#105559, and RT#111780.

Mertens, David

  I am thankful for the tip on making PDL + MCE a possibility. Workers were
  crashing during exiting. Adding "sub PDL::CLONE_SKIP { 1 }" inside MCE.pm
  solved this issue entirely. Also, for tips on effectively using PDL in
  his example at https://gist.github.com/run4flat/4942132.

  Barrier synchronization in MCE came from reading parallel_sync in David's
  Perl module; PDL::Parallel::threads::SIMD.

Mughal, Zakariyya

  First, Demian Riccardi introduced me to Devel::IPerl by Zakariyya. The die
  handlers in MCE broke the user experience. It seems that looking at $^S
  (for Perl state) is not enough. MCE v1.600 fixes this once and for all.
  The answer is in perl-5.20.1/cpan/CGI/lib/CGI/Carp.pm (Carp::longmess).

###############################################################################
#  * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # *  #
###############################################################################

Ogulin, Joseph

  For providing the initial perl-MCE.spec file.
  For the cool remarks at work about MCE being really big :)

Riccardi, Demian

  I received a message from Demian asking for mce_map. His message requested
  ( my @a = mce_map { ... } 1..100 ). Five models were introduced in v1.500;
  MCE::Flow, MCE::Grep, MCE::Loop, MCE::Map, and MCE::Stream.

  Find capability were limited in MCE::Shared 1.001. For example, not able to
  mix :AND(s) and :OR(s) together. One day, I clicked on recent distributions
  on metacpan.org and came across HackaMol::Roles::SelectionRole. The trick
  is to eval the query string together with grep or map. Thank you Demian.

Rowe, Jillian

  For reporting on IO::File handles not working with MCE. Fixed in 1.515.
  For believing in possibilities beyond text-book thinking.

Roy, Sylvia (spouse)

  For supporting me from v1.0, especially during later 1.6 revisions and 1.7.
  But not to forget the journey during 1.8. So little was known on the hurdles
  to cross. It's all over now with MCE 1.827 and MCE::Shared 1.823.

Šabata, Petr

  For initial package submission at Red Hat.
  https://bugzilla.redhat.com/show_bug.cgi?id=1162531

Sasser, Tom

  For reporting on bin/mce_grep failing with Perl 5.8.x.

Schlichting, Florian

  For making a Debian package for MCE: libmce-perl

Shen, Wei

  For the BioUtil CPAN module. BioUtil::Seq::FastaReader inspired me to
  create parallel examples (biofasta folder) in MCE 1.600. We tried to
  make FastaReader in BioUtil::Seq even faster.

###############################################################################
#  * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # * # *  #
###############################################################################

Sjøgren, Adam

  For reporting on running /usr/bin/time mce_script.pl and pressing Ctrl-C
  failing. The default behavior in MCE::Signal is to call setpgrp. MCE 1.403
  adds a new option to MCE::Signal to not call setpgrp. This also gives the
  developer finer control as to where setpgrp should be called, if ever
  needed, such as in the child process and not the manager process.
  
    use MCE::Signal qw(-no_setpgrp);
    use MCE;

Smith, Marcus

  For reporting a problem with MCE including a sample script demonstrating
  MCE not 100% UTF-8 safe. All this time, totally overlooked the fact that
  the length function deals in logical characters, not physical bytes. :)

Timmermans, Leon

  The inspiration for not having to backslash variables for MCE::Shared came
  from reading Const::Fast: e.g. mce_share my @array => ( list );

Thalhammer, Jeffrey Ryan

  For submitting a feature request for lazy arrays support.

  Although a lazy array cannot be used directly for input_data, support for
  iterator objects was added to MCE 1.505; e.g. input_data => $iterator_ref;
  This enabled support for lazy arrays using closures.

  I am thankful for the small code snippet highlighting a use-case of MCE.
  I would have failed initially without it. Your example was placed under
  the "SYNTAX for INPUT_DATA" section in MCE::Core for others to see on
  how to process output from Path::Iterator::Rule in parallel.

Warm regards,
Mario