1. Users can choose to use/enable PnetCDF non-blocking APIs by setting the
"enable_pnetcdf_bput" option to ".true." in the namelist.
2. If "enable_pnetcdf_bput" option is set to ".false.", the blocking APIs
will be used.
3. The "enable_pnetcdf_bput" option only affects the PnetCDF method (i.e.
io_form=11).

This commits add a subroutine "BputCalcBufferSize". It calculates the
buffer size needed by PnetCDF non-blocking APIs (bput calls). The
returned buffer size will equal the total amount of writes (in bytes)
for a particular timestep, excluding PnetCDF headers (only the data
section).

PnetCDF bput calls need additional buffer space because they cache write
reqs in memory and flush them to file system altogether later on. The
additional buffer space is for the caching purpose.

----------

Below explains the implementaion of this new subroutine.

The implementation codes are borrowed from
share/output_wrf.F: subroutine "output_wrf".

The subroutine "output_wrf" is to write all variables to file.
The implementation logic of "output_wrf" is that:

  Iterate over all variables (do while loop):
      Some "if" conditions:
          write/output the variable (of size amoutX) to file
      End IF
  End loop.

The new subroutine "BputCalcBufferSize" is to return the buffer size
(i.e. the total amount of writes). The logic here is to replace
"write/output the variable to file" in the original logic with
"increment total size":

  totalSize = 0
  Iterate over all variables (do while loop):
      Some "if" conditions:
          totalSize = totalSize + amoutX
      End IF
  End loop.

PNC-nb APIs refer to PnetCDF non-blocking APIs.

This commit provide subroutine to tell PnetCDF the size of buffer to be
used by PnetCDF non-blocking bput calls.

PNC-nb refers to PnetCDF non-blocking APIs.

Using the non-blocking APIs, multiple write requests will be coalesced,
and then flushed when an explict "flush" is called. This commits provide
subroutines to perform the explicit flush. The newly added subroutines
in essential call nfmpi_wait_all API to perform the flush.

PNC-nb refers to PnetCDF Non-Blocking APIs.

In this commit, we
1) call BputCalcBufferSize to calculate the buffer size needed by PNC-nb
2) call BputSetBufferSize to tell PnetCDF library the buffer size
3) call BputWait to flush pending write reqs.
4) tell PnetCDF library to detach the buffer at file close.

The call flow is:

  For each time step:
      if buffer_size is not calculated:
          buffer_size = BputCalcBufferSize(...)
      if not yet told PnetCDF the needed buffer_size:
          BputSetBufferSize(buffer_size, ...)

      For each variable:
          post write reqs (but not actually flushed)

      call BputWait(...) to flush pending write reqs

  When closing the file:
      tell PnetCDF library to detach the buffer

Here are some important facts:
0) Posting writ reqs is not yet implemented. Next commit will address
   this.
1) Currently only restart and history file can enable PnetCDF
   non-blocking APIs. It is enabled only if the namelist set
   "enable_pnetcdf_bput" to ".true.", and io_form_history,
   io_form_restart, or both are set to 11.
2) Buffer size calculation and setting buffer size happens before any
   write reqs.
3) Buffer size calculation happens at most once for all history files.
   This is because variables are same across different time steps and
   different history files. So calculating once is enough. Same for
   restart files.
4) BputSetBufferSize(...) is called at most once per file.
5) We flush pending/cached reqs to file at the end of each time step.
6) The PnetCDF library will only detach the buffer for non-blocking
   APIs during file close.

PNC-nb refers to PnetCDF non-blocking APIs.
In this commit, we call NFMPI_BPUT instead of NFMPI_PUT if non-blocking
APIs is enabled.

This is an optimization that applies to both blocking and non-blocking
PnetCDF APIs.

In PnetCDF, an application creating a file will first enter
"define mode", in which it can describe all attributes, dimensions,
types and structures of variables. The program will then exit
"define mode" and enter data mode, in which it actually performs I/O.

Previously in WRF, it enters and exits define mode several times,
while only once is enough/necessary. This commit remove redundant enter/
exit of define mode.

This is an optmization for PnetCDF (both blocking and non-blocking APIs)
. In WRF, there are two types of variables, paritioned and
non-partitioned. Non-partitioned variables are small variables that all
process write the entire variables with the same values to the file.
Partitioned varaibles are large variables; each process is responsible
for a (non-overrlapped) sub-region of such a variable.

When writing a non-partitioned variable, there's no need for every
process to make the PnetCDF write call.
1) If non-blocking APIs are used, only rank 0 will post write reqs
   using nfmpi_bput. Other processes do not need to call nfmpi_bput
   because nfmpi_bput does not require collective call.
2) If using blocking APIs, all processes still need to call
   nfmpi_put_all (which requires collective call). But only process of
   rank 0 has req size > 0, all other processes have req size = 0 (i.e.
   vcount = 0)