
TopoFlow 3.5 Release Notes
February 26, 2017

===============================
  Enhancements & New Features
===============================
* Improved installation instructions.  See "How_to_Install_TopoFlow_3.5.pdf"
  in the "docs" folder.

* Added "__main__.py" in the "topoflow" package folder which allows
  TopoFlow model runs to be launched at a terminal prompt with a
  much shorter command.  See the new installation instructions.

* New "path info configuration files":  [cfg_prefix]_path_info.cfg
  Previously, four "path info variables":  in_directory, out_directory,
  site_prefix and case_prefix were repeated in the CFG file for every
  component in topoflow/components.  Now, these four variables are
  listed in a single, new "path info CFG file".  Since the new CFG
  file is read by the "read_path_info()" method in BMI_base.py *before*
  reading a component's CFG file with "read_config_file()", it is
  important to remove these four lines from pre-existing CFG files.

  This greatly simplifies the preparation of CFG files for new model runs,
  but now all components must use the same in_directory and out_directory.
  (These 2 directories can still be different, allowing the same input files
  to be shared by multiple users without making copies, while output files
  are saved in a user-owned directory with write permission.)

* Previously, TopoFlow relied on a module for computing D8 flow directions
  and derived grids in /topoflow/utils/tf_d8_base.py, while the Erode
  landscape evolution model in /topoflow/components/erode_d8_global.py
  used a different D8 module in /topoflow/components/d8_global.py.
  Now the latter (and better) D8 module is used for both TopoFlow and Erode.
  This module is written as a component and is therefore configured with its
  own CFG file:  [cfg_prefix]_d8_global.cfg.  TopoFlow model runs now
  require this additional CFG file.

* Addition of initialize_var() and update_var() methods to utils/BMI_base.py.
  The initialize_var() method calls either the initialize_grid() or
  initialize_scalar() method.
  initialize_var() is only called in channels_base.py, so far.
  update_var() is now called in:
      topoflow/components/channels_base.py
      topoflow/components/met_base.py
      topoflow/utils/model_input.py 
  Inclusion in model_input.py reduces runtime for every case where a
  time-varying input variable is read from a file into memory by any
  component.   Using these new methods ensures that references to numpy
  arrays, including "0d arrays", are preserved and properly updated and
  this results in shorter runtimes.  See comments in the code for details.
  These new commands are not yet used throughout the code.

* Improved the output printed to the console at the end of a TopoFlow
  model run by the component "topoflow_driver.py".  Note that any component
  can be the "driver" of a model run, by setting the driver_comp_name
  flag.  However, the "topoflow_driver" component collects information
  from all of the other process components in the model run and then prints
  a nice report at the end of the model run.

* Added a mediator for automatic unit conversion to EMELI that uses
  the cfunits Python package that requires first installing the UDUNITS2
  library from Unidata.  However, since installing UDUNITS2 on MacOS, and
  especially on Windows, can be a difficult process, and since existing
  components do not require unit conversion, by default this release uses
  a version of EMELI that does not require cfunits:
     topoflow/framework/emeli.py
  but also includes a version that does use cfunits:
     topoflow/framework/emeli_with_cfunits.py

* Performed basic testing of every TopoFlow component with the Treynor
  example data set.  CFG files, model output files and complete console
  output for a set of 12 tests are provided in:
     topoflow/examples/Treynor_Iowa_30m/00_Output_Files

* Ran basic, successful tests of the TF 3.5 package on Windows with an
  Anaconda Python distribution.  One required change was the replacement of
  "x.astype(np.int32)" with "np.int32(x)" to change the type of an array
  to int32, within a call to np.concatenate. (In infil_base.py.) It is not
  clear if a similar change will be needed anywhere else.
  
* Added a new infiltration process component (infil_beven.py).
  This hydrologic infiltration component implements the model introduced
  by Beven (1984) where saturated hydraulic conductivity, Ks, decreases
  exponentially with subsurface depth, z.

* Note that CSDMS introduced some small changes to the Basic Model Interface
  (BMI) in the past few years, which could be called BMI v2.  Two new methods
  are:  get_var_nbytes() and get_var_itemsize(), which have both been
  implemented for this release, in BMI_base.py.  Four other new BMI v2 methods
  provide a new way to describe the computational grid used by each variable
  in the model.  These new methods/functions are:
      get_grid_type(), get_grid_rank(), get_grid_size(), and
      get_var_grid().
  These methods have not yet been implemented for this release.  These
  four BMI v2 methods support regridding, which isn't needed for the TF 3.5
  components.

  Additional changes in BMI v2 are:
      (1) removal of the "get_attribute()" method, and
      (2) addition of the "update_until()" method.

* Fixed some "array ordering" bugs in topoflow/utils/BMI_base.py:
    In get_grid_shape(),   ordering is now [nz, ny, nx].
    In get_grid_spacing(), ordering is now [dz, dy, dx].
    In get_grid_origin(),  ordering is now [z0, y0, x0].

* New initialize_basin_vars() in BMI_base.py that uses outlets.py.

* Removed "feature" where the midpoint of a DEM was used for the basin
  outlet in the absence of an outlet_file: [case_prefix]_outlets.txt.
  This resulted in strange behavior and very small discharge values.
  TopoFlow will no longer run without a valid "outlet_file".

* Removed numerous, obsolete code blocks that had been commented out.

* Updated EMELI to remove references to the obsolete "port" concept.
  See: topoflow/framework/emeli.py.

* Fixed bug where files were written to user's home directory when
  out_directory = "." in the path info CFG file.

* Fixed bugs in the Diversions component and added the "plane" test data
  set in topoflow/examples/Test_Plane_Canal.  Ran a series of successful
  tests described in: "00_README_Test_Plan_Canal.txt". Also see the
  movies in the 00_Movies folder.

* NetCDF output files now use NETCDF3_CLASSIC vs. NETCDF4_CLASSIC format.
  As with previous versions, TF 3.5 uses the netCDF4 Python package to
  write model output to NetCDF files.  This is done with the utilities:
      topoflow/utils/ncts_files.py    (for NetCDF Time Series files)
      topoflow/utils/ncgs_files.py    (for NetCDF Grid Stack  files)
      topoflow/utils/ncps_files.py    (for NetCDF "Profile Series" files)
      topoflow/utils/nccs_files.py    (for NetCDF "Cube Stack" files)
  (This still seems to be the best overall Python package for NetCDF.)
  TopoFlow output files from these utilities contain 0D, 2D, 1D or 3D in
  their names, respectively.  The netCDF4 package can write NetCDF3 or
  NetCDF4 formats, but the NetCDF4 files are typically MUCH bigger than
  NetCDF3 files with the same data.  This results from "time" having an
  "unlimited" dimension (since the number of time slices is typically not
  known in advance).  NetCDF4 uses "chunking" in this case, but even
  after adjusting the "chunksizes" keyword, filesizes for the Treynor
  test data set created using the NETCDF4 or NETCDF4_CLASS formats were
  found to be from 5 to 800+ times bigger than those created using the
  NETCDF3_CLASSIC format.  Note that NetCDF3 files can be converted to
  NetCDF4 files, if necessary, using the nccopy utility from UniData.

* Fixed a bug in topoflow/componenets/gc2d.py, a component that can
  model the dynamics of valley glaciers.  The bug was in the filter2d()
  function, where the order of arguments to convolve() had to be reversed.
  Also fixed cause of "invalid division" error in the basal_shear_stress()
  function.

* Implemented a new version of the "update_velocity()" method in:
     topoflow/components/channels_dynamic_wave.py.  For details, see:
  "Peckham_2017_Routing_Notes.pdf" in the "docs" folder.  The dynamic
  wave component for D8 channel routing should be used with caution as it
  is susceptible to subtle numerical stability issues.

=================================
  Known Issues & Possible Bugs
=================================

* Many components & utilities do not have "triple quote" help strings.

* Most of the component unit tests (in "tests" folders) are broken or missing.
  (However, all components were tested for Treynor data set: see above.)

* In satzone_base.py, must have water table elevation < land surface
  elevation (DEM) everywhere at start of a model run.

* Should add more detail to the README.md file in the package.  However,
  there is now a lot of documentation in the "docs" folder.

* TF 3.5 writes "long names" from variables to NetCDF files, but these
  names do not yet use the CF Standard Names nor the CSDMS Standard Names.

* There are still some calls to "idl_func" methods that should be updated.
  These are holdovers from the initial conversion from IDL to Python using
  I2PY.

* Python exec() command is still used too much in model_output.py.

* The update_water_balance() method is commented out in:
     topoflow/components/evap_base.py.
  It has not been examined/tested for a long time.






