Retrain with new input files?

[ronandquinn]

Hi
Isn't it possible to retrain the model with new data, to improve output?
When I train the model with my own txt file, then retrain with a new txt file, using the init_from arg, the first issue is the new data and loaded model will obviously disagree on word set and dictionary mappings, but even if these assertions are removed, I still get:

`2018-03-27 10:41:11.721945: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Traceback (most recent call last):
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
    return fn(*args)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
    status, run_metadata)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,725] rhs shape= [256,726]
	 [[Node: save/Assign_26 = Assign[T=DT_FLOAT, _class=["loc:@rnnlm/softmax_w"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](rnnlm/softmax_w/Adam_1, save/RestoreV2_26)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "retrain.py", line 128, in <module>
    main()
  File "retrain.py", line 56, in main
    train(args)
  File "retrain.py", line 96, in train
    saver.restore(sess, ckpt.model_checkpoint_path)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1666, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,725] rhs shape= [256,726]
	 [[Node: save/Assign_26 = Assign[T=DT_FLOAT, _class=["loc:@rnnlm/softmax_w"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](rnnlm/softmax_w/Adam_1, save/RestoreV2_26)]]

Caused by op 'save/Assign_26', defined at:
  File "retrain.py", line 128, in <module>
    main()
  File "retrain.py", line 56, in main
    train(args)
  File "retrain.py", line 93, in train
    saver = tf.train.Saver(tf.global_variables())
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1218, in __init__
    self.build()
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1227, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1263, in _build
    build_save=build_save, build_restore=build_restore)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 751, in _build_internal
    restore_sequentially, reshape)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 439, in _AddRestoreOps
    assign_ops.append(saveable.restore(tensors, shapes))
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 160, in restore
    self.op.get_shape().is_fully_defined())
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/state_ops.py", line 276, in assign
    validate_shape=validate_shape)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/gen_state_ops.py", line 57, in assign
    use_locking=use_locking, name=name)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
    op_def=op_def)
  File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1470, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [256,725] rhs shape= [256,726]
	 [[Node: save/Assign_26 = Assign[T=DT_FLOAT, _class=["loc:@rnnlm/softmax_w"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](rnnlm/softmax_w/Adam_1, save/RestoreV2_26)]]`

Incompatible shapes

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [256,725] rhs shape= [256,726]

How is the init_from arg supposed to be used if not for retraining with new data?