After train my new add layer, how to fine turn with InceptionResnetV2 together next??

[RossTntt]

I have added some layer after InceptionResnetV2, and add them to variables_to_train. Then, only train the new layer, but the weight in InceptionResnetV2 did not change. Everything is fine by now.
However, I want to do next, turn down learning rate and then train the layer in InceptionResnetV2 and my new add layer together. At this time, I got the NOT FOUND ERR like "InceptionResnetV2/Repeat_1/block17_20/Conv2d_1x1/weights/Adam_1 when they are not found in the .ckpt file"
I have read the Q&A, but I have to do the fine turn like what I said, what should I do next??