file_copy.py error

[aroramanish2009]

Hi,
I was getting the following error when trying to use file_copy.py.

'''
failed: [10.x.x.55] => {"failed": true, "parsed": false}
Traceback (most recent call last):
File "/home/manish/.ansible/tmp/ansible-tmp-1450376959.58-153940520304606/nxos_file_copy", line 1761, in
main()
File "/home/manish/.ansible/tmp/ansible-tmp-1450376959.58-153940520304606/nxos_file_copy", line 136, in main
if not fc.file_already_exists():
File "/usr/lib/python2.7/site-packages/pycsco/nxos/utils/file_copy.py", line 57, in file_already_exists
dst_hash = self.get_remote_md5()
File "/usr/lib/python2.7/site-packages/pycsco/nxos/utils/file_copy.py", line 82, in get_remote_md5
md5_body = md5_dict['ins_api']['outputs']['output']['body']
KeyError: 'body'
'''

Resolved by making following changes :

'''
def get_remote_md5(self):
"""Return the md5 sum of the remote file,
if it exists.
"""
md5_dict = xmltodict.parse(self.device.show(
'show file {0} md5sum'.format(self.dst), text=False)[1])
#md5_body = md5_dict['ins_api']['outputs']['output']['body']
md5_body = md5_dict['ins_api']['outputs']['output']
if md5_body:
#return md5_body['file_content_md5sum']
return md5_body
'''

Thanks

[mzbenami]

@aroramanish2009 Thanks for finding this. We are having trouble reproducing this. Can you give some more information about your environment? What platform and software version are you using on the Nexus switch?

[aroramanish2009]

hi @mzbenami
I used it against Nexus 9000 C9372PX running 7.0(3)I1(2) & 7.0(3)I2(2).
Thanks

[mzbenami]

@aroramanish2009 Hmm, we tested on that platform with no errors. I see you are using the Ansible module. Can you share what parameters you are passing for source_file and dest_file? Are there any special characters or spaces in those parameters?

The device should always return a 'body' tag, even if the file doesn't exist. The only time it wouldn't, from our testing, is if there's an error. The only way we can see an error being generated on the command is if the file name has something unexpected about it, like special characters or spaces.

[aroramanish2009]

Here's the Error again:
File "/usr/lib/python2.7/site-packages/pycsco/nxos/utils/file_copy.py", line 82, in get_remote_md5
md5_body = md5_dict['ins_api']['outputs']['output']['body']
KeyError: 'body'

File: nxos.7.0.3.I2.2.bin

Here's Ansible Task:
#SCP[put] the BIN only if not present on the switch, Make sure source file path is correct.
- name: Sending File over to the switch bootflash.
nxos_file_copy: source_file='nxos.7.0.3.I2.2.bin' host={{ inventory_hostname }}

The above task works just fine when I changed your to :
#md5_body = md5_dict['ins_api']['outputs']['output']['body']
md5_body = md5_dict['ins_api']['outputs']['output']
if md5_body:
#return md5_body['file_content_md5sum']
return md5_body

Also, the content of body was just the MD5 of the file.
Manish

[mzbenami]

@aroramanish2009 Are you able to go to the API sandbox through your browser and try the command in the screenshot below and let me know your output. As you see in the shot below, we have the same file on the same software version, and the md5sum is returned in the body tag, not in the output tag.

Are you saying your fix works even when the file is already copied, meaning that Ansible doesn't transfer the file if it is already there? I can see your fix working, but causing Ansible to always transfer the file no matter what.

[aroramanish2009]

1. I am making Ansible to do file check in a task earlier, so it skips that file transfer task anyways if the file is already present.
2. I am getting an error on the Sandbox with that command ( running version 7.0(3)I1(2) ) :
   

[mzbenami]

@aroramanish2009 Sorry can you do it one more time with message format set to "XML" and and command type set to "cli_show". The library is using XML.

[aroramanish2009]

Ok It looks like I get the error when I am running version 7.0(3)I1(2) but the output matches your when I upgrade the switch to version 7.0(3)I2(2).

Error Page:

Success Page after NXOS upgrade:

I think, the code should be updated to try/expect to solve this problem so that both outputs are supported.
Manish

[aroramanish2009]

It is confirmed that the code breaks when using "NXOS: version 7.0(3)I1(2)" even though NXAPI version is same for I1(2) and I2(2).
I was able to get the file copy work on both versions by :

    try:
           md5_body = md5_dict['ins_api']['outputs']['output']['body']
           if md5_body:
               return md5_body['file_content_md5sum']
        except KeyError:
           md5_body = md5_dict['ins_api']['outputs']['output']
           if md5_body:
               return md5_body

The code fails to work when file is already present and I am using "NXOS: version 7.0(3)I1(2)", for now I am using the work around to check for file.

[aroramanish2009]

Latest update to fix the code for both versions, might not be completely optimal but works well :-)

    def get_remote_md5(self):
        """Return the md5 sum of the remote file,
        if it exists.
        """
        try:
           md5_dict = xmltodict.parse(self.device.show(
               'show file {0} md5sum'.format(self.dst), text=False)[1])
        except:
           md5_dict = xmltodict.parse(self.device.show(
               'show file {0} md5sum'.format(self.dst), text=True)[1])
        try:
           md5_body = md5_dict['ins_api']['outputs']['output']['body']
           if md5_body:
               return md5_body['file_content_md5sum']
        except TypeError:
           md5_body = md5_dict['ins_api']['outputs']['output']['body']
           if md5_body:
               return md5_body
        except KeyError:
           md5_body = md5_dict['ins_api']['outputs']['output']
           if md5_body:
               return md5_body

Please let me know if you come up with better solution.
Thanks
Manish

[mzbenami]

@aroramanish2009 I put a fix in this branch https://github.com/jedelman8/pycsco/tree/file_copy_bug_fix. Are you able to test it?

[aroramanish2009]

@mzbenami
Still getting a "KeyError: 'body'" with your version of the fix on the switch with "7.0(3)I1(2)".

[aroramanish2009]

@mzbenami
If I modify the code like below then it works with file being present/absent on both Cisco codes

    def get_remote_md5(self):
        """Return the md5 sum of the remote file,
        if it exists.
        """
        try:
            md5_dict = xmltodict.parse(self.device.show(
                'show file {0} md5sum'.format(self.dst), text=False)[1])
        except Exception as e:
            # bug in 7.0(3)I1(2)
            if 'Structured output unsupported' in e.msg:
                md5_body = e.err
                if md5_body:
                    return md5_body
            raise e
        else:
            try:
                md5_body = md5_dict['ins_api']['outputs']['output']['body']
                if md5_body:
                    return md5_body['file_content_md5sum']
            except KeyError:
                md5_body = md5_dict['ins_api']['outputs']['output']
                if md5_body:
                    return md5_body

[mzbenami]

@aroramanish2009 It's not clear for whether a CLIError is being raised when you run the code against the old device, but one should be. I'm not sure what version of code you are using, but it makes sense that it might not be raised for you if you're not on a recent version.

I understand that fix you gave works for you, but it would break the idempotency of the Ansible module. In the case of the KeyError, md5_body wouldn't contain the hash value of the remote file if it exists, so it would appear as if the remote file doesn't exist.

How did you install the fix I provided? You would need to clone the whole branch and run setup.py. I can give more detailed instructions if you need. Or if you prefer to install via pip, we will make sure the fix gets into the next version on PyPI, and let you know when it's available for download over pip.

[aroramanish2009]

I can try to clone the repo and use setup.py ( yes, it would help a lot if you can provide some detailed instructions).

[mzbenami]

@aroramanish2009 No problem it would be something like this:

git clone https://github.com/jedelman8/pycsco.git
cd pycsco
git checkout file_copy_bug_fix
sudo python setup.py install -f

[aroramanish2009]

@mzbenami
I tried the code again using the above process but still got the same results where if the file is not present then KeyError: 'body' thrown out but code works well if the file already exists because proper error is raised.
I agree with you that the Cisco NXOS old version doesn't raise CLIError when the file is not present which causes the last part of code in the function to execute and raise KeyError because body doesn't exists.
I am soon going to be upgrading my switches to new NXOS version anyways so I am not going to spend more time on this since I already have a workaround.
Thank you so much for your help & keep up the good work.

[bobbywatson3]

I'm seeing this issue as well when trying to copy a file that doesn't already exist on the switch. I just pulled the latest copy of pycsco, and I still have the issue. Upgrading NXOS on our switches is planned, but not going to happen right now. What are my options?

[bobbywatson3]

I'm going to try the ugly hack of having an ansible task that ensures the file is absent, then remove the
if not fc.file_already_exists():
command from the module and just copy it.

[jedelman8]

I'd actually recommend checking out one of these (plan is to maintain these more going forward):

https://github.com/networktocode/pyntc (uses pynxos below)

device.file_copy('newconfig.cfg')

Supports NXOS and IOS too:

>>> # CREATE DEVICE OBJECT FOR AN IOS DEVICE
>>> 
>>> csr1 = NTC(host='csr1', username='ntc', password='ntc123', device_type='cisco_ios_ssh')
>>>

>>> # CREATE DEVICE OBJECT FOR A NEXUS DEVICE
>>> 
>>> nxs1 = NTC(host='nxos-spine1', username='ntc', password='ntc123', device_type='cisco_nxos_nxapi')
>>> 

Can you try this?

[jedelman8]

Any luck @robertwatson3 ?

[bobbywatson3]

I'm using nxos-ansible, so I'd need to switch to ntc-ansible. My hack works for now, so I'm going to roll with it. I'll move on to ntc-ansible (and therefore pyntc) on the next project.