Faraday-Proxy PyPi Support #203

kb1lqc · 2017-06-03T20:18:52Z

Related to Issue #191

This PR is intended to get the Proxy application into the pip index. It also allows for "editable" mode installation with command line operation of proxy.

This PR will definitely need some love before merging but it's a good starting point for discussion

Operation

Navigate to the main folder and pip install -e . to install in editable mode. One can also follow the python packaging guides to create an uploadable distribution for the PyPi servers.

Once installed simply run "Faraday-Proxy" to run the proxy program from command line.

You will need to configure the proxy application by first seeing the options by running faraday-proxy -h

First you will need to create a new configure file by running faraday-proxy --configure then run a second Faraday-Proxy command with the appropriate options to set the callsign, node Id, and port (usually comx on windiws).

This is the best I got for remembering from memory try to figure it out the best you can with the help options and then asking me the rest. I don't have access to the software as I'm traveling internationally at this time without a laptop!

Initial add of setup.py for setuptools use when installing with pip.

Added Faraday specific data into the setup.py file.

Removed test folder indicated in setup.py as well as changed the README to REAMDE.md in the same file.

Still having 'pip install faraday' issues but it uploades to pypi!

Specified requirements in setup.py

Will move over to pbr and this requires a simplified setup.py.

@hdmike

Faraday-proxy appears to work. Now I need to fix the formatters ini filer error per @hdmike.

@hdkmike

Used @hdkmike suggested import code and the issue was fixed. Now I need to do the same for proxy.ini.

Successfully made it past the proxy.ini import. Now we need to get the sqlite schema file imported as it fails when running faraday-proxy.

After installing with pip in editable mode I can use the command faraday-proxy to start up the proxy application correctly! I simply brute forced the relative file paths for the database and database schema files.

Imported faraday/telemetry.py directly as well as fixed up paths to configuration files and databases.

Modified paths to configuration files in faraday/aprs.py as well as added correct lines to setup.cfg to properly create console entry points for faraday-aprs.

Very simple update of logging file locations as well as setup.cfg to point to the correct entry points.

Fixed data_file formatting and also forced a difference base version so I can upload to pypi without committing.

Updated proxy.py to use configuration files from etc/ folder.

To ensure all files are in a common faraday folder.

updated file paths to enable use of /etc/faraday/

@hdkmike

After talking to @hdkmike and @reillyeon we figured out a path forward for python packaging and config file locations. This is a sort of brute force attempt to get this working so not pretty but I don't want to optimise until I know we actually understand this.

Had to change some files around but this version works! Added a new path element as well. Still very brute forced. The user MUST copy proxy.sample.ini over to proxy.ini and edit it all manually right now. Also, the log file is created in /etc and it shouldn't but one step at a time.

Decided to save the log files in ~/faraday/.faraday/lib which now works. Not sure if long term want to keep this current structure with lib in user area and etc in python sys.prefix. Again, brute force trying to understand the workings of it all. Progress!

Ran pytest and fixed the PEP8 errors/warnings.

kb1lqd · 2017-06-04T21:27:12Z

faraday-proxy --configure is unrecognized

@reillyeon any thoughts?

kb1lqd · 2017-06-04T21:37:43Z

Oh no, I think this is messing with all my other faraday branches I have checked out... I can run applications and such from other branches after install this PR... I need to uninstall to run any other heritage branches?

EDIT:

I needed to pip uninstall faraday to run any other GITHUB branches in our heritage fashion.

el-iso · 2017-06-04T22:18:07Z

Yes, I experienced a similar problem. I think the package installed via pip takes precedence over some other stuff in the code. After I uninstalled the package it fixed it

…

On Jun 4, 2017 4:37 PM, "Brenton Salmi" ***@***.***> wrote: Oh no, I think this is messing with all my other faraday branches I have checked out... I can run applications and such from other branches after install this PR... I need to uninstall to run any other heritage branches? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#203 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AXZdHos9X68SRYj1Ih3vWoDn1m_0sbR3ks5sAyOngaJpZM4NvMMk> .

kb1lqc · 2017-06-05T03:58:39Z

Yes please do not use pip install Faraday at this time. In learning PyPi I uploaded a non working package to the main PyPi site instead of test PyPi. That version us very old and won't work. You must install editable or from PyPi test server.

However the PyPi test server will often fail on a fresh install since not all required packages have test PyPi uploads with the latest versions so you need to install thise from normal PyPi instead of test.

https://stackoverflow.com/a/34561435

This is the latest test PyPi which should be the same version in my git PR
https://testpypi.python.org/pypi/faraday/0.0.1009.dev988

Use the stack overflow method above to install from the test PyPi if you want. However I still suggest editable mode!

kb1lqc · 2017-06-05T04:04:48Z

@kb1lqd @el-iso woops! --init-config is what you want to create the initial config file, sorry about the confusion!

After that runs you'll have a config file with REPLACEME placeholders. Then run in one or more command line entries

Faraday-Proxy --callsign CALLSIGN --nodeid NODEID --port PORT

Make sure you remember port is com port so COM5 or similar on Windows and dev/ttyusb or similar on Linux and mac

kb1lqd · 2017-06-07T08:31:35Z

@kb1lqc

pip install -e . Looks to have successfully installed faraday:

I made sure to restart the command window just incase.

faraday-proxy --init-config fails!

It looks like it is not expecting arguments but getting them?

Also, I definitely could NOT run my RFDataport branch with this installed so I had to remove it again.

kb1lqc · 2017-06-08T08:35:53Z

Hmm make sure you have the python module shutil though I think it comes with python anyways. That said weird, it's complaining that I passed the initial argument which should be true or false...

kb1lqc · 2017-06-08T08:39:32Z

Yup @kb1lqd looks like line 86 of proxy.py clearly shows no arguments yet like 215 passes them. Not sure how this happened. Try modifying line 215 to remove the argument passed. Installed in editable mode literally means you can edit the file and try again, no need to re install

kb1lqd · 2017-06-18T02:23:55Z

@kb1lqc @reillyeon I verified that Bryce's suggestion to remove line 215 argument in proxy.py worked and I can now run proxy as described:

kb1lqc · 2017-06-18T22:15:59Z

Running the current commit b6798b4 results in the following error if not launched from the main folder. This test was performed by moving up one directory.

Bryce@BRYCE-SP4 MINGW64 ~/Documents/GitHub
$ faraday-proxy
Traceback (most recent call last):
  File "C:\Python27\Scripts\faraday-proxy-script.py", line 11, in <module>
    load_entry_point('faraday', 'console_scripts', 'faraday-proxy')()
  File "C:\python27\lib\site-packages\pkg_resources\__init__.py", line 560, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "C:\python27\lib\site-packages\pkg_resources\__init__.py", line 2648, in load_entry_point
    return ep.load()
  File "C:\python27\lib\site-packages\pkg_resources\__init__.py", line 2302, in load
    return self.resolve()
  File "C:\python27\lib\site-packages\pkg_resources\__init__.py", line 2308, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "c:\users\bryce\documents\github\faradayrf-kb1lqc\faraday\proxy.py", line 216, in <module>
    configureProxy(args, proxyConfigPath)
  File "c:\users\bryce\documents\github\faradayrf-kb1lqc\faraday\proxy.py", line 173, in configureProxy
    config.set(unit, 'BAUDRATE', args.baudrate)
  File "C:\python27\lib\ConfigParser.py", line 396, in set
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'UNIT0'

It appears that the code for finding proxy.ini is wrong and defaulting to the current directory.

Not bothering making faraday-telemetry and other apps since we are only focusing on getting faraday-proxy to run correctly right now.

Keeping it simple for the README by linking to the quickstart tutorial which will be updated with correct installation instructions.

Performing a logging debug message of the proxy.ini path instead of a print.

kb1lqc · 2017-06-18T23:29:21Z

faraday/proxy.py

+        path = location
+        break
+    except ConfigParser.NoSectionError:
+        pass


Because I don't fail gracefully this code allows editable mode (or other installations) to fail if the file is never found. The code simply moves on with a blank path. Woops. This is related
to IT #206

kb1lqc · 2017-06-18T23:31:39Z

faraday/proxy.py

+
+#Create Proxy configuration file path
+proxyConfigPath = os.path.join(path, "proxy.ini")
+logger.debug('Proxy.ini PATH: ', proxyConfigPath)


Woops, this is wrong. I need to use a + instead of comma to join the two items inside the logging.debug() code. Also, this is redundant and I should delete line 47

kb1lqc · 2017-06-18T23:39:47Z

faraday/proxy.py

+parser.add_argument('--unit', type=int, default=0, help='Specify Faraday unit to configure')
+
+#Proxy options
+parser.add_argument('--number', type=int, default=0, help='Set number of Faraday radios to use')


This may be dangerous and/or not work. Keep an eye on this one. Might need to default to 1?

This is actually OK. Defaulting to 0 basically keeps the 1 in the proxy.ini file until you specify something other than 0 or 1 (since 0 is ignored... you'd never have 0 units to startup. Maybe in testmode but that's getting in the weeds for now). Programming a number.

This does allow negative numbers and any number that is positive

OK this is addressed in Issue #207

kb1lqc · 2017-06-19T00:22:46Z

faraday/proxy.py

+    logger.info("Log initialization complete")
+
+
+def saveProxyLog(name, config):


May not work without pip install creating ~user/.faraday/lib

Not applicable, folder created if not already existing whenever proxy is run:

https://github.com/FaradayRF/Faraday-Software/pull/203/files#diff-a9d2cba55827323291e9d0c49fd6f720R629

kb1lqc · 2017-06-19T00:23:07Z

faraday/proxy.py

+    sys.exit(0)
+
+
+def showProxyLogs():


May not work without pip install creating ~user/.faraday/lib

Not applicable, folder created if not already existing whenever proxy is run:

https://github.com/FaradayRF/Faraday-Software/pull/203/files#diff-a9d2cba55827323291e9d0c49fd6f720R629

kb1lqc · 2017-06-19T00:36:58Z

requirements.txt

@@ -1,4 +1,5 @@
 appdirs==1.4.3
+argparse==1.4.0


Shouldn't need this since python >2.6 has argparse in the stdlib

Also should possibly add setuptools?

Corrected erroneous comma instead of a plus sign for combining strings in a logging.debug() command. Also removed an extra an unneeded logging.debug() command.

kb1lqd

I sat down with Bryce and we went through all of the code updates. I verified that I could install, configure, and start proxy from the command line. I also verified that telemetry and and APRS also worked (also commanding local and remote) using traditional methods of manually running the github scripts.

It's part of stdlib in Python >2.6 so it's already there.

Needed to change it so that uploading to PyPi works. However it should be oding this by commit anywyas so might not be needed but hell... Let's start on 0.0.1010 with this PR. For funzies.

kb1lqc · 2017-06-19T02:14:54Z

Uploaded to PyPi servers! @el-iso @reillyeon @kb1lqd

TestPyPi: https://testpypi.python.org/pypi/faraday/0.0.1010.dev996
PyPi: https://pypi.python.org/pypi/faraday/0.0.1010.dev996

kb1lqc added 30 commits April 30, 2017 15:02

Added setup.py

dc61bf1

Initial add of setup.py for setuptools use when installing with pip.

Updated setup.py for Faraday

ac7fbc2

Added Faraday specific data into the setup.py file.

Updated setup.py to register with pypi

991dfcf

Removed test folder indicated in setup.py as well as changed the README to REAMDE.md in the same file.

Initial pypi files that sucessfully upload

27c4649

Still having 'pip install faraday' issues but it uploades to pypi!

Added requirements files

6562a0f

Specified requirements in setup.py

Removed bad Proxy folder from packages list

e2ea05f

Moved setup.py to setup.old.py for pbr

c0d1def

Will move over to pbr and this requires a simplified setup.py.

Minimal pbr setup.py

989bd4a

Initial Faraday pbr configuration

5f5cb97

pbr Based setup.cfg so far so good

b087fc9

Faraday-proxy appears to work. Now I need to fix the formatters ini filer error per @hdmike.

Fixed logging.ini import error

5c96094

Used @hdkmike suggested import code and the issue was fixed. Now I need to do the same for proxy.ini.

Proxy.ini now imports

1d11677

Successfully made it past the proxy.ini import. Now we need to get the sqlite schema file imported as it fails when running faraday-proxy.

WORKING FARADAY-PROXY AFTER PIP INSTALL!

872db0e

After installing with pip in editable mode I can use the command faraday-proxy to start up the proxy application correctly! I simply brute forced the relative file paths for the database and database schema files.

Initial faraday-telemetry command line working

7cb74e4

Imported faraday/telemetry.py directly as well as fixed up paths to configuration files and databases.

faraday-aprs working!

89ba681

Modified paths to configuration files in faraday/aprs.py as well as added correct lines to setup.cfg to properly create console entry points for faraday-aprs.

faraday-simpleui now works!

48ca3e0

Very simple update of logging file locations as well as setup.cfg to point to the correct entry points.

Updated setup.cfg for correct file import in pbr

f0196f1

Added README

3cfb0eb

Updated setup.cfg to fix data_file

7428534

Fixed data_file formatting and also forced a difference base version so I can upload to pypi without committing.

Added updates for config file relative paths.

404b02f

Moved proxy.sample.ini to main project folder

48323aa

Moved proxy.sample.ini into a new etc/ folder

c25bcb4

Moved proxy files to etc

9f2dba2

Updated proxy.py to use configuration files from etc/ folder.

Added test.txt to create etc/faraday folder

2b8f194

Moved /etc files into /etc/faraday

3d35f34

To ensure all files are in a common faraday folder.

Added updated proxy.py for /etc/faraday files

e230a4f

updated file paths to enable use of /etc/faraday/

Fixed PEP8 Errors Per pytest

326ee12

Ran pytest and fixed the PEP8 errors/warnings.

Removed unneeded and failing argument line 215 proxy.py

b6798b4

kb1lqc mentioned this pull request Jun 18, 2017

faraday-proxy Must Be Run From Git Directory When Installed in Editable Mode #206

Open

kb1lqc added 3 commits June 18, 2017 15:46

Remove Unecessary setuptools options

cfd406c

Not bothering making faraday-telemetry and other apps since we are only focusing on getting faraday-proxy to run correctly right now.

Update README with simple link to quickstart

8d72854

Keeping it simple for the README by linking to the quickstart tutorial which will be updated with correct installation instructions.

Changed proxy.ini path to logger

e6e382f

Performing a logging debug message of the proxy.ini path instead of a print.

kb1lqc requested a review from kb1lqd June 18, 2017 23:13

kb1lqc commented Jun 18, 2017

View reviewed changes

kb1lqc commented Jun 19, 2017

View reviewed changes

Removed unecessary path print and fixed logging

01bf0bb

Corrected erroneous comma instead of a plus sign for combining strings in a logging.debug() command. Also removed an extra an unneeded logging.debug() command.

kb1lqc mentioned this pull request Jun 19, 2017

faraday-proxy Command Line Configuration Doesn't Sanitize User Input #207

Open

kb1lqd approved these changes Jun 19, 2017

View reviewed changes

kb1lqc added 2 commits June 18, 2017 18:47

Removed unecesary argparse requirement

bf66c04

It's part of stdlib in Python >2.6 so it's already there.

Changed version in setup.cfg

13746c0

Needed to change it so that uploading to PyPi works. However it should be oding this by commit anywyas so might not be needed but hell... Let's start on 0.0.1010 with this PR. For funzies.

kb1lqc merged commit fcdadae into FaradayRF:master Jun 19, 2017

kb1lqc mentioned this pull request Jun 21, 2017

Use Python Setuptools to Install Faraday Software #191

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Faraday-Proxy PyPi Support #203

Faraday-Proxy PyPi Support #203

kb1lqc commented Jun 3, 2017 •

edited

Loading

kb1lqd commented Jun 4, 2017 •

edited

Loading

kb1lqd commented Jun 4, 2017 •

edited

Loading

el-iso commented Jun 4, 2017 via email

kb1lqc commented Jun 5, 2017

kb1lqc commented Jun 5, 2017

kb1lqd commented Jun 7, 2017 •

edited

Loading

kb1lqc commented Jun 8, 2017

kb1lqc commented Jun 8, 2017

kb1lqd commented Jun 18, 2017

kb1lqc commented Jun 18, 2017

kb1lqc Jun 18, 2017

kb1lqc Jun 18, 2017

kb1lqc Jun 18, 2017

kb1lqc Jun 19, 2017

kb1lqc Jun 19, 2017

kb1lqc Jun 19, 2017

kb1lqc Jun 19, 2017

kb1lqc Jun 19, 2017

kb1lqc Jun 19, 2017

kb1lqc Jun 19, 2017

kb1lqc Jun 19, 2017

kb1lqd left a comment

kb1lqc commented Jun 19, 2017

		logger.info("Log initialization complete")


		def saveProxyLog(name, config):

Faraday-Proxy PyPi Support #203

Faraday-Proxy PyPi Support #203

Conversation

kb1lqc commented Jun 3, 2017 • edited Loading

kb1lqd commented Jun 4, 2017 • edited Loading

kb1lqd commented Jun 4, 2017 • edited Loading

el-iso commented Jun 4, 2017 via email

kb1lqc commented Jun 5, 2017

kb1lqc commented Jun 5, 2017

kb1lqd commented Jun 7, 2017 • edited Loading

kb1lqc commented Jun 8, 2017

kb1lqc commented Jun 8, 2017

kb1lqd commented Jun 18, 2017

kb1lqc commented Jun 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kb1lqd left a comment

Choose a reason for hiding this comment

kb1lqc commented Jun 19, 2017

kb1lqc commented Jun 3, 2017 •

edited

Loading

kb1lqd commented Jun 4, 2017 •

edited

Loading

kb1lqd commented Jun 4, 2017 •

edited

Loading

kb1lqd commented Jun 7, 2017 •

edited

Loading