Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Python Setuptools to Install Faraday Software #191

Closed
kb1lqc opened this issue Apr 29, 2017 · 44 comments
Closed

Use Python Setuptools to Install Faraday Software #191

kb1lqc opened this issue Apr 29, 2017 · 44 comments
Assignees

Comments

@kb1lqc
Copy link
Member

kb1lqc commented Apr 29, 2017

Summary

Let's face it, installing and running Faraday software is a pain right now. Thanks to #178, #182, #184, #185, and #187 we can finally move towards using setuptools. The end result will be that Faraday Software can be installed with python setup.py <some_command> <options> and run by just invoking the name of the software. Additionally, released Faraday Software would be registered with Pypi allowing installation from pip instead of relying on GitHub (unless you want the latest version).

Problem Explanation

'nuff said.

Environment

Software

Latest Master version as of 4-29-2017 and the general Python environment.

Hardware

N/A

Supporting Information

setuptools
Official pypi distribution guide

@kb1lqc kb1lqc added this to the Alpha Software milestone Apr 29, 2017
@kb1lqc
Copy link
Member Author

kb1lqc commented Apr 30, 2017

Hey that's cool! haven't tested it from the repo but apparently I followed the pypi steps necessary to get Faraday into the package list! @kb1lqd @reillyeon

Actual Package (not playing well with markup readme.md): https://pypi.python.org/pypi/faraday/0.0.1a1

image

@kb1lqc
Copy link
Member Author

kb1lqc commented Apr 30, 2017

Yup work to do but neat to be able to even get a response running pip install faraday 🍾

image

@el-iso
Copy link
Contributor

el-iso commented Apr 30, 2017

@kb1lqc I got the same result. I'm glad you posted. I was trying to figure out what I did wrong! haha

@kb1lqc
Copy link
Member Author

kb1lqc commented May 2, 2017

Update! I used the pip --log parameter to generate a log file of my pip failure while using a virtualenv. Interesting. I have the following gist:

pipfaraday.txt

For some reason pypi is defaulting to V0.0.1 instead of the newer versions... possibly because it's the only tar.gz file?

Also I fell victim to this pip bug that I am not sure is actually solved...

@kb1lqc
Copy link
Member Author

kb1lqc commented May 2, 2017

D'oh I didn't see that pypi has a test server: https://wiki.python.org/moin/TestPyPI

That would have prevented a broken build from going into pypi... will start using that.

@kb1lqc
Copy link
Member Author

kb1lqc commented May 2, 2017

Alright I used the testpypi server and was able to upload correct files to the test pip and install them!

image

I believe what happened was I created new wheel files without making new sdist files and when uploaded to the pypi server would not be seen as a complete version.

running faraday or faraday.proxy still does not work, will look into it. I suspect I'm just missing files. This is what is in the tar.gz source code folder:

image

image

@kb1lqc
Copy link
Member Author

kb1lqc commented May 2, 2017

Cool good lessons learned here. I jumped the gun and worked with pypi prior to using pip install -e . or editable mode to actually test these commands and folder structure.

@kb1lqc
Copy link
Member Author

kb1lqc commented May 2, 2017

@reillyeon I am making progress. In my latest setup.py I have placed the required packages into the setup file as I can't figure out if the requirements.txt is sufficient. I do not think so. I can confirm this installs faraday and all dependencies when running pip install -e . in the direction with setup.py in it.

However, I cannot run any command line faraday, faraday.proxy, or proxy commands. I do see some progress when running python I can:

Import Faraday

import faraday (imports)

From Faraday Import Proxy

from faraday import proxy

The following error:

>>> from faraday import proxy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "faraday\proxy.py", line 30, in <module>
    logging.config.fileConfig(filename)
  File "C:\python27\Lib\logging\config.py", line 77, in fileConfig
    formatters = _create_formatters(cp)
  File "C:\python27\Lib\logging\config.py", line 113, in _create_formatters
    flist = cp.get("formatters", "keys")
  File "C:\python27\Lib\ConfigParser.py", line 607, in get
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'formatters'

The no section formatters essentially means that proxy.py is being run but the Proxy/logging.ini folder relative to it does not exist. So some progress but our goal is to run faraday.proxy from the command line after a pip install...

I am not sure this is setup correctly. I don't understand at the moment how I am to get pip to recognize a command line version of faraday.proxy. Thoughts?

@hdkmike
Copy link

hdkmike commented May 2, 2017

@kb1lqc , where is "loggingConfig.ini" actually supposed to live?

@hdkmike
Copy link

hdkmike commented May 3, 2017

I did a quick check in Linux to see if I could resolve your config file path issue:

  • Have the filename in faraday/proxy.py be a relative path based on the Proxy module
  • Modification of setup.py to use find_modules for faraday and submodules as well as Proxy
  • Modification of setup.py to include package data (i.e. non-python files) for Proxy

I'm not sure if these are the "proper" ways to do this but they seem not too hackish and I was able to get from faraday import proxy to work.

diff --git a/faraday/proxy.py b/faraday/proxy.py
index 81c9c4e..c8af3ee 100644
--- a/faraday/proxy.py
+++ b/faraday/proxy.py
@@ -26,7 +26,8 @@ from flask import request
 from faraday.uart import layer_4_service

 # Start logging after importing modules
-filename = os.path.abspath("loggingConfig.ini")
+#filename = os.path.abspath("loggingConfig.ini")
+filename = os.path.join(os.path.dirname(__file__), '..', 'Proxy', 'loggingConfig.ini')
 logging.config.fileConfig(filename)
 logger = logging.getLogger('Proxy')

diff --git a/setup.py b/setup.py
index 960bd6f..bb4dc88 100644
--- a/setup.py
+++ b/setup.py
@@ -22,7 +22,9 @@ setuptools.setup(
     license = "GPLv3",
     keywords = "faraday radio faradayrf ham amateur",
     url = "https://github.com/FaradayRF/Faraday-Software",
-    packages = ['faraday','proxy', 'Proxy'],
+    #packages = ['faraday', 'Proxy'],
+    packages = setuptools.find_packages(include=['faraday', 'faraday.*']) + ['Proxy'],
+    package_data = {'Proxy': '*'},
     long_description = "FaradayRF software",
     classifiers = [
         "Development Status :: 2 - Pre-Alpha",
@@ -64,4 +66,4 @@ setuptools.setup(
                         'requests',
                         'six',
                         'Werkzeug'],
-)
\ No newline at end of file
+)

@hdkmike
Copy link

hdkmike commented May 3, 2017

After our phone conversation, it seems like you want to pip install all of the libraries and applications necessary for Faraday to run and we're well on track for that. I'm looking into a reference on how to get the wrapper applications installed in the PATH that use the libraries in the PYTHONPATH. I'm still not sure on the intended location of the configuration and database files. Possibly command line parameters with a "default" config that's included in the install directory?

@hdkmike
Copy link

hdkmike commented May 3, 2017

If you do get wrapper applications into the PATH, consider manpages when installed on Linux.

@hdkmike
Copy link

hdkmike commented May 3, 2017

Looks like you want to use Scripts to get modules into the PATH: https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation with pip

Apparently, this even works with Windows by creating EXEs automatically.

@hdkmike
Copy link

hdkmike commented May 3, 2017

The application that I've been looking at is python-swiftclient of the OpenStack project. When you pip install it, it puts an executable in the PATH but it also puts the Python libraries in the PYTHONPATH.

@kb1lqc
Copy link
Member Author

kb1lqc commented May 3, 2017

Awesome, I started using pbr and it's working wonderfully. Thanks @hdkmike ! Also your suggestion @reillyeon on faraday-proxy worked well too. I am currently importing both logging.ini and proxy.ini. Now the error I get is when trying to setup the sqlite3 database with db.sql schema file:

$ faraday-proxy
2017-05-02 21:38:14,150 - Proxy - INFO - Starting proxy server
Traceback (most recent call last):
  File "C:\Users\Bryce\Documents\GitHub\faradayrf-kb1lqc\testfaraday\Scripts\faraday-proxy-script.py", line 11, in <module>
    load_entry_point('faraday', 'console_scripts', 'faraday-proxy')()
  File "c:\users\bryce\documents\github\faradayrf-kb1lqc\faraday\proxy.py", line 579, in main
    initDB()  # Initialize database for logging
  File "c:\users\bryce\documents\github\faradayrf-kb1lqc\faraday\proxy.py", line 447, in initDB
    with open(dbSchema, 'rt') as f:
IOError: [Errno 2] No such file or directory: 'db.sql'

@kb1lqc
Copy link
Member Author

kb1lqc commented May 3, 2017

WOOHOO! @hdkmike @reillyeon @kb1lqd !

$ faraday-proxy
2017-05-02 21:45:28,519 - Proxy - INFO - Starting proxy server
2017-05-02 21:45:30,275 - Proxy - INFO - Starting Thread For Unit: KB1LQC-2
2017-05-02 21:45:30,276 - Proxy - INFO - Starting uart_worker thread

image

@kb1lqc
Copy link
Member Author

kb1lqc commented May 3, 2017

Onto telemetry. Word of warning. For some reasong the logging.ini code in faraday/telemetry.py doesn't immediately fail. I let it run for a while with no output which was confusing. It eventually spat out:

image

This clued me into needing to fix up the importing of logging.ini. Glad I let it sit for a while...

@kb1lqc
Copy link
Member Author

kb1lqc commented May 3, 2017

Got faraday-telemetry!

image

The only really wierd part is where the telemetry.db log is stored. Currently it is still in Applications/Telemetry/telemetry.db. We may want to consider where we put Telemetry databases in the future. Readily goes with where/how we do config files for people who don't already have configs like I do...

image

@kb1lqc
Copy link
Member Author

kb1lqc commented May 3, 2017

faraday-aprs working!

image

Similar comments as that for Telemetry. @reillyeon we are no longer using the Applications/APRS/aprs.py files anymore with this method. Not sure we even need them now...

@reillyeon
Copy link
Collaborator

Once we update the documentation we should remove them.

@kb1lqc
Copy link
Member Author

kb1lqc commented May 3, 2017

Agreed, just a heads up. The PR for #191 will absolutely break lots of documentation including the tutorials/start. Going to be a big 📌

@kb1lqc
Copy link
Member Author

kb1lqc commented May 3, 2017

faraday-simpleui now works too! As does faraday-proxy, faraday-telemetry, and faraday-aprs. Check this out. This is the current method I use to start all the necessary components for viewing telemetry with simpleui:

  • faraday-proxy
  • faraday-telemetry
  • faraday-simpleui

I place faraday-proxy on a different screen since faraday-telemetry causes output every second indicating an empty queue (nominal). Overall, lots of work left to do but it's looking real now 🍾 !

@kb1lqd @reillyeon @hdkmike

simpleui-pipinitial

@kb1lqc
Copy link
Member Author

kb1lqc commented May 3, 2017

OK if you are interested you can for the time being (it's periodically cleaned) install Faraday from the testpypi server:

pip install -i https://testpypi.python.org/pypi faraday

This will install the packaged version of kb1lqc@3cfb0eb

However the following is not including and should be updated:

  • No proxy.ini in pip/python package. I tried manually creating proxy.ini in testfaraday/Proxy (virtualenv)
    image

  • Telemetry, SimpleUI, and APRS will also suffer from not having their *.ini files

When I run faraday-proxy from this testpypi install I get the following error:

(testfaraday) C:\Users\Bryce\Documents\GitHub>faraday-proxy
Traceback (most recent call last):
  File "C:\python27\Lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\python27\Lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Users\Bryce\Documents\GitHub\faradayrf-kb1lqc\testfaraday\Scripts\faraday-proxy.exe\__main__.py", line 5, in <module>
  File "c:\users\bryce\documents\github\faradayrf-kb1lqc\testfaraday\lib\site-packages\faraday\proxy.py", line 30, in <module>
    logging.config.fileConfig(filename)
  File "C:\python27\Lib\logging\config.py", line 77, in fileConfig
    formatters = _create_formatters(cp)
  File "C:\python27\Lib\logging\config.py", line 113, in _create_formatters
    flist = cp.get("formatters", "keys")
  File "C:\python27\Lib\ConfigParser.py", line 607, in get
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'formatters'

It appears as though I'm having errors importing loggingConfig.ini only when I used the package. This doesn't happen when I run from a pip install -e .

Thoughts on how to proceed? Progress!

@hdkmike
Copy link

hdkmike commented May 3, 2017

@kb1lqc , nice job! You'll have to make some decisions on where the *.ini config files live and the DB files. Typically, you don't want them (other than the absolute default/samples) living in the package.

Some options/suggestions (I haven't specifically looked at which options are available as CLI args):

  • You could make a subcommand to your console_scripts that take the packaged sample file and creates an initial config file named the default way or customizable. This way, a user can easily create a config file without hunting and copy pasting
    • For example, faraday-proxy createconfig that copies the packaged proxy.sample.ini file into the current directory and renames it proxy.ini
    • You can also make it arg driven and/or prompt driven to insert required parameters such as callsign on generation
  • You could make the console scripts search for their files the current directory which may or may not be a good idea.
    • You can consider making config files part of the home directory structure like a .faraday folder with all the different configs there (and then whatever generically can be implemented to support OSX and Windows)
    • Alternatively, if there are not a lot of config files, you can explicitly specify them on the command line such as faraday-proxy --config /path/to/file/proxy.ini

@hdkmike
Copy link

hdkmike commented May 3, 2017

As for missing files, pip really only packages up *.py files unless you specify package_data options in setup.py or, in the case of pbr, data_files in setup.cfg. With pip install -e ., the -e just creates a symlink to wherever . is so it just uses the .ini files in your directory. I've been testing with a completely different directory and virtualenv to avoid references or paths to files that don't come from the pip install.

Also, tried to install from the test server and got:

Collecting faraday
  Downloading https://testpypi.python.org/packages/c0/36/23974ceb440e65c0c053643114d5326d37a8b8bf11ea4428342679d2c52b/faraday-0.0.1.dev966-py2-none-any.whl (594kB)
    100% |████████████████████████████████| 604kB 1.1MB/s
Collecting pyserial==3.2.1 (from faraday)
  Could not find a version that satisfies the requirement pyserial==3.2.1 (from faraday) (from versions: )
No matching distribution found for pyserial==3.2.1 (from faraday)

Definitely progress! 👍

@kb1lqc
Copy link
Member Author

kb1lqc commented May 4, 2017

@hdkmike and @reillyeon I found this very useful for packaging data and configuration files. Pretty sure it's along the lines of what we need for including and excluding files with setuptools.

https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files

@kb1lqc
Copy link
Member Author

kb1lqc commented May 4, 2017

I'm going to give this a little more thought. I may not get to it tonight. Your opinion would be much appreciated. Here's what I'm thinking for the faraday package:

SetupTools/pbr

Included Data Files

  • All relevant *.py
  • *.sample.ini configuration files
  • *.sql SQLite3 schema files
  • Documentation *.md markdown files (necessary?)
  • Images for documentation (necessary?)
  • Necessary HTML and Javascript files/packages (JQuery, Bootstrap, etc for faraday-simpleui)
  • AUTHORS & ChangeLog files from setuptools/pbr

Excluded Data Files

  • Any *.db files as they are automatically generated on startup

Configuration

I think we should consider using a command such as faraday-config, possibly taking arguments for which applications to install, copying the *.sample.ini files into a common locations such as in the egg itself (package_resources) or a .faradayrc files in some predefined location suitable for Linux, OS X, and Windows. A local folder could also work such that my workflow is mkdir faraday -> cd faraday -> faraday-config which installs configuration data to that directory.

However, I'd also consider moving all configuration to a common file, one big faraday.ini. This could also be moved over upon command but may be easier to code since you would know exactly what to look for (no need to understand which Applications are currently installed). Then a command line and/or web interface can be launched allowing one to configure Faraday details.

  • This should be persistent between updates/installations of faraday packages. Possibly allowing user to choose upon uninstalling if they want to keep that file or simply just always keep it
  • I really want to make configuration easier, supporting both command line and future ability by web/ui is highly encouraged.

Thoughts

I think this is achievable though I am a tad overwhelmed. setuptools and pbr seem to be able to do this but it's not intuitive. What do you think?

@reillyeon
Copy link
Collaborator

Let's check in the code that just sets up the commands assuming that you are sitting in the right directories in the source tree. That is, finding config and database in the current directory as we currently do. Then we can follow up with packaging data files. One thing at a time and it sounds like getting this to work was hard enough.

@kb1lqc
Copy link
Member Author

kb1lqc commented May 4, 2017

@reillyeon and @hdkmike I uploaded the latest commit to issue191 branch. Additionally, I created a wheel and source distribution package here:

https://pypi.python.org/pypi/faraday/0.0.1001.dev966

You should be able to get this with a simple pip install faraday. However I still get the formatters error even if I'm operating this inside a virtualenvironment within a freshly cloned folder of Faraday-Software master.

(testfaraday) C:\Users\Bryce\Documents\GitHub\faradayrf-kb1lqc>faraday-proxy
Traceback (most recent call last):
  File "C:\python27\Lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\python27\Lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Users\Bryce\Documents\GitHub\faradayrf-kb1lqc\testfaraday\Scripts\faraday-proxy.exe\__main__.py", line 5, in <module>
  File "c:\users\bryce\documents\github\faradayrf-kb1lqc\testfaraday\lib\site-packages\faraday\proxy.py", line 30, in <module>
    logging.config.fileConfig(filename)
  File "C:\python27\Lib\logging\config.py", line 77, in fileConfig
    formatters = _create_formatters(cp)
  File "C:\python27\Lib\logging\config.py", line 113, in _create_formatters
    flist = cp.get("formatters", "keys")
  File "C:\python27\Lib\ConfigParser.py", line 607, in get
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'formatters'

I'm starting to think my importing of the INI files and stuff isn't working right with pbr? https://docs.openstack.org/developer/pbr/#files How do I tell if my source and wheel are going to work?

@kb1lqc
Copy link
Member Author

kb1lqc commented May 4, 2017

OK I tested the latest version (https://pypi.python.org/pypi/faraday/0.0.1002.dev967) and it now simply prints a ERROR with the loggingConfig.ini filename to the screen.

# Start logging after importing modules
filename = os.path.join(os.path.dirname(__file__), '..', 'Proxy', 'loggingConfig.ini')
logging.error(filename)
logging.config.fileConfig(filename)
logger = logging.getLogger('Proxy')

Running faraday-proxy from:
C:\Users\Bryce\Documents\GitHub\faradayrf-kb1lqc

With a virtualenvironment in faradayrf-kb1lqc calls testfaraday you can see the following output from faraday-proxy

ERROR:root:c:\users\bryce\documents\github\faradayrf-kb1lqc\testfaraday\lib\site-packages\faraday\..\Proxy\loggingConfig.ini

It's trying to go up ..\Proxy\loggingConfig.ini which will fail because my site-packages doesn't contain a Proxy folder. This means that none of my pbr files directives seem to be working or at least we're pointing to the wrong place. I am not sure where to look for those files. Thoughts?

AHHH... They are a few folders up.. this is wierd
image

proxy.py running from command line executed from the location \testfaraday\Lib\site-packages
which means that this could work if I increased the folder move to:

../../../Proxy

I'll try it but this doesn't feel right 😟

@kb1lqc
Copy link
Member Author

kb1lqc commented May 4, 2017

Yes this did it. the Proxy folder is a few folders up... (faraday 0.0.1004.dev967)

image

When run in a brand new Git clone in a different virtual environment I get this error as expected since I don't have a proxy.ini (only proxy.sample.ini)

image

@kb1lqc
Copy link
Member Author

kb1lqc commented May 4, 2017

Yes, in my faradayenv environements (a different one, the windows command prompt above) I have the following freshly pip install faraday Proxy folder contents
image

Manually placing proxy.ini inside:
image

Doesnt do much because apparently it's looking for PROXY section inside loggingConfig.ini which is weird.

image

Ok figuring this one out another night.

@kb1lqc
Copy link
Member Author

kb1lqc commented May 5, 2017

Really struggling to see how I can package these configuration files anywhere other than the current locations which is causing issues. The following suggests keeping them inside the egg but fails to mention how to get them into it in the first place.

http://peak.telecommunity.com/DevCenter/setuptools#non-package-data-files

@kb1lqc
Copy link
Member Author

kb1lqc commented May 5, 2017

So @reillyeon and @hdkmike why not include the configuration files in a proxy folder inside our faraday package folder. It appears to be suggested:

https://docs.python.org/3/distutils/setupscript.html#setup-script

this would cause the following structure in our software:

faraday

  • Proxy
    -- proxy.ini
    -- loggingConfig.ini

@kb1lqc
Copy link
Member Author

kb1lqc commented May 10, 2017

@reillyeon @hdkmike I noticed this lovely tidbit:

With a known working repo (I can python Proxy/proxy.py from cmd)
image

however if I install from the test repo a distribution of Faraday based on the same code:
image

This isn't failing in faraday/proxy.py exactly... it's a mix up when importing from faraday import proxy from Proxy/proxy.py. I can recreate it if if pop open command line python and manually from faraday import proxy which will return the same formatters error.

Thoughts?

@el-iso
Copy link
Contributor

el-iso commented May 11, 2017

Is it possible that installing the code as a python module causes the path to "loggingConfig.ini" to be incorrect? I tested it and it looks like os.path.abspath("loggingConfig.ini") will create a path even if it isn't valid. So, when it tells you there's no "formatters" section, maybe it can't find config file at all? If "formatters" is the first section it looks for, this would make sense. I couldn't find any information on what section it looks for first though.

Just an idea....

@kb1lqc
Copy link
Member Author

kb1lqc commented May 11, 2017

Progress!

After a great #Slack discussion with @reillyeon and @hdkmike the three of us realized what was going on and figure out a path forward. I have implemented a very brute force method of this which confirms that we are on the correct path albeit I didn't optimize it. Thanks @el-iso for the suggestion, you'll see the answer below.

I Was Looking At Paths Incorrectly

I completely missed the meaning of this part of packaging data files. I thought I knew it but didn't. The sys.prefix location being the path always installed to was missed. This resulted in (Windows) a installed python package always moving a data_file to C:\python27\. If it was a virtual environment then it was relative to that.

This causes the installation of pip install -e . to look in completely different places than a pip install faraday to look. The editable install will not move any files. Therefore we must check several locations.

Checking For Config Files

I probably implemented this in the poorest way but it works. As @hdkmike would resoundingly disagree with me "if it compiles, ship it". Just kidding. Anyways.

The code below is a bit of a brute force hack I placed in faraday/proxy.py. I define several paths where the loggingConfig.ini file could be. These are essentially relative paths to an etc/faraday folder locally, the sys.prefix path to a /etc/faraday folder and finally the userspace (i.e. C:\User\Bryce\) with a hidden folder .faraday. While not predefined I also check the local folder the code is run in too.

The loop iterates through each of these paths and breaks out once a file is successfully opened. If this occurs I save the path to a path variable that I can easily use later.

relpath1 = os.path.join('etc', 'faraday')
relpath2 = os.path.join('..', 'etc', 'faraday')
setuppath = os.path.join(sys.prefix, 'etc', 'faraday')
userpath = os.path.expanduser('~/.faraday')
path = ''

for location in os.curdir, relpath1, relpath2, setuppath, userpath:
    try:
        print location
        logging.config.fileConfig(os.path.join(location, "loggingConfig.ini"))
        path = location
        break
    except ConfigParser.NoSectionError:
        pass

This is also repeated in a somewhat similar way for the SQLite3 database files though in a different location of proxy.py

Current Folder Structure

It's a bit messy but I save the config files to:
sys.prefix/ + etc/faraday

All log files are saved to:
<userhome>/ + .faraday/lib

Summary

Progress! Finally! I'll explain in my next comment how to run it. This can absolutely be optimized and better laid out.

@kb1lqc
Copy link
Member Author

kb1lqc commented May 11, 2017

Test PyPi Package

I updated the latest code with a working proxy application to faraday 0.0.1004.dev978 on the pypi test server

To install it simply run the following code:

pip install -i https://testpypi.python.org/pypi faraday

You should see faraday 0.0.1004.dev978 install at the end of all the requirements installation. If you get an error such as a package version can't be found, just install that package with the normal pip server

When all is set and done your sys.prefix folder will have some new additions, on windows this is C:\python27 which will now have a C:\python27\etc folder if you didn't already have one.

Inside there will be a faraday folder:

image

Inside that folder will be our initial configuration files:
image

Configuring Faraday

This is a bit annoying but it's getting there. We can totally automate this out later. We will need to copy proxy.sample.ini into proxy.ini manually and edit it's contents to what Proxy normally expects.

image

Running Faraday

Simply running from command line faraday-proxy will bring up the Faraday Proxy application.

image

As you can see above the file paths print out until success (debug code) so I found my configs at C:\python27\etc\faraday as expected. The paths prior failed because relative to where I ran the code ~/Documents/GitHub the configuration files were not found.

Looking at my user home space I now have a .faraday folder with another lib folder inside.

image

image

The log above is saved to when logging mode is enabled. If you are running a testmode log then it will be relative to this folder be careful setting the name in proxy.ini!

Final test

Yes it works, here's my flask server response!

image

@reillyeon and @hdkmike I'd like to clean up this code. I know I didn't use a config.py file but now that I now this works I'll gladly move over. Do you see cleaner ways to implement this?

https://github.com/kb1lqc/Faraday-Software/tree/issue191

@kb1lqd

@kb1lqc
Copy link
Member Author

kb1lqc commented May 27, 2017

Installing on my desktop I've noticed the following problems:

  • proxy.ini was not found, however, it appears that loggingConfig.ini was since that's called prior. Could be specific to the code looking for proxy.ini

'''
Bryce@Bryce-PC MINGW64 ~/Documents/GitHub/kb1lqc-software (issue191)
$ faraday-proxy.exe
.
etc\faraday
2017-05-27 14:33:22,141 - Proxy - ERROR - ConfigParse.Error: No section: 'PROXY'
'''

@kb1lqc
Copy link
Member Author

kb1lqc commented May 29, 2017

Getting there @reillyeon, @kb1lqd , and @hdkmike ! I'll update documentation soon too but proxy is running from a pip install (local editable) with command line configuration. If you change configuration it is changed in the proxy.ini file. Also, running faraday-proxy --init-config will copy proxy.sample.ini into proxy.ini which removes the need to do any file copy pasting manually.

image

Test PyPi: https://testpypi.python.org/pypi/faraday/0.0.1009.dev988
Local Branch: https://github.com/kb1lqc/Faraday-Software/tree/issue191

I'll be testing the purely pypi installed version soon. Then I may need to consider moving on from proxy or buttoning up a Proxy only PR for python packaging. Thoughts?

@kb1lqc
Copy link
Member Author

kb1lqc commented Jun 21, 2017

Alright! with #203 pulled in the next step is getting Telemetry into the package. This should be much more straightforward with the foundation that #203 laid. Will start doing this in the next day or so.

@kb1lqc
Copy link
Member Author

kb1lqc commented Jun 22, 2017

PR #211 now covers the telemetry application!

@kb1lqc
Copy link
Member Author

kb1lqc commented Jun 29, 2017

Argh! Figured out why device configuration isn't working if I try to use a form of simpleconfig implemented as a function in the deviceconfiguration.py program... The argument ends up being parsed and acted upon before the Flask server is up. This explains why it fails for me but if I then use the external simpleconfig program it works just fine (the server has started up at this point).

Honestly @kb1lqd device configuration almost belongs inside proxy since it's interfacing the hardware. I'll go ahead and make the awkward device configuration server + simpleconfig PR for now but we should consider improving that workflow. Does this sound good to you?

@kb1lqc
Copy link
Member Author

kb1lqc commented Jul 4, 2017

This is closed for all currently supported applications as of PR #222. Bugs and documentation updates as needed will be address with future issue tickets. Thanks @reillyeon @el-iso @kb1lqd !

@kb1lqc kb1lqc closed this as completed Jul 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants