-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use Python Setuptools to Install Faraday Software #191
Comments
Hey that's cool! haven't tested it from the repo but apparently I followed the pypi steps necessary to get Faraday into the package list! @kb1lqd @reillyeon Actual Package (not playing well with markup readme.md): https://pypi.python.org/pypi/faraday/0.0.1a1 |
@kb1lqc I got the same result. I'm glad you posted. I was trying to figure out what I did wrong! haha |
Update! I used the For some reason pypi is defaulting to V0.0.1 instead of the newer versions... possibly because it's the only tar.gz file? Also I fell victim to this pip bug that I am not sure is actually solved... |
D'oh I didn't see that pypi has a test server: https://wiki.python.org/moin/TestPyPI That would have prevented a broken build from going into pypi... will start using that. |
Alright I used the testpypi server and was able to upload correct files to the test pip and install them! I believe what happened was I created new wheel files without making new sdist files and when uploaded to the pypi server would not be seen as a complete version. running |
Cool good lessons learned here. I jumped the gun and worked with pypi prior to using |
@reillyeon I am making progress. In my latest setup.py I have placed the required packages into the setup file as I can't figure out if the requirements.txt is sufficient. I do not think so. I can confirm this installs However, I cannot run any command line Import Faraday
From Faraday Import Proxy
The following error:
The no section formatters essentially means that I am not sure this is setup correctly. I don't understand at the moment how I am to get pip to recognize a command line version of |
@kb1lqc , where is "loggingConfig.ini" actually supposed to live? |
I did a quick check in Linux to see if I could resolve your config file path issue:
I'm not sure if these are the "proper" ways to do this but they seem not too hackish and I was able to get diff --git a/faraday/proxy.py b/faraday/proxy.py
index 81c9c4e..c8af3ee 100644
--- a/faraday/proxy.py
+++ b/faraday/proxy.py
@@ -26,7 +26,8 @@ from flask import request
from faraday.uart import layer_4_service
# Start logging after importing modules
-filename = os.path.abspath("loggingConfig.ini")
+#filename = os.path.abspath("loggingConfig.ini")
+filename = os.path.join(os.path.dirname(__file__), '..', 'Proxy', 'loggingConfig.ini')
logging.config.fileConfig(filename)
logger = logging.getLogger('Proxy')
diff --git a/setup.py b/setup.py
index 960bd6f..bb4dc88 100644
--- a/setup.py
+++ b/setup.py
@@ -22,7 +22,9 @@ setuptools.setup(
license = "GPLv3",
keywords = "faraday radio faradayrf ham amateur",
url = "https://github.com/FaradayRF/Faraday-Software",
- packages = ['faraday','proxy', 'Proxy'],
+ #packages = ['faraday', 'Proxy'],
+ packages = setuptools.find_packages(include=['faraday', 'faraday.*']) + ['Proxy'],
+ package_data = {'Proxy': '*'},
long_description = "FaradayRF software",
classifiers = [
"Development Status :: 2 - Pre-Alpha",
@@ -64,4 +66,4 @@ setuptools.setup(
'requests',
'six',
'Werkzeug'],
-)
\ No newline at end of file
+) |
After our phone conversation, it seems like you want to pip install all of the libraries and applications necessary for Faraday to run and we're well on track for that. I'm looking into a reference on how to get the wrapper applications installed in the PATH that use the libraries in the PYTHONPATH. I'm still not sure on the intended location of the configuration and database files. Possibly command line parameters with a "default" config that's included in the install directory? |
If you do get wrapper applications into the PATH, consider manpages when installed on Linux. |
Looks like you want to use Scripts to get modules into the PATH: https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation with pip Apparently, this even works with Windows by creating EXEs automatically. |
The application that I've been looking at is
|
Awesome, I started using
|
WOOHOO! @hdkmike @reillyeon @kb1lqd !
|
Onto telemetry. Word of warning. For some reasong the This clued me into needing to fix up the importing of |
Got The only really wierd part is where the |
Similar comments as that for Telemetry. @reillyeon we are no longer using the |
Once we update the documentation we should remove them. |
Agreed, just a heads up. The PR for #191 will absolutely break lots of documentation including the |
I place |
OK if you are interested you can for the time being (it's periodically cleaned) install Faraday from the testpypi server:
This will install the packaged version of kb1lqc@3cfb0eb However the following is not including and should be updated:
When I run
It appears as though I'm having errors importing loggingConfig.ini only when I used the package. This doesn't happen when I run from a Thoughts on how to proceed? Progress! |
@kb1lqc , nice job! You'll have to make some decisions on where the Some options/suggestions (I haven't specifically looked at which options are available as CLI args):
|
As for missing files, Also, tried to install from the test server and got:
Definitely progress! 👍 |
@hdkmike and @reillyeon I found this very useful for packaging data and configuration files. Pretty sure it's along the lines of what we need for including and excluding files with setuptools. https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files |
I'm going to give this a little more thought. I may not get to it tonight. Your opinion would be much appreciated. Here's what I'm thinking for the SetupTools/pbrIncluded Data Files
Excluded Data Files
ConfigurationI think we should consider using a command such as However, I'd also consider moving all configuration to a common file, one big
ThoughtsI think this is achievable though I am a tad overwhelmed. |
Let's check in the code that just sets up the commands assuming that you are sitting in the right directories in the source tree. That is, finding config and database in the current directory as we currently do. Then we can follow up with packaging data files. One thing at a time and it sounds like getting this to work was hard enough. |
@reillyeon and @hdkmike I uploaded the latest commit to https://pypi.python.org/pypi/faraday/0.0.1001.dev966 You should be able to get this with a simple
I'm starting to think my importing of the INI files and stuff isn't working right with |
OK I tested the latest version (https://pypi.python.org/pypi/faraday/0.0.1002.dev967) and it now simply prints a ERROR with the
Running With a virtualenvironment in faradayrf-kb1lqc calls
It's trying to go up ..\Proxy\loggingConfig.ini which will fail because my AHHH... They are a few folders up.. this is wierd
I'll try it but this doesn't feel right 😟 |
Really struggling to see how I can package these configuration files anywhere other than the current locations which is causing issues. The following suggests keeping them inside the egg but fails to mention how to get them into it in the first place. http://peak.telecommunity.com/DevCenter/setuptools#non-package-data-files |
So @reillyeon and @hdkmike why not include the configuration files in a https://docs.python.org/3/distutils/setupscript.html#setup-script this would cause the following structure in our software: faraday
|
@reillyeon @hdkmike I noticed this lovely tidbit: With a known working repo (I can however if I install from the test repo a distribution of Faraday based on the same code: This isn't failing in Thoughts? |
Is it possible that installing the code as a python module causes the path to "loggingConfig.ini" to be incorrect? I tested it and it looks like os.path.abspath("loggingConfig.ini") will create a path even if it isn't valid. So, when it tells you there's no "formatters" section, maybe it can't find config file at all? If "formatters" is the first section it looks for, this would make sense. I couldn't find any information on what section it looks for first though. Just an idea.... |
Progress!After a great #Slack discussion with @reillyeon and @hdkmike the three of us realized what was going on and figure out a path forward. I have implemented a very brute force method of this which confirms that we are on the correct path albeit I didn't optimize it. Thanks @el-iso for the suggestion, you'll see the answer below. I Was Looking At Paths IncorrectlyI completely missed the meaning of this part of packaging data files. I thought I knew it but didn't. The This causes the installation of Checking For Config FilesI probably implemented this in the poorest way but it works. As @hdkmike would resoundingly disagree with me "if it compiles, ship it". Just kidding. Anyways. The code below is a bit of a brute force hack I placed in The loop iterates through each of these paths and breaks out once a file is successfully opened. If this occurs I save the path to a
This is also repeated in a somewhat similar way for the SQLite3 database files though in a different location of Current Folder StructureIt's a bit messy but I save the config files to: All log files are saved to: SummaryProgress! Finally! I'll explain in my next comment how to run it. This can absolutely be optimized and better laid out. |
Test PyPi PackageI updated the latest code with a working To install it simply run the following code:
You should see faraday 0.0.1004.dev978 install at the end of all the requirements installation. If you get an error such as a package version can't be found, just install that package with the normal pip server When all is set and done your Inside there will be a Inside that folder will be our initial configuration files: Configuring FaradayThis is a bit annoying but it's getting there. We can totally automate this out later. We will need to copy Running FaradaySimply running from command line As you can see above the file paths print out until success (debug code) so I found my configs at Looking at my user home space I now have a The log above is saved to when Final testYes it works, here's my flask server response! @reillyeon and @hdkmike I'd like to clean up this code. I know I didn't use a |
Installing on my desktop I've noticed the following problems:
''' |
Getting there @reillyeon, @kb1lqd , and @hdkmike ! I'll update documentation soon too but proxy is running from a pip install (local editable) with command line configuration. If you change configuration it is changed in the proxy.ini file. Also, running Test PyPi: https://testpypi.python.org/pypi/faraday/0.0.1009.dev988 I'll be testing the purely pypi installed version soon. Then I may need to consider moving on from proxy or buttoning up a Proxy only PR for python packaging. Thoughts? |
PR #211 now covers the telemetry application! |
Argh! Figured out why device configuration isn't working if I try to use a form of simpleconfig implemented as a function in the deviceconfiguration.py program... The argument ends up being parsed and acted upon before the Flask server is up. This explains why it fails for me but if I then use the external simpleconfig program it works just fine (the server has started up at this point). Honestly @kb1lqd device configuration almost belongs inside proxy since it's interfacing the hardware. I'll go ahead and make the awkward device configuration server + simpleconfig PR for now but we should consider improving that workflow. Does this sound good to you? |
This is closed for all currently supported applications as of PR #222. Bugs and documentation updates as needed will be address with future issue tickets. Thanks @reillyeon @el-iso @kb1lqd ! |
Summary
Let's face it, installing and running Faraday software is a pain right now. Thanks to #178, #182, #184, #185, and #187 we can finally move towards using
setuptools
. The end result will be that Faraday Software can be installed withpython setup.py <some_command> <options>
and run by just invoking the name of the software. Additionally, released Faraday Software would be registered with Pypi allowing installation from pip instead of relying on GitHub (unless you want the latest version).Problem Explanation
'nuff said.
Environment
Software
Latest Master version as of 4-29-2017 and the general Python environment.
Hardware
N/A
Supporting Information
setuptools
Official pypi distribution guide
The text was updated successfully, but these errors were encountered: