re-factor to make it easy to deploy and support other web technologies.

[kern3020]

Hello Mike,

I was looking at the d3py python module. At the moment, it is tightly coupled with python’s SimpleHTTPServer. From a desktop, this works well interactively.

Have you considered a Model/View split? This would provide a good way to deploy d3. Figure and its subclasses would make fine models, but all code which is specific to a web server would be refactored into a class which knows about the Simple HTTP Server. Another class would know how to write to the file system. It might look something like:

// encapsulates all code related to the SimpleHTTPServer implementation.
vHttp = createHTTP(“www.example.com”, 8080)
// creates a class which knows one to write the d3 code to the filesystem for deployment.
vFs = createFS(“~/workspace/opportunity/static”)

// This call is just a placeholder. It will take fewer args as web specific args
// would be passed to creatHTTP().
fig = PandasFigure(...)

// The user can show it. The functionality will be the as it is now.
vHttp.show(fig)
// and/or they can save it to a file system which could be handed off to deploy on the website.
vFs.save(fig)

The added benefit of this pattern is it can extended. This may be particularly useful for HTTP as there are a plethora of web servers and web technologies. New classes could be created for integration and/or deployment.

What do you think?

Regards,
John

[mikedewar]

Hi John! Sorry about the slow response. d3py is a bit of a slow-burn project!

As for the model / view split - it all sounds good! I'm very keen to do the following;

• Keep the user experience really simple
• decouple the javascript generation from the geom definition
• allow server tech to be swapped out as necessary.
• use websockets to allow animation / interaction / data updating

I think splitting this out along the lines you describe would be a good idea! Would you be up for implementing it? If you can get a pull request together, I'd love to take a look, and I'm sure @mynameisfiber would be too.

[kern3020]

Hello Mike,

Yes, I would like to implement it. This week I'll be going to pyData but should have time to work on it after that.

Regards,
John

[mikedewar]

Awesome! I'm looking forward to it. Enjoy pyData!

[mynameisfiber]

@kern3020 You can add the following into __init__.py:

from util import display, deploy

and then use d3.display/d3.deploy in the examples. I'm not that big of a fan of having to cary around two objects, the display mechanism and the figure, since it can complicate things for the user, however if we just make sure these utility functions are as accessible as possible it may be okay. In this same vein, the display function doesn't seem to provide for the background HTTP server which, in my opinion, is the most useful. What I would like to do, in ipython for example, is something like:

fig = d3.Figure(...)
fig += ....
fig.show(background=True)

fig += ...

And be able to just refresh my browser to see the changes

Another note.... do you think we could change the terminology to be more specific? Namely, display doesn't always mean display through HTTP. Similarly, deploy doesn't necessarily always mean save to disk.

[kern3020]

Hello Micha,

Thanks for your feedback. It got me thinking about a refinement which will
make the Figure class a bit simpler.

Here is what an example looks like with this pull request

    fig = d3py.PandasFigure(df, 'd3py_area', width=500, height=250) 
    fig += d3py.geoms.Area('x', 'y', 'y0')
    fig += d3py.geoms.xAxis('x')
    fig += d3py.geoms.yAxis('y')

    with SimpleServer(fig) as server:
           server.show()

By default, why does the user of d3py need to know about the SimpleServer? What if Figure contained both a displayable and deployable and provided defaults from them. [aside: there is a typo in the pull request. SimpleServer is spelled SimplyServer.] Then the code fragment becomes...

    fig = d3py.PandasFigure(df, 'd3py_area', width=500, height=250) 
    fig += d3py.geoms.Area('x', 'y', 'y0')
    fig += d3py.geoms.xAxis('x')
    fig += d3py.geoms.yAxis('y')

    fig.show()

If someone wanted a custom displayable, they would need to create a SimpleServer or IPython when that work is completed and provide it as an argument to figure constructor. What do you think about this refinement?

-jk

[kern3020]

Hello Mike,

"The first is the loss of the 'with' idiom - why has that been removed?"

Unfortunately, the word 'removed' is ambiguous here. It has not been deleted. It has been moved into the SimpleServer class. So, with pull request #64 it became

    with SimpleServer(fig) as server:
               server.show()

I think the modifications I mentioned in response to Micha's feedback would address your feedback, too. Both the displayable and deployable will be encapsulated in figure. By default, the user won't need to know anything about them.

"The other is the util.display replacing fig.show(). I'm totally happy having additional display options in a util module, and I'm even happy having a discussion about the default show() method, but I'm less happy that the user has to import a separate module and make a decision about how to show the figure."

Everyone shares the importance adhering to a public API. I didn't understand that adding a function to the examples directory represented d3py's API. The util module will be deleted along with the display function. Any bits which are still interesting will be encapsulated into a well-known class.

"Would it be possible to get the above additions into a place that doesn't involve any changes to the standard examples?"

I think so.

"Then, to demonstrate this pull requests additions, new examples could be added?"

Good idea! I would like add an example showing how to configure d3py for something other than localhost.

-jk

[mikedewar]

Thanks for all this John! I think it makes sense to use the examples as a guide for d3py's set of idioms, so that we can get a hearty discussion going each time there's a change. With the advent of trifacta's vega project, I think d3py's set of idioms are going to become even more important! There's an opportunity for us to shape how plotting in python will take place over the next few years!