Tests fail on v0.5

[ranjanan]

Tests fail for me on v0.5 rc2:

Error During Test
  Test threw an exception of type MethodError
  Expression: to_dot(attrs) == "[\"foo\"=\"bar\"]"
  MethodError: no method matching similar(::Dict{String,Any}, ::Type{String})
  Closest candidates are:
    similar(::SubArray{T,N,P,I,L}, ::Type{T}, ::Tuple{Vararg{Int64,N}}) at subarray.jl:52
    similar{T}(::Array{T,1}, ::Type{T}) at array.jl:122
    similar{T}(::Array{T,2}, ::Type{T}) at array.jl:123
    ...
   in _collect(::Dict{String,Any}, ::Base.Generator{Dict{String,Any},Graphs.##33#34}, ::Base.EltypeUnknown, ::Base.HasLength) at ./array.jl:318
   in to_dot(::Dict{String,Any}) at /Users/ranjan/.julia/v0.5/Graphs/src/dot.jl:66
   in include_from_node1(::String) at ./loading.jl:426
   in macro expansion; at /Users/ranjan/.julia/v0.5/Graphs/test/runtests.jl:28 [inlined]
   in anonymous at ./<missing>:?
   in include_from_node1(::String) at ./loading.jl:426
   in process_options(::Base.JLOptions) at ./client.jl:262
   in _start() at ./client.jl:318
ERROR: LoadError: LoadError: There was an error during testing
 in record(::Base.Test.FallbackTestSet, ::Base.Test.Error) at ./test.jl:397
 in do_test(::Base.Test.Threw, ::Expr) at ./test.jl:281
 in include_from_node1(::String) at ./loading.jl:426
 in macro expansion; at /Users/ranjan/.julia/v0.5/Graphs/test/runtests.jl:28 [inlined]
 in anonymous at ./<missing>:?
 in include_from_node1(::String) at ./loading.jl:426
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /Users/ranjan/.julia/v0.5/Graphs/test/dot.jl, in expression starting on line 22
while loading /Users/ranjan/.julia/v0.5/Graphs/test/runtests.jl, in expression starting on line 25

[tkelman]

What is still using this package? It is not actively developed or maintained and should be considered effectively deprecated.

[papamarkou]

@tkelman, despite this package not being actively maintained, it is used by other packages (including Lora that I am developing), so it is a dependency for other packages, and for this reason Graphs can't be deprecated.

[dehann]

IncrementalInference.jl package also heavily uses Graphs.jl

[tkelman]

Then the question is why are you using an unmaintained package that doesn't pass its own tests when an multiple actively maintained alternatives are available?

[papamarkou]

@tkelman, if you have the time to make the necessary software changes and submit a PR in Lora that migrates functionality from Graphs to LightGraphs, you are welcome to do so. I had spent a few hundreds of hours to software-engineer the graph-based model of Lora on the basis of Graphs, which required both conceiving and implementing the Graphs-centric approach. I simply don't have enough time to move to another package for accommodating the graph models of Lora. I think this answers your questions as to why things are the way they are.

Although Graphs is not actively maintained, it has sufficient functionality to be used by the community. Besides, developers don't have the time to switch packages depending on which packages in the ecosystem fall in or out of flavour or fashion, especially when we are talking about packages like Graphs, which have required hundreds of hours to develop in the first place and ship with important functionality. In an imaginary situation, if my code in Lora had become compatible with another graph-related package (which is not possible anyhow for the reasons explained above), then what would happen if the developer of a new graphs package stops maintaining it in turn, would I have again to migrate to a third graph-related Julia package?

One possibility is that an attempt is made in the next GSoC or JSoC to recruit a student to maintain and update Graphs, considering how important and good this package is anyhow, as I think you haven't appreciated enough the quality and importance of Graphs when you suggested it is deprecated. The second alternative is that we simply ensure that Graphs, as it stands, can be used across new Julia versions (and if someone comes along to make more serious maintenance of it down the road, all the better). The third option as you proposed it, which is not really an option, would be to deprecate Graphs, but then Lora will effectively not be working anymore, and as I explained I have my own roadmap for Lora and no time to change to another graph-based package - Lora is used by the community, I use it for conducting research, writing research articles and possibly a book. If you take such a drastic measure with Graphs that renders Lora unusable, you are effectively asking me to stop using Lora or Julia for that matter, which I don't think it is a nice thing to ask.

[tkelman]

In #218 you said you would "do a bit of housekeeping, at least to ensure that the package remains alive." 2 small PR's were merged, but other than that? Given Lora seems to work on 0.5 as of its latest tag, at least some of Graphs.jl apparently works there. All this package needs is some active maintenance keeping it up to date, then I wouldn't be calling it unmaintained or effectively deprecated.

https://github.com/JuliaLang/Graphs.jl/graphs/contributors speaks for itself, it's not a question of fashion or importance or quality, it's a question of ongoing engineering hours being put into it. Deprecating the package would not change anything for users of existing versions of Julia, it would only put an upper Julia version ceiling on whether or not the package is installable.

[papamarkou]

Yes, my statement about housekeeping Graphs had been ambitious given my time constraints. Same goes for Sundials, I said about 3 years ago that I would contribute more actively to it too. If we trace my github trajectory, I am sure we will uncover more online messages of mine along these lines. Bottom line, it happens frequently to find oneself overwhelmed with work and not being able to contribute to multiple fronts. Although I am not sure I am not the only one who spreads thin across packages or turns to be too optimistic about their levels of contribution, I can apologize if it makes anyone feel better, or simply be twice as careful with my (promises for) level of involvement if it helps. At the same time, we are all aware that I am not a full-time employee of Julia, I just use it and develop with it as much as I can as part of my own academic work, so I can't be seriously held accountable about Graphs or any other important package if I am not the actual creator of it.

This is what I am trying to explain, that it is not acceptable to put an upper Julia version limit on Graphs, this will effectively prevent Graphs being easily installed in upcoming Julia versions. Please, do not do that. Let's just ensure that Graphs is tweaked as much as it is needed so that it remains simply compatible with new Julia versions. Thanks.

cc @StefanKarpinski @ViralBShah

[tkelman]

There was actually a GSoC project working on a graphs package, just not this one - https://github.com/pranavtbhat/Graft.jl. Any such project would need a mentor, and when a GSoC involves working on an existing package then that would need to be someone with the appropriate time for code review and assisting with maintenance.

You do need to be at least a little bit accountable for your dependencies when you decide to start using them, especially ones that run the risk of becoming unmaintained and no longer working.

[papamarkou]

Right, when I used Graphs as a dependency, it was in JuliaLang (as it still is) and it was developed at the time by one of the core Julia members of the community, @lindahua. I wouldn't say that at the time Graphs seemed to be at risk of becoming unmaintained given these facts, quite the opposite.

I would not have the time to consistently supervise a student for Graphs myself.

In my humble opinion, maintaining Graphs just so that it is not deprecated is at this stage the most reasonable avenue. I would like to kindly ask you not to deprecate Graphs, but I can't do or say anything else other than ask for this. Many thanks.

[ViralBShah]

Clearly, Graphs.jl is not getting maintained, and if tests are not passing, does Lora.jl want to be depending on it? I am ok to keep it under maintenance as long as it keeps receiving updates with julia versions and keeps passing its tests.

Do we know what the effort is to make it work again? If so, we can leave it as it is.

[ViralBShah]

@ranjanan and I are looking into what needs to happen for 0.5. Hopefully not much.

[papamarkou]

It depends really on what level of maintenance we have in mind. If by maintenance we basically want to ensure that Graphs.jl is not broken between versions, that is probably very minimal work and I can certainly look into it @ViralBShah. If not mistaken, what @tkelman means is that there are some more serious matters of maintenace, such as bug fixing and active package development, that's what I don't manage to find the time to do.

I can see the point of avoiding dependence of Lora on a non-actively developing package, as Graphs has ended up being, but migrating to LightGraphs is not entirely trivial time-wise... plus Graphs, despite its non-hyperactive state, I think it is a good package.

[ViralBShah]

Well, the maintenance should be what is required to keep Graphs.jl free of deprecation warnings and passing the testsuite. I don't think @tkelman means maintenance as active package development, but we certainly have to fix bugs that come up - which can be addressed as happens.

[papamarkou]

I will have a quick look now to see if deprecating warnings and test passing is easy to fix, hopefully it will be ok. Bug fixing can be a bit trickier because it requires a better reading of the package's code - I do have a ground understanding of it as I needed to develop my own graph type, but certainly I don't know deeply all the nitty-gritty details of it so as to be proficient at its bug-fixing.

I will let you know in the next few mins how easy it is to fix deprecation warnings and test failures.

[ranjanan]

@Scidom @ViralBShah I found the issue. On v0.4:
map(a -> (a[1],a[2]), dict) creates an array of tuples of all keys and values. This doesn't work on v0.5. An appropriate fix would be doing map(a -> (a[1], a[2]), [dict...]), which explicitly constructs an array of Pairs.

I'm working on a PR for this.

[papamarkou]

Thanks a lot @ranjanan, will look into the error in a few moments, I just managed so far to eliminate the deprecation warnings the last few mins.

[papamarkou]

@ranjanan, just to ensure we coordinate (as it seems I spent the last hour duplicating work that you also did), did your recent PR fix everything, should I drop my efforts? Thanks.

[ranjanan]

@Scidom I fixed the tests, and I'm now fixing the depwarns. Looks like I ran into a hiccup or two in fixing the dep warns, but I'll iron them out.

[papamarkou]

Ok, thanks @ranjanan. I just asked because we were working in parallel but with slightly different modifications, thus wasting time (my fault, I should had clarified from the beginning who is doing what right now). I will hold back, and will let you proceed with the work you have been doing.

[dehann]

Hi, haven't read all yet, but can help out with maintenance once I'm back from travel. About 2 weeks though...

[evanfields]

TravelingSalesmanHeuristics also depends on Graphs for spanning trees used in lower bounds. Here the dependence is just to avoid reimplementing a minimum spanning tree algorithm - not particularly painful.

I remember developing TravelingSalesmanHeuristics and being unsure which graphs package to use. Given that Graphs has a rather special name and lives under JuliaLang, it appears rather official. For the sake of new Julia users and package developers, if Graphs is not the de facto graphs package, the readme and documentation should say so in big letters.

[tkelman]

The name and being under JuliaLang are historical. Yes the readme should say that this is unmaintained and new projects should avoid it, and we should move the repo to a different organization. @sbromberger was against moving it to JuliaGraphs.

[sbromberger]

For the record, my objection still stands, and this thread is perhaps further justification. JuliaGraphs is an active ecosystem for graph modeling, and Graphs, as has been pointed out, has not been actively maintained. I don't think there's any benefit to moving a moribund package to another organization. If/when it resumes development, I'm happy to reopen #165.

The original proposal to move to JuliaGraphs was made a year and a half ago, and there was no action from any of the maintainers until I finally closed it 6 months later.

As another example, a PR I made to fix a bug that was discovered in LightGraphs and confirmed in Graphs.jl (#188) has sat unmerged since June 2015 - 14 months.

I'll leave my personal opinions regarding the wisdom of relying on a package that has had no real development in almost two years out of this comment but would like to suggest that I'd be happy to begin discussions as to whether LightGraphs is an appropriate package for Lora and others. (It might not be.)

[sbromberger]

For the record, as of yesterday:

julia> Pkg.dependents("Graphs")
11-element Array{AbstractString,1}:
 "KCores"
 "ISPC"
 "OpenStreetMap"
 "TravelingSalesmanHeuristics"
 "Mamba"
 "IncrementalInference"
 "GraphCentrality"
 "Metis"
 "Lora"
 "MinimalPerfectHashes"
 "TikzGraphs"

GraphCentrality has been deprecated for a while.

[ViralBShah]

We can also just move it to JuliaArchives.

[ViralBShah]

Also, I thought that this issue was fixed. @ranjanan ?

[tkelman]

#228 looks okay to me and I suppose it's okay to merge, I was mostly waiting for someone who uses this package and might be more familiar with the code base to test it in combination with their own package and give the all-clear.

[tkelman]

@sbromberger your objection makes sense to me. JuliaGraphs would be a better home for this in terms of naming and subject matter than the generic JuliaArchive (you already have one package in JuliaGraphs labeled "NOT ACTIVELY MAINTAINED") but I can understand you not wanting to assume any responsibility for this package if you don't use it or work on it yourself.

Your offer to help look into whether LightGraphs would be an appropriate replacement is very much appreciated, and I'd advise the developers of those Pkg.dependents("Graphs") to take you up on that offer.

[tkelman]

I've granted @dehann and @evanfields collaborator access to this repository. If we move it, I think github should maintain that but ping me if that happens and there are any problems.

[papamarkou]

@sbromberger there are several species of software, including languages and libraries, that are several decades old and are still used, even if they are not maintained nowadays. My main point is that Graphs is a good package, and any unfixed bugs (which are nearly always present in any software) are not the only or decisive factor for characterizing the package as obsolete or moribund. What I don't understand is why I feel I am kind of alone in showing some appreciation for the work that Lin Dahua has put into Graphs. In my opinion (as previously stated), the natural choice is to embrace Graphs with further support via GSoC or JSoC, seeking both students and supervisors for it.

One more very important thing, that I think it is being neglected in the discussions about Graphs; it is not just a matter of preference, I simply have no bandwidth/time to rediscover the wheel once again and rethink-recode Lora's graph model to work with a new Julia package for graphs. I am not sure that the time I have put (or any other developers that rely on Graphs) into developing Lora using Graphs is being considered as an important factor in the discussion.

Anyhow, I don't think it is very constructive to recycle my points, esp if I feel that for one or another reason they are not heard, however much I stress them...

As far as LightGraphs is concerned, I obviously don't have anything against the package, and I would be very happy to migrate Lora's graph infrastructure to it. However, once again, this is a hypothetical scenario, assuming that I would find the time to do all this, but my guess is that this will simply remain an open issue for so long that at the end it will drag Lora down into becoming obsolete following the unfortunate path of Graphs.

[tkelman]

Yes, we're repeating ourselves - we all appreciate the work that Dahua put into this and many other packages, but recognize that he's not around anymore and hasn't been for some time, and Julia changes rapidly. @Scidom can you review and test whether #228 will work okay for Lora?

Just to put you at ease, I don't plan on formally deprecating this package in metadata in the 0.5 timeframe. But we should move it to a non-JuliaLang organization and document the current maintenance status more clearly to warn potential users.

[papamarkou]

I understand this about Graphs @tkelman, but there is only so much I can adapt in terms of my available time. I can try to look into the migration of Lora's graph model to a new package in the future, but I must say that I am alarmed about how this endeavour (and Lora) will end up, knowing the reality of my time constraints and busy academic schedule. Indeed, it is not helpful to recycle our main points...

I am overwhelmed with some research work today and tomorrow, but will be able to take a few mins and check #228 over the weekend.

[ViralBShah]

Appreciated and understand. Should hopefully not take much time, and I am happy to help as necessary. But certainly need to move to a different org. Unfortunate that the name Graphs.jl will not be available.

I would have preferred the actively developed packages to be called Graphs.jl. Oh well...

[tkelman]

It would actually be possible to do a merge commit replacing all of the content with one of "the actively developed packages." That way all the history would still be here and past tags would work (we'd add upper bounds to all the packages that use the current Graphs.jl API). If needed, we could also make a copy of this repository and call it LegacyGraphs.

[papamarkou]

That would work indeed - if LegacyGraphs will be available through METADATA, Lora won't be disrupted in the near future. I will get back to you over the weekend about #228.

[sbromberger]

@Scidom whether to use Graphs.jl or some other package is entirely your choice; I'm not trying to influence your decision in any way whatsoever. However, each choice you make comes with risk/cost. If you'd like to discuss migration, I'm available to chat; if not, that's OK too.

As for appreciation for this package, it was the inspiration for LightGraphs and several functions were lifted directly (with appropriate credit); it's fair to say LG wouldn't be where it is today without Graphs.jl and the committers who took time to help me out when I was writing LG core.

That said, Julia's only 3.5 years old, and Graphs.jl has been more or less dormant for half that time. There's no issue with using "old" packages in an old ecosystem, but I would have big concerns with using something that hasn't seen growth commensurate with its underlying language.

@tkelman the "NOT ACTIVELY MAINTAINED" package is GraphCentrality, which was my first package that was written for Graphs.jl (since the committers didn't want to merge centrality measures into Graphs directly). Since Graphs.jl is not fixing bugs that impact GraphCentrality, I've decided to not spend time developing it any more. If nobody's using it, I suppose I can deregister.

[tkelman]

I suppose I can deregister.

We can discuss this elsewhere, but the process for that is adding an upper Julia version bound to all past tags of the package in metadata. See JuliaLang/METADATA.jl#6084 for an example.

[sbromberger]

@tkelman yes, bounding seems appropriate. Thanks for the suggestion.

[mlubin]

FYI, JuMP/ReverseDiffSparse dropped its dependency on Graphs.jl over a year ago

[papamarkou]

Do we know if this package will stick around in the archive for a period of two years, being accessible from metadata? I need it for a book I am writing and before I finish the book, I will almost certainly have no time for migrations due to a number of work responsibilities. If you cant guarantee this package will stick around for about two years please let me know as there is no point writing a book anymore for a deprecated Lora. Thanks.

[tkelman]

It should remain available in its current state, but nothing is an absolute guarantee. Continuing to be available for future releases of julia depends entirely on whether anyone spends the time to keep it updated. Changes in Julia itself could easily cause the package to stop working again. Without any time spent on maintenance here, you will likely find the code in your book only works on what will end up being outdated versions of Julia.

[lindahua]

I just noticed this conversation. I contributed to a number of Julia packages in early days, including this one. It is unfortunate that I no longer has the capacity to actively maintain these packages (e.g. fixing all the broken places caused by the changes in Julia itself), due to the overwhelming workload in academic commitment, and more importantly, the switch of my research area (to deep learning). Over the past couple years, I have successfully converted several of my PhD students to be Julia users. Whereas I wish that they can take my place to maintain these packages, I am not able to ask them to do so as these packages are not pertinent to their research topics.

The unfortunate situation of this package is only one example. I believe that there are plenty of other packages facing similar difficulties. Complete reliance on a single developer is not sustainable -- in these days, especially in the field of computer science, things evolve very rapidly and people often change their interest. The community may need to pay attention to such issues, which is very important for cultivating a sustainable ecosystem.

BTW, for the package naming issue, what about we consider adopting a domain.package_name pattern -- in this way, I can change this to dahua.Graphs etc, instead of occupying a valuable slot in the global package namespace.

[ViralBShah]

@Scidom The package will stay in JuliaArchive for as long as it is necessary. If changes are minor, we will try keep it working with new versions of julia. It is up to you whether you want to keep depending on a package that is known not to be maintained broadly - we have tried our best to support you.

Given the amount of discussion this has generated, you haven't yet said what amount of effort it is for Lora to migrate to LightGraphs.jl. Are we talking days, weeks or months?

[tkelman]

Package namespaces is a definite todo for the upcoming iteration of the package manager.

[ViralBShah]

Graphs.jl master is passing Pkg.test().

[papamarkou]

@ViralBShah you know how things work with software development better than me. Sometimes it is a little bit difficult to quantify the amount of time required for achieving the end result. I know that the change will be non-trivial, mainly because LightGraphs define lightweight graphs, whereas the generic graph I have defined in Lora using Graphs defines nodes that are model parameters themselves. I would have to sit down and think, till I feel that a LightGraphs approach clicks to me, and can't predict in advance how much development time the implementation will entail. This might for instance be a month of full time work, which is virtually impossible for me to find.

I am facing the same difficulties from the standpoint of academic commitment. It was easier for me to catch up with the flux of transient states in the Julia ecosystem 2 or 3 years ago as a postdoctoral researcher. At this stage, despite my good will, it is practically impossible for me to adapt as flexibly to the changes related to important packages.

In my opinion, it is a problem that independent efforts are made to develop separate packages for the same domain of application.

I honestly appreciate the effort made to support me. What I am trying to say is not that I am irritated with anyone in the thread, but that I am facing a practical problem of available time to migrate to LightGraphs (or to another graphs-related package if LightGraphs gets deprecated in the future for the same reasons), that's the only issue here.

[ViralBShah]

Well, everything is working right now. Let's keep it going. Let's revisit around 0.6, and if the effort is not too much, we will keep Graphs.jl going. Of course, this cannot be a guarantee, and hopefully best effort is good enough.

I think we have discussed this satisfactorily and have a plan.

[papamarkou]

Yes, that's ok for now, and thanks for the time and effort to maintain Graphs alive. I understand that no long-term commitments can be made about Graphs.

I will try to migrate Lora to a graph implementation based on LightGraphs if I find the time to do so. I am afraid it won't be possible to achieve this very soon, but at least I will do my best.

[ViralBShah]

Fixed in #228

[timholy]

Nice to see you, @lindahua ---all the best!

[sbromberger]

In my opinion, it is a problem that independent efforts are made to develop separate packages for the same domain of application.

I strongly disagree. The issue – particularly with something like graph modeling – is that different approaches / design decisions yield different benefits based on what's important to the author/user. For example: MatrixNetworks.jl chooses to use sparse matrices to describe graphs via adjacency lists; this has huge benefits to spectral graph operations but equally large disadvantages for other operations (graph modification, for example, is relatively expensive, and there are other algorithms that just won't take advantage of the speed that algebraic treatment affords).

LightGraphs is very fast and simple, but at the expense of not being able to handle graph metadata.

The diversity of the ecosystem is one of its strengths, not a weakness: just as with other languages, having multiple packages within a particular domain, each with its own set of features and tradeoffs, gives people choice.

The primary issue I see here is that Graphs.jl was originally created under the JuliaLang organization and remained there even when development stalled. IMO, packages under this organization should have some minimum guarantees that they will continue to function at least with the current -stable release and should be promptly moved/deprecated when it's apparent that they can't. This is (mostly) a volunteer open source exercise, after all, and expecting specific performance / commitment from developers who are volunteering their time is inappropriate and selfish.

[papamarkou]

I agree that diversity is important when it covers different lines of development. An adjacency matrix-based representation of graphs is different than an adjacency-list one or than a sparse matrix-based one. There is a fine line between packages and in some cases there is need for more communication between developers of packages in overlapping areas, in my opinion.

I don't expect commitment from a specific developer or set of developers. I am hoping for a more global coordinated community-based maintenance. The tentative solution I proposed, to possibly recruit a student to help Graphs flourish further or to keep the package alive with some minimal effort of fixing inter-version issues, pointed to this direction, irrespectively of whether my solution was a good or bad one. In my opinion, there is an issue about how to keep packages alive independently of their creators; this is not an expectation from one specific individual, quite the opposite.

I am volunteering my time too, like most of us here. I am investing energy on Julia and try to find a viable solution so that my research and academic activity, which I chose to carry out using Julia, can operate. If the Graphs package had survived, the community would had benefited, not only me. If I would manage to write a scientific book on Monte Carlo methods with Julia, I would benefit by further expanding my knowledge horizons, but the community would benefit too assuming it wouldn't be a bad book.

I would like to kindly ask you to be more careful with personal attacks, with comments related to my levels of wisdom or selfishness, it is the second time it happens.

[tkelman]

Let's please stop this discussion, it's not going anywhere productive and we've collectively wasted way more time on it than was really called for, given the fix in #228 was 15 lines of code and would have only taken a few minutes to review and test.