Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is restfuse dead? #63

Open
mrkdev opened this issue Nov 21, 2014 · 18 comments
Open

Is restfuse dead? #63

mrkdev opened this issue Nov 21, 2014 · 18 comments

Comments

@mrkdev
Copy link

mrkdev commented Nov 21, 2014

Please tell us what you plan to do with restfuse and where it will go. As of now, there seams to be no current commit nighter some other work happening. Also there are multiple long pending pull requests in this repo. Is restfuse dead or still alive? It is an interesting project and if it's still alive, I'm interested to contribute and add some new features.

@kamal2222ahmed
Copy link

i think we (the contributors) should pickup this where it left off. This is ONE of a KIND project, and if you try to look for a REST testing framework, you would not find ANY that is reasonably good and Free

@bbwinter
Copy link

HTML Unit.  I lost patience with Restfuse as it was buggy and didn't do everything it said it should.  html unit has been very nice so far. 

 On Friday, December 19, 2014 7:28 AM, kamal2222ahmed <[email protected]> wrote:

i think we (the contributors) should pickup this where it left off. This is ONE of a KIND project, and if you try to look for a REST testing framework, you would not find ANY that is reasonably good and Free—
Reply to this email directly or view it on GitHub.

@smoyer64
Copy link

I looked at Restfuse, saw comments similar to those in this thread and decided to use RestAssured instead. I like the programming model of Restfuse ... it's so much better than the (Ruby like?) DSL you use in RestAssured. I do like that the DSL includes methods to test the response contents including parsing JSON structures (I can add JSONUnit to Restfuse and get the same functionality).

In any case, I'd be happy to pitch in and help with the Restfuse project IF the current maintainer will let someone take it over.

@kamal2222ahmed
Copy link

Sent an email to Holger Staudacher , asking his permission, but i personally would think it would be ok to start at least fixing the build and work on the bugs. As for RestAssured, i did look at it while evaluating a REST testing framework, and it has the disadvantage of Groovy being used, which is yet another language to learn and increases the pre-requisite knowledge in order to use the framework

@kamal2222ahmed
Copy link

I have started working on the bugs , and once i get a bit deeper understanding of restfuse, i will perhaps contribute to enhancements. My question is: what is the best way to maintain restfuse.

  1. Should we fork it, and THEN Enhance it
  2. Should we maintain THIS repo
    also, what is the process of fixing bugs, enhancements , etc since the owner is not available. Here is what i found as an example:

how to use GitHub effectively for your project

GitHub provide an awesome set of tools for opensource developers. For Dancer, we use them as much as possible. I'll show and explain how we do our development.

code review

Dancer's development goes fast. We do our best to ship often, we do a lot of refactoring, and we listen our users. This means processing pull request and issues as fast as possible.
pull request

There is five developers with write access to the main repository. Each one is asked to do a review of pending pull request every morning (this is not required, neither enforced, it's a "if you have ten minutes in the morning while drinking your cofee, please, review the PR").

When we're reviewing something, most of the time we pull the submited patches into a branch named review/username. If the reviewer is happy with the modifications, he will add a comment to the pull request: "approved" (and some other comments if it's required). In case he's not happy, he will comment disaproved and give his reasons (tests doesn't passes, or there is no tests for the patch provided, or this is not something we want, etc). If the PR is about something the developer doesn't really knows, or has a doubt, he skip this one, and ask for someone else to take a look.

In order to merge the branch review/username, we need two developers to comment approved. Of course, for something simple, like a documentation fix, or changing a few line somewhere, we can merge this without applying this process.

As the work consists to read the code and comment, it's quiete easy to handle two/three pull request each day for a developer. When one of the developper with access to the repository find some time, he can go through the pull request, and merge the one marqued as approved, since he knows that the people who had already approved them understand the code.
issues

We don't use RT to manage our issues. It's not that RT is bad or that we don't like it, it's just that GitHub's issues are more integrated with our workflow. GitHub's issues are really simple (one can even say naive), but most of the time it's ok for our usage. We don't need advanced features like "track how much time you've spend on this ticket" (even if I do track my time spent on Dancer, using orgmode).

One of the nice feature of GitHub's issues, is that you can close them with a commit. If the commit's message looks like 'closes GH-123', the issue 123 will be closed, with a link to the commit in the comment (take a look). I find this feature really useful, since when refering to a closed issue, you can find the commit inside the ticket.

Once or twice a week, we try to proceed a triage, where we go through the issues, and tag them.

When someone report something on the mailing list or on irc, we ask them if they can open an issue, since it's easier for us to track the request.

An issue doesn't need to be about a bug, it could be:

a feature request (I want to do x or y with dancer)
something that need to be refactored
an issue reported by an user that need feedback from the developers

commenting on code

Another nice feature is the possibility to comment the code. Most of the time you'll do it while reviewing a pull request. But a user can also comment on something.

Sometimes we push a branch that need some feedback, and a discussion will be started.
comparing branches

You can easily diff two branches. With this url you can do a quick diff of the changes between master and devel, see the list of commits, and which files have changed.

This is usefull when you want to have an overview of the work.
gitflow

We're using gitflow to work. Gitflow is a nice tool that help you creating and merging branches. If you don't know gitflow, there is a good article that explain the reasons behind it.

Ok, this has nothing to do with GitHub, but I'll explain quickly what we do with it.

an article on gitflow: http://scottchacon.com/2011/08/31/github-flow.html

We use the following conventions:

master: only for release
devel: this is the development branch. This one should always work (tests can't be broken)
topic/$name: when we start to develop a new feature, we create a topic branch
hotfix/$name: when we need to release a new version right now, without merging devel before, we create a hofrix branch
release/$version: when we're ready to ship, we create a new branch

It's very rare that we need to push a hotfix or release branch to GitHub: thoses branches had a really short life span. But topic branches can be pushed, when we want feedback from users or other developers.
future

We're already using jitterbug to do some continuous integration. When we push to GitHub, a payload is posted to jitterbug, and a build is triggered. In a near future, we will use git's notes with jitterbug. The idea is to store inside a note, for each build, the duration of the build, and the result of the build (failure / success) and maybe the TAP output in case of a failure. This will allow developers to see directly in the logs the status. Of course, GiHub already display them, so this will be very useful for all of us.

lumberjaph.net is © 2008 – 2014 Franck Cuny // @franckcuny

references:

  1. http://code.tutsplus.com/articles/team-collaboration-with-github--net-29876
  2. https://gun.io/blog/how-to-github-fork-branch-and-pull-request/

@GGBeer
Copy link

GGBeer commented Jan 13, 2015

+1 for option "1. Fork it, and THEN Enhance it."
Optional creating a new repo "RestFuse-NG" would give us a clean start.

@smoyer64
Copy link

@GGBeer I came here to play devil's advocate to your comment (+1 for option "1. Fork it, and THEN Enhance it."). Now that I've pulled the code and attempted to build and test it in a few different configurations, my opinion has changed dramatically. To be fair to the discussion, I'll include the pros I was originally going to promote:

  • Taking over the original project would keep the file edit history.
  • Taking over the original project would let us update the project's web site.
  • Taking over the original project would keep its URL on Github.
  • Taking over the original project would keep the groupId and artifactId the same on Maven central.

I'll admit I'm a fan of more minimalist projects and therefore not a fan of some of the (what I consider) Maven bastardizations. The main point of Maven is "convention over configuration" and I think this project is a great example of what NOT to do:

  • Creating the library JAR file as an OSGI feature.
  • Adding in Tycho build tools to perform tasks that Maven does natively.
  • Changing the directory structure needlessly (or maybe it matches a default Eclipse project ... or Ant).
  • Committing code in a non-functional state (or perhaps it's not providing developer documentation that's sufficient to build and test the project ... I can't tell). I spent a couple hours trying to get the testRemovesProxyProperties() method in com.eclipsesource.restfuse.internal.HttpTestStatement_Test working ... I can see the embedded Jetty server starting, yet I never seem to null out the proxy configuration.
  • Broken (or inattentive?) release engineering - release v1.2.0 is in Maven central but the library's project is still set to v1.2.0-SNAPSHOT.

I haven't tried analyzing the project with Sonar yet, but I'll give my impressions ... I think the code quality is actually pretty good, but there's an almost complete lack of comments in the classes I looked at.

I spent a couple more hours carefully merging the existing pull requests into restfuse and while I'm not sure I understand all of the Tycho related cruft, it does seem to build the library. One integration test is failing as noted above. You can find the (hopefully temporary) project here:

https://github.com/selesy/restfuse

and a JAR file containing the results of my work here:

https://github.com/selesy/restfuse/releases/tag/v1.2.1-SNAPSHOT

For the most part, the example code works ... there are two TestCases with two tests each that fail:

  • DynamicPathTest fails because search.maven.org returns a status code of 302 (Found) instead of 200 (Ok). I'm guessing this test started failing due to changes to search.maven.org rather than the software regressing.
  • DynamicBodyTest fails because there isn't a server running on localhost. This test depends on the configuration of the host machine but should be environment independent.

If the group is so inclined, I'd be more than happy to delete the unnecessary build tools and set the project up so that it's built in a conventional Maven fashion. The only issues I have with doing so is whether we'd conform to the EPL license with a dramatically restructured project, and whether it's fair to the original developer(s). It's entirely possible either don't understand what they were trying to accomplish with OSGI/Tycho or they were working towards a bigger goal that isn't visible to the group.

For posterity, here's the process I used to create v1.2.1-SNAPSHOT:

  1. Forked the restfuse project.
  2. Tagged the fork to preserve a starting point.
  3. Pulled and merged a pull request into the project.
  4. Built the library from the root of the project with: mvn install -f com.eclipsesource.restfuse/pom.xml
  5. Ran the tests from the root of the project with: mvn test -f com.eclipsesource.restfuse.test/pom.xml
  6. Tagged the project with a checkpoint.
  7. Repeat from step three for each pull request.
  8. Ran the example tests (in Eclipse since there's no pom.xml in that project).
  9. Updated the project version to match the new OSGI bundle version.
  10. Pushed the project back up to Github.
  11. Created a manual release (uploading the created JAR.

Sorry for posting such a long comment ... where should we go from here?

@GGBeer
Copy link

GGBeer commented Jan 15, 2015

Had the same problems with the directory structure and build tools. As I am currently not using Eclipse and are not that familiar with OSGi/Tycho/etc. I just created a new local maven pom.xml with a standard directory structure to get going, but copied my changes back into the original dir structure before commiting to my fork.

Your pros are definitely true and we should respect the original project developers work and efforts. Also don't know if the EPL license model is still OK or should be changed.
Anyway the fork you made is certainly a good way to get things going and bring restfuse back to live again.

@smoyer64
Copy link

I have confirmed that the DynamicPathTest.java tests failed due to changes in search.maven.org and have modified the tests to expect a 302 HTTP status code (the latest is at: https://github.com/selesy/restfuse). I'm tempted to make a branch with the project moved into a conventional Maven project ... I think a parent project with one child module for the library and another for the examples would be enough.

The test module could be moved into the library project under src/it/java and run with the maven-failsafe-plugin. I can provide CI and analysis via Sonar but I think before we get too far along we should make some decisions about project name (if it needs to change), how to update the documentation (especially developer documentation), etc.

@yurgisbaykshtis
Copy link

Thanks smoyer64 for taking over this great project. Any plans to update an artifact in maven repo?

@smoyer64
Copy link

smoyer64 commented Feb 7, 2015

I started working on simplifying the project and believe it's back to
working the way it did before. I was going to convert the issues in the
original repository and start fixing them to (I've already applied the pull
requests).

I was waiting to hear what the group wanted to do long-term ... I feel as
though I've stolen the project in a way but my intent was just to move it
forward a bit. What does the community want?

In any case, I'll be generating releases as I go but I'm not sure I can get
them into Central under the original GAV. Does anyone have a contact at
EclipseSource? How can we get an official blessing and take over the
web-site, etc?

On Fri, Feb 6, 2015 at 5:55 PM, yurgisbaykshtis [email protected]
wrote:

Thanks smoyer64 for taking over this great project. Any plans to update an
artifact in maven repo?


Reply to this email directly or view it on GitHub
#63 (comment)
.

Sent from my iBerry MacPhone

@kamal2222ahmed
Copy link

smoyer64, The name would depend on what are we going with restfuse . Here are some which we can ammend, or augment.

  1. Generates test run reports in JUnit XML format. Which is compaitlble with most CI servers
  2. Provides benchmarking
  3. Orchestration
  4. Extensible
  5. supports POST, GET, PUT, DELETE, OPTIONS, PATCH and HEAD
  6. Support Response types, JSON, XML , HTML etc
  7. Coverage Reporting

@drejc
Copy link

drejc commented Feb 8, 2015

Nice to hear that RestFuse is moving again.
What I personally miss the most is a way to perform scenario tests. I have submitted a patch to add an Order property to the test, forcing RestFuse to execute tests in a certain order. This solves some of the scenario issues but not all of them.

An addition pain is the use of dynamically generated paths. It can be achieved but is very cumbersome.
Especially if you want the next REST call to use the return value of a previous one (scenarios again).

Just to illustrate I will give you an example what I'm currently testing with REST fuse:

  • create account
  • login with wrong credentials
  • login with correct credentials
  • make call to a restricted REST interface
  • log out

In this scenario the login call creates a cookie (session), which must be than provided on the next calls an so on.

Currently the code is way to complicated for what it does.
Just a simple history stack with return values of all calls in a scenario would reduce my code in half.

@smoyer64
Copy link

smoyer64 commented Feb 8, 2015

Unit tests are supposed to be idempotent ... the changes you're describing
sound more like a Selenium test (which we've used to do UI testing) and I
can see a place for them testing RESTful APIs as well.

REST isn't supposed to rely on state being managed on the server and in the
case you've shown above, I would tend to make a parameterized test that
tested without any credentials, with the wrong credentials and with the
correct credentials. Using annotations to define these are, as you've
noted, somewhere between impossible and cumbersome.

I think testing security is a very common use case and can be broadly
applied (having a Cookie is roughly equivalent to having a Bearer-Token in
a Header). So the real question is what's the best way to allow
programmatic configuration of the call. At some point, what if the
@httptest annotation was empty and you had to trigger the call within the
test?

I've used RestAssured and it's a lot more like what you describe ... the
downside is that it's all programmatic (and based on a Groovy DSL).

This should probably be a separate conversation ... I think there's a lot
to discuss!

On Sun, Feb 8, 2015 at 6:42 AM, drejc [email protected] wrote:

Nice to hear that RestFuse is moving again.
What I personally miss the most is a way to perform scenario tests. I have
submitted a patch to add an Order property to the test, forcing RestFuse to
execute tests in a certain order. This solves some of the scenario issues
but not all of them.

An addition pain is the use of dynamically generated paths. It can be
achieved but is very cumbersome.
Especially if you want the next REST call to use the return value of a
previous one (scenarios again).

Just to illustrate I will give you an example what I'm currently testing
with REST fuse:

  • create account
  • login with wrong credentials
  • login with correct credentials
  • make call to a restricted REST interface
  • log out

In this scenario the login call creates a cookie (session), which must be
than provided on the next calls an so on.

Currently the code is way to complicated for what it does.
Just a simple history stack with return values of all calls in a scenario
would reduce my code in half.


Reply to this email directly or view it on GitHub
#63 (comment)
.

Sent from my iBerry MacPhone

@yurgisbaykshtis
Copy link

Starting from junit 11 you can execute tests in name order. What we do is just prefixing names with test###_ (like test002_userPut).
That works for us with restfuse.
The only (minor) problem is that you can't mix @httptest and @test with the latter ones always being executed after the former.

On Feb 8, 2015, at 3:42 AM, drejc <[email protected]mailto:[email protected]> wrote:

Nice to hear that RestFuse is moving again.
What I personally miss the most is a way to perform scenario tests. I have submitted a patch to add an Order property to the test, forcing RestFuse to execute tests in a certain order. This solves some of the scenario issues but not all of them.

An addition pain is the use of dynamically generated paths. It can be achieved but is very cumbersome.
Especially if you want the next REST call to use the return value of a previous one (scenarios again).

Just to illustrate I will give you an example what I'm currently testing with REST fuse:

  • create account
  • login with wrong credentials
  • login with correct credentials
  • make call to a restricted REST interface
  • log out

In this scenario the login call creates a cookie (session), which must be than provided on the next calls an so on.

Currently the code is way to complicated for what it does.
Just a simple history stack with return values of all calls in a scenario would reduce my code in half.


Reply to this email directly or view it on GitHubhttps://github.com//issues/63#issuecomment-73407392.

@smoyer64
Copy link

smoyer64 commented Feb 8, 2015

Maybe this is one of those cases where it's best to work backwards ...
could you provide a complete test class that completes the test you
describe above and is structured the way you envision it? I realize it
won't run on the current Restfuse but it's a great way to discuss how we'd
accomplish such a test.

On Sun, Feb 8, 2015 at 12:48 PM, yurgisbaykshtis [email protected]
wrote:

Starting from junit 11 you can execute tests in name order. What we do is
just prefixing names with test###_ (like test002_userPut).
That works for us with restfuse.
The only (minor) problem is that you can't mix @httptest and @test with
the latter ones always being executed after the former.

On Feb 8, 2015, at 3:42 AM, drejc <[email protected]<mailto:
[email protected]>> wrote:

Nice to hear that RestFuse is moving again.
What I personally miss the most is a way to perform scenario tests. I have
submitted a patch to add an Order property to the test, forcing RestFuse to
execute tests in a certain order. This solves some of the scenario issues
but not all of them.

An addition pain is the use of dynamically generated paths. It can be
achieved but is very cumbersome.
Especially if you want the next REST call to use the return value of a
previous one (scenarios again).

Just to illustrate I will give you an example what I'm currently testing
with REST fuse:

  • create account
  • login with wrong credentials
  • login with correct credentials
  • make call to a restricted REST interface
  • log out

In this scenario the login call creates a cookie (session), which must be
than provided on the next calls an so on.

Currently the code is way to complicated for what it does.
Just a simple history stack with return values of all calls in a scenario
would reduce my code in half.


Reply to this email directly or view it on GitHub<
#63 (comment)

.


Reply to this email directly or view it on GitHub
#63 (comment)
.

Sent from my iBerry MacPhone

@GGBeer
Copy link

GGBeer commented Feb 9, 2015

Currently I am using RestFuse to test my JSON-based Webservices. For testing a specific WS-Function I have to make sure that all objects which are required for executing that function have been generated/added in advance and the TestCase must use/reference the specific UUIDs associated to these (temporary) objects. Also some parameters of the testcases must be generated on TC-execution e.g. include timestamps, provide unique object names, etc. The ordered testcase execution and DynamicBody features can be used for this but are difficult to use in large test scenarios.

For that reason I extended RestFuse to have full control over the HTTP headers and body just before the message is sent out via HttpRequest. At that point also access to the HttpTest-Annotations was important to me for defining further test specifc parameters/details (e.g. the fields containing the UUIDs to be used). Via reflection an instance of a ProcessRequest-class (definable by an HttpTest annotation parameter) is created in the HttpStatement.evaluate() phase. This ProcessRequest class can do just anything to/with the HTTP contents at that point. Also via reflection the test specifc parameters are retrieved by their field getters.
Of course this kind of RestFuse usage is not an idempotent JUnit test anymore (but solves my testcase writing problems).

My restfuse changes are far from perfect and have lots of room for improvement, but maybe they are usefull to the community.

Thx,
Guus

@drejc
Copy link

drejc commented Feb 9, 2015

Ok ... here is an example ... of a simple test of mine ... just for illustration purposes.
The destination is dynamically created on each call ... once the cookie is filled (with previous call) the cookie is used:

@Rule
public Destination destination = getDestination();

// provide dynamic destination according to stored cookie
private static String authCookie;
private Destination getDestination() {
        Destination destination = new Destination( this, TestSettings.HOST);

        if (authCookie != null)
            destination.getRequestContext().addHeader("Cookie", authCookie);
        return destination;
    }

@HttpTest(method = Method.POST,
              path = "/rest/public/login",
              content = LOGIN_JSON,
              type = MediaType.APPLICATION_JSON,
              order = 20)
    public void login() {

        // login user
        assertEquals(HttpStatus.SC_OK, response.getStatus());
            List<String> cookie = response.getHeaders().get("Set-Cookie");
                // cookie is store in static variable ...
        authCookie = cookie.get(0);
    }

// not public ...
@HttpTest(method = Method.GET,
              path = "/rest/user/info",
              order = 30)
    public void getInfo() {
        assertEquals(HttpStatus.SC_OK, response.getStatus());
        String body = response.getBody();
        ... assert response ...
    }

Current disadvantage is that @path, @content and @authentication can not be dynamically generated so you are kind of limited with the current implementation.

While the static annotation principle is nice for simple tests ... the dynamic aspect should also be considered ... for instance a call to GET /test/user/info/{userId} is not possible at the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants