[Bug][API] project check url encoding

[jeffschaper]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

We're rolling out DORA metrics at my company and we chose to use a forward slash (/) as a delimiter in our project naming convention. For example, team/ci/app.

I wrote a Python script to onboard projects to DevLake and part of the logic includes checking to see if the project already exists. The /check endpoint doesn't seem to not be handling url encoding the way I'd expect.

Here's a sample curl request curl http://localhost:8080/api/rest/projects/team%2Fci%2Fapp/check -H "Authorization: Bearer <token>"

The API response returns a 404 HTTP error: 404 page not found

What do you expect to happen

I'd expect the API response to return a json object as described in the API docs

{
  "exist": false
}

How to reproduce

Make a call to the /check endpoint with a project name using %2F to delimit a project name such as team/ci/app

Sample curl request: curl http://localhost:8080/api/rest/projects/team%2Fci%2Fapp/check -H "Authorization: Bearer <token>"

Anything else

The error seems to only be happening in our dev environment. Localhost seems to be handling this scenario without any issues.

Further, the error only happens when I substitute a forward slash (/) for %2F. Single word project names seem to be handled okay in dev.

I think I tracked down the functions responsible:

• incubator-devlake/backend/server/services/project.go

  Lines 421 to 431 in f99d5bd

  	func getProjectByName(tx dal.Dal, name string, additionalClauses ...dal.Clause) (*models.Project, errors.Error) {
  	project := &models.Project{}
  	err := tx.First(project, append([]dal.Clause{dal.Where("name = ?", name)}, additionalClauses...)...)
  	if err != nil {
  	if tx.IsErrorNotFound(err) {
  	return nil, errors.NotFound.Wrap(err, fmt.Sprintf("could not find project [%s] in DB", name))
  	}
  	return nil, errors.Default.Wrap(err, "error getting project from DB")
  	}
  	return project, nil
  	}

• incubator-devlake/backend/server/api/project/project.go

  Lines 36 to 75 in f99d5bd

  	// @Summary Get a project
  	// @Description Get a project
  	// @Tags framework/projects
  	// @Accept application/json
  	// @Param projectName path string true "project name"
  	// @Success 200 {object} models.ApiOutputProject
  	// @Failure 400 {string} errcode.Error "Bad Request"
  	// @Failure 500 {string} errcode.Error "Internal Error"
  	// @Router /projects/{projectName} [get]
  	func GetProject(c *gin.Context) {
  	projectName := c.Param("projectName")
  	projectOutput, err := services.GetProject(projectName)
  	if err != nil {
  	shared.ApiOutputError(c, errors.Default.Wrap(err, "error getting project"))
  	return
  	}
  	shared.ApiOutputSuccess(c, projectOutput, http.StatusOK)
  	}
  	// @Summary Get project related check
  	// @Description Get project related check info, such existence, token validity
  	// @Tags framework/projects
  	// @Accept application/json
  	// @Param projectName path string true "project name"
  	// @Success 200 {object} models.ApiProjectCheck
  	// @Failure 400 {string} errcode.Error "Bad Request"
  	// @Failure 500 {string} errcode.Error "Internal Error"
  	// @Router /projects/{projectName}/check [get]
  	func GetProjectCheck(c *gin.Context) {
  	projectName := c.Param("projectName")
  	projectOutputCheck := &models.ApiProjectCheck{}
  	_, err := services.GetProject(projectName)
  	if err != nil {
  	projectOutputCheck.Exist = false
  	} else {
  	projectOutputCheck.Exist = true
  	}
  	shared.ApiOutputSuccess(c, projectOutputCheck, http.StatusOK) // //shared.ApiOutputSuccess(c, projectOutputCheck, http.StatusOK)
  	}

Version

v1.0.1@e061ef2

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[dosubot]

The issue you're experiencing with URL encoding in the /check endpoint might be related to how the Gin framework handles URL paths. In the code, the UseRawPath setting is enabled, which should allow special characters like %2F to be processed correctly ^[1].

However, since the problem occurs only in your dev environment and not on localhost, there might be differences in configuration or middleware that affect URL handling. Check if there are any discrepancies in the CORS or authentication middleware settings between the two environments ^[1][2].

Additionally, ensure that the server in the dev environment is configured to handle encoded URLs properly. You might want to verify any reverse proxy settings or server configurations that could be affecting the request path.

If these suggestions don't resolve the issue, consider debugging the request handling in the dev environment to see how the path is being interpreted.

To continue talking to Dosu, mention @dosu.

[jeffschaper]

I'll check with my platform team on these questions

[michelelcham]

also i am facing this on my devlake, unable to load data from branches that contains slashes "/"

[jeffschaper]

I'll check with my platform team on these questions

As a follow-up to my earlier comment, I asked my platform team about this issue, and they confirmed our internal platform is not normalizing %2F to / anywhere. My platform team pointed me to look into nginx, which seems to be running in the config-ui.

At this time, my team has decided to change our project name delimiter from forward slashes (/) to underscores (_) as a workaround. Example: From my-team/my-ci/my-app to my-team_my-ci_my-app

I'll keep the issue open, as it seems at least one other person is facing the same issue.

[klesh]

Thanks for reporting the issue, to be triaged.

[github-actions]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions]

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.