automate update of certifi

[benoitc]

we should automate the update of certifi to make sure we are using the last updates from Mozilla.

cc @fenollp

[fenollp]

That's really just up to the TravisCI owner: https://docs.travis-ci.com/user/cron-jobs/
Once that job is set up & there's a diff the build will fail. Maybe there's an email notif but we should maybe look at something like IfThisThenThat to trigger the openning of an issue or directly of a PR.

I don't know a good solution but I'm very interested in something that's automatic! (while safe).

[benoitc]

i think it should be feasible to create a release directly sent to hex.pm one tagged. I will have a look

[benoitc]

looking at #32 i wonder if we could not have some kind of warning system included in certifi instead of rebuilding it:

• certifi is updated regularly using an update hooks
• a background daemon would check the last version in hex or something and log a warning if it needs to be updated. Then up to the user it would redeploy the app or dl a new file and do what Automatic updates at run time #32 aims to provide.

Thoughts?

[fenollp]

I think a @daily build of master that cuts a minor release if there's a non-empty git diff (so no user interaction) would be best.
Not sure which kind of scrutiny you want here. I'd prefer fully automated/open/inpectable builds & deploys personally.

[paulo-ferraz-oliveira]

Two things to consider in the scope of this Issue:

a. automatic updates via cron aren't working since .travis.yml states TRAVIS_OTP_RELEASE=20.2, which is not declared in otp_release (in the same file)
b. it is possible that erlang-certifi stops using Travis CI soon

This Issue requires a refresh 😄

[paulo-ferraz-oliveira]

Regarding 2., which now seems closer to the complete objective (getting rid of Travis CI), how does this affect 1.? Do we still want certifi to build automatically? I'm assuming the goal is to:

1. check the "ca bundle" origin,
2. build, tag (git), publish (Hex) and release (GitHub)
   Also, what does this mean for consumers of master. Do we assume that everybody's using a "lock" file and somehow updating from master is considered innocuous?

[tsloughter]

Is this still an issue?

I see the repo is now using github actions but doesn't have an action for a daily check of certs to make an update.

My suggestion would be an action that runs daily and opens a PR with the update. That would then notify the maintainers of changes, allowing them to cut a release to hex after merging, while not being an automatic publish to hex that could go wrong :)

[benoitc]

Thanks for the suggestion but I don't want any automatic build using github actions or anything requiring an external service not in my control or better user control.

I have developed some new service that will be soon available to handle such updates and do it directly from any release system and notify the user. It's already deployed for customers, i will open it ASAP. Then maybe there will be a way to use github actions to do it though I am focusing in ensuring users doesn't depend on any external dependency for their build and updates.

[tsloughter]

Cool. But I would argue it isn't really an external service since it is the same service hosting the code and running the CI already :). And it is just a daily job to diff and open a PR if necessary, sounds like overkill to require an actual external service with full control for that.

Using this method is also what I found castore to be doing when I went looking for examples https://github.com/elixir-mint/castore/blob/master/.github/workflows/outdated.yml

[paulo-ferraz-oliveira]

@tsloughter, as of late Sep'21, https://github.com/g-andrade/tls_certificate_check/pulls?q=is%3Apr+is%3Aclosed is doing what I believe is what you're suggesting. Maybe erlang-certifi could do the same? ie. if I got your suggestion write, the automation would be in "importing the certificates", "creating a pull request", with tests (they won't run automatically, either) and merged by a human (no automatic releases).

[tsloughter]

Yea, looks like it.

[benoitc]

Cool. But I would argue it isn't really an external service since it is the same service hosting the code and running the CI already :). And it is just a daily job to diff and open a PR if necessary, sounds like overkill to require an actual external service with full control for that.

Using this method is also what I found castore to be doing when I went looking for examples https://github.com/elixir-mint/castore/blob/master/.github/workflows/outdated.yml

Right. We can do that indeed. There are not that many updates (unless you're trying to do weird stuff in incorporating the certificates) so that should work.

By itself I think it should also be interresting to not rely totally on mkcert.org but that a second step. I will work on a change during the week unless someone take care of it first.

[benoitc]

@tsloughter, as of late Sep'21, https://github.com/g-andrade/tls_certificate_check/pulls?q=is%3Apr+is%3Aclosed is doing what I believe is what you're suggesting. Maybe erlang-certifi could do the same? ie. if I got your suggestion write, the automation would be in "importing the certificates", "creating a pull request", with tests (they won't run automatically, either) and merged by a human (no automatic releases).

If an action can be triggerred by a webhook then maybe we can pass also some information to validate automatically that the bundle signature is OK during tests and then automatically trigger a release. There is no need for an intermediate action to merge a PR since I already have an automatic way to be notified when the bundle is updated. I will look if we can do that way.

[tofran]

@benoitc Regarding mkcert.org: why not just use the well trusted CCADB directly?

[neuromantik33]

We're currently encountering a show-stopper as letsencrypt root CA expired as of 30-09-2021 and it is impossible to make HTTPS requests to sites signed with acme's generated letsencrypt certificates. An update of this library is very important!

[benoitc]

@neuromantik33 what version your library is using? also please open a separate issue for it please.

Current cert bundle include "ISGR Root X1" so it should pass.

➜  erlang-certifi git:(master) grep -r -e "ISRG Root X1" priv/cacerts.pem
priv/cacerts.pem:# Issuer: CN=ISRG Root X1 O=Internet Security Research Group
priv/cacerts.pem:# Subject: CN=ISRG Root X1 O=Internet Security Research Group
priv/cacerts.pem:# Label: "ISRG Root X1"

[benoitc]

@benoitc Regarding mkcert.org: why not just use the well trusted CCADB directly?

certifi projects was based on mkcert.org. But we can revise it