DOC: Add core migration guide

[seanpmorgan]

Closes #686

[bhack]

What will be the pipeline from keras-contrib to core?
See keras-team/keras-contrib#519

[seanpmorgan]

What will be the pipeline from keras-contrib to core?
See keras-team/keras-contrib#519

Hi @bhack thanks for bringing this up! So given that keras-contrib is being deprecated I think a logical path is for contributions to be submitted to addons prior to requesting a move to Keras core.

I checked some issues on keras-contrib and most issues are without responses (last PR was in the summer) so AFAIK there should be no more submissions to keras-contrib as PRs. Does that conflict with your understanding?

[bhack]

IMHO we could put keras-contrib in read only mode as soon as it is possible.

But then it will hard to explain to user that you need to go here in a Tensorflow repo to then go on Keras core that It Is not in TF owned repo but still Tensorflow only API from April 2020.

What kind of narrative we are building for new users?

[bhack]

Do we have any kind of telemetry?

[seanpmorgan]

#236 needs to be completed. We were/are waiting for adoption to pick up along with TF2 adoption:
https://pypistats.org/packages/tensorflow-addons

[seanpmorgan]

Agree that it is a splintered experience for new users. Some questions that I can not provide answers to:

• What needs to be done in order to sunset keras-contrib? Some appropriate documentation and limit pull requests to bug fixes only? It's an obvious place for new users to land because of it's name and history. cc @gabrieldemarmiesse
• Will tensorflow/keras repository be the new location for modular tf.keras? If so, I think that helps to unify where users should look for complimentary repositories. cc @karmel @fchollet

Ultimately it's something I think we need to streamline... but this particular PR is the path from Addons to TF-core and I don't think that process changes based on the answers to the above questions.

[bhack]

@seanpmorgan I agree with your points but I really don't understand when Keras it will be only an API for Tensorflow (from april 2020) what tf.keras will be?
And I think it will impact this PR cause if keras-contib it will be defacto Addons I think that we need to really understand what is storytelling here.
Is Addons in the ideal staging queue for Keras core like the old keras-contrib or it will be only for Tf.Keras or what else?
It is really hard to create a linear story telling at the point where Keras will be only for Tensorflow and anymore a "vendor neutral" frontend API.
I think that forcibly keeping a concept originated by the history of the framework when the multibackeds nature is changed now it is going only to generate confusion.

[gabrieldemarmiesse]

@seanpmorgan , to answer your questions:

• I'll put the repo in read-only mode. I didn't know it was possible. Thanks for the suggestion. In the main page, it's said that we're migrating, so it's clear to the users that they should go to tensorflow/addons. Our workforce is very small, so we can't continue to maintain this repo anyway.
• tf.keras will live in keras-team/keras, not in tensorflow/keras. Doing pip install keras will install tf.keras. In the future, if you do import keras, it'll have the same effect as import tensorflow.keras. We have a messy history, sure, but we need to abandon the multi-backend codebase. It's better for the long term.

If you have any other questions, feel free to ask.

[gabrieldemarmiesse]

Actually I can't archive keras-contrib, I don't think I have the rights. I changed the readme to make it clear that this repo is deprecated.

[bhack]

I am ok to abandon the multi-backend but I really don't understand ehy Keras needs to be under another org if It will be Tensorflow only.
All other sigs are under Tensorflow.

[gabrieldemarmiesse]

Looks good to me.

[gabrieldemarmiesse]

We don't want Keras to be tied to Google's rules and processes. We want it to be really open source. Unlike tensorflow, google won't have an internal version of keras. Everything will be public, on the github repo. That's the direction here. We'll see how it turns out in practice.

[bhack]

@gabrieldemarmiesse So what we will do with https://github.com/tensorflow/tensorflow/labels/comp%3Akeras ?
How we will avoid the natural behavior to create these there?

[gabrieldemarmiesse]

It's similar to when users open issues on scipy and those issues get migrated to numpy. We're not the only ones having the task of dispatching issues. For exemple, people might open an issue in the torchvision repo and this issue gets redirected to pytorch. Do you know a way to handle it automatically? El jue., 19 dic. 2019 21:54, bhack <[email protected]> escribió:

…

@gabrieldemarmiesse <https://github.com/gabrieldemarmiesse> So what we will do with https://github.com/tensorflow/tensorflow/labels/comp%3Akeras ? How we will avoid the natural behavior to create these there? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#774?email_source=notifications&email_token=ADCLMK3DMCLWE7FNVSX74Y3QZPNSFA5CNFSM4J3BYIZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHLABFI#issuecomment-567672981>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADCLMK7VQKEAFSDY3MK7X53QZPNSFANCNFSM4J3BYIZA> .

[bhack]

It Is not the same IMHO as scipy or Torchvision examples.
Tensorflow had an heavy marketing run over months about "Keras It is now our official high level API ".
It will be very hard to tell to the users/devs that now Keras it Is an optional external project again and so "please interact with that project and not with Tensorflow directly".
Just my 2c

[bhack]

Also It will be quite complex to request to users/devs to split issues/PRs between Keras and TF repo when the feautre It Is going to impact or request TF changes.

[gabrieldemarmiesse]

Well, it may be an issue, we'll see with the time. @bhack Do you have another organisation/process in mind that would be better?

[bhack]

I honestly don't know how we got into this situation because I didn't attend the internal strategica meetings about Keras roadmap.
I have my opinion that I've expressed in other tickets like adopting Keras namespace across subprojects and use the Keras brand as "Keras API design philosophy" or instead simply let disappear the brand cause It Is now only defined in a python context.

I.e. Things like the one below without a Keras branded namespace IMHO it Is not going to be a robust storytelling as Keras will be soon totally dedicated to the Tensorflow project:

The Layers API of TensorFlow.js is modeled after Keras and we strive to make the Layers API as similar to Keras as reasonable given the differences between JavaScript and Python.

[seanpmorgan]

@seanpmorgan , to answer your questions:

• I'll put the repo in read-only mode. I didn't know it was possible. Thanks for the suggestion. In the main page, it's said that we're migrating, so it's clear to the users that they should go to tensorflow/addons. Our workforce is very small, so we can't continue to maintain this repo anyway.

@gabrieldemarmiesse Thanks! The new README page seems clear to me

• tf.keras will live in keras-team/keras, not in tensorflow/keras. Doing pip install keras will install tf.keras. In the future, if you do import keras, it'll have the same effect as import tensorflow.keras. We have a messy history, sure, but we need to abandon the multi-backend codebase. It's better for the long term.

Thanks for the info. IMO its a bit confusing, but I understand the desire to keep it as a fully open-source project with its own governance.

@gabrieldemarmiesse So what we will do with https://github.com/tensorflow/tensorflow/labels/comp%3Akeras ?
How we will avoid the natural behavior to create these there?

Not exactly a fix, but because there is a paid team to manage/triage issues on TF-core, end users should at least be directed to the correct place after review.

[seanpmorgan]

Thanks for the review all! I do think this has diverged a bit from the PR... probably best to discuss these roadmap decisions on keras-governance.

[gabrieldemarmiesse]

Sorry about invading this PR @seanpmorgan . And thanks for the PR :)

[bhack]

The governance original thread was at keras-team/governance#2 but nobody reply.

[tanzhenyu]

@seanpmorgan @gabrieldemarmiesse @bhack
keras-team/governance#22
keras-team/governance#23

-- I think we are meant to schedule a meeting to discuss this, not sure why it hasn't happen yet, meanwhile JFYI

[bhack]

@tanzhenyu I am ok with this. Waiting for a @seanpmorgan free slot.

[seanpmorgan]

@tanzhenyu @bhack We have our monthly meeting this Thursday (Sept 3rd 11am PT):
https://docs.google.com/document/d/1kxg5xIHWLY7EMdOJCdSGgaPu27a9YKpupUz2VTXqTJg/edit#

Do you think we could cover this with @karmel and your team during that or would you prefer a focused meeting to discuss?

[tanzhenyu]

@tanzhenyu @bhack We have our monthly meeting this Thursday (Sept 3rd 11am PT):
https://docs.google.com/document/d/1kxg5xIHWLY7EMdOJCdSGgaPu27a9YKpupUz2VTXqTJg/edit#

Do you think we could cover this with @karmel and your team during that or would you prefer a focused meeting to discuss?

I believe it's be great to have separate focused meeting for both of them, so whoever is interested in cv or nlp can attend. But if not, 09/03/20 11am PST works for me -- Can you invite me (so I will invite Francois)