Disable dynamic ES mapping and fix test fixtures

[MrOrz]

This PR updates src/schema to latest and fix unit test fixtures that would insert extra fields by accident.

Migration needed

Extra fields that needs removal

export ES_HOST=localhost:62223

# articles: 3~15 minutes
curl "$ES_HOST/articles/doc/_update_by_query?conflicts=proceed" -H 'content-type:application/json' -d '{"script":{"source":"ctx._source.remove(\"appUserId\"); ctx._source.remove(\"tags\"); ctx._source.remove(\"normalArticleReplycount\"); for (int i = 0; i < ctx._source.articleCategories.length; ++i) { ctx._source.articleCategories[i].remove(\"appUserId\"); } for (int i = 0; i < ctx._source.references.length; ++i) { ctx._source.references[i].remove(\"appUserId\"); } for (int i = 0; i < ctx._source.articleReplies.length; ++i) { ctx._source.articleReplies[i].remove(\"appUserId\"); }"}}'

# articlereplyfeedbacks: 2 minutes
curl "$ES_HOST/articlereplyfeedbacks/doc/_update_by_query?conflicts=proceed" -H 'content-type:application/json' -d '{"script":{"source":"ctx._source.remove(\"appUserId\");"},"query":{"exists": {"field":"appUserId"}}}'

# replyrequests: 42s
curl "$ES_HOST/replyrequests/doc/_update_by_query?conflicts=proceed" -H 'content-type:application/json' -d '{"script":{"source":"ctx._source.remove(\"appUserId\");"},"query":{"exists": {"field":"appUserId"}}}'

# user special case - weird nameless situation
curl -XPOST "$ES_HOST/users/doc/CAdXE3EBrhVJn3LNMoiA/_update" -H 'content-type:application/json' -d '{"doc":{"name":""}}'

# user special case - blocked by accident
curl -XPOST "$ES_HOST/users/doc/_bdjyJIBDfH0hhhZtJMk/_update" -H 'content-type:application/json' -d '{"script":{"source":"ctx._source.remove(\"blockedReason\");"}}'

Also need to manually reload (reindex) all db indexes. May need to take down API servers so that no insertion happens during reindex process (prevent new docs inserted into old index during re-index).

Stats on localhost DB & staging:

• articlecategoryfeedbacks, categories, cooccurrences: quite fast
• airesponses: 30~45 secs
• articlereplyfeedbacks: 42 secs
• replyrequests: 45s
• ydocs: 4s ~ 1 min
• replies: 1.5~7 mins
• users: 32 secs
• articles: 2min
• analytics: 25 mins
• urls:

[coveralls]

coverage: 83.365% (-5.1%) from 88.512%
when pulling 55b5710 on strict-schema
into 7b2d03a on master.

[MrOrz]

Migration script executed; all indices reloaded.

docker@debian:~/rumors-db$ npm run reload -- articlecategoryfeedbacks

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "articlecategoryfeedbacks"

Source:  articlecategoryfeedbacks_v1_1_1
Target:  articlecategoryfeedbacks_v1_1_2
Reindexed from articlecategoryfeedbacks_v1_1_1 to articlecategoryfeedbacks_v1_1_2 in 1 seconds.
Setup articlecategoryfeedbacks_v1_1_2 -> articlecategoryfeedbacks alias and remove articlecategoryfeedbacks_v1_1_1.
docker@debian:~/rumors-db$ npm run reload -- categories

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "categories"

Source:  categories_v1_0_2
Target:  categories_v1_1_1
Reindexed from categories_v1_0_2 to categories_v1_1_1 in 0 seconds.
Setup categories_v1_1_1 -> categories alias and remove categories_v1_0_2.
docker@debian:~/rumors-db$ npm run reload -- cooccurrences

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "cooccurrences"

Source:  cooccurrences_v1_0_0
Target:  cooccurrences_v1_0_1
Reindexed from cooccurrences_v1_0_0 to cooccurrences_v1_0_1 in 0 seconds.
Setup cooccurrences_v1_0_1 -> cooccurrences alias and remove cooccurrences_v1_0_0.
docker@debian:~/rumors-db$ npm run reload -- airesponses

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "airesponses"

Source:  airesponses_v1_0_0
Target:  airesponses_v1_0_1
Reindexed from airesponses_v1_0_0 to airesponses_v1_0_1 in 23 seconds.
Setup airesponses_v1_0_1 -> airesponses alias and remove airesponses_v1_0_0.
docker@debian:~/rumors-db$ npm run reload -- articlereplyfeedbacks

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "articlereplyfeedbacks"

Source:  articlereplyfeedbacks_v1_2_0
Target:  articlereplyfeedbacks_v1_2_1
Reindexed from articlereplyfeedbacks_v1_2_0 to articlereplyfeedbacks_v1_2_1 in 23 seconds.
Setup articlereplyfeedbacks_v1_2_1 -> articlereplyfeedbacks alias and remove articlereplyfeedbacks_v1_2_0.
docker@debian:~/rumors-db$ npm run reload -- replyrequests

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "replyrequests"

Source:  replyrequests_v1_1_1
Target:  replyrequests_v1_1_2
Reindexed from replyrequests_v1_1_1 to replyrequests_v1_1_2 in 24 seconds.
Setup replyrequests_v1_1_2 -> replyrequests alias and remove replyrequests_v1_1_1.
docker@debian:~/rumors-db$ npm run reload -- ydocs

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "ydocs"

Source:  ydocs_v1_0_1
Target:  ydocs_v1_0_2
Reindexed from ydocs_v1_0_1 to ydocs_v1_0_2 in 2 seconds.
Setup ydocs_v1_0_2 -> ydocs alias and remove ydocs_v1_0_1.
docker@debian:~/rumors-db$ npm run reload -- replies

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "replies"

Source:  replies_v1_1_0
Target:  replies_v1_1_1
Reindexed from replies_v1_1_0 to replies_v1_1_1 in 53 seconds.
Setup replies_v1_1_1 -> replies alias and remove replies_v1_1_0.
docker@debian:~/rumors-db$ npm run reload -- users

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "users"

Source:  users_v1_2_1
Target:  users_v1_2_2
Reindexed from users_v1_2_1 to users_v1_2_2 in 17 seconds.
Setup users_v1_2_2 -> users alias and remove users_v1_2_1.
docker@debian:~/rumors-db$ npm run reload -- articles

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "articles"

Source:  articles_v1_4_0
Target:  articles_v1_4_1
Reindexed from articles_v1_4_0 to articles_v1_4_1 in 54 seconds.
Setup articles_v1_4_1 -> articles alias and remove articles_v1_4_0.
docker@debian:~/rumors-db$ 
docker@debian:~/rumors-db$ npm run reload -- analytics

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "analytics"

Source:  analytics_v1_2_0
Target:  analytics_v1_2_1

Reindexed from analytics_v1_2_0 to analytics_v1_2_1 in 724 seconds.
Setup analytics_v1_2_1 -> analytics alias and remove analytics_v1_2_0.
docker@debian:~/rumors-db$ 
docker@debian:~/rumors-db$ 
docker@debian:~/rumors-db$ npm run reload -- urls

> [email protected] reload
> babel-node --extensions .ts,.js db/reloadSchema.js "urls"

Source:  urls_v1_1_0
Target:  urls_v1_1_1
Reindexed from urls_v1_1_0 to urls_v1_1_1 in 251 seconds.
Setup urls_v1_1_1 -> urls alias and remove urls_v1_1_0.