Problems compiling (warnings_as_errors) #23

Iulian7 · 2017-06-28T11:02:05Z

I am compiling on macOS Sierra, rebar 3.4.1 on Erlang/OTP 20 Erts 9.0 and Elixir 1.4.5.

I managed to compile but only after overriding some dependencies in mix.exs and manually removing warnings_as_errors from some rebar.config files (e.g. cuttlefish and a few others).

The relevant part of my mix.exs looks like this:

  defp deps do
    [
      {:riak_core, "~> 3.0", hex: :riak_core_ng},
      {:poolboy, "~> 1.5", override: true},
      {:cuttlefish, github: "basho/cuttlefish", tag: "2.0.11", manager: :rebar3, override: true},
      {:lager, github: "basho/lager", tag: "3.2.4", manager: :rebar3, override: true},
      {:goldrush, github: "basho/goldrush", tag: "0.1.9", manager: :rebar3, override: true}
    ]
  end

The warning I am getting from cuttlefish for example is export_all flag enabled - all functions will be exported. There are also quite a lot of deprication warnings (e.g. the 'random' module is deprecated; use the 'rand' module instead from basho_stats).

Are these problems due to an incompatibility with Erlang 20 or is there something else I should be doing? Editing the rebar config files of dependencies seems kind of wrong.

basho/cuttlefish#234

The text was updated successfully, but these errors were encountered:

Licenser · 2017-06-28T13:15:51Z

Hi @Iulian7,
this is a bug in mix. warning_as_errors should not lead to dependencies failing to compile, it should only evaluate it on the top level project.

Aside from fixing mix, the alternative is making all the dependencies compile w/o warning in Erlang 20. Which probably will eventually happen, but if you like to speed it up I suggest making PR's towards the dependencies in question. From we can do inside this repository, is only updating to newer versions of dependencies as they come along.

Licenser · 2017-06-28T13:16:21Z

Oh I forgot to say, I'm not yet on Erlang 20 so I don't know what problems will arise from it.

Iulian7 · 2017-07-19T21:30:46Z

I also realise the cuttlefish fork https://github.com/Kyorai/cuttlefish which fixes the warnings_as_errors does not compile with mix for another reason. Using {:cuttlefish, "~> 2.0", override: true} gives back the error Uncaught error: {badmatch,error}.

I am running DEBUG=1 mix deps.compile cuttlefish and getting this:

...
===> Building escript...
===> Creating escript file /Users/iulian/elixir/riak/deps/cuttlefish/_build/default/bin/cuttlefish
===> processing <<"cuttlefish">>
===> new deps of <<"cuttlefish">> found to be []
===> processing <<"getopt">>
===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace or consult rebar3.crashdump
===> Uncaught error: {badmatch,error}
===> Stack trace to the error location:
[{rebar_prv_escriptize,find_deps_of_deps,3,
                       [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_escriptize.erl"},
                        {line,246}]},
 {rebar_prv_escriptize,find_deps,2,
                       [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_escriptize.erl"},
                        {line,240}]},
 {rebar_prv_escriptize,escriptize,2,
                       [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_escriptize.erl"},
                        {line,104}]},
 {rebar_prv_escriptize,do,1,
                       [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_escriptize.erl"},
                        {line,80}]},
 {rebar_core,do,2,
             [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_core.erl"},
              {line,153}]},
 {rebar_hooks,run_provider_hooks_,6,
              [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_hooks.erl"},
               {line,49}]},
 {rebar_hooks,run_all_hooks,6,
              [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_hooks.erl"},
               {line,16}]},
 {rebar_prv_compile,compile,3,
                    [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_compile.erl"},
                     {line,129}]}]
...

It is worth noting that {:cuttlefish, github: "basho/cuttlefish", tag: "2.0.11", manager: :rebar3, override: true} compiles if I modify rebar.config accordingly.

More details about the environment (as per rebar3 report) for completeness:

-----------------
Operating System: x86_64-apple-darwin16.6.0
ERTS: Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:0] [hipe] [kernel-poll:false] [dtrace]
Root Directory: /usr/local/Cellar/erlang/20.0/lib/erlang
Library directory: /usr/local/Cellar/erlang/20.0/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.3.0
certifi: 0.4.0
cf: 0.2.2
common_test: 1.15
compiler: 7.1
crypto: 4.0
cth_readable: 1.2.6
dialyzer: 3.2
edoc: 0.9
erlware_commons: 1.0.0
eunit: 2.3.3
eunit_formatters: 0.3.1
getopt: 0.8.2
inets: 6.4
kernel: 5.3
providers: 1.6.0
public_key: 1.4.1
relx: 3.23.1
sasl: 3.0.4
snmp: 5.2.6
ssl_verify_fun: 1.1.1
stdlib: 3.4
syntax_tools: 2.1.2
tools: 2.10

-----------------
Escript path: /usr/local/bin/rebar3
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs release relup report shell state tar tree unlock update upg
rade upgrade upgrade version xref

Also, if I simply run rebar3 compile in deps/cuttlefish with {:cuttlefish, "~> 2.0", override: true} it compiles fine:

===> Verifying dependencies...
===> Fetching getopt ({pkg,<<"getopt">>,<<"0.8.2">>})
===> Downloaded package, caching at /Users/iulian/.cache/rebar3/hex/default/packages/getopt-0.8.2.tar
===> Fetching lager ({pkg,<<"lager">>,<<"3.2.1">>})
===> Downloaded package, caching at /Users/iulian/.cache/rebar3/hex/default/packages/lager-3.2.1.tar
===> Fetching goldrush ({pkg,<<"goldrush">>,<<"0.1.8">>})
===> Downloaded package, caching at /Users/iulian/.cache/rebar3/hex/default/packages/goldrush-0.1.8.tar
===> Compiling goldrush
===> Compiling lager
===> Compiling getopt
===> Compiling cuttlefish
===> Building escript...

Any ideas what might be going on or how this can be addressed?

Licenser · 2017-07-20T11:07:53Z

@Iulian7 have you asked the mix folks about it? It seems the package itself compiles fine but only files when compiled with mix instead rebar.

My knowledge about mix ends about at knowing what it rougly is, I could imagine it calls some old version of rebar3 perhaps? Or uses an outdated plugin. I am really not sure here.

Iulian7 · 2017-07-20T15:09:40Z

@Licenser Thanks for the pointers. It turns out the rebar3 version mix was using was slightly older. I fixed this but it was not the problem. I will write to the Elixir/mix guys and see if I get anything.

Licenser · 2017-07-21T12:13:56Z

Awesome, if there is anything we can do let us know!

Iulian7 · 2017-07-21T14:17:46Z

I did some more experimentation. For some of these I had to remove warnings_as_errors from rebar.config because some have not been changed for Erlang/OTP 20.

Configuration 1 (compiles):

 {:riak_core, "~> 3.0", hex: :riak_core_ng},
 {:cuttlefish, github: "basho/cuttlefish", tag: "2.0.11", manager: :rebar3, override: true},
 {:lager, github: "basho/lager", tag: "3.2.4", manager: :rebar3, override: true},
 {:goldrush, github: "basho/goldrush", tag: "0.1.9", manager: :rebar3, override: true}

Configuration 2 (does not work):

      {:riak_core, "~> 3.0", hex: :riak_core_ng},
      {:cuttlefish, "~> 2.0", override: true}, 
      {:lager, github: "basho/lager", tag: "3.2.4", manager: :rebar3, override: true},
      {:goldrush, github: "basho/goldrush", tag: "0.1.9", manager: :rebar3, override: true}

Basically it works with the initial basho/cuttlefish release 2.0.11 but not with Kyorai/cuttlefish release 2.0.12 (which as far as I can tell is what the hex package cuttlefish 2.0.12 is currently linked to). The 2.0.12 release compiles on Erlang 20 without modification (only with rebar3). For the 2.0.11 release I had to remove warnings_as_errors as mentioned.

I find the whole thing extremely strange to say the least and I would like to find out what exactly is going on. I compared the rebar.config file for both versions and they do differ.

cuttlefish 2.0.11 rebar.config

{require_otp_vsn, "R16|17|18"}.

{erl_opts, [warnings_as_errors, {parse_transform, lager_transform}, debug_info, warn_untyped_record]}.

{eunit_opts, [verbose]}.
{cover_enabled, true}.

{escript_emu_args, "%%! -escript main cuttlefish_escript -smp disable +A 0\n"}.
{escript_incl_apps, [goldrush, getopt, lager]}.

{xref_checks, []}.
{xref_queries, [{"(XC - UC) || (XU - X - B - \"(rebar.*|mustache)\" : Mod)", []}]}.

{deps, [
    {getopt, ".*", {git, "https://github.com/basho/getopt.git", {tag, "v0.8.2"}}},
    {lager, ".*", {git, "https://github.com/basho/lager.git", {tag, "3.2.4"}}},
    {neotoma, ".*", {git, "https://github.com/basho/neotoma.git", {tag, "1.7.4"}}}
  ]}.

{post_hooks, [
    {"-win32", compile, "rebar escriptize"},
    {"^((?!-win32).)*$", compile, "./rebar escriptize"}
  ]}.

cuttlefish 2.0.12 rebar.config

{minimum_otp_vsn, "R16"}.

{erl_opts, [warnings_as_errors,
           {parse_transform, lager_transform},
           debug_info,
           warn_untyped_record]}.

{deps, [getopt, lager]}.

{escript_emu_args, "%%! -escript main cuttlefish_escript -smp disable +A 0\n"}.
{escript_incl_apps, [goldrush, getopt, lager, cuttlefish]}.
{escript_main_app, cuttlefish}.

{provider_hooks, [{post, [{compile, {default, escriptize}}]}]}.

{eunit_opts, [verbose]}.
{cover_enabled, true}.

{profiles, [{dev, [{deps, [neotoma]},
                  {plugins, [rebar3_neotoma_plugin]}]}]}.

There are some things that differ in these configurations. As I am not that familiar with Erlang I can't really say what implications these would have.

Perhaps some more seasoned Erlang devs can comment on this.

Iulian7 · 2017-07-21T15:02:13Z

OK, the problem is with the post provider_hooks.

If I comment out {provider_hooks, [{post, [{compile, {default, escriptize}}]}]}. from rebar.config everything compiles fine. This escriptize hook doesn't seem to play well with mix. It's probably looking for some files in the wrong directories or something. Maybe...

I think this is related: erlang/rebar3#1560

Does anyone think this is an issue that needs to be taken up to the main Elixir guys? Or is something else going on here?

fire · 2017-08-14T03:51:40Z

To compile riak on erlang 20:

Add this to mix.exs.

{:riak_core, github: "Kyorai/riak_core", branch: "fifo-merge"},

Modify rebar.conf in poolboy to remove warnings as errors:

{erl_opts, [debug_info,
            {platform_define, "^[0-9]+", namespaced_types}]}.
{eunit_opts, [verbose]}.
{cover_enabled, true}.

Modify rebar.conf in cuttlefish to remove warnings as errors and provider_hooks:

{minimum_otp_vsn, "R16"}.

{erl_opts, [{parse_transform, lager_transform},
            debug_info,
            warn_untyped_record]}.

{deps, [getopt, lager]}.

{escript_emu_args, "%%! -escript main cuttlefish_escript -smp disable +A 0\n"}.
{escript_incl_apps, [goldrush, getopt, lager, cuttlefish]}.
{escript_main_app, cuttlefish}.

{eunit_opts, [verbose]}.
{cover_enabled, true}.

{profiles, [{dev, [{deps, [neotoma]},
                   {plugins, [rebar3_neotoma_plugin]}]}]}.

Modify rebar.conf in riak_ensemble to remove warnings as errors:

{deps, [{lager, "~>3.2.1"}]}.

{erl_opts, [debug_info,
           warn_untyped_record,
           {platform_define, "^[0-9]+", namespaced_types},
           {platform_define, "^((1[8|9])|2)", rand_module},
           {parse_transform, lager_transform}]}.

{eunit_opts, [verbose]}.
{edoc_opts, [preprocess,
             {dir, "edoc"}]}.
{cover_enabled, true}.
{xref_checks, [undefined_function_calls]}.

{profiles, [{test, [{deps, [{riak_test, ".*", {git, "git://github.com/basho/riak_test", {branch, "develop"}}}]}]}]}.

Modify rebar.conf in riak_core to remove warnings as errors:

{erl_first_files, ["src/gen_nb_server.erl", "src/riak_core_gen_server.erl",
		   "src/riak_core_stat_xform"]}.
%{cover_enabled, true}.
{erl_opts, [{parse_transform, lager_transform},
            debug_info, {platform_define, "^[0-9]+", namespaced_types},
            {platform_define, "18", old_rand},
            {platform_define, "17", old_rand},
            {platform_define, "^R15", old_hash}]}.
{edoc_opts, [{preprocess, true}]}.
{eunit_opts, [verbose]}.
{xref_checks, []}.
{xref_queries, [{"(XC - UC) || (XU - X - B - \"(cluster_info|dtrace)\" : Mod)", []}]}.

{deps, [
  %% Stuff we need to work with rebar3
  {goldrush, "~>0.1.8"},
  {cuttlefish, "~>2.0.12"},
  {clique, "~>0.3.9"},
  %% We now include folsom
  folsom,
  %% Normal deps
  {lager, "~>3.2.4"},
  {poolboy, "0.8.2", {pkg, basho_poolboy}},
  {basho_stats, "~>1.0.3"},
  {riak_sysmon, "~>2.1.3"},
  {riak_ensemble, "~>2.4.0", {pkg, riak_ensemble_ng}},
  {pbkdf2, "~>2.0.0"},
  {blume, "~>0.1.0"},
  {chash, "~>0.1.1"},
  {eleveldb, "~>2.2.20"},
  %%{eleveldb, {git, "https://github.com/project-fifo/eleveldb.git", {branch, "hex-2.2.19-diayzer"}}},
  {exometer_core, "~>1.0.0", {pkg, basho_exometer_core}}
]}.


%%-------------------------------------------------------------------
%% Profiles
%%-------------------------------------------------------------------

{profiles,
 [{test, [{deps,
           [{mustache, ".*", {git, "https://github.com/mojombo/mustache.erl.git", {tag, "v0.1.1"}}}]}]},
  {docs, [{deps, [{edown, "0.7.0"}]}]},
  {eqc, [{erl_opts, [{d, 'TEST'}]}, {deps, [meck]}, {plugins, [rebar_eqc]}]},
  {prod, [{relx, [{dev_mode, false}]}]},
  {lint,
   [{plugins,
     [{rebar3_lint,
       {git, "https://github.com/project-fifo/rebar3_lint.git",
        {tag, "0.1.4"}}}]}]}
 ]}.
{overrides,
 [
  %% R19
  {override, pooler,
   [{erl_opts,
     [{platform_define, "^[0-9]+", namespaced_types},
      debug_info,
      inline]}]},
  %% Normal
  {override, setup, [{post_hooks, []}]},
  {override, eleveldb,
   [{pre_hooks, [{compile, "c_src/build_deps.sh get-deps"},
                 {compile, "c_src/build_deps.sh"}]},

    {post_hooks, [{clean, "c_src/build_deps.sh clean"}]},

    {plugins,
     [pc]},

    {provider_hooks,
     [{post,
       [{compile, {pc, compile}},
        {clean, {pc, clean}}]}]}]}]}.

Make sure you have sudo apt-get install erlang-eunit erlang-os-mon.

I was unable to start riak though.

fire · 2017-08-14T04:01:15Z

Copying https://github.com/basho/riak_core/blob/develop/priv/riak_core.schema into priv/ I was able to get riak to run.

The other riak_core.schema didn't work.

Licenser · 2017-08-14T16:02:48Z

@fire what branch was that on? master or fifo-merge? If it's the fifo branch what was the error prompted?

Also, I think we really need a decent solution to deal with warn_on_errors, disabling them in all projects doesn't seem like a good solution :( of cause making them all error free would be the best :P

facundoolano · 2017-12-14T14:06:18Z

Oh I forgot to say, I'm not yet on Erlang 20 so I don't know what problems will arise from it.

@Licenser I'm trying it now with Erlang 20, and I have these errors:

poolboy has warnings_as_errors and uses export_all
riak_ensemble and riak_core have warnings_as_errors and uses gen_fsm

(I'm overriding erl_opts to work around them)

Licenser · 2017-12-15T09:08:12Z

removing warnings as errors is only a short-term remedy, those warnings will pop up as real errors eventually.

Sadly I'm a bit preoccupied with other things at the moment and don't have an erlang 20 installed, so unless someone wants to step up and properly fix the warnings and make a PR it will have to wait a bit until I get some air.

bieniusa · 2018-02-04T15:00:36Z

I would be happy to help in upgrading and giving back to this awesome community!
But since all these forks are really hard to follow for me, could you point me out where I find the poolboy repo that I should clone, fix and do a pull request on??
The link on hex is broken 😕 -> Ah, educated guess: There is an "o" missing in the url!

ghost · 2018-02-04T23:09:41Z

Also, I think we really need a decent solution to deal with warn_on_errors, disabling them in all projects doesn't seem like a good solution :( of cause making them all error free would be the best :P

Just a thought - but rebar3 now has the means to override properties in all child projects - would be ideal if mix could do the same - will create an issue and link back to this.

Licenser · 2018-02-05T12:03:11Z

@bieniusa sorry for the wrong link, I see you found it already and thanks for updating poolboy!!!

Licenser · 2018-02-05T12:10:56Z

since @bieniusa was so kind to get the erlang 20 ball rolling I added a https://github.com/Kyorai/riak_core/tree/erl20 branch to track it :)

Licenser · 2018-05-03T19:22:43Z

See #33

* Test with several slave nodes in a cluster (configured in config/test.exs). To prevent running RiakCore on the master node (harmless, but clutters the logs), invoke with `mix test --no-start` * Fixed a few warnings * Updated riak_core.schema Notes: * Running this project on Elixir 1.6.5 with Erlang 20.3.6. I had to comment out "warnings_as_errors" in `deps/riak_ensemble/rebar.config` and in `deps/riak_core/rebar.config` * This project uses riak_core_ng v3.0.9. Later riak_core_ng commits introduced gen_fsm_compat, which fails with newer Elixir+rebar3 because of "missing erl_vsn" issue: rebar_erl_vsn is a pre-compile hook in rebar.config of gen_fsm_compat and few other projects. Error seems to be caused by Mix not handling rebar3 hooks properly. See elixir-lang/elixir#7733 and Kyorai/riak_core#23 This issue is not specific to erl_vsn: for example, the forked https://github.com/gpad/cuttlefish (see mix.exs) differs from the official version only by rebar.config commenting out: % {provider_hooks, [{post, [{compile, {default, escriptize}}]}]}.

Licenser marked this as a duplicate of #20 Jul 21, 2017

Licenser mentioned this issue Jul 21, 2017

Failed to compile #20

Closed

Licenser marked this as a duplicate of #21 Jul 21, 2017

Licenser mentioned this issue Jul 21, 2017

Fails to compile with latest Rebar3 #21

Closed

Licenser mentioned this issue Aug 14, 2017

ideas how to deal with warnings_as_errors in dependencies erlang/rebar3#1607

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Problems compiling (warnings_as_errors) #23

Problems compiling (warnings_as_errors) #23

Iulian7 commented Jun 28, 2017 •

edited

Loading

Licenser commented Jun 28, 2017

Licenser commented Jun 28, 2017

Iulian7 commented Jul 19, 2017

Licenser commented Jul 20, 2017

Iulian7 commented Jul 20, 2017

Licenser commented Jul 21, 2017

Iulian7 commented Jul 21, 2017 •

edited

Loading

Iulian7 commented Jul 21, 2017 •

edited

Loading

fire commented Aug 14, 2017

fire commented Aug 14, 2017

Licenser commented Aug 14, 2017 •

edited

Loading

facundoolano commented Dec 14, 2017 •

edited

Loading

Licenser commented Dec 15, 2017

bieniusa commented Feb 4, 2018 •

edited

Loading

ghost commented Feb 4, 2018

Licenser commented Feb 5, 2018

Licenser commented Feb 5, 2018

Licenser commented May 3, 2018

Problems compiling (warnings_as_errors) #23

Problems compiling (warnings_as_errors) #23

Comments

Iulian7 commented Jun 28, 2017 • edited Loading

Licenser commented Jun 28, 2017

Licenser commented Jun 28, 2017

Iulian7 commented Jul 19, 2017

Licenser commented Jul 20, 2017

Iulian7 commented Jul 20, 2017

Licenser commented Jul 21, 2017

Iulian7 commented Jul 21, 2017 • edited Loading

cuttlefish 2.0.11 rebar.config

cuttlefish 2.0.12 rebar.config

Iulian7 commented Jul 21, 2017 • edited Loading

fire commented Aug 14, 2017

fire commented Aug 14, 2017

Licenser commented Aug 14, 2017 • edited Loading

facundoolano commented Dec 14, 2017 • edited Loading

Licenser commented Dec 15, 2017

bieniusa commented Feb 4, 2018 • edited Loading

ghost commented Feb 4, 2018

Licenser commented Feb 5, 2018

Licenser commented Feb 5, 2018

Licenser commented May 3, 2018

Iulian7 commented Jun 28, 2017 •

edited

Loading

Iulian7 commented Jul 21, 2017 •

edited

Loading

Iulian7 commented Jul 21, 2017 •

edited

Loading

Licenser commented Aug 14, 2017 •

edited

Loading

facundoolano commented Dec 14, 2017 •

edited

Loading

bieniusa commented Feb 4, 2018 •

edited

Loading