template sources seem to not get merged, only the last one is used

[EugenMayer]

This is the common.yaml file: https://goo.gl/osCnxs
i have this template present local as file:

ls /etc/tiller/templates
mysql.erb

Since i added "consul" to the template_sources (plural) the local file /etc/tiller/templates/mysql.erb is no longer even recognised, no matter, if consul has a template of this name or not under configuration/templates/sources/mysql.erb

If i add a consul template, the file gets generated.

I do not get, why several sources are able to get used, but the values are not merged?

[EugenMayer]

interestingly, flipping the values to

template_sources: [ "consul", "file" ]

does work consisten, this means:

• if the file i available locally, use it
• if the file is not available localy, ask consul

Since this behaviour makes sense and is the way you describe the order ( right superseeds left ) the describe issue above is a bug, since

template_sources: [ "file", "consul" ]

Does work as "exclusively consul, never use file"

[markround]

Checking this out now, thanks for the report!

[markround]

Hmmm. I'm having some trouble replicating this. Here's an example common.yaml:

---
consul:
  url: "http://localhost:8500"
data_sources: [ "defaults" , "file" , "consul" ]
template_sources: [ "file" , "consul" ]

environments:
  development:
    local.erb:
      target: local.txt

My Consul is populated with the same test data as used in the fixtures. I then run Tiller:

$ ./bin/tiller -b ./t -n -v 
tiller v0.7.9 (https://github.com/markround/tiller) <[email protected]>
Using common.yaml v2 format configuration file
Using configuration from ./t
Using plugins from /usr/local/lib/tiller
Using environment development
Template sources loaded [FileTemplateSource, ConsulTemplateSource]
Data sources loaded [DefaultsDataSource, FileDataSource, ConsulDataSource]
Available templates : ["local.erb", "template1.erb", "template2.erb"]
Building template local.erb  <------ LOCAL TEMPLATE GETTING BUILT
Not running as root, so not setting ownership/permissions on local.txt
Warning, merging duplicate data values.
consul_per_env => 'per-env global for development enviroment' being replaced by : 'This is over-written for template1 in development' from ConsulDataSource
Warning, merging duplicate target values.
target => 'template1-local.txt' being replaced by : 'template1.txt' from ConsulDataSource
Building template template1.erb
Not running as root, so not setting ownership/permissions on template1.txt
Building template template2.erb
Not running as root, so not setting ownership/permissions on template2.txt
Template generation completed

$ cat local.txt
This is a local template from the file template source.
Here is a global value form consul : per-env global for development enviroment

This appears to be working as intended - my definition for local.erb comes from the file data-source, and is not present anywhere in Consul, so it gets used and picked up from the file template source. If I swap the order, all that happens is that the Consul templates get built first, before my local.erb.

Do you have any other values (target or otherwise) at all for mysql.erb in Consul ?

[EugenMayer]

remove the consul template, your local template will not be picked up

[EugenMayer]

@markround sorry, missed that you did not had any consul template. I guess what you should add, eventhough it should not be the cause, is the custom template confuguration.

And now, consul is entirely empty during my test, not a single value in there. Add this to your consul block:

  templates: '/configuration/templates/sources'
  values:
    global: '/configuration/globals/all'
    per_env: '/configuration/globals/%e'
    template: '/configuration/services/%e/%t'
    target: '/configuration/templates/targets/%t/%e'

[markround]

Hmmm. This is strange. I have now deleted everything from Consul (K/V store is totally empty), My common.yaml now looks like:

---
consul:
  url: "http://localhost:8500"
  templates: '/configuration/templates/sources'
  values:
    global: '/configuration/globals/all'
    per_env: '/configuration/globals/%e'
    template: '/configuration/services/%e/%t'
    target: '/configuration/templates/targets/%t/%e'

data_sources: [ "defaults" , "file" , "consul" ]
template_sources: [ "consul" , "file" ]

environments:
  development:
    local.erb:
      target: local.txt

And yet....

$ ./bin/tiller -b ./t -n -v 
tiller v0.7.9 (https://github.com/markround/tiller) <[email protected]>
Using common.yaml v2 format configuration file
Using configuration from ./t
Using plugins from /usr/local/lib/tiller
Using environment development
Template sources loaded [ConsulTemplateSource, FileTemplateSource]
Data sources loaded [DefaultsDataSource, FileDataSource, ConsulDataSource]
Consul : No templates could be fetched from /configuration/templates/sources
Available templates : ["local.erb"]
Building template local.erb
Not running as root, so not setting ownership/permissions on local.txt
Template generation completed

[EugenMayer]

ok, seems like you cannot reproduce it. Let me try to either create a docker image with the test example or try to fiddle out wich detail is the cause for this. thanks for the effort, i report back!

it would be great if you could have a look at #26 also

[EugenMayer]

@markround tried to reproduce - but i could not. It works as desired. I cannot really explain why this is not reproduceable, but it worked out here - so this issue can be closed sorry. #26 is pretty severe though

[EugenMayer]

closing this for now, until i can reproduce this. Thank you for your effort and sorry