google-fluentd looking for non-existent redis.conf file

[tkeemon]

Running the google-fluentd role on a Wharfhouse node produced this error:

TASK [google-fluentd : Copy custom configurations] *************************************************************

TASK [google-fluentd : Copy custom configurations] *************************************************************
changed: [35.237.211.232] => (item=apt_history.conf)
changed: [35.237.211.232] => (item=authlog.conf)
changed: [35.237.211.232] => (item=kernlog.conf)
changed: [35.237.211.232] => (item=letsencrypt.conf)
failed: [35.237.211.232] (item=redis.conf) => {"ansible_loop_var": "item", "changed": false, "item": "redis.conf", "msg": "Could not find or access 'redis.conf'\nSearched in:\n\t/home/tj/Programming/wharf/wharf/deploy/appsembler_roles/google-fluentd/templates/redis.conf\n\t/home/tj/Programming/wharf/wharf/deploy/appsembler_roles/google-fluentd/redis.conf\n\t/home/tj/Programming/wharf/wharf/deploy/appsembler_roles/google-fluentd/tasks/templates/redis.conf\n\t/home/tj/Programming/wharf/wharf/deploy/appsembler_roles/google-fluentd/tasks/redis.conf\n\t/home/tj/Programming/wharf/wharf/deploy/templates/redis.conf\n\t/home/tj/Programming/wharf/wharf/deploy/redis.conf on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}

I'm on the latest commit on develop: cde4ede

I think there are two issues here. First, there are two tasks with the same name:

roles/google-fluentd/tasks/main.yml

Lines 31 to 43 in cde4ede

	- name: Copy custom configurations
	copy:
	src: "{{ item }}"
	dest: "/etc/google-fluentd/config.d/{{ item | basename }}"
	backup: true
	with_items: "{{ google_fluent_custom_config_files }}"
	- name: Copy custom configurations
	template:
	src: "{{ item }}"
	dest: "/etc/google-fluentd/config.d/{{ item | basename | regex_search('(.+\\.conf)') }}"
	backup: true
	with_items: "{{ google_fluent_custom_config_templates }}"

In my experience, this causes Ansible to silently skip the first task (which it looks like happened here). If the older task isn't needed, then it should probably be deleted.

I'm having more trouble figuring out what's happening with the second issue (the actual failure). It looks like Ansible is looking locally for a redis.conf file and not finding it. I'm not sure why Ansible is looking for that file because there's no template with that name in the repo: https://github.com/appsembler/roles/tree/develop/google-fluentd/templates

I'm not sure how google_fluent_custom_config_templates is being set in the first place because its default value is an empty list:

roles/google-fluentd/defaults/main.yml

Line 8 in cde4ede

google_fluent_custom_config_templates: []

and we're not overriding that anywhere in the Wharf repo.

One thing worth noting is that the remote directory the config files are supposed to be copied into is full of files not created by Ansible:

tj@prod-itential-wharfhouse-5:~$ vi /etc/google-fluentd/config.d/
apache.conf             letsencrypt.conf        redis.conf
apt_history.conf        magento.conf            redmine.conf
authlog.conf            mediawiki.conf          salt.conf
cassandra.conf          memcached.conf          solr.conf
chef.conf               mongodb.conf            sugarcrm.conf
forward.conf            mysql.conf              syslog.conf
gitlab.conf             nginx.conf              syslog_endpoint.conf
jenkins.conf            postgresql.conf         tomcat.conf
jetty.conf              puppet.conf             zookeeper.conf
joomla.conf             puppet-enterprise.conf  
kernlog.conf            rabbitmq.conf           

[thraxil]

@tkeemon wharf is setting google_fluent_custom_config_templates here: https://github.com/appsembler/wharf/blob/develop/deploy/base_vars/all#L47

[thraxil]

Also, the first task is running, it just doesn't do anything because google_fluent_custom_config_files is an empty list. I've never seen ansible skip a task based on the name.