When launching a Kubernetes cluster, everything appears fine. When trying to add an additional node via the actions>scaleup nodes option, a new vm is created, registered in etcd on the kuberntes masternode, and then everything seems to stall (i.e. no more progress is made). The new vm has been registred on the mater and master-add-node.sh is called (but nothing after that in the murano-agent log exsits), and only the initial download of the murano-agent configuration settings happens on the new vm (i.e. no instructions passed the "these are the login and connection information" stage to setup murano.conf in /etc/..).
Furthermore, no real logs seem to show up on the controller logs for murano-engine or murano-api.
/var/log/murano/murano-engine.log after initializing a scaleup action:
log_http_response /usr/lib/python2.7/site-packages/muranoclient/common/http.py:124
2015-09-09 14:57:57.469 8987 DEBUG murano.dsl.executor [-] 0a88641ca9294ca1bf10b5b8642a4fd0: Begin execution: io.murano.system.Resources.yaml (-7294027655241992074) called from File "/tmp/murano-packages-cache/2f7479e9-a241-4954-a6d0-32e72497f450/io.murano.apps.docker.kubernetes.KubernetesCluster/Classes/KubernetesCluster.yaml", line 123:13 in method deploy of class io.murano.apps.docker.kubernetes.KubernetesCluster
$.minionNodes.take($.nodeCount).select($.setupEtcd()) _invoke_method_implementation /usr/lib/python2.7/site-packages/murano/dsl/executor.py:142
2015-09-09 14:57:57.470 8987 DEBUG murano.dsl.executor [-] 0a88641ca9294ca1bf10b5b8642a4fd0: End execution: io.murano.system.Resources.yaml (-7294027655241992074) _invoke_method_implementation /usr/lib/python2.7/site-packages/murano/dsl/executor.py:160
2015-09-09 14:57:57.471 8987 DEBUG murano.dsl.executor [-] 0a88641ca9294ca1bf10b5b8642a4fd0: Begin execution: io.murano.apps.docker.kubernetes.KubernetesNode.getIp (-6747789967329102446) called from File "/tmp/murano-packages-cache/2f7479e9-a241-4954-a6d0-32e72497f450/io.murano.apps.docker.kubernetes.KubernetesCluster/Classes/KubernetesCluster.yaml", line 123:13 in method deploy of class io.murano.apps.docker.kubernetes.KubernetesCluster
$.minionNodes.take($.nodeCount).select($.setupEtcd()) _invoke_method_implementation /usr/lib/python2.7/site-packages/murano/dsl/executor.py:142
2015-09-09 14:57:57.471 8987 DEBUG murano.dsl.executor [-] 0a88641ca9294ca1bf10b5b8642a4fd0: End execution: io.murano.apps.docker.kubernetes.KubernetesNode.getIp (-6747789967329102446) _invoke_method_implementation /usr/lib/python2.7/site-packages/murano/dsl/executor.py:160
2015-09-09 14:57:57.473 8987 DEBUG murano.dsl.executor [-] 0a88641ca9294ca1bf10b5b8642a4fd0: Begin execution: io.murano.system.Agent.call (-7715528276152306858) called from File "/tmp/murano-packages-cache/2f7479e9-a241-4954-a6d0-32e72497f450/io.murano.apps.docker.kubernetes.KubernetesCluster/Classes/KubernetesCluster.yaml", line 123:13 in method deploy of class io.murano.apps.docker.kubernetes.KubernetesCluster
$.minionNodes.take($.nodeCount).select($.setupEtcd()) _invoke_method_implementation /usr/lib/python2.7/site-packages/murano/dsl/executor.py:142
2015-09-09 14:58:08.149 8987 DEBUG murano.engine.system.agent_listener [-] Got execution result: id 'ece45aa739c4482bb4d83db2922ca5d6' body '{u'Body': u'gateway-1=http://200.10.0.4:7001,MyVM-3=http://200.10.0.6:7001,MyVM-1=http://200.10.0.2:2380,MyVM-1=http://200.10.0.2:7001,MyVM-2=http://200.10.0.5:7001,MyVM-4=http://200.10.0.7:7001', u'SourceID': u'ece45aa739c4482bb4d83db2922ca5d6', u'ErrorCode': 0, u'FormatVersion': u'2.0.0', u'Time': u'2015-09-09 19:58:07.902294', u'Action': u'Execution:Result', u'ID': u'd00f7481e45c4013874b0bdf92044c87'}' _receive /usr/lib/python2.7/site-packages/murano/engine/system/agent_listener.py:88
In this case, MyVM-4 is the newly created vm to add into the cluster. This is running in Openstack Juno. The interface just sits there with "Configureing etcd node MyVM-4" with the status Deploy in progress, this will remain for hours until I manually delete the stack and remove the environment from the mysql database manually (abort still doesn't exist in the murano environment yet!).