persistence of firewall rules

I just noticed on the QA cluster that the containers were unable to reach each other as the iptable rules were somehow reset. Docker sets up custom iptables rules to enable the container’s virtual network interfaces to connect with the machine’s network interface. Since these rules were removed, the containers could no longer reach each other. We must

  1. persist iptable rules created by the framework
  2. add restart-policy management inside all of the containers, so that restarting the docker daemon on each machine would re-create the services

This behavior has not been observed in the sites where SIMPLE has been deployed. The current fix is to rollback to deploy, pre-deploy, config (on CM and LC) stages and re-run config, pre-deploy and deploy.

The following issue can be used to track progress:

The firewall resets must be due to something CERN-specific (Puppet left-over?) on those nodes, as their OS rather is CC7 instead of CentOS 7…