Added cilium support (#2236)

* Added cilium support

* Fix typo in debian test config

* Remove empty lines

* Changed cilium version from <latest> to <v1.0.0-rc3>

* Add missing changes for cilium

* Add cilium to CI pipeline

* Fix wrong file name

* Check kernel version for cilium

* fixed ci error

* fixed cilium-ds.j2 template

* added waiting for cilium pods to run

* Fixed missing EOF

* Fixed trailing spaces

* Fixed trailing spaces

* Fixed trailing spaces

* Fixed too many blank lines

* Updated tolerations,annotations in cilium DS template

* Set cilium_version to iptables-1.9 to see if bug is fixed in CI

* Update cilium image tag to v1.0.0-rc4

* Update Cilium test case CI vars filenames

* Add optional prometheus flag, adjust initial readiness delay

* Update README.md with cilium info

roles/kubernetes/node/templates/kubelet.rkt.service.j2

@@ -39,7 +39,7 @@ ExecStart=/usr/bin/rkt run \
 {% if local_volume_provisioner_enabled == true %}
         --volume local-volume-base-dir,kind=host,source={{ local_volume_base_dir }},readOnly=false,recursive=true \
 {% endif %}
-{% if kube_network_plugin in ["calico", "weave", "canal", "flannel", "contiv"] %}
+{% if kube_network_plugin in ["calico", "weave", "canal", "flannel", "contiv", "cilium"] %}
         --volume etc-cni,kind=host,source=/etc/cni,readOnly=true \
         --volume opt-cni,kind=host,source=/opt/cni,readOnly=true \
         --volume var-lib-cni,kind=host,source=/var/lib/cni,readOnly=false \