Commit Graph

80 Commits

Author SHA1 Message Date
Max Gautier
77bfb53455 Fix ci-matrix pre-commit hook
- Remove dependency of pydblite which fails to setup on recent pythons
- Discard shell script and put everything into pre-commit
2024-05-28 13:26:44 +02:00
Payback159
5d01dfa179 add auto generate documentation sidebar script, introduce script as pre-commit-hook, adapt existing scripts to work with documentation structure 2024-05-17 15:09:54 +02:00
Max Gautier
0b2533143f Don't force python interpreter on ubuntu (#11183)
This allows local override in tasks for using virtualenvs.
2024-05-13 19:51:11 -07:00
Max Gautier
d321e42d9e CI: Factorize ansible-playbook flags (#11173)
We have inconsistent sets of options passed to the playbooks during our
CI runs.

Don't run ansible-playbook directly, instead factorize the execution in
a bash function using all the common flags.
Also remove various ENABLE_* variables and instead directly test for the
relevant conditions at execution time, as this makes it more obvious and
does not force one to go back and forth in the script.
2024-05-13 03:20:48 -07:00
Max Gautier
ef95eb078a Cleanup unused CI tooling (#11014) 2024-03-15 03:57:27 -07:00
Kay Yan
90b0151caf support node feature discovery (#10861)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
2024-03-05 08:36:08 -08:00
Max Gautier
e90cae9344 Refactor check_galaxy + fix version (#10729)
* Remove checks for docs using exact tags

Instead use a more generic documentation for installing kubespray as a
collection from git.

* Check that we upgraded galaxy.yml to next version

This is only intented to check for human error. The version in galaxy
should be the next (which does not mean the same if we're on master or a
release branch).

* Set collection version to KUBESPRAY_NEXT_VERSION
2024-01-11 15:49:31 +01:00
Max Gautier
51069223f5 Decouple kubespray-defaults from download (#10626)
* Decouple role kubespray-defaults from download

Avoids doing re-importing the download role on every invocation of
kubespray-defaults (and skipping everything).

This has a measurable effect on playbook performance.

* Update docs refering to moved download defaults
2023-12-11 16:56:17 +01:00
Max Gautier
d2944d2813 Check jinja templates for syntax error (#10667)
Allow to fail early (pre-commit time) for jinja error, rather than
waiting until executing the playbook and the invalid template.

I could not find a simple jinja pre-commit hook in the wild.
2023-12-06 03:05:24 +01:00
Florian Ruynat
9696936b59 Fixup recover control plane playbook + add debian12/cilium test (#10411)
* Add debian12 cilium testing

* Fixup recover control plane playbook
2023-09-05 10:42:52 -07:00
Luke Simmons
1955943d4a Removes Ansible reinstall from pipeline (#10032) 2023-08-14 05:11:21 -07:00
Arthur Outhenin-Chalandre
25cb90bc2d Upgrade ansible (#10190)
* project: update all dependencies including ansible

Upgrade to ansible 7.x and ansible-core 2.14.x. There seems to be issue
with ansible 8/ansible-core 2.15 so we remain on those versions for now.
It's quite a big bump already anyway.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* tests: install aws galaxy collection

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* ansible-lint: disable various rules after ansible upgrade

Temporarily disable a bunch of linting action following ansible upgrade.
Those should be taken care of separately.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve deprecated-module ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve no-free-form ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[meta] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[playbook] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[tasks] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-file-permissions ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-shell-pipe ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: remove deprecated warn args

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use fqcn for non builtin tasks

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve syntax-check[missing-file] for contrib playbook

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use arithmetic inside jinja to fix ansible 6 upgrade

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-06-26 03:15:45 -07:00
Victor Morales
bf31a3a872 Split defaults main file (#10121) 2023-06-22 02:19:40 -07:00
Arthur Outhenin-Chalandre
edc73bc3c8 project: upgrade test dependencies and drop ansible-core 2.11 (#10034)
Molecule 5.0 require ansible-core 2.12.10.
So this commit we update ansible-core from 2.12.5 to 2.12.10.
We also drop supporting two ansible-core version. Also we now use the "oldest"
still supported ansible-core version as both 2.11 is EOL and not
supported by molecule.



tests/molecule: remove linting in molecule to support molecule 5



tests/molecule: remove role name check for molecule 5 support

Kubespray doesn't use ansible galaxy style naming so we have to disable
that check.



contrib/inventory_builder: fix tox.ini for tox4



tests/molecule: fix get_playbook in testinfra tests



tests: upgrade most tests requirements

Exclude ansible-lint for now, I will do that in a separate PR.



tests/molecule: force kvm driver option

If we don't do this it fallbacks to qemu emulated on our CI for some
reasons.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-06-02 20:40:40 -07:00
Kay Yan
82633c6f61 Remove the Support of Debian 9 because Debian 9 is EOF (#10097)
* remove-debian9-support

* Add six module into openstack-cleanup/requirements.txt (#10099)

To fix tf-elastx_cleanup job which was failed with the following error:

   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/generic/password.py", line 16, in <module>
     from keystoneauth1.identity import v3
   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/v3/__init__.py", line 27, in <module>
     from keystoneauth1.identity.v3.oauth2_mtls_client_credential import *  # noqa
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/v3/oauth2_mtls_client_credential.py", line 17, in <module>
     import six
 ModuleNotFoundError: No module named 'six'

---------

Co-authored-by: Kenichi Omichi <ken1ohmichi@gmail.com>
2023-05-18 15:42:33 -07:00
Florian Ruynat
951face343 Migrate CI_BUILD_ID to CI_JOB_ID and CI_BUILD_REF to CI_COMMIT_SHA (#10063) 2023-05-11 04:21:17 -07:00
Luke Simmons
acbf44a1b4 Adds support for Ansible collections (#9582) 2023-03-27 02:25:55 -07:00
Kenichi Omichi
7fec254f62 Drop part for supporting ansible 2.9 and 2.10 (#9842)
requirements-$ANSIBLE_VERSION.yml doesn't exist in Kubespray repo.
That was for supporting ansible 2.10-, and now Kubespray supports
2.11+. So this drops the part to avoid confusion.
2023-03-02 01:54:58 -08:00
Bas
2c93c997cf pre-commit autocorrected files (#9750) 2023-02-06 01:35:16 -08:00
Florian Ruynat
1f36df666d Update fedora35 vagrant box url (#9699)
* Update fedora35 vagrant box url

* Update Terraform to 1.3.7

* Update Vagrant to 2.3.4
2023-01-26 21:28:25 -08:00
Cyclinder
bf8c64af08 feature request: add cyclinder to reviewers (#9646)
Signed-off-by: cyclinder qifeng.guo@daocloud.io

Signed-off-by: cyclinder qifeng.guo@daocloud.io
2023-01-10 03:05:27 -08:00
Kenichi Omichi
ed38d8d3a1 Add ingress-nginx check for updating README (#9533)
To detect the version mismatch.
2022-12-01 01:16:27 -08:00
Kenichi Omichi
e5ec0f18c0 Add packet_ubuntu20-calico-aio-hardening (#9359)
To verify the hardening method works always.
The configuration comes from docs/hardening.md

Fix yaml format of hardening.yml

Add condition to skip 040 test for hardening
2022-10-19 05:35:04 -07:00
Kenichi Omichi
24632ae81b Add check_typo job (#9361)
To block merging pull requests which contain typo automatically.
2022-10-07 02:21:53 -07:00
Kenichi Omichi
7e862939db Add kube-vip check to check_readme_versions.sh (#9155)
To check the kube-vip version between readme.md and the default value
on the role, this updates check_readme_versions.sh
2022-08-06 08:26:20 -07:00
Kenichi Omichi
7414409aa0 Add target components on check_readme_versions.sh (#9045)
This adds target components on check_readme_versions.sh after
merging https://github.com/kubernetes-sigs/kubespray/pull/9044
In addition, this fixes typo on check_readme_versions.sh

This adds `foo_version` variables for some components because
check_readme_versions.sh verifies the corresponding version for
`<component name>_version` from main.yml. This change also makes
consistency in the main.yml. In long-term, we will be able to
remove the existing `foo_image_tag` variables, but that is not now
for backwards compatibility for users.
2022-07-05 08:02:47 -07:00
Kenichi Omichi
6cd243f14e Add component version check for README.md (#9042)
During code-review, reviwers needed to take care of README.md also
should be updated when the pull request updated component versions.
This adds the corresponding check to reduce reviwer's burden.
2022-06-29 00:14:05 -07:00
Calin Cristian Andrei
9143810a4d [CI] add remove node job 2022-05-04 06:35:51 -07:00
Kenichi Omichi
f184725c5f Use ansible 2.12 for testcases_prepare (#8763)
tests/requirements.txt links to tests/requirements-2.12.txt, so
Kubespray uses ansible 2.12 by default for testing. However we
forgot to update testcases_prepare.sh to use ansible 2.12.
This updates testcases_prepare to use ansible 2.12.
2022-05-02 11:34:31 -07:00
Cristian Calin
835fd86a08 [CI] split molecule testes to run in parallel (#8756)
* add parametrization to molecule_run.sh

* [CI] split molecule tests to allow parallelization of work
2022-04-29 00:09:12 -07:00
Cristian Calin
a090038d02 [CI] add ara to collect CI job logs (#8545) 2022-02-23 07:36:19 -08:00
Cristian Calin
2f44b40d68 OEL7: Fix CentOS7 Extras for OEL7 (#8219)
* OEL7: Fix CentOS7 Extras for OEL7

* Molecule: add logs collection for jobs
2021-11-29 13:39:21 -08:00
Florian Ruynat
a5f88e14d0 Cleanup tests (#8234)
* Add Fedora 35 image, support and CI

* Cleanup tests and allow_failure for vagrant
2021-11-26 09:00:51 -08:00
Cristian Calin
e78bda65fe Defaults: replace docker with containerd as our default container_manager (#8175)
* Defaults: replace docker with containerd as our default container_manager

* CI: Use docker for download_localhost test

* Defaults: with container_manager=containerd we need etcd_deployment_type=host

* CI: Run weave jobs with docker

* CI: Vagrant don't download_force_cache

* CI: Fix upgrade tests

* should run compatible with old settings, this means docker
* we need to run with a distro that has at least modern containerd,
  this means move from debian9 to debian10 to allow `containerd_version`
  to match between 2.17 and master
2021-11-25 06:54:33 -08:00
Cristian Calin
0d0468e127 Exercise multiple ansible versions in CI (#8172)
* Ansible: separate requirements files for supported ansible versions

* Ansible: allow using ansible 2.11

* CI: Exercise Ansible 2.9 and Ansible 2.11 in a basic AIO CI job

* CI: Allow running a reset test outside of idempotency tests and running it in stage1

* CI: move ubuntu18-calico-aio job to stage2 and relay only on ubuntu20 with the variously supported ansible versions for stage1

* CI: add capability to install collections or roles from ansible-galaxy to mitigate missing behavior in older ansible versions
2021-11-10 16:11:50 -08:00
Cristian Calin
8d553f7e91 Mitogen: deprecate the use of mitogen and remove coverage from CI (#8147) 2021-11-05 00:57:52 -07:00
Cristian Calin
2ed211ba15 Fix-CI: python was upgraded in CI to 3.10 and pathlib is now included in python base making this dependency break the CI (#8153) 2021-11-03 12:52:32 -07:00
Florian Ruynat
c7529270ff Fix CI script for Terraform >0.15 (#7928) 2021-09-01 04:30:59 -07:00
jayonlau
6b5da84014 Clean up extra spaces last one (#7904)
Although these errors are not important, they affect the code specification.
2021-08-25 12:06:09 -07:00
Cristian Calin
7516fe142f Move to Ansible 3.4.0 (#7672)
* Ansible: move to Ansible 3.4.0 which uses ansible-base 2.10.10

* Docs: add a note about ansible upgrade post 2.9.x

* CI: ensure ansible is removed before ansible 3.x is installed to avoid pip failures

* Ansible: use newer ansible-lint

* Fix ansible-lint 5.0.11 found issues

* syntax issues
* risky-file-permissions
* var-naming
* role-name
* molecule tests

* Mitogen: use 0.3.0rc1 which adds support for ansible 2.10+

* Pin ansible-base to 2.10.11 to get package fix on RHEL8
2021-07-12 00:00:47 -07:00
Florian Ruynat
d1388d69d0 Fix tests following python change (#7775)
* Fix ansible detection for python3 and ubuntu

* Fix oracle missing centos-extras repo for containerd/docker dependencies
2021-07-08 18:52:53 -07:00
Kenichi Omichi
486b223e01 Replace kube-master with kube_control_plane (#7256)
This replaces kube-master with kube_control_plane because of [1]:

  The Kubernetes project is moving away from wording that is
  considered offensive. A new working group WG Naming was created
  to track this work, and the word "master" was declared as offensive.
  A proposal was formalized for replacing the word "master" with
  "control plane". This means it should be removed from source code,
  documentation, and user-facing configuration from Kubernetes and
  its sub-projects.

NOTE: The reason why this changes it to kube_control_plane not
      kube-control-plane is for valid group names on ansible.

[1]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint/README.md#motivation
2021-03-23 17:26:05 -07:00
Florian Ruynat
f6eed8091e Remove contiv related files (#6964) 2020-11-30 06:48:50 -08:00
Sergey
4a8a52bad9 containerd docker hub registry mirror support (#6962)
* containerd docker hub registry mirror support

* add docs

* fix typo

* fix yamllint

* fix indent in sample
and ansible-playbook param in testcases_run

* fix md

* mv common vars to tests/common/_docker_hub_registry_mirror.yml

* checkout vars to upgrade tests
2020-11-30 00:22:49 -08:00
Sergey
d739a6bb2f add Google proxy-mirror-cache for docker hub to CI tests (#6957) 2020-11-27 03:24:48 -08:00
Maxime Guyot
686316b390 Cleanup virsh volumes in Vagrant CI (#6688) 2020-09-17 08:04:45 -07:00
Florian Ruynat
705ad84ce7 Update third party librairies and tools (#6669) 2020-09-17 02:36:46 -07:00
Barry Melbourne
058438a25d Remove support for CoreOS Container Linux (#6576) 2020-08-28 02:28:53 -07:00
Florian Ruynat
f2ca929a4a Move nodes readiness test before pods readiness (#6089) 2020-06-09 05:23:18 -07:00
Maxime Guyot
fd59556222 Add Elastx CI (#6127) 2020-06-03 00:00:17 -07:00