Commit Graph

96 Commits

Author SHA1 Message Date
Kubernetes Prow Robot
acfaef2adf Merge pull request #11797 from VannTen/ci/fix_collection_testing
CI: build collection before runnings tests
2024-12-16 11:12:51 +01:00
Max Gautier
7da317348c CI: remove IDEMPOT_CHECK (#11796)
There is no test with IDEMPOT_CHECK=true since commit 7b78e6872 (disable
idempotency tests (#1872), 2017-10-26)

Remove the related infra from our CI scripts.
2024-12-13 20:02:26 +01:00
Max Gautier
18fab585ad CI: build collection before runnings tests
We were running the playbooks before building the collections, so this
probably wasn't testing much.
2024-12-13 16:44:30 +01:00
Max Gautier
630e9de658 CI: drop special casing for Opensuse and CoreOS (#11791)
- special casing should be in Kubespray, not in the test. It makes no
  sense to do something in tests which won't be done in actual usage.
- We don't actually test CoreOS at all in the CI.
2024-12-13 03:52:26 +01:00
Max Gautier
b9e9364c50 Remove ara support in CI
We do not use it and it has some overhead.
2024-12-11 17:28:35 +01:00
Max Gautier
236a7486f4 CI: directly use ANSIBLE_VERBOSITY instead of tweaking command line
https://docs.ansible.com/ansible/latest/reference_appendices/config.html#envvar-ANSIBLE_VERBOSITY
2024-11-14 09:41:01 +01:00
Max Gautier
ceb4b2fa7d CI: use current tests/ directory for upgrade testing
We should not rollback our test setup during upgrade test.
The only reason to do that would be for incompatible changes in the test
inventory, and we already checkout master for those (${CI_JOB_NAME}.yml)

Also do some cleanup by removing unnecessary intermediary variables
2024-11-14 09:41:00 +01:00
Max Gautier
a8e7238c9f CI: make the ansible inventory a directory
Not constraining the inventory to .ini allows us to use dynamic
inventory, which is needed for simplifying kubevirt jobs inventory.

Also reduces the scope of the ANSIBLE_INVENTORY variable.
2024-11-13 17:32:52 +01:00
Max Gautier
1818993a8a CI: for upgrade testing, checkout old version before provisioning
Otherwise, modifying the test inventory + it's expected structure will
always fail.
2024-09-21 15:10:59 +02:00
Max Gautier
76c42b4d3f CI: cleanup '-scale' tests infra (#11535)
There is actually no test using this since ad6fecefa8,
so there is no reason to keep that infra in our tests scripts.
2024-09-18 13:04:50 +01:00
Bogdan Sass
4b324cb0f0 Rename master to control plane - non-breaking changes only (#11394)
K8s is moving away from the "master" terminology, so kubespray should follow the same naming conventions. See 65d886bb30/sig-architecture/naming/recommendations/001-master-control-plane.md
2024-09-06 07:56:19 +01:00
Max Gautier
b0be5f2dad Print the name of faulty jinja templates in pre-commit (#11484) 2024-08-30 06:43:30 +01:00
Takuya Murakami
60b323b17f [CI] Add a CI job to test cluster upgrading, and fix bug of testcases_run.sh (#11458)
* Fix: fix testcases_run.sh for upgrade tests

Need to git checkout ${CI_COMMIT_SHA} before running upgrade playbook (revert #11173 partially)

* feat: add CI job to test upgrade

Add a packet_ubuntu22-calico-all-in-one-upgrade job
2024-08-29 15:47:32 +01:00
Antoine Legrand
a0587e0b8e CI: rework pipeline: short/extended based on labels (#11324)
* CI: reduce VM resources requests to improve scheduling

* CI: Reduce default jobs; add labels(ci-full/extended) to run more test

* CI: use jobs dependencies instead of stages

* precommit one-job

* CI: Use Kubevirt VM to run Molecule and Vagrant jobs
2024-07-01 03:25:36 -07:00
Max Gautier
19851bb07c collection-build-install convert to pre-commit 2024-05-28 13:26:46 +02:00
Max Gautier
7f7b65d388 Convert check_typo to pre-commit + use maintained version
client9/misspell is unmaintained, and has been forked by the golangci
team, see https://github.com/client9/misspell/issues/197#issuecomment-1596318684.

They haven't yet added a pre-commit config, so use my fork with the
pre-commit hook config until the pull request is merged.
2024-05-28 13:26:45 +02:00
Max Gautier
77bfb53455 Fix ci-matrix pre-commit hook
- Remove dependency of pydblite which fails to setup on recent pythons
- Discard shell script and put everything into pre-commit
2024-05-28 13:26:44 +02:00
Payback159
5d01dfa179 add auto generate documentation sidebar script, introduce script as pre-commit-hook, adapt existing scripts to work with documentation structure 2024-05-17 15:09:54 +02:00
Max Gautier
0b2533143f Don't force python interpreter on ubuntu (#11183)
This allows local override in tasks for using virtualenvs.
2024-05-13 19:51:11 -07:00
Max Gautier
d321e42d9e CI: Factorize ansible-playbook flags (#11173)
We have inconsistent sets of options passed to the playbooks during our
CI runs.

Don't run ansible-playbook directly, instead factorize the execution in
a bash function using all the common flags.
Also remove various ENABLE_* variables and instead directly test for the
relevant conditions at execution time, as this makes it more obvious and
does not force one to go back and forth in the script.
2024-05-13 03:20:48 -07:00
Max Gautier
ef95eb078a Cleanup unused CI tooling (#11014) 2024-03-15 03:57:27 -07:00
Kay Yan
90b0151caf support node feature discovery (#10861)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
2024-03-05 08:36:08 -08:00
Max Gautier
e90cae9344 Refactor check_galaxy + fix version (#10729)
* Remove checks for docs using exact tags

Instead use a more generic documentation for installing kubespray as a
collection from git.

* Check that we upgraded galaxy.yml to next version

This is only intented to check for human error. The version in galaxy
should be the next (which does not mean the same if we're on master or a
release branch).

* Set collection version to KUBESPRAY_NEXT_VERSION
2024-01-11 15:49:31 +01:00
Max Gautier
51069223f5 Decouple kubespray-defaults from download (#10626)
* Decouple role kubespray-defaults from download

Avoids doing re-importing the download role on every invocation of
kubespray-defaults (and skipping everything).

This has a measurable effect on playbook performance.

* Update docs refering to moved download defaults
2023-12-11 16:56:17 +01:00
Max Gautier
d2944d2813 Check jinja templates for syntax error (#10667)
Allow to fail early (pre-commit time) for jinja error, rather than
waiting until executing the playbook and the invalid template.

I could not find a simple jinja pre-commit hook in the wild.
2023-12-06 03:05:24 +01:00
Florian Ruynat
9696936b59 Fixup recover control plane playbook + add debian12/cilium test (#10411)
* Add debian12 cilium testing

* Fixup recover control plane playbook
2023-09-05 10:42:52 -07:00
Luke Simmons
1955943d4a Removes Ansible reinstall from pipeline (#10032) 2023-08-14 05:11:21 -07:00
Arthur Outhenin-Chalandre
25cb90bc2d Upgrade ansible (#10190)
* project: update all dependencies including ansible

Upgrade to ansible 7.x and ansible-core 2.14.x. There seems to be issue
with ansible 8/ansible-core 2.15 so we remain on those versions for now.
It's quite a big bump already anyway.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* tests: install aws galaxy collection

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* ansible-lint: disable various rules after ansible upgrade

Temporarily disable a bunch of linting action following ansible upgrade.
Those should be taken care of separately.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve deprecated-module ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve no-free-form ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[meta] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[playbook] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[tasks] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-file-permissions ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-shell-pipe ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: remove deprecated warn args

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use fqcn for non builtin tasks

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve syntax-check[missing-file] for contrib playbook

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use arithmetic inside jinja to fix ansible 6 upgrade

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-06-26 03:15:45 -07:00
Victor Morales
bf31a3a872 Split defaults main file (#10121) 2023-06-22 02:19:40 -07:00
Arthur Outhenin-Chalandre
edc73bc3c8 project: upgrade test dependencies and drop ansible-core 2.11 (#10034)
Molecule 5.0 require ansible-core 2.12.10.
So this commit we update ansible-core from 2.12.5 to 2.12.10.
We also drop supporting two ansible-core version. Also we now use the "oldest"
still supported ansible-core version as both 2.11 is EOL and not
supported by molecule.



tests/molecule: remove linting in molecule to support molecule 5



tests/molecule: remove role name check for molecule 5 support

Kubespray doesn't use ansible galaxy style naming so we have to disable
that check.



contrib/inventory_builder: fix tox.ini for tox4



tests/molecule: fix get_playbook in testinfra tests



tests: upgrade most tests requirements

Exclude ansible-lint for now, I will do that in a separate PR.



tests/molecule: force kvm driver option

If we don't do this it fallbacks to qemu emulated on our CI for some
reasons.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-06-02 20:40:40 -07:00
Kay Yan
82633c6f61 Remove the Support of Debian 9 because Debian 9 is EOF (#10097)
* remove-debian9-support

* Add six module into openstack-cleanup/requirements.txt (#10099)

To fix tf-elastx_cleanup job which was failed with the following error:

   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/generic/password.py", line 16, in <module>
     from keystoneauth1.identity import v3
   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/v3/__init__.py", line 27, in <module>
     from keystoneauth1.identity.v3.oauth2_mtls_client_credential import *  # noqa
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/v3/oauth2_mtls_client_credential.py", line 17, in <module>
     import six
 ModuleNotFoundError: No module named 'six'

---------

Co-authored-by: Kenichi Omichi <ken1ohmichi@gmail.com>
2023-05-18 15:42:33 -07:00
Florian Ruynat
951face343 Migrate CI_BUILD_ID to CI_JOB_ID and CI_BUILD_REF to CI_COMMIT_SHA (#10063) 2023-05-11 04:21:17 -07:00
Luke Simmons
acbf44a1b4 Adds support for Ansible collections (#9582) 2023-03-27 02:25:55 -07:00
Kenichi Omichi
7fec254f62 Drop part for supporting ansible 2.9 and 2.10 (#9842)
requirements-$ANSIBLE_VERSION.yml doesn't exist in Kubespray repo.
That was for supporting ansible 2.10-, and now Kubespray supports
2.11+. So this drops the part to avoid confusion.
2023-03-02 01:54:58 -08:00
Bas
2c93c997cf pre-commit autocorrected files (#9750) 2023-02-06 01:35:16 -08:00
Florian Ruynat
1f36df666d Update fedora35 vagrant box url (#9699)
* Update fedora35 vagrant box url

* Update Terraform to 1.3.7

* Update Vagrant to 2.3.4
2023-01-26 21:28:25 -08:00
Cyclinder
bf8c64af08 feature request: add cyclinder to reviewers (#9646)
Signed-off-by: cyclinder qifeng.guo@daocloud.io

Signed-off-by: cyclinder qifeng.guo@daocloud.io
2023-01-10 03:05:27 -08:00
Kenichi Omichi
ed38d8d3a1 Add ingress-nginx check for updating README (#9533)
To detect the version mismatch.
2022-12-01 01:16:27 -08:00
Kenichi Omichi
e5ec0f18c0 Add packet_ubuntu20-calico-aio-hardening (#9359)
To verify the hardening method works always.
The configuration comes from docs/hardening.md

Fix yaml format of hardening.yml

Add condition to skip 040 test for hardening
2022-10-19 05:35:04 -07:00
Kenichi Omichi
24632ae81b Add check_typo job (#9361)
To block merging pull requests which contain typo automatically.
2022-10-07 02:21:53 -07:00
Kenichi Omichi
7e862939db Add kube-vip check to check_readme_versions.sh (#9155)
To check the kube-vip version between readme.md and the default value
on the role, this updates check_readme_versions.sh
2022-08-06 08:26:20 -07:00
Kenichi Omichi
7414409aa0 Add target components on check_readme_versions.sh (#9045)
This adds target components on check_readme_versions.sh after
merging https://github.com/kubernetes-sigs/kubespray/pull/9044
In addition, this fixes typo on check_readme_versions.sh

This adds `foo_version` variables for some components because
check_readme_versions.sh verifies the corresponding version for
`<component name>_version` from main.yml. This change also makes
consistency in the main.yml. In long-term, we will be able to
remove the existing `foo_image_tag` variables, but that is not now
for backwards compatibility for users.
2022-07-05 08:02:47 -07:00
Kenichi Omichi
6cd243f14e Add component version check for README.md (#9042)
During code-review, reviwers needed to take care of README.md also
should be updated when the pull request updated component versions.
This adds the corresponding check to reduce reviwer's burden.
2022-06-29 00:14:05 -07:00
Calin Cristian Andrei
9143810a4d [CI] add remove node job 2022-05-04 06:35:51 -07:00
Kenichi Omichi
f184725c5f Use ansible 2.12 for testcases_prepare (#8763)
tests/requirements.txt links to tests/requirements-2.12.txt, so
Kubespray uses ansible 2.12 by default for testing. However we
forgot to update testcases_prepare.sh to use ansible 2.12.
This updates testcases_prepare to use ansible 2.12.
2022-05-02 11:34:31 -07:00
Cristian Calin
835fd86a08 [CI] split molecule testes to run in parallel (#8756)
* add parametrization to molecule_run.sh

* [CI] split molecule tests to allow parallelization of work
2022-04-29 00:09:12 -07:00
Cristian Calin
a090038d02 [CI] add ara to collect CI job logs (#8545) 2022-02-23 07:36:19 -08:00
Cristian Calin
2f44b40d68 OEL7: Fix CentOS7 Extras for OEL7 (#8219)
* OEL7: Fix CentOS7 Extras for OEL7

* Molecule: add logs collection for jobs
2021-11-29 13:39:21 -08:00
Florian Ruynat
a5f88e14d0 Cleanup tests (#8234)
* Add Fedora 35 image, support and CI

* Cleanup tests and allow_failure for vagrant
2021-11-26 09:00:51 -08:00
Cristian Calin
e78bda65fe Defaults: replace docker with containerd as our default container_manager (#8175)
* Defaults: replace docker with containerd as our default container_manager

* CI: Use docker for download_localhost test

* Defaults: with container_manager=containerd we need etcd_deployment_type=host

* CI: Run weave jobs with docker

* CI: Vagrant don't download_force_cache

* CI: Fix upgrade tests

* should run compatible with old settings, this means docker
* we need to run with a distro that has at least modern containerd,
  this means move from debian9 to debian10 to allow `containerd_version`
  to match between 2.17 and master
2021-11-25 06:54:33 -08:00