Commit Graph

8091 Commits

Author SHA1 Message Date
Max Gautier d8629b8e7e download: separate static metadata into it's own file
By separating logic from data, we should make it easier to add new
components.
2025-01-15 14:32:49 +01:00
Max Gautier 4d3f06e69e download: cleanup graphQL query
- remove unused parts in the response
- clarify variables names
2025-01-14 17:04:29 +01:00
Max Gautier d17bd286ea download: allow excluding some component
This is handy when some component releases is buggy (missing file at the
download links) to not block everything else.

Move the filtering up the stack so we don't have to do it multiples
times.
2025-01-14 17:04:28 +01:00
Max Gautier 55cff4f3d3 download: get checksums file relative to git root
This means the update-hashes command can be run anywhere in Kubespray
repository without having to figure out the correct path.
2025-01-14 17:04:28 +01:00
Max Gautier 76e07daa12 download: put grapqQL query in package + read from importlib 2025-01-14 17:04:27 +01:00
Max Gautier a551922c84 Adapt download.py to run as a package script 2025-01-14 17:04:27 +01:00
Max Gautier ba3258d7f0 Move download_hash.py into a python package
Can operate on several branches without the need for backport
2025-01-14 17:04:26 +01:00
Max Gautier 9b56840d51 download: create pyproject.toml 2025-01-14 17:04:24 +01:00
Max Gautier 4351b47ebe download: convert to logging 2025-01-14 17:04:18 +01:00
Max Gautier b08c5e8b14 download: Log Github rate-limit status 2025-01-14 17:02:29 +01:00
Max Gautier 81790cab91 download: remove unneeded imports 2025-01-14 10:41:42 +01:00
Max Gautier 9fbc566d98 download: Support adding new versions and update the doc 2025-01-14 10:41:41 +01:00
Max Gautier ff768cc9fe download: support multiple hash algorithm 2025-01-14 10:41:41 +01:00
Max Gautier ff3d9a0443 download: Support for gvisor (part 2)
Gvisor releases, besides only being tags, have some particularities:
- they are of the form yyyymmdd.p -> this get interpreted as a yaml
  float, so we need to explicitely convert to string to make it work.
- there is no semver-like attached to the version numbers, but the API
  (= OCI container runtime interface) is expected to be stable (see
  linked discussion)
- some older tags don't have hashs for some archs

Link: https://groups.google.com/g/gvisor-users/c/SxMeHt0Yb6Y/m/Xtv7seULCAAJ
2025-01-14 10:41:40 +01:00
Max Gautier 6608efb2c4 download: compute version from Github tags for gvisor
Gvisor is the only one of our deployed components which use tags instead
of proper releases. So the tags scraping support will, for now, cater to
gvisor particularities, notably in the tag name format and the fact that
some older releases don't have the same URL scheme.
2025-01-14 10:41:39 +01:00
Max Gautier 479fda6355 download: support cri-dockerd, youki, kata, crun 2025-01-14 10:41:39 +01:00
Max Gautier 3a44411aa1 Support project using alternates names for arch
(the url should use `alt_arch` instead of `arch` for those)
2025-01-14 10:41:38 +01:00
Max Gautier 9334bc1fee support components with no premade hashes 2025-01-14 10:41:38 +01:00
Max Gautier c94daa4ff5 download: Update yaml data with new hashes 2025-01-14 10:41:37 +01:00
Max Gautier 5be8155394 remove old loops and generators 2025-01-14 10:41:36 +01:00
Max Gautier 08913c4aa0 Don't use 'checksum' in the components names 2025-01-14 10:41:36 +01:00
Max Gautier 38dd224ffe Extract get_hash into it's own function
Also, always raise even for 404 not found (should not happen now that
we'll use GraphQL to find the exact set of versions)
2025-01-14 10:41:36 +01:00
Max Gautier 24c59cee59 download_hash: adapt download urls to v-less versions 2025-01-14 10:41:35 +01:00
Max Gautier 2be54b2bd7 Filter new versions for new ones and same minor releases
We're only interested in new patch releases for auto-update.
2025-01-14 10:41:35 +01:00
Max Gautier ae68766015 Filter by github results InvalidVersion
Containerd use the same repository for releases of it's gRPC API (which
we are not interested in).
Conveniently, those releases have tags which are not valid version
number (being prefixed with 'api/').

This could also be potentially useful for similar cases.
The risk of missing releases because of this are low, since it would
require that a project issue a new release with an invalid format, then
switch back to the previous format (or we miss the fact it's not
updating for a long period of time).
2025-01-14 10:41:34 +01:00
Max Gautier 9f58ba60f3 download: compute new versions from Github API
We obtain the set of version from Github, then for each component we do
a set comparison to determine which versions we don't have.
2025-01-14 10:41:34 +01:00
Max Gautier a6219c84c9 Put graphql query in it's own file 2025-01-14 10:41:33 +01:00
Max Gautier 7941be127d downloads: add graphql node ids
The Github graphQL API needs IDs for querying a variable array of
repository.

Use a dict for components instead of an array of url and record the
corresponding node ID for each component (there are duplicates because
some binaries are provided by the same project/repository).
2025-01-14 10:41:33 +01:00
Max Gautier c938dfa634 scripts: get_nodes_ids.sh
Add the script used to obtain graphql node IDs from Github so it's
easier to add a new component.
2025-01-14 10:41:31 +01:00
ChengHao Yang 5a353cb04f Add manual option to the external_cloud_provider variable (#11883)
* Add `manual` option in the `external_cloud_provider` value

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

* Update external cloud provider description in roles & sample inventory

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

---------

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
2025-01-13 00:12:34 -08:00
kyrie 1f186ed451 add containerd registry mirror certificate configuration (#11857)
Signed-off-by: KubeKyrie <shaolong.qin@daocloud.io>
2025-01-09 01:48:31 -08:00
Chad Swenson 8443f370d4 Structured AuthorizationConfiguration (#11852)
Adds the ability to configure the Kubernetes API server with a structured authorization configuration file.

Structured AuthorizationConfiguration is a new feature in Kubernetes v1.29+ (GA in v1.32) that configures the API server's authorization modes with a structured configuration file.
AuthorizationConfiguration files offer features not available with the `--authorization-mode` flag, although Kubespray supports both methods and authorization-mode remains the default for now.

Note: Because the `--authorization-config` and `--authorization-mode` flags are mutually exclusive, the `authorization_modes` ansible variable is ignored when `kube_apiserver_use_authorization_config_file` is set to true. The two features cannot be used at the same time.

Docs: https://kubernetes.io/docs/reference/access-authn-authz/authorization/#configuring-the-api-server-using-an-authorization-config-file
Blog + Examples: https://kubernetes.io/blog/2024/04/26/multi-webhook-and-modular-authorization-made-much-easier/
KEP: https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3221-structured-authorization-configuration

I tested this all the way back to k8s v1.29 when AuthorizationConfiguration was first introduced as an alpha feature, although v1.29 required some additional workarounds with `kubeadm_patches`, which I included in example comments.

I also included some example comments with CEL expressions that allowed me to configure webhook authorizers without hitting kubeadm 1.29+ issues that block cluster creation and upgrades such as this one: https://github.com/kubernetes/cloud-provider-openstack/issues/2575.
My workaround configures the webhook to ignore requests from kubeadm and system components, which prevents fatal errors from webhooks that are not available yet, and should be authorized by Node or RBAC anyway.
2025-01-07 09:14:28 +01:00
ChengHao Yang 1801debaea Add Flatcar 4081.2.1 image to test-infra (#11849)
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
2025-01-07 08:38:28 +01:00
Kay Yan 369be00960 increase the memory requirement to 2GB (#11864)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
2025-01-07 08:00:28 +01:00
Kay Yan ae1805587b cleaup for 2.27.0 (#11854)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
2025-01-07 05:06:29 +01:00
Noam 55d1e4a4b5 enable bash completion tasks for Suse OS family (#11860)
* remove check for os family on bash completion tasks

* add Suse
2025-01-06 15:36:16 +01:00
Max Gautier ac9b76eb2e Ignore Mem preflight errors on ubuntu upgrade testcase (#11859) 2025-01-06 11:52:16 +01:00
Mohamed Omar Zaian 9ec9b3a202 [ingress-nginx] upgrade to 1.12.0 (#11846) v2.27.0 2025-01-02 04:58:14 +01:00
Antoine Legrand 0222a2a634 Add option to skip network plugin installation (#11844) 2024-12-31 12:52:13 +01:00
Kubernetes Prow Robot 57490d5e5e Merge pull request #11793 from VannTen/cleanup/ci_testcases_better_callback
Use debug stdout callback in ci rather than manual debug
2024-12-27 18:40:12 +01:00
Kubernetes Prow Robot 5af3a34de8 Merge pull request #11819 from VannTen/cleanup/preinstall_fact
Cleanups in kubernetes/preinstall (DNS stuff)
2024-12-27 18:04:11 +01:00
ChengHao Yang 54a01f2774 Bump: Containerd upgrade to 1.7.24 & runc upgrade to v1.2.3 (#11833)
* Bump: Containerd upgrade to 1.7.24

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

* Docs: README.md update Containerd version 1.7.24

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

* Bump: runc upgrade to v1.2.3

Runc upgrade to v1.2.3, and add v1.1.15, v1.2.x checksum

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

---------

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
2024-12-27 13:36:11 +01:00
Max Gautier 6f6da3d3c7 Update SECURITY_CONTACTS with active maintainers (#11827)
* Update SECURITY_CONTACTS with active maintainers

* Add yankay to SECURITY_CONTACTS
2024-12-27 06:26:13 +01:00
ChengHao Yang a6bc327d63 Bump: Helm upgrade to v3.16.4 (#11832)
* Bump: Helm default version v3.16.4

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

* Docs: README.md update helm version

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

---------

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
2024-12-26 14:24:11 +01:00
Mohamed Omar Zaian 25d0380db7 [calico] Add version 3.29.1 and make it default (#11798) 2024-12-25 23:14:11 +01:00
ChengHao Yang 3305ae9235 Bump: Kubernetes default version v1.31.4 (#11828)
* Bump: kubernetes upgrade to 1.31.4

Add Kubernetes 1.31.4, 1.30.8 and 1.29.12 version

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

* Docs: Upgrade Kubernetes version to 1.31.4

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>

---------

Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
2024-12-25 23:10:13 +01:00
kyrie e7a5e3ca5c Fix using the default network manager in reset.yml (#11678)
* enhance reset network service

Signed-off-by: KubeKyrie <shaolong.qin@daocloud.io>

* reset network service: use systemd module directly

---------

Signed-off-by: KubeKyrie <shaolong.qin@daocloud.io>
Co-authored-by: Max Gautier <mg@max.gautier.name>
2024-12-24 15:50:11 +01:00
dependabot[bot] 6c69ffed5b build(deps): bump molecule-plugins[vagrant] from 23.5.3 to 23.6.0 (#11826)
Bumps [molecule-plugins[vagrant]](https://github.com/ansible-community/molecule-plugins) from 23.5.3 to 23.6.0.
- [Release notes](https://github.com/ansible-community/molecule-plugins/releases)
- [Commits](https://github.com/ansible-community/molecule-plugins/compare/v23.5.3...v23.6.0)

---
updated-dependencies:
- dependency-name: molecule-plugins[vagrant]
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-24 14:46:11 +01:00
Max Gautier d173f1d951 Only consider host in 'k8s_cluster' when checking if ip is a cached fact (#11817)
This avoids spurious failure with 'localhost'.

It should also be more correct the inventory contains uncached hosts
which are not in `k8s_cluster` and therefore should not be Kubespray
business.

(We still use hostvars for uncached hosts, because it's easier to select
on 'ansible_default_ipv4' that way and does not change the end result)
2024-12-23 08:48:10 +01:00
Max Gautier 91ad58a185 Update ansible-lint pre-commit + drop jsonschema dep (#11818) 2024-12-20 03:00:09 +01:00