Cluster K8s

Fixed version

Composants	PROD_Actuel	PROD_Futur	Version_latest (app)	Version_chart_helm
Cert-manager	1.11.0	1.15.0	1.15.0	1.15.0
FluxCD	no flux	2.3.0	2.3.0	2.13.0
Git-agent	16.9.3	16.9.3	17.1.0	2.30
Git-runner	16.9.1	16.9.1	17.0.1	0.62.1
Prometheus	0.60.1	0.74.0	0.74.0	60.2.0
Traefik	2.9.6	3.0.2	3.0.2	28.3.0
Velero	1.10.1	1.13.2	1.13.2	6.7.0
Vault	no vault	1.16.1	1.16.1	0.28.0

How does it work ?

---
title: Git architecture
---
%%{init: { 'gitGraph': {'mainBranchName':'local'} } }%%
gitGraph
    branch develop
    commit id: "first"
    branch main
    checkout local
    commit id: "dev"
    commit id: "1.0.0-dev"
    checkout develop
    merge local id: "1.0.0-staging"
    branch deploy/staging
    checkout deploy/staging
    commit id: "dev-k8s-ldc-web:1.0.0"
    checkout main
    merge develop id: "1.0.0"
    branch deploy/production
    checkout deploy/production
    commit id: "k8s-ldc-web:1.0.0"

---
title: Process architecture
---
flowchart TD
    subgraph "Gitlab"
        subgraph "CI/CD staging"
            B1{{job deploy:staging}}
            B1 -. Fill manifests values .-> B2{{manifests ready}}
            B2 -. Connect to dev-k8s-ldc-web .-> B3{{cluster connected}}
            B3 -. Deploy secrets .-> B4{{secrets applied}}
            B4 -. Commit filled manifests .-> B5{{branch ready}}
        end
        subgraph "CI/CD production"
            C1{{job deploy:production}} -. Fill manifests values .-> C2{{manifests ready}}
            C2 -. Connect to k8s-ldc-web .-> C3{{cluster connected}}
            C3 -. Deploy secrets .-> C4{{secrets applied}}
            C4 -. Commit filled manifests .-> C5{{branch ready}}
        end
        subgraph "Flux staging"
            D1{{common-services changes detected}} -. Changes dry-run deployment .-> D2{{dry-run passed}}
            D2 -. Changes deployment .-> D3{{releases deployed}}
            D3 -. Watch loop cluster state .-> D4{{dev-k8s-ldc-web up to date}}
        end
        
        subgraph "Flux production"
            E1{{manifests changes detected}} -. Changes dry-run deployment .-> E2{{dry-run passed}}
            E2 -. Changes deployment .-> E3{{releases deployed}}
            E3 -. Watch loop cluster state .-> E4{{k8s-ldc-web up to date}}
        end
        A1[local] -- Merge request --> A2[develop]
        A2 -- Merge request --> A3[main]
        A2 -- Set staging CI/CD variables --> B1
        B5 -- Git push --> A4[deploy/staging]
        A3 -- Set production CI/CD variables --> C1
        C5 -- Git push --> A5[deploy/production]
        A4 -- Synchronize with Flux --> D1
        A5 -- Synchronize with Flux --> E1
    end

Init flux agent

Install flux CLI

curl -s https://fluxcd.io/install.sh | sudo bash 

Create flux agent for staging

You need to be connected in dev-cluster-k8s by kubeconfig.

export GITLAB_TOKEN=<cluster-k8s-flux-token-api>
flux bootstrap gitlab \
--hostname=lab.s-web.fr \
--owner=ldc-web \
--repository=cluster-k8s \
--branch=deploy/staging \
--path=flux/staging \
--deploy-token-auth

Then you need to add a Kustomization resource for each root folder you want to synchronize with the cluster at the end of the file flux/staging/gotk-sync.yaml. For staging environment:

---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: namespaces
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./namespaces
  prune: true
  sourceRef:
    kind: GitRepository
    name: cluster-k8s
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: common-services
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./releases/common-services
  prune: true
  sourceRef:
    kind: GitRepository
    name: cluster-k8s

Create flux agent for production

You need to be connected in cluster-k8s by kubeconfig.

export GITLAB_TOKEN=<cluster-k8s-flux-token-api>
flux bootstrap gitlab \
--hostname=lab.s-web.fr \
--owner=ldc-web \
--repository=cluster-k8s \
--branch=deploy/production \
--path=flux/production \
--deploy-token-auth

Then the resources to add in flux/production/gotk-sync.yaml for production environment:

---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: namespaces
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./namespaces
  prune: true
  sourceRef:
    kind: GitRepository
    name: cluster-k8s
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: releases
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./releases
  prune: true
  sourceRef:
    kind: GitRepository
    name: cluster-k8s

Note that in production we follow the entire folder ./manifests to deploy services as velero or custom services for ldc web.