Cluster K8s

Fixed version

Composants	PROD_Actuel	PROD_Futur	Version_latest (app)	Version_chart_helm
Cert-manager	1.11.0	1.15.0	1.15.0	1.15.0
FluxCD	no flux	2.3.0	2.3.0	2.13.0
Git-agent	16.9.3	16.9.3	17.1.0	2.30
Git-runner	16.9.1	16.9.1	17.0.1	0.62.1
MariaDB	10.10	1.13.2	1.13.2	18.2.4
Prometheus	0.60.1	0.74.0	0.74.0	60.2.0
Traefik	2.9.6	3.0.2	3.0.2	28.3.0
Velero	1.10.1	1.13.2	1.13.2	6.7.0
Vault	no vault	1.16.1	1.16.1	0.28.0

How does it work ?

---
title: Git architecture
---
%%{init: { 'gitGraph': {'mainBranchName':'local'} } }%%
gitGraph
    branch develop
    commit id: "first"
    branch main
    checkout local
    commit id: "dev"
    commit id: "1.0.0-dev"
    checkout develop
    merge local id: "1.0.0-staging"
    branch deploy/staging
    checkout deploy/staging
    commit id: "dev-k8s-ldc-web:1.0.0"
    checkout main
    merge develop id: "1.0.0"
    branch deploy/production
    checkout deploy/production
    commit id: "k8s-ldc-web:1.0.0"

---
title: Process architecture
---
flowchart TD
    subgraph "Gitlab"
        subgraph "CI/CD staging"
            B1{{job deploy:staging}}
            B1 -. Fill manifests values .-> B2{{manifests ready}}
            B2 -. Connect to dev-k8s-ldc-web .-> B3{{cluster connected}}
            B3 -. Deploy secrets .-> B4{{secrets applied}}
            B4 -. Commit filled manifests .-> B5{{branch ready}}
        end
        subgraph "CI/CD production"
            C1{{job deploy:production}} -. Fill manifests values .-> C2{{manifests ready}}
            C2 -. Connect to k8s-ldc-web .-> C3{{cluster connected}}
            C3 -. Deploy secrets .-> C4{{secrets applied}}
            C4 -. Commit filled manifests .-> C5{{branch ready}}
        end
        subgraph "Flux staging"
            D1{{common-services changes detected}} -. Changes dry-run deployment .-> D2{{dry-run passed}}
            D2 -. Changes deployment .-> D3{{releases deployed}}
            D3 -. Watch loop cluster state .-> D4{{dev-k8s-ldc-web up to date}}
        end
        
        subgraph "Flux production"
            E1{{manifests changes detected}} -. Changes dry-run deployment .-> E2{{dry-run passed}}
            E2 -. Changes deployment .-> E3{{releases deployed}}
            E3 -. Watch loop cluster state .-> E4{{k8s-ldc-web up to date}}
        end
        A1[local] -- Merge request --> A2[develop]
        A2 -- Merge request --> A3[main]
        A2 -- Set staging CI/CD variables --> B1
        B5 -- Git push --> A4[deploy/staging]
        A3 -- Set production CI/CD variables --> C1
        C5 -- Git push --> A5[deploy/production]
        A4 -- Synchronize with Flux --> D1
        A5 -- Synchronize with Flux --> E1
    end

Init flux agent

Install flux CLI

curl -s https://fluxcd.io/install.sh | sudo bash 

Create flux agent for staging

You need to be connected in dev-cluster-k8s by kubeconfig.

export GITLAB_TOKEN=<cluster-k8s-flux-token-api>
flux bootstrap gitlab \
--hostname=lab.s-web.fr \
--owner=ldc-web \
--repository=cluster-k8s \
--branch=deploy/staging \
--path=flux/staging \
--deploy-token-auth

Then you need to add a Kustomization resource for each root folder you want to synchronize with the cluster at the end of the file flux/staging/gotk-sync.yaml. For staging environment:

---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: namespaces
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./namespaces
  prune: true
  sourceRef:
    kind: GitRepository
    name: cluster-k8s
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: common-services
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./releases/common-services
  prune: true
  sourceRef:
    kind: GitRepository
    name: cluster-k8s

Create flux agent for production

You need to be connected in cluster-k8s by kubeconfig.

export GITLAB_TOKEN=<cluster-k8s-flux-token-api>
flux bootstrap gitlab \
--hostname=lab.s-web.fr \
--owner=ldc-web \
--repository=cluster-k8s \
--branch=deploy/production \
--path=flux/production \
--deploy-token-auth

Then the resources to add in flux/production/gotk-sync.yaml for production environment:

---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: namespaces
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./namespaces
  prune: true
  sourceRef:
    kind: GitRepository
    name: cluster-k8s
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: releases
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./releases
  prune: true
  sourceRef:
    kind: GitRepository
    name: cluster-k8s

Note that in production we follow the entire folder ./manifests to deploy services as velero or custom services for ldc web.