Network simulator

How this works in a nutshell

Simulator generates default values.yaml and wraps helm and kubectl commands for shorthand.

Prerequisites

Simulator depends on project structure and works only inside git project root:

git clone https://github.com/broxus/tycho.git
cd ./tycho

• docker

• docker-buildx or

• podman

• docker-buildx

• k3s

• helm

• kubectl

Warning
DON'T SKIP firewall configuration, otherwise, the simulator will not work properly.

Helm monitoring should be installed before simulator is run.

Usage

Prepare:

# Generate zerostate config stub (with optional --force flag):
just init_zerostate_config
# Generate node config sub (with optional --force flag):
just init_node_config
# Generate a network of 4 nodes (with optional --force flag):
just gen_network 4
# Only for k3s after each reboot (requires sudo)
chmod 644 /etc/rancher/k3s/k3s.yaml
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Install simulator:

cargo install --path ./simulator
  # Or alias via `alias simulator="cargo run --bin simulator --"`

At this point you may edit default values in generated files before they are applied to helm templates:

• ${git_project_root}/config.json
• ${git_project_root}/logger.json

Use simulator:

simulator prepare
simulator build local
simulator start
simulator node logs -f
simulator node shell -n 0
simulator stop
simulator clean

helm monitoring

helm repo add vm https://victoriametrics.github.io/helm-charts/
helm repo update vm
kubectl create namespace monitoring
helm install vmks vm/victoria-metrics-k8s-stack -n monitoring --set vmagent.spec.scrapeInterval=10s --atomic --wait

Get your Grafana password for admin user:

kubectl get secret vmks-grafana --template='{{ index .data "admin-password" | base64decode}}' -n monitoring

For more details see https://docs.victoriametrics.com/helm/victoriametrics-k8s-stack/#install-operator-separately, but note that vmks release name and monitoring namespace are used in VMPodScrape to scrape metrics from pods.

Open VictoriaMetrics UI to check tycho target scraping health:

export POD_NAME=$(kubectl get pods -n monitoring -l "app.kubernetes.io/name=vmagent,app.kubernetes.io/instance=vmks-victoria-metrics-k8s-stack" -o jsonpath="{.items[0].metadata.name}")
kubectl -n monitoring port-forward $POD_NAME 8429

Open Grafana to import Tycho dashboard:

export POD_NAME=$(kubectl get pods -n monitoring -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=vmks" -o jsonpath="{.items[0].metadata.name}")
kubectl -n monitoring port-forward $POD_NAME 3000

Generate dashboard into Wayland clipboard:

../scripts/install-python-deps.sh
python ../scripts/gen-dashboard.py | wl-copy

GKE

• Update cluster to 1.32+ to avoid gke bug
• Enable k8s egress to pull images from public repos, see StackOverflow answer.
• To build the image in k8s, see builder subchart

Set kubectl to the right cluster.

Use prepare to create default values.yaml that slightly differ from k3s.

Modify generated builder values for your registry and versions.

Every call to prepare overwrites tycho values, but lets builder values intact.

Use build install to spawn k8s builder job and remove to clean up when it finishes.

export KUBECONFIG=~/.kube/config
kubectl config use-context ${CONTEXT}

simulator prepare
simulator build install
simulator build remove

simulator start
simulator stop

To disable logs completely just leave empty json object {} in logger config.

Helm chaos-mesh

As per docs:

• chaos dashboard here does not require auth
• default namespace is a target for chaos

helm repo add chaos-mesh https://charts.chaos-mesh.org
helm repo update chaos-mesh
kubectl create ns chaos-mesh

kubectl annotate ns default chaos-mesh.org/inject=enabled

For k3s

helm install chaos-mesh chaos-mesh/chaos-mesh -n chaos-mesh \
--set chaosDaemon.runtime=containerd \
--set chaosDaemon.socketPath=/run/containerd/containerd.sock \
--set controllerManager.enableFilterNamespace=true \
--set dashboard.securityMode=false \
--atomic --wait

For GKE

helm install chaos-mesh chaos-mesh/chaos-mesh -n chaos-mesh \
--set controllerManager.enableFilterNamespace=true \
--set dashboard.securityMode=false \
--atomic --wait

Open control dashboard and create experiments:

kubectl port-forward -n chaos-mesh svc/chaos-dashboard 2333:2333