Skip to content

Cloud Native

Jump to bottom Edit New page
Youngwoo Kim edited this page Mar 12, 2021 · 145 revisions

k8s yaml:

https://github.com/crossplane/crossplane

https://github.com/BrandonPotter/kubergui

secrets in git:

https://github.com/bambash/helm-cronjobs

UT/IT

pg in k8s:

Storage:

  • Longhorn
  • ChubaoFs

GCP AI/ML:

kubeflow:

https://qconsf.com/system/files/presentation-slides/control_theory_in_container_orchestration.pdf

MLOPs, Feature Store:

https://kubernetes.io/docs/reference/kubectl/cheatsheet/

https://appsody.dev/

TODO
1. Prometheus + VictoriaMetrics
2. Redash, https://github.com/getredash/contrib-helm-chart
-- 
* TSDB
- VictoriaMetrics
- M3DB
- TimescaleDB

* OLAP
- Druid
- Pinot
- ClickHouse

* HTAP
- Tidb (mysql w/p)
- Vitess (sharding mysql)

--
https://github.com/apache/incubator-pinot/issues/3
https://medium.com/@leventov/comparison-of-the-open-source-olap-systems-for-big-data-clickhouse-druid-and-pinot-8e042a5ed1c7
https://blog.cloudflare.com/http-analytics-for-6m-requests-per-second-using-clickhouse/
https://medium.com/faun/victoriametrics-creating-the-best-remote-storage-for-prometheus-5d92d66787ac

https://www.inovex.de/blog/spinnaker-vs-argo-cd-vs-tekton-vs-jenkins-x/

https://github.com/fabric8io/kubernetes-client/blob/master/doc/CHEATSHEET.md

KAAS, https://github.com/oneinfra/oneinfra

https://github.com/askmeegs/learn-istio

ES:

Flink operator:

https://lapee79.github.io/article/setup-production-ready-kubernetes-on-baremetal-with-kubespray/

https://lapee79.github.io/article/loki-kubernetes-logging/

MetalLB:

https://github.com/ahmetb/kubernetes-network-policy-recipes

https://ruzickap.github.io/k8s-postgresql/#requirements

kafka operator, https://medium.com/@prune998/banzaicloud-kafka-operator-tour-56fca7d6261e

MLHub is based on JupyterHub with complete focus on Docker and Kubernetes., https://github.com/ml-tooling/ml-hub

k8s authentication:

https://github.com/stefanprodan/k8s-prom-hpa

Druid on k8s:

Flink k8s operator:

Air-gapped:

Ceph & BlueStore:

kubeconfig:

Backup and data migration:

Cloud native storage - https://thenewstack.io/ebooks/storage/state-of-state-cloud-native-storage-for-developers/

k8s benchmark:

https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/configuration/assign-pod-node/

https://github.com/ibm-cloud-architecture/refarch-eda/

https://www.youtube.com/watch?time_continue=14&v=b1RsNXGLuUk&feature=emb_logo

k8s debug:

https://www.cloudops.com/2019/05/the-ultimate-rook-and-ceph-survival-guide/

https://kubesail.com/blog/storage-on-kubernetes/

HCI + k8s:

API Gateway:

https://thebsdbox.co.uk/2020/01/02/Designing-Building-HA-bare-metal-Kubernetes-cluster/

Cloud IDE:

https://www.cncf.io/wp-content/uploads/2019/06/2019_04_Kubernetes-in-Highly-Secure-Environments-2-1.pdf

https://github.com/apache-spark-on-k8s/kubernetes-HDFS

https://medium.com/@vamsi.chemitiganti/how-kubernetes-transforms-your-business-26f931d5e263

In a nutshell, Containers are changing the way applications are being architected, designed, developed, packaged, delivered and managed.

https://medium.com/faun/development-in-the-cloud-4aa2cabd3880

https://sysdig.com/blog/kubernetes-limits-requests/

https://medium.com/@imarunrk/continuous-gitops-the-way-to-do-devops-in-kubernetes-896b0ea1d0fb

https://www.magalix.com/blog/kubernetes-authorization

Helm:

Rook & Ceph:

Presto on K8s:

k8s, rook-ceph, presto:

service catalog:

k8s ingress & bare metal:

Dell k8s ref arch.:

Procella:

https://monitoring2.substack.com/p/big-prometheus

https://github.com/Aracki/toolnetes

Docker image downloader:

https://engineering.linecorp.com/en/blog/monitoring-a-spring-boot-app-in-kubernetes-what-i-learned-from-devoxx-belgium-2019/

kubectl web ui:

Kubernetes patterns:

k8s offline install:

k8s & HCI:

참고:

azure stack:

https://github.com/lyft/flyte

benchmark:

https://github.com/Leverege/kubernetes-book

https://github.com/shubheksha/Kubernetes-Up-and-Running-Notes

https://github.com/dennyzhang/cheatsheet-kubernetes-A4

https://github.com/dennyzhang/kubernetes-yaml-templates

Minikube + docker registry:

k8s networking:

https://blogs.vmware.com/cloudnative/2019/10/22/five-key-decisions-to-make-before-running-kubernetes-in-production/

https://kccnceu19.sched.com/event/MQhi/keynote-debunking-the-myth-kubernetes-storage-is-hard-saad-ali-senior-software-engineer-google

Storage: https://blog.calsoftinc.com/2019/10/demystifying-persistent-storage-myths-for-stateful-workloads-in-kubernetes.html https://calsoftinc.com/download/25085/

https://vitobotta.com/2019/08/06/kubernetes-storage-openebs-rook-longhorn-storageos-robin-portworx/

https://platform9.com/blog/kubernetes-storage-dynamic-volumes-and-the-container-storage-interface/

https://platform9.com/blog/tutorial-dynamic-provisioning-of-persistent-storage-in-kubernetes-with-minikube/

https://www.cloudjourney.io/articles/cloudnative/packaging-applications-for-k8s-di/

k8s + python, https://srcco.de/posts/kubernetes-and-python.html

Operator (with Helm or Ansible):

# operator-sdk
$ brew install operator-sdk

# OLM
$ kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/0.12.0/crds.yaml
$ kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/0.12.0/olm.yaml

or 
$ curl -L https://github.com/operator-framework/operator-lifecycle-manager/releases/download/0.12.0/install.sh -o install.sh
$ chmod +x install.sh
$ ./install.sh 0.12.0

Automated provisioning in Kubernetes

Prometheus

Minikube:

cd $BIGTOP_HOME

minikube start --cpus 4 --memory 4096 --container-runtime=cri-o 

kubectl cluster-info

minikube mount .:/bigtop

minikube ssh 
$ ls -als /bigtop

podman, buildah, cri-o

https://prefetch.net/blog/2019/10/16/the-beginners-guide-to-creating-kubernetes-manifests

https://github.com/kubeapps/kubeapps

PV, PVC & StorageClass, https://www.youtube.com/watch?v=qktFhjJmFhg&feature=share

https://rancher.com/blog/2019/2019-03-21-comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/

https://www.weave.works/blog/kubernetes-faq-configure-storage-for-bare-metal-cluster

k8s storage & SAN

https://github.com/mhausenblas/stateful-kubernetes

https://github.com/openebs/node-disk-manager

https://softwareengineeringdaily.com/2019/01/11/why-is-storage-on-kubernetes-is-so-hard/

k8s web ui:

https://platform9.com/blog/kubernetes-helm-why-it-matters/

https://thenewstack.io/big-data-google-replaces-yarn-with-kubernetes-to-schedule-apache-spark/

private registry + helm chart repo

https://www.awsgeek.com/

k8s storage:

https://github.com/operator-framework/awesome-operators https://github.com/schoolofdevops/ultimate-kubernetes-bootcamp

https://github.com/cncf/landscape

https://dataworkssummit.com/san-jose-2018/session/containers-and-big-data/

https://www.abhishek-tiwari.com/kubernetes-for-big-data-workloads/

https://dantehranian.wordpress.com/2015/03/25/how-should-i-get-application-configuration-into-my-docker-containers/

https://lentiq.com/assets/docs/Introduction_to_Kubernetes.pdf

GUI

https://www.sugarkube.io/

k8s

https://www.youtube.com/watch?time_continue=48&v=8C_SCDbUJTg&feature=emb_logo

https://github.com/kubernetes/examples

conf validator / linter

k8s + vagrant + ansible (for dev)

https://yk.surfingstudio.com/articles/automatic-setup-kubernetes-with-kubeadm-by-using-ansible-in-vagrant-environment/

There are already mature solutions for automatic Kubernetes setup, e.g.:

kubespray
kop

https://github.com/yklin/play_k8s_on_vagrant_with_ansible

https://spr.com/4-tools-to-automate-kubernetes-cluster-deployments/

https://github.com/ahmetb/kubectx

https://github.com/kubernetes-sigs/krew/

Docker images

Superset

Presto

Kafka

Flink

Jupyter

docker run --rm --user root -p 8888:8888 -e GRANT_SUDO=yes -e JUPYTER_ENABLE_LAB=yes -v "$PWD":/work jupyter/datascience-notebook:latest
Add a custom footer
Add a custom sidebar
Clone this wiki locally