INTRODUCTIONKubernetes is an open-source, container managementdeployment, but also for managing multiple containerssolution originally announced by Google in a single entity for the purposes of scaling,After its initial release in July 2015, Google donatedavailability, and so on.Kubernetes to the Cloud Native Computing Foundation.Since then, several stable versions have been releasedBeing infrastructure agnostic, Kubernetes clusters canunder Apache installed on a variety of public and private clouds(AWS, Google Cloud, Azure, OpenStack) and on bareFor a developer, Kubernetes provides a manageablemetal servers. Additionally, Google Container Engineexecution environment for deploying, running,can provide a deployed Kubernetes cluster. This makesmanaging, and orchestrating containers across clustersKubernetes similar to Linux kernel, which providesor clusters of hosts. For devops and administrators,consistency across different hardware platforms, orKubernetes provides a complete set of buildingJava, which runs on almost any operating system.blocks that allow the automation of many operationsfor managing development, test, and productionenvironments. Container orchestration enablescoordinating containers in clusters consisting of multiplenodes when complex containerized applicationsare deployed. This is relevant not only for the initial3Everything Kubernetes: A Practical GuideStratoscale

KUBERNETES — HIGHLEVEL ARCHITECTURENODEA Kubernetes cluster consists of one or more nodes managed by Kubernetes. The nodes are bare-metal servers,on-premises VMs, or VMs on a cloud provider. Every node contains a container runtime (for example, Docker Engine),kubelet (responsible for starting, stopping, and managing individual containers by requests from the Kubernetescontrol plane), and kube-proxy (responsible for networking and load balancing).MASTER NODEA Kubernetes cluster also contains one or more master nodes that run the Kubernetes control plane. The control planeconsists of different processes, such as an API server (provides JSON over HTTP API), scheduler (selects nodes to runcontainers), controller manager (runs controllers, see below), and etcd (a globally available configuration store).DASHBOARD AND CLIA Kubernetes cluster can be managed via the Kubernetes Dashboard, a web UI running on the master node. The clustercan also be managed via the command line tool kubectl, which can be installed on any machine able to access the APIserver, running on the master node. This tool can be used to manage several Kubernetes clusters by specifying a contextdefined in a configuration file.4Everything Kubernetes: A Practical GuideStratoscale

InternetNodeKubletProxyKubecti(CLI)DockerMaster Everything Kubernetes: A Practical GuideContainerStratoscale

KUBERNETESBUILDING BLOCKSKubernetes provides basic mechanisms for the deployment, maintenance, and scaling of containerized applications. Ituses declarative primitives, or building blocks, to maintain the state requested by the user, implementing the transitionfrom the current observable state to the requested state.THE BASICSPODA pod is the smallest deployable unit that can be managed by Kubernetes. A pod is a logical group of one or morecontainers that share the same IP address and port space. The main purpose of a pod is to support co-locatedprocesses, such as an application server and its local cache. Containers within a pod can find each other via localhost,and can also communicate with each other using standard inter-process communications like SystemV semaphoresor POSIX shared memory. In other words, a pod represents a “logical host”. Pods are not durable; they will notsurvive scheduling failures or node failures. If a node where the pod is running dies, the pod is deleted. It can then bereplaced by an identical pod, with even the same name, but with a new unique identifier (UID).LABELSELECTORA label is a key/value pair that is attached toA label selector can be used to organize KubernetesKubernetes resource, for example, a pod. Labelsresources that have labels. An equality-based selectorcan be attached to resources at creation time, asdefines a condition for selecting resources that havewell as added and modified at any later time.the specified label value. A set-based selector definesa condition for selecting resources that have a labelvalue within the specified set of values.6Everything Kubernetes: A Practical GuideStratoscale

CONTROLLERREPLICATION CONTROLLERA controller manages a set of pods and ensures thatA replication controller is responsible for running thethe cluster is in the specified state. Unlike manuallyspecified number of pod copies (replicas) across thecreated pods, the pods maintained by a replicationcluster.controller are automatically replaced if they fail,get deleted, or are terminated. There are severalcontroller types, such as replication controllers ordeployment controllers.DEPLOYMENT CONTROLLERREPLICA SETA deployment defines a desired state for logicalA replica set is the next-generation replicationgroup of pods and replica sets. It creates newcontroller. A replication controller supports onlyresources or replaces the existing resources, ifequality-based selectors, while a replica set supportsnecessary. A deployment can be updated, rolledset-based selectors.out, or rolled back. A practical use case for adeployment is to bring up a replica set and pods,then update the deployment to re-create thepods (for example, to use a new image). Later,the deployment can be rolled back to an earlierrevision if the current deployment is not stable.SERVICEA service uses a selector to define a logical group of pods and defines a policy to access such logical groups. Becausepods are not durable, the actual pods that are running may change. A client that uses one or more containers withina pod should not need to be aware of which specific pod it works with, especially if there are several pods (replicas).There are several types of services in Kubernetes, including ClusterIP, NodePort, LoadBalancer. A ClusterIP serviceexposes pods to connections from inside the cluster. A NodePort service exposes pods to external traffic byforwarding traffic from a port on each node of the cluster to the container port. A LoadBalancer service also exposespods to external traffic, as NodePort service does, however it also provides a load balancer.7Everything Kubernetes: A Practical GuideStratoscale

USING LABELS AND SELECTORS FORFINE-GRAINED CONTROLA Kubernetes controller, for example, uses a selector to define a set of managed pods so that pods in that set have thecorresponding label. A label is just a key/value pair that is attached to Kubernetes resources such as pods. Labels canbe attached to resources when they are created, or added and modified at any time. Each resource can have multiplelabels. For example:release: stableenvironment: devA label selector defines a set of resources by specifying a requirements for their labels. For example:environment devenvironment ! liveenvironment in (dev, test)environment notin (live)release stable, environment devThe first two selectors have an equality-based requirement, the third and fourth selectors have a set-basedrequirement. The last selector contains the comma separator, which acts as a logical “AND” operator, so the selectordefines a set of resources where the label “release” equals “stable” and the label “environment” equals “dev.”SERVICE DISCOVERYKubernetes supports finding a service in two ways: through environment variables and using DNS.ENVIRONMENT VARIABLESKubernetes injects a set of environment variables into pods for each active service. Such environment variablescontain the service host and port, for example:MYSQL SERVICE HOST SERVICE PORT 3306An application in the pod can use these variables to establish a connection to the service.The service should be created before the replication controller or replica set creates a pod’s replicas. Changes made toan active service are not reflected in a previously created replica.DNSKubernetes automatically assigns DNS names to services. A special DNS record can be used to specify port numbersas well. To use DNS for service discovery, a Kubernetes cluster should be properly configured to support it.8Everything Kubernetes: A Practical GuideStratoscale

3 STORAGE BUILDING BLOCKSVOLUMEA container file system is ephemeral: if a container crashes, the changes to its file system are lost. A volumeis defined at the pod level, and is used to preserve data across container crashes. A volume can be also usedto share data between containers in a pod. A volume has the same lifecycle as the the pod that encloses it—when a pod is deleted, the volume is deleted as well. Kubernetes supports different volume types, which areimplemented as plugins.PERSISTENT VOLUMEA persistent volume represents a real networked storage unit in a cluster that has been provisioned by anadministrator. Persistent storage has a lifecycle independent of any individual pod. It supports different accessmodes, such as mounting as read-write by a single node, mounting as read-only by many nodes, and mountingas read-write by many nodes. Kubernetes supports different persistent volume types, which are implemented asplugins. Examples of persistent volume types include AWS EBS, vSphere volume, Azure File, GCE Persistent Disk,CephFS, Ceph RBD, GlusterFS, iSCSI, NFS, and Host Path.PERSISTENT VOLUME CLAIMA persistent volume claim defines a specific amount of storage requested and specific access modes. Kubernetesfinds a matching persistent volume and binds it with the persistent volume claim. If a matching volume does notexist, a persistent volume claim will remain unbound indefinitely. It will be bound as soon as a matching volumebecome available.9Everything Kubernetes: A Practical GuideStratoscale

CHOOSING THE RIGHT BLOCK FORTHE JOBDesigned as a simple building block; a replicationselectors. From this perspective, a replica set is justcontroller’s only responsibility is to maintain the specifieda more advanced version of a replication controller.number of replicas. A replication controller countsonly live pods;, terminated pods are excluded. OtherUsing only pods and replication controllersKubernetes building blocks should be used togetherto deploy an application is, at least in part, anwith replication controllers for more advanced tasks.imperative form of managing software, becauseFor example, an autoscaler can monitor application-it usually requires manual steps. A Kubernetesspecific metrics and dynamically change the number ofdeployment is an alternative that enablesreplicas in the existing replication controller. In addition,completely declarative application deployment.a replication controller does not support schedulingpolicies, meaning you cannot provide rules for choosingcluster nodes to run pods from the managed set.A replica set is another Kubernetes building block. Themajor difference between it and a replication controller isthat replication controllers do not support selectors withset-based requirements, while replica sets support suchSECRETCONFIG MAPA Kubernetes secret allows users to pass sensitiveA Kubernetes config map allows users toinformation, such as passwords, authenticationexternalize application configuration parameterstokens, SSH keys, and database credentials, tofrom a container image and define applicationcontainers. A secret can then be referenced whenconfiguration details, such as key/value pairs,declaring a container definition, and read fromdirectory content, or file content. Config mapwithin containers as environment variables orvalues can be consumed by applications throughfrom a local disk.environment variables, local disks, or commandline arguments.10Everything Kubernetes: A Practical GuideStratoscale

JOBA job is used to create one or more pods and ensure that a specified number of them successfully terminate.It tracks the successful completions, and when a specified number of successful completions is reached, thejob itself is complete. There are several types of jobs, including non-parallel jobs, parallel jobs with a fixedcompletion count, and parallel jobs with a work queue. A job should be used instead of a replication controllerif you need to spread pods across cluster nodes and ensure, for example, so that each node has only onerunning pod of the specified type.DAEMON SETNAMESPACEA daemon set ensures that all or some nodesA namespace provides a logical partition of therun a copy of a pod. A daemon set tracks thecluster’s resources. Kubernetes resources canadditional and removal of cluster nodes and addsuse the same name when found in differentpods for nodes that are added to the cluster,namespaces. Different namespaces can beterminates pods on nodes that are being removedassigned different quotas for resource limitations.from a cluster. Deleting a daemon set will cleanup the pods it created. A typical use case for adaemon set is running a log collection daemon ora monitoring daemon on each node of a cluster.QUOTAA quota sets resource limitations, such as CPU,memory, number of pods or services, for a givennamespace. It also forces users to explicitlyrequest resource allotment for their pods.11Everything Kubernetes: A Practical GuideStratoscale

IMPERATIVE VS.DECLARATIVEORCHESTRATIONBefore getting to the practical steps of the KubernetesA declarative approach for administrative tasks isdeployment, it’s important to understand the keyintended to solve such challenges. With a declarativeapproaches to orchestration.approach, an administrator defines a target state fora system (application, server, or cluster). Typically, aThe classic imperative approach for managing softwaredomain-specific language (DSL) is used to describeinvolves several steps or tasks, some of which are manual.the target state. An administrative tool, such asWhen working in a team, it is usually required that theseKubernetes, takes this definition as an input andsteps be documented, and, in an ideal case, automated.takes care of how to achieve the target state from thePreparing good documentation for a classic imperativecurrent observable state.administrative procedure and automating these steps canbe non-trivial tasks, even if each of the steps is simple.12Everything Kubernetes: A Practical GuideStratoscale

HANDS-ON:GETTING STARTEDMinikube is an ideal tool for getting started withIn the following instructions, Minikube is usedKubernetes on a single computer. It enables running ofto install a single-node Kubernetes cluster on aa single-node Kubernetes cluster in a virtual machine.machine with 64 bit GNU/Linux (Debian or Ubuntu)It can be used on GNU/Linux or OS X and requiresand KVM. Refer to the Minikube documentation ifVirtualBox, KVM (for Linux), xhyve (OS X), or VMwareyou want to use an alternative configuration.Fusion (OS X) to be installed on your computer. Minikubecreates a new virtual machine with GNU/Linux, installsand configures Docker and Kubernetes, and finally runs aKubernetes cluster.13Everything Kubernetes: A Practical GuideStratoscale

INSTALLATION1. Install the kubectl command line tool locally: curl -Lo kubectl \release/v1.3.0/bin/linux/amd64/kubectl \&& chmod x kubectl \&& sudo mv kubectl /usr/local/bin/2. Next, install the KVM driver: sudo curl -L eleases/download/v0.7.0/docker-machine-driver-kvm \-o /usr/local/bin/docker-machine-driver-kvm sudo chmod x /usr/local/bin/docker-machine-driver-kvm3. Install Minikube: curl -Lo minikube /v0.6.0/minikube-linux-amd64 \&& chmod x minikube \&& sudo mv minikube /usr/local/bin/14Everything Kubernetes: A Practical GuideStratoscale

4. Start the Minikube cluster: minikube start --vm-driver kvmStarting local Kubernetes cluster.Kubernetes is available at is now configured to use the cluster.The Kubernetes cluster is up and running.Let’s start with a simple deployment using an existing image: kubectl run hello-minikube \--image containers/echoserver:1.4 \--port 8080deployment “hello-minikube” created kubectl expose deployment hello-minikube --type NodePortservice “hello-minikube” exposed5. Check that the pod is up and running: kubectl get ESTARTS1/1AGERunning04m“Running” should appear in the STATUS field. If “ContainerCreating” appears instead,wait a few moments, then repeat the last command.15Everything Kubernetes: A Practical GuideStratoscale

6. Check that the service works: curl (minikube service hello-minikube --url)CLIENT VALUES:client address GETreal path /query nilrequest version 1.1request uri VALUES:server version nginx: 1.10.0 - lua: 10001HEADERS RECEIVED:accept */*host curl/7.35.0BODY:-no body in request-7. Execute the following command to open the KubernetesDashboard in your web browser: minikube dashboard16Everything Kubernetes: A Practical GuideStratoscale

8. To stop the cluster (shut down the virtual machine andpreserve its state), execute the following command: minikube stopStopping local Kubernetes cluster.Stopping “minikubeVM”.9. To start the cluster again and restore it to the previous state,execute the following command: minikube start --vm-driver kvm10. To delete the cluster (delete the virtual machine and its state),execute the following command: minikube deleteNote: There are other open-source tools, such as kubeadm, that simplify installation ofKubernetes cluster in public clouds, on-premises virtual machines, and bare-metal servers.However, there are still many things that are out of scope. For example, you still needa reliable distributed block or file storage, you still need to think about HA, scalability,networking and security. Sometimes, it is simpler to use Kubernetes as a Service.17Everything Kubernetes: A Practical GuideStratoscale

LOGGINGBasic logging in Kubernetes behaves much like logging in Docker. There is a kubectl logs command thatwill show you all the information written to stdout and stderr for a given container in a pod. If a pod hasonly one container, there is no need to specify it explicitly, however when a pod has several containerswe need to add -c container to the end of the command. As with Docker, we can opt to followlogs, to reduce the number of recent lines with --tail and we can filter them by date. Unlike Docker,Kubernetes enables us to check the logs of a container that crashed using the --previous option.The ability to keep the logs of a previous container is available as long as the pod it was run in remainsavailable. When a pod is removed, so are its logs.Log rotation is performed by Kubernetes. The default values are daily rotation or 10 MB to avoid log filestaking up all the available disk space. Up to five rotations are kept for historical evidence. Remember thatonly the last rotation is displayed with kubectl logs; if you want to access an earlier one, you must do somanually.Per our example with the Hello Minikube service, we can use kubectl logs hello-minikube2433534028-ouxw8 to view the access log containing our curl request.All information so far concerns per-node log files. There are no cluster-level logging capabilities built intoKubernetes, but there are some common methods that can be implemented for this purpose.DEDICATED AGENT RUNNING ON EVERY NODEIn this approach, a logging agent is run on every node, preferably through a DeamonSet replica. Apopular choice in this space is fluentd. It can be configured with various backends among which areGoogle Cloud Platform and Elasticsearch.Fluentd even goes as far as to provide a ready to use DaemonSet YAML on GitHub. All that needsto be done is edit its configuration to point at our logging backend.DEDICATED CONTAINER INSIDE A PODThere are several use cases for this approach. The general idea is that a dedicated loggingcomponent in each pod either writes logs to its stdout and stderr or delivers the logs directly to alogging backend. With such a sidecar approach, we can aggregate all logs from different containersin a pod to a single stream that can be accessed with kubectl logs or we can split logs of oneapplication into different logical streams. For example, Webservers’ access.log and error.log can eachbe streamed by a dedicated container to its stdout, so we can check them separately with kubectllogs podname -c access-log and kubectl logs podname -c error-log.DIRECT LOGGING FROM AN APPLICATIONIf the application running in a pod can already communicate with a logging backend, it is possible toskip the Kubernetes logging options altogether. While direct logging may offer some performanceadvantages and provide slightly better security (all data stored in just one place), it also prevents usfrom using kubectl logs. In most cases, this approach is discouraged.18Everything Kubernetes: A Practical GuideStratoscale

MONITORINGAs Kubernetes containers are actually Linux processes, we can use our favourite tools to monitorcluster performance. Basic tools, such as top or kubectl top, will behave as expected. It’s alsopossible to use solutions that are dedicated to Kubernetes. One such solution is Heapster. Heapsteraggregates events and data from across the cluster. It runs as a pod in system namespace. Discoveryand querying of resources is done automatically with data coming from a kubelet managing node.Storage is configurable with InfluxDB and Google Cloud Monitoring being the most popular choices.When building a DIY cluster InfluxDB with Grafana for visualization is the preferred choice. Usingit with Minikube requires two easy steps. First, enable the Heapster addon (minikube addonsenable heapster) by opening the dashboard with Minikube addons. Then open Heapster.By default, dashboards are available for Cluster and for Pods, one for monitoring running nodes andoverall cluster utilization, the other for running pods. The information presented for pods includesCPU usage, memory usage, network usage, and filesystem usage. It is possible to edit existing graphsor add custom ones based on data exported by Heapster.WORKING WITH MULTIPLECLUSTERSSo far, we have used kubectl to connect to only one cluster created by Minikube. But kubectl canbe configured to use multiple clusters and multiple contexts to connect to them. To check availablecontexts, we use kubectl config get-contexts.We can confirm that only one context and only one cluster is defined by kubectl config view. It shouldlook like this:apiVersion: v1clusters:- cluster:certificate-authority: HOME/.minikube/ca.crtserver: minikubecontexts:- context:cluster: minikubeuser: minikubename: minikubekind: Configpreferences: {}users:- name: minikubeuser:client-certificate: HOME/.minikube/apiserver.crtclient-key: HOME/.minikube/apiserver.key19Everything Kubernetes: A Practical GuideStratoscale

The config file used by kubectl is stored at /.kube/config. We can edit it with a text editor andadd another cluster, context and user. When ready, kubectl config get-contexts should showour newly added context without marking it as current. This is the desired state:apiVersion: v1clusters:- cluster:certificate-authority: HOME/.minikube/ca.crtserver: minikube- cluster:certificate-authority: HOME/.minikube/ca.crtserver: secondkubecontexts:- context:cluster: minikubeuser: minikubename: minikube- context:cluster: secondkubeuser: secondkubename: secondkubecurrent-context: secondkubekind: Configpreferences: {}users:- name: minikubeuser:client-certificate: HOME/.minikube/apiserver.crtclient-key: HOME/.minikube/apiserver.key- name: secondkubeuser:client-certificate: HOME/.minikube/apiserver.crtclient-key: HOME/.minikube/apiserver.keyTo switch context, we use kubectl config use-context secondkube. We can verify theswitch was successful again with kubectl config get-contexts. The marker for current shouldhave moved to the new context. All kubectl commands from now on will be executed in a selectedcontext (which in our example is exactly the same as the first one).20Everything Kubernetes: A Practical GuideStratoscale

HANDS-ON: DEPLOYINGAN APPLICATIONIn this example, we deploy the WordPress content management system with a MySQL backend. It is a classic two-tierapplication, where the first tier is the application server (WordPress) that uses the second tier for data persistence (MySQL).STEP 1. CREATE A KUBERNETES SECRETAs discussed above, Kubernetes secrets allow users to pass sensitive information, such aspasswords, database credentials, to containers. In the first step, we need to define a Kubernetessecret that contains a database name, user, and password. It should also contain the rootpassword for MySQL.Before creating a secret, we need to encode such information in Base64 format. Let’s assume wewant to use the following values in our application: “app-db” as a database name “app-user” as a database user name “app-pass” as a database password “app-rootpass” as a database root passwordNote that we need to provide a database root password to allow WordPress to create therequired database. To encode these values to Base64 format, we can use the standard base64utility that is available in almost all Linux distributions: echo -n “app-db” base64YXBwLWRi echo -n “app-user” base64YXBwLXVzZXI echo -n “app-pass” base64YXBwLXBhc3M echo -n “app-rootpass” base64YXBwLXJvb3RwYXNz21Everything Kubernetes: A Practical GuideStratoscale

We use the “-n” option to make sure that the new line symbol (“\n”) is not included in theencoded value.To define a new Kubernetes secret, create a new file, app-secret.yaml, with the followingcontent:apiVersion: v1kind: Secretmetadata:name: app-secrettype: Opaquedata:dbname: YXBwLWRidbuser: YXBwLXVzZXI dbpass: YXBwLXBhc3M dbrootpass: YXBwLXJvb3RwYXNzNote:Kubernetes allows its building blocks to be defined using JSON and YAML formats. Theseformats are quite popular now as a more lightweight alternative to XML. The idea behind XML,JSON, and YAML is to provide a universal text notation to serialize data in both machine- andhuman-readable form. In this example, we will use YAML.In the app-secret.yaml file, we specified the required Kubernetes API version and thedata type to let Kubernetes know that we are defining a secret. In addition, the file defines fourkeys (dbname, dbuser, dbpass, dbrootpass) with the corresponding values we encodedabove. Now we can create our Kubernetes secret using its definition in the app-secret.yamlfile: kubectl create -f app-secret.yamlsecret “app-secret” createdLet’s verify the secret creation: kubectl get secretsNAMEapp-secrets22Everything Kubernetes: A Practical GuideTYPEOpaqueDATA4AGE1mStratoscale

STEP 2. CREATE A PERSISTENT VOLUMENext, we will create a Kubernetes persistent volume to provide the underlying storage for ourMySQL database. To define a new persistent volume, create a new file, app-pv.yam, with thefollowing content:apiVersion: v1kind: PersistentVolumemetadata:name: app-pvlabels:vol: mysqlspec:capacity:storage: 1GiaccessModes:- ReadWriteOncehostPath:path: /data/appIn this file, we use a HostPath volume, which will just pass a directory from the host into acontainer for consumption. To specify that the persistent volume will be used for our application,we added a label (vol: mysql) that can be used later in a selector.Before creating the actual persistent volume, verify that the directory /data/app exists: sudo mkdir -p /data/appNow we can create our persistent volume: kubectl create -f app-pv.yamlpersistentvolume “app-pv” created23Everything Kubernetes: A Practical GuideStratoscale

Let’s verify that the persistent volume is available: kubectl describe ol mysqlReclaim Policy: RetainAccess Path (bare host directory volume)/data/appNo events.STEP 3. CLAIM A PERSISTENT VOLUMEFor MySQL, we need to claim our previously created persistent volume. Create a new file, apppvc.yaml, with the following content:kind: PersistentVolumeClaimapiVersion: v1metadata:name: app-pvcspec:accessModes:- ReadWriteOnceresources:requests:storage: 1Giselector:matchLabels:vol: mysqlThe label selector “matchLabels” is used to make the association to the persistent volume thatwe created early. To create the persistent volume claim using its definition execute the following: kubectl create -f app-pvc.yamlpersistentvolumeclaim “app-pvc” created24Everything Kubernetes: A Practical GuideStratoscale

STEP 4. DEPLOY MYSQLNow we will create a new deployment for MySQL using the existing Docker image. Create a newfile, mysql-deployment.yaml, with the following content:apiVersion: extensions/v1beta1kind: Deploymentmetadata:name: mysql-deploymentspec:replicas: 1template:metadata:labels:app: mysqlspec:containers:- name: mysqlimage: “mysql:5.6”ports:- cont

Kubernetes resource, for example, a pod. Labels can be attached to resources at creation time, as well as added and modified at any later time. THE BASICS POD LABEL SELECTOR KUBERNETES BUILDING BLOCKS Kubernetes provides basic mechanisms for the deployment, m