首頁資訊 Pod的健康檢查機(jī)制

Pod的健康檢查機(jī)制

來源：泰然健康網(wǎng) 時(shí)間：2024年12月29日 15:41

1 探針簡介

對(duì)線上業(yè)務(wù)來說，保證服務(wù)的正常穩(wěn)定是重中之重，對(duì)故障服務(wù)的及時(shí)處理避免影響業(yè)務(wù)以及快速恢復(fù)一直是開發(fā)運(yùn)維的難點(diǎn)。Kubernetes提供了健康檢查服務(wù)，對(duì)于檢測(cè)到故障服務(wù)會(huì)被及時(shí)自動(dòng)下線，以及通過重啟服務(wù)的方式使服務(wù)自動(dòng)恢復(fù)。

2 使用Liveness及Readness探針

Liveness探針：主要用于判斷Container是否處于運(yùn)行狀態(tài)，比如當(dāng)服務(wù)crash或者死鎖等情況發(fā)生時(shí)，kubelet會(huì)kill掉Container，然后根據(jù)其設(shè)置的restart policy進(jìn)行相應(yīng)操作（可能會(huì)在本機(jī)重新啟動(dòng)Container，或者因?yàn)樵O(shè)置Kubernetes QoS，本機(jī)沒有資源情況下會(huì)被分發(fā)的其他機(jī)器上重新啟動(dòng)）

Readness探針：主要用于判斷服務(wù)是否已經(jīng)正常工作，如果服務(wù)沒有加載完成或工作異常，服務(wù)所在的Pod的IP地址會(huì)從服務(wù)的Endpoints中被移除，也就是說，當(dāng)服務(wù)沒有ready時(shí)，會(huì)將其從服務(wù)的load balancer中移除，不會(huì)再接受或響應(yīng)任何請(qǐng)求。

2.1 服務(wù)可用性和自動(dòng)恢復(fù)

如果服務(wù)的健康檢查（readiness）失敗，故障的服務(wù)實(shí)例從service endpoint中下線，外部請(qǐng)求將不會(huì)再轉(zhuǎn)發(fā)到該服務(wù)上，一定程度上保證正在提供的服務(wù)的正確性，如果服務(wù)自我恢復(fù)了（比如網(wǎng)絡(luò)問題），會(huì)自動(dòng)重新加入service endpoint對(duì)外提供服務(wù)。

另外，如果設(shè)置了Container（liveness）的探針，對(duì)故障服務(wù)的Container（liveness）的探針同樣會(huì)失敗，container會(huì)被kill掉，并根據(jù)原設(shè)置的container重啟策略，系統(tǒng)傾向于在其原所在的機(jī)器上重啟該container、或其他機(jī)器重新創(chuàng)建一個(gè)pod;

2.2 建議

1 . 對(duì)全部服務(wù)同時(shí)設(shè)置服務(wù)（readiness）和Container（liveness）的健康檢查。

2 . 通過TCP對(duì)端口檢查（TCPSocketAction），僅適用于端口已關(guān)閉或進(jìn)程停止情況。因?yàn)榧词狗?wù)異常，只要端口是打開狀態(tài)，健康檢查仍然是通過的。

3 . 基于第二點(diǎn)，一般建議用ExecAction自定義健康檢查邏輯，或采用HTTP Get請(qǐng)求進(jìn)行檢（HTTPGetAction）

4 . 無論采用哪種類型的探針，一般建議設(shè)置檢查服務(wù)（readiness）的時(shí)間短于檢查Container（liveness）的時(shí)間，也可以將檢查服務(wù)（readiness）的探針與Container（liveness）的探針設(shè)置為一致。目的是故障服務(wù)先下線，如果過一段時(shí)間還無法自動(dòng)恢復(fù)，那么根據(jù)重啟策略，重啟該Container、或其他機(jī)器重新創(chuàng)建一個(gè)Pod恢復(fù)故障服務(wù)。

2.3 example liveness.yaml

容器會(huì)創(chuàng)建一個(gè)文件/tmp/healthy,30秒后刪除；探針5秒會(huì)檢查一次，檢查方式為cat /tmp/healthy文件是否存在，檢查到容易有問題，探測(cè)失敗3次，則重建容器

[root@k8s-master-10 Liveness]# cat liveness.yaml apiVersion: v1 kind: Pod metadata: labels: test: liveness name: liveness-exec-changjian spec: containers: - name: liveness image: busybox args: - /bin/sh - -c - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600 livenessProbe: exec: command: - cat - /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5

3 存活探針

默認(rèn)情況下：以鏡像打包基于容器資源編排系統(tǒng)創(chuàng)建的pod對(duì)于用戶和服務(wù)就相當(dāng)于一個(gè)黑盒。想要探測(cè)容器中的用戶部署的應(yīng)用和服務(wù)是否正常，都被容器編排系統(tǒng)(k8s)所阻擋。正常情況下，任何一個(gè)為云原生開發(fā)的程序都會(huì)考慮到這個(gè)問題，為了監(jiān)視容器中運(yùn)行的“應(yīng)用”正常各種指標(biāo)應(yīng)該向外輸出，比如，健康狀態(tài)，實(shí)現(xiàn)的方式，比如使用一個(gè)URL或者VirtualHost的單獨(dú)容器對(duì)外輸出健康狀態(tài)指標(biāo)。暴露給容器邊界的外部。

Metrics:提供的指標(biāo)數(shù)據(jù)，有業(yè)務(wù)級(jí)，系統(tǒng)級(jí)等 tracing: 分布式鏈路追蹤可能會(huì)使用，可以認(rèn)為埋點(diǎn)或者叫做探針探針就相當(dāng)于一個(gè)接口，一個(gè)管道。后期直接使用即可，可以理解為URI或者端口探測(cè) readiness: 就緒狀態(tài)檢測(cè) liveness: 存活狀態(tài)檢測(cè)

Pod默認(rèn)提供探針的接口: Liveness

下面非常重要:

判斷一個(gè)pod中容器運(yùn)行健康狀態(tài)與否：有兩種探針，而且周期性運(yùn)行 liveness和 readiness；

liveness：如果容器沒有正常運(yùn)行，或者沒有得到正確的值，kubelet會(huì)根據(jù)容器重啟策略，重啟或者殺死容器；

readiness: 判斷容器內(nèi)的應(yīng)用程序從啟動(dòng)，到應(yīng)用程序是否正常運(yùn)行，能夠提供用戶正常訪問和接受客戶端請(qǐng)求，如果一個(gè)容器沒有通過就緒檢測(cè)，而容器可能會(huì)重啟它，service會(huì)把對(duì)應(yīng)的主機(jī)ip從后端移除，直到下次健康檢測(cè)正常才把它加進(jìn)來。

spec: containers: - name: … image: … livenessProbe: exec <Object> # 命令式探針 httpGet <Object> # http GET類型的探針 tcpSocket <Object> # tcp Socket類型的探針 initialDelaySeconds <integer> # 發(fā)起初次探測(cè)請(qǐng)求的延后時(shí)長建議設(shè)置長一點(diǎn)，有些服務(wù)啟動(dòng)很慢 periodSeconds <integer> # 請(qǐng)求周期 timeoutSeconds <integer> # 超時(shí)時(shí)長 successThreshold <integer> # 成功閾值 (表示探測(cè)n次成功，才表示容器是健康的，狀態(tài)改變之后的確認(rèn)次數(shù)) failureThreshold <integer> # 失敗閾值狀態(tài)改變之后，探測(cè)n此失敗才確認(rèn)失敗

Pod默認(rèn)提供的三種探針方式:

1 . LivenessProbe: 周期性探測(cè), 檢測(cè)未通過時(shí)，kubelet會(huì)根據(jù)restartPolicy的定義來決定是否會(huì)重啟該容器；未定義時(shí)，Kubelet認(rèn)為只容器未終止，即為健康;

以下是對(duì)存活探針方式的三種訪問參數(shù),ReadinessProbe訪問方式一樣:

命令查看readinessProbe有哪些參數(shù)

[root@k8s-master-10 Liveness]# kubectl explain pods.spec.containers.readinessProbe KIND: Pod VERSION: v1 RESOURCE: readinessProbe <Object> DESCRIPTION: Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic. FIELDS: exec<Object> One and only one of the following should be specified. Exec specifies the action to take. failureThreshold<integer> Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. httpGet<Object> HTTPGet specifies the http request to perform. initialDelaySeconds<integer> Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes periodSeconds<integer> How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. successThreshold<integer> Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. tcpSocket<Object> TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported timeoutSeconds<integer> Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes

查看exec探測(cè)幫助:

[root@k8s-master-10 Liveness]# kubectl explain pods.spec.containers.readinessProbe.exec KIND: Pod VERSION: v1 RESOURCE: exec <Object> DESCRIPTION: One and only one of the following should be specified. Exec specifies the action to take. ExecAction describes a "run in container" action. FIELDS: command<[]string> Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.

查看http方式幫助:

[root@k8s-master-10 Liveness]# kubectl explain pods.spec.containers.readinessProbe.httpGet KIND: Pod VERSION: v1 RESOURCE: httpGet <Object> DESCRIPTION: HTTPGet specifies the http request to perform. HTTPGetAction describes an action based on HTTP Get requests. FIELDS: host<string> Host name to connect to, defaults to the pod IP. You probably want to set "Host" in httpHeaders instead. httpHeaders<[]Object> Custom headers to set in the request. HTTP allows repeated headers. path<string> Path to access on the HTTP server. port<string> -required- Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. scheme<string> Scheme to use for connecting to the host. Defaults to HTTP.

注意:

initialDelaySeconds 表示容器在啟動(dòng)之后，如果不設(shè)置時(shí)間，可能就是馬上進(jìn)行存活檢測(cè)，因?yàn)榇藭r(shí)有些大應(yīng)用可能還沒有啟動(dòng)，就檢測(cè)失敗了，檢測(cè)失敗之后又自動(dòng)重啟了，所以就處于重啟的循環(huán)當(dāng)中。所以此處應(yīng)當(dāng)設(shè)置一個(gè)延時(shí)等待時(shí)間。等容器中應(yīng)用都啟動(dòng)好之后，再進(jìn)行檢測(cè)。

ReadinessProbe:

周期性探測(cè),檢測(cè)未通過時(shí)，與該P(yáng)od關(guān)聯(lián)的Service,會(huì)將Pod從Service的后端可用端點(diǎn)列表中刪除;直接再次就緒，重新添加回來。未定義時(shí)，只要容器未終止就是就緒;

StartProbe:

1.16版本之后支持，啟動(dòng)狀態(tài)檢測(cè)，檢測(cè)容器剛剛啟動(dòng)是成功的，只有他通過之后，查看是否有LivenessProbe,然后生效LivenessProbe,一般用于大型服務(wù)啟動(dòng)時(shí)檢測(cè);

以上的三種探針都支持以下三種類似的檢測(cè)方式:

下面三種檢測(cè)方法:

1 . ExecAction: 直接執(zhí)行命令，命令成功返回探測(cè)成功;制作容器時(shí)候預(yù)留一個(gè)探測(cè)接口ip/api;

2 . TCPSocketAaction: 端口能夠正常打開，對(duì)于握手能夠正常相應(yīng)，發(fā)起三次握手。握手相應(yīng)，表示成功;

3 . HTTPGetAction: 向指定的Path發(fā)起HTTP請(qǐng)求,2xx,3xx表示響應(yīng)碼成功;

3.1 基于存活探針使用命令測(cè)試

apiVersion: v1 kind: Pod metadata: name: liveness-exec-demo namespace: default spec: containers: - name: demo image: ikubernetes/demoapp:v1.0 imagePullPolicy: IfNotPresent livenessProbe: exec: command: ['/bin/sh', '-c', '[ "$(curl -s 127.0.0.1/livez)" == "OK" ]'] initialDelaySeconds: 5 timeoutSeconds: 1 periodSeconds: 5 #執(zhí)行上面的demo，判斷執(zhí)行的命令是否等于ok,如果等于ok，表示探測(cè)是成功的。 kubectl apply -f .

提交post請(qǐng)求,容器會(huì)自動(dòng)重啟

[root@k8s-master1 ~]# curl 10.244.135.28/livez OK[root@k8s-master1 ~] curl -XPOST -d "livez=Faile" 10.244.135.28/livez3.3 使用tcp測(cè)試

example1

[root@k8s-master1 lianxi]# cat liveness-tcpsocket-demo.yaml apiVersion: v1 kind: Pod metadata: name: liveness-tcpsocket-demo namespace: default spec: containers: - name: demo image: ikubernetes/demoapp:v1.0 imagePullPolicy: IfNotPresent ports: - name: http containerPort: 80 securityContext: capabilities: add: - NET_ADMIN #方便后續(xù)添加iptables測(cè)試 livenessProbe: tcpSocket: port: http periodSeconds: 5 initialDelaySeconds: 5 [root@k8s-master-10 Liveness]# kubectl exec liveness-tcpsocket-demo -- iptables -A INPUT -p tcp --dport 80 -j REJECT # 容器立馬探測(cè)失敗,5s之后就會(huì)重啟 [root@k8s-master-10 ~]# kubectl get pods -w NAME READY STATUS RESTARTS AGE liveness-tcpsocket-demo 1/1 Running 0 4h23m liveness-tcpsocket-demo 1/1 Running 1 4h24m

example2

[root@k8s-master-10 Liveness]# cat nginx-tcp-liveness.yaml apiVersion: v1 kind: Pod metadata: name: nginx-tcp-liveness-probe spec: containers: - name: nginx-tcp-liveness-probe image: nginx:latest ports: - name: http-80-port protocol: TCP containerPort: 80 livenessProbe: tcpSocket: port: 80 initialDelaySeconds: 3 periodSeconds: 10 timeoutSeconds: 3 [root@k8s-master-10 Liveness]# kubectl apply -f nginx-tcp-liveness.yaml pod/nginx-tcp-liveness-probe created [root@k8s-master-10 Liveness]# kubectl get pods -o wide |grep nginx nginx-tcp-liveness-probe 1/1 Running 0 2m5s 172.17.66.5 192.168.0.14 <none> <none> # 模擬故障,進(jìn)入Pod安裝htop [root@k8s-master-10 Liveness]# kubectl exec -it nginx-tcp-liveness-probe bash root@nginx-tcp-liveness-probe:/# apt-get update root@nginx-tcp-liveness-probe:/# apt-get install htop root@nginx-tcp-liveness-probe:/# htop CPU[|| 5.4%] Tasks: 4, 0 thr; 1 running Mem[|||||||||||||| 792M/5.61G] Load average: 0.02 0.06 0.05 Swp[ 0K/0K] Uptime: 2 days, 22:17:44 PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command 1 root 20 0 10652 3344 2532 S 0.0 0.1 0:00.00 nginx: master process 30 nginx 20 0 11056 1764 460 S 0.0 0.0 0:00.00 nginx: worker process 31 root 20 0 3976 2072 1576 S 0.0 0.0 0:00.00 bash 311 root 20 0 4780 1676 1240 R 0.0 0.0 0:00.00 htop 運(yùn)行htop查看進(jìn)程,容器進(jìn)程通常為1; root@nginx-tcp-liveness-probe:/# kill 1 root@nginx-tcp-liveness-probe:/# command terminated with exit code 137 [root@k8s-master-10 Liveness]# kubectl get pods -w |grep nginx nginx-tcp-liveness-probe 1/1 Running 0 4m47s nginx-tcp-liveness-probe 0/1 Completed 0 4m51s nginx-tcp-liveness-probe 1/1 Running 1 5m7s [root@k8s-master-10 Liveness]# kubectl describe pod nginx-tcp-liveness-probe |tail node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 5m39s default-scheduler Successfully assigned default/nginx-tcp-liveness-probe to 192.168.0.14 Normal Pulling 48s (x2 over 5m39s) kubelet, 192.168.0.14 Pulling image "nginx:latest" Warning Unhealthy 48s kubelet, 192.168.0.14 Liveness probe failed: dial tcp 172.17.66.5:80: connect: connection refused Normal Pulled 33s (x2 over 5m23s) kubelet, 192.168.0.14 Successfully pulled image "nginx:latest" Normal Created 33s (x2 over 5m23s) kubelet, 192.168.0.14 Created container nginx-tcp-liveness-probe Normal Started 33s (x2 over 5m23s) kubelet, 192.168.0.14 Started container nginx-tcp-liveness-probe3.4 通過http進(jìn)行探測(cè)

只看響應(yīng)碼，如果是2xx,3xx表示正常，如果是4xx,5xx就是表示失敗

[root@k8s-master1 lianxi]# cat liveness-httpget-demo.yaml apiVersion: v1 kind: Pod metadata: name: liveness-httpget-demo namespace: default spec: containers: - name: demo image: ikubernetes/demoapp:v1.0 imagePullPolicy: IfNotPresent livenessProbe: httpGet: path: '/livez' port: 80 scheme: HTTP initialDelaySeconds: 5 [root@k8s-master-10 Liveness]# curl 172.17.66.2/livez OK

4 就緒探針

就緒檢查用于應(yīng)用接入到service的場景，用于判斷應(yīng)用是否已經(jīng)就緒完畢，即是否可以接受外部轉(zhuǎn)發(fā)的流量，健康檢查正常則將pod加入到service的endpoints中，健康檢查異常則從service的endpoints中刪除，避免影響業(yè)務(wù)的訪問。

下面是容器啟動(dòng)之后通過15s之后進(jìn)行就緒緒檢測(cè)，在15s之前不會(huì)將訪問容器的流量調(diào)度到后端的service ip，導(dǎo)致訪問失敗。通過檢測(cè)之后才將service 添加到k8s后端的service ip中;

Example1

[root@k8s-master1 lianxi]# cat readiness-httpget-demo.yaml apiVersion: v1 kind: Pod metadata: name: readiness-httpget-demo namespace: default spec: containers: - name: demo image: ikubernetes/demoapp:v1.0 imagePullPolicy: IfNotPresent readinessProbe: httpGet: path: '/readyz' port: 80 scheme: HTTP initialDelaySeconds: 15 timeoutSeconds: 2 periodSeconds: 5 failureThreshold: 3 restartPolicy: Always

Example2

創(chuàng)建一個(gè)Pod,使用httpGet的健康檢查機(jī)制，定義readiness就緒檢查探針檢查路徑/test.html

[root@k8s-master-10 Liveness]# cat httpget-liveness-readiness-probe.yaml apiVersion: apps/v1 kind: Deployment metadata: name: nginx-tcp-liveness-probe labels: app: nginx-server spec: replicas: 1 selector: matchLabels: app: nginx-server template: metadata: labels: app: nginx-server spec: containers: - name: nginx-tcp-liveness-probe image: nginx:latest imagePullPolicy: IfNotPresent ports: - containerPort: 80 livenessProbe: #存活檢查探針 httpGet: port: 80 path: /index.html scheme: HTTP initialDelaySeconds: 3 # 容器啟動(dòng)等待多少秒后存活和就緒探測(cè)器才會(huì)被初始化,默認(rèn)0; periodSeconds: 10 # 執(zhí)行探測(cè)的時(shí)間間隔,默認(rèn)10，最小1 successThreshold: 1 # 探測(cè)器失敗后，被視為最小成功連續(xù)數(shù),默認(rèn)值為1,存活和啟動(dòng)探測(cè)這個(gè)值必須是1; timeoutSeconds: 3 # 探測(cè)超時(shí)等待多少秒,默認(rèn)1; failureThreshold: 3 # 探測(cè)失敗,k8s的重試次數(shù)，存活探測(cè)情況下的放棄意味著重啟容器, 就緒探測(cè)情況>下放棄Pod會(huì)被打上未就緒標(biāo)簽,默認(rèn)3; readinessProbe: #就緒檢查探針 httpGet: port: 80 path: /test.html scheme: HTTP initialDelaySeconds: 3 periodSeconds: 10 timeoutSeconds: 3 [root@k8s-master-10 Liveness]# cat nginx-service.yaml apiVersion: v1 kind: Service metadata: labels: app: nginx name: nginx-service spec: ports: - name: http port: 80 protocol: TCP targetPort: 80 selector: app: nginx-server type: ClusterIP # 我們可以看到就緒探針檢測(cè)不通過 [root@k8s-master-10 Liveness]# kubectl describe pod nginx-tcp-liveness-probe-57bb9c7668-qcssq |tail Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 101s default-scheduler Successfully assigned default/nginx-tcp-liveness-probe-57bb9c7668-qcssq to 192.168.0.14 Normal Pulled 101s kubelet, 192.168.0.14 Container image "nginx:latest" already present on machine Normal Created 101s kubelet, 192.168.0.14 Created container nginx-tcp-liveness-probe Normal Started 101s kubelet, 192.168.0.14 Started container nginx-tcp-liveness-probe Warning Unhealthy 6s (x10 over 96s) kubelet, 192.168.0.14 Readiness probe failed: HTTP probe failed with statuscode: 404 # 查看service的endpoints，發(fā)現(xiàn)此時(shí)endpoints為空, 因?yàn)閞eadiness就緒檢測(cè)異常,kubelet認(rèn)為此時(shí)pod并未就緒,因此并未將其加入到endpoints中 [root@k8s-master-10 Liveness]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 28d nginx-service ClusterIP 10.0.0.69 <none> 80/TCP 4m32s [root@k8s-master-10 Liveness]# kubectl describe svc nginx-service Name: nginx-service Namespace: default Labels: app=nginx Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-service","namespace":"default"},"s... Selector: app=nginx-server Type: ClusterIP IP: 10.0.0.69 Port: http 80/TCP TargetPort: 80/TCP Endpoints: Session Affinity: None Events: <none> # 進(jìn)入到pod中手動(dòng)創(chuàng)建網(wǎng)站文件，使readiness健康檢查正常 [root@k8s-master-10 ~]# kubectl exec -it nginx-tcp-liveness-probe-57bb9c7668-qcssq bash root@nginx-tcp-liveness-probe-57bb9c7668-qcssq:/# echo "readiness probe demo" > /usr/share/nginx/html/test.html [root@k8s-master-10 Liveness]# kubectl describe svc nginx-service Name: nginx-service Namespace: default Labels: app=nginx Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-service","namespace":"default"},"s... Selector: app=nginx-server Type: ClusterIP IP: 10.0.0.69 Port: http 80/TCP TargetPort: 80/TCP Endpoints: 172.17.66.2:80 Session Affinity: None Events: <none> [root@k8s-master-10 Liveness]# kubectl describe endpoints nginx-service Name: nginx-service Namespace: default Labels: app=nginx Annotations: endpoints.kubernetes.io/last-change-trigger-time: 2021-09-11T22:25:15+08:00 Subsets: Addresses: 172.17.66.2 NotReadyAddresses: <none> Ports: Name Port Protocol ---- ---- -------- http 80 TCP Events: <none>

5 Pod中容器使用的兩種鉤子介紹

對(duì)容器主要做一些初始化的操作

post start hook: 容器啟動(dòng)初始化啟動(dòng)后的鉤子 pre stop hook: 容器結(jié)束之前執(zhí)行的操作

[root@k8s-master1 lianxi]# cat lifecycle-demo.yaml apiVersion: v1 kind: Pod metadata: name: lifecycle-demo namespace: default spec: containers: - name: demo image: ikubernetes/demoapp:v1.0 imagePullPolicy: IfNotPresent securityContext: capabilities: add: - NET_ADMIN livenessProbe: httpGet: path: '/livez' port: 80 scheme: HTTP initialDelaySeconds: 5 lifecycle: postStart: #啟動(dòng)前執(zhí)行的鉤子 exec: command: ['/bin/sh','-c','iptables -t nat -A PREROUTING -p tcp --dport 8080 -j REDIRECT --to-ports 80'] preStop: #容器結(jié)束前執(zhí)行的鉤子 exec: command: ['/bin/sh','-c','while killall python3; do sleep 1; done'] restartPolicy: Always

在容器啟動(dòng)的時(shí)候自動(dòng)執(zhí)行添加規(guī)則，容器結(jié)束的執(zhí)行會(huì)執(zhí)行上面的殺死python命令

[root@k8s-master-10 Liveness]# kubectl exec lifecycle-demo -- iptables -vnL -t nat Chain PREROUTING (policy ACCEPT 6 packets, 360 bytes) pkts bytes target prot opt in out source destination 0 0 REDIRECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 redir ports 80 Chain INPUT (policy ACCEPT 6 packets, 360 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 5.1 鉤子三種實(shí)行方法

鉤子也是支持三種實(shí)行的方法：對(duì)于postStart和preStart 都是一樣的

[root@k8s-master-10 Liveness]# kubectl explain pods.spec.containers.lifecycle.postStart KIND: Pod VERSION: v1 RESOURCE: postStart <Object> DESCRIPTION: PostStart is called immediately after a container is created. If the handler fails, the container is terminated and restarted according to its restart policy. Other management of the container blocks until the hook completes. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks Handler defines a specific action that should be taken FIELDS: exec<Object> One and only one of the following should be specified. Exec specifies the action to take. httpGet<Object> HTTPGet specifies the http request to perform. tcpSocket<Object> TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported

6 Pod中多容器模式運(yùn)行

Sidecar:為了讓外部的請(qǐng)求更好的接入Pod中容器而設(shè)計(jì)的(主要是代理);

Adapater: 主要是讓容器請(qǐng)求更好的適配外部的請(qǐng)求而設(shè)計(jì);

Ambassador: 主要為了讓Pod中容器請(qǐng)求更好接入外部的環(huán)境設(shè)計(jì);

(訪問數(shù)據(jù)庫，容器把數(shù)據(jù)給base容器，由base容器訪問數(shù)據(jù)庫)

加入定義兩個(gè)初始化容器: Pod生命周期

容器初始化1 ---> 成功完成 ----> 第二個(gè)初始化容器2 ---> 成功完成 ---> 主容器(如果定義了sidecar可能和主容器一起啟動(dòng))

說明:

一般在容器中使用初始化容器做特權(quán)操作，而不是直接定義特權(quán)字段，初始化容器執(zhí)行完之后就會(huì)終止了, 比如添加iptables規(guī)則, 可以通過初始化容器來實(shí)現(xiàn), 添加完iptables之后自動(dòng)終止,然后運(yùn)行主容器，而主容器沒有net-admin權(quán)限;

Example

[root@k8s-master-10 Liveness]# cat sidecar-container-demo.yaml apiVersion: v1 kind: Pod metadata: name: sidecar-container-demo namespace: default spec: containers: - name: proxy image: envoyproxy/envoy-alpine:v1.14.1 command: ['/bin/sh','-c'] args: ['sleep 5 && envoy -c /etc/envoy/envoy.yaml'] lifecycle: postStart: exec: command: ['/bin/sh','-c','wget -O /etc/envoy/envoy.yaml http://ilinux.io/envoy.yaml'] - name: demo image: ikubernetes/demoapp:v1.0 imagePullPolicy: IfNotPresent env: - name: HOST value: "127.0.0.1" - name: PORT value: "8080"

通過envoys代理訪問內(nèi)部demo,demo只是監(jiān)聽在127.0.0.1的8080端口

[root@k8s-master-10 Liveness]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES lifecycle-demo 1/1 Running 0 21m 172.17.66.3 192.168.0.14 <none> <none> liveness-httpget-demo 1/1 Running 0 48m 172.17.66.2 192.168.0.14 <none> <none> sidecar-container-demo 2/2 Running 0 2m1s 172.17.66.4 192.168.0.14 <none> <none> [root@k8s-master-10 Liveness]# curl 172.17.66.4 iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: sidecar-container-demo, ServerIP: 172.17.66.4!

我們看看具體envoy的定義

[root@k8s-master-10 Liveness]# cat /etc/envoy/envoy.yaml admin: access_log_path: /tmp/admin_access.log address: socket_address: { address: 0.0.0.0, port_value: 9901 } static_resources: listeners: - name: listener_0 address: socket_address: { address: 0.0.0.0, port_value: 80 } filter_chains: - filters: - name: envoy.http_connection_manager config: stat_prefix: ingress_http codec_type: AUTO route_config: name: local_route virtual_hosts: - name: local_service domains: ["*"] routes: - match: { prefix: "/" } route: { cluster: local_service } http_filters: - name: envoy.router clusters: - name: local_service connect_timeout: 0.25s type: STATIC lb_policy: ROUND_ROBIN load_assignment: cluster_name: local_service endpoints: - lb_endpoints: - endpoint: address: socket_address: address: 127.0.0.1 port_value: 8080