Ingress用法
macbook m1 DockerDesktop 似乎没法实现，因为DockerDesktop上的node IP是一个虚拟ip，虽然能部署成功但是无法访问。本次实验使用的是macbook air m1 虚拟机，但是需要把ingress-nginx镜像换成arm64架构的dyrnq/ingress-nginx-controller:v1.5.0本次环境：k8s1.25.13ingress 1.5.2
·
一、Ingress介绍
macbook m1 DockerDesktop 似乎没法实现，因为DockerDesktop上的node IP是一个虚拟ip，虽然能部署成功但是无法访问。
本次实验使用的是macbook air m1 虚拟机，但是需要把ingress-nginx镜像换成arm64架构的dyrnq/ingress-nginx-controller:v1.5.0
本次环境：k8s1.25.13 ingress 1.5.2
在k8s1.25.13+ingress1.5.2环境中pod查看日志是可以收到来自客户端的真实ip的，如果前端做了负载均衡，那么pod查看日志只会收到负载均衡的ip地址。
k8s1.28+ingress1.9.4环境中开启X-Forwarded-For后，pod依然没有收到来自客户端的真实ip(目前这里还没找到解决办法)
Service对集群之外暴露服务的主要方式有两种：NotePort和LoadBalancer，但是这两种方式，都有一定的缺点：
• NodePort方式的缺点是会占用很多集群机器的端口，那么当集群服务变多的时候，这个缺点就愈发明显
• LB方式的缺点是每个service需要一个LB，浪费、麻烦，并且需要kubernetes之外设备的支持
基于这种现状，kubernetes提供了Ingress资源对象，Ingress只需要一个NodePort或者一个LB就可以满足暴露多个Service的需求
ingress有三种部署方式
Deployment+LoadBalancer模式的service
用Deployment部署igress-controller，创建一个type为LoadBalancer的service关联这组pod。大部分公有云，都会为LoadBalancer的service自动创建一个负载均衡器，通常还绑定了公网地址。只要把域名解析指向改地址，就实现了集群服务的对外暴露
Deployment+NodePort模式的service
同样用deployment模式部署ingress-controller，然后创建对应的ingress svc，type为NodePort。这样，ingress就会暴露在集群节点ip的特定端口上。由于nodeport暴露的端口是随机端口，一般会在前面再搭建一套负载均衡器来转发请求。改方式一般用于宿主机是相对固定的环境ip地址不变的场景。
NodePort方式暴露ingress虽然简单方便，但是NodePort多了一层NAT,在请求量级很大时可能对性能会有一定的影响
DaemonSet+HostNetwork(+nodeSelector)
用DaemonSet 结合nodeselector来部署ingress-controller到特定的Node上，然后使用HostNetwork直接把该pod与宿主机node的网络打通，直接使用宿主机的80/443端口就能访问服务。这时，ingress-controller所在的node机器就很类似传统架构的边缘节点，比如机房的入口nginx服务器。该方式整个请求链路最简单，性能相对nodeport模式更好。缺点是由于直接利用宿主机节点的网络和端口，一个node只能部署一个ingress-controller pod。比较适合大并发的生产环境使用。
Daemonset和Deployment部署ingress区别：
Deployment:这种方式部署的ingress只会在某一台node节点创建ingress-controll命名的pod，之后创建的ingress-rule规则也是在同样的node节点上生成，当用户访问域名时绑定的hosts只能是这个node节点的ip，当这个节点挂掉后ingress-controll和ingress-rule会飘到其他node节点上，用户还需要修改hosts IP地址才能访问。
DaemonSet:这种方式部署的ingress会在所有节点启动一个ingress-controll命名的pod，之后创建的ingress-rule规则也是在每个node节点上生成，当挂掉其中一个节点后，ingress-rule规则也会删除一个节点，用户可以绑定其他节点hosts访问，但是这样需要去修改hosts,如果不想修改hosts可以直接keepalived+nginx+ingress实现高可用访问ingress.
# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-httpd-dp-svc-rules <none> www.qiang.com 192.168.1.111,192.168.1.112,192.168.1.113 80 22h
# 挂掉192.168.1.111节点
# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-httpd-dp-svc-rules <none> www.qiang.com 192.168.1.112,192.168.1.113 80 22h
ingress其实就是k8s中的一个组件，比如api-server，ingress主要负责建立规则,比如哪个域名对应哪个svc
svc在这里只用于对应一组pod，不对外提供服务。
ingress controller 是实现Ingress的程序，负责监听ingress规则并转化成配置文件，这个ingress controller 可以是nginx 或haproxy
Ingress（以Nginx为例）的工作原理如下：
- 用户编写Ingress规则，说明哪个域名对应kubernetes集群中的哪个Service
- Ingress控制器动态感知Ingress服务规则的变化，然后生成一段对应的Nginx反向代理配置
- Ingress控制器会将生成的Nginx配置写入到一个运行着的Nginx服务中(这个运行的nginx就是ingress控制器)，并动态更新
- 到此为止，其实真正在工作的就是一个Nginx了，内部配置了用户定义的请求转发规则
https://blog.csdn.net/WuDan_1112/article/details/126234642
https://blog.csdn.net/guijianchouxyz/article/details/128160289
https://betheme.net/qianduan/68349.html?action=onClick
二、部署
1、部署过程
(1) 首先启动一组pod当作后端web服务器并能够被访问
svc: nginx-deployment 这个svc的类型可以是Nodeport 或clusterIP 或无头服务
pod: nginx-deployment-5fb8c4df95-s7h7l
nginx-deployment-5fb8c4df95-6558x
(2) 部署Ingress Controller(ingress控制器)
ingress controller其实就是一组pod（这个pod可以是nginx/haproxy/其他反向代理服务）,一般设置为Daemonset保证每个node节点都有一个pod(ingress控制器)，类型可以是NodePort或LoadBalancer，会直接在节点上暴露80 443端口,一旦监听到ingress规则的变化就会更新ingressController所在pod的nginx.conf内容
(3) 创建Ingress规则并通过浏览器访问域名
指定哪些域名去寻找哪些svc资源。
2、部署Ingress controller
去github上找
(1) 准备被访问的web服务，我这里只准备了一个nginx服务，下面ingress规则还有一个tomcat服务可以自行准备。
# cat nginx.yaml.bak
apiVersion: apps/v1 # pod控制器版本号通过kubectl api-resources查看与pod控制器对应
kind: Deployment # 选择pod控制器
metadata: # 元数据属性必填
name: nginx-dp-wen # pod名称
namespace: devops # 指定名称空间(可选)
spec: # 元数据属性必填
selector: # 标签选择器必填
matchLabels: # 标签匹配属性 必填
prod: mynginx-wen # pod标签名称(之后与svc标签绑定)
replicas: 4 # pod副本数
template: # 模版 必填
metadata: # 模版元数据
labels: # 定义模版标签
prod: mynginx-wen # 模版标签名(与pod标签和svc标签做绑定)
spec: # 元数据属性必填
containers: # 定义容器属性
- name: my-nginx-wen # 第一个容器名称通过kubectl describe可以看到这个名字
ports:
- containerPort: 80 # 容器端口
image: nginx:latest # 来源镜像
imagePullPolicy: IfNotPresent # 优先拉取本地镜像，Never只拉本地 Always只拉线上
securityContext: # 安全模式(可选)
privileged: true # 特权模式
---
apiVersion: v1 # 定义svc的版本
kind: Service # pod控制器类型
metadata: # 元数据
name: nginx-dp-svc-wen # svc名称
namespace: wenqiang
spec:
# type: NodePort # svc类型,作为ingress后端可以不指定类型
ports:
- port: 81 # svc的cluster端口，这个是虚拟端口
targetPort: 80 # 容器内部端口(不写也能自动识别)
# nodePort: 30382 # 对外暴露端口，作为ingress后端服务不暴露端口
selector: # 标签选择器
prod: mynginx-wen
# kubectl get pod -n wenqiang
NAME READY STATUS RESTARTS AGE
nginx-dp-wen-68f944f44f-4zb7q 1/1 Running 5 (3h44m ago) 5d16h
# kubectl get svc -n wenqiang
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-dp-svc-wen ClusterIP 10.68.56.219 <none> 81/TCP 7m53s
(2) 下载ingress controller yaml文件或从giethub上找,如果无法下载就浏览器打开然后复制粘贴(需要翻墙)
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.5.1/deploy/static/provider/baremetal/deploy.yaml
(3) 修改ingress-controlller内容
# vim deploy.yaml
apiVersion: apps/v1
kind: DaemonSet #原先是Deployment，需要改成DaemonSet确保每个node节点都有一个ingress控制器,如果是1.9.4版本需要删除strategy字段
metadata:
labels:
xxxxxxx
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
xxxxxx
template:
metadata:
labels:
xxxxxx
spec:
hostNetwork: true # 增加，把该pod与宿主机node的网络打通，直接使用宿主机的80/433端口就能访问服务
containers:
- args:
- /nginx-ingress-controller
- --election-id=ingress-nginx-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx # 注意后续创建ingress规则时需要和此处设置一致
# 镜像如果拉不下来需要替换，替换后先执行delete再执行apply
image: easyyun/ingress-nginx-controller:v1.5.1
image: dyrnq/kube-webhook-certgen:v20220916-gd32f8c343
(4) 启动ingress controller,会在node节点暴露80 443等端口
# kubectl apply -f deploy.yaml
# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-tf869 0/1 Completed 0 39m
ingress-nginx-admission-patch-sdslg 0/1 Completed 0 39m
ingress-nginx-controller-jkmq8 1/1 Running 0 6m12s
# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.68.141.34 <none> 80:31452/TCP,443:32075/TCP 39m
ingress-nginx-controller-admission ClusterIP 10.68.112.240 <none> 443/TCP 39m
(5) 创建ingress规则,一般这个规则和pod svc写一个yaml文件中。
# cat ingress-rules.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress # 类型是ingress控制器
metadata:
name: ingress-nginx-rules # ingress控制器名称
namespace: wenqiang # 针对哪个名称空间svc生效，同一名称空间下可以有多个虚拟机主机规则
annotations:
kubernetes.io/ingress.class: "nginx" # 注意这里的nginx必须和ingressController中的ingress.class一样
spec:
rules: # 规则
- host: www.wenqiang.com # 虚拟主机域名
http: # 协议
paths: # 指定路径，可以有多个
- path: / # 根路径 表示访问域名的根站点目录
pathType: Prefix
backend: # 后端服务
service:
name: nginx-dp-svc-wen # 这是svc名字需要与之前创建的svc名称对应
port:
number: 81 # 注意这里要有空格，这个是svc虚拟端口，表示访问www.wenqiang.com:80时就会跳转到nginx-dp-svc-wen:81上，之前创建的svc的虚拟端口就是81
- host: www.qiangqiang.com # 第二个虚拟主机
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: httpd-dp-svc-wen
port:
number: 81
# kubectl apply -f ingress-rules.yaml
# 查看ingress规则
# kubectl get ingress -n wenqiang
NAME CLASS HOSTS ADDRESS（这是nodeIP）PORTS AGE
ingress-nginx-rules <none> www.wenqiang.com 192.168.1.15 80 80m
# kubectl describe ingress ingress-nginx-rules -n wenqiang
Name: ingress-nginx-rules
Labels: <none>
Namespace: wenqiang
Address: 192.168.1.15 # 这是node节点ip
Ingress Class: <none>
Default backend: <default>
Rules:
Host(虚拟主机) Path(站点根目录) Backends(后端svc:虚拟端口(集群内容器))
---- ---- --------
www.wenqiang.com
/ nginx-dp-svc-wen:81 (172.20.29.213:80,172.20.29.219:80,172.20.29.224:80
Annotations: kubernetes.io/ingress.class: nginx（描述）
Events: <none>
# 查看ingressController的变化
# kubectl exec -it ingress-nginx-controller-rpwtc -n ingress-nginx -- cat /etc/nginx/nginx.conf
## start server www.wenqiang.com
server {
server_name www.wenqiang.com ;
listen 80 ;
listen [::]:80 ;
listen 443 ssl http2 ;
listen [::]:443 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "wenqiang";
set $ingress_name "ingress-nginx-rules";
set $service_name "nginx-dp-svc-wen";
set $service_port "81";
set $location_path "/";
set $global_rate_limit_exceeding n;
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
preserve_trailing_slash = false,
use_port_in_redirects = false,
global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
})
balancer.rewrite()
plugins.run()
}
(6) 访问
3、创建基于https的ingress
• 经测试发现k8s1.28中的ingress-1.9.4默认自带证书,如果想使用自己的证书仍然需要使用上面的方法。
• kubectl get ingress ingress-name |grep -i -A 1 tls 查看ingress规则是否使用证书
(1) 由于没有购买ssl证书，所以我们创建自建ssl证书
# 创建www.tomcat.com证书私钥
openssl genrsa -out server.key 2048
# 创建利用证书私钥去创建证书公钥，最终谁生成server.key 和 server.crt(可改名.pem)两个证书
openssl req -new -x509 -key server.key -out server.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=www.tomcat.com
------------------
C=国家 ST=省 L=市 O=部门 CN=域名
(2) 将证书转换为k8s能用的格式（将证书写到secret控制器中）
kubectl create secret tls tomcat-ingress-secret --cert=server.crt --key=server.key
# kubectl get secret
NAME TYPE DATA AGE
tomcat-ingress-secret kubernetes.io/tls 2 43m
--------------
创建一个secret加密控制器类型为tls,该控制器名称叫tomcat-ingress-secret
(3) 准备tomcat pod资源用于访问测试
# cat /soft/tomcat.yaml
apiVersion: apps/v1 # pod控制器版本号通过kubectl api-resources查看与pod控制器对应
kind: Deployment # 选择pod控制器
metadata: # 元数据属性必填
name: tomcat-dp-wen # pod名称
namespace: devops # 指定名称空间(可选)
spec: # 元数据属性必填
selector: # 标签选择器必填
matchLabels: # 标签匹配属性 必填
prod: mytomcat-wen # pod标签名称(之后与svc标签绑定)
replicas: 2 # pod副本数
template: # 模版 必填
metadata: # 模版元数据
labels: # 定义模版标签
prod: mytomcat-wen # 模版标签名(与pod标签和svc标签做绑定)
spec: # 元数据属性必填
containers: # 定义容器属性
- name: my-tomcat-wen # 第一个容器名称通过kubectl describe可以看到这个名字
ports:
- name: tomcat-8080
containerPort: 8080 # 容器端口
- name: tomcat-8009
containerPort: 8089 # 容器端口
image: tomcat:latest # 来源镜像
imagePullPolicy: IfNotPresent # 优先拉取本地镜像，Never只拉本地 Always只拉线上
securityContext: # 安全模式(可选)
privileged: true # 特权模式
---
apiVersion: v1 # 定义svc的版本
kind: Service # pod控制器类型
metadata: # 元数据
name: tomcat-dp-svc-wen # svc名称
namespace: devops
spec:
ports:
- name: tomcat-8080 # 第一个端口名称
protocol: TCP # 协议
port: 8080 # svc虚拟端口
targetPort: 8080 # 容器内端口
- name: tomcat-8089 # 第二个端口名称
protocol: TCP
port: 8009
targetPort: 8009
selector: # 标签选择器
prod: mytomcat-wen
# kubectl get pod -n devops |grep tomcat
tomcat-dp-wen-8495f84785-9rn6j 1/1 Running 0 27m
tomcat-dp-wen-8495f84785-q7khk 1/1 Running 0 27m
# kubectl get svc -n devops |grep tomcat
tomcat-dp-svc-wen ClusterIP 10.68.97.136 <none> 8080/TCP,8009/TCP 27m
(4) 创建ingress规则，可以和之前的tomcat.yaml写在一个文件中，也可以单独写一个文件
# cat ingress-tomcat-rules.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-tomcat-rules # 创建的ingress规则的名称
namespace: devops
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false" # ingress1.9.4增加这行后即使配置了TLS证书依然可以访问80端口，否则只能访问443端口。
spec:
tls: # ssl证书标签
- hosts:
- www.tomcat.com # 为哪个域名配置ssl证书，域名和证书要能对上
secretName: tomcat-ingress-secret # 使用哪个ssl证书，可以通过kubectl get secret查看
rules:
- host: www.tomcat.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: tomcat-dp-svc-wen # 访问https://www.qiangqiang.com时就跳转到这个svc的8080虚拟端口上
port:
number: 8080
# 查看ingress规则
# kubectl get ingress -n devops
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-nginx-rules <none> www.wenqiang.com,www.qiangqiang.com 192.168.1.15 80 84m
ingress-tomcat-rules <none> www.tomcat.com 192.168.1.15 80, 443 20m
#查看ingress规则使用的证书
# kubectl describe ingress ingress-tomcat-rules |grep -i -A 1 tls
TLS:
tomcat-ingress-secret terminates www.tomcat.com # 证书是tomcat-ingress-secret证书签发的域名是www.tomcat.com
# 查看ingress证书包含哪些域名
# 1. 获取 Secret 对象的详细信息，提取证书字段的 Base64 编码值
secret_info=$(kubectl get secret <secret_name> -o json)
cert_value=$(echo "$secret_info" | jq -r '.data."tls.crt"')
# 2. Base64 解码证书值到文件
echo "$cert_value" | base64 -d > certificate.crt
# 3. 使用 OpenSSL 查看证书的域名信息
openssl x509 -in certificate.crt -noout -text |grep 'Subject: '|awk -F= '{print $2}'
# 4. 清理临时文件
rm certificate.crt
(5) 访问测试
(6) 开启ingress的X-Forwarded-For功能让真实ip传到pod中
找到kind: ConfigMap 增加data三条内容
apiVersion: v1
data:
allow-snippet-annotations: "true"
use-forwarded-headers: "true" # 增加
compute-full-forwarded-for: "true" # 增加
forwarded-for-header: "X-Forwarded-For" #增加
kind: ConfigMap
# 查看pod日志，客户端192.168.1.9通过代理服务器192.168.1.114访问到了node节点192.168.1.111上的pod资源
192.168.1.111 - - [21/Dec/2023:11:22:45 +0000] "GET / HTTP/1.1" 200 2 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" "192.168.1.9, 192.168.1.114"
192.168.111 是node节点ip
192.168.1.9 是真实客户端ip，客户端真实ip在代理服务器ip最前面
192.168.1.114 是负载均衡代理服务器IP，如果有多个代理会用，分割
(7) deplay.yaml全部内容，下面的注释内容是需要增加的
# cat deploy.yaml
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resourceNames:
- ingress-nginx-leader
resources:
- configmaps
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- coordination.k8s.io
resourceNames:
- ingress-nginx-leader
resources:
- leases
verbs:
- get
- update
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- list
- watch
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- list
- watch
- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
# 创建configmap保持一些非加密文件
apiVersion: v1
data:
allow-snippet-annotations: "true"
use-forwarded-headers: "true" # 增加这三项可以把客户端真实ip传给pod，相当于开启nginx的forword转发
compute-full-forwarded-for: "true" # 增加
forwarded-for-header: "X-Forwarded-For #增加
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-controller
namespace: ingress-nginx
---
# 创建ingress-controller的svc并暴露80 443在宿主机
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: NodePort # 这里是nodeport类型
---
# 用于验证webhook删除此项会导致ingress无法创建/更新
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
# 创建ingress-controller控制器
apiVersion: apps/v1
kind: DaemonSet #原先是Deployment，需要改成DaemonSet确保每个node节点都有一个ingress控制器
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
hostNetwork: true # 增加，把该pod与宿主机node的网络打通，直接使用宿主机的80/433端口就能访问服务
containers:
- args:
- /nginx-ingress-controller
- --election-id=ingress-nginx-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx # 后续创建ingress规则时需要和此处设置一致
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: easyyun/ingress-nginx-controller:v1.5.1
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
- containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 101
volumeMounts:
- mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
- name: webhook-cert
secret:
secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission-create
spec:
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: dyrnq/kube-webhook-certgen:v20220916-gd32f8c343
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: dyrnq/kube-webhook-certgen:v20220916-gd32f8c343
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
欢迎加入DeepSeek 技术社区。在这里，你可以找到志同道合的朋友，共同探索AI技术的奥秘。
更多推荐
16
0- 0
已为社区贡献3条内容
所有评论(0)