当前位置：IT运维 >  【Kubernetes】k8s使用minio作为对象存储

【Kubernetes】k8s使用minio作为对象存储

白山茶AI 2024-09-04 09:07:14 阅读 58

在这里插入图片描述

一、部署minio operator

k8s version：v1.20.15 minio version ：v4.4.16

（1）安装kubectl-minio插件

自选minio-operaterd的版本下载包

minio-operater plugin

<code># 上传服务器并解压

unzip kubectl-minio_linux_amd64.zip "kubectl-minio" -d /usr/local/bin/

# 查看是否安装成功。如返回版本信息则安装成功

kubectl minio version

# 初始化

kubectl minio init

# operator默认部署到minio-operator命名空间中，如果需要指定命名空间，可使用kubectl minio init --namespace {YOUR-NAMESPACE}。

# 使用群集。在配置operator的DNS主机名时，将本地作为群集域。指定kubectl minio init --cluster域参数以设置不同的集群域值。

# 验证是否安装成功

[root@k8s-master-4 ~]# kubectl get all -n minio-operator

NAME READY STATUS RESTARTS AGE

pod/console-5f4f574656-5s2wx 1/1 Running 0 5h52m

pod/minio-operator-c78cb4c65-875rt 1/1 Running 0 5h52m

pod/minio-operator-c78cb4c65-x4869 1/1 Running 0 5h52m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

service/console ClusterIP 10.98.66.116 <none> 9090/TCP,9443/TCP 5h52m

service/operator ClusterIP 10.96.133.155 <none> 4222/TCP,4221/TCP 5h52m

NAME READY UP-TO-DATE AVAILABLE AGE

deployment.apps/console 1/1 1 1 5h52m

deployment.apps/minio-operator 2/2 2 2 5h52m

NAME DESIRED CURRENT READY AGE

replicaset.apps/console-5f4f574656 1 1 1 5h52m

replicaset.apps/minio-operator-c78cb4c65 2 2 2 5h52m

# 访问Operator Console，新开一个终端窗口

[root@k8s-master-4 ~]# kubectl minio proxy -n minio-operator

Starting port forward of the Console UI.

To connect open a browser and go to http://localhost:9090

Current JWT to login: eyJhbGciOiJSUzI1NiIsImtpZCI6Ii1MY0QybWl6YklaUzE1MHZQNzcxcFNpc2JrUk1NdGU1X1MzaHUtY0hmMWcifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtaW5pby1vcGVyYXRvciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjb25zb2xlLXNhLXRva2VuLThwcTRtIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImNvbnNvbGUtc2EiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIyMzY4ODJjZS1hMDhhLTQ0MzEtOWY5MC1iN2IxMWJkZWJmMDEiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6bWluaW8tb3BlcmF0b3I6Y29uc29sZS1zYSJ9.VpR7HaUT6AU_u61AgGAe5thnXoWcPP7nSK2S7zorUXVShdKfNIO1pdvTz1XJ9X1b7hlk3Yqn6LfLz1WJXxDBhEz_pHSqJ0s-RCv00hEHGRLPzyUdI-s9zjOwwqU7yITxkyNswodiZ6jRCOiMwqPRZEHKetzMZlckK6xvIO6gpcQd0dDwXagNyHGArb8-zvBLzqWhgx0PoB3B-FHuy7VabGzVDoFdMQiIU20KOjvdQEXPPZRfk-RBMM-hy7HdCM-rk4sTYEhTGz9dUzj1_umvQbupZw7kEcw588MMF9Q2Q66hOCOC-6YbQTPJIHzqwuYNv4SOzkj5hds2eUi_1FHHDA

Forwarding from 0.0.0.0:9090 -> 9090

Handling connection for 9090

# JWT的值为后续登陆界面时使用

img

访问地址：http://ip:9090

二、创建StorageClass

1、sc-minio.yaml

<code>apiVersion: storage.k8s.io/v1

kind: StorageClass

metadata:

name: minio-local-storage # SC-NAME

provisioner: kubernetes.io/no-provisioner

volumeBindingMode: WaitForFirstConsumer

创建

kubectl apply -f sc-minio.yaml

# 查看是否创建成功

[root@k8s-master-4 ~]# kubectl get sc

NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE

minio-local-storage kubernetes.io/no-provisioner Retain WaitForFirstConsumer false 5h38m

2、创建所需的永久卷

序号 路径 容量 (G) 说明 所在节点
1 /data/1 5 租户使用 3个节点各1个
2 /data/log1 5 租户使用 3个节点各1个
3 /data/minio-tenant-1-log-0 5 审计日志 k8s-master-4
4 /data/minio-tenant-1-prometheus 5 监控数据 k8s-master-4

租户：30G

<code># 创建目录(三个节点)

mkdir -p /data/1 /data/log1

# 在K8s-master-4创建审计日志目录、监控目录

mkdir -p /data/minio-tenant-1-log-0 /data/minio-tenant-1-prometheus

三、PV && PVC

为MinIO租户中的每个卷创建一个PV。例如，假设一个Kubernetes集群有3个节点，每个节点有2个本地连接的驱动器，那么总共创建6个本地PV。名称、容量大小、目录等可根据个人环境进行配置:

1、pv-tenant.yaml

apiVersion: v1

kind: PersistentVolume

metadata:

name: minio-k8s-master-4-data1 # PV-NAME

spec:

capacity:

storage: 5Gi # capacity

volumeMode: Filesystem

accessModes:

- ReadWriteOnce # default ReadWriteOnce

persistentVolumeReclaimPolicy: Retain

storageClassName: minio-local-storage # SC-NAME

local:

path: /data/1 # SC local-path

nodeAffinity:

required:

nodeSelectorTerms:

- matchExpressions:

- key: kubernetes.io/hostname

operator: In

values:

- k8s-master-4 # NODE-NAME

---

apiVersion: v1

kind: PersistentVolume

metadata:

name: minio-k8s-master-4-log1 # PV-NAME

spec:

capacity:

storage: 5Gi # capacity

volumeMode: Filesystem

accessModes:

- ReadWriteOnce # default ReadWriteOnce

persistentVolumeReclaimPolicy: Retain

storageClassName: minio-local-storage # SC-NAME

local:

path: /data/log1 # SC local-path

nodeAffinity:

required:

nodeSelectorTerms:

- matchExpressions:

- key: kubernetes.io/hostname

operator: In

values:

- k8s-master-4 # NODE-NAME

---

apiVersion: v1

kind: PersistentVolume

metadata:

name: minio-k8s-node1-5-data1 # PV-NAME

spec:

capacity:

storage: 5Gi # capacity

volumeMode: Filesystem

accessModes:

- ReadWriteOnce # default ReadWriteOnce

persistentVolumeReclaimPolicy: Retain

storageClassName: minio-local-storage # SC-NAME

local:

path: /data/1 # SC local-path

nodeAffinity:

required:

nodeSelectorTerms:

- matchExpressions:

- key: kubernetes.io/hostname

operator: In

values:

- k8s-node1-5 # NODE-NAME

---

apiVersion: v1

kind: PersistentVolume

metadata:

name: minio-k8s-node1-5-log1 # PV-NAME

spec:

capacity:

storage: 5Gi # capacity

volumeMode: Filesystem

accessModes:

- ReadWriteOnce # default ReadWriteOnce

persistentVolumeReclaimPolicy: Retain

storageClassName: minio-local-storage # SC-NAME

local:

path: /data/log1 # SC local-path

nodeAffinity:

required:

nodeSelectorTerms:

- matchExpressions:

- key: kubernetes.io/hostname

operator: In

values:

- k8s-node1-5 # NODE-NAME

---

apiVersion: v1

kind: PersistentVolume

metadata:

name: minio-k8s-node2-6-data1 # PV-NAME

spec:

capacity:

storage: 5Gi # capacity

volumeMode: Filesystem

accessModes:

- ReadWriteOnce # default ReadWriteOnce

persistentVolumeReclaimPolicy: Retain

storageClassName: minio-local-storage # SC-NAME

local:

path: /data/1 # SC local-path

nodeAffinity:

required:

nodeSelectorTerms:

- matchExpressions:

- key: kubernetes.io/hostname

operator: In

values:

- k8s-node2-6 # NODE-NAME

---

apiVersion: v1

kind: PersistentVolume

metadata:

name: minio-k8s-node2-6-log1 # PV-NAME

spec:

capacity:

storage: 5Gi # capacity

volumeMode: Filesystem

accessModes:

- ReadWriteOnce # default ReadWriteOnce

persistentVolumeReclaimPolicy: Retain

storageClassName: minio-local-storage # SC-NAME

local:

path: /data/log1 # SC local-path

nodeAffinity:

required:

nodeSelectorTerms:

- matchExpressions:

- key: kubernetes.io/hostname

operator: In

values:

- k8s-node2-6 # NODE-NAME

创建审计日志PV、PVC配置文件(后续备用)：

2、pv-pvc-minio-tenant-1-log-0.yaml

apiVersion: v1

kind: PersistentVolume

metadata:

name: minio-tenant-1-log # PV-NAME

namespace: minio-tenant-1

spec:

capacity:

storage: 5Gi # capacity

volumeMode: Filesystem

accessModes:

- ReadWriteOnce # default ReadWriteOnce

persistentVolumeReclaimPolicy: Retain

storageClassName: minio-local-storage # SC-NAME

local:

path: /data/minio-tenant-1-log-0 # SC local-path

nodeAffinity:

required:

nodeSelectorTerms:

- matchExpressions:

- key: kubernetes.io/hostname

operator: In

values:

- k8s-master-4 # NODE-NAME

---

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

labels:

v1.min.io/log-pg: minio-tenant-1-log

name: minio-tenant-1-log-minio-tenant-1-log-0

namespace: minio-tenant-1

spec:

accessModes:

- ReadWriteOnce

resources:

requests:

storage: "5368709120"

storageClassName: minio-local-storage

volumeMode: Filesystem

volumeName: minio-tenant-1-log

创建租户监控prometheus使用的PV、PVC配置文件(后续备用)：

3、pv-pvc-minio-tenant-prometheus.yaml

apiVersion: v1

kind: PersistentVolume

metadata:

name: minio-tenant-1-prometheus # PV-NAME

namespace: minio-tenant-1

spec:

capacity:

storage: 5Gi # capacity

volumeMode: Filesystem

accessModes:

- ReadWriteOnce # default ReadWriteOnce

persistentVolumeReclaimPolicy: Retain

storageClassName: minio-local-storage # SC-NAME

local:

path: /data/minio-tenant-1-prometheus # SC local-path

nodeAffinity:

required:

nodeSelectorTerms:

- matchExpressions:

- key: kubernetes.io/hostname

operator: In

values:

- k8s3-master # NODE-NAME

---

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

labels:

v1.min.io/prometheus: minio-tenant-1-prometheus

name: minio-tenant-1-prometheus-minio-tenant-1-prometheus-0

namespace: minio-tenant-1

spec:

accessModes:

- ReadWriteOnce

resources:

requests:

storage: 5Gi

storageClassName: minio-local-storage

volumeMode: Filesystem

volumeName: minio-tenant-1-prometheus

# 为MinIO Tenant创建命名空间

kubectl create namespace minio-tenant-1

# 创建租户pvc

kubectl apply -f pv-tenant.yaml

# 查看创建的pv

kubectl get pv | grep minio

img

四、创建MinIO Tenant租户

使用kubectl minio租户创建命令创建minio租户。该命令始终使用MinIO服务器和MinIO控制台的最新稳定Docker映像。

以下示例创建了一个3节点MinIO部署，6个驱动器的总容量为30Gi。此时会同时创建审计日志、prometheus监控相关pod。

<code># 查看minio支持的api-resource版本

[root@k8s-master-4 ~]# kubectl api-resources

NAME SHORTNAMES APIVERSION NAMESPACED KIND

......

tenants tenant minio.min.io/v2 true Tenant

......

[root@k8s-master-4 ~]# kubectl minio tenant create minio-tenant-1 \

> --servers 3 \

> --volumes 6 \

> --capacity 30Gi \

> --storage-class minio-local-storage \

> --namespace minio-tenant-1

Tenant 'minio-tenant-1' created in 'minio-tenant-1' Namespace

Username: W175STC2EU3QG0KXEEUN

Password: BhbYGhIZamMlYQXjYOZaD8TZzJno9zNwXa0VUmrq

Note: Copy the credentials to a secure location. MinIO will not display these again.

APPLICATION SERVICE NAME NAMESPACE SERVICE TYPE SERVICE PORT

MinIO minio minio-tenant-1 ClusterIP 443

Console minio-tenant-1-console minio-tenant-1 ClusterIP 9443

# 查看租户状态

# 当前状态在初始化过程中，等待Log Search就绪，此过程可能要花费一段时间，租户成功后，状态会显示Initialized：

[root@k8s-master-4 ~]# kubectl get tenants -n minio-tenant-1

NAME STATE AGE

minio-tenant-1 Initialized 19h

# 各pod、服务等对象成功状态

kubectl -n minio-tenant-1 get all

# 查看创建的pvc（如图二）

kubectl -n minio-tenant-1 get pvc

# 查看当前租户

[root@k8s-master-4 ~]# kubectl minio tenant list

Tenant 'minio-tenant-1', Namespace 'minio-tenant-1', Total capacity 30 GiB

Current status: Initialized

MinIO version: minio/minio:RELEASE.2022-04-16T04-26-02Z

# 可以使用kubectl端口转发程序临时公开每个服务。运行以下示例将流量从运行kubectl的主机转发到Kubernetes集群内运行的服务。

# 租户端口转发

[root@k8s-master-4 ~]# kubectl port-forward service/minio 443:443 -n minio-tenant-1

Forwarding from 127.0.0.1:443 -> 9000

Forwarding from [::1]:443 -> 9000

[root@k8s-master-4 ~]# kubectl port-forward service/minio-tenant-1-console 9443:9443 -n minio-tenant-1

Forwarding from 127.0.0.1:9443 -> 9443

img

img

现在我们进入minio的控制台查看

img

img

租户状态已经ok了

五、其他操作

<code># 删除minio-operater

kubectl minio delete

# 删除tenant

kubectl minio tenant delete minio-tenant-1 --namespace minio-tenant-1

# 删除pvc、pv

kubectl delete pvc --all -n minio-tenant-1

kubectl delete pv --all -n minio-tenant-1

# 删除命名空间

kubectl delete ns minio-tenant-1

上一篇： VMware创建新虚拟机教程（保姆级别）

下一篇： Ubuntu20.04的ROS环境安装ORB-SLAM3详解

本文标签

【Kubernetes】k8s使用minio作为对象存储   

声明

本文内容仅代表作者观点，或转载于其他网站，本站不以此文作为商业用途
如有涉及侵权，请联系本站进行删除
转载本站原创文章，请注明来源及作者。

相关文章

阅读排行

热门文章