Skip to content

V
velero-plugin
Commit f2ca8729 by Administrator
Options

添加使用文档

parent 5425cf88
Showing with 286 additions and 30 deletions
go.mod
module github.com/AliyunContainerService/velero-plugin
module github.com/JokerDevops/velero-plugin
go 1.13
require (
github.com/Azure/go-autorest/autorest v0.9.3 // indirect
github.com/Azure/go-autorest/autorest/adal v0.8.1 // indirect
github.com/aliyun/alibaba-cloud-sdk-go v1.60.302
github.com/aliyun/aliyun-oss-go-sdk v2.0.4+incompatible
github.com/hashicorp/go-plugin v1.0.1 // indirect
github.com/heptio/velero v1.2.0
github.com/imdario/mergo v0.3.8 // indirect
github.com/joho/godotenv v1.3.0
github.com/pkg/errors v0.8.1
github.com/sirupsen/logrus v1.4.2
github.com/spf13/cobra v0.0.5 // indirect
github.com/stretchr/testify v1.4.0
github.com/vmware-tanzu/velero v1.2.0
golang.org/x/time v0.0.0-20191024005414-555d28b269f0 // indirect
google.golang.org/grpc v1.26.0 // indirect
k8s.io/api v0.17.0
k8s.io/apimachinery v0.17.0
k8s.io/client-go v11.0.0+incompatible // indirect
k8s.io/utils v0.0.0-20191218082557-f07c713de883 // indirect
)
go 1.15
go.sum
......@@ -236,8 +236,7 @@ gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
k8s.io/api v0.17.0 h1:H9d/lw+VkZKEVIUc8F3wgiQ+FUXTTr21M87jXLU7yqM=
k8s.io/api v0.17.0/go.mod h1:npsyOePkeP0CPwyGfXDHxvypiYMJxBWAMpQxCaJ4ZxI=
k8s.io/apimachinery v0.17.0 h1:xRBnuie9rXcPxUkDizUsGvPf1cnlZCFu210op7J7LJo=
k8s.io/apimachinery v0.17.0/go.mod h1:b9qmWdKlLuU9EBh+06BtLcSf/Mu89rWL33naRxs1uZg=
k8s.io/client-go v11.0.0+incompatible h1:LBbX2+lOwY9flffWlJM7f1Ct8V2SRNiMRDFeiwnJo9o=
......
install/credentials-velero
ALIBABA_CLOUD_ACCESS_KEY_ID=
ALIBABA_CLOUD_ACCESS_KEY_SECRET=
ALIBABA_CLOUD_OSS_ENDPOINT=
ALIBABA_CLOUD_ACCESS_KEY_ID=<ALIBABA_CLOUD_ACCESS_KEY_ID>
ALIBABA_CLOUD_ACCESS_KEY_SECRET=<ALIBABA_CLOUD_ACCESS_KEY_SECRET>
ALIBABA_CLOUD_OSS_ENDPOINT=<ALIBABA_CLOUD_OSS_ENDPOINT>
\ No newline at end of file
use.md 0 → 100644
# Velero
Velero 是 VMWare 开源的 k8s 集群备份、迁移工具。可以帮助我们完成 k8s 的例行备份工作，以便在出现上面问题的时候可以快速进行恢复。同时也提供了集群迁移功能，可以将 k8s 资源迁移到其他 k8s 集群的功能。Velero 将集群资源保存在对象存储中，默认情况下可以使用 AWS、Azure、GCP 的对象存储，同时也给出了插件功能用来拓展其他平台的存储，这里我们用到的就是阿里云的对象存储 OSS，阿里云也提供了 Velero 的插件，用于将备份存储到 OSS 中。下面我就介绍一下如何在阿里云容器服务 ACK 使用 Velero 完成备份和迁移。
>Velero 地址：https://github.com/vmware-tanzu/velero
>ACK 插件地址：https://github.com/AliyunContainerService/velero-plugin
# 下载 Velero 客户端
Velero 由客户端和服务端组成，服务器部署在目标 k8s 集群上，而客户端则是运行在本地的命令行工具。
- 前往 [Velero 的 Release](https://github.com/vmware-tanzu/velero/releases) 页面 下载客户端，直接在 GitHub 上下载即可
- 解压 release 包
- 将 release 包中的二进制文件 velero 移动到 $PATH 中的某个目录下
- 执行 velero -h 测试
# 创建 OSS bucket
创建一个 OSS bucket 用于存储备份文件，这里也可以用已有的 bucket，之后会在 bucket 中创建 backups、metadata、restores三个目录，这里建议在已有的 bucket 中创建一个子目录用于存储备份文件。
创建 OSS 的时候一定要选对区域，要和 ACK 集群在同一个区域，存储类型和读写权限选择标准存储和私有：
![](http://images.joker.irybd.com/images/20201103180542_4Owuf6_Screenshot.jpeg)
# 创建阿里云 RAM 用户
这里需要创建一个阿里云 RAM 的用户，用于操作 OSS 以及 ACK 资源。
- 新建权限策略
![](http://images.joker.irybd.com/images/20201103181002_vWU3Ra_Screenshot.jpeg)
策略内容如下：
```bash
{
"Version": "1",
"Statement": [
{
"Action": [
"ecs:DescribeSnapshots",
"ecs:CreateSnapshot",
"ecs:DeleteSnapshot",
"ecs:DescribeDisks",
"ecs:CreateDisk",
"ecs:Addtags",
"oss:PutObject",
"oss:GetObject",
"oss:DeleteObject",
"oss:GetBucket",
"oss:ListObjects",
"oss:ListBuckets"
],
"Resource": [
"*"
],
"Effect": "Allow"
}
]
}
```
- 新建用户
在新建用户的时候要选择 编程访问，来获取 AccessKeyID 和 AccessKeySecret，这里请创建一个新用于用于备份，不要使用老用户的 AK 和 AS。另外新建用户之后添加权限绑定上面自定义的权限策略。
![创建用户](http://images.joker.irybd.com/images/20201103181201_slfO0t_Screenshot.jpeg)
![](http://images.joker.irybd.com/images/20201103181244_Z3C7On_Screenshot.jpeg)
![](http://images.joker.irybd.com/images/20201103181322_vYCX12_Screenshot.jpeg)
# 部署服务端
- 拉取 Velero 插件 到本地
```git clone git@git.zhiweidata.top:root/velero-plugin.git```
- 配置修改
1. 修改 install/credentials-velero 文件，将新建用户中获得的 AccessKeyID 和 AccessKeySecret 填入，这里的 OSS EndPoint 为之前 OSS 的访问域名（注：这里需要选择外网访问的 EndPoint。）：
![](http://images.joker.irybd.com/images/20201103181416_1kT5Ew_Screenshot.jpeg)
```
ALIBABA_CLOUD_ACCESS_KEY_ID=<ALIBABA_CLOUD_ACCESS_KEY_ID>
ALIBABA_CLOUD_ACCESS_KEY_SECRET=<ALIBABA_CLOUD_ACCESS_KEY_SECRET>
ALIBABA_CLOUD_OSS_ENDPOINT=<ALIBABA_CLOUD_OSS_ENDPOINT>
```
2. 修改 `install/01-velero.yaml`，将 OSS 配置填入：
```yaml
---
apiVersion: velero.io/v1
kind: BackupStorageLocation
metadata:
labels:
component: velero
name: default
namespace: velero
spec:
config: {}
objectStorage:
bucket: <ALIBABA_CLOUD_OSS_BUCKET> # OSS bucket 名称
prefix: <OSS_PREFIX> # bucket 子目录
provider: alibabacloud
---
apiVersion: velero.io/v1
kind: VolumeSnapshotLocation
metadata:
labels:
component: velero
name: default
namespace: velero
spec:
config:
region: <REGION> # 地域，如果是华东2（上海），则为 cn-shanghai
provider: alibabacloud
```
3. kubernetes 部署 velero 服务
```shell
# 新建 namespace
kubectl create namespace velero
# 部署 credentials-velero 的 secret
kubectl create secret generic cloud-credentials --namespace velero --from-file cloud=install/credentials-velero
# 部署 CRD
kubectl apply -f install/00-crds.yaml
# 部署 Velero
kubectl apply -f install/01-velero.yaml
```
4. 测试 Velero 状态
```shell
[root@node47 velero-plugin]# velero version
Client:
Version: v1.2.0
Git commit: 5d008491bbf681658d3e372da1a9d3a21ca4c03c
Server:
Version: v1.2.0
```
5. 服务端清理
```
kubectl delete namespace/velero clusterrolebinding/velero
kubectl delete crds -l component=velero
```
# 备份测试
velero-plugin 项目中已经给出 example 用于测试备份。
- 部署测试服务
```kubectl apply -f examples/base.yaml```
- 对 nginx-example 所在的 namespace 进行备份
```velero backup create nginx-backup --include-namespaces nginx-example --wait```
- 模拟 namespace 被误删
```kubectl delete namespaces nginx-example```
- 使用 Velero 进行恢复
```velero restore create --from-backup nginx-backup --wait```
# 集群迁移
迁移方法同备份，在备份后切换集群，在新集群恢复备份即可。
# 高级用法
- 定时备份
对集群资源进行定时备份，则可在发生意外的情况下，进行恢复（默认情况下，备份保留 30 天）。
```shell
# 每日1点进行备份
velero create schedule <SCHEDULE NAME> --schedule="0 1 * * *"
# 每日1点进行备份，备份保留48小时
velero create schedule <SCHEDULE NAME> --schedule="0 1 * * *" --ttl 48h
# 每6小时进行一次备份
velero create schedule <SCHEDULE NAME> --schedule="@every 6h"
# 每日对 web namespace 进行一次备份
velero create schedule <SCHEDULE NAME> --schedule="@every 24h" --include-namespaces web
```
>定时备份的名称为：`<SCHEDULE NAME>-<TIMESTAMP>`，恢复命令为：`velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>`。
- 备份删除
直接执行命令进行删除
`velero delete backups <BACKUP_NAME>`
- 备份资源查看
备份查看
`velero backup get`
- 查看定时备份
`velero schedule get`
- 查看可恢复备份 ` velero restore get`
- 备份排除项目
可为资源添加指定标签，添加标签的资源在备份的时候被排除。
```
# 添加标签
kubectl label -n <ITEM_NAMESPACE> <RESOURCE>/<NAME> velero.io/exclude-from-backup=true
# 为 default namespace 添加标签
kubectl label -n default namespace/default velero.io/exclude-from-backup=true
```
# 问题汇总
## 时区问题
进行定时备份时，发现备份使用的事 UTC 时间，并不是本地时间，经过排查后发现是 velero 镜像的时区问题，在调整后就会正常定时备份了，这里我重新调整了时区，直接调整镜像就好，修改 install/01-velero.yaml 文件，将镜像替换为 registry-vpc.cn-shanghai.aliyuncs.com/keking/velero:latest。
```yaml
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: velero
namespace: velero
spec:
replicas: 1
selector:
matchLabels:
deploy: velero
template:
metadata:
annotations:
prometheus.io/path: /metrics
prometheus.io/port: "8085"
prometheus.io/scrape: "true"
labels:
component: velero
deploy: velero
spec:
serviceAccountName: velero
containers:
- name: velero
# sync from gcr.io/heptio-images/velero:latest
image: registry-vpc.cn-shanghai.aliyuncs.com/keking/velero:latest # 修复时区后的镜像
imagePullPolicy: IfNotPresent
command:
- /velero
args:
- server
- --default-volume-snapshot-locations=alibabacloud:default
env:
- name: VELERO_SCRATCH_DIR
value: /scratch
- name: ALIBABA_CLOUD_CREDENTIALS_FILE
value: /credentials/cloud
volumeMounts:
- mountPath: /plugins
name: plugins
- mountPath: /scratch
name: scratch
- mountPath: /credentials
name: cloud-credentials
initContainers:
- image: registry.cn-hangzhou.aliyuncs.com/acs/velero-plugin-alibabacloud:v1.2
imagePullPolicy: IfNotPresent
name: velero-plugin-alibabacloud
volumeMounts:
- mountPath: /target
name: plugins
volumes:
- emptyDir: {}
name: plugins
- emptyDir: {}
name: scratch
- name: cloud-credentials
secret:
secretName: cloud-credentials
```
\ No newline at end of file
vendor/modules.txt
# cloud.google.com/go v0.26.0
cloud.google.com/go/compute/metadata
# github.com/Azure/go-autorest/autorest v0.9.3
## explicit
github.com/Azure/go-autorest/autorest
github.com/Azure/go-autorest/autorest/azure
# github.com/Azure/go-autorest/autorest/adal v0.8.1
## explicit
github.com/Azure/go-autorest/autorest/adal
# github.com/Azure/go-autorest/autorest/date v0.2.0
github.com/Azure/go-autorest/autorest/date
......@@ -12,6 +14,7 @@ github.com/Azure/go-autorest/logger
# github.com/Azure/go-autorest/tracing v0.5.0
github.com/Azure/go-autorest/tracing
# github.com/aliyun/alibaba-cloud-sdk-go v1.60.302
## explicit
github.com/aliyun/alibaba-cloud-sdk-go/sdk
github.com/aliyun/alibaba-cloud-sdk-go/sdk/auth
github.com/aliyun/alibaba-cloud-sdk-go/sdk/auth/credentials
......@@ -24,6 +27,7 @@ github.com/aliyun/alibaba-cloud-sdk-go/sdk/responses
github.com/aliyun/alibaba-cloud-sdk-go/sdk/utils
github.com/aliyun/alibaba-cloud-sdk-go/services/ecs
# github.com/aliyun/aliyun-oss-go-sdk v2.0.4+incompatible
## explicit
github.com/aliyun/aliyun-oss-go-sdk/oss
# github.com/davecgh/go-spew v1.1.1
github.com/davecgh/go-spew/spew
......@@ -55,19 +59,23 @@ github.com/googleapis/gnostic/extensions
# github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd
github.com/hashicorp/go-hclog
# github.com/hashicorp/go-plugin v1.0.1
## explicit
github.com/hashicorp/go-plugin
github.com/hashicorp/go-plugin/internal/plugin
# github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb
github.com/hashicorp/yamux
# github.com/heptio/velero v1.2.0
## explicit
github.com/heptio/velero/pkg/test
# github.com/imdario/mergo v0.3.8
## explicit
github.com/imdario/mergo
# github.com/inconshreveable/mousetrap v1.0.0
github.com/inconshreveable/mousetrap
# github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
github.com/jmespath/go-jmespath
# github.com/joho/godotenv v1.3.0
## explicit
github.com/joho/godotenv
# github.com/json-iterator/go v1.1.8
github.com/json-iterator/go
......@@ -82,25 +90,30 @@ github.com/modern-go/reflect2
# github.com/oklog/run v1.0.0
github.com/oklog/run
# github.com/pkg/errors v0.8.1
## explicit
github.com/pkg/errors
# github.com/pmezard/go-difflib v1.0.0
github.com/pmezard/go-difflib/difflib
# github.com/sirupsen/logrus v1.4.2
## explicit
github.com/sirupsen/logrus
# github.com/spf13/afero v1.2.2
github.com/spf13/afero
github.com/spf13/afero/mem
# github.com/spf13/cobra v0.0.5
## explicit
github.com/spf13/cobra
# github.com/spf13/pflag v1.0.5
github.com/spf13/pflag
# github.com/stretchr/objx v0.1.1
github.com/stretchr/objx
# github.com/stretchr/testify v1.4.0
## explicit
github.com/stretchr/testify/assert
github.com/stretchr/testify/mock
github.com/stretchr/testify/require
# github.com/vmware-tanzu/velero v1.2.0
## explicit
github.com/vmware-tanzu/velero/pkg/apis/velero/v1
github.com/vmware-tanzu/velero/pkg/buildinfo
github.com/vmware-tanzu/velero/pkg/client
......@@ -141,6 +154,7 @@ golang.org/x/text/transform
golang.org/x/text/unicode/bidi
golang.org/x/text/unicode/norm
# golang.org/x/time v0.0.0-20191024005414-555d28b269f0
## explicit
golang.org/x/time/rate
# google.golang.org/appengine v1.4.0
google.golang.org/appengine
......@@ -156,6 +170,7 @@ google.golang.org/appengine/urlfetch
# google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
google.golang.org/genproto/googleapis/rpc/status
# google.golang.org/grpc v1.26.0
## explicit
google.golang.org/grpc
google.golang.org/grpc/attributes
google.golang.org/grpc/backoff
......@@ -201,6 +216,7 @@ gopkg.in/ini.v1
# gopkg.in/yaml.v2 v2.2.4
gopkg.in/yaml.v2
# k8s.io/api v0.17.0
## explicit
k8s.io/api/admissionregistration/v1beta1
k8s.io/api/apps/v1
k8s.io/api/apps/v1beta1
......@@ -238,6 +254,7 @@ k8s.io/api/storage/v1
k8s.io/api/storage/v1alpha1
k8s.io/api/storage/v1beta1
# k8s.io/apimachinery v0.17.0
## explicit
k8s.io/apimachinery/pkg/api/equality
k8s.io/apimachinery/pkg/api/errors
k8s.io/apimachinery/pkg/api/meta
......@@ -278,6 +295,7 @@ k8s.io/apimachinery/pkg/watch
k8s.io/apimachinery/third_party/forked/golang/json
k8s.io/apimachinery/third_party/forked/golang/reflect
# k8s.io/client-go v11.0.0+incompatible
## explicit
k8s.io/client-go/discovery
k8s.io/client-go/discovery/fake
k8s.io/client-go/dynamic
......@@ -388,6 +406,7 @@ k8s.io/klog
# k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
k8s.io/kube-openapi/pkg/util/proto
# k8s.io/utils v0.0.0-20191218082557-f07c713de883
## explicit
k8s.io/utils/integer
# sigs.k8s.io/yaml v1.1.0
sigs.k8s.io/yaml
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment