Migrating from FlexVolume to Container Storage Interface for a StatefulSet in Oracle Cloud Infrastructure
3 min readMay 22, 2021
Existing persistent volumes(PV) provisioned using FlexVolume provisioner: oracle.com/oci can be migrated to be managed by Container Storage Interface with oci-bv StorageClass installed in Oracle Container Engine for Kubernetes (OKE) clusters.
Steps which can be followed for migrating from FlexVolume to CSI for a StatefulSet:
- Change reclaim policy of existing FlexVolume PV to `Retain`
- Create a new PV and bind it to the volume in Step#1 by setting `volumeHandle` to the OCID of the volume from Step#1
- Delete the StatefulSet
- Delete FlexVolume PVC and PV
- Create a new PVC(with storageClassName: oci-bv in PVC) and create a new StatefulSet to use this new PVC
StatefulSet: flex-stateful.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: "nginx"
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx-slim:0.8
ports:
- containerPort: 80
name: web
volumeMounts:
- name: persistent-storage
mountPath: /usr/share/nginx/html
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: flex-claimapiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: flex-claim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
storageClassName: oci
1. Change reclaim policy of existing FlexVolume PV to `Retain`.
### Edit the Reclaim Policy from Delete to Retain in the PV's ## definition. This will ensure the block volume is not deleted.+ kubectl edit pv ocid1.volume.oc1.iad.aaaaaaaaaapersistentvolume/ocid1.volume.oc1.iad.aaaaaaaaaa edited+ kubectl describe pv ocid1.volume.oc1.iad.aaaaaaaaaaName: ocid1.volume.oc1.iad.aaaaaaaaaaLabels: failure-domain.beta.kubernetes.io/region=iadfailure-domain.beta.kubernetes.io/zone=US-ASHBURN-AD-1Annotations: ociAvailabilityDomain: US-ASHBURN-AD-1ociCompartment: ocid1.compartment.oc1..aaaaaaaaaaociProvisionerIdentity: ociProvisionerIdentityociVolumeID: ocid1.volume.oc1.iad.aaaaaaaaaapv.kubernetes.io/provisioned-by: oracle.com/ociFinalizers: [kubernetes.io/pv-protection]StorageClass: ociStatus: BoundClaim: default/flex-claimReclaim Policy: RetainAccess Modes: RWOVolumeMode: FilesystemCapacity: 50GiNode Affinity: <none>Message:Source:Type: FlexVolume (a generic volume resource that is provisioned/attached using an exec based plugin)Driver: oracle/ociFSType: ext4SecretRef: nilReadOnly: falseOptions: map[]Events: <none>
2. Create a new PV and bind it to the above volume by setting `volumeHandle` to OCID of the volume:
File: csi-static-pv.yaml
kind: PersistentVolume
apiVersion: v1
metadata:
name: csi-vol-0
annotations:
# fake it by indicating this is provisioned dynamically, so the system
# works properly
pv.kubernetes.io/provisioned-by: blockvolume.csi.oraclecloud.com
spec:
claimRef:
namespace: default
name: persistent-storage-web-0
storageClassName: oci-bv
# by default, the volume will be not deleted if you delete the PVC, change to
# "Delete" if you wish the volume to be deleted automatically with the PVC
persistentVolumeReclaimPolicy: Retain
capacity:
storage: 50Gi
accessModes:
- ReadWriteOnce
csi:
driver: blockvolume.csi.oraclecloud.com
fsType: ext4
volumeHandle: ocid1.volume.oc1.iad.aaaaaaaaaaCreate the PV:
+ kubectl apply -f csi-static-pv.yamlpersistentvolume/csi-vol created
— — —
In your use case, if you have multiple PVCs and you want to reserve this PV for a specific PVC, please use `claimRefs` under `spec` to specify the PVC name and namespace. For example:
claimRef:
name: <pvc_name>
namespace: <namespace>If the nodes are spread over ADs and you want to ensure the pod gets scheduled in the AD in which the PV is, please use `nodeAffinity`[ https://kubernetes.io/docs/concepts/storage/persistent-volumes/#node-affinity ] in PV under `spec`:
spec:
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: failure-domain.beta.kubernetes.io/zone
operator: In
values:
- <AD_OF_THE_EXISTING_BLOCK_VOLUME>3. Delete StatefulSet:
kubectl delete -f flex-stateful.yaml4. Delete FlexVolume PVC and PV:
+ kubectl delete flex-claimpersistentvolumeclaim "flex-claim" deleted+ kubectl delete pv ocid1.volume.oc1.iad.aaaaaaaaaapersistentvolume "ocid1.volume.oc1.iad.aaaaaaaaaa" deleted
5. Create a PVC using CSI StorageClass: `oci-bv` and create a new StatefulSet to use this new PVC :
To make sure that the new PVC gets bound to the CSI PersistentVolume created in previous step, specify the name of the CSI PersistentVolume in your PVC using the volumeName field.
File: csi-static-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: csi-claim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
storageClassName: oci-bvCreate the PVC:
+ kubectl apply -f csi-static-pvc.yamlpersistentvolumeclaim/csi-claim created
csi-statefulset.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: "nginx"
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx-slim:0.8
ports:
- containerPort: 80
name: web
volumeMounts:
- name: persistent-storage
mountPath: /usr/share/nginx/html
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: csi-claimCreate a StatefulSet:
+ kubectl apply -f csi-statefulset.yamlservice/nginx unchangedstatefulset.apps/web configured+ kubectl get podNAME READY STATUS RESTARTS AGEweb-0 1/1 Running 0 40sweb-1 1/1 Running 0 3s
