I was trying to setup a SVM DR relationship of a recently migrated 7-mode vfiler and it would never initialise. Needed some help troubleshooting the issue but the key command to use on the target cluster to get the failure reason (nothing seems to be in log files now):
snapmirror show -destination-vserver vnas-homes -fields last-transfer-error,unhealthy-reason -expand
This resulted in the following reason:
vnas-homes: vnas-homes-sm: Failed to apply the source Vserver configuration. Reason: Unrecoverable error during configuration replication, requires manual intervention. Error: Schedule CronJob_vnas-homes_vol_homes_0 not found. Execute "snapmirror show -destination-vserver vnas-homes-sm -fields last-transfer-error,unhealthy-reason -expand" to check if the constituent volumes have enountered errors. Transfer failed vnas-homes:vol_homes vnas-homes-sm:vol_homes -
So it turns out that if the source SVM uses:
-
cluster based schedules for snapshots
-
cluster based policies for snapshots
-
SVM based snapshot policies that refer to cluster based schedules
Then these schedules and policies MUST either:
-
exist on the target cluster
-
be removed from the source cluster
- not referred to by the source SVM
If not the ALL SVM DR relationships between the two clusters will fail.
In this can I wanted to remove them So I need to find the Job schedules and then work out which policies referred to them and remove said policies.
Firstly in my setup I create cluster wide schedules and snapshot policies and associate SVM volume snapshot schedules to these as I don’t have lots of various unique backup requirements. When using the 7MMT to move a vfiler into a C-mode cluster it creates cluster based Job schedules and SVM based snapshot policies that refer to them.
The root cause of the failure was the schedule CronJob_vnas-homes_vol_homes_0 so if I try to delete that as I don’t need it on the target cluster I get:
netapp02::> job schedule delete -name CronJob_vnas-homes_vol_homes_0 Error: command failed: Cannot delete job schedule. Schedule referenced by 1 volume Snapshot policy entries, 0 SnapMirror entries, 0 antivirus on-demand entries, 0 SIS policy entries, and 0 configuration backup settings. Remove references to schedule first.
So a single volume snapshot policy refers to that schedule. As it’s blocking the vnas-homes SVM update I know it’s there so I check the snapshot policies of the source SVM and the source cluster, just to be sure. I can then see that there is a single SVM based snapshot policy SnapShot_vnas-homes_vol_homes_0 which is using three schedules, all automatically created by the migration tool:
netapp02::> snapshot policy show -vserver vnas-homes Vserver: vnas-homes Number of Is Policy Name Schedules Enabled Comment ------------------------ --------- ------- ---------------------------------- SnapShot_vnas-homes_vol_homes_0 3 true - Schedule Count Prefix SnapMirror Label ---------------------- ----- ---------------------- ------------------- CronJob_vnas-homes_vol_homes_0 26 CronJob_vnas-homes_vol_homes_0 - CronJob_vnas-homes_vol_homes_1 7 CronJob_vnas-homes_vol_homes_1 - CronJob_vnas-homes_vol_homes_2 9 CronJob_vnas-homes_vol_homes_2 - netapp02::> snapshot policy show -vserver netapp02 Vserver: netapp02 Number of Is Policy Name Schedules Enabled Comment ------------------------ --------- ------- ---------------------------------- std 1 true - Schedule Count Prefix SnapMirror Label ---------------------- ----- ---------------------- ------------------- daily 7 daily - std_homes 3 true - Schedule Count Prefix SnapMirror Label ---------------------- ----- ---------------------- ------------------- daily 7 daily - weekly 26 weekly - 2hour 24 2hour - default 3 true Default policy with hourly, daily & weekly schedules. Schedule Count Prefix SnapMirror Label ---------------------- ----- ---------------------- ------------------- hourly 6 hourly - daily 2 daily daily weekly 2 weekly weekly default-1weekly 3 true Default policy with 6 hourly, 2 daily & 1 weekly schedule. Schedule Count Prefix SnapMirror Label ---------------------- ----- ---------------------- ------------------- hourly 6 hourly - daily 2 daily - weekly 1 weekly - none 0 false Policy for no automatic snapshots. Schedule Count Prefix SnapMirror Label ---------------------- ----- ---------------------- ------------------- - - - -
I need to verify no volumes on the SVM vnas-homes are using the SVM based policy I need to remove:
netapp02::> volume show -vserver vnas-homes -volume * -fields snapshot-policy vserver volume snapshot-policy ---------- --------------- --------------- vnas-homes vnas_homes_vol0 std vnas-homes vol_homes std_homes 2 entries were displayed.
As they’re not it’s safe to remove the snapshot policy:
netapp02::> snapshot policy delete -vserver vnas-homes -policy SnapShot_vnas-homes_vol_homes_0 netapp02::> snapshot policy show -vserver vnas-homes There are no entries matching your query.
Now from the target cluster I should now be able to initialise the DR relationship:
netapp04::> snapmirror initialize vnas-homes-sm: netapp04::> snapmirror show -vserver vnas-homes-sm Progress Source Destination Mirror Relationship Total Last Path Type Path State Status Progress Healthy Updated ----------- ---- ------------ ------- -------------- --------- ------- -------- vnas-homes: DP vnas-homes-sm: Uninitialized Transferring 93.33GB false <date/time> netapp04::> snapmirror show -vserver vnas-homes-sm Progress Source Destination Mirror Relationship Total Last Path Type Path State Status Progress Healthy Updated ----------- ---- ------------ ------- -------------- --------- ------- -------- vnas-homes: DP vnas-homes-sm: Snapmirrored Idle - true -
Fixed!
Hi ,
Just want to know, is there any way if we can check/verify the snapmirror is configured on SVM or volume or both.
because when i run Snapmirror show command. it shows both relationship. the SVM and SVM:volumeName.
regards
VK
Hi Vinay
I’m not sure what you mean? If you want to know if the snapmirror relationship is a “DR” type or a more traditional “per volume” type you could run the following command on the target cluster:
snapmirror show -fields identity-preserve
If it’s “true” then it’s a “DR” type relationship and is snapmirroring the entire SVM.
Or just looking at the outpout of the “snapmirror show” an identity preserving relationship with only show the source and target SVM’s followed with a colon, no mention of the volumes, for example:
vnas-homes: DP vnas-homes-sm: Snapmirrored Idle – true –
HTH
thank you nullgenome !