TEST # 1 CUT PATHS FROM HOST TO IOGRP0: THE INITIAL STATE OF THE VOLUMES BEFORE CUTTING ACCESS FROM THE HOST TO IOGRP0 THE HOST MULTIPATHING INFO AND IOMETER WORKLOAD IMMEDIATELY AFTER STARTING IOMETER: 1
CISCO SWITCH VIEW: Traffic flows to IOGRP0 only on adapters: 20:00:5d and 20:00:5e MIRRORED COPY IS IN CONSISTENT SYNCHRONIZED STATE (Master to Auxiliary direction) 2
STOP TRAFFIC ON ACTIVE ADAPTERS USING CISCO SWITCH, DISABLED V7000 INTERFACES TO NODE1 AND NODE2 (GBURG-03 SIITE) THE HOST MULTIPATH LOOKS LIKE THIS (4 CLOSED PATHS ON EACH VOLUME): 3
THE V7000 STOPS IO FOR 30 SECONDS AND RESUMES, NOW USING ADAPTERS ON IOGRP1, BUT STILL USING THE STORAGE POOL FROM GBURG-03 SITE THE CISCO SWITCH SHOWS TRAFFIC MOVES TO THE ADAPTERS OF IOGRP1: 4
THE V700 VOLUMES KEEP THE SAME STATE : WE GET THE FOLLOWING EVENTS IN V7000: 5
AFTER 20 MINUTES, THE V7000 DROPS ALL IO FOR ABOUT 1 MINUTE AND THEN RESTARTS, THIS TIME ACCESSING THE DATA LOCALLY (ON THE GBURG-05 POOL) THE MIRROR RELATIONSHIP CHANGES DIRECTION, MAKING THE AUXILIARY (GBURG-05) THE MASTER 6
RESTORE THE TWO ORIGINAL PORTS ON IOGRP0: ALL PATHS ARE OPEN TRAFFIC ON THE CISCO SWITCH CONFIRMS THAT THE DATA IS FLOWING TO IOGRP0 ADAPTERS: 7
THE MIRROR RELATIONSHIP CONTINUES WITH AUXILIARY AS MASTER : AFTER ABOUT 20 MINUTES, THERE IS AN IO INTERRUPTION OF ABOUT 1 MINUTE, THE MIRROR RETURNS TO THE NATURAL MASTER-- AUX DIRECTION, THE PERFORMANCE RETURNS TO THE NORMAL LEVELS AS THERE IS NOT AS MUCH BACK-AND-FORTH TRAFFFIC BETWEEN IOGRPS: 8
THE MIRROR RETURNS TO NORMAL MASTER -- AUX DIRECTION: 9
TEST # 2 LOSS OF INTERNAL STORAGE ON IOGRP0: HERE IS THE STATUS AT THE HOST WHEN STARTING THE IOMETER WORKLOAD: THE INITIAL STATE OF THE VOLUMES BEFORE CUTTING ACCESS TO INTERNAL STG. TO IOGRP0: 10
CISCO SWITCH SHOWS DATA FLOWING ON THE PORTS 20:00:5D and 20:00:5E, USED BY IOGRP0: 11
MIRRORED COPY IS IN CONSISTENT SYNCHRONIZED STATE (Master to Auxiliary direction) 12
BREAK THE CONNECTIONS TO THE STORAGE POOL (POWER OFF EXPANSION ENCLOSURE ON GBURG03_POOL): ERRORS ARE REPORTED ON V7000: ACCESS TO STORAGE CONTINUES, THEN AFTER A PAUSE ON IO FOR ABOUT 40 SECONDS, IT RESUMES AT A LOWER PERFORMACE (USING IOGRP1 STORAGE POOL) 13
HOST CONTINUES TO USE IOGRP0 ADAPTERS, BUT NOW ACCESSING THE STORAGE ON THE GBUR05_POOL: THE STORAGE POOL ON GBURG-03 IS OFFLINE, BUT THE SYSTEM, TO KEEP THE HOST WORKING WITHOUT CHANGE, SHOWS THAT THE TWO VOLUMES (GBURG_VOL10 AND GBURG_VOL20) ARE STILL ONLINE WITH THEIR ORIGINAL ASSOCIATION WITH POOL GBURG03_POOL, WHICH WE KNOW IS OFFLINE, BUT INTERNALLY THE V7000 IS WORKING THAT OUT AS THE VOLUMES BEING USED ARE GBURG_AUX10 AND AUX20. 14
THE MIRROR STATE IS IN CONSISTENT COPYING STATE, BUT NOW AUX - MASTER DIRECTION WITH A FREEZE TIME: 15
RESTORE POWER TO EXPANSION ENCLOSURE, TO RESUME ACCESS TO STORAGE ON IOGRP0: ALL EVENTS AUTOMATICALLY FIXED: A LITTLE DIP IN PERFORMACE, AS THE MIRROR STARTS SYNCHRONIZING FROM AUX - MASTER: 16
THERE IS A PAUSE IN IO FOR ABOUT A MINUTE, TO START USING THE LOCAL STORAGE AFTER THE MIRROR FINISHED SYSNCHORNIZATION, ONCE LOCAL STORAGE IS USED, THE PERFORMANCE RECOVERS TO THE ORIGINAL LEVELS. 17
THE HOST CONTINUED IOMETER WORKLOAD, WITHOUT A PROBLEM: MIRROR IS NOW CONSISTENT SYNCHRONIZED (Master to Auxiliary direction): 18
TEST # 3 LOOSE IOGRP0: INITIAL STATUS ON THE VOLUMES: THE HOST MULTIPATHING INFO AND IOMETER WORKLOAD IMMEDIATELY AFTER STARTING LOAD: 19
CISCO SWITCH VIEW: Traffic flows to IOGRP0 only on adapters: 20:00:5d and 20:00:5e 20
PERFORMANCE STATISTICS ON THE V7000: MIRRORED COPY IS IN CONSISTENT SYNCHRONIZED STATE (Master to Auxiliary direction) 21
BREAK CONNECTIONS ON ALL IOGRP0 ADAPTERS (LOOSE THE IOGRP0 CONTROL ECLOSURE): THE V7000 PERFORMANCE SHOWS IO STOPS FOR ABOUT 60 seconds, ON A REDUCED PERFORMANCE RATE : 22
THE HOST SHOWS 4 PATHS CLOSED (THE ONES FROM IOGRP0): THE MIRROR RELATIONSHIP SHOWS CONSISTENT COPYING BUT DIRECTION HAS CHANGED, THE MIRROR IS COPYING FROM AUXILIARY TO MASTER WITH A FREEZE TIME: 23
RESTORE IOGRP0 FIBER CHANNEL CONNECTIONS: IO STOPS FOR ABOUT40 SECONDS, THEN RESUMES AT THE SAME PERFORMANCE LEVEL: THE MULTIPATH SDD ON THE WINDOWS SERVER, SHOWS PORTS FROM IOGRP0 ARE CLOSED, AT THIS POINT THE FIBER CHANNEL ADAPTERS ON IOGRP0 ARE ACTIVE, BUT STILL NO TRAFFIC FLOWS: 24
TRAFFIC FLOWING THROUGH IOGRP1 ADAPTERS: THE MIRROR CONTINUES COPYING FROM IOGRP1 to IOGRP0 25
ONCE THE COPY OPERATIONS COMPLETE, THE MIRROR GOES TO CONSISTENT SYNCHRONIZED, BUT STILL THE DIRECTION IS FROM AUX-- MASTER 26
REBOOTED BOTH NODE1 and NODE2 (Components of IOGRP0 in this test). THE SYSTEM RETURNS TO ITS ORIGINAL STATE OF USING IOGRP0 HOST ADAPTERS AND GBURG03_POOL STORAGE: THE MIRROR RETURNS TO ITS NORMAL FLOW, FROM MASTER- AUX 27
TEST # 4 LOSE QUORUM SITE: THE STATUS OF THE VOLUMES ON SITE GBURG-03 (MAIN SITE) IS: HERE IS THE STATUS AT THE SERVER SHHOWING MULTIPATHING AND THE IOMETER WORKLOAD: THE STATUS OF THE QUORUM IS: 28
THE QUORUM SITE RESIDES IN THE DS8K (EXTERNAL STORAGE): WORKLOAD IN THE V7000: 29
BREAK THE CONNECTIONS TO THE CONTROLLER THAT HAS THE ACTIVE EXTERNAL QUORUM USING THE CISCO SWITCH: HERE IS THE STATUS OF THE QUORUM AFTER FAILING EXTERNAL THIRD SITE: THE V7000 REPORTS THE LOSS: 30
STARTED RECOMMENDED ACTION TO FIX QUORUM DISK NOT AVAILABLE: CONTINUE WITH SEVERAL PANELS OF RECOMMENDED ACTIONS UNTIL WE GET TO THIS LAST ONE: AT THIS POINT THE HARDWARE PROBLEM HAS TO BE ADDRESSED TO FIX THE EXTERNAL QUORUM DRIVE TO GET THROUGH THE RECOMMENDED MANAGEMENT PROCEDURE. 31
THE APPLICATION CONTINUES WITHOUT A PROBLEM: REACTIVATE CONNECTIONS TO THE ACTIVE EXTERNAL QUORUM USING THE CISCO SWITCH: 32
EVENTS IN THE V7000 GET AUTOMATICALLY FIXED AND QUORUM REACTIVATES IN THE EXTERNAL CONTROLLER LOCATED ON SITE QUORUM: 33
TEST # 5 LOSE MDISK HOLDING THE QUORUM (Controller still online): QUORUM DISK STILL LOCATED ON MDISK3 ON THE EXTERNAL CONTROLLER: USING THE REMOVE MDISK FUNCTION, WE REMOVE THE ACTIVE QUORUM MDISK FROM THE QUORUM_POOL: THE DS800 CONTROLLER SHOWS THAT MDISK3 IS UNMANAGED: 34
V7000 IMMEDIATELY GRABS ANOTHER MDISK FROM THE QUORUM EXTERNAL CONTROLLER, TO HOUSE THE QUORUM: WE CAN REASSIGN THE MDISK3 TO THE POOL, WITHOUT AFFECTING THE CURRENT ALLOCATION OF THE QUORUM MDISK: THE QUORUM MDISK CONTINUES WHERE IT WAS BEFORE: 35
36