HDS VSP Storage Daily Maintenance Manual

HDS VSP Storage

User Maintenance Manual

(May 2013)

I. Equipment Maintenance

1.1 VSP Configuration Information

1.2 VSP System Architecture

1.2.1 VSP System Architecture Diagram

Its system architecture adopts a switched architecture, as shown below.

Figure 2-1: VSP Hardware Technical Architecture

1.2.2 VSP Main Components

Figure 2-2: VSP Hardware Main Components Diagram

The main components of the VSP include:

1. Front-end Card (CHA): Responsible for connecting external hosts or storage switching devices, such as minicomputers;
2. Back-end Card (DKA): Responsible for connecting internal disk storage devices, such as hard drives;
3. Data Cache Card (CPC): Data cache area when CHA accesses back-end disks;
4. Memory Module (CM): Memory modules of CM;
5. PCI-Express Switch Card (ESW): DKC data switching control card;
6. Processor Board (MPB): Responsible for processing front-end and back-end data;
7. Disk (HDD): Disks of the storage device;
8. Spare Disk: Disks used as spares;
9. Battery (Battery): Battery to save data during system power failure;
10. Power Supply (PS): Power supply for system operation and switching;
11. Fan (FAN): Fan for system cooling;
12. Console (SVP): Console for system configuration, management, and monitoring (one laptop);
13. Cable: Cables connecting internal and external components of the storage;
14. Fiber Optic Connector (SFP): Fiber optic conversion connector;

1.2.3 VSP Storage Disk Distribution Diagram

The disk distribution of the VSP is identified as follows:

HDDxyz_dd

Where: xy=Cabinet;

z=HDU number

dd=HDD number

Figure 2-3: VSP Disk Location Diagram

1.3 Common Maintenance Commands

When a failure occurs in the storage system—such as an abnormality in a disk, control card, power supply, memory, or battery—SIM information is generated. At the same time, depending on the severity of the fault, the storage system illuminates the Message or Alarm light. When the Message light is on, system usage is not affected, but it should be addressed promptly. When the Alarm light is on, system usage will be affected and should be addressed as soon as possible.

1.3.1 SVP Login

On-site maintenance personnel can perform daily maintenance work by remotely logging into the VSP storage console SVP via Remote Desktop:

VSP:

Click the "History" and "Register" buttons in sequence to access the SVP.

1.3.2 Interface After SVP Login

The following shows the interface seen by the maintenance engineer after logging into the SVP;

1.3.3 SIM Information Check

If there is a fault in the storage device, the console will display SIM Message: Pending SIM exists. At the same time, the yellow SIM alarm light on the top-left corner of the front panel of the storage device will light up.

Click the Information button to view the SIM log:

Click Log...

List SIM:

1. Status column shows "Initial": New SIM information exists and requires attention.
2. Status column shows "Completed": SIM information that has been manually (or automatically) terminated; this is generally performed by an engineer completing the Complete operation after confirming the storage maintenance is finished and the system has returned to normal.

Select a row of SIMs and click "Content" to view detailed information:

If possible, it is best to save the displayed content via screen capture and email it to the HDS engineer.

1.3.4 Storage Component Status Check

In addition to viewing SIMs on the SVP interface, you can also enter the Maintenance menu to check the current storage operating status:

If there is hardware or other faults, the corresponding component locations will flash to indicate an alarm.

1.3.5 Clearing Information in SIM

First, enter Modify mode from View Mode by clicking 'View Mode':

The SVP changes to 'Modify Mode':

Navigate sequentially to: Information -> Log… -> List SIM, select the SIM information requiring completion, and click Complete:

Confirm to proceed.

Before exiting the SVP, change Modify Mode back to View Mode!

1.3.6 Collecting DUMP Information

Select the Auto Dump button:

In the following interface, select the dump type and transfer method. Typically, select Normal for Type; choose Detail when performance analysis is required. Media is generally set to HDD (SVP local disk). No other options need modification.

After selecting OK, the dump collection progress will be displayed:

Upon completion of dump data collection, compression begins:

After compression finishes, a completion notification appears:

After collection is complete, copy the file c:\dkc200\tmp\hdcp.tgz out and upload it to TUF.

1.3.7 Partitioning LUNs Using SVP

Modify mode, enter the LUN configuration interface

1. Set the CHIP type as needed
2. Configure port parameters and host mode

Enable port security mode:

Select Enable

After clicking OK, two system security prompts appear; select 'Yes'

Select the correct connection method:

A system security prompt appears; select 'Yes' to complete.
Creating Host Groups, Binding Host WWNs, and Mapping LDEVs:

Right-click and select "New".

Select the correct HBA card WWN for the corresponding port and click OK.

Click Next upon completion.

Select the operating system platform type and associated options. If the host is AIX with an HA environment, select HACMP.

A prompt will appear stating that I/O interruption will occur on this host port; do you wish to continue? Click "Yes" to proceed (since I/O interruption is momentary and will not affect host production; however, performing this operation during peak business hours may impact host production).

The figure below shows the completed binding of host HBA card WWNs within the host group.

Select LUN in the Display option to begin mapping host disks.

After selecting Add, a disk selection window appears. Select the required number of LUNs and corresponding number of LDEVs, click "Set", then click OK to complete disk addition:

A prompt will appear stating that I/O interruption will occur on this host port; do you wish to continue? Click "Yes" to proceed (since I/O interruption is momentary and will not affect host production; however, performing this operation during peak business hours may impact host production).

The figure below shows the state after disk addition within the host group is complete:

Note the difference between the figure above and this one: a "+" sign appears after the LDEV. The presence of "+" indicates that this disk has been mapped twice.

To add a new host and create a new host group, perform WWN binding and disk mapping; once completed, the host can recognize the disks. The operation method remains exactly the same. Note that in the disk selection window when mapping or adding new disks, ensure you select disks without a "+" sign. Disks without a "+" sign are unused.

After completing the operation, exit and save the new configuration file.

Upon completion of all steps, return to "View Mode" to prevent erroneous operations.

Close the remote desktop connection and select "OK".

1.4 VSP Emergency Plan - Power On/Off Operations

Caution: Powering on/off affects host I/O; please take note!

Power-On Procedure

1. Turn on the main circuit switches on the four PDU power modules located on both sides at the rear of the control cabinet and disk cabinet. Confirm that the yellow BASE ON indicator light on the front main panel (DKC PANEL) of the rack illuminates, and simultaneously, the System Control Computer (SVP) located at the lower front of the control cabinet begins to start.

1. Push the "PS ON/PS OFF" switch to the "ON" position, and simultaneously push the "PS SW ENABLE" switch to the ENABLE position.
2. Confirm that the green PS-ON indicator light on the controller panel is lit, and you can hear the sound of the internal fans spinning.
3. System startup may take 15~25 minutes; boot time varies depending on the configuration. Upon completion of startup, the green READY indicator light on the controller panel will illuminate. If the READY indicator does not light up after a long period, contact HDS technical support immediately.
   Shutdown Procedure
4. Confirm that the host connected to the storage has no I/O access.
5. Push the "PS ON/PS OFF" switch to the "OFF" position, and simultaneously push the "PS SW ENABLE" switch to the ENABLE position.

1. Normal system shutdown may take 15~20 minutes; shutdown time varies depending on the configuration.
2. Confirm that upon completion of system shutdown, all indicator lights on the controller panel extinguish except for the yellow BASE ON indicator, which remains lit, and the internal fans stop spinning. If any abnormal conditions are observed, contact HDS technical support immediately.
3. Turn off the main circuit switches on the four PDU power modules located on both sides at the rear of the control cabinet and disk enclosure, and confirm that the yellow BASE ON indicator light on the front main panel of the cabinet extinguishes.
4. If a long-term shutdown exceeding 24 hours is required, disconnect the "Battery Cable". Contact an HDS engineer if needed.

1.5 Equipment Inspection

1.5.1 Health Inspection Content

Health checks for the HDS storage system primarily include the following five aspects:

1. Equipment Site and Environment Check

Definition: Is the external operating environment status and numerical values of the equipment normal? Are there any abnormalities or faults detected?

Scope: Includes: anti-static flooring and installation condition, thermometer/hygrometer readings, equipment ventilation status, power supply status, and other specific information or values;

1. Equipment Hardware Check

Definition: Is the equipment hardware status and information normal? Are there any abnormalities or faults detected?

Scope: Power supply, batteries, connected fiber optics, main board cards (DKA/CHA/Cache/Memory/FSW) status, hard disk status within HDU, and system status information collected and monitored via the SVP console.

1. Equipment Software Check

Definition: Check the status and information of major software running on the equipment. Are system logs and dump files normal? Are there any abnormalities or faults detected?

Scope: Includes: HDLM multipath software, ShadowImage mirror software, TC/UR synchronous/asynchronous data replication software configuration information and status;

1. System Performance Check

Definition: Are system performance parameters normal? Are certain measures required?

Scope: Includes: Cache write wait time, channel card utilization, channel efficiency, etc.;

1. Equipment Security Check

Definition: Are there alarm or error messages in system information or logs? Are other measures required?

Scope: Includes: SIM information, SSB logs, DUMP information, Microcode version, backup disks, and redundancy link light checks;

1. Fibre Channel Switch Check

Definition: Check if the hardware indicator lights of the storage system are normal? Are there any abnormalities or faults detected?

Scope: FC switch power supply, fans, status lights, Zone configuration, etc.;

1.5.2 VSP System Health Check Report

VSP Series Device Health Check List

This table is for inspection by users who have not installed Hi-Track. If any issues arise, please contact the HDS China Representative Office: Beijing (10-85182238), Shanghai (021-62882700), Guangzhou (020-38251188), Chengdu (028-86788686).

#pairdisplay –g <group> -fcx

#pairdisplay –g <group> -fcx

#usr/DynamicLinkManager/bin/dlnkmgr view –path –c -srt lu

#oslevel –r 

#oslevel –l

#ioslevel

II. Microcode Upgrade

2.1 Significance of Microcode Upgrade

HDS storage devices release new microcode versions at regular intervals to fix system bugs, improve device availability, and reduce abnormal failures. After running for a long period, HDS storage devices may require related spare part version updates. For disk devices, the spare parts library will no longer provide certain older disk versions; they are replaced by newer disk versions.

HDS recommends that the microcode running on storage devices should not be lower than MGA (i.e., Minimum Supported Version). Microcode upgrades generally consider using RGA (i.e., Recommended Version). Newly installed devices must reach at least PBL (i.e., Factory Installed Version).

HDS defines microcode versions as follows:

2.2 Microcode Upgrade Strategy

We recommend upgrading the microcode version twice a year. When encountering major bugs with high frequency related to the customer environment, HDS will promptly provide upgrade recommendations and suggest customers upgrade the microcode version of relevant devices in advance.

2.3 Microcode Upgrade Steps

2.3.1 Microcode Upgrade Approval Process

To ensure safe and reliable microcode upgrade services, HDS engineers will collect VSP log information in advance, submit CHIP to the Technical Center, and perform a deep check of the device operational status.

2.3.2 Microcode Upgrade Operation Steps

HDS engineers perform an HDS storage health check to confirm that the device hardware status is normal;

1. VSP Microcode Upgrade Steps

1. The HDS engineer begins the microcode upgrade. The upgrade process automatically copies the new microcode to the HDS storage device SVP;

2. The microcode upgrade process will proceed component-by-component automatically, as shown in the table below:

First, upgrade the SVP. After rebooting, upgrade all devices except DKU, and finally upgrade the DKU. During the upgrade process, microcode is written to every CPU on the front-end ports and back-end ports. During the upgrade process, we select Non-stop mode, i.e., without stopping the system.

1. After the upgrade is completed, confirm that the microcode upgrade has finished, back up the configuration file, restart the SVP, and check the new microcode version and hardware status;

1. HDS engineers and system coordination personnel check the link status of the equipment involved in the upgrade to confirm that the system status is normal.
2. Execution Process

(1)<Initial screen>

(2)Change the mode to [Modify Mode] (CL).Select (CL) [Install].

(3)In the ‘Install’ window, select (CL) [Micro Program Install].

(4)Select (CL) [Micro Program]. Select (CL) [OK]. Selecting (CL) [Cancel] returns you to step(3).

(5)In the ‘Microprogram Exchange’ dialog box, [How] : select (CL) [Online].

[From]: select (CL) CD-ROM.

(6)The ‘Microprogram Exchange’ dialog box appears. Select (CL) one or more items from the list of the type of micro-programs, and select (CL) [OK].

(7)<If SCSI channel adapter is equipped> Select (CL) one of the exchange modes of DKCMAIN micro-program as follows, and select (CL) [OK].

(7-1) <If "Non Stop SCSI host" is selected > Select [OK] (CL) after checking the right preconditions. Go to step (8).

(8)When DKCMAIN, or RAMBOOT were selected, you can select the reboot pattern from the list.

(9)On the ‘Software Maintenance’ dialog box, the micro-program types, current versions, new versions, and message are displayed.

(10)In response to the message "An exchange of a microprogram finished. Please check subsystem status and microprogram version with MAINTENANCE.", select (CL) [OK].

III. Fault Handling

In daily maintenance work for storage systems, manufacturer engineers and customer administrators most frequently deal with fault handling and troubleshooting. Different fault phenomena or types require different handling solutions or implementation strategies to ensure rapid problem resolution while minimizing impact on the customer's production and operational systems.

3.1 Fault Handling Plan

For the types of faults and situations that may occur with VSP devices on-site, the following table provides some suggested handling measures and solutions so that on-site engineers can perform corresponding handling and maintenance work based on the fault phenomena or issues. For specific details and operation guides, please refer to relevant documents.

As on-site customer personnel, upon discovering the following faults, you must notify HDS service team personnel immediately or report the service issue via the HDS 400 service number (400-6786783) to ensure rapid problem handling and resolution.

Table 5-1: On-site Fault Repair Plan

IV. Cases

4.1 Hard Disk Replacement Steps

When performing disk replacement, strict anti-static measures must be implemented to eliminate static electricity from the spare parts; otherwise, other components may malfunction.

1. Use a qualified anti-static wrist strap and connect it to the device's metal grounding component;
2. Before touching components, touch the device's grounded metal parts with both hands for more than 5 seconds to eliminate human body static electricity;
3. No component should be loosely inserted into the device;
4. Before installing the disk, touch the disk's metal casing to the device's metal grounding component to fully eliminate static electricity from the disk casing;
5. During installation, keep your palm wearing the anti-static wrist strap in contact with the disk's metal casing;
6. For specific measures, consult an HDS support engineer.

When performing disk replacement, confirm the status of the disk to be replaced, including the status displayed on the Maintenance screen, SIM logs, etc.

1. If the disk to be replaced is a data disk and data has already been copied to the Hot Spare disk, proceed with the normal disk replacement steps. As shown in the figure, Device Status=Failed, Group=Spare.

1. If the disk to be replaced is a data disk and the disk array is currently copying data to the Hot Spare, pause the replacement operation and proceed with the normal disk replacement steps after the data copy is complete. The figure shows the state where the data disk is undergoing Sparing, Device Status displays the Copy progress, and Group status displays the original RG.

1. If the disk to be replaced is a data disk but is in a normal state, perform preventive maintenance steps: copy the data from the data disk to the Hot Spare, then proceed with the replacement. At this time Device Status=Normal, Group status displays the original RG.
2. If the faulty disk itself is a Hot Spare, you can proceed with the normal disk replacement steps.
3. If the faulty disk only has an RG defined but no LDEV defined, even if Device Status=Failed, the disk array will not start copying data to the Hot Spare, and you can proceed with the normal disk replacement steps.

Normal Disk Replacement Steps

1. Change the SVP to Modify mode;
2. Enter the Maintenance screen, select the disk to be replaced, and confirm that the disk status meets the normal replacement conditions (see the explanation earlier in this chapter).
3. Select [Replace]-[Replace] and follow the SVP prompts to perform the replacement.
4. When the SVP displays the following prompt, answer [Yes].

1. After data Copy Back starts, the SVP prompts that the disk replacement is complete. At this point, you can view the Copy progress via the Copy Status on the Maintenance screen. This screen does not update automatically; you can refresh the progress status by clicking [Renew].

Disk Preventive Maintenance

If the data disk status is Normal (Device Status=Normal) and the disk needs to be replaced due to special reasons, such as a port error on the disk, use the preventive maintenance steps. Before implementation, confirm the necessity of the maintenance with an HDS support engineer.

To ensure data security, copy the data from the data disk to the Hot Spare, then proceed with the replacement.

Before implementing preventive maintenance, confirm that the disk array has a suitable unused Hot Spare..

1. Change the SVP to Modify mode;
2. Enter the Maintenance screen, select the disk to be replaced, and confirm that the disk status is Normal.
3. Select [Other]-[Spare Disk]. When the SVP displays the following prompt, answer [Yes].

1. After data copying is complete, the data disk status changes to Failed/ Spare.
2. Perform disk replacement according to the normal disk replacement procedure.

4.2 CHA Replacement Procedure

[Note] The following steps are simulated based on a laboratory environment, including Pre-replacement (PRE), Replacement, and Post-replacement (POST);

1. The port numbers for the two storage CHAs (1EU, 2QU) are respectively

1EU-1A 3A 5A 7A 1B 3B 5B 7B, 2QU-2A 4A 6A 8A 2B 4B 6B 8B

2. Multiple hosts (OS—aix) with two HBA cards each, each card having two fiber ports.

3. Two switches.

SAN Architecture Diagram:

4. Port 8B has failed. CHA2QU needs to be replaced.

Procedure:

PRE PROCEDURE

1. Use the rdpexe program to search for the array, connect, and install the USB device simultaneously.

2. Change the operation mode to "Modify" and enter the maintenance window.

3. Set the path to offline or switch the channel.

A. View the correspondence between all host ChaIDs and the storage front-end ports.

#/usr/DynamiclinkManager/bin/dlnkmgr view –cha

B. set path offline (disconnect all paths related to 2A, 4A, 6A, 8A, 2B, 4B, 6B, 8B from all hosts)

#/usr/DynamicLinkManager/bin/dlnkmgr offline -chaid 00003

#/usr/DynamicLinkManager/bin/dlnkmgr offline -chaid 00002

4. In the "Maintenance" window, select [DKCBOX].

5. Select [logic box front or back].

6. Select CHAXX.

7. Specify the corresponding CHA board and perform Replace.

8. When the following window appears, select "yes".

9. Collect error information

10. Enter the information and select "OK".

Confirm that the information has been stored.

11. Confirm channel is offline

Perform the above confirmation for serial ports or mainframe fiber front-end boards.

Perform the above confirmation for fiber or iSCSI front-end boards.

For mainframe fiber front-end boards where CUIR function is enabled, the "Confirm channel offline" message will not appear; instead, the following message will be displayed.

12. Confirm the following information

13. Close CHA information.

14. Check if the shut down LED is blinking.

Select “No” twice to force the shut down LED to turn on.

15. Begin replacement, then perform confirmation.

Hardware PROCEDURE

1. Remove the CHA card.

Check that the shut down LED is lit; if not, insert the maintenance jumper into the shutdown connector (online replacement only).

Remove fiber cables from the card.

Loosen screws and remove the card.

Remove the maintenance jumper.

2. Install spare card

Install in the original position and tighten screws.

3. Clean fiber cable connectors.

4. Connect fiber cables.

POST PROCEDURE

1. Wait for Power Event

2. Perform CUIR recovery when replacing the mainframe fiber card

3. Check CHA recovery

4. Set paths to online on the host side.

#/usr/DynamicLinkManager/bin/dlnkmgr online -chaid 00002

#/usr/DynamicLinkManager/bin/dlnkmgr online -chaid 00003

5. Complete SIM logs.

6. Check the system status in the "Maintenance Window"; if there is no abnormality, close the Maintenance Window.

7. Change the mode to "View".

8. Close remote desktop connection.

4.3 DKA Replacement Steps

PRE PROCEDURE

1. Use the rdpexe program to search for the array, connect, and install the USB device simultaneously.

2. Change the operation mode to "Modify" and enter the maintenance window.

3. In the "Maintenance" window, select [DKCBOX].

4. Select [logic box front or back].

5. Select DKAXX.

6. Specify the corresponding DKA card and perform Replace.

7. When the following window appears, select "yes".

8. Collect error information

9. Enter information and select 'OK'.

Confirm information storage.

10. Confirm the following information

11. Block DKA information.

12. Check if the shut down LED is flashing.

Select “No” twice to force the shut down LED to turn on. 'No' twice to forcibly turn on the 'shut down LED'.

13. Begin replacement, then confirm upon completion.

Hardware PROCEDURE

1. Remove the DKA card.

Check that the shut down LED is lit. If not, insert the maintenance jumper into the shutdown connector (online replacement only).

Remove the CABLE cables from the card.

Loosen screws and remove the card.

Remove the maintenance jumper.

2. Install spare card

Install in the original position and tighten the screws.

Connect the Cable cables to the replaced card.

POST PROCEDURE

1. Wait for Power Event

2. For DKA

* For DKA

"DKU PATH INLINE is now running..." is displayed.

3. Check the recovery process

* For DKA

"Restoring the DKA..."

4. Check CHA recovery

5. Complete SIM log.

6. Check the system status in the "Maintenance Window"; if there is no abnormality, close the Maintenance Window. 'Maintenance Window'. If no abnormalities are found, close the maintenance window.

7. Change the mode to "View". 'View'.

8. Close remote desktop connection.

4.4 CM Replacement Steps

PRE PROCEDURE

1. Use the rdpexe program to search for the array, connect, and install the USB device simultaneously.

2. Change the operation mode to "Modify" and enter the maintenance window. 'Modify' and enter the maintenance window.

3. In the 'Maintenance' window, select [DKCBOX].

4. Select [Logic Box front or Back].

5. Select Cache and perform Replace.

6. Perform pre-replacement checks.

7. Compress error information

8. Input information and select 'OK'.

Confirm information storage.

9. Check DKP closure

"The Cache Memory PCB (CACHE-nnn) is being blocked." is displayed.

10. Check if the shut down LED is lit

If not lit, use a jumper.

11. CACHE replacement

Hardware PROCEDURE

1. Remove the CM card.

Check if the shut down LED is lit; if not, install a jumper to the shut down connector (online replacement only).

Remove screws and the CM card.

Remove the jumper.

2. Replace damaged components.

When replacing the CM card, transfer all memory modules (including blanking plates) to the same positions on the spare card.

When a memory module fails, replace only the memory module.

3. Install the CM card and tighten the screws.

POST PROCEDURE

1. Confirm information

2. Verify that the replacement steps are complete.

3. Mark SIM logs as complete.

4. Check system status in the 'Maintenance Window'; if no anomalies are found, close the maintenance window.

5. Change mode to 'View'.

6. Close the remote desktop connection.

Images localized into English for illustration; labels may vary by product version.