MPxIO

Configuring MPXIO in Solaris 10

This document discusses the configuration of MPXIO on Solaris 10 for failover with a single-port HBA. While this configuration isn't as redundant as one with multiple HBAs, configuring multipathing access may still be necessary. For example, your array may present multiple paths to the same LUN over the single HBA. Without controlling access to the LUN with MPXIO (or another product such as Veritas DMP), failover will not occur properly if one of the paths to the LUNs disappears.

Environment Configuration
* Sunfire T1000
* QLE2460 4Gbps single-port HBA
* Hitachi AMS200 Array
* McData Switches

Output of format:

1. c1t50060E801049CF50d0
/pci@780/SUNW,qlc@0/fp@0,0/ssd@w50060e801049cf50,0
...
3. c1t50060E801049CF52d0
/pci@780/SUNW,qlc@0/fp@0,0/ssd@w50060e801049cf52,0

These two devices refer to the same LUN on the AMS (LUN 0). This is determined by the 'd0' at the end of the device name. What we are observing here are two different paths to the same device. One path to LUN 0 is over the 50 interface (c1t50060E801049CF50d0). The other path to LUN 0 is over the 52 interface (c1t50060E801049CF52d0).

What's wrong with this picture?
The issue is that we need to mount only one of these paths to the LUN. The path that we need to choose is the path that is the 'owner' of the LUN. The LUN's owner can be determined on the SAN management station. If, for example, the owner is the 50 interface, we should mount c1t50060E801049CF50d0. However, if the 50 interface has issues and we lose that path, we will lose access to the LUN.

We need a layer to abstract the two paths into a single device that we use. This layer would present a new virtual device that could be mounted. Underneath this virtual device would be the two paths to the LUN, the owner and the non-owner. This resolves the issue of what device to use if an interface to the LUN fails. The multipathing layer would simply start accessing the device over the alternate path and would not require any changes to what device the OS accesses the filesystem over.

In Solaris 10, MPXIO provides this layer of abstraction to solve this issue.

Configuring MPXIO
There are a few files that we need to modify to enable MPXIO.

/kernel/drv/fp.conf
To configure this file, you need to determine the path to the fp devices. To do this, run the following command:

# ls -l /dev/fc
lrwxrwxrwx 1 root root 46 Aug 15 18:36 fp0 -> ../../devices/pci@780/SUNW,qlc@0/fp@0,0:devctl

We are interested in the path after '/devices' and before '/fp@0,0...'
In this case, the path we are interested in is: '/pci@780/SUNW,qlc@0'. This path is
associated with port 0 of the fp driver (from the '0' in fp0).

We need to turn on MPXIO for this device and port by setting the following in
/kernel/drv/fp.conf:

mpxio-disable="yes";
name="fp" parent="/pci@780/SUNW,qlc@0" port=0 mpxio-disable="no";

This disables MPXIO globally and enables it for the above device.

/kernel/drv/scsi_vhci.conf
The scsi_vhci driver is responsible for hiding the owner and non-owner paths to the LUNs. To enable this capability, the driver must know what type of device that the HBA is connected to. To do this, you need to set the following in /kernel/drv/scsi_vhci.conf:

device-type-scsi-options-list="HITACHI DF600F", "symmetric-option";
symmetric-option = 0x1000000;

We knew to set the option to "HITACHI DF600F" from the output of format:

1. c1t50060E801049CF50d0 <HITACHI-DF600F-0000 cyl 981 alt 2 hd 50 sec 768>

After making these changes, reboot the server with the following command:
# reboot -- -r

After reboot, format will show the following:

1. c2t4849544143484920373330343031383130303030d0 /scsi_vhci/ssd@g4849544143484920373330343031383130303030

Note: the order of the devices that are displayed may be altered! What used to be #1 in the output of format may show up as #2.
Additionally, notice that we are now accessing the devices via the scsi_vhci driver.

To determine what devices are behind these virtual devices run the following command:

# stmsboot -L
non-STMS device name STMS device name
------------------------------------------------------------------
/dev/rdsk/c1t50060E801049CF50d0 /dev/rdsk/c2t4849544143484920373330343031383130303030d0

/dev/rdsk/c1t50060E801049CF52d0 /dev/rdsk/c2t4849544143484920373330343031383130303030d0

This shows that both paths to LUN 0 (50 and 52) are now behind the device:

/dev/rdsk/c2t4849544143484920373330343031383130303030d0

Finally, reconfigure the system's vfstab and other configuration files to start using the filesystems on the devices now under MPXIO control.