 | |||||||||||
| |||||||||||
 |
|||||||||||
| |
| ||||||||||||||||
| ShaoLin HA Cluster 1.0 Installation Manual and Operations Guide | ||
|---|---|---|
| Prev | Chapter 2. Cluster Hardware Configurations and Installation | Next |
ShaoLin HA Cluster requires a reliable shared storage device to act as a shared storage for the clusters. In the event of hardware failure, it is desirable to be able to disconnect one cluster member from the shared storage for repair without disrupting the other cluster. Shared storage is truly vital to the cluster configuration.
The following storage types are supported by ShaoLin HA Cluster.
This refers to disk arrays that is just a bunch of disk. JBOD storage does not come with any redundancy, you have to use Linux software RAID (MD) to provide on-line redundancy. ShaoLin HA Cluster supports software RAID, you may refer to your Linux distribution for more information on setting up software RAID during OS installation. Or you may refer to the document Software RAID HOWTO on information setting up software RAID in Linux.
 | Non-RAID storage |
|---|---|
Using non-RAID storage can result in a single point of failure. |
This type of disk array have RAID controllers built-in, and emulate as a normal SCSI device which works independent of the operating systems and does not require special drivers.
This refers to using host bus adapters that has RAID controller built-in, and connect to a JBOD storage disk array. Not all host adapter RAID supports cluster configuration, please refer to Section 2.2.1.3 for more information and refer to the vendor of your host adapter.
ShaoLin HA Cluster uses the shared storage will as a startup drive for the clusters. It is required the shared storage is bootable from the clusters simultaneously. Most of the SCSI and fibre-channel shared storage devices are bootable, it depends on your host adapter settings. Please consult your hardware vendor for detail configuration of your SCSI or fibre-channel host adapter for bootable configurations.
Other considerations which not limited to ShaoLin HA Cluster is the configuration of disk arrays. It is highly recommended to use RAID-1 or RAID-5 in your disk array for redundancy and no single point of failure. Do not use RAID-0 which doesn't have any redundancy features. ShaoLin HA Cluster supports both hardware RAID and software RAID which provided by the Linux MD driver. The best selection would be using hardware RAID enclosure disk array boxes , in which the RAID controller is built into the disk array subsystem. Other host adapter hardware RAID is also supported by setting the host adapter RAID to use write through cache scheme. Using write back cache scheme will cause data lost in case your host adapter malfunction or disconnection of the failed cluster with pending buffers in the adapter cache. However, not all host bus adapter RAID is capable to run in a cluster configuration. Please ask your product vendor for information on using the host adapter RAID in a cluster configuration.
FC and SAN virtualized storage also have benefits from storage management. By using appropriate SAN management software, such as the IBM SAN Manager, you can configure multiple cluster pairs to share on a single storage pool. This also ensure storage consolidation and maximum utilization in management point of view. If budget is available, it is highly recommended to use SAN storage which benefits from future expansion and easy management.
It is not recommended to use multi-initiator parallel SCSI bus. It is also difficult to achieve a data rates above 80MB/sec. with the use of standard SCSI adapters. The use of multi-initiator requires Y-cable and external terminators to make sure the SCSI bus terminates properly even when one of the cluster node disconnected at either end. For single-initiator parallel SCSI buses, it requires the storage device has a multi-port backplane, or fibre channel connection are required. Multi-port single-initiator SCSI can use internal termination provided by the host bus adapter which is much easy to setup, tests also shows it can easily achieve data rate above 80MB/sec. . Therefore, single-single initiator SCSI or fibre-channel storage are recommended for cluster storage.
Using single initiator shared SCSI requires the disk array subsystem has dual SCSI ports which allow both clusters to connect to the individual SCSI ports using isolated SCSI buses. Each SCSI bus should uses internal termination provided by HBA and the disk array subsystem's controller. The disk controller should have onboard SCSI termination to provide appropriate SCSI termination for each SCSI bus. In such configurations, you may configure your HBA to enable internal termination.
Although using multi-initiator SCSI is not recommended, however it is a low cost alternative to all other possible solutions. You have to make sure you understand the concept of SCSI bus termination to continue the setup. The difficulty of using multi-initiator SCSI is the possibility of getting correct SCSI termination even in the event of a cluster disconnection. It is likely multi-initiator SCSI to achieve a low performance due to length of cable and also other factors. you should use external terminators and Y-cables for proper SCSI termination. You should also test the configuration thoroughly by triggering the failover manually through the console with Cluster Manager a couple of times before you system get into production. If you encounter problems, you should go to your BIOS setup of the HBA, adjust the bus speed to a slower speed to get possibly stability. Test has shown a 40MB/sec setup has a higher chance of stability.
Host bus adapters should be configured to boot from the shared disks. If you have multiple SCSI channels on your host bus adapter, make sure it is configured to boot from the channel that is connected to the shared SCSI bus.
SCSI IDs are used to identify SCSI devices on a SCSI bus. Since cluster configuration of shared storage requires 2 hosts connect to the same SCSI bus, the host adapter of each cluster must be different. It is required to configure the host adapter SCSI ID to the highest priority. Wide SCSI priority order as 7,6,5,4,3,2,1,0,15,14,13,12,11,10,9.8 and normal SCSI is 7,6,5,4,3,2,1,0 . You should configure the host adapters to 7 or 6 to ensure the host adapters get the highest priority.
A single initiator fibre channel has only one cluster connected to one fibre channel host port. If your fibre channel disk array subsystem has dual controllers, you can connect each cluster to each of the fibre channel host port. The dual controller of the disk array must allow the two clusters to view all the logical devices in the disk array and has the same set of logical unit numbers (LUNS).
A single port fibre channel disk array can be used in a cluster system with an additional fibre channel switch or hub. However, the fibre channel switch or hub can be a single point of failure of the system. Administrators must be aware of the risk of setting up such systems.
If you are using a SAN subsystem, and your SAN controller supports groups. Make sure you have defined the host ports of the cluster in the same group such that both clusters can boot from and see the same logical units. If you are planning to setup multiple cluster pairs, you should define one group per cluster pairs. Please consult your SAN storage vendor for more details.
It is suggested to use storage systems that has dual power supplies. This way, it allows the storage array to connect to dual power sources and have power supply unit redundancy. Please see Section 2.4.2 for more information on power supply connections.
ShaoLin HA Cluster allow all data to stored only on the shared storage and able to operate without any local or internal storage devices. The consideration and requirements of using local disks depends on your need on cluster operations and administration. Local disks are recommended to use as swap disks, since swap data are temporary data which is not important to share among the clusters. However, there are couple of reason to use local disks or not.
Using local storage allow you to spread the load on the shared storage in case of swap. Swapping to local disk avoid the I/O traffic going to the shared storage. Since local storage and external shared storage locate on a different I/O path, this will improve performance significantly in low memory systems.
Another advantage to have local disks is to allow on-line upgrade the clusters without taking down the system. Some Linux distribution installation process will automatically wake up detected swap partitions during the OS installation phase. If this happens while the ACTIVE NODE is in production during the installation or new OS on the STANDBY NODE, it is dangerous and may cause memory corruption of the production cluster. If you are using internal swap partitions, you are safe to install a new operating system on an unused shared volume to perform OS installation without interference of the clusters.
It is recommended to use RAID-1 on local disk drives to eliminate the single point of failure of the swap partitions.
Using internal disk drives can gain performance and flexibility. On the down side, managing diskless servers can gain advantages of reducing chance of failure. Running an additional disk array for each cluster can increase the chance of failure and added load to system administration.
For absolute reliability and manageability, using internal disks is not recommended.
