Each redundant FC-AL pair has its own pair of KGPSA-BC Host Adapters in
each cluster node, a pair of storage hubs, and RA8000/ESA12000 Storage
Subsystem enclosures (with two HSG80 Array Controllers in each master
enclosure). In the diagram, the hardware components that constitute the second
redundant FC-AL pair are shaded.
24 drives 24 drives
HSG80 Ctrl A HSG80 Ctrl A
HSG80 Ctrl B HSG80 Ctrl B
RA8000/ RA8000/
ESA12000 ESA12000
(Master) (Master)
24 drives
24 drives
Storage Hub
Storage Hub
KGPSA-BC
Host Adapters (4)
ProLiant ProLiant
Server Server
Storage Hub
Storage Hub
HSG80 Ctrl A HSG80 Ctrl A
HSG80 Ctrl B HSG80 Ctrl B
RA8000/ RA8000/
ESA12000 ESA12000
(Master) (Master)
24 drives
24 drives
24 drives
24 drives
24 drives
24 drives
Figure 2-3. Two-node PDC/O5000 cluster with two redundant Fibre Channel
Arbitrated Loops
In Figure 2-3, the first redundant FC-AL pair contains six RA8000/ESA12000
Storage Subsystem enclosures. The second redundant FC-AL pair contains
four RA8000/ESA12000 Storage Subsystem enclosures.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
2-8 Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Shared Storage Subsystem Components
The PDC/O5000 is based on a cluster architecture known as shared storage
clustering, in which clustered nodes share access to a common set of shared
disk drives.
RA8000/ESA12000 Storage Subsystem
The RA8000/ESA12000 Storage Subsystem is the shared storage solution for
the PDC/O5000. Each storage subsystem contains one master enclosure and
up to two expansion enclosures. The master enclosure contains the two
dual-ported HSG80 Array Controllers that connect the RA8000/ESA12000
Storage Subsystem to the cluster's redundant FC-AL paths.
Each enclosure holds up to 24 SCSI disk drives. This allows a maximum
configuration of 72 disk drives in each RA8000/ESA12000 Storage
Subsystem. These disk drives are called Storage Building Blocks (SBBs).
The Compaq StorageWorks Command Console (SWCC) provides a graphical
user interface to set up, configure, monitor, and troubleshoot the
RA8000/ESA12000 Storage Subsystem components.
For detailed information about storage subsystem components, refer to the
StorageWorks documentation provided with the RA8000/ESA12000 Storage
Subsystem.
HSG80 Array Controllers
Two dual-port HSG80 Array Controllers are installed in each
RA8000/ESA12000 Storage Subsystem master enclosure. HSG80 Array
Controllers are not installed in the RA8000/ESA12000 expansion enclosures.
To ensure fault tolerance, each controller is connected to a different storage
hub.
From the perspective of the cluster nodes, each HSG80 Array Controller is
simply another device connected to one of the cluster's two FC-AL (I/O)
paths. Consequently, each node sends its I/O requests to the HSG80 Array
Controller just as it would to any SCSI device. The HSG80 Array Controller
receives the I/O requests from the nodes and directs them to the shared storage
disks to which it has been configured. Because the array controller processes
the I/O requests, the cluster nodes are not burdened with the I/O processing
tasks associated with reading and writing data to multiple shared storage
devices.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
Architecture 2-9
Each HSG80 Array Controller combines the logical disk drives that have been
configured to it into a single, high-performance storage unit called a
storageset. RAID technology ensures that every unpartitioned storageset,
whether it uses 2 disks or 10, looks like a single storage unit to the cluster
nodes.
Both HSG80 Array Controllers in the master enclosure are configured as
active, but access to the same logical unit number (LUN) is restricted to one
HSG80 Array Controller (on one path of the redundant FC-AL) to ensure data
integrity. In this configuration, both ports on each HSG80 Array Controller are
receiving and transmitting data for the same LUNs. This increases the data
throughput for the cluster as compared to a nonredundant Fibre Channel path
configuration.
If the first HSG80 Array Controller in an RA8000/ESA12000 Storage
Subsystem master enclosure fails, the other controller (with the assistance of
Secure Path) will take over the role of the failed controller, in addition to its
own role, and continue operation. The two ports on each HSG80 Array
Controller are connected by Fibre Channel cables to two ports on the same
storage hub, so that both HSG80 Array Controllers are connected to a different
storage hub. Connecting the HSG80 Array Controllers in this manner ensures
two independent FC-AL paths to the shared storage subsystem.
Storage Hubs
The PDC/O5000 uses redundant Fibre Channel Arbitrated Loops (FC-ALs) to
communicate between the cluster nodes and the shared storage subsystems.
One storage hub is used exclusively by each FC-AL path. On each storage
hub, one port is used by each cluster node and one port is used to connect to
one of the two HSG80 Array Controllers installed in each RA8000/ESA12000
Storage Subsystem master enclosure.
The PDC/O5000 allows the use of either the Storage Hub 7 (with 7 ports) or
the Storage Hub 12 (with 12 ports). Using the Storage Hub 7 limits the size of
the PDC/O5000 cluster: four cluster nodes with two RA8000/ESA12000
Storage Subsystem master enclosures cannot be achieved with Storage Hub 7s.
If your cluster will contain more than three nodes or is likely to grow, you are
advised to use Storage Hub 12s. This allows the combined number of cluster
node ports and array controller ports for each FC-AL path to be up to 12.
To ensure no single point of failure, each redundant FC-AL pair in a
PDC/O5000 cluster uses a dedicated pair of storage hubs; one for each FC-AL
path.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
2-10 Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
KGPSA-BC Host Adapters
Each cluster node contains two KGPSA-BC Host Adapters; one for each
FC-AL path. The leftmost (or top) KGPSA-BC Host Adapter in each node is
connected to the same storage hub. The rightmost (or bottom) KGPSA-BC
Host Adapter in each node is connected to the other storage hub.
If the cluster contains multiple redundant FC-AL pairs, each redundant FC-AL
pair must also have its own dedicated pair of KGPSA-BC Host Adapters in
each cluster node.
Secure Path software is installed on each cluster node to ensure the proper
operation of components along each FC-AL path. For information on
installing Secure Path, see "Installing Secure Path for Windows NT" in
Chapter 5, "Installation and Configuration."
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
Architecture 2-11
Fibre Channel Arbitrated Loop Data Paths
Each redundant FC-AL pair in a PDC/O5000 cluster has two distinct data
paths:
One data path runs from the leftmost (or top) KGPSA-BC Host Adapter
s
in each of the nodes to one storage hub, then on to both ports of the top
HSG80 Array Controller in each RA8000/ESA12000 Storage
Subsystem master enclosure.
A second data path runs from the rightmost (or bottom) KGPSA-BC
s
Host Adapter in each of the nodes to the second storage hub, then on to
both ports of the lower HSG80 Array Controller in each
RA8000/ESA12000 Storage Subsystem master enclosure.
Host Adapter-to-Storage Hub Data Paths
Figure 2-4 highlights the two data paths where they run between the
KGPSA-BC Host Adapters and the storage hubs.
KGPSA-BC KGPSA-BC
Host Adapters (2) Host Adapters (2)
ProLiant ProLiant
Server Server
Storage Hub
Storage Hub
HSG80 Ctrl A HSG80 Ctrl A
HSG80 Ctrl B HSG80 Ctrl B
RA8000/ RA8000/
ESA12000 ESA12000
6-24 drives 6-24 drives
Figure 2-4. Host adapter to storage hub data paths
Secure Path monitors the status components along both FC-AL paths. If
Secure Path detects the failure of a KGPSA-BC Host Adapter, Fibre Channel
cable, or storage hub along one path , it automatically transfers all I/O activity
on that path to the other FC-AL path.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
2-12 Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Storage Hub-to-RA8000/ESA12000 Storage
Subsystem Data Path
Figure 2-5 highlights the two data paths where they run between the storage
hubs and the two HSG80 Array Controllers installed in each
RA8000/ESA12000 Storage Subsystem master enclosure.
KGPSA-BC KGPSA-BC
Host Adapters (2) Host Adapters (2)
ProLiant ProLiant
Server Server
Storage Hub
Storage Hub
HSG80 Ctrl A HSG80 Ctrl A
HSG80 Ctrl B HSG80 Ctrl B
RA8000/ RA8000/
ESA12000 ESA12000
6-24 drives 6-24 drives
Figure 2-5. Storage hub to RA8000/ESA12000 Storage Subsystem data paths
If any component along this connection experiences a fault, Secure Path
automatically transfers all I/O activity to the components on the other FC-AL
path, including the storage hub and the other HSG80 Array Controller in the
master enclosure.
SCSI Disks
The RA8000/ESA12000 Storage Subsystem uses standard hot-plug,
Wide-Ultra SCSI disk drives, ensuring investment protection for existing SCSI
users.
Each RA8000/ESA12000 Storage Subsystem master enclosure can connect by
SCSI cables to one or two expansion enclosures. Each enclosure can hold up
to 24 Wide-Ultra SCSI drives.
In a shared disk clustering scheme like the PDC/O5000, the SCSI drives must
be 100 percent compatible with the clustering software. Compaq has worked
diligently to ensure that Compaq SCSI drives meet the stringent needs of
clustering.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
Architecture 2-13
IMPORTANT: Other than the Compaq StorageWorks SCSI drives, do not directly install
any other SCSI drives in the RA8000/ESA12000 Storage Subsystem.
Cluster Interconnect
The cluster interconnect is the data path over which all of the nodes in a
PDC/O5000 cluster communicate.
NOTE: Several terms for cluster interconnect are used throughout the industry. Others
are: private LAN, private interconnect, system area network (SAN), and private network.
Throughout this guide, the term cluster interconnect is used.
The nodes use the cluster interconnect data path to:
Communicate individual resource and overall cluster status
s
Send and receive heartbeat signals
s
Coordinate database locks through the Oracle Integrated Distributed
s
Lock Manager
Redundant Ethernet Cluster Interconnect
The PDC/O5000 uses a redundant Ethernet cluster interconnect. A redundant
cluster interconnect uses redundant hardware to ensure no single point of
failure along the cluster interconnect path.
IMPORTANT: The cluster management software for the Ethernet cluster interconnect
requires the use of TCP/IP. When configuring the Ethernet cluster interconnect, be sure to
enable TCP/IP.
NOTE: Refer to the technical white paper Supported Ethernet Interconnects for Compaq
Parallel Database Clusters Using Oracle Parallel Server for detailed information about
configuring redundant Ethernet cluster interconnects. This document is available at
www.compaq.com/highavailability/
A redundant Ethernet cluster interconnect uses the following component sets:
Two 100-Mbit/sec Ethernet adapters in each cluster node
s
Two 100-Mbit/sec Ethernet switches or hubs for two-node clusters
s
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
2-14 Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Two 100-Mbit/sec Ethernet switches for clusters with three or more
s
nodes
One Ethernet crossover cable installed between the two Ethernet
s
switches or hubs
IMPORTANT: Compaq recommends Service Pack 4 of Windows NT Server for a
redundant Ethernet cluster interconnect or client LAN. However, because certain
applications may not work with Service Pack 4, consult with your software expert to
confirm that your applications can run with Service Pack 4. Using Service Pack 3 requires
installing the approved Microsoft patch (hot fix) article ID Q156655 (Title: Memory Leak
and STOP Screens Using Intermediate NDIS Drivers).
Ethernet Cluster Interconnect Adapters
To implement the Ethernet cluster interconnect, each cluster node must be
equipped with Ethernet adapters capable of 100-Mbit/sec transfer rates. Some
adapters may be capable of both 10-Mbit/sec and 100-Mbit/sec; however,
adapters used for the Ethernet cluster interconnect must run at 100-Mbit/sec.
The Ethernet adapters must have passed Windows NT Server 4.0 hardware
compatibility test (HCT) certification.
NOTE: For recommended dual-port and single-port Ethernet adapters for your cluster,
see the document Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8
Parallel Server Release 8.0.5 Certification Matrix at www.compaq.com/highavailability/
If you are using dual-port Ethernet adapters, you can use one port for the
Ethernet cluster interconnect and the second port for the client LAN. Refer to
the technical white paper Supported Ethernet Interconnects for Compaq
Parallel Database Clusters Using Oracle Parallel Server at:
www.compaq.com/highavailability/
For detailed information about installing a redundant Ethernet cluster
interconnect, see Chapter 5, "Installation and Configuration."
Ethernet Switch
IMPORTANT: The switch or switches used for the Ethernet cluster interconnect must be
dedicated to the cluster interconnect. They cannot be connected to the client LAN or to
cluster nodes that are not part of the PDC/O5000 cluster.
In a two-node PDC/O5000 cluster, two 100-Mbit/sec Ethernet switches or
hubs are connected by cables to the two Ethernet adapters in each node.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
Architecture 2-15
In PDC/O5000 clusters with three or more nodes, two 100-Mbit/sec Ethernet
switches are required for the cluster interconnect path; hubs cannot be used.
The 100-Mbit/sec Ethernet switch handles higher network loads, which is
essential to the uninterrupted operation of the cluster. Ethernet hubs cannot be
used in clusters with three or more nodes.
Figure 2-6 shows the redundant Ethernet cluster interconnect components used
in a two-node PDC/O5000 cluster.
Ethernet Switch/Hub #1
for Cluster Interconnect
Ethernet Switch/Hub #2
Crossover
for Cluster Interconnect
Ca ble
Dual-por t Dual-por t
Ether net Ether net
Adapters (2) Adapters (2)
Client LAN
Hub/Switch #1
Node 1 Node 2
Crossover
Ca ble
Client LAN
Hub/Switch #2
Figure 2-6. Redundant Ethernet cluster interconnect components in a
two-node PDC/O5000 cluster
In the configuration example shown in Figure 2-6, the top port on each
dual-port Ethernet adapter connects by cable to one of the two Ethernet
switches or hubs provided for the cluster interconnect. A crossover cable is
installed between the two Ethernet switches or hubs used for the cluster
interconnect. The bottom port on each adapter connects by Ethernet cable to
the client LAN for the PDC/O5000 cluster.
A crossover cable should be installed between client LAN hubs or switches to
provide fault tolerance in the client LAN path should a failure occur on one of
the hubs or switches.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
2-16 Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Local Area Network
IMPORTANT: For the PDC/O5000, the client LAN and the cluster interconnect are
separate networks. Do not use either network to handle the other network's traffic.
Every client/server application requires a local area network (LAN) over
which client machines and servers communicate. In the case of a cluster, the
hardware components of the client LAN are no different than in a stand-alone
server configuration.
The software components used by network clients should have the ability to
detect node failures and automatically reconnect the client to another cluster
node. For example, Net8, Oracle Call Interface (OCI), and Transaction Process
Monitors can be used to address this issue.
For complete information on how to ensure client auto-reconnect in an
Oracle8 Parallel Server environment, refer to the Oracle white paper Oracle
Parallel Server on Windows NT Clusters: Implementing Client Failover.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: c-ch2 Architecture.doc Last Saved On: 9/8/99 2:58 PM
3
Chapter
Cluster Software Components
The Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8
Release 8.0.5 (PDC/O5000) combines software from several leading computer
vendors. The cluster management software is a combination of Compaq
operating system dependent modules (OSDs) and Oracle8 Enterprise Edition
Release 8.0.5 with the Oracle8 Parallel Server Option Release 8.0.5. The
integration of these components creates a stable cluster management
environment in which the Oracle database can operate.
NOTE: For information about currently supported software revisions for the PDC/O5000,
refer to the Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Parallel
Server Release 8.0.5 Certification Matrix at www.compaq.com/highavailability
The software components include:
Microsoft Windows NT Server 4.0
s
For the PDC/O5000, the only supported operating system is Microsoft
Windows NT Server.
Compaq software
s
G Compaq SmartStart and Support Software
G Compaq System Configuration Utility
G Compaq Insight Manager
G Compaq Options ROMPaq
G Compaq StorageWorks Command Console
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: d-ch3 Cluster Software Components.doc Last Saved On: 9/8/99 2:58 PM
3-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
G Compaq StorageWorks Secure Path for Windows NT
G Compaq Operating System Dependent Modules
Oracle software
s
G Oracle8 Server Release 8.0.5
G Oracle8 Parallel Server Option Release 8.0.5
G Oracle8 Enterprise Manager Release 8.0.5
Microsoft Windows NT Server 4.0
The PDC/O5000 supports only version 4.0 of Microsoft Windows NT Server.
Both Windows NT Server 4.0 Standard Edition and Windows NT Server 4.0
Enterprise Edition are supported. Windows NT Server 4.0 with Service Pack 3
or Service Pack 4 is required.
IMPORTANT: Compaq recommends Service Pack 4 of Windows NT Server for a
redundant Ethernet cluster interconnect or client LAN. However, because certain
applications may not work with Service Pack 4, consult with your software expert to
confirm that your applications can run with Service Pack 4. Using Service Pack 3 requires
installing the approved Microsoft patch (hot fix) article ID Q156655 (Title: Memory Leak
and STOP Screens Using Intermediate NDIS Drivers).
IMPORTANT: The PDC/O5000 does not work in conjunction with Microsoft Cluster
Server. Do not install Microsoft Cluster Server on any of the cluster nodes.
Compaq Software
Compaq offers an extensive set of features and optional tools to support
effective configuration and management of your PDC/O5000:
Compaq SmartStart and Support Software CD
s
Compaq System Configuration Utility
s
Compaq Insight Manager
s
Compaq Options ROMPaq
s
Compaq StorageWorks Command Console
s
Compaq StorageWorks Secure Path for Windows NT
s
Compaq operating system dependent modules (OSDs)
s
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: d-ch3 Cluster Software Components.doc Last Saved On: 9/8/99 2:58 PM
Cluster Software Components 3-3
Compaq SmartStart and Support Software
SmartStart, which is located on the SmartStart and Support Software CD, is
the best way to configure Windows NT Server on a PDC/O5000 cluster.
SmartStart uses a step-by-step process to configure the operating system and
load the system software.
The Compaq SmartStart and Support Software CD also contains device drivers
and utilities that enable you to take advantage of specific capabilities offered
on Compaq products. These drivers are provided for use with Compaq
hardware only.
For information about SmartStart, refer to the Compaq Server Setup and
Management pack.
Compaq System Configuration Utility
The SmartStart and Support Software CD also contains the Compaq System
Configuration Utility. This utility is the primary means to configure hardware
devices within your servers, such as I/O addresses, boot order of disk
controllers, and so on.
For information about the System Configuration Utility, see the Compaq
Server Setup and Management pack.
Compaq Insight Manager
Compaq Insight Manager, loaded from the Compaq Management CD, is a
software utility used to collect information about the servers in the cluster.
Compaq Insight Manager performs these functions:
Monitors server fault conditions and status
s
Forwards server alert fault conditions
s
Remotely controls servers
s
The Integrated Management Log is used to collect and feed data to Compaq
Insight Manager. This log is used with the Compaq Integrated Management
Display (IMD), the optional Remote Insight controller, and SmartStart.
In Compaq servers, each hardware subsystem, such as nonshared disk storage,
system memory, and system processor, has a robust set of management
capabilities. Compaq Full Spectrum Fault Management notifies the end user of
impending fault conditions.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: d-ch3 Cluster Software Components.doc Last Saved On: 9/8/99 2:58 PM
3-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
For information about Compaq Insight Manager, refer to the documentation
you received with your Compaq ProLiant server.
Compaq Options ROMPaq
The Compaq Options ROMPaq diskettes allow a user to upgrade the ROM
Firmware images for Compaq System product options, such as array
controllers, disk drives, and tape drives used for nonshared storage.
Compaq StorageWorks Command Console
The Compaq StorageWorks Command Console (SWCC) is a graphical user
interface (GUI) used to create and manage RA8000/ESA12000 Storage
Subsystem resources in a PDC/O5000 cluster, including RAID
implementation, disk mirroring, and disk partitioning.
The SWCC provides a user-friendly method of executing the Command Line
Interpreter (CLI) commands for the HSG80 Array Controllers. While the CLI
Window provides very detailed control over the storage subsystem, the SWCC
replicates most of the CLI Window functions in a graphic form and provides a
user-friendly method of executing CLI commands.
For details about using SWCC, refer to the documentation provided with the
HSG80 Array Controller.
Compaq StorageWorks Secure Path
for Windows NT
Compaq StorageWorks Secure Path for Windows NT (Secure Path) must be
installed on each server in a PDC/O5000 cluster. Secure Path allows an
RA8000/ESA12000 Storage Subsystem with two HSG80 Array Controllers to
be cabled to both paths of a redundant FC-AL pair, using two separate
KGPSA-BC Host Adapters in each server. If a failure occurs on one path's
KGPSA-BC Host Adapters, Fibre Channel cables, storage hub, or HSG80
Array Controllers, the failure is detected and I/O activity is automatically
rerouted to the second FC-AL path.
For detailed information about installing Secure Path in a PDC/O5000 cluster,
see "Installing Secure Path for Windows NT" in Chapter 5, "Installation and
Configuration." For more detailed information about the Secure Path software,
refer to the online documentation provided on the Secure Path CD.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: d-ch3 Cluster Software Components.doc Last Saved On: 9/8/99 2:58 PM
Cluster Software Components 3-5
Compaq Operating System Dependent Modules
Compaq supplies low-level services that are required by Oracle8 Parallel
Server Release 8.0.5. The OSD layer monitors critical clustering hardware
components, constantly relaying cluster state information to Oracle8 Parallel
Server. Oracle8 Parallel Server monitors this information and takes pertinent
action as needed.
For example, the OSD layer is responsible for monitoring the performance of
each node in the cluster. The OSD layer determines if one of the nodes is no
longer responding to the cluster heartbeat. If the node still does not respond,
the OSD layer determines it is unavailable and communicates this information
to Oracle8 Parallel Server. Oracle8 Parallel Server then evicts the node from
the cluster, recovers the part of the database affected by that node, and
reconfigures the cluster with the remaining nodes.
The OSD software is found on the Compaq Parallel Database Cluster
PDC/O5000 Clustering Software CD, included in the Compaq PDC/O5000
Kit.
For a detailed description of how the OSD layer interacts with Oracle8 Parallel
Server, refer to the Oracle8 Enterprise Edition Getting Started Release 8.0.5
for Windows NT manual.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: d-ch3 Cluster Software Components.doc Last Saved On: 9/8/99 2:58 PM
3-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Oracle Software
NOTE: The PDC/O5000 uses Oracle8 Release 8.0.5. If a release other than 8.0.5 is being
used, visit the Compaq website at www.compaq.com/highavailability/ to confirm that the
release has been certified on the PDC/O5000.
Oracle8 Server Enterprise Edition Release 8.0.5
The Oracle8 Server Enterprise Edition Release 8.0.5 provides the following:
Oracle8 Server Release 8.0.5
s
Oracle8 Parallel Server Option Release 8.0.5
s
Oracle8 Enterprise Manager Release 8.0.5
s
Oracle8 Server Release 8.0.5
Oracle8 Server Release 8.0.5 is the database application software and must be
installed on each node in the PDC/O5000 cluster. Refer to the documentation
for the Oracle8 Server Release 8.0.5 for additional information.
Oracle8 Parallel Server Option Release 8.0.5
Oracle8 Parallel Server Option Release 8.0.5 is the key component in the
Oracle8 clustering architecture. Oracle8 Parallel Server allows the database
server to divide its workload among the physical cluster nodes. This is
accomplished by running a distinct instance of Oracle8 Server on each node in
the PDC/O5000 cluster.
Oracle8 Parallel Server manages the interaction between these instances.
Through its Integrated Distributed Lock Manager, Oracle8 Parallel Server
manages the ownership of database records that are requested by multiple
instances.
At a lower level, Oracle8 Parallel Server monitors cluster membership. It
interacts with the OSDs, exchanging information about the state of each
cluster node.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: d-ch3 Cluster Software Components.doc Last Saved On: 9/8/99 2:58 PM
Cluster Software Components 3-7
For additional information, refer to:
Oracle8 Enterprise Edition Getting Started Release 8.0.5 for
s
Windows NT
Other Oracle documentation for Oracle8 Server Release 8.0.5 and
s
Oracle 8 Parallel Server Release 8.0.5
Oracle8 Enterprise Manager Release 8.0.5
Oracle8 Enterprise Manager Release 8.0.5 is responsible for monitoring the
state of both the database entities and the cluster members. It primarily
manages the software components of the cluster. Hardware components are
managed with Compaq Insight Manager.
Do not install Oracle8 Enterprise Manager on any of the PDC/O5000 cluster
nodes. It must be installed on a separate server that is running Oracle8
Release 8.0.5 and has network access to the cluster nodes. Before installing
Oracle8 Enterprise Manager, read its documentation to ensure it is installed
and configured correctly for an Oracle8 Parallel Server environment.
Client Reconnect Software
When a network client computer operates in a clustered environment, it must
be more resilient than when operating with a stand-alone server. Because a
client can access the database through any of the cluster nodes, the failure of
its current node does not prevent the client from reattaching to the cluster and
continuing its work.
Several software packages can be used to automatically reconnect a client in
the event of a node failure. The software is responsible for detecting when the
client's cluster node is no longer available and then connecting the client to
one of the remaining, operational cluster nodes. For example, Oracle Net8 and
Oracle Call Interface (OCI) can address this issue.
NOTE: For complete information on how to ensure client auto-reconnect in an Oracle8
Parallel Server environment, contact your Oracle representative.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: d-ch3 Cluster Software Components.doc Last Saved On: 9/8/99 2:58 PM
3-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Oracle Certification
To ensure that Oracle8 Parallel Server Release 8.0.5 is used in a compatible
hardware environment, Oracle has established a certification process, which is
a series of test scripts designed to stress an Oracle8 Parallel Server
implementation and verify stability and full functionality.
All hardware providers who choose to deliver platforms for use with Oracle8
Parallel Server Release 8.0.5 must demonstrate the successful completion of
the Oracle8 Parallel Server for Windows NT Certification. Neither Oracle nor
Compaq will support any implementation of Oracle8 Parallel Server that does
not strictly conform to the configurations certified with this process. For a
complete list of certified Compaq servers, see the Compaq Parallel Database
Cluster Model PDC/O5000 for Oracle8 Parallel Server Release 8.0.5
Certification Matrix at
www.compaq.com/highavailability/
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: d-ch3 Cluster Software Components.doc Last Saved On: 9/8/99 2:58 PM
4
Chapter
Planning
Before connecting any cables or powering up any hardware on your Compaq
Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5
(PDC/O5000), it is important that you understand how all the various cluster
components fit together to meet your operational requirements. The major
topics discussed in this chapter are:
Site planning
s
Capacity planning for cluster hardware
s
Planning the cluster configuration
s
RAID planning
s
Planning the grouping of physical storage space
s
Disk drive planning
s
Network planning
s
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
4-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Site Planning
You must carefully select and prepare the site to ensure a smooth installation
and a safe and efficient work environment. To select and prepare a location for
your cluster, consider the following:
The path from the receiving dock to the installation area
s
Availability of appropriate equipment and qualified personnel
s
Space for unpacking, installing, and servicing the computer equipment
s
Sufficient floor strength for the computer equipment
s
Cabling requirements, including the placement of network and Fibre
s
Channel cables within one room (under the subfloor, on the floor, or
overhead) and possibly between rooms
Client LAN resource planning, including the number of hubs or
s
switches and cables to connect to the cluster nodes
Environmental conditions, including temperature, humidity, and air
s
quality
Power, including voltage, current, grounding, noise, outlet type, and
s
equipment proximity
IMPORTANT: Carefully review the power requirements for your cluster components to
identify special electrical supply needs in advance (for example, the 220 VAC source
power required for the PDU in the ESA12000 cabinet).
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
Planning 4-3
Capacity Planning for Cluster Hardware
Capacity planning determines how much computer hardware is needed to
support the applications and data on your clustered servers. Given the size of
your database and the performance you expect, you must decide how many
servers and storage arrays the cluster needs.
Compaq ProLiant Servers
To determine the number of servers you install in a PDC/O5000 cluster, take
into account the levels of availability and scalability your site requires. Plan
your cluster so that the failure of a single node will not adversely impact
cluster operations. For example, when running a two-node cluster, the failure
of one node leaves the one remaining node to service all clients. This could
result in an unacceptable level of performance.
Within each server, the appropriate number and speed of the CPUs and
memory size are all determined by several factors. These include the types of
database applications being used and the number of clients connecting to the
servers.
IMPORTANT: All servers in a PDC/O5000 cluster must be the same model type. All the
servers must be identically configured; the same adapter card must be in the same slot in
each server, the amount of memory identical, the number of CPUs identical, and so on.
NOTE: For an up-to-date list of Compaq Parallel Database Cluster-certified servers for the
PDC/O5000 and detailed information about minimum and maximum cluster node
configurations, refer to the Compaq Parallel Database Cluster Model PDC/O5000 for
Oracle8 Parallel Server Release 8.0.5 Certification Matrix. This document is available on
the Compaq website at www.compaq.com/highavailability/
Planning Shared Storage Components
Several key components make up the shared storage subsystem for the
PDC/O5000. Each redundant FC-AL pair in a PDC/O5000 cluster provides
two FC-AL paths. Collectively, the following cluster components are used by
these two FC-AL paths:
Two KGPSA-BC Host Adapters in each server (one adapter for each
s
path)
Two storage hubs (each of which connects all of the cluster nodes to one
s
FC-AL path)
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
4-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
One or two sets of RA8000/ESA12000 Storage Subsystem enclosures.
s
Each set consists of a master enclosure and up to two expansion
enclosures. Each RA8000/ESA12000 Storage Subsystem enclosure
holds up to 24 disk drives.
Two dual-port HSG80 Array Controllers installed in each
s
RA8000/ESA12000 Storage Subsystem master enclosure. Each
controller is connected to a different storage hub.
NOTE: For more information about redundant Fibre Channel Arbitrated Loops (FC-ALs) in
a PDC/O5000 cluster, see Chapter 2, "Architecture."
Use Table 4-1 to identify which storage hubs to use for each redundant FC-AL
in your PDC/O5000 cluster.
Table 4-1
Storage Hub Selection Matrix
Number of Nodes Number of Number of Recommended
in the Cluster RA8000/ESA12000 Ports Used on Storage Hubs
Expansion Cabinets each Storage
Hub
2 1 4 Storage Hub 7s
2 2 6 Storage Hub 7s
3 1 5 Storage Hub 7s
3 2 7 Storage Hub 7s
4 1 6 Storage Hub 7s
4 2 8 Storage Hub 12s
As Table 4-1 shows, you can use two Storage Hub 7s for every PDC/O5000
cluster configuration except one: a four-node cluster with two HSG80 Array
Controllers in each of two RA8000/ESA12000 Storage Subsystem master
enclosures.
The number of RA8000/ESA12000 Storage Subsystem enclosures and SBBs
used in a PDC/O5000 cluster depends on the following:
The amount of shared storage space required by the database
s
The hardware RAID levels used on the shared storage disks
s
The number and storage capacity of disk drives installed in the
s
enclosures
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
Planning 4-5
NOTE: For improved I/O performance and cluster integrity, as you increase the number of
nodes in a PDC/O5000 cluster, you should also increase the shared storage by adding
more or higher-capacity disk drives.
Planning Cluster Interconnect and Client LAN
Components
Planning a Redundant Ethernet Cluster
Interconnect
IMPORTANT: Compaq recommends Service Pack 4 of Windows NT Server for a
redundant Ethernet cluster interconnect or client LAN. However, because certain
applications may not work with Service Pack 4, consult with your software expert to
confirm that your applications can run with Service Pack 4. Using Service Pack 3 requires
installing the approved Microsoft patch (hot fix) article ID Q156655 (Title: Memory Leak
and STOP Screens Using Intermediate NDIS Drivers).
NOTE: Refer to the technical white paper Supported Ethernet Interconnects for Compaq
Parallel Database Clusters Using Oracle Parallel Server for detailed information about
configuring redundant Ethernet cluster interconnects. This document is available at
www.compaq.com/highavailability/
When installing a redundant Ethernet cluster interconnect in a PDC/O5000
cluster, review these planning considerations:
Whether to use two Ethernet switches or two Ethernet hubs for the
s
cluster interconnect. If your cluster will contain or grow to three or more
nodes, you must use two Ethernet switches.
Whether to use two dual-port Ethernet adapters in each node that will
s
connect to both the cluster interconnect and the client LAN or to use
separate single-port adapters for the Ethernet cluster interconnect and
the client LAN.
NOTE: For information about recommended dual-port and single-port Ethernet adapters
for your redundant Ethernet cluster interconnect, see the document Compaq Parallel
Database Cluster Model PDC/O5000 for Oracle8 Parallel Server Release 8.0.5 Certification
Matrix. This document is available at www.compaq.com/highavailability/
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
4-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Planning the Client LAN
Every client/server application requires a local area network (LAN) over
which client machines and servers communicate. In the case of a cluster, the
hardware components of the client LAN are no different than in a stand-alone
server configuration.
Reference Material for Hardware Sizing
For information about hardware sizing for online transaction processing
(OLTP) databases, see the guidelines in the Oracle white paper Sizing Compaq
ProLiant Servers for Oracle OLTP Applications, May 1997. You can find this
white paper at:
www.compaq.com
Planning the Cluster Configuration
IMPORTANT: Use the Oracle documentation to obtain detailed information about planning
for the Oracle software. Once the required level of performance, the size of the database,
and the type of database have been determined, use this Oracle documentation to
continue the planning of the cluster's physical components.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
Planning 4-7
Sample Small Configuration for the PDC/O5000
Figure 4-1 shows an example of a small PDC/O5000 cluster: a two-node
cluster with one RA8000/ESA12000 Storage Subsystem master enclosure.
Client LAN
Redundant Cluster Interconnect
KGPSA-BC KGPSA-BC
Host Adapters (2) Host Adapters (2)
ProLiant ProLiant
Server Server
Storage Hub
Storage Hub
HSG80 Ctrl A
HSG80 Ctrl B
RA8000/
ESA12000
6-24 drives
Figure 4-1. Two-node PDC/O5000 cluster with one RA8000/ESA12000
Storage Subsystem enclosure
The sample small configuration shown in Figure 4-1 contains these major
cluster components:
Two ProLiant servers (cluster nodes)
s
Two KGPSA-BC Host Adapters in each server
s
Two storage hubs
s
One RA8000/ESA12000 Storage Subsystem master enclosure. The
s
enclosure contains up to 24 disk drives.
Two dual-port HSG80 Array Controllers installed in the
s
RA8000/ESA12000 Storage Subsystem master enclosure. The two ports
on each controller are connected to the same storage hub.
Redundant Ethernet NIC adapters, cables, and Ethernet switches or hubs
s
for the Ethernet cluster interconnect
Ethernet NIC adapters, switches or hubs (not shown), and cables for the
s
client LAN
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
4-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Sample Large Configuration for the PDC/O5000
Figure 4-2 shows an example of a large PDC/O5000 cluster: a four-node
cluster with six fully-populated RA8000/ESA12000 Storage Subsystem
enclosures.
Client LAN
ProLiant ProLiant
KGPSA-BC
Servers Servers
Host Adapters (2)
KGPSA-BC
Host Adapters (2)
Storage Hub 12
Storage Hub 12
HSG80 Ctrl A HSG80 Ctrl A
HSG80 Ctrl B HSG80 Ctrl B
RA8000/ RA8000/
ESA12000 ESA12000
(Master) (Master)
24 drives
24 drives
24 drives
24 drives
24 drives
24 drives
Figure 4-2. Four-node PDC/O5000 cluster with six RA8000/ESA12000
Storage Subsystem enclosures
The sample large configuration shown in Figure 4-2 contains these major
cluster components:
Four ProLiant servers (cluster nodes)
s
Two KGPSA-BC Host Adapters in each server
s
Two storage hub 12s
s
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
Planning 4-9
Two sets of RA8000/ESA12000 Storage Subsystem enclosures. In this
s
sample configuration, each set consists of one master enclosure and two
expansion enclosures. Each RA8000/ESA12000 enclosure contains up
to 24 disk drives, providing a total of 144 disk drives in the cluster
(6 enclosures with 24 disks each).
Two dual-port HSG80 Array Controllers installed in each
s
RA8000/ESA12000 Storage Subsystem master enclosure. Both ports on
each controller are connected to the same storage hub.
Cluster interconnect hardware: redundant Ethernet NIC adapters, cables,
s
and Ethernet switches for the Ethernet cluster interconnect
Ethernet NIC adapters, switches or hubs (not shown), and cables for the
s
client LAN
RAID Planning
Storage subsystem performance is one of the most important aspects of tuning
database cluster servers for optimal performance. Efforts to plan, configure,
and tune a PDC/O5000 cluster should focus on getting the most out of each
disk drive and having an appropriate number of shared storage drives in the
cluster. When properly configured, the I/O subsystem should not be the
limiting factor in overall cluster performance.
RAID technology provides cluster servers with more consistent performance,
higher levels of fault- tolerance, and easier fault recovery than non-RAID
systems. RAID uses redundant information stored on different disks to ensure
that the cluster can survive the loss of any disk in the array without affecting
the availability of data to users.
RAID also uses the technique of striping, which involves partitioning each
drive's storage space into units ranging from a sector (512 bytes) up to several
megabytes. The stripes of all the disks are interleaved and addressed in order.
In a PDC/O5000 cluster, each node is connected to shared storage disk drive
arrays housed in RA8000/ESA12000 Storage Subsystem enclosures. The
maximum number of arrays in one cluster depends on several factors,
including the number of ports available on the storage hubs, the number of
nodes in the cluster, and the number of redundant FC-AL paths installed in the
cluster. Refer to "Planning Shared Storage Components" in this chapter for
more information.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
4-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
The PDC/O5000 implements RAID at the hardware level, which is faster than
software RAID. When you implement RAID on shared storage arrays, you use
the hardware RAID to perform such functions as making copies of the data or
calculating checksums. To implement the RAID sets, use the Compaq
StorageWorks Command Console (SWCC) or the Command Line Interpreter
(CLI) Window for the HSG80 Array Controllers.
IMPORTANT: Do not use the software RAID offered by the operating system to configure
your shared storage disks.
Supported RAID Levels
RAID provides several fault- tolerant options to protect your cluster's shared
data. However, each RAID level offers a different mix of performance,
reliability, and cost.
The PDC/O5000 supports these RAID levels:
RAID 0
s
RAID 1
s
RAID 0+1
s
RAID 3/5
s
NOTE: RAID 0 does not provide the fault- tolerance feature of other RAID levels.
For RAID level definitions and information about configuring hardware
RAID, refer to the following:
Refer to the information about RAID configuration contained in the
s
StorageWorks documentation provided with the HSG80 Array
Controller.
Refer to the Compaq white paper Configuring Compaq RAID
s
Technology for Database Servers, #ECG 011/0598, available at the
Compaq website at
www.compaq.com
Refer to the various white papers on Oracle8, which are available at the
s
Compaq ActiveAnswers website at
www.compaq.com/activeanswers/
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
Planning 4-11
Raw Data Storage and Database Size
Raw data storage is the amount of storage available before any RAID levels
have been configured. It is called raw storage because RAID volumes require
some overhead. The maximum size of a database stored in a RAID system will
always be less than the amount of raw data storage available, except for
RAID 0, where no storage overhead is required.
To calculate the amount of raw data storage in a cluster, multiply the capacity
of a particular RA8000/ESA12000 by the number of storage subsystems that
you have.
The amount of disk space required for a given database size may be affected
by the RAID levels you select and the overhead required for indexes, I/O
buffers, and logs.
Selecting RAID Levels
Many factors will affect the RAID levels you select for your cluster database.
These include the specific availability, performance, reliability, and recovery
capabilities required from the database. Each cluster must be evaluated
individually by qualified personnel.
The following general guidelines apply to RAID selection for a PDC/O5000
cluster using Oracle8 Parallel Server:
Oracle recommends that some form of disk fault-tolerance be
s
implemented in the cluster.
In order to ease the difficulty of managing dynamic space allocation in
s
an Oracle Parallel Server (OPS) raw volume environment, Oracle
recommends the creation of "spare" raw volumes that can be used to
dynamically extend tablespaces when the existing datafiles approach
capacity. The number of these spare raw volumes should represent from
10 to 30 percent of the total database size. To allow for effective load
balancing, the spares should be spread across a number of disks and
controllers. The database administrator should decide, on a case by case
basis, which spare volume to use based on which volume would have
the least impact on scalability (for both speedup and scaleup).
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
4-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Planning the Grouping of Physical Disk
Storage Space
Figure 4-3 shows an example of how the physical storage space in one
six-drive array in an RA8000/ESA12000 Storage Subsystem enclosure is
grouped for an Oracle8 Parallel Server database.
RA8000/ESA12000
Disk Storage Array
Disk Disk Disk Disk Disk Disk
Drive Drive Drive Drive Drive Drive
Create RAIDset
RAIDset
with CLI or SWCC
Create a logical drive
RAID Logical Drive (Virtual Disk)
with CLI or SWCC
Create one extended
par tition with Extended Partition
NT Disk Administrator
...
Create logical
...
par titions with
NT Disk Administrator
Figure 4-3. RA8000/ESA12000 Storage Subsystem disk grouping for a
PDC/O5000 cluster
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
Planning 4-13
The RA8000/ESA12000 Storage Subsystem enclosure contains up to four
rows of six drives each. You are advised to configure each six-drive row as a
single, separate virtual disk. As Figure 4-3 shows, this requires that you create
a RAID set, create one logical drive, create one extended partition, and then
create logical partitions. For detailed information about these procedures, refer
to "Configuring a Storageset" and "Creating Partitions" in Chapter 5,
"Installation and Configuration."
You use either the CLI Window for the HSG80 Array Controllers or the
Compaq StorageWorks Command Console (SWCC) to create, configure, and
partition the logical disks. While the CLI Window provides very detailed
control over the RA8000/ESA12000 Storage Subsystem, the SWCC replicates
most of the CLI Window functions in a graphic form and provides a more
user-friendly tool for executing CLI commands.
IMPORTANT: To perform partitioning, all required drivers must already be installed for
each server. These include the Emulex (LP6NDS35. SYS) driver for the KGPSA-BC Host
Adapters, the HSG80 Array Controller (HSZDISK.SYS) driver, and the Secure Path
(RAIDISK.SYS) driver.
Disk Drive Planning
NonShared Disk Drives
Nonshared disk drives, or local storage, operate the same way in a cluster as
they do in a single-server environment. These drives can be in the server drive
bays or in an external storage enclosure. As long as they are not accessible by
multiple servers, they are considered nonshared.
Treat nonshared drives in a clustered environment as you would in a
nonclustered environment. In most cases, some form of RAID is used to
protect the drives and aid in restoration of a failed drive. Since the Oracle
Parallel Server application files are stored on these drives, it is recommended
that you use hardware RAID.
Hardware RAID is the recommended solution for RAID configuration because
of its superior performance. For the PDC/O5000, hardware RAID for the
RA8000/ESA12000 Storage Subsystem can be implemented using SWCC or
the CLI Window for the HSG80 Array Controller.
Compaq Confidential Need to Know Required
Writer: EGardner Project: Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide Comments:
Part Number: 153668-001 File Name: e-ch4 Planning.doc Last Saved On: 9/8/99 3:00 PM
4-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8 Release 8.0.5 Administrator Guide
Shared Disk Drives
Shared disk drives are contained in the disk arrays that are accessible to each
node in a cluster. It is recommended that you use hardware RAID levels 0, 1,
0+1, or 3/5 on your shared disk drives.
If a logical drive is configured with a RAID level that does not support
fault-tolerance (for example, RAID 0), then the failure of the shared disk
drives in that logical drive will disrupt service to all Oracle databases that are
dependent on that disk drive. See "Selecting RAID Levels" earlier in this
chapter.
The HSG80 Array Controller is equipped with the RAID capability. The
HSG80 Array Controller RAID is configured with RAID storageset
commands issued from the CLI Window or from the SWCC.
The HSG80 Array Controller ACS monitors the status of the shared disk
drives. If it detects a drive failure, it puts that drive in the failed set. From the
StorageWorks CLI or SWCC GUI, you can monitor the status of shared
storage disk drives and failed sets. A failed drive is displayed from the SWCC
GUI with an "X" marked over it. Refer to "Replacing a Failed Drive in a
Storage Subsystem" in Chapter 6, "Cluster Management," for further
information.
Network Planning
Windows NT Server Hosts Files for the Ethernet
Cluster Interconnect
For the redundant Ethernet cluster interconnect, the Compaq operating system
dependent modules (OSDs) require a unique entry in the hosts and lmhosts
files located at %SystemRoot%\system32\drivers\etc for each network port on
each node.
Each node needs to be identified by the IP address assigned to the Ethernet
adapter port used by the Ethernet cluster interconnect and by the IP address
| 153668-001
|
