Dolphin Express DX Installation and Reference Guide

For users of SISCI, MySQL, Oracle RAC, - Linux version

Dolphin Interconnect Solutions ASA

This document describes the installation and usage of the Dolphin Express software stack version 3.6.x in combination with Dolphin DX interconnect hardware.

Published under Gnu Public License v2

December 21st, 2010


Table of Contents

Abstract
1. Introduction & Overview
1. Who needs Dolphin Express and SuperSockets?
2. How do Dolphin Express and SuperSockets work?
3. What's Special about Dolphin Express and SuperSockets?
4. Contents of this Document
5. Terminology
6. Contact & Feedback: Dolphin Support
2. Quick Installation Guide
3. Requirements and Planning
1. Supported Platforms
1.1. Hardware
1.1.1. Dolphin Express DX Hardware
1.1.2. Supported Platforms
1.1.3. Recommended Node Hardware
1.1.4. Recommended Frontend Hardware
1.2. Software Support
1.2.1. Linux
1.2.2. Windows
1.2.3. Solaris
2. Interconnect Planning
2.1. Nodes to Equip with Dolphin Express Interconnect
2.1.1. MySQL Server and MySQL Cluster
2.1.2. Oracle RAC
2.2. Dolphin DX Interconnect Topology
2.3. Physical Node Placement
4. Initial Installation
1. Installation Overview
1.1. Requirements
1.2. Installation Variants
1.2.1. Live Installation
1.2.2. Non-GUI Installation
1.3.
1.4. Installing the Front-end software
1.5. Install the compute node software
1.6. Installation Result
2. Dolphin DX Adapter Card Installation
3. Software and Cable Installation
3.1. Overview
3.2. Starting the Software Installation
3.3. Post installation
3.4. Working with the Dolphin Network Configurator, dis_netconfig
3.4.1. Cluster Edit
3.4.2. Node Arrangement
3.4.3. Cabling Instructions
3.5. Cluster Cabling
3.5.1. Connecting the Dolphin DX cables
3.5.2. Verifying the Cabling
3.6. Finalising the Software Installation
3.6.1. Static Connectivity Test
3.6.2. SuperSockets Configuration Test
3.6.3. SuperSockets Performance Test
3.7. Handling Installation Problems
3.8. Interconnect Validation using the management GUI
3.8.1. Installing dxadmin
3.8.2. Starting dxadmin
3.8.3. Cluster Overview
3.8.4. Cabling Correctness Test
3.8.5. Fabric Quality Test
3.9. Making Cluster Application use Dolphin Express
3.9.1. Generic Socket Applications
3.9.2. Kernel Socket Services
3.9.3. Native SISCI Applications
5. Update Installation
1. Complete Update
2. Rolling Update
6. Manual Installation
1. Installation under Load
2. Installation of a Heterogeneous Cluster
3. Manual RPM Installation
3.1. RPM Package Structure
3.2. RPM Build and Installation
4. Unpackaged Installation
7. Interconnect Maintenance
1. Verifying Functionality and Performance
1.1. Availability of Drivers and Services
1.2. Cable Connection Test
1.3. Static Interconnect Test
1.4. Interconnect Load Test
1.4.1. Test Execution from sciadmin GUI
1.4.2. Test Execution from Command Line
1.5. Interconnect Performance Test
2. Replacing Interconnect Cables
3. Replacing an Adapter
4. Physically Moving Nodes
5. Replacing a Node
6. Adding Nodes
7. Removing Nodes
8. SISCI API
1. Enable applications to use the SISCI API
9. SuperSockets
1. Make Generic Applications use SuperSockets
1.1. Launch via Wrapper Script
1.2. Launch with LD_PRELOAD
2. SuperSockets Functionality and Performance
2.1. SuperSockets Status
2.2. SuperSockets Functionality
2.2.1. latency_bench
2.2.2. sockperf
2.2.3. latencyplot
3. Troubleshooting
4. SuperSockets Utilization
5. Kernel Socket Services
6. Command Reference
10. MySQL Operation
1. MySQL Cluster
1.1. SuperSockets Poll Optimization
1.2. NDBD Deadlock Timeout
1.3. SCI Transporter
2. MySQL Replication
11. Oracle RAC Operation
1. Oracle RAC
2. Installation
2.1. Oracle
2.2. Dolphin SuperSockets
2.3. Enabling Oracle Booster
3. Known issues
12. Advanced Topics
1. Notification on Interconnect Status Changes
1.1. Interconnect Status
1.2. Notification Interface
1.3. Setting Up and Controlling Notification
1.3.1. Configure Notification via the dis_netconfig
1.3.2. Configure Notification Manually
1.3.3. Verifying Notification
1.3.4. Disabling and Enabling Notification Temporarily
2. High-Availability Features
3. Managing IRM Resources
3.1. Updates with Modified IRM Configuration
4. Using dxdiag
13. FAQ
1. Dolphin DX Hardware
2. Software
A. Self-Installing Archive (SIA) Reference
1. SIA Operating Modes
1.1. Full Cluster Installation
1.2. Node Installation
1.3. Frontend Installation
1.4. Installation of Configuration File Editor
1.5. Building RPM Packages Only
1.6. Extraction of Source Archive
2. SIA Options
2.1. Node Specification
2.2. Installation Path Specification
2.3. Installing from Binary RPMs
2.4. Preallocation of Memory - Communication Buffers
2.5. Enforce Installation
2.6. Configuration File Specification
2.7. Batch Mode
2.8. Non-GUI Build Mode
2.9. Software Removal
B. dxadmin Reference
1. Startup
2. Interconnect Status View
2.1. Icons
2.2. Operation
2.2.1. Cluster Status
2.2.2. Node Status
3. Node and Interconnect Control
3.1. Admin Menu
3.2. Cluster Menu
3.3. Node Menu
3.4. Cluster Settings
3.5. Adapter Settings
4. Interconnect Testing & Diagnosis
4.1. Cable Test
4.2. Fabric Test
5. Troubleshooting Best Practice
C. Configuration Files
1. Cluster Configuration
1.1. dishosts.conf
1.1.1. Basic settings
1.1.2. SuperSockets settings
1.1.3. Miscellaneous Notes
1.2. networkmanager.conf
1.3. cluster.conf
2. SuperSockets Configuration
2.1. supersockets_profiles.conf
2.2. supersockets_ports.conf
3. Driver Configuration
3.1. dis_irm.conf
3.1.1. Resource Limitations
3.1.2. Real time behavior
3.1.3. Memory Preallocation
3.1.4. Logging and Messages
3.2. dis_ssocks.conf
D. Platform Issues and Software Limitations
1. Platforms with Known Problems vs Dolphin DX
2. SuperSockets
E. DX Firmware upgrade

List of Figures

4.1. Cluster Edit dialog of dis_netconfig
4.2. Main dialog of dis_netconfig
4.3. Node dialog of dis_netconfig
4.4. DX x8 connection
4.5. dxadmin, connect
9.1. Working 2 node setup
9.2. latencyplot for 4 byte messages
9.3. latencyplot compare with Ethernet
9.4. latencyplot reload
B.1. Fabric is UP
B.2. Fabric is DEGRADED
B.3. Fabric is REDUCED
B.4. Fabric has FAILED due to loss of connectivity
B.5. Fabric has FAILED due to dead nodes
B.6. Options in the Admin menu
B.7. Options in the Cluster menu
B.8. Options in the Node menu
B.9. Cluster configuration in dxadmin
B.10. Advanced settings for a node
B.11. Result of running cable test on a good cluster
B.12. Result of cable test on a problematic cluster
B.13. Result of fabric test without installing all the necessary rpms
B.14. Result of fabric test on a proper fabric

List of Tables

3.1. Dolphin DX cables
B.1. Node or Adapter State
B.2. Link State