3. Node and Interconnect Control

3.1. Admin Menu

The items in the Admin menu specifies information that are relevant for the Dolphin Admin GUI

Figure B.5. Options in the Admin menu

Options in the Admin menu

  • Connect to the network manager running on the local or a remote machine.

  • Disconnect from the network manager.

  • Refresh Status of the node and interconnect (instead of waiting for the update interval to expire).

  • Switch to Debug Statistics View will show the value of selected counters of each adapter instead of the node icons which is useful for debugging fabric problems.

3.2. Cluster Menu

The commands in the cluster menu are executed on all nodes in parallel and the results are displayed by sciadmin. When choosing one of the fabric options the command will be executed on all nodes in that fabric.

Figure B.6. Options in the Cluster menu

Options in the Cluster menu

Each fabric in the cluster has a sub-menu Fabric <X>. Within this sub-menu, the Diag (-V 0), Diag (-V 1), Diag (-V 9) are diagnostics functions that can be used to get more detailed information about a fabric that shows problem symptoms.

  • Diag (-V 0) prints only errors that have been found.

  • Diag (-V 1) prints more verbose status information (verbosity level 1).

  • Diag (-V 9) prints the full diagnostic information including all error counters (verbosity level 9).

  • Diag -clear clears all the error counters in the Dolphin Express interconnect adapters. This helps to observe if error counters are changing.

  • Diag -prod prints production information about the Dolphin Express interconnect adapters (serial number, card type, firmware revision etc)

  • The Test option is described in Chapter 4, Initial Installation, Section 4.2, “Fabric Test”

The other commands in the Cluster menu are:

  • Settings displays the Cluster Settings dialog (see below).

  • Reboot cluster nodes reboot all cluster nodes after a confirmation.

  • Power down cluster nodes powers down all cluster nodes after a confirmation.

  • Toggle Network Manager Verbose Settings to increase/decrease the amount of logging from the Dolphin Network Manager.

  • Ethernet to test the quality of your ethernet connections in the cluster.

  • Select the Arrange Fabrics option to make sure that the different adapters in your hosts are connected to the same fabric. This option is only displayed for clusters with more than one fabric.

  • Test Cable Connections is described in Chapter 4, Initial Installation, Section 4.1, “Cable Test”t

3.3. Node Menu

The options in the Node menu are identical to the options in the Cluster and Cluster Fabrics <X> menu, only that commands are executed on the selected node only. The only additional option is Settings that is described in the Section 3.5, “Adapter Settings”.

Figure B.7. Options in the Node menu

Options in the Node menu

3.4. Cluster Settings

The Dolphin Interconnect Manager provides you with several options on how to run the cluster.

Figure B.8. Cluster configuration in sciadmin

Cluster configuration in sciadmin

  • Check Interval Admin alters the number of seconds between each time the Network Manager sends updates to the <Filename>Sciadmin</Filename> GUI.

  • Check Interval Network Manager alters the number of seconds between each time the Network Manager receives updates from the Node Managers.

  • Topology specifies that topology that you configured the cluster in, while Topology found displays the auto-determined topology. Changes to the topology setting can be performed with dishostseditor.

  • Auto Rerouting lets you decide to enable automatic fail over recovery (On), choose to freeze the routing to a current state (Off), or use the default routing tables in the driver (Default), the latter also means that no automatic rerouting will take place.

  • Nodes in X,Y,Z dimension shows how the interconnect is currently dimensioned. Changes to the dimension settings can be performed with dishostseditor.

  • Remove Session to dead nodes lets you decide whether to remove the session to nodes that are unavailable.

  • Wait before removing session defines the number of seconds to wait until removing sessions with a node that has died or became inaccessible by other means.

  • Automatic Create Sessions to new nodes lets you decide if the Network Manager shall create sessions to all available nodes.

  • Alert script lets you choose to enable/disable the use of a script that may alert the cluster status to an administrator.

  • IRM Driver lets you choose to enable/disable the IRM driver.

3.5. Adapter Settings

The Advanced Settings button in the node menu allows you to retrieve more detailed information about an adapter and to disable/enable links of this adapter.

Figure B.9. Advanced settings for a node

Advanced settings for a node


  • Link Frequency sets the frequency of a link. It is not recommended to change the default setting.

  • Prefetch Memsize shows the maximum amount of remote memory that can be accessed by this node.

    A changed value will not become effective until the IRM driver is restarted on the node, which has to be done outside of sciadmin. Setting this value too high (> 512MB) can cause problem with some machines, especially for 32bit platforms.

  • SCI LINK 0 / 1 / 2 allows to set the way a link is controlled:

    • Automatic lets the network manager control the link to enable and disable it as required by the link and the interconnect status.

    • Disabled forces a link down. This is a per-session setting (the link will be under control of the network manager if it is restarted), and only required as a temporary measure for trouble shooting.

      The disable link option can also be used as a temporary measure to disable an unstable adapter or ringlet so that it does not impose unnecessary noise on the adapters. If such an unlikely event occurs, please contact Dolphin support.

      A manually disabled link is marked blue in the sciadmin interconnect display, as shown in the screenshot below.

    Warning

    Please note that when Auto Rerouting is enabled (default setting), disabling a link within a ringlet will disable the complete ringlet. Disabling to many links can thus isolate nodes from access to the Dolphin Express interconnect.

Figure B.10. Link disabled by istrator (Disabling the links on the machine with hostname tiger-5 takes down the corresponding links on the other machines that share the same ringlet.).

Link disabled by istrator (Disabling the links on the machine with hostname tiger-5 takes down the corresponding links on the other machines that share the same ringlet.).