Dell DX6004S DX Content Router Setup and Configuration Guide
Dell DX6004S Manual
View all Dell DX6004S manuals
Add to My Manuals
Save this manual to your list of manuals |
Dell DX6004S manual content summary:
- Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 1
Content Router Setup and Configuration Guide Version 2.2 - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 2
Content Router Setup and Configuration Guide: Version 2.2 Copyright © 2010 Caringo, Inc. All Rights Reserved. No part of this document may be reproduced, transmitted, or transcribed without the written consent of Caringo, Inc. - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 3
Content Router Services 4 3.1. Basic Architecture ...4 3.1.1. Structure of a DX Content Router Node 5 3.1.2. Publisher Service 7 3.1.3. Replicator Service 10 22 5.5.3. Request for Source Cluster IP Addresses 23 6. Support for Content Restoration and Fail-Over 24 6.1. Administrative Disaster - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 4
B.1. Publisher Configuration 26 B.2. Replicator Configuration 30 C. Enumerator API ...34 C.1. Enumerator Types 34 C.2. Enumerator Start ...34 C.2.1. Enumerator Start Query Arguments 35 C.2.2. Enumerator Start Response 35 C.3. Enumerator Next ...36 C.3.1. Enumerator Next Query Arguments 36 - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 5
Router 1.1. Overview of DX Content Router A DX Storage cluster routinely replicates content objects to other nodes in the same cluster in order to improve technical architects in designing scalable, highly redundant, cost effective application storage solutions. 1.2.2. Scope This document covers the - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 6
independent DX Storage clusters, connected to one another via DX Content Router nodes. DX Content Router supports several alternative to Cluster B, and vice versa. The administrator of each cluster can decide, based on stream metadata, which streams should be replicated to the other cluster. It - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 7
between cooperating organizations. With a little forethought and planning when storing descriptive metadata with each stream, a very sophisticated data distribution and storage infrastructure can be created. Because you have the ability to create your own rules, a dynamic pool of data can be created - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 8
. Dotted lines represent local cluster communication. Solid lines represent TCP traffic over the HTTP protocol between DX Content Router services and also between storage nodes within the two independent clusters. Alternatively, if network configuration prevents direct communication between the - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 9
and must be installed on a server in the same subnet as its target storage cluster. A 3rd party application, which may or may not be installed on a node with other DX Content Router services, can also function as a subscriber by integrating with the Enumerator API defined in the Appendix. DX Content - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 10
Similar to the previous proxy-enabled example, if the storage nodes are not able to communicate directly in the configured network topology, the Publisher can be configured to send responses via an SCSP Proxy in a mirrored - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 11
are designed and deployed as independent processes running on a server. We discuss each of these services in more detail in the following sections. 3.1.2. Publisher Service The Publisher service collects a comprehensive list of all the UUIDs stored in the cluster (as well as those that have been - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 12
cluster as the destination of some of the locally stored streams, we want to find all streams whose content metadata contains a header called "DX Storage-priority" whose value starts with either a "1", a "2" or one of the words "high" or "medium". Note that the header name is not case sensitive but - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 13
clause. In the example above, where there are two publish clauses, all content streams can be queued for remote replication once for each publish. In addition, when there are two select clauses in a given publish clause, the content metadata is evaluated against each select clause's filter set. The - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 14
one and two years old or on Mondays"). 3.1.3. Replicator Service A DX Content Router node's Replicator service serves as a subscriber to one remote cluster's Publisher service. The Replicator's purpose is to receive UUIDs from the Publisher service, and then schedule those UUIDs for replication (or - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 15
or Linux distributions are not currently supported. Subsequent installation instructions will assume an existing RHEL supported network services and protocols. 4.1.3.1. Required Communications A DX Content Router node must be able to initiate TCP connections with all nodes in a DX Storage - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 16
Content Router servers and DX Storage cluster nodes. A DX Content Router node should use the same speed connection as the fastest DX Storage node it communicates with to prevent bottlenecks. 4.1.3.4. IP Address Configuration DX Content Router Services expect a predictable communication pattern and - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 17
is installed on a Cluster Services Node (CSN), the necessary software services on a single server. Both the Publisher and Replicator services can be configured from the CSN management console. Please see the 'Configuration' appendix for a full list of all parameters and best practice instructions - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 18
installed Ubuntu-based version of Storage features. It is also recommended that all Replicators be upgraded prior to the Publisher being upgraded for similar reasons. Both services customers should contact their designated support resource for downgrade instructions. 4.4. Removing DX Content Router - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 19
to ensure DX Content Router has the necessary information to start correctly. If a service was stopped for any reason it can be manually started with a standard init.d script. For mirrored configurations with both services on the same server, Publisher and Replicator must be started separately as - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 20
="someHeaderName"/> (not contains('.txt')) - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 21
olderThan('Tue, 16 Oct 2007 00:00:00 GMT') olderThan('365d') matches('.*filename\s*\=.*\.txt.*') - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 22
address field of the browser. The location of the console differs when the Publisher is installed on a Cluster Services Node (CSN). Please reference the CSN Installation and Configuration Guide for related details. Copyright © 2010 Caringo, Inc. All rights reserved 18 Version 2.2 December 2010 - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 23
: the version of DX Content Router installed on the Publisher node • Uptime: the amount of time elapsed since the DX Content Router Publisher service was last started • Source Cluster: the group multicast address of the DX Storage cluster from which the Publisher is gathering UUIDs • Stream Events - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 24
• Working: the subscriber is available and actively processing streams • Offline: the subscriber has not contacted the DX Content Router Publisher node within the expected configurable frequency • Idle: the subscriber is available but is not currently processing any streams • Busy: the subscriber - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 25
in the response in the same order with every request. Enumerators may be deleted, however, so the index may change. If either DX Content Router service is installed on a Cluster Services Node (CSN), the response from the request is used to populate the SNMP MIB for the respective - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 26
[enumerator1] uuid=... channel={rules file channel for which the enumerator was instantiated} context={optional description for enumerator} host=... version=... status=... eventsMatched=... backlogTotal=... transmitQueue=... subscriberQueue=... subscriberInProgress=... droppedEvents=... - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 27
Publisher clients such as Replicator may need to know IP addresses of nodes in the CAStor source cluster that Publisher listens to (in order to make SCSP requests to this cluster). The Publisher Channel Server will support a request for a list of source cluster IP addresses and SCSP port numbers - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 28
Chapter 6. Support for Content Restoration and FailOver Some common uses of DX Content Router Content Router nodes. Note Administrators who temporarily install a Replicator or Publisher service on a DX Content Router node as part of a DR event should plan to uninstall the temporary service when all - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 29
stored with each stream in DX Storage based on the cluster name specified in your cluster or node configuration files. This attribute ensures that section of the DX Storage Application Guide or the Running and Managing CFS section of the CFS Setup and Configuration Guide respectively for a complete - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 30
address. Local log file name when not using a syslog. Syslog host name or IP address. Content Router services log to the local 5 syslog facility. The parameter should not be set if local file-based logging will be used instead of syslog. The number of older, rotated log files to keep when file - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 31
following supported values: 50=critical, 40=error, 30=warning, 20=info, 10=debug, 0=no log The multicast group address for the DX Storage cluster SCSP connection port for the DX Storage cluster listed in the group parameter. A comma-separated list of DX Storage node IP addresses used to validate - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 32
the anticipated size of the data store. Approximate GB required per 100 million streams = 16 + 0.07 (average metadata size per DX Storage object, in bytes). For example, if DX Storage objects have 200 bytes of metadata on average, the requirement will be: 16 + 14 = 30 GB. Time in seconds before the - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 33
Option Name Default subscriberTimeout Interval 90000 Mandatory no maxActiveEvents 20 no snmpCommunity ourpwdofchoicehere no consoleReportStyleURL none no consoleStyleURL none no errorRetentionDays 4 no enumeratorDefault 5000 no MaxItems Copyright © 2010 Caringo, Inc. All rights - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 34
feedback on error conditions. Description The Replicator's IP address. Local log file name when not using a syslog. Syslog host name or IP address. Content Router services log to the local 5 syslog facility. The parameter should not be set Version 2.2 December 2010 - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 35
Description if local file-based logging will be used instead of syslog. The level of logging verbosity with the following supported values: 50=critical, 40 to 239.255.255.255. A comma-separated list of DX Storage node IP addresses used to validate version compatibility at boot time. Either scsphosts - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 36
Option Name Default castorProxyIp castorProxyPort subscribeTo none none none Mandatory no no yes subscriptionCheck 10 no Interval offlineAfter 120 no errOfflineAfter 1800 no timeoutInterval 86400 no consolePort Copyright © 2010 Caringo, Inc. All rights reserved 8088 no 32 - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 37
Option Name Default Mandatory storageDir /var/opt/caringo/ no contentrouter/ replicator maxActiveEvents 20 no ignoreDeleteEvents 0 no Description state on this port. There is not currently a separate console for Replicator. A unique, writable directory path for use in storing in process - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 38
service. The purpose of the HTTP interface is to enable a simple standards based approach for building "plug-in like" DX Storage object revisions across different nodes and clusters. This Storage object types, including named, unnamed and alias objects, please refer to the DX Storage Application Guide - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 39
when configuring a DX Content Router replicator service. It corresponds to one of the The following query arguments are supported for the Start command. enumerated. All time stamp comparisons are based on file creation timestamps. An ISO problem. Copyright © 2010 Caringo, Inc. All rights reserved - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 40
Note the maxItems query argument is only supported for the UUID and Event enumerator types. provides the subscriber a way to report possible trouble conditions to the Publisher. Subscribers should take not be resent by the Publisher without manual intervention using the Republish or Republish All - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 41
only contain metadata for a single object, but UUID and Event enumerators support inclusion of data for multiple objects. If for any reason the request is with a descriptive message in the response body as to the encountered problem. C.4. Enumerator End The Enumerator End command is called to end an - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 42
a descriptive message in the response body as to the encountered problem. C.5. Enumerator Timeout The DX Content Router Publisher will repurpose the timeout period applies to enumerators created for any channel of a Publisher service. It is configurable in the publisher.cfg file, with a default - Dell DX6004S | DX Content Router Setup and Configuration Guide - Page 43
from the Publisher's "subscriberErrOfflineAfter" configuration value. Number of seconds before the Publisher will terminate an enumerator if it has not been heard from. The minimum supported value is 600 seconds. Copyright © 2010 Caringo, Inc. All rights reserved 39 Version 2.2 December 2010
Content Router Setup
and Configuration Guide
Version 2.2