Transcription
Configuration and DeploymentDell EMC ECS: Using Veritas Enterprise VaultAbstractThis document explains how to use Dell EMC ECS object storage asPrimary Storage for Veritas Enterprise Vault .April 2020H15309
RevisionsRevisionsDateDescriptionOctober 2018Release 8December 2018Release 9January 2019Release 10April 2019Release 11January 2020Release 12April 2020Release 13AcknowledgementsAuthor: Paul McKeownThe information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in thispublication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.Use, copying, and distribution of any software described in this publication requires an applicable software license.Copyright 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or itssubsidiaries. Other trademarks may be trademarks of their respective owners. [4/22/2020] [Configuration and Deployment] [H15309]2Dell EMC ECS: Using Veritas Enterprise Vault H15309
Acknowledgements3Dell EMC ECS: Using Veritas Enterprise Vault H15309
Table of contentsTable of contentsRevisions.2Acknowledgements .2Table of contents .4Executive summary.6Audience .6Terminology .61Solution overview .71.1ECS Streamer overview .71.2ECS: multi-protocol, API-accessible storage .81.2.1 Simple multi-tenancy .81.2.2 Multi-site, active-active architecture and access .82ECS configuration details .92.134Namespace compliance .9ECS Streamer installation .123.1ECS Streamer driver install (simple) .123.2ECS Streamer driver install (manual) .143.3ECS Streamer driver remote installation .14ECS Streamer configuration .174.1Add ECS as a storage option in Enterprise Vault .174.2Configuring an ECS Streamer-based Enterprise Vault Store partition .174.2.1 Host property .195ECS Streamer driver details .235.1Network ports.235.2Retention .235.2.1 Retention policies .235.3WORM and NON-WORM support .235.4Safe Copy Check .245.5Support of large partition listings .265.6Handling failed ECS nodes .265.7Windows performance monitoring support .265.7.1 Global counter object .265.7.2 Instance counter object .275.7.3 Process counter object .285.7.4 Reset performance monitor maximum counters: EVResetPerf.exe .295.7.5 Log-level registry value to troubleshoot performance monitor support .304Dell EMC ECS: Using Veritas Enterprise Vault H15309
Table of contents6Troubleshooting .316.1Windows event log.316.2Fiddler .326.3DebugView .366.4ECSCHECK.EXE.386.4.1 Test dtquery .4077.1Veritas Enterprise Vault .417.2Dell EMC ECS .418Streamer release information .429Performance information .45A5Supported environments .419.1Testing configuration .459.2Results .45Technical support and resources .46Dell EMC ECS: Using Veritas Enterprise Vault H15309
Executive summaryExecutive summaryTo support Dell EMC ECS object storage with Veritas Enterprise Vault , the Dell EMC ECS Streamerdriver is used to translate the Veritas Streamer API to the ECS S3 API. This allows content to be archivedfrom Enterprise Vault servers to ECS, as well as queried, restored, and deleted. The ECS Streamer driverleverages the ECS S3 API extensions for retention support and for replication status checking required byEnterprise Vault.This document discusses how to configure Enterprise Vault to use ECS as an archive target.AudienceThis document is intended for storage administrators and Dell EMC professional services personnel who wishto learn how to deploy and configure Dell EMC ECS object storage with Veritas Enterprise Vault.TerminologyEV: Veritas Enterprise VaultS3: Simple Storage Service (API)LB: IP Network Load BalancerVDC: ECS Virtual Data CenterBucket: Logical unit of storage on an ECS system in which objects (including their metadata) are stored6Dell EMC ECS: Using Veritas Enterprise Vault H15309
Solution overview1Solution overviewThis section provides an overview of the integration of ECS with Veritas Enterprise Vault and the keytechnologies used.1.1ECS Streamer overviewVeritas has developed a Storage Streamer API for Enterprise Vault (EV) which archive storage vendors mustintegrate with to allow Enterprise Vault to archive files, email, and other items to their storage systems. DellEMC has developed the ECS Storage Streamer driver to allow Veritas Enterprise Vault to archive to ECS.The ECS Streamer driver translates the Streamer API calls to the ECS S3 API.Customers can now create Vault Store partitions of type Dell EMC ECS within the Veritas Enterprise VaultVAC. These Vault Store partitions are associated with S3 buckets on the ECS cluster where savesetsarchived to the Vault Store will reside.ECS buckets reside within ECS namespaces that may be compliant or non-compliant. Both compliant andnon-compliant namespaces allow retention to be propagated from Enterprise Vault to the ECS cluster,however compliant namespaces store archive data in tamper-resistant storage which meets strict SEC 17a4(f) rules for electronic record-keeping.Example of an ECS-based storage solution for Veritas Enterprise VaultThe Veritas Enterprise Vault system has archive policies that archive files to a Vault Store in which an ECSbased partition has been defined. The ECS Streamer driver uses the ECS S3 API to store/access objects inthe S3 bucket on the ECS cluster.ECS systems are clusters of 4 or more individual ECS nodes. Customers can use the ECS Streamer built inload balancer (from revision 2.0) to distribute I/O across all nodes in the ECS cluster or deploy an IP LoadBalancer to distribute I/O.7Dell EMC ECS: Using Veritas Enterprise Vault H15309
Solution overview1.2ECS: multi-protocol, API-accessible storageECS is a massively scalable, software-defined object storage platform that enables any organization to store,access and manipulate unstructured data as objects. ECS provides support for industry standard APIs suchas Amazon S3. In addition, ECS extends the Amazon S3 API with support for retention, byte range updatesand atomic appends.1.2.1Simple multi-tenancyECS delivers a flat software architecture ideal for multi-tenant environments. Configuring, provisioning,creating a namespace and managing a multitenant platform has never been easier. Key metrics and reportingare provided for capacity, object count, objects created, objects deleted and inbound/outbound bandwidth.These activities are all visible via the ECS GUI and through the REST API.1.2.2Multi-site, active-active architecture and accessECS features a truly geo-efficient architecture that stores, distributes and protects data both locally andgeographically. This eliminates any single point of failure and provides seamless failover from site to site withno impact to the business. ECS automatically maximizes throughout, maintains high availability and datadurability, and increases capacity and the reliability of applications. In terms of geographic limitations – thereare none – providing users with a single global namespace with anywhere access to contentToday’s modern applications demand a different architecture. Managing both cost and accessibility asstorage environments grow and become more complex is one of the biggest challenge’s organizations facetoday. Developers are finding it easier to go to public cloud alternatives putting data at risk and increasingmanagement costs. ECS provides a cloud-scale storage architecture that is specifically designed to supportmodern applications with unparalleled availability, protection, simplicity and scale.8Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS configuration details2ECS configuration detailsHow the ECS is configured (for example, the number of ECS nodes, number of VDCs, GEO protection, andapplication location access) is decided by the solution architect designing the ECS configuration, and for themost part, Veritas Enterprise Vault is unaware of the design. However, the solution architect should considerthe compliance requirements of the customer’s Enterprise Vault solution.2.1Namespace complianceIf the customer wishes to have an SEC-compliant solution, the namespace that the buckets (that allEnterprise Vault partitions will use) should be configured as compliant.When creating the namespace in the ECS Management GUI, ensure the Compliance option is selected.Namespace creationWhen using ECS buckets in a compliant ECS namespace to meet SEC 17-A4 requirements, be aware of thefollowing potential issues when choosing between WORM and NON-WORM partition types9Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS configuration detailsNON-WORM mode EV partitionsIf you are setting an ECS Streamer partition to NON-WORM mode, EV will write objects with no retentionperiod even if you are setting a retention period in the EV Retention Category. If you are using NON-WORMpartitions, it is likely that you should not be using a compliant ECS namespace.WORM modeIf you are using a compliant namespace, you must give a minimum of a one-second retention to the bucketretention value. This means you must set the following option after the Host IP address when creating theECS Streamer Partition:DISABLE WRITE TESTIf you do not set this option, the ECS Streamer will fail connectivity checking because it will create an objectand then delete it as part of its checks. This create/delete will occur within one second and will fail and causethe connectivity check fail.Revision 2.0.1 of the ECS Streamer driver will check if there is a retention period set on the bucket and if so, itwill not perform the write test.Later revisions of the ECS Streamer driver reintroduced write check when there is bucket level retention butnow retry to ensure the operation is performed over a 1 second timeframe.ECS Bucket default retention settingCreate the bucket for the EV Vault Store in the compliant Namespace.Ensure that the Bucket Retention is set to a value that does not conflict with any Enterprise Vault RetentionCategory, ideally just 1 second. This must be set, or an error will occur when you attempt to create thebucket.Administrators should take care when setting the ECS bucket-level retention. Enterprise Vault will expect tobe able to delete expired content and although EV will cleanly handle deletion errors on content that is stillunder ECS retention, it will post and error in the Microsoft Windows Event log.If a customer is using a NON-WORM EV partition it is recommended that ECS bucket level retention is notapplied as EV will expect all the savesets written to the partition to have no retention.10Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS configuration detailsBucket creationConsideration should be given to the interaction between Enterprise Vault and bucket-level retention if morethan one second is used for Enterprise Vault Partitions.11Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer installation3ECS Streamer installation3.1ECS Streamer driver install (simple)You can use the following procedure to download the ECS Streamer driver and quickly perform a defaultinstallation of the driver on a Windows serverGo to www.dell.com/support to locate the ECS Streamer driver installer. Once you have download the ECSStreamer driver onto the Enterprise Vault server, double click the installation file and installation will startUnless they have already been installed on this server you will be prompted to allow the installer to install theVC Redistributed fileECS Streamer driver install - 1Click Install to install the Visual C redistributed packages. You will be asked to reboot the serverECS Streamer driver install - Reboot12Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer installationClick Yes to reboot the server.When you log in after the restart you may be asked to install the VC 32bit version of the distribution, followthe instructions are reboot.When you log in after the reboot the installer will automatically continue the installationClick Next to install the ECS Streamer driver and click through the remaining screens to finish the installation.ECS Streamer installation13Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer installationECS Streamer installation – change installation folderYou can change the installation folder for the ECS Streamer driver at this screen if you wish, otherwise clickthrough the remaining prompts.The ECS Streamer driver should be installed on all Enterprise Vault servers.Please refer to the ECS Streamer release notice for further details on installing the driver.3.2ECS Streamer driver install (manual)Download the ECS Streamer driver as per the above section.Admins can run the ECS Streamer installation from a CLI (DOS) shell and specify The log file pathnameSet the installation directory by specifying a path for the INSTALLDIR propertyFor example: ECSStreamer.2.0.5.7.exe /V"/log c:\setuplogs\install.log INSTALLDIR f:\EVStreamer"Please refer to the ECS Streamer release notice for further details on installing the driver3.3ECS Streamer driver remote installationA utility is now part of the distribution that aids in scripting remote installations.EVRemoteInstall.exe is a stand-alone executable that can be used in a script. It takes as parameters the ECSStreamer setup file, along with a remote computer target. It also has extensive logging options.This utility can be run in one of two ways: 14Install ECS Streamer on a remote system. This utility can be used in a script to install on any number ofsystems. Use the /r parameter to specify the remote server.Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer installation Specify a file containing a list of remote systems and install/uninstall/get status on all the servers in thelist. Use the /rf parameter to specify the list of remote servers.Command line usage:C:\ EVRemoteInstall.exe /?Invalid Parameter: /?Product: EVStreamerUsage:EVRemoteInstall.exe options [ properties ] options :/i path to setup file - Install (UNC path or local path)/x- Uninstall/s- Check Install Status[/u domain\user ]- Connect with specified 'domain\user'[/a password ]- If /u specified, use password to connect[/r remote server ]- Remote server[/rf file ]- File contains one server per-line[/n number ]- Number of parallel installs (default 4)[/p]- Display progress[/pv]- Verbose progress (list actions)[/b]- Reboot after install/uninstallLogging Options/l[i w e a r u c m o p v x ! *] LogFile i - Status messagesw - Nonfatal warningse - All error messagesa - Start up of actionsr - Action-specific recordsu - User requestsc - Initial UI parametersm - Out-of-memory or fatal exit informationo - Out-of-disk-space messagesp - Terminal propertiesv - Verbose outputx - Extra debugging information - Append to existing log file! - Flush each line to the log* - Log all information, except for v and x options/log LogFile Equivalent of /l* LogFile properties - any number of install properties of the format:PROPERTYNAME value if /rf is specified, then /r, /p or /pv are ignored.if /rf is not specified, then /n is ignored.15Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer installationSample usage:This command will install the EV streamer on “remotecomputer”. It will set the install folder tof:\EVStreamer and save the installation log:C:\ EVRemoteInstall.exe /i c:\setupfiles\ECSStreamer.2.0.5.7.exe /r remotecomputer /p/log c:\setuplogs\ECSStreamer.remotecomputer.log INSTALLDIR f:\EVStreamerRequirements: EVRemoteInstall.exe does not require installation and can be run on any supported version ofWindows Server. Windows Server 2008R2, 2012, 2012R2 and 2016.Client versions of Windows (7, 8, 8.1 and 10) are not supported, either for the system runningEVRemoteInstall.exe, or the system where the streamer is being installed.All servers must have file sharing enabled.o File sharing is used to copy files to and from the target system, as well as install a remoteservice and access remote registry.o A named pipe connection is created at the local system to the target system.Additional Information:You will notice that no progress is displayed for the first stage of the install. This is because the MicrosoftRedistributables are being installed during this time. After they are installed, you’ll see progress indications forthe install.The /p option should not be used in a script. This option is good for showing progress when using this utilityinteractively. It will display a row of dots to show progress.This command will set an exit code of zero if the installation was successful, or non-zero if it was notsuccessful. You should capture any standard output, plus save a log file (/log option) to be able to analyzewhy the installation failed.When specifying /rf file and a log file (/l) parameter, it will create one log file for the overall operation, plusone log file for each server in the file list. The server name is added to the log file name specified.If the remote server is in a different domain, then you must specify credentials: “/u domain\user ” and“/a password ” on the command line.When specifying /rf file , the file should contain a list of servers, one per line. The syntax of each lineis:[#] server name/IP [; domain/user ; password ]Where:oooIf the line starts with ‘#’ the line is ignoredBlank lines are ignoredIf “; domain/user ; password ” is included, those values override the same parameters specified on thecommand line, if any. This is useful if servers in the list are in different domains.Please refer to the ECS Streamer release notice for further details on installing the driver16Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer configuration4ECS Streamer configuration4.1Add ECS as a storage option in Enterprise VaultRefer to the following procedure on the Veritas support web site:https://www.veritas.com/support/en US/article.000114337This procedure is only required to be performed on one Enterprise Vault server instance (the primary EVserver typically).Note: With Enterprise Vault revision 12.3 and later, the ECS Streamer driver configuration files (not the ECSStreamer Driver itself) are already included in the EV distribution and this task does not need to be performed.Always check with the Veritas document linked above in case Veritas have made any changes to thisprocedure.4.2Configuring an ECS Streamer-based Enterprise Vault Store partitionIn the VAC, expand the Enterprise Vault Site and Vault Store Groups and right-click the Vault Store you wishto create a new Vault Store Partition for. Click New Partition and click Next.Choose an appropriate Name and Description and click Next.Choose Dell EMC ECS from the Storage Type drop-down menu. Click Next.New Partition menuEnter the connectivity details for the ECS cluster to be connected to and select Test to check theconfiguration details.17Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer configurationPartition definitionPartition propertiesPropertyDescriptionHOSTEnter the IP address or FDQN of the LB service or ECS nodes. See section4.2.1.PortDefault 9020. Enter the port number being used for data connection to the ECS(or Load Balancer).Bucket NameEnter the name of the ECS bucket to be used.ECS buckets used by Enterprise Vault should not be file system enabled.Access KeyEnter the ECS access key (object user) to be used.Secret KeyEnter the Access key secret key.Safe Copy Check0, 1 or 2. See section 5.4.Use proxySet to 0 if no proxy is being used or to the port the proxy is using.Click the Test button to perform a check of the configuration settings. If you get an error, recheck the settingsas entered. Also check what errors are being logged in the Windows Event Log.Pre ECS 3.4 - the namespace property is only used in the dtquery request (used when safe copy check is 2,see the section below). The namespace property is irrelevant to the actual archiving process and it is notused in S3 object requests. The object user is globally unique and belongs to a namespace, therefore thenamespace is implicitly identified.ECS 3.4 – the ECS Streamer Driver can be configured to use an EMC ECS extension to the S3 API todiscover the replication status of a saveset and this uses the same network port as normal S3 data access.18Dell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer configurationIf the Test is successful, click Next.In the next GUI form, select the appropriate WORM setting for the partition. The ECS Streamer driver hasbeen validated for NON-WORM and well as WORM Enterprise Vault partitions, check or un-check this radiobutton appropriately.WORM or NON-WORM settingComplete the forms that follow as required.4.2.1Host propertyWith revision 2.0.1.1, it is possible to specify a comma-separated list of ECS Node IP or FQDN addresses asthe Host value in the Partition Configuration GUI. See the following 0.0.2,10.0.0.3,10.0.0.4You may also specify an IP address range, for example:10.0.0.1-10.0.0.8When specifying the host addresses or range, use of the ECS Streamer driver internal IP Load Balancer isenabled.The ECS Streamer driver will not attempt to auto-discover any other ECS Node IPs when a list or range isspecified.If you do use the ;LB true option, the ECS streamer driver will auto-discover all Node IPs and ignore your listor range.With version 1.0.10 of the Dell ECS Streamer driver, some additions were made to the syntax of the Hostproperty to allow the administrator to set some internal settings of the Dell ECS Streamer driver. The optionsare as follows: 19MAX TRIESDISABLE WRITE TESTMAX CONNSDTHOSTDell EMC ECS: Using Veritas Enterprise Vault H15309
ECS Streamer configuration RDTSHOWCHUNKSLB trueDONT TEST STOPPINGIGNORE BAD COMMON NAMES3V4To use the options, the Host IP (or name) must come first and the other options can be specified using asemicolon separator in name value format, for example:10.1.83.51;MAX TRIES 2;MAX CONNS 25;DISABLE WRITE TEST 1With revision 2.0.2.1 of the ECS streamer driver, you may just specify the value to have it set to 1 (or true).The following example is the same as the prior example:10.1.83.51;MAX TRIES 2;MAX CONNS 25;DISABLE WRITE TEST4.2.1.1MAX TRIESWhen any server failure response is received, the Streamer will attempt a total of 5 tries by default beforereturned bad status to Evault. There is a standard exponential back-off algorithm between retries. This valuecan be overridden by the MAX TRIES option which must be greater than 0 and less than 5.4.2.1.2MAX CONNSThis controls the maximum number of simultaneous safe copy checks that can occur at one time. The defaultis 65 as of revision 1.0.14 of the Streamer driver. Customers should not need to change this number but itmay be set between 2 and 100.Note: As of revision 2.0 of the Streamer driver, this setting is ignored if used.4.2.1.3DISABLE WRITE TESTWhen creating a Vault Store Partition for the first time when there is a Temporary Site Outage (TSO), thetests performed when the user clicks on the Test button will fail because of the TSO. We recommend that youdo not create Vault Store Partitions while a TSO is in effect, but if needed, set DISABLE WRITE TEST 1 todisable the test. With revision 2.0.2.1 of the Streamer driver, you need only specify DISABLE WRITE TES
from Enterprise Vault servers to ECS, as well as queried, restored, and deleted. The ECS Streamer driver leverages the ECS S3 API extensions for retention support and for replication status checking required by Enterprise Vault. This document discusses how to configure E
