mopdf.com

WHY DOES SAS ENVIRONMENT MANAGER SHOW ALL RESOURCES AS (!) (DOWN)?If all resources are stopped, you should correctly see all resources as down in SAS EnvironmentManager. However, there are cases that the resources might actually be up, but they are shown as down.SAS Environment Manager uses the server-agent mode for monitoring resources. There is one serverand one or many agents. All the resource metrics are first collected by the agents and then sent from theagents to the server. In addition, every resource has an availability metric. If the availability metric is true,the resource is shown as running or up; if the availability metric is false, the resource is shown as stoppedor down. The availability metric has a Boolean value similar as many other metrics. However, there is adifference. If we assume that the availability metric was true at some point and the server has notreceived any new availability metric data for a period of time, the resource will be shown as down eventhough the resource might be actually up.Keep in mind that all the metrics are sent from the agents to the server. Further, each platform has onlyone agent, so, all metric data on that platform will be sent by this one agent. If all resources associatedwith this same agent are shown as down in the server, then it is likely that the server cannot receive datafrom the agent, rather than the resources actually being down.There are various reasons why the server cannot get data from the agent. You would need to checkthese areas: The server The network between server and agent The agent platform The agent process.In most cases, you need to check the log files to get details for further diagnosis. You can also delete theagent cache directory and restart the agent. The cache data file folder is typically located in the directory SAS Config aBecause SAS Environment Manager treats the availability like a normal metric, it’s possible that theresource is shown as down, but the other associated metrics are all valid. Hence, you might still be ableto view other metrics for a Down resource.WHAT IS THE BEST WAY TO ENABLE AN ALERT FOR A RESOURCE?The best way to define and enable an alert is through the SAS Environment Manager web interface.Follow the following steps to define a new alert:1. Under the Resources tab, navigate to or search a resource by resource name or type.2. Go to the Detail page by clicking the resource name.3. Go to the Alert List page by clicking the ‘Alert’ button.4. Go to the Alert Definition List page by clicking the ‘Configure’ button.5. Go to the New Alert Definition page by clicking the ‘New ’ button.6. Enter all information and click the ‘OK’ button to create a new alert definitionFollow the following steps to enable an inactive alert definition:1. Go to the Alert Center page by clicking the ‘Alert Center’ button from the ‘Analyze’ drop-down menu.2. Go to the Alert Definition List page by clicking the ‘Definition’ button.3. Find the specified alert definition and click its name to go into the detail page.4. Edit the alert definition to activate the alert definition.THE AGENT CANNOT CONNECT TO THE SERVERTypically, this problem is due to the incorrect setup of the environment, which results in the SASEnvironment Manager agent failing to start up or failing to connect to the Environment Manager server. Itis rarely caused by the application errors. Five of the common specific causes are listed below.5

Permission issue The SAS Environment Manager agent registers and connects to the server by usingthe SAS Environment Manager Service Account user account. If you have not assigned proper roles inSAS Environment Manager, the agent will fail to start up and connect to the Server. If this happens, a“Permission denied” error message will be displayed in the agent start up console or in the Agent log file.See Figure 5 for an example.Figure 5 - Agent Start Up ErrorThe SAS Environment Manager Service Account (sasevs@saspw) user account should be assigned theSuper User Role in SAS Environment Manager by default. If the account is not assigned to the properrole (for example, it’s assigned to only the Guest Role), then the SAS Environment Manager server willnot accept the agent’s connection request and it will report the permission exception.You can check the user role in SAS Environment Manager or examine the logs to confirm the reason. Ifthe user is not assigned to the Super User Role, you can assign the proper role to the user by using theSAS Environment Manager web interface.Incorrect machine host and port setup The agent and the server IP addresses and the ports must beconfigured correctly. Especially in a multiple server-multiple machine environment, the IP addresses andports for both SAS Environment Manager server and the agents should always be available on thenetwork. Figure 6 is an example agent configuration properties file, which is typically located at SAS Config f/agent.properties.Figure 6 – Agent Configuration Properties File6

If the SAS Environment Manager agent or server listening ports are changed, the properties should alsobe changed accordingly.Cache data expired or not valid anymore The agent will create a data cache file when it starts up andconnects to the server. Afterward, it will use the content in the file for the future communication to the server.There are cases where the agent has been changed (for example, during an upgrade in place or an agentupgrade), but the cache data file did not get updated accordingly. This mismatch can cause a connectionfailure between the agent and the server.A common solution to fix this problem is to clear the cache data folder and then restart the agent. Thecache data file folder is typically located at SAS Config This is also the first recommendation to resolve any connection failure issues between the agent and theserver.JAR version mismatch When the agent starts up, it needs to interact with the native code implementedby the open-source third party JAR files. If the system environment changes, such as through a patch beingapplied, but the corresponding JAR files are not updated, the agent might fail to connect to the server.For example, one customer ran into the connection issue after applying a new security patch. We foundout there was a problem with the version of the sigar.jar file that was used. After upgrading the JAR file tothe newer version, the problem was resolved.Upgrade non-SSL to SSL environment If you change from a non-SSL to an SSL implementation, thenthe existing connection data for the agent will not work anymore. In this case, you should delete the SASEnvironment Manager agent data file and restart the agent to generate the new SSL connection data forthe agent and server communication.Decryption error Several agent properties are encrypted after the first successful start up. The key that isused to encrypt the values is saved in the file SASConfig /LevX/Web/ SASEnvironmentManager/ agent5.0.0-EE/conf/agent.scu. The next time an agent starts, it attempts to use this file to decrypt those encryptedproperties. However, the error sibleException” will bereturned if the process failed to decrypt the values. This seems to be an agent start up error, rather than anerror with the connection to the Server. However, you might see the following errors at the console: “Errorcontacting agent: error sending argument: unable to connect to 127.0.0.1:2144: connection refused, retried5 times, cmd agent:ping”. The error includes the word “connection”, so we are including it with otherconnection errors. To fix the problem, use the following workaround:1. Stop your SAS Environment Manager agent.2. Delete or rename the data directory SAS Config \LevX\Web\SASEnvironmentManager\agent5.0.0-EE.3. Delete or rename the agent.scu file in the SASConfig f directory.4. Modify the encrypted property to a plain text value. The following two files contain the encryptedproperties, which should be formatted to ENC(XXXXXXXXXX). SASConfig f/agent.properties file; setthe agent.setup.camPword property to a plain password text value. SASConfig f/auto-approve.propertiesfile; all values inside auto-approve.properties file should be changed to true.5. Execute the following command: SASConfig \hq-7

agent.bat/sh restartAGENT DOES NOT SUCCESSFULLY COLLECT METRICS OR DISCOVER RESOURCESThere are many reasons that the agent does not successfully collect metrics or discover resources.These are the four common causes.Server configuration issue Every resource has its own specific configuration properties. To collectmetrics, the values for these properties must be filled out correctly. Go to the following locations in SASEnvironment Manager to find the configuration properties and make edits as needed. Resources - browse- Host Name Server Name - Inventory - Configuration Properties Resources - browse - Host Name Server Name - Service Name - Inventory - Configuration PropertiesMetric enablement Not every metric is defined to be collected by default. If you can’t find a metric, go toone of the following locations to check whether it’s being collected: Resources - browse - Host Name Server Name - MonitorResources - browse - Host Name Server Name - Service Name - MonitorClick the arrow button to the right of the label “Show All Metrics. See Figure 7.Figure 7 - Show All MetricsThen select the metrics you want and fill out the box for “Collection Interval for Selected” at the bottom.See Figure 8. Click the arrow icon to save the changes.8

Figure 8 - Metric Collection IntervalUpgrade in place (UIP) issue UIP or hotfix processes might upgrade the versions of SAS EnvironmentManager plugins. However, after the UIP completes or the hotfix is applied, the data for the old pluginsmight not be removed from the system. This might prevent the discovery of the new versions of theresources. A manual rediscovery is needed to discover the resources, and the manual changes to theconfiguration properties might be needed in order to discover all the sub-resources and metrics. Forexample, the following figures show how to do a rediscovery of ActiveMQ 5.12 after the ActiveMQ serveris upgraded from 5.7 to 5.12.1. Log on to SAS Environment Manager, navigate to the Resources - Servers page and delete theActiveMQ5.7 server. See Figure 9.Figure 9 - Delete the ActiveMQ Server Resource9

2. Click Platforms- “Rdcesx****”. See Figure 10.Figure 10 - Select the Platform3. Select “New Auto-Discovery” from the “Tools Menu” drop-down list. See Figure 11.Figure 11 - New Auto-Discovery Selection4. Select the check box for ActiveMQ 5.12 and then click “OK”. See Figure 12.10

Figure 12 - Select ActiveMQ for Auto-Discovery5. When you open the Dashboard again, and you can see the newly displayed ActiveMQ server. Clickthe button “Add to Inventory” to add the server. Keep in mind that it might take a few minutes forthe server to show up on the Dashboard. See Figure 13.Figure 13 - Add Server to Inventory6. When you open the Resources - Servers page, you should see the new server you just added. SeeFigure 14.Figure 14 - Viewing the New Server11

After you successfully rediscover the server with the new version, you might still not be able to find theservices of the new server. Let us assume that you see the screen in Figure 15, which shows you couldnot find the services (No health data).Figure 15 - No Health Data AvailableYou can follow these steps to fix the problem.1. Log on to SAS Environment Manager and navigate to Resources - Servers. Click the ActiveMQserver such as “RDCESX***ActiveMQ 5.12 localhost”. See Figure 16.Figure 16 - Select Server2. Click the Inventory tab (Figure 17).12

Figure 17 - Select the Inventory Tab3. At the bottom of the Inventory page, click the Edit button. See Figure 18.Figure 18 - Select Edit4. Check the “-Dcom.sun.management.jmxremote.authenticate” property that can be found inwrapper.conf in the directory C:\SAS\Config\Lev1\Web\activemq\bin\win64. If the property value iswrapper.java.additional.15 -Dcom.sun.management.jmxremote.authenticate true, then specifyvalues for the “jmx.username” and “jmx.password” on the Configuration Properties page. See Figure19. Otherwise, you do not have to specify these values.5. Update the port number as needed for “jmx.url”. The port can be found in wrapper.conf in thedirectory java.additional.14 -Dcom.sun.management.jmxremote.port 11099. See Figure 19.13

Figure 19 - Server Configuration Properties6. Click OK to save the properties.7. In the Resources - Servers page, click the server “RDCESX***ActiveMQ 5.12 localhost”. You shouldnow be able to see the health data as shown in Figure 20.Figure 20 - Health Data AvailableEnvironment issue You might have a special hardware environment. Consider this example. Acustomer reports that the “Hyperic Apache Tomcat 6.0” can’t be discovered in their productionenvironment. After much debugging, it turns out that there were two Network Interface Controllers (NICs)on the server and an unused NIC was enabled. This caused an incorrect IP address to be picked up.Bugs in the plugins It is possible there are bugs in the plugins code. We have encountered this a fewtimes. However, this is not common. We encourage you to contact SAS Technical Support if all otheravenues have been exploited but you still have problems.14

HOW TO CONFIGURE HTTPS FOR SAS ENVIRONMENT MANAGER WEB CONSOLEYou should follow the recommendation in the paper “Advanced Topics in SAS Environment Manager” toconfigure the HTTPS for SAS Environment Manager. (See References.) The details are described in thesection “SSL configuration for SAS Environment Manager”. In summary, you can use the SASDeployment Wizard (SDW) to configure HTTPS or SSL support automatically. When using SDW, bydefault, both HTTP and HTTPS are configured. You do need to pay attention to what certificates to use,as described in the paper.Another question related to HTTPS configuration is this: if I have configured SAS Environment Managerto use self-signed certificates, how can I change to use the customer Certificate Authority (CA) or sitesigned certificates? Follow these steps to address this question.1. Prepare your certificate from CA or your company’s IT group.2. Use SAS Deployment Manager (SDM) to replace the certificates in the SAS Deployment Managersecurity framework keystore.3. Use a program such as OpenSSL to transfer the CA certificate to the Java keystore format certificateand replace it in the SAS Environment Manager Server conf folder.4. Change the callback URL in the spring security configuration file.5. Remove the old keystore data from the SAS Environment Manager database.6. Use SAS Management Console to change the SAS logon property.7. Restart SAS services after you have made all changes.For more detailed instructions, please contact SAS Technical Support.HOW TO CONFIGURE THE AGENTS WITH CERTIFICATES OTHER THAN THE DEFAULTSELF-SIGNED CERTIFICATESThe SAS Environment Manage agent can use certificates other than the default self-signed certificates(such as CA or site-signed certificates). Here are the options to accomplish this.Use SAS Deployment Wizard (SDW) When running SDW to configure the SAS Environment Manageragent, you will see the screen in Figure 21. Make sure you check “Establish secure communication”.Figure 21 - Specify Agent Communication15

On the next screen (Figure 22), change the drop-down selection to “Use a customer-supplied JKS formatkeystore”.Figure 22 – Select Customer JKS Format KeystoreIn the next screen (Figure 23), specify the path to your customer certificates file.Figure 23 - Specify Path to Keystore File16

Manual update If you choose the “Use the default JKS format keystore ” option in the SAS DeploymentWizard (Figure 24), then by default, the self-signed certificate is used.Figure 24 - Select Default JKS Format KeystoreIf you select the self-signed certificate, you can use the manual process described in the previous sectionto change it to use the customer CA or site-signed certificates. The process in that section is for changingthe certificates for the SAS Environment Manager server. After you have completed that process, youneed to make the following manual changes to the agent.1. Stop the agent2. Delete all files in cache data directory, which is typically SAS Config a3. Start the agentThis will trigger the download of the certificates from the server database to the agent.WHAT ARE THE COMBINATION OF ROLES NEEDED TO SEE THE RESOURCES TAB?There are no permissions that control the visibility of the Resources tab. That also means any SASEnvironment Manager user can see the Resources tab even though that user does not have any role orpermissions. However, even though the user can see the Resources tab, that does not mean that theuser can see resources under the tab. What resources a user can see is controlled by the roles that userhas. The role definition further defines the permissions of resources and the groups of the resourcesassigned to the role. See Figure 25.17

Figure 25 – Role PermissionsThe resources that can have permissions include “Groups”, “Platforms”, “Servers”, “Services”, and“Applications”. The possible permissions are: “None”, “Read Only”, “Read/Write” and “Full”. The followingtwo simple examples demonstrate the process of assigning permission and what a user can see basedon that assignment.In the first example, Figure 26 shows a user “ruser1”, who has a role “None Resources.” This role doesnot have any permissions (except Read permission for Escalations). With this role, the user “ruser1” cansee the “Resources” tab, but cannot see any resources.Figure 26 - Permissions for "None Resources" Role18

In the second example, Figure 27 shows a user “ruser2”, who has a role “Groups Resources.” This rolehas Read permissions on Servers and Groups and Read permission for Escalations). The role has anassigned group called “agents”, which contains two server resources.Figure 27 – Permissions for “Groups Resources” RoleWith this role, the user “ruser2” can see the “Resources” tab and can also see two server resources in thegroup “agents”. See Figures 28 and 29.Figure 28 - Servers Visible to User "ruser2"19

Figure 29 - Groups Visible to User "ruser2"CONCLUSIONThis paper identified several most common and frequently asked questions from our customers. Thequestions are mainly in the areas of SAS Environment Manager server and agent and include topics suchas SAS Environment Manager web interface usage; server functions such as alerts, controls, andresource discovery; agent issues; and security issues such as user management and SSL configuration.The paper describes the solutions and workarounds. We also plan to address many of these issues infuture SAS Environment Manager releases.Several other frequently asked questions are listed below. We encourage you to contact SAS TechnicalSupport if you need solutions for any of these issues. What are the details and definitions for the configurable products? Need more information about the meaning of specific alerts., For example, these two alerts come upfrequently: TCP Attempt Fails. What does this mean? How is this alert triggered and how can the problembe fixed? HQ Time Agent Spends Fetching Metrics. What causes this alert, and how it can the problem befixed? Why does the server create lots of temporary tables to store the metrics? Can these tables bedeleted? Why does the agent fail to start and returns the message “- Unable to register agent: Permissiondenied”? Can SAS Environment Manager be configured behind SAS Web Server or any Reverse ProxyServer?REFERENCESSAS Institute Inc. 2016. What's New in SAS 9.4. Cary, NC: SAS Institute Inc. Available snew/64788/PDF/default/whatsnew.pdfSAS Institute Inc. 2015. SAS 9.4 Intelligence Platform: Installation and. Configuration Guide. Cary, NC:SAS Institute Inc. Available /69172/PDF/default/biig.pdf20

SAS Institute Inc. 2016. SAS Environment Manager 2.5 User’s Guide. Cary, NC: SAS Institute Inc.Available evcdc/2.5 M1/docsets/evug/content/evug.pdf?locale en#nameddest titlepagePeters, Amy, Bonham, Bob, and Li, Zhiyong. 2013, “Monitoring 101: New Features in SAS 9.4 forMonitoring Your SAS Intelligence Platform.” Proceedings of the SAS Global Forum 2013 Conference.Cary, NC: SAS Institute Inc. Available at gs13/4632013.pdfLi, Zhiyong, and Fernandez, Alec. 2014, “Migrating SAS Java EE Applications from WebLogic,WebSphere, and JBoss to Pivotal tc Server.” Proceedings of the SAS Global Forum 2014 Conference.Cary, NC: SAS Institute Inc. Available ngs14/SAS357-2014.pdfLi, Zhiyong, and Thorland, Mike. 2015, “Your Top Ten SAS Middle-Tier Questions.” Proceedings of theSAS Global Forum 2015 Conference. Cary, NC: SAS Institute Inc. Available ngs15/SAS1904-2015.pdfLi, Zhiyong, and Gilles Chrzaszcz 2016, “Advanced Topics in SAS Environment Manager.” Proceedingsof the SAS Global Forum 2016 Conference. Cary, NC: SAS Institute Inc. Available athttp://s

These resources include: Apache Tomcat 6.0; Pivotal Web Server 5.4 WebServer; PostgreSQL 9.x localhost:9432; tc Runtime SASServer1_1. If you are an authorized user, you can use SAS Environment Manager to invoke a control action, schedule an action for a future time, or sche