How do I increase the open files limit for a non-root user

Recently, I have tried to increase the number of open files allowed for a user, as for carrying out a load test for a Java application server where the current limit of 1024 is not enough.

As the beginning you have to change the SOFT limit and the HARD limit, by changing following file as follows;
/etc/security/limits.conf.

* soft nofile 4096
* hard nofile 4096

The first column describes WHO the limit is to apply for. '*' is a wildcard, meaning all users. To raise the limits for root, you have to explicitly enter 'root' instead of '*'.

You also need to edit /etc/pam.d/common-session* and add the following line to the end:

session required pam_limits.so

This will help you to get rid of "Too many Open File" exception!.

Configuring WSO2 API Manager 1.5.0 with WSO2BAM to support for Load Balanced data publishing

Here I'm going to explain about how to configure WSO2BAM (Download here)to collect, analyze and display the statistics from WSO2 API Manager. (Download here)

Configuring WSO2 API Manager

Step 1: Configure the <APIUsageTracking> in the <APIM_home>/repository/ conf/api-manager.xml as follows;

<APIUsageTracking>

<!-- Enable/Disable the API usage tracker. -->

<Enabled>true</Enabled>

<!-- API Usage Data Publisher. --> <PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>

<!-- Thrift port of the remote BAM server. ->

<ThriftPort>7612</ThriftPort>

<!-- Server URL of the remote BAM/CEP server used to collect statistics. Must

be specified in protocol://hostname:port/ format.

-->

<BAMServerURL>tcp://192.168.1.3:7612,tcp://192.168.1.3:7613</BAMServerURL>

<!-- Administrator username to login to the remote BAM server. -->

<BAMUsername>admin</BAMUsername>

<!-- Administrator password to login to the remote BAM server. -->

<BAMPassword>admin</BAMPassword>

<!-- JNDI name of the data source to be used for getting BAM statistics.This data source should be defined in the master-datasources.xml file in conf/datasources directory. -->

<DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>

</APIUsageTracking>

Make sure to mention both of BAM server URL's in <BAMServerURL> separated by a (,). Also give JNDI name of the data source to be used for getting BAM statistics in later part of the config <DataSourceName> as mention above.

Step 2: Configure the data source definition in master-datasources.xml. Here I have point the db to a MySql database.

<datasource>

<name>WSO2AM_STATS_DB</name>

<description>The datasource used for getting statistics to API Manager</description>

<jndiConfig>

<name>jdbc/WSO2AM_STATS_DB</name>

</jndiConfig>

<definition type="RDBMS">

<configuration>

<url>jdbc:mysql://localhost:3306/BAMDB</url>

<username>root</username>

<password>root</password>

<driverClassName>com.mysql.jdbc.Driver</driverClassName>

<maxActive>50</maxActive>

<maxWait>60000</maxWait>

<testOnBorrow>true</testOnBorrow>

<validationQuery>SELECT 1</validationQuery>

<validationInterval>30000</validationInterval>

</configuration>

</definition>

</datasource>

Configuring WSO2BAM

Step 1: If you’re running a multiple WSO2 products in same host, change the port offset of BAM, carbon.xml in <BAM_home>/repository/conf/carbon.xml

<!-- Ports offset. This entry will set the value of the ports defined below to the define value + Offset. e.g. Offset=2 and HTTPS port=9443 will set the effective HTTPS port to 9445 -->

<Offset>1</Offset>

Step 2: Copy the file <APIM_HOME>/statistics/API_Manager_Analytics.tbox to directory, <BAM_HOME>/repository/deployment/server/bam-toolbox. If you use Oracle DB to store statistical data, copyAPI_Manager_Analytics_Oracle.tbox instead.

Step 3: Configure the data source definition in master-datasources.xml.

<datasource>

<name>WSO2AM_STATS_DB</name>

<description>The datasource used for getting statistics to API Manager </description>

<jndiConfig>

<name>jdbc/WSO2AM_STATS_DB</name>

</jndiConfig>

<definition type="RDBMS">

<configuration>

<url>jdbc:mysql://localhost:3306/BAMDB</url>

<username>root</username>

<password>root</password>

<driverClassName>com.mysql.jdbc.Driver</driverClassName>

<maxActive>50</maxActive>

<maxWait>60000</maxWait>

<testOnBorrow>true</testOnBorrow>

<validationQuery>SELECT 1</validationQuery>

<validationInterval>30000</validationInterval>

</configuration>

</definition>

</datasource>

If you have change the default BAM port as mentioned in the step 1, you must have to change the Cassandra port accordingly in the JDBC connection url as well.

<datasource>

<name>WSO2BAM_CASSANDRA_DATASOURCE</name>

<description>The datasource used for Cassandra data</description>

<definition type="RDBMS">

<configuration>

<url>jdbc:cassandra://localhost:9161/EVENT_KS</url>

<username>admin</username>

<password>admin</password>

</configuration>

</definition>

</datasource>

For load balanced data publishing, follow the same steps mention in the Configuring WSO2BAM section and specify the correct URL in the <BAMServerURL> section in the <APIM_home>/repository/conf/api-manager.xml.

Restart WSO2BAM server by running <BAM_HOME>/bin/wso2server.[sh/bat].

And then the APIM server by running <APIM_home>/bin/wso2server.[sh/bat].

Fail Over testing

Once the APIM busy, serving requests, check the statistics mentioned in the APIM publisher. Then stop one BAM server instance and once the JVM stops, check whether the statistics still updating fine in the publisher. Since the APIM set up is configured to serve load balanced data publishing, statistics should be update correctly even from one BAM node. (refer this article to setup a BAM cluster setup.)

Enabling One Time Password (OTP) in WSO2 Identity Server

OTP is an identity management feature which is allowing the user to have a new password each time when they login. This makes the user to secure from password theft and do not want to be annoyed to remember the password all the time.

Step 1

Set the following configurations in {carbon_home}/repository/conf/security/identity-mgt.properties file.

Identity.Listener.Enable=true

Notification.Sending.Enable=true

Notification.Expire.Time=7200

Notification.Sending.Internally.Managed=true

Authentication.Policy.Enable=true

Authentication.Policy.Check.OneTime.Password=true

Step 2

Start the identity server. Once after the server start up login to the portal as the admin user and set the following claim for the OTP and map it with a valid attribute with the underlying data store. 

http://wso2.org/claims/identity/otp

Make sure you have select Supported by Default attribute, so the OTP attribute will display by default in the user profile.

Step 3

Create a new user with the role permission to login to the IS portal. Then log in to the portal using new user credentials. Go to the user profile as shown below. Update the profile details and set the One Time Password attribute to true.

Step 4

Sign out from the user and you will get only permit to login only the next time with your current password. So once you logged out from the next login session you will get a mail from the portal with the details about your new password as follows;

WSO2 Stratos 1.6 – Testing the Application Server worker nodes behavior using soapUI

In this example I'm going to explain how to test the behavior of Application Server worker nodes when its deal with a heavy load of web requests which is generated using soapUI.

Prerequisites

Please refer the my previous blog to find the information about how to download, install and initialize the servers.

Step 1:

Write a simple Hello service as follows and export project as deployable archive.

package org.wso2.sample;

import java.util.Date;

public class HelloService{

public String sayHello(String name) {

String message = "Hello, " + name + "! " + System.currentTimeMillis();

System.out.println(message + " " + new Date());

System.out.flush();

return message;

}

}

Step 2:

Log in to the management console of Application Server management node and deploy the created Axis2 Web service (Go to Manage --> Axis2 Services --> Add --> AAR service). Once the service is deployed in the management node go to the services list and click on 'Try this service' to make sure that the created service functioning well.

Step 3:

Then create a new soapUI project using the Hello service, WSDL1.1.

After that, right click on the 'Request 1' and add it to Test case and then the project list will be updated as follows;

To create a heavy load of request, you have to create new load test by right click on 'Load Tests' link. Once you create the load test you will get a window as follows to adjust the load according to you requirement. i.e. increasing the no of Threads in to 100.

Step 4:

Click on 'run' button and check the server logs which passing in to Application Server worker node01 and node02.

WSO2 Stratos 1.6 – Setting up an additional WSO2 Application Server Worker node to the WSO2 Stratos ELB.

In this example I'm going to explain you the minimum configuration that would guide you to set up another worker node to the Stratos ELB in a case if you need to configure another worker node to the Stratos setup.

Prerequisites

Please refer the Udara Liyanage's blog to find the information about how to download, install and initialize the servers.

Step 1:

Follow the <wso2-stratos-setup-home>/worker path and check available worker nodes using following command;

~/wso2-stratos-local-setup-1.6.0/worker$ ls

Get a copy of available Application Server instance and paste it on the same folder by renaming it.

Ex: wso2as-5.0.2 as wso2as-5.0.2-2

Step 2 (axis2.xml configuration):

We need to change the localMemberBindPort and the localMemberPort in the axis2.xml to make a separate communication port to the load balancer with the newly created worker node. Open <wso2-stratos-setup-home>/worker/wso2as-5.0.2/repository/conf/axis2/axis2.xml and update the configuration as shown below.

<parameter name="localMemberBindPort">4305</parameter>

<parameter name="localMemberPort">4305</parameter>

Step 3 (carbon.xml configuration):

We going to run multiple WSO2 Carbon based products in same host, so we need to change the port offset to avoid the possible port conflicts. In order to do that, open <wso2-stratos-setup-home>/worker/wso2as-5.0.2/repository/conf/carbon.xml and increment available offset by 1.

i.e. <Offset>4</Offset>

Step 4 (stratos.sh configuration):

To start the newly created Application Server worker node as ./stratos.sh you need to update the stratos.sh with the new worker node information as follows;

Open the stratos.sh file and add the new entry to the list as follows;

as_mgt=wso2as-5.0.2

as_worker=wso2as-5.0.2

as_worker2=wso2as-5.0.2-2

bps_mgt=wso2bps-3.0.1

bps_worker=wso2bps-3.0.1

cep=wso2cep-2.0.1

dss_mgt=wso2dss-3.0.1

dss_worker=wso2dss-3.0.1

greg=wso2greg-4.5.3

mb=wso2mb-2.1.0

manager=wso2stratos-manager-2.0.2

bam=wso2bam-2.1.0

brs_mgt=wso2brs-2.0.0

brs_worker=wso2brs-2.0.0

cg=wso2cg-1.0.0

esb_mgt=wso2esb-4.6.0

esb_worker=wso2esb-4.6.0

is=wso2is-4.0.1

ss=wso2ss-1.0.2

ts=wso2ts-1.0.1

elb=wso2elb-2.0.2

Also add the new worker node to the start_stop() method as follows;

elif [ $1 == "as_worker" ] || [ $1 == "as_worker2" ] || [ $1 == "esb_worker" ] || [ $1 == "dss_worker" ] || [ $1 == "bps_worker" ] || [ $1 == "brs_worker" ] ;

At the end add it to the echo list that show the available services as well.

echo "manager as_mgt as_worker as_worker2 esb_mgt esb_worker dss_mgt dss_worker bps_mgt bps_worker brs_mgt brs_worker greg bam cep mb cg is ss ts"

So once you run ./stratos.sh start command you will see the newly created Application Server node in the list as follows;

Step 5 (start up new worker node):

Run following command to start up the new worker node.

./stratos.sh start 'as_worker2'

Once the new worker node joins the cluster, you will see the following messages in WSO2 Load Balancer logs. Check the different ports assigned to Worker node1 (4300) and node2 (4305) in LB logs as follows;

node 1:
TID: [0] [ELB] [2013-07-06 13:45:08,687]  INFO {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler} -  Received JOIN message from 127.0.0.1:4300(wso2.appserver.domain) {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler}
TID: [0] [ELB] [2013-07-06 13:45:19,689]  INFO {org.apache.axis2.clustering.management.DefaultGroupManagementAgent} -  Application member Host:127.0.0.1, Remote Host:null, Port: 4300, HTTP:9766, HTTPS:9446, Domain: wso2.appserver.domain, Sub-domain:worker, Active:true joined application cluster {org.apache.axis2.clustering.management.DefaultGroupManagementAgent}
TID: [0] [ELB] [2013-07-06 13:45:19,690]  INFO {org.apache.axis2.clustering.tribes.MembershipManager} -  Application member 127.0.0.1:4300(wso2.appserver.domain) joined group wso2.appserver.domain {org.apache.axis2.clustering.tribes.MembershipManager}

node 2:

TID: [0] [ELB] [2013-07-06 13:47:05,817]  INFO {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler} -  Received JOIN message from 127.0.0.1:4305(wso2.appserver.domain) {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler}

TID: [0] [ELB] [2013-07-06 13:47:16,821]  INFO {org.apache.axis2.clustering.management.DefaultGroupManagementAgent} -  Application member Host:127.0.0.1, Remote Host:null, Port: 4305, HTTP:9767, HTTPS:9447, Domain: wso2.appserver.domain, Sub-domain:worker, Active:true joined application cluster {org.apache.axis2.clustering.management.DefaultGroupManagementAgent}

TID: [0] [ELB] [2013-07-06 13:47:16,821]  INFO {org.apache.axis2.clustering.tribes.MembershipManager} -  Application member 127.0.0.1:4305(wso2.appserver.domain) joined group wso2.appserver.domain {org.apache.axis2.clustering.tribes.MembershipManager}

Testing the cluster

To test that whether the startup worker nodes are working simultaneously you can follow the way as describe below;

• Log in to the management console of Application Server management node
• Deploy a new Axis2 Web service (Go to Manage --> Axis2 Services --> Add --> AAR service)
• Once the service is deployed in the management node go to the services list and click on Tryit
• Invoke the service

Once you send bulk set of requests to the load balancer you will notice that the requests are simultaneously handle by the worker node 1 and worker node 2 by analyzing the server log entries of worker node 1 and worker node 2. (Testing the cluster using soapUI - next blog)