The WebSphere Runbook

Wednesday, 20 April 2016

Stopping and starting WAS application modules separately via wsadmin

Allow starting and stopping an individual Web or EJB modules using an undocumented call to the ApplicationManager object.

Get a reference to the relevant ApplicationManager object in the usual way:

appManager = AdminControl.queryNames

('type=ApplicationManager,process=myClusterMember1,node=myNode,*')

List the operations available for the appManager object...

print Help.operations(appManager)

This results in the following output …

void startApplication(java.lang.String)  
java.lang.Boolean _canStopApplication(java.lang.String) 
void stopApplication(java.lang.String) 
void _startModule(java.lang.String, java.lang.String) 
void _stopModule(java.lang.String, java.lang.String) 
void _applicationInstalled(java.lang.String) 
void _applicationUninstalled(java.lang.String)

The two methods we're interested in are _startModule and _stopModule, both of which take two arguments of type String.

True to form the IBM documentation was awful.

print Help.operations(appManager, '_startModule')

Provides the following following output:

void _startModule(java.lang.String, java.lang.String)
Description: Start Application Module - This method is intended for WebSphere internal use only.  Its function is undocumented and is subject to change at any time.

Parameters:
Type  java.lang.String
Name  applicationName
Description  Application Name

Type  java.lang.String
Name  moduleURI
Description  Module Name

It took me a while to work out how to format the arguments, here's the correct form:

AdminControl.invoke(<reference to app manager>, '<action>', '<app name> <module name>')

For example:

AdminControl.invoke(appManager,  '_startModule',  'MyApp  MyWebModule' )

As you can see, there are three arguments as follows:
- The reference to the ApplicationManager object
- The action
- A string which is the application name concatenated with the module name (separated by a space)

So, a nice easy way to do this would be:

appManager = AdminControl.queryNames('type=ApplicationManager,process=server1,node=node1,*')
appName = 'MyApp'
moduleName = 'MyWebModule'
args = appName + '  ' + moduleName
adminControl.invoke(appManager,  '_startModule',  args )

I also found out that if you're using a later wsadmin client (e.g. the one that ships with WAS 8.5 and above) that the third argument can be an array:

AdminControl.invoke(appManager,  '_startModule' ,  ['MyApp' , 'MyWebModule'] )

Here's an example of how I used this in the real world when I needed to start a WebModule after hot deploying it.

Wednesday, 25 March 2015

Simple shell script to create and package a Servlet

#!/bin/sh

SERVLET_NAME=$1

rm -rI ${SERVLET_NAME}

# Create WAR file structure
echo "Creating WAR directory structure"
mkdir -p ${SERVLET_NAME}/src/co/uk/bobclarke
mkdir -p ${SERVLET_NAME}/WEB-INF/classes

# Create java file
echo "Creating ${SERVLET_NAME}.java"
cat > ${SERVLET_NAME}/src/co/uk/bobclarke/${SERVLET_NAME}.java <<EOF1
package co.uk.bobclarke;

import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class ${SERVLET_NAME} extends HttpServlet{

        public void doGet ( HttpServletRequest request, HttpServletResponse response) throws IOException {
                PrintWriter out = response.getWriter();
                out.println("<html>");
                out.println("<body>");
                out.println("<p>Hello World</p>");
                out.println("</body>");
                out.println("</html>");
        }
}
EOF1


# Create deployment descriptor
echo "Creating web.xml"
cat > ${SERVLET_NAME}/WEB-INF/web.xml <<EOF2
<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
        <servlet>
                <servlet-name>${SERVLET_NAME}</servlet-name>
                <servlet-class>co.uk.bobclarke.${SERVLET_NAME}</servlet-class>
        </servlet>

        <servlet-mapping>
                <servlet-name>${SERVLET_NAME}</servlet-name>
                <url-pattern>/${SERVLET_NAME}</url-pattern>
        </servlet-mapping>
</web-app>
EOF2

# Compile
echo "Compiling ${SERVLET_NAME}.java"
javac -classpath /home/autowas/was/dev/JavaEE/j2ee.jar \
        ${SERVLET_NAME}/src/co/uk/bobclarke/${SERVLET_NAME}.java -d \
        ${SERVLET_NAME}/WEB-INF/classes

# Create WAR file
echo "Creating ${SERVLET_NAME}.war"
jar -cvf ${SERVLET_NAME}.war -C ${SERVLET_NAME} .

Thursday, 19 March 2015

Configure JVM heap for WAS cluster members

Nice and simple if you have many JVM's in your cluster.
Note the wildcard feature when listing AdminConfig types (Line 24), very handy.

#=================================================================
# Setup
#=================================================================
import sys
clusterName = sys.argv[0]
min = sys.argv[1]
max = sys.argv[2]

#=================================================================
# Subs
#=================================================================
def getJvms(clusterName):
        # Get the cluster ID
        cluster = AdminConfig.list('ServerCluster', '*'+clusterName+'*')

        # Get a list of the clusters members
        members = AdminConfig.showAttribute(cluster, 'members')
        memberArray = members.split('[')[1].split(']')[0].split(' ')

        # Detemine JVM's
        jvms = []
        for member in memberArray:
                memberName = AdminConfig.showAttribute(member, 'memberName')
                jvms.append(AdminConfig.list('JavaVirtualMachine', '*'+memberName+'*'))
        return jvms

 

def setHeap(jvms, min, max):
        for jvm in jvms:
                print 'Setting '+jvm+' heap sizes of '+str(min)+' and '+str(max)
                AdminConfig.modify(jvm, [['initialHeapSize', min]])
                AdminConfig.modify(jvm, [['maximumHeapSize', max]])


#=================================================================
# Main
#=================================================================
jvms = getJvms(clusterName)
setHeap(jvms, min, max)
AdminConfig.save()

Thursday, 5 March 2015

wsadmin AdminApp.install with wildcards (and what on earth were they thinking with the nested argument structure?)

This always used to drive me bonkers because of the parameters needed for
-MapModulesToServers
-MapWebModToVH
-CtxRootForWebMod.

Then I realised these could be wild-carded. The result still looks a bit archaic but it's a big improvement.

import sys

war = sys.argv[0]
name = sys.argv[1]
cluster = sys.argv[2]
cr = sys.argv[3]
cellId = AdminConfig.list('Cell')
cell = AdminConfig.showAttribute(cellId, 'name')

 AdminApp.install(war, '[ -appname '+name+' -contextroot '+cr+' 
        -MapModulesToServers [[ .* .* WebSphere:cell='+cell+',cluster='+cluster+' ]] 
        -MapWebModToVH [[ .* .* default_host ]] 
        -CtxRootForWebMod [[ .* .* '+cr+' ]]]' )

AdminConfig.save()

It's worth noting that only -MapWebModToVH and -MapModulesToServer are required, therefore a if you're in a hurry you can use a shorter version:

import sys

war = sys.argv[0]
cluster = sys.argv[1]

AdminApp.install(war, '[-MapWebModToVH [[ .* .* default_host ]] 
        -MapModulesToServers [[ .* .* WebSphere:cluster='+cluster+']]]')

AdminConfig.save()

If you chose this simpler method the Application name is chosen for you and the context root is set to /

Lastly (and this is where my OCD really kicked in) ... I really didn't like the way the second argument to AdminApp.install was just a long string, so I worked out that you can also do it like this... (not really sure if it's any simpler though)

import sys

war = sys.argv[0]
cluster = sys.argv[1]

AdminApp.install( 
    war, [
        '-MapWebModToVH', 
        [ [ '.*', '.*', 'default_host'] ],  
        '-MapModulesToServers', 
        [ [ '.*', '.*', 'WebSphere:cluster='+cluster ] ] 
    ]
)
AdminConfig.save()

Monday, 23 February 2015

Midvision Rapid Deploy - command line operation

A run script could look like this....

#!/bin/sh

java -Xmx1536m -classpath \

jars/rapiddeploy-orchestration.jar:\

jars/rapiddeploy-remoting-jsch.jar:\

jars/rapiddeploy-core-plugins-prd-ibm-websphere7.jar:\

jars/rapiddeploy-core-plugins-prd-ibm-websphere85.jar:\

jars/rapiddeploy-core-plugins-tools-maven.jar:\

jars/rapiddeploy-remoting-remoting.jar:\

jars/rapiddeploy-core-plugins-prd-ibm-websphere8.jar:\

jars/rapiddeploy-ext-wmq-jar.jar \

com/midvision/rapiddeploy/domain/task/core/DefaultBatch \

definition=mv-rapiddeploy-orchestration.xml \

log4j=log4j.properties \

name=RD-TEST \

server=10.1.1.20 \

environment=autowas \

instance=RapidDeployTestInstance \

application=my_application \

deployFile=tars/rd-test.tar \

taskflowLock=rd-test.lock

Structure of the RD runtime

/home/autowas/rapid-deploy

builds/ This is where RD unpacks the tar

jars/ The RD runtime jars

mv-rapiddeploy-orchestration.xml The main orchestration file

run-rd.sh* A script to invoke RD (example above)

tars/ Where I placed my custom tar (structure below)

unpack/ Where I unpack the tar to manipulate contents during test

Structure RD expects in a tar

db_libs/

_Documentation

install.dat

integrators.txt

j2ee/ This is where all the "deployment" files live

jenkinsbuild.log

lib/

log4j.properties

midvision-deploy-orchestration.xml

midvision-jee-deploy-orchestration.xml The main sub orchestration file

mv-rapiddeploy-orchestration.xml Not sure why this is duplicated here

scripts/

SharedLibraries/

url_libs/

UrbanCode properties

The basic form is .... ${p:scope/propertyName}

where scope is application, environment, resource (there are more, I've found these the most useful)
All scopes have a build in allProperties array, for example ${p:environment/allProperties}
Array elements are comma separated with each element of the form key=value

Create a new WAS job in UrbanCode Deploy

Create a Top Level Resource Group
Add the Agent on the DMGR to this Group
Create a Component
Create an Application
Add the Component to the Application
Create a Process (this is the low level process) under the Component of type "configuration deployment"
In this process create your *.py file. The contents of the file will be our wsadmin script
Also in this Process add a Shell step which invokes wsadmin and passes args
Save the Process
Create an Environment under the Application
Click on the Environment name and add a Base Resource (the whole folder representing the Top Level Resource Group you created in step 1)
Under Resources > Resources, navigate to the agent under the Top Level Resource you created earlier click Actions and add the Component you created earlier (this is called mapping / associating the component to the machine where it will run) - NOTE: I'm not sure whether you need to add this to the Agent or the containing folder - TBC
Under the Application create a high level Process, because I added the Component to the Application in step 3 it will available to drag onto the canvas.

Friday, 6 February 2015

wsadmin script to create a datasource (and of course a JDBC provider and auth alias) on WebSphere

Just a quicky so I can remember this.
Will need to modify the provider if you want to use something other than Oracle.


#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Set up
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
import sys

cellName = AdminConfig.list('Cell')
dsName=sys.argv[0]
userName=sys.argv[1]
password=sys.argv[2]
driverPath=sys.argv[3]
dbHost=sys.argv[4]
dbPort=sys.argv[5]
dbSid=sys.argv[6]

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Subs
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
def createProvider():
      print "Creating JDBC Provider"
      provider = AdminTask.createJDBCProvider('[-scope Cell='+cellName+' 
        -databaseType Oracle -providerType "Oracle JDBC Driver" 
        -implementationType "XA data source" -name "Oracle JDBC Driver (XA)" -description "" 
        -classpath ['+driverPath+'] -nativePath "" ]') 
 return(provider)

def createAuthAlias():
      print "Creating Auth Alias"
      security = AdminConfig.getid('/Security:/')
      alias = ['alias', dsName+'_alias']
      userid = ['userId', userName]
      pw = ['password', password]
      jaasAttrs = [alias, userid, pw]
      aliasId = AdminConfig.create('JAASAuthData', security, jaasAttrs)
      return(aliasId)

def createDataSource(provider, aliasId):
      print "Creating DataSource"
      aliasName =  AdminConfig.showAttribute(aliasId, 'alias') 
      ds = AdminTask.createDatasource(provider, '[-name '+dsName+' -jndiName jdbc/'+dsName+' 
        -dataStoreHelperClassName   com.ibm.websphere.rsadapter.Oracle11gDataStoreHelper 
        -containerManagedPersistence true -componentManagedAuthenticationAlias 
        -xaRecoveryAuthAlias '+aliasName+' 
        -configureResourceProperties [[URL java.lang.String jdbc:oracle:thin:@'+dbHost+':'+dbPort+':'+dbSid+']]]')
      AdminConfig.create('MappingModule', ds, '[[authDataAlias '+aliasName+'] [mappingConfigAlias ""]]')

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Main program
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
provider = createProvider()
aliasId = createAuthAlias()
createDataSource(provider, aliasId)

Monday, 19 January 2015

VirtualBox guest with Host Only and Bridged network interfaces

My CentOS guest has interfaces
- 192.168.100.102 - host-only adapter
- 192.168.0.105 - bridged adapter

If I run ifconfig on my Mac (which is running the hypervisor) I see it has the same subnet (192.168.0.0) as the guests bridged adapter. The host only adapter appears to be represented by a new interface in the listing called vboxnet0 with an ipaddress of 192.168.100.100. This means the Mac is directly connected to both subnets and therefore doesn't need to route.

The guest should be able to route to internet (if available) via bridged adapter so therefore the default gateway should be the same as the one the Mac uses, i.e when I'm at home this is the wireless router and when I'm mobile it's my phones hotspot router. These are on different subnets and my guest interface is hard coded for a class C 192.168.0.0 network, this is OK for my home but my phones hotspot has a network address of 192.168.43.0. To work around this I've been changing the eth0 interface IP Address in /etc/sysconfig/network-scripts/ifcfg-eth0. I probably need to add another interface so that I'll have three in total (1 for host only and two bridged, 1 for the mobile subnet (192.168.43.0) and another for home wireless subnet 192.168.0.0). Not sure how to configure the routing table to contain both but I'll work that out. Alternatively I could just configure the home wireless router to be on subnet 192.168.43.0 but that wouldn't be as interesting.

To add a default router manually (not sure how to make this permanent as /etc/defaultgateway doesn't seem to be working) I run.
route add default gw 192.168.0.1 eth0

I would normally use netstat -r to look at the routing table but it takes ages to display the last entry (the one I'm interested in) for some reason. I've found that ip route list is faster.

Setting the DNS Server is just a case if updating /etc/resolve.conf.

Monday, 5 January 2015

wsadmin - list methods of a Python module

This can be quite handy.

To see a list of references (to modules, strings, arrays etc) type dir() from the wsadmin prompt as follows...

wsadmin>dir()

The response will be something like...
['AdminApp', 'AdminApplication', 'AdminAuthorizations', 'AdminBLA', 'AdminClusterManagement', 'AdminConfig', 'AdminControl', 'AdminJ2C', 'AdminJDBC', 'AdminJMS', 'AdminLibHelp', 'AdminNodeGroupManagement', 'AdminNodeManagement', 'AdminResources', 'AdminServerManagement', 'AdminTask', 'AdminUtilities', 'ApplyPerfTuning', 'Help', 'LTPA_LDAPSecurityOff', 'LTPA_LDAPSecurityOn', 'TypedProxy', '__builtin__', '__doc__', '__name__', 'bsf', 'cellName', 'checkuserpw', 'doAuthenticationMechanism', 'doGlobalSecurity', 'doGlobalSecurityDisable', 'doLDAPUserRegistry', 'domainHostname', 'exportLTPAKey', 'flag', 'foo', 'forceSync', 'generateLTPAKeys', 'getLDAPUserRegistryId', 'getLTPAId', 'getSecId', 'getSecurityAdminMbean', 'imp', 'java', 'ldapPassword', 'ldapPort', 'ldapServer', 'ldapServerId', 'ldapUserRegistryId', 'lineSeparator', 'ltpaId', 'main', 'nodeName', 'osgiApplicationConsole', 'secMbean', 'securityId', 'securityoff', 'securityon', 'sleep', 'sys', 'whatEnv']

For those that are python modules (for example AdminResources is a module) you can run dir(<module name>) to list the methods within the function, fot exmaple...

wsadmin>dir(AdminResources)

Produces the following output...
['AdminUtilities', '__doc__', '__file__', '__name__', 'bundleName', 'createCompleteMailProvider', 'createCompleteMailProviderAtScope', 'createCompleteResourceEnvProvider', 'createCompleteResourceEnvProviderAtScope', 'createCompleteURLProvider', 'createCompleteURLProviderAtScope', 'createJAASAuthenticationAlias', 'createLibraryRef', 'createMailProvider', 'createMailProviderAtScope', 'createMailSession', 'createMailSessionAtScope', 'createProtocolProvider', 'createProtocolProviderAtScope', 'createResourceEnvEntries', 'createResourceEnvEntriesAtScope', 'createResourceEnvProvider', 'createResourceEnvProviderAtScope', 'createResourceEnvProviderRef', 'createResourceEnvProviderRefAtScope', 'createScheduler', 'createSchedulerAtScope', 'createSharedLibrary', 'createSharedLibraryAtScope', 'createURL', 'createURLAtScope', 'createURLProvider', 'createURLProviderAtScope', 'createWorkManager', 'createWorkManagerAtScope', 'help', 'resourceBundle', 'sys']

You'll see that one of the methods is createJAASAuthenticationAlias, which means you can run

wsadmin>AdminResources.createJAASAuthenticationAlias( authAlias, uid, password)

NOTE: I found out the args by running AdminResources.help('createJAASAuthenticationAlias')

Here's an example of importing a python module and querying/using it accordingly ...

wsadmin>import time

wsadmin>dir()
['AdminApp', 'AdminApplication', 'AdminAuthorizations', 'AdminBLA', 'AdminClusterManagement', 'AdminConfig', 'AdminControl', 'AdminJ2C', 'AdminJDBC', 'AdminJMS', 'AdminLibHelp', 'AdminNodeGroupManagement', 'AdminNodeManagement', 'AdminResources', 'AdminServerManagement', 'AdminTask', 'AdminUtilities', 'ApplyPerfTuning', 'Help', 'LTPA_LDAPSecurityOff', 'LTPA_LDAPSecurityOn', 'TypedProxy', '__builtin__', '__doc__', '__name__', 'bsf', 'cellName', 'checkuserpw', 'doAuthenticationMechanism', 'doGlobalSecurity', 'doGlobalSecurityDisable', 'doLDAPUserRegistry', 'domainHostname', 'exportLTPAKey', 'flag', 'foo', 'forceSync', 'generateLTPAKeys', 'getLDAPUserRegistryId', 'getLTPAId', 'getSecId', 'getSecurityAdminMbean', 'imp', 'java', 'ldapPassword', 'ldapPort', 'ldapServer', 'ldapServerId', 'ldapUserRegistryId', 'lineSeparator', 'ltpaId', 'main', 'nodeName', 'osgiApplicationConsole', 'secMbean', 'securityId', 'securityoff', 'securityon', 'sleep', 'sys', 'time', 'whatEnv']

wsadmin>time
<jclass org.python.modules.time at -1272186292>

wsadmin>type(time)
<jclass org.python.core.PyJavaClass at -779978699>

wsadmin>dir(time)
['__doc__', 'accept2dyear', 'altzone', 'asctime', 'classDictInit', 'clock', 'ctime', 'daylight', 'gmtime', 'localtime', 'mktime', 'sleep', 'strftime', 'time', 'timezone', 'tzname']

wsadmin>time.ctime()
'Tue Jan 06 10:20:33 2015'

Saturday, 25 October 2014

Export private and public keys from a PKCS12 keystore

Both keys into the same PEM file

openssl pkcs12 -in keystore.p12 -passin "pass:password" -nodes -out keypair.pem

Separate PEM files

openssl pkcs12 -in keystore.p12 -passin "pass:password" -nodes -nocerts -out private_key.pem

openssl pkcs12 -in keystore.p12 -passin "pass:password" -nokeys -out public_key.pem

Sunday, 14 September 2014

Shared folders in VirtualBox

Example using MacOSX host and CentOS guest

Install Guest Additions on the guest...
https://www.virtualbox.org/manual/ch04.html

From the host command line run...
VBoxManage sharedfolder add <VM name> --name <share name> --hostpath <host directory>

For example...
VBoxManage sharedfolder add CentOS6.4 --name myshare --hostpath /Users/clarkeb/automation

Now you have the host mapped to a shared device on the guest. Now you need to mount that device within the guest OS to a point on the file system

mount -t vboxsf <share name> <empty directory>

For example

mount -t vboxsf myshare /home/clarkeb/automation

Thursday, 4 September 2014

Create JNDI entries for scalars with wsadmin

# createJndiEntry.py

# Author: Bob Clarke

# Date: 28/08/2014

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# Define subs

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

def repExists(repName):

print 'Checking if Resource Env Provider '+repName+' exists'

rep = AdminConfig.getid('/ResourceEnvironmentProvider:'+repName+'/')

return rep

def createRep(repName):

print 'Creating Resource Env Provider with name '+repName

rep = AdminConfig.create('ResourceEnvironmentProvider', cellId, [['name',repName]] )

# Not sure why I need this "Referencable", seems to be a requirement

AdminConfig.create('Referenceable', rep, [['factoryClassname', 'fClass1'],['classname', 'Class1']])

return rep

def reeExists(key):

print 'Checking if jndi entry of salsa/'+key+' exists'

ree = AdminConfig.getid('/ResourceEnvEntry:'+key+'/')

return ree

def createJndiRef(rep,key):

print 'Creating JNDI entry salsa/'+key

ree = AdminConfig.create('ResourceEnvEntry', rep, [['name', key], ['jndiName', 'salsa/'+key]])

return ree

def createPropSet(ree,key):

print 'Creating an empty property set'

propSet = AdminConfig.create('J2EEResourcePropertySet',ree, [])

return propSet

def createProp(propSet,key):

print 'Creating property, key='+key+' : value='+val

AdminConfig.create('J2EEResourceProperty', propSet, [['name', key], ['type', 'java.lang.String'], ['value', val], ['required', 'false']])

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# Setup

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

repName = sys.argv[0]

key = sys.argv[1]

val = sys.argv[2]

cellId = AdminConfig.getid('/Cell:/')

rep=""

ree=""

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# Main program

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

existingRep=repExists(repName)

if existingRep:

print repName+' exists'

print 'Setting rep to '+str(existingRep)

rep=existingRep

else:

rep=createRep(repName)

existingRee=reeExists(key)

if existingRee:

print 'salsa/'+key+' exists'

print 'Setting ree to '+str(existingRee)

ree=existingRee

else:

ree=createJndiRef(rep,key)

propSet=createPropSet(ree,key)

createProp(propSet,key)

AdminConfig.save()

Avoid getting tied in knots with nested jython lists

I usually get tied in knots trying to use nested lists when creating / modifying config with wsadmin.

Consider this example:

AdminConfig.create( 'J2EEResourceProperty', propSet,

    '[[name "key"]

     [type "java.lang.String"]

     [description "My property"]

     [value "Hello"]

     [required "false"]]' )

When you add some passed in params to the mix it gets messy:

AdminConfig.create( 'J2EEResourceProperty', propSet, 
    '[[name " ’ + keyParam + ’ "] 
     [type "java.lang.String"] 
     [description "My property"] 
     [value " ’ + valueParam + ’ "] 
     [required "false"]]' )

The above looks fiddly to me, much better to use the following approach:

AdminConfig.create( 'J2EEResourceProperty', propSet, 
    [['name', keyParam], 
    ['type', 'java.lang.String'], 
    ['description', 'My property'], 
    ['value', valueParam], 
    ['required', 'false']] )

Monday, 4 August 2014

VBoxManage common commands

VBoxManage list vms

VBoxManage list runningvms

VBoxManage list hdds (gets size of vdi)

VBoxManage showvminfo <vm name>

VBoxManage clonevm <existing vmname> --name <new vm name> --basefolder <path> --register --mode all

VBoxManage modifyvm <vm name> --ostype Linux

VBoxManage modifyvm <vm name> --memory 8192

VBoxManage modifyhd "<path to .vdi file>" --resize 32768

VBoxManage registervm “path to .vbox file”

VBoxManage startvm <vm name> --type headless

VBoxManage guestproperty enumerate <vm name> | grep IP

VBoxManage snapshot list <vm name>

VBoxManage snapshot <vm name> take <new snapshot name>

VBoxManage snapshot <vm name> restore <snapshot name>

VBoxManage controlvm <vm name> poweroff

VBoxManage controlvm <vm name> savestate

Thursday, 31 July 2014

Installing Oracle XE and SQL Plus on CentOS

Dependencies

rpm --install libaio-0.3.107-10.el6.x86_64.rpm

rpm --install bc-1.06.95-1.el6.x86_64.rpm

Oracle XE Install and post config (note, you need to have populated xe.rsp)

rpm --install oracle-xe-11.2.0-1.0.x86_64.rpm

/etc/init.d/oracle-xe configure responseFile=xe.rsp

Instant Client and SQLPlus install

rpm --install oracle-instantclient11.2-basic-11.2.0.4.0-1.x86_64.rpm

rpm --install oracle-instantclient11.2-sqlplus-11.2.0.4.0-1.x86_64.rpm

Environment setup

export LD_LIBRARY_PATH=/usr/lib/oracle/11.2/client64/lib

export ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe

export ORACLE_SID=XE

Test with

/usr/lib/oracle/11.2/client64/bin/sqlplus system/oracle@XE

Resources

http://pkgs.org

http://www.oracle.com

Wednesday, 30 July 2014

Resize VirtualBox VM file system (and swap) under LVM

Quick overview of LVM

- Volume groups contain PV's and LV's, think of them as the glue between the two

- LV's are the things that are referred to in /etc/fstab (e.g thngs you out file systems on or use as swap space)

- You add space to VG's (by way of PV's) and you can then extend LV's which are members of these VG's

- The basic process is ... 1./ create disk partition, 2./ register it as a PV, 3./ add this PV to VG, 4./ extend LV

The high level process goes as follows....

1./ Shutdown VM

2./ Increase the size of the Virtual disk image

3./ Startup VM

4./ Create a new primary disk partition to fill the newly created VDI space

5./ Register the new partition as a PV (Physical Volume) with LVM

6./ Add the new PV to the existing Volume Group

7./ Extend the relevant Logical Volumes to use the newly available space in the Volume Group

8./ Resize the filesystem or swap accordingly depending on what you need to use the extra space for.

Here are the details....

First resize the Virtual disk image

Get your VM name - sudo VBoxManage list runningvms

Get the VDI path for your VM - sudo VBoxManage showvminfo <vm name> | grep -i vdi (alternatively use VBoxManage list hdds)

- Power off the VM - sudo VBoxManage controlvm <VM uuid> poweroff

- Now resize the VDI - sudo VBoxManage modifyhd <path.vdi|VM uuid> --resize 32768 (for 32GB)

Create the primary disk partition

- Start up the VM - sudo VBoxManage startvm <uuid | name>

- log on to the VM OS

- su to root

- fdisk /dev/sda

- u (twice) to change display units to cylinders

- p to print the partition table (you'll need this to obtain statring sector of the new partition)

- n to add a partion

- p to add primary partition

- select a partition number

- Select starting cylinder (the default should be correct)

- +16G (for adding 16GB)

- p to print the partition table to check it

- v to verify

- w to write the table

- reboot

Register the newply created partition with LVM

- running lvm with no args places you in the lvm prompt. Type help to see all commands

- We need to Create a physical volume (PV) for volume manager

- pvdisplay - will show you current PV’s

- pvcreate --verbose /dev/sda3 - add a new PV

Extend volume group

- vgdisplay - will show you which volume groups exist - we have one called "VolGroup" with a Free PE / Size of 0/0

- vgextend <Vol group name> /dev/sda3 - will add our new physical volume to this volume group

- if we now run vgdisplay again we'll see the Free PE / Size has increased accordingly

Extend the logical volume

- lvextend --size +16G /dev/VolGroup/lv_root - this adds 16GB to the volume the logical volume named lv_root

- Or we could have specified the target size "lvextend --size 32G /dev/VolGroup/lv_root"

Display LV details

- lvdisplay

Now get the OS to actually use the added space by either resizing the filesystem or adding swap or both ...

To resize the filesystem

- resize2fs /dev/VolGroup/lv_root (uses all available size)

- now take a look with df –k

To add swapspace

- swapoff /dev/VolGroup/lv_swap –v (or swapoff –a) to disable swap on all devices)

- lvextend /dev/VolGroup/lv_swap --size +1G (assuming you have 1GB spare in your volume group)

- mkswap /dev/VolGroup/lv_swap

- swapon /dev/VolGroup/lv_swap

- now take a look with free -m

Monday, 9 June 2014

AdminTask help using wildcards

Quite handy ... in this example I'm searching for AdminTask commands that contain the string SDK

wsadmin> print AdminTask.help('-commands', '*SDK*')

Friday, 16 May 2014

wsadmin script to gather Dynacache stats

# dynaCacheMon.py

# Author: Bob Clarke
# Date: 05/01/2015
#

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Set up
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
import time
count = 0
month = time.ctime().split()[1]
day = time.ctime().split()[2]
year = time.ctime().split()[4]
timeStamp = time.ctime().split()[3]
logFile='/tmp/dynaCacheMon_'+timeStamp+'_'+month+'_'+day+'_'+year+'.log'
file = open(logFile,'a')

if(len(sys.argv) == 2):
        maxLoops = float(sys.argv[0])
        waitTime = float(sys.argv[1])
else:
        # Default values
        maxLoops = 3
        waitTime = 10

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Main program
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
print '\nRunning '+str(maxLoops)+' loops with '+str(waitTime)+' seconds delay. Logging results to '+logFile

while (count < maxLoops):
        print "Loop number "+str(count)
        mBeans = AdminControl.queryNames('type=DynaCache,*')
        for mBean in mBeans.splitlines():
                process = mBean.split(',')[1]
                jvm = AdminControl.queryNames('type=JVM,'+process+',*')
                freeHeap = AdminControl.invoke(jvm, 'getFreeMemory')
                maxHeap = AdminControl.invoke(jvm, 'getMaxMemory')
                file.write('------------------- JVM Details --------------------\n')
                file.write('Loop: '+str(count)+'\n')
                file.write('Time: '+str(time.ctime())+'\n')
                file.write('JVM: '+process+'\n')
                file.write('Max heap: '+maxHeap+'\n')
                file.write('Free heap: '+freeHeap+'\n')
                file.write('---------- Cache instances for this JVM ------------\n')

                for cacheInstance in AdminControl.invoke(mBean, 'getCacheInstanceNames').splitlines():
                        #stats = AdminControl.invoke(mBean, 'getCacheStatistics', cacheInstance+' [MemoryCacheSizeInMB MemoryCacheEntries]')
                        stats = AdminControl.invoke(mBean, 'getAllCacheStatistics', cacheInstance)
                        file.write('### Instance Name: '+cacheInstance+' ###\n')
                        file.write(stats+',\n\n')
        file.flush()
        count = count+1
        sleep(waitTime)

Friday, 25 April 2014

Lightweight in-memory Database

http://h2database.com/html/main.html

Thursday, 24 April 2014

Some basic Oracle-xe and Sqlplus notes

Query

----------

sqlplus <user>/<password>@<SID> <<EDF

select * from dba_users;

EOF

Setting the editor

-------------------------------

SQL> define_editor=’vi’

Oracle XE Status / stop / start

-----------------------

/etc/init.d/oracle-xe status (as root)

lsnrctl status (as oracle user)

lsnrctl start (as oracle user)

Oracle XE Install on Ubuntu
---------------------------------
Download oracle-xe-universal_10.2.0.1-1.1_i386.deb
/opt/oracle/src/oracle-xe-universal_10.2.0.1-1.1_i386.deb
edit /etc/apt/sources.list as follows...
deb file:/opt/oracle/src raring main
Create a Packages.gz file as follows...
cd /opt/oracle/src
dpkg-scanpackages . | gzip > /opt/oracle/src/dists/raring/main/binary-i386/Packages.gz

Update apt cache...
apt-get update

Install ...
apt-get install oracle-xe-universal
ureadahead will be reprofiled on next reboot
Setting up oracle-xe-universal (10.2.0.1-1.1) ...
update-rc.d: warning: /etc/init.d/oracle-xe missing LSB information
update-rc.d: see <http://wiki.debian.org/LSBInitScripts>
Executing Post-install steps...
You must run '/etc/init.d/oracle-xe configure' as the root user to configure the database
HTTP Admin port = 40000
DB Port = 40521
password for SYS and SYSTEM = oracle123
started on boot = no
The DB is now started .. and it's running as user 'oracle'. not sure how it knew to run as this user, I don't recall

providing the installer that info.

Now it's installed we need to run it
------------------------------------
Already running as user 'oracle'. not sure how it knew to run as this user, I don't recall providing the installer that
info.

Wednesday, 16 April 2014

AdminControl invoke with multiple arguments

Basic syntax:

AdminControl.invoke( <MBean string>, '<method>', '<arg1>  <arg2>' )

So it seems that AdminControl.invoke needs the third argument to be a string (I've often seen the error message "Third argument cannot be coerced into a string")

Here's an example of running getCacheStatistics on a DynaCache mbean:

AdminControl.invoke(

    myBean,

    'getCacheStatistics',

    '/services/cache/appdata CacheHits'

The third argument in the command above is providing the cache instance name (/services/cache/appdata) followed by a space and then the name of the metric we're after (CacheHits)

In the case of <DynaCache MBean>.getCacheStatistics you can provide multiple metrics to retrieve by providing an array as follows:

AdminControl.invoke(

    myBean,

    'getCacheStatistics',

    '/services/cache/appdata [CacheHits DiskCacheSizeInMB]'

Taken from the following article...

http://pic.dhe.ibm.com/infocenter/wasinfo/v8r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Frdyn_mbeancachestat.html

Wednesday, 2 April 2014

Centos iptables causing 'No route to host' message

This had me going for a while...

Two machines on the same subnet, client machine pings server machine fine, I can ssh from client to server, but when trying to connect to any other TCP service I get a 'No route to host' message!

I'm staring at the screen thinking "How can the TCP layer possibly have a bearing on routing at the IP layer below?.... that's bonkers". For a minute the long held belief that I had understood the core fundamentals of TCP/IP wavered. Thankfully I spotted a sentence on a blog stating that on some Linux distro's iptables erroneously reported 'No route to host' when dropping packets. Sure enough I checked my iptables rules and all ports were blocked except 22.... phew, sane again :-)

Friday, 22 November 2013

Getting JConsole to connect to a secured WebSphere ND Cell

I've often used JConsole to get the basic stats for JVM's, but I wanted to get right into all the juicy stuff - JDBC connection pools, ThreadPools ... and even better, Lombardi Event Manager and checking for failed BPD Instances !!.. now that's more like it.

The problem I had initially with getting this to work was that I kept getting authentication failures .. the classic "Cannot connect due to insufficient or empty credentials". I kept seeing this even though com.ibm.ws.admin.client_7.0.0.jar was in my classpath and I had correctly a correctly populated sas.client.props file. They key to this problem was that (strangely enough) the com.ibm.CORBA package (used when you set com.ibm.CORBA.ConfigURL as a JVM param - see below) which is NOT in com.ibm.ws.admin.client_7.0.0.jar !!!

This meant that the JVM param I refer to (i.e com.ibm.CORBA.ConfigURL) was being ignored and the sas.client.props file wasn't being read.... no wonder I wasn't authenticating!

So, I thought to myself, this is easy enough, I just need to find that JAR that does contain the com.ibm.CORBA package. At this timely point, my colleague Jeff leaned over and said "Why don't you just use an IBM runtime, I bet this will work" .... I said that this wasn't logical because surely I just needed to find the right JAR... but I was running low on time, so I reluctantly plugged in the IBM runtime and it sprung into life. Great stuff, it works, but I would have preferred not to be forced into having to use the IBM runtime
... all a bit too mysterious and black boxy for my liking ... I must return to solve it properly one day, not that I'm OCD or anything :-)

So, moving on....
Assuming you have an X-Window server running, the example script below will let you wield all sorts of power over your WAS Cell.

$HOST is the Deployment Manager
$PORT can be the Deployment Manager's RMI port or it's ORB Port.

#!/bin/bash

export BASE=/opt/bpm
ex port JAVA_HOME=${BASE}/java
export CLASSPATH=${JAVA_HOME}/lib/jconsole.jar
export CLASSPATH=${CLASSPATH}:${JAVA_HOME}/lib/tools.jar
export CLASSPATH=${CLASSPATH}:${BASE}/runtimes/com.ibm.ws.admin.client_7.0.0.jar
export HOST=saturnbpm.stack1.com
export PORT=11004
export DISPLAY=<your desktop IP>:0
export JCP=java.class.path=${CLASSPATH}
export CLIENTSSL=com.ibm.SSL.ConfigURL="file:${BASE}/bob/ssl.client.props"
export CLIENTSAS=com.ibm.CORBA.ConfigURL="file:${BASE}/bob/sas.client.props"
export JMXURL=service:jmx:iiop://${HOST}:${PORT}/jndi/JMXConnector

${JAVA_HOME}/bin/jconsole -J-Djavax.net.debug=ssl \
                      -J-D${JCP} -J-D${CLIENTSSL} -J-D${CLIENTSAS} $JMXURL

... and here's what your sas.client.props and ssl.client.props should contain

sas.client.props

com.ibm.CORBA.securityEnabled=true
com.ibm.CORBA.authenticationTarget=BasicAuth
com.ibm.CORBA.authenticationRetryEnabled=true
# com.ibm.CORBA.loginSource can be set to prompt, stdin or properties
# If properties is specified you'll need to also specify 
# com.ibm.CORBA.loginUserid and com.ibm.CORBA.loginPassword
com.ibm.CORBA.loginSource=properties
com.ibm.CORBA.loginUserid=admin
com.ibm.CORBA.loginPassword={xor}< Your XOR'd password >
com.ibm.CORBA.requestTimeout=180
com.ibm.CORBA.validateBasicAuth=true
com.ibm.CORBA.authenticationRetryCount=3

ssl.client.props

com.ibm.ssl.defaultAlias=DefaultSSLSettings
com.ibm.ssl.alias=DefaultSSLSettings
com.ibm.ssl.protocol=SSL_TLS
com.ibm.ssl.trustManager=IbmPKIX
com.ibm.ssl.enableSignerExchangePrompt=gui
com.ibm.ssl.trustStoreName=ClientDefaultTrustStore
com.ibm.ssl.trustStore=myTrustStore.p12
com.ibm.ssl.trustStorePassword={xor}< The XOR'd password of your SSL truststore>
com.ibm.ssl.trustStoreType=PKCS12
com.ibm.ssl.trustStoreProvider=IBMJCE
com.ibm.ssl.trustStoreFileBased=true
com.ibm.ssl.trustStoreReadOnly=false

Thursday, 14 November 2013

A quick guide to using the WebSphere app server AdminControl object

When I need to monitor some realtime stats on WAS using wsadmin, this is the process I go through....

Fire up a wsadmin command line, you'll now be at the wsadmin> prompt
Grab an array full of the particular objects you want to monitor. In this example I need to get stats from ThreadPools, so I run tps = AdminControl.queryNames('type=ThreadPool,*').splitlines()
tps is now a PyList containing a shed load of ThreadPool objects
So let's pull a random one out for investigation by running tp = tps[1]
if you type tp the object will be printed to the screen.. it'll look something like this ...

'WebSphere:name=BPESchedulerWorkManager.Alarm Pool,process=PSUAT003.AppTarget_2.plbpmu07psuat003node001.0,platform=dynamicproxy,node=plbpmu07psuat003node001,version=7.0.0.25,type=ThreadPool,mbeanIdentifier=BPESchedulerWorkManager.Alarm Pool,cell=psuat003cell001,spec=1.0'

Let's see what type of attributes this type of object has by running print Help.attributes(tp). This gives the following output...

Attribute                       Type                            Access
name                            java.lang.String                RO
maximumSize                     int                             RW
minimumSize                     int                             RW
inactivityTimeout               long                            RW
growable                        boolean                         RW
stats                           javax.management.j2ee.statistics.Stats  RO

Let's also see what type of operations we can invoke on this type of object has by running print Help.operations(tp). This gives the following output...

java.lang.String getName()
int getMaximumPoolSize()
void setMaximumPoolSize(int)
int getMinimumPoolSize()
void setMinimumPoolSize(int)
long getKeepAliveTime()
void setKeepAliveTime(long)
boolean isGrowAsNeeded()
void setGrowAsNeeded(boolean)
javax.management.j2ee.statistics.Stats getStats()

Seems to me that the stats attribute might be interesting... we can take a look with print AdminControl.getAttribute(tp, 'stats')
This will return a horibly formatted PyString simiolar to the one below. Put this through a splash of regex to pull out the stats you need and you're done.

'\n
Stats

name=BPESchedulerWorkManager.Alarm Pool, type=threadPoolModule\n

\n

name=ActiveCount, 
ID=3, 
description=The number of concurrently active threads.,
unit=N/A, 
type=BoundedRangeStatistic,
lowWaterMark=0,
highWaterMark=1, 
current=0, 
integral=10121.0, 
lowerBound=0, 
upperBound=0

\n\n
name=PoolSize, 
ID=4, 
description=The average number of threads in a pool., 
unit=N/A, 
type=BoundedRangeStatistic, 
lowWaterMark=1, 
highWaterMark=10, 
current=1, 
integral=0.0, 
lowerBound=10, 
upperBound=10\n

}'

Friday, 4 October 2013

wsadmin - passing arguments when usng execfile()

I understand the correct way to execute a script with arguments from within wsadmin is to use the subprocess as follows

wsadmin>import subprocess
wsadmin>subprocess.call([sys.executable, 'abc.py', 'argument1', 'argument2'])

However, I can't seem to successfully import the subprocess module

This being the case I'm using the following approach...

wsadmin>import sys
wsadmin>sys.argv = ['argument1', 'argument2']
wsadmin>execfile('abc.py')

Wednesday, 2 October 2013

Getting PMI metrics using wsadmin

Get a PyJavaInstance ref to the thing you want to check (In this case a JVM)

jvmO = AdminControl.makeObjectName(

    AdminControl.queryNames('type=JVM,J2EEServer=BobServerA,*'))

Check it's of type jclass org.python.core.PyJavaInstance

type(jvmO)

Now get it's PerfMBean Object

perfO =  AdminControl.makeObjectName(

    AdminControl.queryNames('type=Perf,process=BobServerA,*'))

Now run getStatObject

jvmStatsO = AdminControl.invoke_jmx(

    perfO,

    'getStatsObject',

    [ jvmO, java.lang.Boolean('false') ],

    [ 'javax.management.ObjectName', 'java.lang.Boolean' ]

Now run the getStatistic method on jvmStatsO to get stats

jvmStatsO.getStatistic('HeapSize').getCurrent()

To work out what other methods you can call either of the following

jvmStatsO.getClass().getMethods()


jvmStatsO.getStatistic('HeapSize').getClass().getMethods()