Preface

This document describes the functionality provided by the IBM WebSphere Application Server (WAS) plugin.

Refer to XL Deploy Reference Manual for background information on XL Deploy and deployment concepts.

Overview

The WAS plugin is a XL Deploy plugin that adds capability for managing deployments and resources on an existing WebSphere application server. It offers out of the box support for deploying/undeploying application artifacts, datasources, JMS resources and other Java EE resources and WAS configurations. See the Features section below for details. It can easily be extended to support more deployment options or management of new artifacts/resources on WAS.

Features

• Deploys and undeploys Java EE application artifacts:
  • Enterprise applications (EAR)
  • Web applications (WAR)
  • Enterprise JavaBeans (EJB JAR)
  • Creates and removes Java EE resources:
    • Oracle datasources
    • DB2 datasources (type 2 and type 4)
    • V5 default JMS resources: queues, topics and connection factories.
    • WebSphere MQ JMS resources: queues, topics and connection factories.
    • SIB JMS resources: queues, topics, connection factories and destinations.
• Creates and removes WAS configuration elements:
  • Shared libraries
  • Virtual hosts
• Discovers WAS topologies: cells, nodes, clusters, server, web servers.

Requirements

• XL Deploy requirements
  • XL Deploy: version 4.0.0+
  • IBM WebSphere Application Server Standard and Network Deployment: 6.1, 7.0, 8.0 and 8.5 (Unix and Windows)
  • Other XL Deploy Plugins: None

Usage in Deployment Packages

The plugin works with the standard deployment package of DAR format. Please see the Packaging Manual for more details about the DAR format and the ways to compose one.

The following is a sample deployit-manifest.xml file that can be used to create a WebSphere specific deployment package. It contain declarations for an Ear, an Oracle datasource spec and a couple of JMS resources.

<?xml version="1.0" encoding="UTF-8"?>
<udm.DeploymentPackage version="1.0-was" application="PetClinic">
    <deployables>
        <jee.Ear name="petClinic-1.0.ear" file="petClinic-1.0.ear">
        </jee.Ear>
        <was.OracleDatasourceSpec name="sampleOracleDatasource">
            <jndiName>jdbc/sampleOracleDatasource</jndiName>
            <jdbcProvider>Oracle JDBC Driver</jdbcProvider>
            <username>{{DATABASE_USERNAME}}</username>
            <password>{{DATABASE_PASSWORD}}</password>
            <datasourceHelperClassname>com.ibm.websphere.rsadapter.Oracle10gDataStoreHelper</datasourceHelperClassname>
        </was.OracleDatasourceSpec>
        <was.SibQueueDestinationSpec name="sampleSibQueueDestination">
            <busName>sampleSIBus</busName>
        </was.SibQueueDestinationSpec>
        <was.SibQueueSpec name="sampleSibQueue">
            <busName>sampleSIBus</busName>
            <description>sample sib queue</description>
        </was.SibQueueSpec>
    </deployables>
</udm.DeploymentPackage>

Using the deployables and deployeds

The following table shows the possible containers a deployable can be targeted to and the deployed that will be created as a result of that.

Deployed actions table

The following table shows the actions taken when creating, modifying or destroying a deployed.

Discovering WAS middleware

The WAS plugin can only discover WebSphere CI's via a Deployment Manager. Discovery via Administrative Agents is not supported.

XL Deploy can scan your environment for as far as possible and create Configuration Items in its repository based on the configurations it encounters during the scan. This process is knows as discovery.

The CIs discovered during discovery will help you in setting up your infrastructure in an easy way. However, they need not be complete: some CIs contain properties that can not be automatically discovered, like passwords. These specific kind of properties will still need to be entered manually.

Because discovery is part of the XL Deploy plugin suite, the exact discovery functionality available varies depending on the middleware platforms present in your environment.

The following steps comprehend discovery:

1. Create a CI representing the starting point for discovery (often a Host CI).
2. Start discovery passing this CI.
3. Store the discoverd CIs in the repository.
4. Complete the discovered CIs manually by providing missing needed properties.
5. Add the discovered CIs to an environment.

Note however that the last step of discovery is optional. The discovered CIs will be stored under the Infrastructure root node in the repository and may be added to an environment at some later time.

Create a Configuration Item starting point

The first step taken in discovery is to create a starting point to kick off the process from. This starting point consists of a Configuration Item specifying at least the host that discovery should start at. Depending on the middleware you are trying to discover, additional parameters may be needed.

Following is an example of how to start discovery for a WebSphere Application Server (WAS). First a CI is created for the host itself and next a Configuration Item is created for the deployment manager running on that ost. The deployment manager CI will be the starting point for discovery.

    # Create a CI with the required discovery parameters filled in
    wasHost = factory.configurationItem('Infrastructure/rs94asob.k94.corp.com', 'overthere.SshHost', {
            'address': 'was-61',
            'username': 'root',
            'password': 'rootpwd',
            'os': 'UNIX',
            'connectionType': 'SFTP'
        })

    repository.create(wasHost)

    # Now create a WAS deployment manager
    dmManager = factory.configurationItem(wasHost.id + '/wasDM', 'was.DeploymentManager', {
            'host':'Infrastructure/rs94asob.k94.corp.com',
            'version':'WAS_61',
            'wasHome':'/opt/ws/6.1/profiles/dmgr',
            'username':'wsadmin',
            'password':'wsadmin'})

Note 1: Right now possible values for version are: WAS_61, WAS_70, WAS_80, WAS_85.

Note 2: Last segment of Configuration Item id ("wasDM" in this example) should match your WebSphere cell name.

Start discovery passing a Configuration Item

After the CI starting point has been created, it can be used to perform discovery. The XL Deploy CLI discovery functionality is synchronous, which means that the CLI will wait until the discovery process finishes.

The process of discovery works exactly like a regular task:

deployit> taskId = deployit.createDiscoveryTask(dmManager)
deployit> deployit.startTaskAndWait(taskId)
deployit> discoveredCIs = deployit.retrieveDiscoveryResults(taskId)

Note there are no single- or double quotes around dmManager, because it's an object and not a string. The result of this command will be an object containing a list of discovered CIs.

Store the CIs in the repository

XL Deploy returns a list of discovered middleware CIs. Note that these are not yet persisted. To store them in the repository, use the following code:

    repository.create(discoveredCIs)

Complete discovered middleware CIs

The easiest way to find out which of the discovered CIs require additional information is by printing them. Any CIs that contain passwords (displayed as '********') will need to be completed. To print the stored CIs, the following code can be used:

    for ci in discoveredCIs: deployit.print(repository.read(ci.id));

Note: the created CIs can also be edited in the GUI using the Repository Browser.

Adding CIs to Environments

Middleware that is used as a deployment target must be grouped together in an environment. Environments are CIs and like all CIs, they can be created from the CLI. The following command can be used for this:

    env = factory.configurationItem('Environments/DiscoveredEnv', 'udm.Environment') 

Add the discovered CIs to the environment:

    env.values['members'] = [ci.id for ci in discoveredCIs]

Note that not all of the discovered CIs should necessarily be stored in an environment. For example, in the case of WAS, some nested CIs may be discovered of which only the top-level one must and can be stored.

Store the new environment:

    repository.create(env)

The newly created environment can now be used as a deployment target.

Note: the user needs specific permission to store CIs in the database. See the XL Deploy System Administration Manual.

Extending the WAS plugin

The WAS plugin is designed to be extended through XL Deploy's Plugin API type system and through the use of custom, user defined WAS Python scripts.

The WAS plugin associates Create, Modify, Destroy and Inspect operations received from XL Deploy with WAS Python scripts that need to be executed for the specific operation to be performed. The operation specific script is given a Python object representation of the Deployed that triggered the operation. The script is then executed using wsadmin.

There also exists an advanced method to extend the WAS plugin, but the implementation of this form of extension needs to be written in the Java programming language and consists of writing so-called Deployed contributors, PlanPreProcessors and Contributors.

Please refer to the Customization Manual for a detailed explanation of the type system and advanced methods of customization of plugins.

Extending using XML and Python scripts

The easiest way of extending the WAS plugin is by using XML and Python scripts. In this way no Java code needs to be written. Extending the behaviour of the plugin is done by simply defining the necessary deployables and deployeds for the specific environment. When XL Deploy starts up, it reads a file called synthetic.xml from the server class-path, i.e. the ext directory of the server.

This file contains the type definitions of the deployables and the deployeds, i.e. the information about the types and their properties. In addition to that, it defines which Python scripts should be executed for a particular operation, as mentioned above. The scripts have all the information from the Deployed at their disposal to perform their work.

As an example, this is the type definition of a virtual host as it appears in synthetic.xml:

<type type="was.VirtualHost" extends="was.Resource" deployable-type="was.VirtualHostSpec" container-type="was.Cell">
    <generate-deployable type="was.VirtualHostSpec" extends="was.Deployable" />
    <property name="createScript" default="was/virtualhost/create-virtual-host.py" hidden="true" />     
    <property name="destroyScript" default="was/virtualhost/destroy-virtual-host.py" hidden="true" />
    <property name="inspectScript" default="was/virtualhost/inspect-virtual-host.py" hidden="true" />   
    <property name="aliases" kind="set_of_string" description="Virtual host aliases - enter alias as: hostname:port" />
</type>

Looking at this type definition more closely, it indicates that a virtual host, the type of which is was.VirtualHost will be created on its target infrastructure, called container, with a container-type of was.Cell. On WAS a virtual host may be created on a cell and that is exactly what this definition means.

The attribute extends tells the plugin what resource is extended by this definition. In this case it's a simple basic resource so it extends the type was.Resource. Since multiple virtual hosts may be created this way, each with its own set of properties, a specification of what will be deployed is created, a so-called deployable-type, and is itself of type was.VirtualHostSpec.

Within the type definition, there is the possibility of specifying properties of exactly how the virtual host is to be created. The first property is called createScript and specifies the script to be executed by wsadmin for the creation of the virtual host. An extension of this plugin could specify a different creation script. The plugin comes with a default creation script (createScript):

create-virtual-host.py

import re

pattern = re.compile('^[^:]+:\d{,5}')
virtualHostParent = AdminConfig.getid('/Cell:%s/' % (deployed.container.cellName))
attributes = [['name', deployed.name]]
attributes.append(['aliases', [[['hostname', alias.split(':')[0]], ['port', alias.split(':')[1]]] 
                    for alias in deployed.aliases if pattern.match(alias) != None]])
re.purge()

print "Creating virtual host %s on target scope %s with attribute(s) %s" % (deployed.name, virtualHostParent, attributes)
AdminConfig.create('VirtualHost', virtualHostParent, attributes)

This script also shows that aliases are also created for the specified virtual host. Aliases may be specified using the property aliases and it takes a set of strings. An example of this property is:

<property name='aliases' kind='set_of_string' value='www.my-domain.com:80,www.proxy-domain.com:8443'/>

When the script executes a predefined variable deployed has a reference to the deployed that is being deployed.

The script executed on the host is created by appending a number of library scripts and adding the script from the type last. Per default you will get the following runtime scripts added:

• From the Python plugin: python/runtime/base.py
• From the WAS plugin: was/runtime/base.py
• From the deployed itself scripts defined in the libraryScripts hidden property.

In addition to a creation script, a destruction script (destroyScript) must also be specified:

destroy-virtual-host.py

virtualHostContainmentPath = '/Cell:%s/VirtualHost:%s' % (deployed.container.cellName, deployed.name)
virtualHostId = validateNotEmpty(AdminConfig.getid(virtualHostContainmentPath),
"Cannot find virtual host with id: %s" % (virtualHostContainmentPath))

print "Destroying virtual host %s" % (deployed.name)
AdminConfig.remove(virtualHostId)

Also for the destroy operation a predefined variable deployed is available with the deployed being destroyed.

Finally, a modification script (modifyScript) may also be specified. If that is not present, the destruction script is invoked to remove the resource with the old settings and then the creation script is invoked to create the resource with the new settings.

The modify script also has access to the deployed variable.

Inspection

One thing left to be mentioned is the property called inspectScript. When the Domain manager of WAS is known, it is possible for XL Deploy to discover on its own most of the WAS topology automatically. This specific script is called upon by XL Deploy when it's trying, in our example, to discover a virtual host on a WAS cell. Following is a simplified script implementation shipped with the WAS plugin:

inspect-virtual-host.py

virtualhosts = AdminConfig.list('VirtualHost')
if virtualhosts != "":
    for virtualhost in virtualhosts.splitlines():
        if virtualhost.find('/nodes/') == -1:
            virtualHostName = AdminConfig.showAttribute(virtualhost, 'name')
            virtualHostId = '%s/%s' % (container.id, virtualHostName)
            discovered(virtualHostId, 'was.VirtualHost')
            virtualHostContainmentPath = '/Cell:%s/VirtualHost:%s' % (container.cellName, virtualHostName)
            virtualHostAliases = AdminConfig.getid(virtualHostContainmentPath + '/HostAlias:/').split()
            inspectedProperty(virtualHostId, 'aliases', [
                AdminConfig.showAttribute(a, 'hostname') + ':' + AdminConfig.showAttribute(a, 'port') 
                for a in virtualHostAliases if a != ""])
            inspectedItem(virtualHostId)

An inspection script generally consists of a number of steps:

• Collect information from WebSphere
• Indicate to XL Deploy that an object with a certain configuration item ID and a certain type was detected. This is achieved by calling discovered with the configuration item ID that the object has to get inside XL Deploy and the type.
• Set the properties on the object. This is done by repeatedly calling inspectedProperty. inspectedProperty takes three parameters, the ID of the configuration item being discovered, the name of the property and the value. The value should be representable as either a string, or a list of strings.
• When the complete object has been processed this is indicated to XL Deploy by calling inspectedItem with the configuration item ID.

The discovered, inspectedProperty and inspectedItem functions are defined in the Python plugin's python/runtime/base.py. This file also contains some additional helper functions.

Discovery starts at the Cell level. If you need to inspect other levels of the hierarchy you need to traverse the children of the Cell yourself. For convenience a function findAllContainers is defined in the runtime script runtime/base.py.

When the discovery script executes two predefined variables are available:

• prototype contains a 'prototype' deployed which can be used to get the type of the CI being discovered.
• container contains the Cell.

More information about the process of discovery and how to use it can be found in the Command Line Interface (CLI) manual.

Adding a property

The architecture of the WAS plugin enables the tranfer of properties from the deployed - was.VirtualHost in this example - to the accompanying Python scripts defined with the properties createScript and destroyScript. In order for this to be possible, the properties are bound to an object called deployed and can be accessed as deployed.<property-name>. This can be seen in the creation script, where the property aliases is available as deployed.aliases and the name of the virtual host as deployed.name. Using this convention as many properties as needed by the scripts can be bound to the type definition. Care should be taken that the scripts use the same type as specified in the definition, so if a property defines a kind=integer, the script should also treat the value of this property as being of type integer.

An example of adding another property is:

<property name='index-range' kind='integer' value='999' description='maximum index of an array of 1000 items'/>

Fixing a property

In the case where it is necessary that a specific property should always contain a fixed predefined value, the attribute hidden=true should be used on the definition of the property. This way an end-user, who is performing the deployment using the XL Deploy user interface, won't be able to see this property and therefore not have the possibility of modifying it. Usually this is done for the Python scripts which once written are not allowed to be changed by an end-user.

Using the example of the previous section and wanting to fix the maximum range, the example becomes:

<property name='index-range' kind='integer' value='999' hidden='true' description='maximum index of an array of 1000 items'/>

Execution order

A deployment process consists of a series of steps that are executed sequentially. Plugins offer the ability to influence the order of execution of the steps contributed to the deployment process in relation to other contributed steps and operations, not necessarily contributed by the same plugin(s), that are part of the deployment process.

The order of execution allows for the chaining of scripts or operations to create a logical sequence of events. In order to specify the order, properties with the name of createOrder and destroyOrder may be used with an attibute called default to specify the order ordinal. For example, the following synthetic.xml snippet says that creation of the virtual host, default=60, will happen before any step with a higher order, for example default=70, but after any step with a lower order, i.e. lower than default=60.

<type type="was.VirtualHost" extends="was.Resource" deployable-type="was.VirtualHostSpec" container-type="was.Cell">
    <generate-deployable type="was.VirtualHostSpec" extends="was.Deployable" />

    <property name="createScript" default="was/virtualhost/create-virtual-host.py" hidden="true" />
    <property name="createVerb" default="Deploy" hidden="true" />
    <property name="createOrder" kind="integer" default="60" hidden="true" />

    <property name="destroyScript" default="was/virtualhost/destroy-virtual-host.py" hidden="true" />
    <property name="destroyVerb" default="Undeploy" hidden="true" />
    <property name="destroyOrder" kind="integer" default="30" hidden="true" />

    <property name="inspectScript" default="was/virtualhost/inspect-virtual-host.py" hidden="true" />   
    <property name="aliases" kind="set_of_string" description="Virtual host aliases - enter alias as: hostname:port" />
</type>

Notice that the destroyOrder has a low order, because when executing a deployment the virtual host should be destroyed before it can be created again.

Extending the plugin with custom control task

The plugin has the capability to add control tasks to ExtensibleDeployed or PythonManagedContainer. The control task can be specified as a python script that will be executed using wsadmin on the target host or as an OS shell script that will be run on the target host. The OS shell script is first processed with FreeMarker before being executed.

Creating a python script control task to test datasources

Synthetic.xml snippet

<type-modification type="was.Datasource">
  <method name="testDatasource" script="was/resources/ds/test-ds.py" language="python"/>
</type>

test-ds.py snippet

datasource = AdminConfig.getid("%s/JDBCProvider:%s/DataSource:%s/" %
 (deployed.container.containmentPath, deployed.jdbcProvider, deployed.name))
if datasource == '':
  print "WARN: No JDBC DataSource '%s' found. Nothing to do" % (deployed.name)
else:
  print "Testing JDBC DataSource '%s' (config ID '%s')" % (deployed.name, datasource)
  AdminControl.testConnection(datasource)

Creating an os script control task to start the DeploymentManager

Synthetic.xml snippet

<type-modification type="was.DeploymentManager">
  <method name="start" script="was/container/start-dm" language="os"/>
</type-modification>

start-dm.sh snippet for *nix

${container.wasHome}/bin/startManager.sh

start-dm.bat snippet for windows

${container.wasHome}\bin\startManager.bat

Release notes

WAS plugin 4.0.0

New features

WAS plugin 4.0.0

Improvement

• [DEPL-5484] - Extend was.WmqQueue type

Bug fixes

• [DEPL-5445] - was.MailSession, was.MailProvider are destroyed in the wrong order

WAS plugin 3.9.0

New features

• [DEPL-3275] - Add support for generic JMS CIs
• [DEPL-3276] - Additional JDBC providers and data sources
• [DEPL-3277] - Add support for CustomService
• [DEPL-3278] - Add support for ThreadPool
• [DEPL-3279] - Add support for ListenerPort
• [DEPL-3280] - Add support for ServerPort
• [DEPL-3281] - Add support for WorkManager
• [DEPL-3282] - Add support for ResourceEnvironmentEntry
• [DEPL-3283] - Add support for ServletCacheInstance
• [DEPL-3287] - Add support for StringNamespaceBinding
• [DEPL-3288] - Add support for Variable
• [DEPL-4248] - Add support for deploying to dynamic clusters
• [DEPL-4249] - Add support for creating / modifying / setting mode of dynamic cluster via control tasks
• [DEPL-4250] - Add support for discovering dynamic clusters
• [DEPL-4251] - Add support for service policies
• [DEPL-4252] - Add support for deployment of health policies
• [DEPL-4254] - Add support for deploying editioned applications
• [DEPL-4255] - Add support to create / modify dynamic clusters of ODRs via control tasks
• [DEPL-4257] - Add support for discovery of dynamic clusters of ODRs
• [DEPL-4316] - Add support for Resource Environment Providers
• [DEPL-4879] - Add support for MailSession and MailProvider
• [DEPL-5028] - Add support for SiBus
• [DEPL-5032] - Add support for Keystore, CopiedKeystoreFile and SSLConfig

Improvements

• [DEPL-3285] - Improve support for JaasAliases, push changes to runtime automatically
• [DEPL-4165] - Improvements to was.WmqQueue
• [DEPL-4167] - Improvements to was.WmqQueueConnectionFactory
• [DEPL-4666] - Add ability to control WAS container and deployeds
• [DEPL-5031] - Add sharedLibraryNames property (LIST_OF_STRING) to was.Module
• [DEPL-5142] - Make JDBC provider and datasource scripts idempotent

Bug fixes

• [DEPL-2757] - Topology discovery fails when there are multiple servers defined
• [DEPL-3695] - Discovery says "Inspecting cell ..." twice
• [DEPL-3735] - Connection test fails when deploying an Oracle datasource
• [DEPL-4281] - Incorrect default for WAS cell port in description
• [DEPL-4350] - Start application step fails - hangs in an infinite loop
• [DEPL-4409] - Make stop, start script on was.Module not required
• [DEPL-4609] - Only WAR's should be mapped to web servers, not EJB's
• [DEPL-4910] - JAAS alias script will fail if multiple Security objects exist
• [DEPL-5006] - Discovery fixes for data sources, JDBC Providers, MQ Queue connection factory, MQ Queue
• [DEPL-5221] - Duplicate shared library check is too broad

Possible breaking changes / different behaviour

The various data source, JDBC provider and JAAS alias scripts were made idempotent. This means they will be more robust against objects already being created. Objects found will be made to look according to what is specified in XL Deploy. If you have local changes to the scripts for these types you will have to verify if your changes are still necessary or work.

A JAAS alias will now be propagated to runtime in an additional step. No server restart needed. This behaviour also applies to data sources were a username and password is specified (and a JAAS alias is created under the hood). As a result the test connection functionality will now work with a newly deployed JAAS alias.

A JAAS alias generated by was.JaasAlias or was.DataSources will now always target Global Security. Previously it would fail if multiple Security objects existed.

Compared to the wasx.ListenerPort type the was.ListenerPort does not support deploying to a Cluster.

Known issues

• The DB2 JDBC Provider and data source types do not work on iSeries DB2

WAS plugin 3.8.3

Re-release of version 3.8.2 to account for some build infrastructure changes. Functionally identical to version 3.8.2.

WAS plugin 3.8.2

Improvements

• [DEPL-3771] - Support existing WAS plugin functionality on WAS 8.5
• [DEPL-4285] - Documentation improvements

Bug fixes

• [DEPL-2563] - The startup order of deployed applications is set to -1 if no value is specified
• [DEPL-2737] - Cannot deploy to a WAS cell that has two servers with the same name on different nodes
• [DEPL-3047] - Can create two shared libraries with the same name in the same scope
• [DEPL-3059] - Can create two JDBC providers with the same name in the same scope
• [DEPL-3426] - Success (return code) of starting app on WAS cluster depends only on the last server
• [DEPL-3882] - Create datasource script does not obtain the global security object correctly

WAS plugin-3.8.1

Improvement

• Improved useability of discovery

WAS plugin-3.8.0

Improvements

• In-depth discovery of WebSphere topologies