# Preface

This document describes the functionality provided by the vSphere plugin.

See the XL Deploy Reference Manual for background information on XL Deploy and deployment concepts.

# Overview

The vSphere plugin is an XL Deploy plugin that supports launching, provisioning and terminating hosts and environments on VMWare vSphere platform.

The vSphere plugin is part of XL Scale. For more information about the XL Scale, see the XL Scale Manual.

## Features

- Deploying vSphere templates as virtual machines from XL Deploy;
- Destroying previously deployed virtual machines;
- Combining vSphere virtual machines into environments which can be used for application deployment;
- Registering middleware as part of a created environment.

# Requirements

- XL Deploy requirements

  - XL Deploy: version 4.0+
  - Other XL Deploy Plugins: XL Scale

- Infrastructural requirements

  - VMWare vSphere platform with vCenter 5.1+

# Usage scenarios

This section describes the most common usage scenarios, further sections contain more detailed configuration instructions.

## Creating a single host

In its simplest form, the vSphere plugin can deploy a single virtual machine from a template and register it in XL Deploy as a CI of type `cloud.SshHost` (when connecting to the host using SSH) or `cloud.CifsHost` (when connecting using CIFS). The resulting host CI can contain middleware CIs that are present on the host and can be used as a normal container for deployment. The host can also be destroyed, which causes XL Deploy to terminate the vSphere instance and remove the host CI and its children from the repository.

There is a special CI type `vsphere.HostTemplate` which is used as a template to define all information about the future virtual machine.

## Creating an environment

For more information about combining cloud hosts into environments, see the XL Scale Manual.

## Provisioning instantiated hosts

When deploying a virtual machine, the template may already have the desired middleware installed. If this is the case, a launched host will be ready for use as soon as it has finished booting. It is also possible to provision a host using Puppet, Chef or a shell command after launching it. This is supported via the the notion of a marker file. If the host template specifies a marker file, XL Deploy will poll the launched instance for its presence. When the file is found on the instance filesystem, XL Deploy will conclude the host is up and ready for deployment. The location of the marker file can be configured in the `vsphere.HostTemplate`.

See the section on the marker file in the XL Scale Manual for additional details on the polling process.

Note:

- it is the responsibility of the template to invoke the provisioning process and to create the marker file when provisioning is completed, signaling that the host and middleware are ready for deployment.
- marker files are only supported when the template is based on a Unix family OS.

# Configuration instructions

## vCenter credentials

The vSphere plugin requires access to vCenter in order to perform operations on vSphere platform. These credentials are specified under the Configuration root node in the repository using a `vsphere.Credentials` CI. The CI has a control task `validateCredentials` that can test that the credentials can be used to communicate with vCenter.

## Host template

The next step is to define host template CIs (`vsphere.HostTemplate`). A host template describes a single host that can be launched on the vSphere platform. In addition to the generic host template properties, it allows some vSphere-specific properties.

See CI reference for list of `vsphere.HostTemplate` properties and their meaning.

See the XL Scale Manual for the generic properties.

Here is an example of an vSphere host descriptor:

    <#escape x as x?xml>
      <list>
        <cloud.SshHost id="${hostsPath}/${hostTemplate.name}_${hostAddress}">
          <template ref="${hostTemplate.id}"/>
          <cloudId>${cloudId}</cloudId>
          <address>${hostAddress}</address>
          <#if hostTemplate.privateKeyFile??><privateKeyFile>${hostTemplate.privateKeyFile}</privateKeyFile></#if>
          <#if hostTemplate.username??><username>${hostTemplate.username}</username></#if>
          <#if hostTemplate.password??><password>${hostTemplate.password}</password></#if>
          <#if hostTemplate.os??><os>${hostTemplate.os}</os></#if>
          <#if hostTemplate.connectionType??><connectionType>${hostTemplate.connectionType}</connectionType></#if>
        </cloud.SshHost>
        <www.ApacheHttpdServer id="${hostsPath}/${hostTemplate.name}_${hostAddress}/httpd">
          <host ref="${hostsPath}/${hostTemplate.name}_${hostAddress}"/>
          <startCommand>sudo apachectl stop</startCommand>
          <startWaitTime>3</startWaitTime>
          <stopCommand>sudo apachectl stop</stopCommand>
          <stopWaitTime>3</stopWaitTime>
          <restartCommand>sudo apachectl restart</restartCommand>
          <restartWaitTime>10</restartWaitTime>
          <defaultDocumentRoot>/var/www</defaultDocumentRoot>
          <configurationFragmentDirectory>/etc/apache2/conf.d</configurationFragmentDirectory>
        </www.ApacheHttpdServer>
      </list>
    </#escape>

Every `vsphere.HostTemplate` CI provides a `validateDescriptor` control task which processes the Freemarker template, parses the resulting XML and reports errors if something is wrong. No actual changes are made to the repository during execution of this control task.

Please note that:

- Hosts which you define here should be either `cloud.SshHost` or `cloud.CifsHost`.
- The `cloud.SshHost` or `cloud.CifsHost` in the template must contain the XML fragments for `address`, `cloudId` and `template`. These are needed for the proper functioning of the plugin.
- You should always use `<list>` as a parent XML element, even if you define only one CI.
- When you use references to `${hostTemplate.*}` properties, make sure they are defined on host template when required by the appropriate CI types. E.g. `cloud.SshHost` requires `connectionType`, when `ec2.HostTemplate` does not.
- Since XML is being generated you have to make sure that values are properly encoded. You can achieve this by enclosing the template in `<#escape x as x?xml>...</#escape>`, or alternatively use `${exampleKey?xml}`. See the Freemarker documentation for details.

## Environment template

For more information about defining environment templates, see the XL Scale Manual.

# Using the vSphere plugin

Please see the XL Scale manual for instructions on how to use the environment and host templates provided in the vSphere plugin.

## Upgrade notes

### Upgrading from version 3.9.4 or below

- Before installing the distribution please make sure that following libraries are not present in the `plugins` folder. Remove them if they are.
  - cloud-plugin-*.jar
  - dom4j-*.jar
  - vijava-*.jar
  - vsphere-plugin-*.jar
  - xml-apis-*.jar

## Release notes

### XL Scale plugin version 4.0.0

As a major change, Cloud plugin has been rebranded to XL Scale.

Improvements:

- [DEPL-5489] - How to update in case of connections of type WINRM, WINRM_HTTP, WINRM_HTTPS have been used.

Fixed Bugs:

- [DEPL-4909] - Cloud instances from the same template get the same instance ID

### vSphere plugin version 4.0.0

Fixed Bugs:

- [DEPL-5636] - vsphere.HostTemplate can not be created when host name is not located in datacenter

### Cloud plugin version 3.9.4

Fixed bugs:

- [DEPLOYITPB-5081] - environmentTemplate is not exposed in the template
- [DEPLOYITPB-4864] - Host template validation exposes password (improvement of the fix made in 3.9.3)

### vSphere plugin version 3.9.3

Improvements:

- [DEPLOYITPB-4864] - Host template validation exposes password

# CI Reference

## Configuration Item Overview

### Containers

### Other Configuration Items

## Configuration Item Details

### cloud.BaseHostTemplate

Virtual Type
Interfaces | udm.ConfigurationItem

Base class for instance templates, all instance templates must extend it

|  | Public Properties
--- | --- | ---
|  | bootTimeout : INTEGER = 500
|  | Maximal amount of time (in seconds) allowed to elapse before the instance is ready.
|  | xmlDescriptor : STRING
|  | Freemarker template of XML which describes instance and middleware
|  | connectionType : ENUM [SFTP, SFTP_CYGWIN, SFTP_WINSSHD, SCP, SUDO, INTERACTIVE_SUDO, TUNNEL, TELNET, WINRM_INTERNAL, WINRM_NATIVE]
|  | Connection type to be used for connecting to the host
|  | markerPath : STRING
|  | Path to the file which should appear on the instance when provisioning completes.
|  | os : ENUM [WINDOWS, UNIX, ZOS]
|  | OS family
|  | password : STRING
|  | Password
|  | privateKeyFile : STRING
|  | Private key file to use for authentication
|  | retryDelay : INTEGER = 5
|  | Delay (in seconds) after each connection attempt.
|  | username : STRING
|  | Username

Control task | Parameter CI | Attributes | Description
--- | --- | --- | ---
instantiate | cloud.HostParameters |  | Create instance from template
validateDescriptor |  |  | Validate XML descriptor

### cloud.CifsHost

Type Hierarchy | overthere.CifsHost >> overthere.RemoteHost >> overthere.Host >> udm.BaseContainer >> udm.BaseConfigurationItem
Interfaces | udm.Taggable, udm.ConfigurationItem, udm.Container, overthere.HostContainer

Cloud host with CIFS access

|  | Public Properties
--- | --- | ---
|  | address : STRING
|  | Address of the host
|  | cloudId : STRING
|  | Unique ID within cloud platform
|  | connectionType : ENUM [TELNET, WINRM_INTERNAL, WINRM_NATIVE] = WINRM_INTERNAL
|  | Type of CIFS connection to create
|  | os : ENUM [WINDOWS, UNIX, ZOS] = WINDOWS
|  | Operating system the host runs
|  | password : STRING
|  | Password to use for authentication
|  | template : CI<cloud.BaseHostTemplate >
|  | Template which was used to create this host
|  | username : STRING
|  | Username to connect with
|  | cifsPort : INTEGER = 445
|  | Port on which the CIFS server runs
|  | jumpstation : CI<overthere.Jumpstation>
|  | Jumpstation that should be used to reach this host
|  | pathShareMappings : MAP_STRING_STRING
|  | Mapping from Windows paths to Windows share names, e.g. C:\IBM\WebSphere -> WebSphereShare
|  | port : INTEGER
|  | Port on which the Telnet or WinRM server runs
|  | stagingDirectoryPath : STRING
|  | Directory into which staged files are stored. Will be cleaned up when the task is finished.
|  | tags : SET_OF_STRING
|  | If set, only deployables with the same tag will be automatically mapped to this container.
|  | temporaryDirectoryPath : STRING
|  | Directory into which temporary files are stored. Will be cleaned up when the connection is closed.
|  | winrmEnableHttps : BOOLEAN = false
|  | Enable SSL communication to the WinRM server
|  | winrsAllowDelegate : BOOLEAN = false
|  | Specifies that the user's credentials can be used to access a remote share, for example, found on a different machine than the target endpoint (WINRM_NATIVE only)
|  | winrsProxy : CI<overthere.Host>
|  | Host on which to run the winrs command. Defaults to localhost. (WINRM_NATIVE only)

|  | Hidden Properties
--- | --- | ---
|  | connectionTimeoutMillis : INTEGER = 1200000
|  | Number of milliseconds Overthere waits for a connection to a remote host to be established
|  | protocol : STRING = cifs
|  | Protocol to use when connecting to this host
|  | tmpFileCreationRetries : INTEGER = 1000
|  | Number of times Overthere attempts to create a temporary file with a unique name
|  | winrmContext : STRING = /wsman
|  | Context used by the WinRM server (WINRM only)
|  | winrmEnvelopSize : INTEGER = 153600
|  | Envelop size for WinRM messages (WINRM only)
|  | winrmHttpsCertificateTrustStrategy : ENUM [STRICT, SELF_SIGNED, ALLOW_ALL] = STRICT
|  | HTTPS certifiacte trust strategy for WinRM over HTTPS (WINRM_INTERNAL only)
|  | winrmHttpsHostnameVerificationStrategy : ENUM [STRICT, BROWSER_COMPATIBLE, ALLOW_ALL] = STRICT
|  | HTTPS host name verification strategy for WinRM over HTTPS (WINRM_INTERNAL only)
|  | winrmLocale : STRING = en-US
|  | Locale to use for WinRM messages (WINRM only)
|  | winrmTimeout : STRING = PT60.000S
|  | Timeout to use for WinRM messages in XML schema duration format
|  | tmpDeleteOnDisconnect : BOOLEAN = true
|  | If true, delete the temporary connection directory when the connection is closed
|  | winrmKerberosAddPortToSpn : BOOLEAN = false
|  | Add the port number (e.g. 5985) to the service principal name (SPN) for which a Kerberos ticket is requested (WINRM_INTERNAL only)
|  | winrmKerberosDebug : BOOLEAN = false
|  | Enable Kerberos debug messages (WINRM only)
|  | winrmKerberosUseHttpSpn : BOOLEAN = false
|  | Use the HTTP protocol in the service principal name (SPN) for which a Kerberos ticket is requested, instead of the default WSMAN protocol (WINRM_INTERNAL only)
|  | winrsCompression : BOOLEAN = false
|  | Turn on compression. Older installations on remote machines may not support compression so it is off by default (WINRM_NATIVE only)
|  | winrsNoecho : BOOLEAN = false
|  | Specifies that echo should be disabled. This may be necessary to ensure that user's answers to remote prompts are not displayed locally (WINRM_NATIVE only)
|  | winrsNoprofile : BOOLEAN = false
|  | Specifies that the user's profile should not be loaded. By default the server will attempt to load the user profile. If the remote user is not a local administrator on the target system then this option will be required (the default willresult in error) (WINRM_NATIVE only)
|  | winrsUnencrypted : BOOLEAN = false
|  | Disable encryption. Requires the remote host to have been configured to allow that (WINRM_NATIVE only)

Control task | Parameter CI | Attributes | Description
--- | --- | --- | ---
checkConnection |  |  | Checks whether XL Deploy can transfer files to and execute commands on this host.
destroy |  | delegate = destroyHost | Shut down EC2 instance and remove all related CIs

### cloud.CloudEnvironmentParameters

Type Hierarchy | udm.Parameters >> udm.BaseConfigurationItem
Interfaces | udm.ConfigurationItem

Parameters for cloud environment instantiation

|  | Public Properties
--- | --- | ---
|  | environmentId : STRING = Environments
|  | Id of the environment you want to create
|  | hostsPath : STRING = Infrastructure
|  | Repository location where all created hosts will appear

### cloud.Environment

Type Hierarchy | udm.Environment >> udm.BaseConfigurationItem
Interfaces | udm.ConfigurationItem

Cloud environment

|  | Public Properties
--- | --- | ---
|  | linkedCis : SET_OF_CI<udm.ConfigurationItem>
|  | CIs that were described in the template of this cloud environment and created along with it
|  | template : CI<cloud.EnvironmentTemplate >
|  | Template which was used to create this environment
|  | dictionaries : LIST_OF_CI<udm.Dictionary>
|  | The dictionaries providing placeholder values. If the same entry exists in multiple dicitonaries, the first one in the list is taken.
|  | members : SET_OF_CI<udm.Container>
|  | The infrastructure components of this Environment
|  | smtpServer : CI<mail.SmtpServer>
|  | The SMTP server used to send mails with when deploying to this Environment.

Control task | Parameter CI | Attributes | Description
--- | --- | --- | ---
destroy |  |  | Shut down all related cloud instances and remove all related CIs

### cloud.EnvironmentTemplate

Interfaces | udm.ConfigurationItem

Cloud environment template

|  | Public Properties
--- | --- | ---
|  | hostTemplates : LIST_OF_CI<cloud.BaseHostTemplate >
|  | Host templates
|  | xmlDescriptor : STRING
|  | Freemarker template of XML which describes environment
|  | description : STRING
|  | Description of the template

Control task | Parameter CI | Attributes | Description
--- | --- | --- | ---
instantiate | cloud.CloudEnvironmentParameters |  | Instantiate environment and all hosts, which templates are linked to this environment template
validateEnvironmentDescriptor |  |  | Validate XML descriptor of the environment template

### cloud.HostParameters

Type Hierarchy | udm.Parameters >> udm.BaseConfigurationItem
Interfaces | udm.ConfigurationItem

Parameters for host templates instantiation

|  | Public Properties
--- | --- | ---
|  | hostsLocation : STRING = Infrastructure
|  | Repository location where all created hosts will appear
|  | instanceName : STRING
|  | Name of the instance after creation

### cloud.SshHost

Type Hierarchy | overthere.SshHost >> overthere.RemoteHost >> overthere.Host >> udm.BaseContainer >> udm.BaseConfigurationItem
Interfaces | udm.Taggable, udm.ConfigurationItem, udm.Container, overthere.HostContainer

Cloud host with SSH access

|  | Public Properties
--- | --- | ---
|  | address : STRING
|  | Address of the host
|  | cloudId : STRING
|  | Unique ID within cloud platform
|  | connectionType : ENUM [SFTP, SFTP_CYGWIN, SFTP_WINSSHD, SCP, SU, SUDO, INTERACTIVE_SUDO, TUNNEL] = SFTP
|  | Type of SSH connection to create
|  | os : ENUM [WINDOWS, UNIX, ZOS]
|  | Operating system the host runs
|  | port : INTEGER = 22
|  | Port on which the SSH server runs
|  | template : CI<cloud.BaseHostTemplate >
|  | Template which was used to create this host
|  | username : STRING
|  | Username to connect with
|  | jumpstation : CI<overthere.Jumpstation>
|  | Jumpstation that should be used to reach this host
|  | passphrase : STRING
|  | Optional passphrase for the private key in the private key file
|  | password : STRING
|  | Password to use for authentication
|  | privateKeyFile : STRING
|  | Private key file to use for authentication
|  | stagingDirectoryPath : STRING
|  | Directory into which staged files are stored. Will be cleaned up when the task is finished.
|  | suPassword : STRING
|  | Password of user to su to when accessing files or executing commands
|  | suUsername : STRING
|  | Username to su to when accessing files or executing commands
|  | sudoUsername : STRING
|  | Username to sudo to when accessing files or executing commands
|  | tags : SET_OF_STRING
|  | If set, only deployables with the same tag will be automatically mapped to this container.
|  | temporaryDirectoryPath : STRING
|  | Directory into which temporary files are stored. Will be cleaned up when the connection is closed.

|  | Hidden Properties
--- | --- | ---
|  | connectionTimeoutMillis : INTEGER = 1200000
|  | Number of milliseconds Overthere waits for a connection to a remote host to be established
|  | interactiveKeyboardAuthRegex : STRING = .*Password:[ ]?
|  | Regular expression to look for in keyboard-interactive authentication before sending the password
|  | protocol : STRING = ssh
|  | Protocol to use when connecting to this host
|  | suCommandPrefix : STRING = su - {0} -c
|  | Su command to prefix to the original command. The placeholder {0} is replaced with the sudoUsername
|  | suPasswordPromptRegex : STRING = .*[Pp]assword.*:
|  | Regular expression to look for before sending the password
|  | sudoCommandPrefix : STRING = sudo -u {0}
|  | Sudo command to prefix to the original command. The placeholder {0} is replaced with the sudoUsername
|  | sudoPasswordPromptRegex : STRING = .*[Pp]assword.*:
|  | Regular expression to look for in interactive sudo before sending the password
|  | tmpFileCreationRetries : INTEGER = 1000
|  | Number of times Overthere attempts to create a temporary file with a unique name
|  | allocateDefaultPty : BOOLEAN = false
|  | If true, a default PTY (dummy:80:24:0:0) is allocated when executing a command
|  | allocatePty : STRING
|  | Specification for the PTY to be allocated when executing a command. The format is TERM:COLS:ROWS:WIDTH:HEIGHT, e.g. xterm:80:24:0:0
|  | suOverrideUmask : BOOLEAN = true
|  | If true, permissions are explicitly changed with chmod -R go+rX after uploading a file or directory
|  | suPreserveAttributesOnCopyFromTempFile : BOOLEAN = true
|  | If true, files are copied from the connection temporary directory using the -p flag to the cp command
|  | suPreserveAttributesOnCopyToTempFile : BOOLEAN = true
|  | If true, files are copied to the connection temporary directory using the -p flag to the cp command
|  | suQuoteCommand : BOOLEAN = true
|  | If true, the original command is quoted when it is prefixed with suCommandPrefix
|  | sudoOverrideUmask : BOOLEAN = true
|  | If true, permissions are explicitly changed with chmod -R go+rX after uploading a file or directory
|  | sudoPreserveAttributesOnCopyFromTempFile : BOOLEAN = true
|  | If true, files are copied from the connection temporary directory using the -p flag to the cp command
|  | sudoPreserveAttributesOnCopyToTempFile : BOOLEAN = true
|  | If true, files are copied to the connection temporary directory using the -p flag to the cp command
|  | sudoQuoteCommand : BOOLEAN = false
|  | If true, the original command is quoted when it is prefixed with sudoCommandPrefix
|  | tmpDeleteOnDisconnect : BOOLEAN = true
|  | If true, delete the temporary connection directory when the connection is closed

Control task | Parameter CI | Attributes | Description
--- | --- | --- | ---
checkConnection |  |  | Checks whether XL Deploy can transfer files to and execute commands on this host.
destroy |  | delegate = destroyHost | Shut down EC2 instance and remove all related CIs

### vsphere.Credentials

Interfaces | udm.ConfigurationItem

vCenter credentials

|  | Public Properties
--- | --- | ---
|  | password : STRING
|  | vCenter password.
|  | url : STRING
|  | vCenter URL, e.g. https://vcenter.example.com/sdk
|  | username : STRING
|  | vCenter username.
|  | ignoreCert : BOOLEAN = true
|  | Ignore SSL certificate warnings

Control task | Parameter CI | Attributes | Description
--- | --- | --- | ---
validateCredentials |  |  | Validate credentials by connecting to vCenter.

### vsphere.HostTemplate

Type Hierarchy | cloud.BaseHostTemplate >> udm.BaseConfigurationItem
Interfaces | udm.ConfigurationItem

vSphere instance template

|  | Public Properties
--- | --- | ---
|  | bootTimeout : INTEGER = 500
|  | Maximal amount of time (in seconds) allowed to elapse before the instance is ready.
|  | cpus : INTEGER
|  | Amount of CPUs to be available for the virtual machine.
|  | credentials : CI<vsphere.Credentials >
|  | vCenter credentials
|  | datacenter : STRING
|  | Name of the datacenter.
|  | host : STRING
|  | Name of the host or cluster on which to run the virtual machine.
|  | memory : INTEGER
|  | Amount of memory to be allocated for the virtual machine in megabytes.
|  | retryDelay : INTEGER = 5
|  | Delay (in seconds) after each connection attempt.
|  | templatePath : STRING
|  | Path to the template relative to the datacenter. The datacenter name should not be included in the path.
|  | xmlDescriptor : STRING
|  | Freemarker template of XML which describes instance and middleware
|  | connectionType : ENUM [SFTP, SFTP_CYGWIN, SFTP_WINSSHD, SCP, SUDO, INTERACTIVE_SUDO, TUNNEL, TELNET, WINRM_INTERNAL, WINRM_NATIVE]
|  | Connection type to be used for connecting to the host
|  | customization : STRING
|  | Name of the existing customization specification.
|  | datastore : STRING
|  | Name of the datastore to be used for the virtual machine.
|  | destinationPath : STRING
|  | Path to the folder where the new virtual machine should be created, relative to the datacenter. Leave this field empty if you want to create the virtual machine directly under the datacenter.
|  | markerPath : STRING
|  | Path to the file which should appear on the instance when provisioning completes.
|  | os : ENUM [WINDOWS, UNIX, ZOS]
|  | OS family
|  | password : STRING
|  | Password
|  | privateKeyFile : STRING
|  | Private key file to use for authentication
|  | resourcePool : STRING
|  | Name of the resource pool to be used for the virtual machine.
|  | username : STRING
|  | Username

Control task | Parameter CI | Attributes | Description
--- | --- | --- | ---
instantiate | cloud.HostParameters |  | Create instance from template
validateDescriptor |  |  | Validate XML descriptor