Elastic Batch Platform

Introduction to EBP

This is part of the Heirloom Computing Elastic Batch Platform forum on

The Heirloom Computing Elastic Batch Platform (EBP), or Job Entry Subsystem (JES), is a cloud-centric batch job execution and operating environment. EBP  is a business process manager in the form of an IBM JES2 subsystem equivalent, complete with MVS Job Control Language (JCL) compatible interfaces, a RESTful Web Services and XML control interface. When coupled with the Heirloom Platform-as-a-Service (Heirloom PaaS) administration dashboard, a cloud based scheduler front-ends many cloud-resident EBP systems allowing jobs to be fanned out to a pool of cloud execution engines.

EBP can also operate on a private cloud, server farm, backoffice server or even a desktop environment. EBP operates under Java EE servers such as Oracle WebLogic, IBM WebSphere, Red Hat Wildfly (JBOSS) and Apache Tomcat on Window, UNIX or Linux based environments.  It can also be started from the command line in "stand-alone mode" thanks to a built-in Eclipse Jetty servlet engine.

EBP supports a subset of the MVS JCL for zOS as defined by Heirloom JCL and the concepts of IBM JES2 including the following:

  • Job Classes which define specific Heirloom PaaS resources that are available for batch programs
  • Job Initiators which define a batch window and address concurrency issues
  • Input Job Queue where submitted and scheduled batch jobs await execution
  • Output Spool where status and held output datasets await retrieval and analysis following a job's execution
  • Job Groups that define interrelationships among jobs that drive the scheduling of the job's execution
  • Output Spoolers output classes and programs that can print, hold, or discard job output datasets
  • Network Job Entry in which batch jobs can be submitted for execution via a RESTful Web services interface
  • Remote Job Entry in which batch jobs can be submitted for execution via other jobs (internal reader)
  • JCL Engine executing a batch job consisting of jobs, steps, data definition assignments, inline and cataloged procedures and control logic that execute Elastic COBOL or other executable programs
  • JCS Engine executing a batch job consisting of IBM VSE style jobs, steps, assignment statements, extents (JCS mode JCL), with job class types and initiators for them
  • JEC Job Scheduler supports the latest IBM Job Execution Control introduced in August, 2015 to manage parallel and dependent jobs
  • Script Engine Batch jobs consisting of Shell scripts or Python scripts are accepted and managed by EBP in the same manner as JCL jobs
  • Program Engine provides a mechanism for command line programs, such as COBOL or Java, can be managed by EBP so that installations that do not require the complexity of JCL can nonetheless run overnight background command line invoking Elastic COBOL
  • IMS-DB supported in EBP and ETP
  • Utility Programs such as flat-file, partitioned dataset and record-sequenced key dataset manipulations
  • Cloud Scheduler allowing multiple EBP nodes (an "EBP-Plex") installations to be coordinated from a central point and work dispatched to servers on a timely basis using the Elastic Scheduling Platform (ESP)

Access the Elastic Batch Platform through the scheduling facilities of the Heirloom PaaS Dashboard at


Central to the EBP system is the statement-oriented language used to control the execution of batch jobs and the standard dataset utilities that may be invoked (along with user programs) by jobs specifications conforming to IBM's Job Control Language, JCL. EBP also supports an IBM variant called JCS available on other systems as well as the definition of Job Groups through the use of Job Execution Control, JEC.  The following documents define the syntax accepted by the subsystem:

Dataset Name Mapping

Datasets are central to JCL, they're used for program storage, cataloged procedures and data.  In JCL datasets are referenced as 5-part names, each of 8 charcters each with an optional qualifier for partitioned dataset member or generation data group (GDG).  Here are some examples of JCL datasets:

  • MNH00.MYGDG.DAT(+1)
  • MNH00.MYGDG.DAT(G019V00)
EBP datasets are represented as directories and files within directories.   For example, the absolute GDG name will be a file within a directory represented by the dataset name and the relative GDG references will map to the appropriate absolute name.  EBP will evaluate a dataset name and find the best match based on the directory structure.  The directory structure is rooted on a series of paths that are specified during configuration.  There are 3 configuration paths:
  • datalib - location of data files referenced on DD cards within job steps
  • systemlib - location of executable programs invoked through job step EXEC cards
  • jcllib - location of cataloged procedures referenced in JCLLIB datasets names or INCLUDE statements, both system and user generated
  • classlib - location of class libraries paths (.jar files or directories with .class files) that will be placed on the class path of invoked programs
Each library path is specified as a distinct directory entry in the EBP Configuration Settings .  For example, datalib is initially defined in these settings as 2 separate directories:
  • ebp.datalib.1=/data
  • ebp.datalib.2=d:\data
Meaning that first directory searched for datasets named on DD cards will be "/data".  If that doesn't exist (e.g. on a Windows system), then "D:\data" is searched next.  If the D drive doesn't exist on the Windows sytem a JCL error will prevent the job from running with a "DATASET NOT FOUND" ABEND.  You can configure any number of datalib.1 through datalib.n entries for EBP to search.  New datasets (DISP=NEW) are created in the first datalib that has the permissions to do so (see Tape Handling below for storage class and unit extension).
The DSNs on DD cards are looked up in the datalibs on a directory tree basis where the "." is a directory separator.   DSNs may be translated to lower case as well.  Two configuration settings influence this translation:
  • ebp.newdsndirectory - look to translate DSNs to directory trees first if yes and flat files if no (default no)
  • ebp.newdsnlower - look at lower case directories and files first if yes (default no).
All dataset-to-directory mappings are searched directory tree order if newdsndirectory is yes then in file order. A DSN of "AAA.BBB.CCC" would look first to find "AAA/BBB/CCC", then "AAA/BBB.CCC" and finally "AAA.BBB.CCC".  A DISP=NEW would create the file "AAA/BBB/CCC".  Couple that with datalib.1 as "/data1" and datalib.2 as "/data2" results in a search order of "/data1/AAA/BBB/CCC", "/data1/AAA/BBB.CCC", "/data1/AAA.BBB.CCC", "/data2/AAA/BBB/CCC", "/data2/AAA/BBB.CCC" and "/data2/AAA.BBB.CCC".   Directory read/write permissions come into play as well.  The first directory tree that allows creation of files and/or directories will determine the DISP=NEW dataset location.  If "/data1" was write-protected, a DISP=NEW DSN would be created as "/data2/AAA/BBB/CCC."  Reversing the lookup with newdsndirectory=no while creating directories corresponding to the high-level qualifiers for certain users (e.g., "MNH00") would result in a DSN of "MNH00.ACCT.DAT" to be created as "/data2/MNH00/DATA.DAT".

Tape Handling and Unit Specification

Tapes are regular files in EBP, but configuration settings may be used to allocate and manage them on potentially slower or cheaper media.  Using VOL=SER=TP451,LABEL=(2,SL) would allocate a file named TP451.L0002 in one of the datalib directories.  Using UNIT=TAPE (or DATACLAS=xxxx, MGMTCLAS=xxxx or STORCLAS=xxxx) on DISP=NEW datasets (or VOL=SER=serial numbered tapes) will look for datalib configurations that include that unit name or storage class.  For example, UNIT=TAPE and datalib.9=/data/tape will restrict the dataset allocation to this directory.  Using multiple datalibs, each referencing both storage class and unit names can segment allocation into certain areas.  Consider the following datalibs used for allocating tape datasets:

  • datalib.1=/data/sysda
  • datalib.2=/data/nfsmount-3350-1
  • datalib.3=/data/nfsmount-3350-2
  • datalib.7=/data/lwrclass/tape/3490
  • datalib.8=/data/lwrclass/tape/3420
  • datalib.9=/data/uprclass/cart/3880

would allow DD names to be allocated on the various classes

//DD1 DD DISP=NEW,DSN=HCI00.MY.DAT       allocate /data/sysda/HCI00.MY.DAT
//DD7 DD DISP=NEW,UNIT=TAPE,VOL=SER=TPS431,LABEL=(1,SL)  allocate /data/lwrclass/tape/3490/TPS431.L0001
//DD8 DD DISP=NEW,UNIT=3420,VOL=SER=TPS432,LABEL=(2,SL)  allocate /data/lwrclass/tape/3420/TPS431.L0002
//DD9 DD DISP=NEW,STORCLAS=UPRCLASS      allocate /data/uprclass/cart/3880/SYSEBP.TEMP2345.T1234567.DAT

Note:  UNIT=SYSDA does not require that the datalib contain "sysda", but those that do are given preferential ordering ahead of others.

RETPD and EXPDT DD parameters and LABEL subparameters work as expected.  Run the standard utility TMSCLEAN to scratch expired datasets and volumes.  Using VOL=(PRIVATE,,,) or VOL=SCRATCH will allocate scratch serial numbered tapes.  See Examples.

Unit specification also implies certain sizes that are calculated and checked during new dataset allocation.  The underlying datalib device must have enough disk space to hold the given allocation using traditional IBM device type specifications even though the space is not allocated until the COBOL program requires it.  If there is not enough space the datalib will be rejected and the next datalib will be examined for the required space.  If no datalib contains enough free space then the job is aborted.  SPACE required is calculated from the traditional UNIT sizes as shown in the following table,

UNIT BLK (bytes)
TRK (bytes)
CYL (trks)
3350 * 19069 30
3380 * 47476 15
3390 * 56664 15
9345 * 46456 15
SYSDA * 100000 10

* - calculated from block size specified in the dataset DCB

Examples of calculating SPACE and UNIT together,

//DD1 DD DISP=NEW,DSN=HCI00.MY1.DAT,UNIT=3350,SPACE=(TRK,(10,10))  allocate if 190KB available
//*                                                                on /data/nfsmount-3350-1 or -2
//DD2 DD DISP=NEW,DSN=HCI00.MY2.DAT,UNIT=SYSDA,SPACE=(CYL,(10,10)) allocate if 10MB available on any datalib
//DD3 DD DISP=NEW,DSN=HCI00.MY3.DAT,SPACE=(CYL,(100,7))            allocate if 100MB available on any datalib
//DD4 DD DISP=NEW,DCB=BLKSIZE=1000,SPACE=(BLK,(5,5))               allocate if 5000bytes available

JCL, Program and Scripts

EBP is capable of executing JCL jobs but also jobs consisting of Python or Shell script code and COBOL or Java programs directly, without the need for them to be contained in JCL.  Specify the job type when defining the EBP job class.  Jobs of all types can be submitted to EBP through the submit Web service or through the internal reader from other JCL jobs.  Submit COBOL or Python jobs by submitting a Java archive (jar) file.  Where possible, EBP will look for hints in a "job card" on the first comment line of a script for information regarding classification.  For example,

# Python script
import re;

Job Class Resource Limits

Job classes are defined to EBP through the define Web service.  A number of resource limits may be specified on the class:

  • Maximum CPU execution time
  • Maximum elapsed (wall-clock) time
  • Maximum memory (region) used for each step
  • Total file sizes created during step execution
  • Total network transmission allowed during the job

The TIME keyword on the JOB card will help EBP determine its class if not specified explicitly on the JOB card.  Elapsed time restrictions are checked regardless, but maximum CPU and other resource limits require checking by a command outside the Java Virtual Machine environment.  Set the ebpstartcommand configuration parameter to a command that can interpret these configuration settings and start an arbitrary command under resource control.  Although not initially configured (i.e., jobs will not ABEND if their steps take too long), ebpstartcommand may be set to any of the available commands in the bin directory within the webapps folder containing EBP.  For example, on Linux 64-bit systems configure ebpstartcommand=bin/ebpstart_linuxx86_64.

Checkpoint / Restart

EBP supports the notion of checkpoint / restart initiated through Web services.  The checkpoint Web service will suspend a running job at its currently executing step and place it in a suspended state.  Dataset locks are kept, the temporary spool (working directory of a job), its output spool (held output datasets) and the input spool (input job) are retained. 

Suspended jobs have their intermediate MSG and ouptut spool datasets available for listing.

A subsequent restart Web service for a job ID will place a suspended job into the queued state where, when an initiator of that class becomes available, it will resume execution at the beginning of the step that was executing at the time of suspension.  The restarted will rescan the JCL and reprocess any symbolic parameters that may be in effect at the time of resumption.  The MSG dataset will reflect both the job execution before and after the restart.

On some systems (e.g., Linux) it is possible to not only restart a job at the beginning of the a step but restart the program that was executing at the point of its last checkpoint.  Such programs restart in the middle of their processing.  Support relies on the availability of an external checkpoint/restart facility, such as the Distributed Multi-Threaded CheckPointing (DMTCP).  Such process checkpointing requires additional EBP configurations to take advantage of these features and these are done on a per-job-class basis.  Those classes whose jobs are eligible for DMTCP checkpoint/restart have the checkpointcommand-classname and restartcommand-classname configurations set to the command sequence needed to carry out process-level checkpoint (which produces a ckpt file at regular intervals) and later restart.  For example, if Job Class R is defined for restartable jobs, the two configurations might be:

checkpointcommand-R: rm ckpt_*;/usr/local/bin/dmtcp_launch --ckpt-open-files
restartcommand-R:/usr/local/bin/dmtcp_restart *.dmtcp

For each step in class R jobs EBP erases all prior checkpoint files created by DMTCP (ckpt_*.dmtcp) and then the command to invoke the step's program (e.g., a standard utility or COBOL application) is pre-pended with the dmtcp_launch command.  And if a job step is coming out of a suspended mode, the dmtcp_restart command will be executed instead of the restarted job step command and given the corresponding ckpt_*.dmtcp file.

A DMTCP coordinator must be started and listening for launch commands and a checkpoint interval must be in place to create checkpoint files every so often.  Start the coordinator with a command such as:

/usr/local/bin/dmtcp_coordinator -i 20

There are other dmtcp_launch and dmtcp_coordinator options that control other checkpointing parameters.  See DMTCP documentation for more info. 

Output Spoolers

EBP supports output classes named A through Z, 0 through 9 as well as DEFAULT such that SYSOUT DD assignments and MSGCLASS designations can define the corresponding output datasets (1) be held in the output spool for later retrieval, (2) designate automatic printing to defined printers, (3) redirect to Java log4j logging systems, (4) sent to arbitrary spool programs, (5) or discarded entirely.  Define the output classes through the EBP Configuration service with the following values:

Output Classes
Output Class Default Description
DEFAULT hold Hold output in output spool
L log Send to Java log4j logging system
P lpr Send to the CUPS printing system
Y hold Hold output in output spool
Z none Discard output
All others hold Same as DEFAULT

Set additional parameters to lpr or define new spooling programs by setting its output class descriptors to a command name optional parameters, including redirection parameters.  For example, output class L is changed to send to a remote printer and output class S is defined to append all output to a file with this configuration:

output.L=lpr -P node22/printer44 -C EBP -T Job-Output
output.S=cat >>/tmp/alloutput.txt


EBP integrates with the supporting J2EE Application Server or Servlet Container to provide three layers of security:

  • Transport Level Security - ensuring data between EBP and the clients is secure
  • Authentication - validating who can use EBP
  • Authorization - indicating who can use what services against which objects

See Operating EBP in a Secure Environment for detailed information and examples for EBP, container and security subsystem configuration.

Once the EBP and container has been set up to provide the three layers of security you add users to the container's security environment and add them to roles.  EBP classifies operations in terms of roles, users and administrators.  By default these role names are EBP-user and EBP-admin but they can be changed to group names you may have defined in your mainframe security system, should you wish to continue to use these.  Users submit jobs, administrators change configuration.  Note that administrators don't ordinarily have the rights for user operations -- the ability to submit jobs.  But, you can define users that take on multiple roles in most server environments. This allows you to define the same groupings that are common in mainframe resource access control environments such as  RACF, TopSecret or ACF2.  In many environments you can configure EBP to use the same RACF system for authentication and authorization using its lightweight directory access protocol (LDAP) facility.  Or, you can extract users and groups from the mainframe system and import them into an interchange format (LDIF) to be imported into local directory systems.

When configured appropriately the EBP Web services (submit, configure, purge, etc.) are protected by the security system and jobs will run under authenticated users.  Generally, the EBP index.html home page and the sysoper.html System Operators console and their components can be allowed to be accessed by all users.  But, your installation may set up other logon pages that set the security contexts before any part of EBP becomes visible.

When security is turned on EBP will enforce a cascading set of ownership of jobs.  The authenticated user (in the EBP-user role) will become the owner of jobs submitted by the submit Web service.  Listing the status of jobs will include that user name in the output, visible on the System Operators Console.  When JCL jobs are run by EBP the &SYSUID symbolic parameter is set to the upper-case representation of the user (&sysuid is the lower case representation). Only that same user or one with both an EBP-user and EBP-admin role may cancel, purge, checkpoint or restart that job.  Further, any jobs submitted to the internal reader either in JCL or through the COBOL API will take on the same user as that which submitted the original job.

The users authenticated by the EBP container will override the USER keyword that may be present on the JOB card. the USER and PASSWORD job card keywords are not used to authenticate against, for example, OpenLDAP.  However, the USER and PASSWORD entry on the job card will still be used to authenticate access to resources such as with database connections performed by DSNTIAD and IKJEFT01.  Note that EBP cannot learn LDAP passwords in order to pass them to database connections.  However, JNDI naming and JDBC datasources can be established to allow the container to validate database connections used by job steps.

IMS DB Support

IMS-DB is supported in EBP with utilities such as DBDGEN and PCBGEN as well as the ability to run IMS DB Batch applications with the DFSRRC00 utility.


The IBM VSE Job Control Subsystem and its flavor of JCL called JCS is fully supported.  Define a JCS job class and start initiators.  Then submit JCS decks with the SUBMIT web service.

The following defines the syntax accepted by the subsystem:


IBM introduced Job Execution Control with z/OS JES2 Version 2, Release 2 (August, 2015).  JEC allows the definition of interrelationships among jobs.  JEC is fully supported within EBP.  Define a JEC job class and start initiators in order to SUBMIT JEC definitions of JOBGROUPs. 

Also introduced is the new SCHEDULE JCL statement.  Jobs can be given instructions on when to start (SCHEDULE HOLDUNTL parameter), indicate that they must begin by a certain time and date (SCHEDULE STARTBY parameter), that they should be initiated at the same time as another job (SCHEDULE WITH parameter) and conform to the more advanced scheduling criteria defined in a JEC JOBGROUP (SCHEDULE JOBGROUP parameter).

The following defines the syntax accepted by the subsystem:

Installing/Starting/Stopping EBP

The Elastic Batch Platform operates under a Java EE server or the simpler servlet management system such as Apache Tomcat. Install Tomcat on Windows, UNIX, Linux (Heirloom PaaS managed clouds have EBP pre-installed and configured), then deploy the EBP system (ebp.war) by using the server's administration interface or copying the deployment file into the webapps directory.  See the Announcements forum for the latest EBP news, downloads and updated installation instructions.  See JBOSS or Weblogic for special installation instructions for those systems.

The EBP subsystem is a licensed program product of Heirloom Computing, Inc. Ensure that the registration file for your account is installed in either (1) the same webapps/ebp folder where EBP has been deployed, or (2) the home directory of the user under which the application server operates (e.g. /home/apache on Linux Tomcat systems operating under that user id).

Start your application server or servlet management server prescribed by the manufacturers documentation. If your EBP is operational it will begin accepting RESTful Web services from any client. Although not the primary interface to EBP, the default index page is acceptable for testing basic operations. If your server is listening on port 8080 you can access the EBP index page with:

Jobs, classes and started iniatiators persist across restart of the application server. If the app server stops while a job is running the job will be terminated and requeued to begin again after the server restarts (depending on your job restart parameters). A redeploy of EBP into the app server will reset the state as will the quiesce Web service call with the resetindication. All jobs will be flushed from the queue.

The EBP default configuration is usually fine for running the subsystem but you can change that either by modifying the file deployed with the subsystem under the server's webapps/ebp folder or by issuing the config Web service. Settings persist through application server restarts unless a redeploy overwrites them.

See the following document for EBP configuration settings:

EBP Web Services

The dashboard controls EBP through the Web services that define an HTTP Web URL with one or more parameters for each service and returning an XML document describing what was done. Report or held output datasets from jobs in the output queue are also retrieved through Web services. See EBP Web Services for a complete list or one of the specific services listed below.

  • define - new job class
  • start - job initiators of a specific class
  • submit - jobs to the input queue to be executed by initiators
  • list - list job classes, job initiators, input jobs, job outputs, output datasets
  • cancel - a running or queued job
  • purge - output of a running or finished job
  • quiesce - suspend job initiation, complete running jobs
  • stop - running job initiators, halting job execution
  • undefine - existing job classes, stopping all initiators of that class
  • checkpoint - a job by saving the current state (at the most recent completed job step)
  • restart - a previously checkpointed (after last successful job step) or held job (at start)
  • config - the Elastic Batch Platform

A typical sequence of Web service invocation to define and start jobs is shown below.

  1. Define    A, JCL
  2. Start     A
  3. Submit    JCL
  4. List
  5. List      jobid
  6. List      outputid
  7. List      outputid,     , MSG
  8. List      outputidstepddname

You can purge old output of completed jobs, cancel running or queued jobs and adjust configuration with other services.


EBP Error Messages

Error messages generated by the EBP subsystem fall into a number of categories:

  • Errors during startup.shutdown
  • Errors issueing Web service requests
  • Errors scanning input job datasets
  • Errors executing JCL
  • Errors occuring during execution of standard utilities
  • Errors as a result of executing scripts, programs or procedures referenced by input jobs

The list of all error messages may be found here:

EBP JCL Examples

See the following document for a list of sample JCL and script job inputs.

EBP System Operators Console

See the following document for a list of sample JCL and script job inputs.


EBP Installation

EBP is distributed as a COBOL and Java-based Web Application Archive (war).  The Announcements forum contains the latest information on new features, functions and fixes as well as the download location.  EBP is distributed as a zip file containing two files:

  • jes-vv.vv.vv.war -- the Web Application Archive used with Java Application Servers and Java Servlet Containers
  • ebp-standalone-vv.vv.vv.jar -- the Java Application Archive used as a stand-alone version of EBP without the need of other software

See your Java Application Server (Apache Geronimo, Oracle Weblogic, IBM Websphere, Red Hat JBOSS) documentation for installation and configuration of WAR files.  Most have a Web interface with a "Deploy" operation or simply allow you to drop war files into a "deploy" or "webapps" folder.  EBP begins operating immediately upon start-up of the container.  Depending on the configuration, you should be able to access the RESTful Web Services control point of EBP from a browser with the Web address:


Alternatively, you can run EBP without the need of a container.  Start EBP in stand-alone mode with the command:

java -jar ebp-standalone-vv.vv.vv.jar

Standalone EBP has simple command-line options to set important options which can be displayed with the --help option:

java -jar ebp-standalone-vv.vv.vv.jar --help
Heirloom Computing Elastic Batch Platform
Usage: java -jar ebp.jar options...
--port 8080 - defines the Web service port
--home <jar location> - EBP Home directory
--logs logs - log directory
--realm - login properties file
--context / - URL path
--help - this message


Use --port option to set the HTTP listening port.  The --home option sets the home directory, if different than the location of the jar file itself.  The --logs option can change the location of the message logs to other than the home directory.  The --context option will change the URL path from the default of "/" (the default when running under Application Servers is typically "/ebp").

java -jar ebp-standalone-vv.vv.vv.jar --port 80 --context /ebp --home /usr/share/EBP
sets up the environment such that the following Web address will access the home page:
Stand-alone EBP uses the Eclipse Jetty servlet container.  Further configuration is available through Eclipse Jetty system property settings.
Was this article helpful?
3 out of 3 found this helpful
Have more questions? Submit a request


  • 0

    Surprisingly complete for a first release.  Descriptions are clear and susinct.   Have not seen these capabilities available on  any other Cloud offering to date.  This is clearly a first for Heirloom!

  • 0

    New JCS and JEC language descriptions added here and in other forum documents

Please sign in to leave a comment.
Powered by Zendesk