* PartitionCollector provides a way to pass data from
* individual partitions to a single point of control running on
* the step's parent thread. The PartitionAnalyzer is used to
* receive and process this data.
*
*/
public interface PartitionCollector {
/**
* The collectPartitionData method receives control
* periodically during partition processing.
* This method receives control on each thread processing
* a partition as follows:
*

*

*
for a chunk type step, it receives control after
* every chunk checkpoint and then one last time at the
* end of the partition;

*
for a batchlet type step, it receives control once
* at the end of the batchlet.

*

*

* Note the collector is not called if the partition
* terminates due to an unhandled exception.
*

* *@return* an Serializable object to pass to the
* PartitionAnalyzer.
* @throws Exception is thrown if an error occurs.
*/
public Serializable collectPartitionData() throws Exception;
}
9.5.4. PartitionAnalyzer Interface
A partition analyzer receives control to process data and final results from partitions. If a partition collector is configured on the step, the partition analyzer receives control to process the data and results from the partition collector. While a separate partition collector instance is invoked on each thread processing a partition, the partition analyzer runs on a single, consistent thread each time it is invoked. The PartitionAnalyzer interface may be used to implement an PartitionAnalyzer batch artifact:
package jakarta.batch.api.partition;
import java.io.Serializable;
import jakarta.batch.runtime.BatchStatus;
/**
* PartitionAnalyzer receives control to process
* data and final results from each partition. If
* a PartitionCollector is configured on the step,
* the PartitionAnalyzer receives control to process
* the data and results from the partition collector.
* While a separate PartitionCollector instance is
* invoked on each thread processing a step partition,
* a single PartitionAnalyzer instance runs on a single,
* consistent thread each time it is invoked.
*
*/
public interface PartitionAnalyzer {
/**
* The analyzeCollectorData method receives
* control each time a Partition collector sends
* its payload. It receives the
* Serializable object from the collector as an
* input.
* @param data specifies the payload sent by a
* PartitionCollector.
* @throws Exception is thrown if an error occurs.
*/
public void analyzeCollectorData(Serializable data) throws
Exception;
/**
* The analyzeStatus method receives control each time a
* partition ends. It receives the batch and exit
* status strings of the partition as inputs.
* @param batchStatus specifies the batch status of a partition.
* @param exitStatus specifies the exit status of a partition.
* @throws Exception is thrown if an error occurs.
*/
public void analyzeStatus(BatchStatus batchStatus, String
exitStatus) throws Exception;
}
package jakarta.batch.api.partition;
import java.io.Serializable;
import jakarta.batch.runtime.BatchStatus;
/**
* The AbstractPartitionAnalyzer provides default
* implementations of less commonly implemented methods.
*/
public abstract class AbstractPartitionAnalyzer implements
PartitionAnalyzer {
/**
* Override this method to analyze PartitionCollector payloads.
*
* @param data specifies the payload sent by the
* PartitionCollector.
* @throws Exception is thrown if an error occurs.
*/
@Override
public void analyzeCollectorData(Serializable data) throws
Exception {
}
/**
* Override this method to analyze partition end status.
* @param batchStatus specifies the batch status of a partition.
* @param exitStatus specifies the exit status of a partition.
* @throws Exception is thrown if an error occurs.
*/
@Override
public void analyzeStatus(BatchStatus batchStatus, String
exitStatus)
throws Exception {
}
}
9.6. Decider Interface
A decider may be used to determine batch exit status and sequencing between steps, splits, and flows in a Job XML. The decider returns a String value which becomes the exit status value on which the decision chooses the next transition. The Decider interface may be used to implement an Decider batch artifact:
package jakarta.batch.api;
import jakarta.batch.runtime.StepExecution;
/**
* A Decider receives control as part of a decision element
* in a job. It is used to direct execution flow during job
* processing. It returns an exit status that updates the
* current job execution's exit status. This exit status
* value also directs the execution transition based on
* next, end, stop, fail child elements configured on the
* same decision element as the decider.
*/
public interface Decider {
/**
* The decide method sets a new exit status for a job.
* It receives an array of StepExecution objects as input.
* These StepExecution objects represent the execution
* element that transitions to this decider as follows:
*

*

*
Step

*

* When the transition is from a step, the decide method
* receives the StepExecution corresponding
* to the step as input.
*
Split

*

* When the transition is from a split, the decide method
* receives a StepExecution from each flow defined to the split
* as input.
*
Flow

*

* When the transition is from a flow, the decide method
* receives a StepExecution corresponding
* to the last execution element that completed in the flow.
* This will be a single StepExecution if the last element
* was a step and multiple StepExecutions if the last element
* was a split.
*

* @param executions specifies the StepExecution(s) of the preceding
* element.
* *@return* updated job exit status
* @throws Exception is thrown if an error occurs.
*/
public String decide(StepExecution[] executions) throws Exception;
}
9.7. Transactionality
Chunk type check points are transactional. The batch runtime uses global transaction mode on the Java EE platform and local transaction mode on the Java SE platform. Global transaction timeout is configurable at step-level with a step-level property:
'jakarta.transaction.global.timeout={seconds} - default is 180 seconds'
Example:
<step id="MyGlobalStep">
<properties>
<property name="jakarta.transaction.global.timeout" value="600"/>
</properties>
</step>
10. Batch Runtime Specification
10.1. Batch Properties Reserved Namespace
The batch runtime supports properties at job, step, partition, and artifact level. The property name prefix, 'jakarta.batch', is reserved for use by the batch runtime, as prescribed by this specification and future revisions of same. Applications and specification implementations must not define properties for their own use that begin with this prefix. Applications that do so risk undefined behavior.
10.2. Job Metrics
The batch runtime supports the following chunk-type step metrics:
-
readCount - the number of items successfully read.
-
writeCount - the number of items successfully written.
-
filterCount - the number of items filtered by ItemProcessor.
-
commitCount - the number of transactions committed.
-
rollbackCount - the number of transactions rolled back.
-
readSkipCount - the number of skippable exceptions thrown by the ItemReader.
-
processSkipCount - the number of skippable exceptions thrown by the ItemProcessor.
-
writeSkipCount - the number of skippable exceptions thrown by the ItemWriter.
These metrics are available through the StepExecution runtime object. See section 10.9.10 for further information on StepExecution.
10.3. Job Runtime Identifiers
Job runtime artifacts are uniquely defined at runtime with the following operational identifiers:
instanceId |
Is a long that represents an instance of a job. A new job instance is created everytime a job is started with the JobOperator "start" method. |
executionId |
Is a long that represents the next attempt to run a particular job instance. A new execution is created the first time a job is started and everytime thereafter when an existing job execution is restarted with the JobOperator "restart" method. Note there can be no more than one executionId in the STARTED state at one time for a given job instance. |
stepExecutionId |
Is a long that represents the attempt to execute a particular step within a job execution. |
Note instanceId, executionId, and stepExecutionId are all globally unique values within a job repository. See section 7.4 for explanation of job repository.
10.4. JobOperator
The JobOperator interface provides a set of operations to start, stop, restart, and inspect jobs. See 10.9.6 for detailed description of this interface. The JobOperator interface is accessible via a factory pattern:
JobOperator jobOper = BatchRuntime.getJobOperator();
See section 10.9.5 for details on the BatchRuntime class.
10.5. Batch Artifact Loading
All batch artifacts comprising a batch application are loadable by the following loaders in the order specified:
-
Implementation-specific loader
The batch runtime implementation _may_provide an implementation-specific means by which batch artifacts references in a Job XML (i.e. via the 'ref=' attribute) are resolved to an implementation class and instantiated. When the batch runtime resolves a batch artifact reference to an instance the implementation-specific mechanism (if one exists) is attempted first. The loader must return an instance or null.
An example of an implementation-specific loader might be CDI or Spring DI.
-
Archive loader
If an implementation-specific mechanism does not exist or fails toresolve a batch artifact reference (returns null), then the batch runtime implementation must resolve the reference with an archive loader. The implementation must provide an archive loader that resolves the reference by looking up the reference in a
batch.xml
file, which maps reference name to implementation class name. The loader must return an instance or null.The
batch.xml
file is packaged by the developer with the application under the ‘META-INF’ directory (‘WEB-INF/classes/META-INF’ for .war files).See 10.7.1 for more about the
batch.xml
file. -
Thread Context Class Loader
If the archive loader fails to resolve a batch artifact reference (returns null), then the batch runtime implementation must resolve the reference by treating the reference as a class name and loading it through the thread context class loader. The loader must return an instance or null.
10.6. Job XML Loading
Job XML is specified by name on the JobOperator.start command (see 10.9.6) to start a job.
All Job XML references are loadable by the following loaders in the order specified:
-
implementation-specific loader
The batch runtime implementation _must_provide an implementation-specific means by which Job XML references are resolved to a Job XML document.
The purpose of an implementation-specific loader is to enable Job XMLloading from outside of the application archive, such as from a repository, file system, remote cache, or elsewhere.
-
archive loader
If the implementation-specific mechanism does fails to resolve a Job XML reference, then the batch runtime implementation must resolve the reference with an archive loader. The implementation must provide an archive loader that resolves the reference by looking up the reference from the
META-INF/batch-jobs
directory.Job XML documents may be packaged by the developer with the application under the ‘META-INF/batch-jobs’ directory ('`WEB-INF/classes/META-INF`/batch-jobs' for .war files).
See 10.7.2 for more about the
META-INF/batch-jobs
.
10.7. Application Packaging Model
The batch artifacts that comprise a batch application requiring no unique packaging. They may be packaged in a standard jar file or can be included inside any Java archive type, as supported by the target execution platform in question.E.g. batch artifacts may be included in wars, EJB jars, etc, so long as they exist in the class loader scope of the program initiating the batch jobs (i.e. using the JobOperator start method).
10.7.1. META-INF/batch.xml
A batch application may use the archive loader (see section 10.5) to
load batch artifacts. The application can direct artifact loading by
supplying an optional batch.xml
file. The batch.xml
file must be stored
under the META-INF
directory. For .jar files it is the standard META-INF
directory. For .war files it is the WEB-INF/classes/META-INF
directory.
The format and content of the batch.xml
file follows:
<batch-artifacts xmlns="https://jakarta.ee/xml/ns/jakartaee">
<ref id="<reference-name>" class="<impl-class-name>" />
</batch-artifacts>
Where:
<reference-name> |
Specifies the reference name of the batch artifact. This is the value that is specified on the ref= attribute of the Job XML. |
<impl-class-name> |
Specifies the fully qualified class name of the batch artifact implementation. |
Notes:
-
If an implementation-specific loader is used (see 10.5) any artifact it loads takes precedence over artifacts specified in
batch.xml
. -
Use of
batch.xml
to load batch artifacts requires the availability of a zero-argument constructor (either a default constructor or an explicitly-defined, no-arg constructor ).
10.7.2. META-INF/batch-jobs
A batch application may use the archive loader (see section 10.6) to
load Job XML documents. The application does this by storing the Job XML
documents under the META-INF/batch-jobs
directory. For .jar files the
batch-jobs directory goes under the standard META-INF
directory. For
.war files it goes under the WEB-INF/classes/META-INF
directory. Note
Job XML documents are valid only in the batch-jobs directory:
sub-directories are ignored.
Job XML documents stored under META-INF/batch-jobs
are named with the
convention `<name>.xml
,Where:
<name> |
Specifies the name of a Job XML. This is the value that is specified on the JobOperator.start command. |
.xml |
Specifies required file type of a Job XML file under
|
Note if an implementation-specific loader (see 10.6) loads a Job XML
document that document takes precedence over documents stored under
META-INF/batch-jobs
.
10.8. Restart Processing
The JobOperator restart method is used to restart a JobExecution. A JobExecution is eligible for restart if:
-
Its batch status is STOPPED or FAILED.
-
It is the most recent JobExecution.
10.8.1. Job Parameters on Restart
Job parameter values are not remembered from one execution to the next. All Job Parameter substitution during job restart is performed based exclusively on the job parameters specified on that restart.
10.8.3. Execution Sequence on Restart – Overview
On the initial execution of a JobInstance, the sequence of execution is essentially:
-
Start at initial execution element
-
Execute the current execution element
-
Either:
-
Transition to next execution element (and go to step 2. above) OR
-
Terminate execution
-
On a restart, i.e. a subsequent execution of a JobInstance, the sequence of execution is similar, but the batch implementation must, in addition, determine which steps it does and does not need to re-execute.
So on a restart, the sequence of execution looks like:
-
Start at restart position
-
Decide whether or not to execute (or re-execute) the current execution element
-
Either:
-
Transition to next execution element (and go to step 2. above) OR
-
Terminate execution
-
So it follows that for restart we need: a definition of where in the job definition to begin; rules for deciding whether or not to execute the current execution element; and rules for performing transitioning, especially taking into account that all steps relevant to transitioning may not have executed on this (restart) execution. These rules are provided below.
10.8.4. Execution Sequence on Restart – Detailed Rules
Upon restart, the job is processed as follows:
-
Job XML Substitution is performed (see section 8.8).
-
Start by setting the current position to the restart position. The restart position is either:
-
the execution element identified by the <stop> elements "restart" attribute if that is how the previous execution ended; else
-
the initial execution element determined the same as upon initial job start, as described in section 8.2.5 Step Sequence;
-
-
Determine if the current execution element should re-execute:
-
If the current execution element is a COMPLETED step that specifies allow-restart-if-complete=false, then transition based on the exit status for this step from the previous completed execution. If the transition is a next transition, then repeat step 3 here with the value of next as the new, "current" execution element. Or, if the transition is a terminating transition such as end, stop, or fail, then terminate the restart execution accordingly.
-
If the current execution element is a COMPLETED step that specifies allow-restart-if-complete=true, then re-run the step and transition based on the new exit status from the new step execution. As above, either repeat step 3 with the next execution element or terminate the new execution as the transition element
-
If the current execution element is a STOPPED or FAILED step then restart the step and transition based on the exit status from the new step execution.+
Note if the step is a partitioned step, only the partitions that did not complete previously are restarted. This behavior may be overridden via a PartitionMapper (see section 10.8.5). Note for a partitioned step, the checkpoints and persistent user data are loaded from the persistent store on a per-partition basis (this is not a new rule, but a fact implied by the discussion of checkpoints in section 8.2.6 and the Step Context in section <IP_ADDRESS>, which is summarized here for convenience).
-
If the current execution element is a decision, execute the decision (i.e. execute the Decider) unconditionally. The Deciders "decide" method is passed a StepExecution array as a parameter. This array will be populated with the most-recently completed StepExecution(s) for each corresponding step.E.g. some StepExecution(s) may derive from previous job executions and some from the current restart (execution). A single decision following a split could even have a mix of old, new StepExecution(s) in the same array.
-
If the current execution element is a flow, transition to the first execution element in the flow and perform step 3 with this as the current element. When restart processing of the flow has completed, then follow the same rules which apply during the original execution (see section 8.9) to transition at the flow level to the next execution element, and repeat step 3 with that element as the current element.
Note the same rules regarding transitioning within a flow during an original execution apply during restart processing as well.
-
If the current execution element is a split, proceed in parallel for each flow in the split. For each flow, repeat step 3 with the flow element as the current element. When all flows in the split have been processed, follow the split’s transition to the next execution element and repeat step 3 with that element as the current element.
-
10.8.5. PartitionMapper on Restart
When the PartitionMapper is invoked at the beginning of a step which has been executed within a previous job execution, the first and most important decision for the mapper implementor to make is whether or not to keep the previous partitions or to begin the new execution with new partition definitions.
This decision is communicated to the batch implementation via the 'partitionsOverride' property of the PartitionPlan built by the mapper, i.e. the result of PartitionPlan’s getPartitionsOverride() method.
This property directs whether or not the partitions used in the previous execution of this step will or will be used (i.e. the relevant data carried forward and applied) within the current execution of this step. (As a consequence, the value of this property has no real meaning when the mapper is first called on the first execution of this step).
<IP_ADDRESS>. partitionsOverride = False
Three rules apply in the case where override is set to 'false':
<IP_ADDRESS>.1. Number of Partitions Must Be Same
The key idea here is that the mapper must build a partition plan with the same number of partitions that were used in the previous execution of this step. As a consequence, it is an error for the partition plan to return (via getPartitions()) a different number than the number of partitions established by the plan the last time this step was executed.
<IP_ADDRESS>.2. Partition Properties Populated From Current Plan
Though the number of partitions in the previous plan is persisted, the Properties[] returned by the previous PartitionPlan’s getPartitionProperties() is not. On a new execution of this step, it is the current return value of PartitionPlan#getPartitionProperties() which is used to populate the pool of potential 'partitionPlan' substitutions (see section <IP_ADDRESS>).
<IP_ADDRESS>.3. "Numbering" of Partitions via Partition Properties
Upon execution of this step, the batch implementation will associate each element of the Properties[] returned by PartitionPlan#getPartitionProperties() with a single partition, in order to potentially resolve 'partitionPlan' substitutions (see section <IP_ADDRESS>) for a single partition. During the course of execution of each partition, the batch implementation will capture data such as checkpoint values, persistent user data, etc.
Upon a new execution of this step during restart, the batch implementation must ensure that a similar mapping occurs. That is, the elements of the new Properties[] returned by the PartitionPlan#getPartitionProperties() built by the mapper must be mapped to the partitions in the same order as the earlier elements of the earlier Properties[] were mapped (for resolving 'partitionPlan' substitutions).
E.g., the following must hold:
Earlier Execution:
partitionPlanProps[] = mapper.getPartitionPlan().getPartitionProperties(); partitionPlanProps[0] ---maps to---> partition leaving off at checkpoints R0, W0 partitionPlanProps[1] ---maps to---> partition leaving off at checkpoints R1, W1
Current Execution:
newPartitionPlanProps[] = mapper.getPartitionPlan().getPartitionProperties(); newPartitionPlanProps[0] ---maps to---> partition resuming at checkpoints R0, W0 newPartitionPlanProps [1] ---maps to---> partition resuming at checkpoints R1, W1
In the shorthand above, "maps to" simply means that the Properties object on the left is used to potentially resolve the 'partitionPlan' substitutions for the give partition, before it executes as described.
<IP_ADDRESS>. partitionsOverride = True
In this case, all partition execution data: checkpoints, persistent user data, etc. from the earlier execution are discarded, and the new PartitionPlan built by the new execution of the PartitionMapper may define either the same or a different number of partitions; the new P artitionPlan’s getPartitionProperties() return value will be used to resolve 'partitionPlan' substitutions.
10.9. Supporting Classes
10.9.1. JobContext
package jakarta.batch.runtime.context;
/**
*
* A JobContext provides information about the current
* job execution.
*
*/
import java.util.Properties;
import jakarta.batch.runtime.BatchStatus;
public interface JobContext
{
/**
* Get job name
* *@return* value of 'id' attribute from <job>
*/
public String getJobName();
/**
* The getTransientUserData method returns a transient data object
* belonging to the current Job XML execution element.
* *@return* user-specified type
*/
public Object getTransientUserData();
/**
* The setTransientUserData method stores a transient data object into
* the current batch context.
* @param data is the user-specified type
*/
public void setTransientUserData(Object data);
/**
* The getInstanceId method returns the current job's instance
* id.
* *@return* job instance id
*/
public *long* getInstanceId();
/**
* The getExecutionId method returns the current job's current
* execution id.
* *@return* job execution id
*/
public *long* getExecutionId();
/**
* The getProperties method returns the job level properties
* specified in a job definition.
*

* A couple notes:
*

*
 There is no guarantee that the same Properties object instance
* is always returned in the same (job) scope.
*
 Besides the properties which are defined in JSL within a child
* <
properties>
element of a <
job>
element, the batch
* runtime implementation may choose to include additional,
* implementation-defined properties.
*

*
* *@return* job level properties
*/
public Properties getProperties();
/**
* The getBatchStatus method simply returns the batch status value * set
by the batch runtime into the job context.
* *@return* batch status string
*/
public BatchStatus getBatchStatus();
/**
* The getExitStatus method simply returns the exit status value stored
* into the job context through the setExitStatus method or null.
* *@return* exit status string
*/
public String getExitStatus();
/**
* The setExitStatus method assigns the user-specified exit status for
* the current job. When the job ends, the exit status of the job is
* the value specified through setExitStatus. If setExitStatus was not
* called or was called with a null value, then the exit status
* defaults to the batch status of the job.
* @param status string
*/
public void setExitStatus(String status);
}
10.9.2. StepContext
package jakarta.batch.runtime.context;
import java.io.Serializable;
import java.util.Properties;
import jakarta.batch.runtime.BatchStatus;
import jakarta.batch.runtime.Metric;
/**
*
* A StepContext provides information about the current step
* of a job execution.
*
*/
public interface StepContext
{
/**
* Get step name
* *@return* value of 'id' attribute from <step>
*
*/
public String getStepName();
/**
* The getTransientUserData method returns a transient data object
* belonging to the current Job XML execution element.
* *@return* user-specified type
*/
public Object getTransientUserData();
/**
* The setTransientUserData method stores a transient data object into
* the current batch context.
* @param data is the user-specified type
*/
public void setTransientUserData(Object data);
/**
* The getStepExecutionId method returns the current step's
* execution id.
* *@return* step execution id
*/
public *long* getStepExecutionId();
/**
* The getProperties method returns the step
level properties
* specified in a job definition.
*

* A couple notes:
*

*
 There is no guarantee that the same Properties object instance
* is always returned in the same (step) scope.
*
 Besides the properties which are defined in JSL within a child
* <
properties>
element of a <
step>
element, the batch
* runtime implementation may choose to include additional,
* implementation-defined properties.
*

* *@return* step level properties
*/
public Properties getProperties();
/**
* The getPersistentUserData method returns a persistent data object
* belonging to the current step. The user data type must implement
* java.util.Serializable. This data is saved as part of a step's
* checkpoint. For a step that does not do checkpoints, it is saved
* after the step ends. It is available upon restart.
* *@return* user-specified type
*/
public Serializable getPersistentUserData();
/**
* The setPersistentUserData method stores a persistent data object
* into the current step. The user data type must implement
* java.util.Serializable. This data is saved as part of a step's
* checkpoint. For a step that does not do checkpoints, it is saved
* after the step ends. It is available upon restart.
* @param data is the user-specified type
*/
public void setPersistentUserData(Serializable data);
/**
* The getBatchStatus method returns the current batch status of the
* current step. This value is set by the batch runtime and changes as
* the batch status changes.
* *@return* batch status string
*/
public BatchStatus getBatchStatus();
/**
* The getExitStatus method simply returns the exit status value stored
* into the step context through the setExitStatus method or null.
* *@return* exit status string
*/
public String getExitStatus();
/**
* The setExitStatus method assigns the user-specified exit status for
* the current step. When the step ends, the exit status of the step is
* the value specified through setExitStatus. If setExitStatus was not
* called or was called with a null value, then the exit status
* defaults to the batch status of the step.
* @param status string
*/
public void setExitStatus(String status);
/**
* The getException method returns the last exception thrown from a
* step level batch artifact to the batch runtime.
* *@return* the last exception
*/
public Exception getException();
/**
* The getMetrics method returns an array of step level metrics. These
* are things like commits, skips, etc.
* *@see* jakarta.batch.runtime.metric.Metric for definition of standard
* metrics.
* *@return* metrics array
*/
public Metric[] getMetrics();
}
10.9.3. Metric
package jakarta.batch.runtime;
/**
*
* The Metric interface defines job metrics recorded by
* the batch runtime.
*
*/
public interface Metric
{
public *enum* MetricType
{
READ_COUNT_, _WRITE_COUNT_,
_COMMIT_COUNT_,
_ROLLBACK_COUNT_, _READ_SKIP_COUNT_, _PROCESS_SKIP_COUNT_,
_FILTER_COUNT_,
_WRITE_SKIPCOUNT
}
/**
* The getName method returns the metric type.
* *@return* metric type.
*/
public MetricType getType();
/**
* The getValue method returns the metric value.
* *@return* metric value.
*/
public *long* getValue();
}
10.9.4. PartitionPlan
package jakarta.batch.api.partition;
/**
*
* PartitionPlan is a helper class that carries partition processing
* information set by the *@PartitionMapper* method.
*
* A PartitionPlan contains:
*

*
number of partition instances

*
number of threads on which to execute the partitions

*
substitution properties for each Partition (which can be
* referenced using the #
{
partitionPlan['propertyName']
}

* syntax.

*

*/
import java.util.Properties;
public interface PartitionPlan
{
/**
* Set number of partitions.
* @param count specifies the partition count
*/
public void setPartitions(int count);
/**
* Specify whether or not to override the partition
* count from the previous job execution. This applies
* only to step restart .
*

* When false is specified, the
* partition count from the previous job execution is used
* and any new value set for partition count in the current run
* is ignored. In addition, partition results from the previous
* job execution are remembered, and only incomplete partitions
* are reprocessed.
*

* When true is specified, the partition count from the current run
* is used and all results from past partitions are discarded. Any
* resource cleanup or back out of work done in the previous run is the
* responsibility of the application. The PartitionReducer artifact's
* rollbackPartitionedStep method is invoked during restart before any
* partitions begin processing to provide a cleanup hook.
*/
public void setPartitionsOverride(boolean override);
/**
* Return current value of partition override setting.
* *@return* override setting.
*/
public boolean getPartitionsOverride();
/**
* Set maximum number of threads requested to use to run
* partitions for this step. A value of '0' requests the batch
* implementation to use the partition count as the thread
* count. Note the batch runtime is not required to use
* this full number of threads;
it may not have this many
* available, and may use less.
*
* @param count specifies the requested thread count
*/
public void setThreads(int count);
/**
* Sets array of substitution Properties objects for the set of
Partitions.
* @param props specifies the Properties object array
* @see PartitionPlan#getPartitionProperties()
*/
public void setPartitionProperties(Properties[] props);
/**
* Gets count of Partitions.
* *@return* Partition count
*/
public int getPartitions();
/**
* Gets maximum number of threads requested to use to run
* partitions for this step. A value of '0' requests the batch
* implementation to use the partition count as the thread
* count. Note the batch runtime is not required to use
* this full number of threads;
it may not have this many
* available, and may use less.
*
* *@return* requested thread count
*/
public int getThreads();
/**
* Gets array of Partition Properties objects for Partitions.
*

* These can be used in Job XML substitution using
* substitution expressions with the syntax:
* #
{
partitionPlan['propertyName']
}

*

* Each element of the Properties array returned can
* be used to resolving substitutions for a single partition.
* In the typical use case, each Properties element will
* have a similar set of property names, with a
* substitution potentially resolving to the corresponding
* value for each partition.
*
* *@return* Partition Properties object array
*/
public Properties[]
getPartitionProperties();
}
package jakarta.batch.api.partition;
import java.util.Properties;
/**
* The PartitionPlanImpl class provides a basic implementation
* of the PartitionPlan interface.
*/
public class PartitionPlanImpl implements PartitionPlan
{
*private* int partitions= 0;
*private* boolean override= *false*;
*private* int threads= 0;
Properties[] partitionProperties= null;
@Override
public void setPartitions(int count)
{
partitions= count;
// default thread count to partition count
*if* (threads == 0) threads= count;
}
@Override
public void setThreads(int count)
{
threads= count;
}
@Override
public void setPartitionsOverride(boolean override)
{
*this*.override= override;
}
@Override
public boolean getPartitionsOverride()
{
return override;
}
@Override
public void setPartitionProperties(Properties[] props)
{
partitionProperties= props;
}
@Override
public int getPartitions()
{
return partitions;
}
@Override
public int getThreads()
{
return threads;
}
@Override
public Properties[] getPartitionProperties()
{
return partitionProperties;
}
}
10.9.5. BatchRuntime
package jakarta.batch.runtime;
/**
* The BatchRuntime represents the batch
* runtime environment.
*
*/
import jakarta.batch.operations.JobOperator;
/**
* BatchRuntime represents the JSR 352 Batch Runtime.
* It provides factory access to the JobOperator interface.
*
*/
public class BatchRuntime
{
/**
* The getJobOperator factory method returns
* an instance of the JobOperator interface.
* *@return* JobOperator instance.
*/
public *static* JobOperator getJobOperator()
{
return null;
}
}
10.9.6. BatchStatus
package jakarta.batch.runtime;
/**
* BatchStatus enum defines the batch status values
* possible for a job.
*
*/
public enum BatchStatus
{
STARTING_, _STARTED_, _STOPPING_,
_STOPPED_, _FAILED_, _COMPLETED_, _ABANDONED_
}
10.9.7. JobOperator
package jakarta.batch.operations;
import java.util.List;
import java.util.Set;
import java.util.Properties;
import jakarta.batch.runtime.JobExecution;
import jakarta.batch.runtime.JobInstance;
import jakarta.batch.runtime.StepExecution;
/**
* JobOperator provide the interface for operating on batch jobs.
* Through the JobOperator a program can start, stop, and restart jobs.
* It can additionally inspect job history, to discover what jobs
* are currently running and what jobs have previously run.
*
* The JobOperator interface imposes no security constraints. However,
* the implementer is free to limit JobOperator methods with a security
* scheme of its choice. The implementer should terminate any method
* that is limited by the security scheme with a JobSecurityException.
*
*/
public interface JobOperator
{
/**
* Returns a set of all job names known to the batch runtime.
*
* *@return* a set of job names.
* @throws JobSecurityException
*/
public Set<String> getJobNames() throws JobSecurityException;
/**
* Returns number of instances of a job with a particular name.
*
* @param jobName
* specifies the name of the job.
* *@return* count of instances of the named job.
* @throws NoSuchJobException
* @throws JobSecurityException
*/
public int getJobInstanceCount(String jobName) throws
NoSuchJobException,
JobSecurityException;
/**
* Returns all JobInstances belonging to a job with a particular name
* in reverse chronological order.
*
* @param jobName
* specifies the job name.
* @param start
* specifies the relative starting number (zero based) to
* return from the
* maximal list of job instances.
* @param count
* specifies the number of job instances to return from the
* starting position of the maximal list of job instances.
* *@return* list of JobInstances.
* @throws NoSuchJobException
* @throws JobSecurityException
*/
public List<JobInstance> getJobInstances(String jobName, int start,
int count)throws NoSuchJobException, JobSecurityException;
/**
* Returns execution ids for job instances with the specified
* name that have running executions.
*
* @param jobName
* specifies the job name.
* *@return* a list of execution ids.
* @throws NoSuchJobException
* @throws JobSecurityException
*/
public List<Long> getRunningExecutions(String jobName) throws
NoSuchJobException, JobSecurityException;
/**
* Returns job parameters for a specified job instance. These are the
* key/value pairs specified when the instance was originally created
* by the start method.
*
* @param executionId
* specifies the execution from which to retrieve the
* parameters.
* *@return* a Properties object containing the key/value job parameter
* pairs.
* @throws NoSuchJobExecutionException
* @throws JobSecurityException
*/
public Properties getParameters(*long* executionId)
throws NoSuchJobExecutionException, JobSecurityException;
/**
* Creates a new job instance and starts the first execution of that
* instance, which executes asynchronously.
*
* Note the Job XML describing the job is first searched for by name
* according to a means prescribed by the batch runtime implementation.
* This may vary by implementation. If the Job XML is not found by that
* means, then the batch runtime must search for the specified Job XML
* as a resource from the `META-INF/batch-jobs` directory based on the
* current class loader. Job XML files under `META-INF/batch-jobs`
* directory follow a naming convention of "name".xml where "name" is
* the value of the jobXMLName parameter (see below).
*
* @param jobXMLName
* specifies the name of the Job XML describing the job.
* @param jobParameters
* specifies the keyword/value pairs for attribute
* substitution in the Job XML.
* *@return* executionId for the job execution.
* @throws JobStartException
* @throws JobSecurityException
*/
public *long* start(String jobXMLName, Properties jobParameters)
throws
JobStartException, JobSecurityException;
/**
* Restarts a failed or stopped job instance, which executes
* asynchronously.
*
* @param executionId
* specifies the execution to to restart. This execution
* must be the most recent execution that ran.
* @param restartParameters
* specifies the keyword/value pairs for attribute
* substitution in the Job XML.
* *@return* new executionId
* @throws JobExecutionAlreadyCompleteException
* @throws NoSuchJobExecutionException
* @throws JobExecutionNotMostRecentException,
* @throws JobRestartException
* @throws JobSecurityException
*/
public *long* restart(*long* executionId, Properties
restartParameters)
throws JobExecutionAlreadyCompleteException,
NoSuchJobExecutionException,
JobExecutionNotMostRecentException,
JobRestartException,
JobSecurityException;
/**
* Request a running job execution stops. This
* method notifies the job execution to stop
* and then returns. The job execution normally
* stops and does so asynchronously. Note
* JobOperator cannot guarantee the jobs stops:
* it is possible a badly behaved batch application
* does not relinquish control.
*

* Note for partitioned batchlet steps the Batchlet
* stop method is invoked on each thread actively
* processing a partition.
*
* @param executionId
* specifies the job execution to stop.
* The job execution must be running.
* @throws NoSuchJobExecutionException
* @throws JobExecutionNotRunningException
* @throws JobSecurityException
*/
public void stop(*long* executionId) throws
NoSuchJobExecutionException,
JobExecutionNotRunningException, JobSecurityException;
/**
* Set batch status to ABANDONED. The instance must have
* no running execution.
*

* Note that ABANDONED executions cannot be restarted.
*
* @param executionId
* specifies the job execution to abandon.
* @throws NoSuchJobExecutionException
* @throws JobExecutionIsRunningException
* @throws JobSecurityException
*/
public void abandon(*long* executionId) throws
NoSuchJobExecutionException,
JobExecutionIsRunningException, JobSecurityException;
/**
* Return the job instance for the specified execution id.
*
* @param executionId
* specifies the job execution.
* *@return* job instance
* @throws NoSuchJobExecutionException
* @throws JobSecurityException
*/
public JobInstance getJobInstance(*long* executionId) throws
NoSuchJobExecutionException, JobSecurityException;
/**
* Return all job executions belonging to the specified job instance.
*
* @param jobInstance
* specifies the job instance.
* *@return* list of job executions
* @throws NoSuchJobInstanceException
* @throws JobSecurityException
*/
public List<JobExecution> getJobExecutions(JobInstance instance)
throws
NoSuchJobInstanceException, JobSecurityException;
/**
* Return job execution for specified execution id
*
* @param executionId
* specifies the job execution.
* *@return* job execution
* @throws NoSuchJobExecutionException
* @throws JobSecurityException
*/
public JobExecution getJobExecution(*long* executionId) throws
NoSuchJobExecutionException, JobSecurityException;
/**
* Return StepExecutions for specified execution id.
*
* @param executionId
* specifies the job execution.
* *@return* step executions (order not guaranteed)
* @throws NoSuchJobExecutionException
* @throws JobSecurityException
*/
public List<StepExecution> getStepExecutions(*long* jobExecutionId)
throws NoSuchJobExecutionException, JobSecurityException;
}
10.9.8. JobInstance
package jakarta.batch.runtime;
public interface JobInstance
{
/**
* Get unique id for this JobInstance.
* *@return* instance id
*/
public *long* getInstanceId();
/**
* Get job name.
* *@return* value of 'id' attribute from <job>
*/
public String getJobName();
}
10.9.9. JobExecution
package jakarta.batch.runtime;
import java.util.Date;
import java.util.Properties;
public interface JobExecution
{
/**
* Get unique id for this JobExecution.
* *@return* execution id
*/
public *long* getExecutionId();
/**
* Get job name.
* *@return* value of 'id' attribute from <job>
*/
public String getJobName();
/**
* Get batch status of this execution.
* *@return* batch status value.
*/
public BatchStatus getBatchStatus();
/**
* Get time execution entered STARTED status.
* *@return* date (time)
*/
public Date getStartTime();
/**
* Get time execution entered end status: COMPLETED, STOPPED, FAILED
* *@return* date (time)
*/
public Date getEndTime();
/**
* Get execution exit status.
* *@return* exit status.
*/
public String getExitStatus();
/**
* Get time execution was created.
* *@return* date (time)
*/
public Date getCreateTime();
/**
* Get time execution was last updated.
* *@return* date (time)
*/
public Date getLastUpdatedTime();
/**
* Get job parameters for this execution.
* *@return* job parameters
*/
public Properties getJobParameters();
}
10.9.10. StepExecution
package jakarta.batch.runtime;
import java.util.Date;
import java.io.Serializable;
public interface StepExecution
{
/**
* Get unique id for this StepExecution.
* *@return* StepExecution id
*/
public *long* getStepExecutionId();
/**
* Get step name.
* *@return* value of 'id' attribute from <step>
*/
public String getStepName();
/**
* Get batch status of this step execution.
* *@return* batch status.
*/
public BatchStatus getBatchStatus();
/**
* Get time this step started.
* *@return* date (time)
*/
public Date getStartTime();
/**
* Get time this step ended.
* *@return* date (time)
*/
public Date getEndTime();
/**
* Get exit status of step.
* *@return* exit status
*/
public String getExitStatus();
/**
* Get persistent user data.
*

* For a partitioned step, this returns
* the persistent user data of the
* StepContext of the "top-level"
* or main thread (the one the PartitionAnalyzer, etc.
* execute on). It does not return the persistent user
* data of the partition threads.
* *@return* persistent data
*/
public Serializable
getPersistentUserData ();
/**
* Get step metrics
* *@return* array of metrics
*/
public Metric[] getMetrics();
}
10.9.11. Batch Exception Classes
This specification defines batch exception classes in package jakarta.batch.operations. Note all batch exceptions are direct subclasses of base class BatchRuntimeException, which itself is a direct subclass of java.lang.RuntimeException. The following batch exception classes are defined:
-
JobExecutionAlreadyCompleteException
-
JobExecutionIsRunningException
-
JobExecutionNotMostRecentException
-
JobExecutionNotRunningException
-
JobRestartException
-
JobSecurityException
-
JobStartException
-
NoSuchJobException
-
NoSuchJobExecutionException
-
NoSuchJobInstanceException
11. Job Runtime Lifecycle
The following sections describe an ordered flow of artifact method invocations. Simple symbols are used to denote actions as follows:
Symbol |
Meaning |
<action> |
An action performed by the batch runtime. |
< - > method |
Invocation of a batch artifact method by the batch runtime. |
[method] |
Optional method. |
// comment |
Comment to clarify behavior. |
LABEL: |
Label used for flow control comments. |
11.1. Batch Artifact Lifecycle
All batch artifacts are instantiated prior to their use in the scope in which they are declared in the Job XML and are valid for the life of their containing scope. There are three scopes that pertain to artifact lifecycle: job, step, and step-partition.
One artifact per Job XML reference is instantiated. In the case of a partitioned step, one artifact per Job XML reference per partition is instantiated. This means job level artifacts are valid for the life of the job. Step level artifacts are valid for the life of the step. Step level artifacts in a partition are valid for the life of the partition.
No artifact instance may be shared across concurrent scopes. The same instance must be used in the applicable scope for a specific Job XML reference.
11.2. Job Repository Artifact Lifecycle
All job repository artifacts are created by the batch runtime during job processing and exist until deleted by an implementation provided means.
11.3. Job Processsing
-
<Create JobContext>
-
<Store job level properties in JobContext>
-
< - >[JobListener.beforeJob…] // thread A
-
<processs execution elements>
-
< - >[JobListener.afterJob…] // thread A
-
<Destroy JobContext>
11.4. Regular Batchlet Processsing
-
<Create StepContext>
-
<Store step level properties in StepContext>
-
< - > [StepListener.beforeStep…] // thread A
-
< - > Batchlet.process // thread A
-
// if stop issued:
-
< - >[Batchlet.stop] // thread B, StepContext is available
-
< - >[StepListener.afterStep…] // thread A
-
<Store StepContext persistent area>
-
<Destroy StepContext>
11.5. Partitioned Batchlet Processsing
-
<Create StepContext>
-
<Store step level properties in StepContext>
-
< - >[StepListener.beforeStep…] // thread A
-
< - >[PartitionReducer.beginPartitionedStep] // thread A
-
< - >[PartitionMapper.mapPartitions] // thread A
-
// per partition:
-
< - >Batchlet.process // thread Px
-
// if stop issued:
-
< - >[Batchlet.stop] // thread Py, StepContext is available
-
< - >[PartitionCollector.collectPartitionData] // thread Px
-
-
// when collector payload arrives:
-
< - >[PartitionAnalyzer.analyzeCollectorData] // thread A
-
// when partition ends:
-
< - >[PartitionAnalyzer.analyzeStatus] // thread A
-
// if rollback condition occurs:
-
< - >[PartitionReducer.rollbackPartitionedStep] // thread A
-
< - >[PartitionReducer.beforePartitionedStepCompletion] // thread A
-
< - >[PartitionReducer.afterPartitionedStepCompletion] // thread A
-
< - >[StepListener.afterStep…] // thread A
-
<Store StepContext persistent area>
-
<Destroy StepContext>
11.6. Regular Chunk Processing
-
<Create StepContext>
-
<Store step level properties in StepContext>
-
< - >[StepListener.beforeStep…] // thread A
-
[<begin transaction> ]
-
< - >ItemReader.open // thread A
-
< - >ItemWriter.open // thread A
-
[<commit transaction> ]
-
// chunk processing:
-
<repeat until no more items (i.e. while readItem hasn’t returned 'null') > \{
-
<begin checkpoint interval [<begin chunk transaction>]>
-
<repeat until checkpoint criteria reached OR readItem returns
null
> \{-
< - >ItemReader.readItem // thread A
-
// if readItem returns non-null
-
< - >ItemProcessor.processItem // thread A
-
// if processItem returns non-null, <add item to writeItems buffer>
-
-
-
}
-
// if at least one non-null value has been successfully read inthe present chunk
-
< - >ItemWriter.writeItems // thread A
-
-
< - >[ItemReader.checkpointInfo] // thread A
-
< - >[ItemWriter.checkpointInfo] // thread A
-
<Store StepContext persistent area>
-
[<commit chunk transaction>]
-
-
}
-
[<begin transaction> ]
-
< - >ItemWriter.close // thread A
-
< - >ItemReader.close // thread A
-
[<commit transaction> ]
-
< - >[StepListener.afterStep…] // thread A
-
<Store StepContext persistent area>
-
<Destroy StepContext>
11.7. Partitioned Chunk Processing
-
<Create StepContext>
-
<Store step level properties in StepContext>
-
< - >[StepListener.beforeStep…] // thread A
-
< - >[PartitionReducer.beginPartitionedStep] // thread A
-
< - >[PartitionMapper.mapPartitions] // thread A
// per partition - on thread Px:-
[<begin transaction> ]
-
< - >ItemReader.open // thread Px
-
< - >ItemWriter.open // thread Px
-
[<commit transaction> ]
-
<repeat until no more items (i.e. while readItem hasn’t returned 'null') > \{
-
<begin checkpoint interval [<begin chunk transaction>]>
-
<repeat until checkpoint criteria reached OR readItem returns 'null'> \{
-
< - >ItemReader.readItem // thread Px
-
// if readItem returns non-null
-
< - >ItemProcessor.processItem // thread Px
-
//if processItem returns non-null, <add item to writeItems buffer>
-
-
-
}
-
//if at least one non-null value has been successfully read in this partition of the present chunk
-
< - >ItemWriter.writeItems // thread Px
-
-
< - >[ItemReader.checkpointInfo] // thread Px
-
< - >[ItemWriter.checkpointInfo] // thread Px
-
<Store (partition-local) StepContext persistent area>
-
[<commit chunk transaction>]
-
< - >[PartitionCollector.collectPartitionData] // thread Px
-
-
}
-
[<begin transaction> ]
-
< - >ItemWriter.close // thread Px
-
< - >ItemReader.close // thread Px
-
[<commit transaction> ]
-
-
[<begin transaction> ] // thread A
-
// Actions 9-12 run continuously until all partitions end.
-
// when collector payload arrives:
-
< - >[PartitionAnalyzer.analyzeCollectorData] // thread A
-
// when partition ends:
-
< - >[PartitionAnalyzer.analyzeStatus] // thread A
-
// Remaining actions run after all partitions end:
-
// if rollback condition occurs:
-
< - >[PartitionReducer.rollbackPartitionedStep] // thread A
-
[<rollback transaction >]
-
// else not rollback
-
< - >[PartitionReducer.beforePartitionedStepCompletion] // thread A
-
[<commit transaction> ] // thread A
-
< - >[PartitionReducer.afterPartitionedStepCompletion] // thread A
-
< - >[StepListener.afterStep…] // thread A
-
<Store StepContext persistent area>
-
<Destroy StepContext>
11.8. Chunk with Listeners (except RetryListener)
-
<Create StepContext>
-
<Store step level properties in StepContext>
-
< - >[StepListener.beforeStep…] // thread A
-
[<begin transaction> ]
-
< - >ItemReader.open // thread A
-
< - >ItemWriter.open // thread A
-
[<commit transaction> ]
-
// chunk processing:
-
<repeat until no more items (i.e. while readItem hasn’t returned 'null') > \{
-
<begin checkpoint interval [<begin chunk transaction>]>
-
< - >[ChunkListener.beforeChunk] // thread A
-
<repeat until checkpoint criteria reached OR readItem returns 'null'> {
-
< - >[ItemReadListener.beforeRead] // thread A
-
< - >ItemReader.readItem // thread A
-
< - >[ItemReadListener.afterRead] // thread A
-
// or:
-
{
-
< - >[ItemReadListener.onReadError] // thread A
-
< - >[SkipListener.onSkipReadItem] // thread A
-
}
-
// if readItem returns non-null
-
< - >[ItemProcessListener.beforeProcess] // thread A
-
< - >ItemProcessor.processItem // thread A
-
< - >[ItemProcessListener.afterProcess] // thread A
-
//if processItem returns non-null,< add item to writeItems buffer>
-
// or:
-
{
-
< - >[ItemProcessListener.onProcessError] // thread A
-
< - >[SkipListener.onSkipProcessItem] // thread A
-
}
-
-
-
}
-
//if at least one non-null value has been successfully read in the present chunk
-
< - >[ItemWriteListener.beforeWrite] // thread A
-
< - >ItemWriter.writeItems // thread A
-
< - >[ItemWriteListener.afterWrite] // thread A
-
// or:
-
{
-
< - >[ItemWriteListener.onWriteError] // thread A
-
< - >[SkipListener.onSkipWriteItems] // thread A
-
}
-
-
< - >[ChunkListener.afterChunk] // thread A
-
< - >[ItemReader.checkpointInfo] // thread A
-
< - >[ItemWriter.checkpointInfo] // thread A
-
<Store StepContext persistent area>
-
[<commit chunk transaction>]
-
-
}
-
[<begin transaction> ]
-
< - >ItemWriter.close // thread A
-
< - >ItemReader.close // thread A
-
[<commit transaction> ]
-
< - >[StepListener.afterStep…] // thread A
-
<Store StepContext persistent area>
-
<Destroy StepContext>
11.9. Chunk with RetryListener
Note rollback processing is also depicted in this section.
-
<Create StepContext>
-
<Store step level properties in StepContext>
-
< - >[StepListener.beforeStep…] // thread A
-
[<begin transaction> ]
-
< - >ItemReader.open // thread A
-
< - >ItemWriter.open // thread A
-
[<commit transaction> ]
-
// chunk processing:
-
<repeat until no more items (i.e. while readItem hasn’t returned 'null') > \{
-
S1:
-
<begin checkpoint interval [<begin chunk transaction>]>
-
<repeat until checkpoint criteria reached OR readItem returns 'null'> \{
-
S2:
-
< - >ItemReader.readItem // thread A
-
// if exception
-
< - >[ItemReadListener.onReadErrror] // thread A
-
< - >[RetryReadListener.onRetryReadException] // thread A
-
// if retryable exception
-
// if no-rollback exception
-
resume S2:
-
// else
-
<end repeat>
-
// else
-
<end repeat>
-
S3:
-
// if readItem returns non-null
-
< - >ItemProcessor.processItem // thread A
-
// if exception
-
< - >[ItemProcessListener.onProcessErrror] // thread A
-
< - >[RetryProcessListener.onRetryProcessException] // thread A
-
// if retryable exception
-
// if no-rollback exception
-
resume S3:
-
// else
-
<end repeat>
-
// else
-
<end repeat>
-
// if processItem returns non-null, <add item to writeItems buffer>
-
-
}
-
// if rollback exception, execute rollback procedure (below) and resume at S1 with item-count=1
-
S4:
-
// if at least one non-null value has been successfully read in the present chunk
-
< - >ItemWriter.writeItems (buffer) // thread A
-
// if exception
-
< - >[ItemWriteListener.onWriteErrror] // thread A
-
< - >[RetryWriteListener.onRetryWriteException] // thread A
-
// if retryable exception
-
// if no-rollback exception
-
resume S4:
-
// else
-
execute rollback procedure (below) and resume S1:
-
// else execute rollback procedure (below) and resume S1:
-
< - >[ItemReader.checkpointInfo] // thread A
-
< - >[ItemWriter.checkpointInfo] // thread A
-
<Store StepContext persistent area> // thread A
-
S5:
-
[<commit chunk transaction>] // thread A
-
// if exception
-
// if retryable exception
-
// if no-rollback exception:
-
resume S5:
-
// else
-
execute rollback procedure (below) and resume S1:
-
// else execute rollback procedure (below) and resume S1:
-
-
}
-
[<begin transaction> ]
-
< - >ItemWriter.close // thread A
-
< - >ItemReader.close // thread A
-
[<commit transaction> ]
-
< - >[StepListener.afterStep…] // thread A
-
<Store StepContext persistent area>
-
<Destroy StepContext>
Rollback Procedure
-
< - >ItemWriter.close // thread A
-
< - >ItemReader.close // thread A
-
[ChunkListener.onError] // thread A
-
[rollback transaction]
-
[<begin transaction> ]
-
< - >ItemWriter.open // thread A, pass last committed checkpoint info
-
< - >ItemReader.open // thread A, pass last committed checkpoint info
-
[<commit transaction> ]
11.10. Chunk with Custom Checkpoint Processing
-
<Create StepContext>
-
<Store step level properties in StepContext>
-
< - >[StepListener.beforeStep…] // thread A
-
[<begin transaction> ]
-
< - >ItemReader.open // thread A
-
< - >ItemWriter.open // thread A
-
[<commit transaction> ]
-
// chunk processing:
-
<repeat until no more items (i.e. while readItem hasn’t returned 'null') > \{
-
[
-
< - >[CheckpointAlgorithm.checkpointTimeout]] // thread A
-
< - >[CheckpointAlgorithm.beginCheckpoint] // thread A
-
<begin checkpoint interval [<begin chunk transaction>]>
-
]
-
<repeat until isReadyToCheckpoint returns 'true' OR readItem returns 'null'> \{
-
< - >ItemReader.readItem // thread A
-
// if readItem returns non-null
-
< - >ItemProcessor.processItem // thread A
-
//if processItem returns non-null, <add item to writeItems buffer>
-
-
< - >CheckpointAlgorithm.isReadyToCheckpoint // thread A
-
-
}
-
//if at least one non-null value has been successfully read in the present chunk
-
< - >ItemWriter.writeItems // thread A
-
-
< - >[ItemReader.checkpointInfo] // thread A
-
< - >[ItemWriter.checkpointInfo] // thread A
-
<Store StepContext persistent area>
-
[<commit chunk transaction>]
-
< - >[CheckpointAlgorithm.endCheckpoint] // thread A
-
-
}
-
[<begin transaction> ]
-
< - >ItemWriter.close // thread A
-
< - >ItemReader.close // thread A
-
[<commit transaction> ]
-
< - >[StepListener.afterStep…] // thread A
-
<Store StepContext persistent area>
-
<Destroy StepContext>
11.13. Stop Processing
The JobOperator.stop operation stops a running job execution. If a step is running at the time the stop is invoked, the batch runtime takes the following actions:
Chunk Step
The job and step batch status is marked STOPPING. Note the batch runtime cannot guarantee the step actually exits. The batch runtime attempts to interrupt the read/process/write chunk processing loop. The batch runtime allows the step to finish processing the current item. This means the current item is read, processed if a processor is configured, and all currently buffered items, if any, including the current item, are written. If the batch artifacts configured on the chunk type step return to the batch runtime, as expected, the job and step batch status is marked STOPPED.
Batchlet Step
The job and step batch status is marked STOPPING. The batch runtime invokes the batchlet’s stop method. Note the batch runtime cannot guarantee the batchlet actually exits. But a well behaved batchlet will. If the batchlet returns to the batch runtime, the job and step batch status is marked STOPPED.
Note for partitioned batchlet steps the Batchlet stop method is invoked on each thread actively processing a partition.
12. Batch XML XSD
<xml version="1.0" encoding="UTF-8">
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
targetNamespace="https://jakarta.ee/xml/ns/jakartaee/"
mlns:jbatch="https://jakarta.ee/xml/ns/jakartaee/" version="1.0">
<xs:element name="batch-artifacts" type="jbatch:BatchArtifacts" />
<xs:complexType name="BatchArtifacts">
<xs:sequence>
<xs:element name="ref" type="jbatch:BatchArtifactRef" minOccurs="0"
maxOccurs="unbounded" />
</xs:sequence>
</xs:complexType>
<xs:complexType name="BatchArtifactRef">
<xs:attribute name="id" use="required" type="xs:string" />
<xs:attribute name="class" use="required" type="xs:string" />
</xs:complexType>
</xs:schema>
13. Job Specification Language
Jobs are described by a declarative Job Specification Language (JSL) defined by an XML schema, also known informally as Job XML.
13.1. Validation Rules
The batch runtime must perform schema validation during JobOperator start processing before the start method returns to the caller. A schema validation error results in JobStartException. The implementation has two choices for handling semantic errors in the JSL:
-
Do semantic validation during JobOperator start processing before returning to the caller. If there is a semantic validation error, the implementation must throw JobStartException.
-
Do semantic validation after job execution begins. If a semantic validation error occurs, the implementation must end the job in the FAILED state. The implementation is advised to log sufficient error information to enable problem resolution.
Typical semantic validation the batch runtime should detect and handle include, but is not limited to:
-
no executable elements
-
non-existent transitions (e.g. next="value" where "value" does not exist)
-
cycles among next values (e.g. step1:next=step2; step2:next=step1)
13.2. JSL XSD
<xml version="1.0" encoding="UTF-8">
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
targetNamespace="https://jakarta.ee/xml/ns/jakartaee/"
xmlns:jsl="https://jakarta.ee/xml/ns/jakartaee/" version="1.0">
<xs:annotation>
<xs:documentation>
Job Specification Language (JSL) specifies a job,
its steps, and directs their execution.
JSL also can be referred to as "Job XML".
</xs:documentation>
</xs:annotation>
<xs:simpleType name="artifactRef">
<xs:annotation>
<xs:documentation>
This is a helper type. Though it is not otherwise
called out by this name
in the specification, it captures the fact
that the xs:string value refers
to a batch artifact, across numerous
other JSL type definitions.
</xs:documentation>
</xs:annotation>
<xs:restriction base="xs:string" />
</xs:simpleType>
<xs:complexType name="Job">
<xs:annotation>
<xs:documentation>
The type of a job definition, whether concrete or
Abstract. This is the type of the root element of any JSL document.
</xs:documentation>
</xs:annotation>
<xs:sequence>
<xs:element name="properties" type="jsl:Properties"
minOccurs="0" maxOccurs="1">
<xs:annotation>
<xs:documentation>
The job-level properties, which are accessible
via the JobContext.getProperties() API in a batch
Artifact.
</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="listeners" type="jsl:Listeners"
minOccurs="0" maxOccurs="1">
<xs:annotation>
<xs:documentation>
Note that "listeners" sequence order in XML does
not imply order of execution by
The batch runtime, per the
specification.
</xs:documentation>
</xs:annotation>
</xs:element>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="decision" type="jsl:Decision" />
<xs:element name="flow" type="jsl:Flow" />
<xs:element name="split" type="jsl:Split" />
<xs:element name="step" type="jsl:Step" />
</xs:choice>
</xs:sequence>
<xs:attribute name="version" use="required" type="xs:string"
fixed="1.0" />
<xs:attribute name="id" use="required" type="xs:ID" />
<xs:attribute name="restartable" use="optional" type="xs:string" />
</xs:complexType>
<xs:element name="job" type="jsl:Job">
<xs:annotation>
<xs:documentation>
The definition of an job, whether concrete or
Abstract. This is the
type of the root element of any JSL document.
</xs:documentation>
</xs:annotation>
</xs:element>
<xs:complexType name="Listener">
<xs:sequence>
<xs:element name="properties" type="jsl:Properties"
minOccurs="0" maxOccurs="1" />
</xs:sequence>
<xs:attribute name="ref" use="required" type="jsl:artifactRef" />
</xs:complexType>
<xs:complexType name="Split">
<xs:sequence>
<xs:element name="flow" type="jsl:Flow" minOccurs="0"
maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="id" use="required" type="xs:ID" />
<xs:attribute name="next" use="optional" type="xs:string" />
</xs:complexType>
<xs:complexType name="Flow">
<xs:sequence>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="decision" type="jsl:Decision" />
<xs:element name="flow" type="jsl:Flow" />
<xs:element name="split" type="jsl:Split" />
<xs:element name="step" type="jsl:Step" />
</xs:choice>
<xs:group ref="jsl:TransitionElements" minOccurs="0"
maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="id" use="required" type="xs:ID" />
<xs:attribute name="next" use="optional" type="xs:string" />
</xs:complexType>
<xs:group name="TransitionElements">
<xs:annotation>
<xs:documentation>
This grouping provides allows for the reuse of the
'end', 'fail', 'next', 'stop' element sequences which
may appear at the end of a 'step', 'flow', 'split' or 'decision'.
The term 'TransitionElements' does not formally appear in the spec, it
is
A schema convenience.
</xs:documentation>
</xs:annotation>
<xs:choice>
<xs:element name="end" type="jsl:End" />
<xs:element name="fail" type="jsl:Fail" />
<xs:element name="next" type="jsl:Next" />
<xs:element name="stop" type="jsl:Stop" />
</xs:choice>
</xs:group>
<xs:complexType name="Decision">
<xs:sequence>
<xs:element name="properties" type="jsl:Properties"
minOccurs="0" maxOccurs="1" />
<xs:group ref="jsl:TransitionElements" minOccurs="0"
maxOccurs="unbounded"
/>
</xs:sequence>
<xs:attribute name="id" use="required" type="xs:ID" />
<xs:attribute name="ref" use="required" type="jsl:artifactRef" />
</xs:complexType>
<xs:attributeGroup name="TerminatingAttributes">
<xs:attribute name="on" use="required" type="xs:string" />
<xs:attribute name="exit-status" use="optional" type="xs:string" />
</xs:attributeGroup>
<xs:complexType name="Fail">
<xs:attributeGroup ref="jsl:TerminatingAttributes" />
</xs:complexType>
<xs:complexType name="End">
<xs:attributeGroup ref="jsl:TerminatingAttributes" />
</xs:complexType>
<xs:complexType name="Stop">
<xs:attributeGroup ref="jsl:TerminatingAttributes" />
<xs:attribute name="restart" use="optional" type="xs:string" />
</xs:complexType>
<xs:complexType name="Next">
<xs:attribute name="on" use="required" type="xs:string" />
<xs:attribute name="to" use="required" type="xs:string" />
</xs:complexType>
<xs:complexType name="CheckpointAlgorithm">
<xs:sequence>
<xs:element name="properties" type="jsl:Properties"
minOccurs="0" maxOccurs="1" />
</xs:sequence>
<xs:attribute name="ref" use="required" type="jsl:artifactRef" />
</xs:complexType>
<xs:complexType name="ExceptionClassFilter">
<xs:sequence>
<xs:element name="include" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence />
<xs:attribute name="class" use="required" type="xs:string" />
</xs:complexType>
</xs:element>
<xs:element name="exclude" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence />
<xs:attribute name="class" use="required" type="xs:string" />
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
<xs:complexType name="Step">
<xs:sequence>
<xs:element name="properties" type="jsl:Properties" minOccurs="0"
maxOccurs="1" />
<xs:element name="listeners" type="jsl:Listeners"
minOccurs="0" maxOccurs="1">
<xs:annotation>
<xs:documentation>
Note that "listeners" sequence order in XML does
not imply order of execution by
The batch runtime, per the
specification.
</xs:documentation>
</xs:annotation>
</xs:element>
<xs:choice minOccurs="0" maxOccurs="1">
<xs:element name="batchlet" type="jsl:Batchlet" />
<xs:element name="chunk" type="jsl:Chunk" />
</xs:choice>
<xs:element name="partition" type="jsl:Partition"
minOccurs="0" maxOccurs="1" />
<xs:group ref="jsl:TransitionElements"
minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
<xs:attribute name="id" use="required" type="xs:ID" />
<xs:attribute name="start-limit" use="optional" type="xs:string" />
<xs:attribute name="allow-start-if-complete" use="optional"
type="xs:string" />
<xs:attribute name="next" use="optional" type="xs:string" />
</xs:complexType>
<xs:complexType name="Batchlet">
<xs:sequence>
<xs:element name="properties" type="jsl:Properties"
minOccurs="0" maxOccurs="1" />
</xs:sequence>
<xs:attribute name="ref" use="required" type="jsl:artifactRef" />
</xs:complexType>
<xs:complexType name="Chunk">
<xs:sequence>
<xs:element name="reader" type="jsl:ItemReader" />
<xs:element name="processor" type="jsl:ItemProcessor"
minOccurs="0" maxOccurs="1" />
<xs:element name="writer" type="jsl:ItemWriter" />
<xs:element name="checkpoint-algorithm" type="jsl:CheckpointAlgorithm"
minOccurs="0" maxOccurs="1" />
<xs:element name="skippable-exception-classes"
type="jsl:ExceptionClassFilter" minOccurs="0" maxOccurs="1" />
<xs:element name="retryable-exception-classes"
type="jsl:ExceptionClassFilter"
minOccurs="0" maxOccurs="1" />
<xs:element name="no-rollback-exception-classes"
type="jsl:ExceptionClassFilter"
minOccurs="0" maxOccurs="1" />
</xs:sequence>
<xs:attribute name="checkpoint-policy" use="optional"
type="xs:string">
<xs:annotation>
<xs:documentation>
Specifies the checkpoint policy that governs
commit behavior for this chunk.
Valid values are: "item" or
"custom". The "item" policy means the
chunk is checkpointed after a
specified number of items are
processed. The "custom" policy means
The chunk is checkpointed
According to a checkpoint algorithm
implementation. Specifying
"custom" requires that the
checkpoint-algorithm element is also
specified. It is an optional
Attribute. The default policy is
"item". However, we chose not to define
A schema-specified default for this attribute.
</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="item-count" use="optional" type="xs:string">
<xs:annotation>
<xs:documentation>
Specifies the number of items to process per chunk
when using the item
checkpoint policy. It must be valid XML integer.
It is an optional
Attribute. The default is 10. The item-count
Attribute is ignored
for "custom" checkpoint policy. However, to
make it easier for implementations to support JSL inheritance
we abstain from defining a schema-specified default for this
Attribute.
</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="time-limit" use="optional" type="xs:string">
<xs:annotation>
<xs:documentation>
Specifies the amount of time in seconds before
taking a checkpoint for the
item checkpoint policy. It must be valid
XML integer. It is an
optional attribute. The default is 0, which
means no limit. However, to
make it easier for implementations to
support JSL inheritance
we abstain from defining a schema-specified
default for this attribute.
When a value greater than zero is
specified, a checkpoint is taken when
time-limit is reached or
item-count items have been processed,
whichever comes first. The
time-limit attribute is ignored for
"custom" checkpoint policy.
</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="skip-limit" use="optional" type="xs:string">
<xs:annotation>
<xs:documentation>
Specifies the number of exceptions a step will
skip if any configured
skippable exceptions are thrown by chunk
processing. It must be a
valid XML integer value. It is an optional
Attribute. The default
is no limit.
</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="retry-limit" use="optional" type="xs:string">
<xs:annotation>
<xs:documentation>
Specifies the number of times a step will retry if
Any configured retryable
exceptions are thrown by chunk processing.
It must be a valid XML
integer value. It is an optional attribute.
 Credits
Section 7 Domain Language of Batch, was adapted from Spring Batch Reference Documentation:
15. Change Log
15.1. Version 1.0 Revision A - Maintenance Release
15.1.1. Issues List
Following these links will show each original issue on our official spec issues tracking list. In most cases the bug report contains the complete text of the spec delta or addition, but not in every single case.
In Sec. 10.7.1, should we have said we require a "no-arg" explicit or implicit constructor rather than a "default constructor" |
|
SPEC: Misspoke on collector role on exit status |
|
Clarify JobContext/StepContext properties; fix TCK to not depend on writable Properties |
|
ItemProcessListener#onProcessError has javadoc from ItemProcessListener#afterProcess |
|
Add "mark FAILED" to BatchStatus state transitions |
|
Spec is unclear whether JobOperator methods may/must execute synchronously or not (with TCK implications) |
|
CheckpointAlgorithm needs to specify timeunit (seconds) and other javadoc fixes |
|
Evaluation order of multiple transition elements |
|
"Looping" should be clarified |
|
Flow/Split transitioning & termination not fully defined |
|
Details of exception handling (by container) |
|
8.6.1 Transition Next Element |
|
SPEC Partition Plan example confusing |
|
stop/end/fail exit-status should affect job exit status, not step (as claimed in spec). |
|
Spec should clarify StepExecution values passed to Decider on a restart |
|
Co-existence of transition elements with @next attribute PLUS behavior if no transition element @on is matched |
|
Spec contradicts itself when talking about uninitialized exit status (TCK assumes 'null') |
|
SPEC Partition Properties example has a invalid tag |
|
@Inject @BatchProperty should work for job level properties |
|
Clarify partition restart processing, PartitionPlan properties, and persistent user data for partitioned steps. |
|
Clarify when CheckpointAlgorithm#beginCheckpoint is invoked |
|
Spec doesn’t fully describe PartitionPlan override and the use of PartitionMapper |
|
When the first readItem() in a chunk return 'null', is this a zero-item chunk or is this not a new chunk after all |
|
Spec unclear on skipping part of an Exception hierarchy |


Card title
Some quick example text to build on the card title and make up the bulk of the card's content.
Read Chapter
Card title
Some quick example text to build on the card title and make up the bulk of the card's content.
Read Chapter
Card title
Some quick example text to build on the card title and make up the bulk of the card's content.


The definition of madness is doing the same thing over and over again and expecting a different result. Yet this is exactly what we have been doing in health care for decades. We have continually invested in the kinds of technologies that create and support chronic disease and yet somehow expect that this will result in wellness, prevention, and cures. Instead we need to harness the vast wealth of knowledge already amassed within the top academic centers and work together to create business models that target major chronic disease areas. 75% of all health care costs are caused by chronic diseases and it is this segment that has continued to grow dramatically. Pundits project these costs will reach $6 trillion dollars by the year 2050 for the US alone, unless something is changed.
Fixing this problem does not require expensive technology or new products. It requires leadership. It requires that we work together across the entire academic community. It requires that we identify, validate and build consensus around models that we can all support and implement. It requires that we be creative in building these models. It requires that we abandon ivory tower and silos thinking and work together and with the community. However, most importantly it requires that we open our eyes and hearts to all solutions and work together to make them even better.
When supported by a consensus of thought leaders medical products have historically moved into the main stream reaching 50% or greater penetration within five years or less (eg many pharmaceutical products). Without such support treatments no matter how well designed and documented rarely reach 10% penetration. Vavici Foundation is a non profit that will work with the top academic community to identify, validate and create consensus around the kinds of business models we need to shift the focus of health care back to wellness, prevention and curative care.


Woman injured, possibly by hit-and-run driver, in Huntington Park
HUNTINGTON PARK (KABC) -- Police are investigating a report of a woman who was possibly struck by a hit-and-run driver in Huntington Park.
The incident reportedly happened around 10 p.m. Friday in the area of Pacific Boulevard and Belgrave Avenue. Pacific Boulevard remained closed Saturday morning between Belgrave and Slauson avenues.
The driver allegedly fled the scene in a white Ford sedan traveling northbound on Pacific Boulevard.
The woman was taken to a local trauma center, where her condition was not immediately released.
An investigation is ongoing.


Our Services
Buying Cat Food
Buying cat food Buy it so that your cat gets the full nutrition. They are as healthy, hygienic and helpful as possible for cats, so when buying cat food it is important for the sitter to understand better. The types of cat food are as follows.
- Fresh food
- Packaged food
TEAM MEMBER
Our Team.
Lorem ipsum dolor sit amet, consectetur adipisicing elit. Illum reiciendis quasi itaque, obcaecati atque sit!


Set An Alarm
Take your next medicine after
00
hours and
00
mins within
00
days
SUCCESS!
Preparing the alarm for
.
SUCCESS!
Alarm set!
ERROR!
Don't put zeroes(0) on both hours and minutes.
ERROR!
You have entered zero(0) on days.
Please re-enter proper value for days.
SKIP


Random Facebook comment picker
Facebook access token
Get one here
Facebook object ID
Number of comments to pick
Pick comments
Picked comments:


WITNESS LIST State Affairs Committee April 26, 1999-2:00P HB 3 On: Raab, Ted Melina (Texas State Employees Union) HB 1088 For: Rhodes, Robert C. (Self) Against: Hernandez, Leslie (National Association of Social Workers-Texas Chapter) HB 2385 For: Smith, Tom "Smitty" (Public Citizens Members) HB 2602 For: Raab, Ted Melina (Texas State Employees Union) On: Cohen, Rachel (State Auditor's Office) Gee, Michelle (State Council on Competetive Government (CCG)) HB 3393 For: Aborn, Justin (GTE Internetworking) Epstein, Keith J. (SBC Internet Services) Kriz, Kenneth (GTE Media Ventures) Ladin Jr, William E. (PDQ.Net, Internet Service Provider) Leslie, M. Scott (Self) Muehlberg, Mark (Self) Perryman, Ray (Texas Open Net Internet Coalition) Raposa, John F. (GTE Corporation) Teplitz, Steven (America Online, Inc) Against: Arnold, W.D. (Self and Texas Cable & Telecommunications Assn) Carey III, William C. (Self and Time Warner Cable; Texas Cable TV Assn.) Garland, Charles (Self) King, Jess (Self) Morgan, Russell (AT&T) Pike, Dan (Self and Texas Cable and Telecommunications Assoc. and Prime Cable) Rutan, Edwin (AT&T) Valdez, Jason L. (Self) HB 3695 For: Kennard, Karen (Texas Municipal League) HCR 128 For: Lawrence, Paul (United Parcel Service) Story, Dave (Self and Coalition Against Unfair USPS Competition CAUUC) HJR 75 For: Ford, Mike (Self and Initiative for Texas Lone Star 2000) Against: Jefferson, John (Texas Chapter of the Wildlife Society) Joiner, Gary (Texas Farm Bureau) Sanchez, Joe (Mexican American Legal Defense and Educational Fund (MALDEF)) SB 223 On: Raab, Ted Melina (Texas State Employees Union) SB 382 For: Ellis, Gregory M. (Self) On: Phillips, Jim (Texas Natural Resource Conservation Commission)


Introducing JSON |
json element value object array string number "true" "false" "null" object '{' ws '}' '{' members '}' members member member ',' members member ws string ws ':' element array '[' ws ']' '[' elements ']' elements element element ',' elements element ws value ws string '"' characters '"' characters "" character characters character '0020' . '10ffff' - '"' - '\' '\' escape escape '"' '\' '/' 'b' 'n' 'r' 't' 'u' hex hex hex hex hex digit 'A' . 'F' 'a' . 'f' number int frac exp int digit onenine digits '-' digit '-' onenine digits digits digit digit digits digit '0' onenine onenine '1' . '9' frac "" '.' digits exp "" 'E' sign digits 'e' sign digits sign "" '+' '-' ws "" '0009' ws '000A' ws '000D' ws '0020' ws
JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.
JSON is built on two structures:
These are universal data structures. Virtually all modern programming languages support them in one form or another. It makes sense that a data format that is interchangeable with programming languages also be based on these structures.
In JSON, they take on these forms:
An object is an unordered set of name/value pairs. An object begins with { (left brace) and ends with } (right brace). Each name is followed by : (colon) and the name/value pairs are separated by , (comma).
An array is an ordered collection of values. An array begins with [ (left bracket) and ends with ] (right bracket). Values are separated by , (comma).
A value can be a string in double quotes, or a number, or true or false or null, or an object or an array. These structures can be nested.
A string is a sequence of zero or more Unicode characters, wrapped in double quotes, using backslash escapes. A character is represented as a single character string. A string is very much like a C or Java string.
A number is very much like a C or Java number, except that the octal and hexadecimal formats are not used.
Whitespace can be inserted between any pair of tokens. Excepting a few encoding details, that completely describes the language.


Quickstart¶
To be able to run your spiders from the Flask API need to create the following files
Create flask app¶
Create a file called app.py with the following code:
from twisted.web.wsgi import WSGIResource
from twisted.web.server import Site
from twisted.internet import reactor
from arachne import Arachne
app = Arachne(__name__)
resource = WSGIResource(reactor, reactor.getThreadPool(), app)
site = Site(resource)
reactor.listenTCP(8080, site)
if __name__ == '__main__':
reactor.run()
Spider¶
Lets say we have a spider that crawls http://dmoz.org that looks like:
import scrapy
class DmozItem(scrapy.Item):
"""Item object to store title, link, description"""
title = scrapy.Field()
url = scrapy.Field()
desc = scrapy.Field()
class DmozSpider(scrapy.Spider):
"""
Spider to crawl Python books and resources on dmoz.org
"""
name = "dmoz"
allowed_domains = ["dmoz.org"]
start_urls = [
"http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
"http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
]
def parse(self, response):
for sel in response.xpath('//ul/li'):
item = DmozItem()
item['title'] = sel.xpath('a/text()').extract()
item['url'] = sel.xpath('a/@href').extract()
item['desc'] = sel.xpath('text()').extract()
yield item
Settings¶
You need to specify the endpoint you would like to run your spider at in your settings.py file. To read more about the SPIDER_SETTINGS variables goto this link. It should look like:
SPIDER_SETTINGS = [
{
'endpoint': 'dmoz',
'location': 'spiders.DmozSpider',
'spider': 'DmozSpider'
}
]
Directory Structure¶
So finally this is what your directory should look like:
.
├── app.py
├── settings.py
└── spiders
├── DmozSpider.py
└── __init__.py # don't forget the __init__.py file
Now, run your application with the python command:
python app.py
Navigate to the URL http://localhost:8080/spiders/ to get a list of the spiders in your project. For the above example you should receive a response as:
{
spiders:['dmoz']
}
To run the dmoz spider you can navigate to the URL http://localhost:8080/run-spider/dmoz.
Note
You can also check the full quickstart project on github.


Even with the help of Java 7’s
Objects.equals()
, theequals()
method is often written in a verbose and messy fashion. This article shows how you can write a succinctequals()
in a format that allows easy checking with visual inspection.
When you create a new class, it automatically inherits class Object
. If you
don’t override equals()
, you’ll get Object
s equals()
method. By default
this compares addresses, so only if you are comparing the exact same objects
will you get true
. The default case is the “most discriminating.”
// DefaultComparison.java
class DefaultComparison {
private int i, j, k;
public DefaultComparison(int i, int j, int k) {
this.i = i;
this.j = j;
this.k = k;
}
public static void main(String[] args) {
DefaultComparison
a = new DefaultComparison(1, 2, 3),
b = new DefaultComparison(1, 2, 3);
System.out.println(a == a);
System.out.println(a == b);
}
}
/* Output:
true
false
*/
Normally you’ll want to relax this restriction. Typically, if two objects are
the same type and have fields with identical values, you’ll consider those
objects equal, but there may also be fields that you don’t want to include in
the equals()
comparison. This is part of the class design process.
A proper equals()
must satisfy the following five conditions:
Reflexive: For any
x
,x.equals(x)
should returntrue
.Symmetric: For any
x
andy
,x.equals(y)
should returntrue
if and only ify.equals(x)
returnstrue
.Transitive: For any
x
,y
, andz
, ifx.equals(y)
returnstrue
andy.equals(z)
returnstrue
, thenx.equals(z)
should returntrue
.Consistent: For any
x
andy
, multiple invocations ofx.equals(y)
consistently returntrue
or consistently returnfalse
, provided no information used in equals comparisons on the object is modified.For any non-
null x
,x.equals(null)
should returnfalse
.
Here are the tests that satisfy those conditions and determine whether the object you’re comparing yourself to (which we’ll call here the rval) is equal to this object:
If the rval is
null
, it’s not equal.If the rval is
this
(you’re comparing yourself to yourself), the two objects are equal.If the rval is not the same class or subclass, the two objects are not equal.
If all the above checks pass, then you must decide which fields in the rval are important (and consistent), and compare those.
Java 7 introduced the Objects
class to help with this process, which we use
to write a better equals()
.
The following examples compare different versions of the Equality
class. To
prevent duplicate code we’ll build the examples using the Factory Method. The
EqualityFactory
interface simply provides a make()
method to produce an
Equality
object, so a different EqualityFactory
can produce a different
subtype of Equality
:
// EqualityFactory.java
import java.util.*;
interface EqualityFactory {
Equality make(int i, String s, double d);
}
Now we’ll define Equality
containing three fields (all of which we consider
important during comparison) and an equals()
method that fulfills the four
checks described above. The constructor displays its type name to ensure we are
performing the tests we think we are:
// Equality.java
import java.util.*;
public class Equality {
protected int i;
protected String s;
protected double d;
public Equality(int i, String s, double d) {
this.i = i;
this.s = s;
this.d = d;
System.out.println("made 'Equality'");
}
@Override
public boolean equals(Object rval) {
if(rval == null)
return false;
if(rval == this)
return true;
if(!(rval instanceof Equality))
return false;
Equality other = (Equality)rval;
if(!Objects.equals(i, other.i))
return false;
if(!Objects.equals(s, other.s))
return false;
if(!Objects.equals(d, other.d))
return false;
return true;
}
public void
test(String descr, String expected, Object rval) {
System.out.format("-- Testing %s --%n" +
"%s instanceof Equality: %s%n" +
"Expected %s, got %s%n",
descr, descr, rval instanceof Equality,
expected, equals(rval));
}
public static void testAll(EqualityFactory eqf) {
Equality
e = eqf.make(1, "Monty", 3.14),
eq = eqf.make(1, "Monty", 3.14),
neq = eqf.make(99, "Bob", 1.618);
e.test("null", "false", null);
e.test("same object", "true", e);
e.test("different type", "false", new Integer(99));
e.test("same values", "true", eq);
e.test("different values", "false", neq);
}
public static void main(String[] args) {
testAll( (i, s, d) -> new Equality(i, s, d));
}
}
/* Output:
made 'Equality'
made 'Equality'
made 'Equality'
-- Testing null --
null instanceof Equality: false
Expected false, got false
-- Testing same object --
same object instanceof Equality: true
Expected true, got true
-- Testing different type --
different type instanceof Equality: false
Expected false, got false
-- Testing same values --
same values instanceof Equality: true
Expected true, got true
-- Testing different values --
different values instanceof Equality: true
Expected false, got false
*/
testAll()
performs comparisons with all different types of objects we ever
expect to encounter. It creates Equality
objects using the factory.
In main()
, notice the simplicity of the call to testAll()
. Because
EqualityFactory
has a single method, it can be used with a lambda expression
as the make()
method.
The above equals()
method is annoyingly verbose, and it turns out we can
simplify it into a canonical form. Observe:
The
instanceof
check eliminates the need to test fornull
The comparison to
this
is redundant. A correctly-writtenequals()
will work properly with self comparison.
Because &&
is a short-circuiting comparison, it quits and produces false
the first time it encounters a failure. So, by chaining the checks together
with &&
, we can write equals()
much more succinctly:
// SuccinctEquality.java
import java.util.*;
public class SuccinctEquality extends Equality {
public SuccinctEquality(int i, String s, double d) {
super(i, s, d);
System.out.println("made 'SuccinctEquality'");
}
@Override
public boolean equals(Object rval) {
return rval instanceof SuccinctEquality &&
Objects.equals(i, ((SuccinctEquality)rval).i) &&
Objects.equals(s, ((SuccinctEquality)rval).s) &&
Objects.equals(d, ((SuccinctEquality)rval).d);
}
public static void main(String[] args) {
Equality.testAll( (i, s, d) ->
new SuccinctEquality(i, s, d));
}
}
/* Output:
made 'Equality'
made 'SuccinctEquality'
made 'Equality'
made 'SuccinctEquality'
made 'Equality'
made 'SuccinctEquality'
-- Testing null --
null instanceof Equality: false
Expected false, got false
-- Testing same object --
same object instanceof Equality: true
Expected true, got true
-- Testing different type --
different type instanceof Equality: false
Expected false, got false
-- Testing same values --
same values instanceof Equality: true
Expected true, got true
-- Testing different values --
different values instanceof Equality: true
Expected false, got false
*/
For each SuccinctEquality
, the base-class constructor is called before the
derived-class constructor. The output shows that we still get the correct
result. You can tell that short-circuiting happens because both the null
test and the “different type” test would otherwise throw exceptions during
the casts that occur further down the list of comparisons in equals()
.
Objects.equals()
shines when you compose your new class using another class:
// ComposedEquality.java
import java.util.*;
class Part {
String ss;
double dd;
public Part(String ss, double dd) {
this.ss = ss;
this.dd = dd;
}
@Override
public boolean equals(Object rval) {
return rval instanceof Part &&
Objects.equals(ss, ((Part)rval).ss) &&
Objects.equals(dd, ((Part)rval).dd);
}
}
public class ComposedEquality extends SuccinctEquality {
Part part;
public ComposedEquality(int i, String s, double d) {
super(i, s, d);
part = new Part(s, d);
System.out.println("made 'ComposedEquality'");
}
@Override
public boolean equals(Object rval) {
return rval instanceof ComposedEquality &&
super.equals(rval) &&
Objects.equals(part, ((ComposedEquality)rval).part);
}
public static void main(String[] args) {
Equality.testAll( (i, s, d) ->
new ComposedEquality(i, s, d));
}
}
/* Output:
made 'Equality'
made 'SuccinctEquality'
made 'ComposedEquality'
made 'Equality'
made 'SuccinctEquality'
made 'ComposedEquality'
made 'Equality'
made 'SuccinctEquality'
made 'ComposedEquality'
-- Testing null --
null instanceof Equality: false
Expected false, got false
-- Testing same object --
same object instanceof Equality: true
Expected true, got true
-- Testing different type --
different type instanceof Equality: false
Expected false, got false
-- Testing same values --
same values instanceof Equality: true
Expected true, got true
-- Testing different values --
different values instanceof Equality: true
Expected false, got false
*/
Notice the call to super.equals()
—no need to reinvent it (plus you don’t
always have access to all necessary parts of a base class).
Equality Across Subtypes
Inheritance suggests that objects of two different subtypes can be “the same”
when they are upcast. Suppose you have a collection of Pet
objects. This
collection will naturally accept subtypes of Pet
: In this example, Dog
s and
Pig
s. Each Pet
has a name
and a size
, as well as a unique internal id
number.
We define equals()
and hashCode()
using the canonical form via the
Objects
class, but we only define them in the base class Pet
, and we do not
include the unique id
number in either one. From the standpoint of
equals()
, this means we only care if something is a Pet
, not whether it is
a specific type of Pet
:
// SubtypeEquality.java
import java.util.*;
enum Size { SMALL, MEDIUM, LARGE }
class Pet {
private static int counter = 0;
private final int id = counter++;
private final String name;
private final Size size;
public Pet(String name, Size size) {
this.name = name;
this.size = size;
}
@Override
public boolean equals(Object rval) {
return rval instanceof Pet &&
// Objects.equals(id, ((Pet)rval).id) && // [1]
Objects.equals(name, ((Pet)rval).name) &&
Objects.equals(size, ((Pet)rval).size);
}
@Override
public int hashCode() {
return Objects.hash(name, size);
// return Objects.hash(name, size, id); // [2]
}
@Override
public String toString() {
return String.format("%s[%d]: %s %s %x",
getClass().getSimpleName(), id,
name, size, hashCode());
}
}
class Dog extends Pet {
public Dog(String name, Size size) {
super(name, size);
}
}
class Pig extends Pet {
public Pig(String name, Size size) {
super(name, size);
}
}
public class SubtypeEquality {
public static void main(String[] args) {
Set<Pet> pets = new HashSet<>();
pets.add(new Dog("Ralph", Size.MEDIUM));
pets.add(new Pig("Ralph", Size.MEDIUM));
pets.forEach(System.out::println);
}
}
/* Output:
Dog[0]: Ralph MEDIUM a752aeee
*/
If we are just thinking about types, it does make sense—sometimes—to only consider the classes from the standpoint of their base type, which is the foundation of the Liskov Substitution Principle. This code fits nicely with that principle because the derived types don’t add any extra functionality (methods) that isn’t in the base class. The derived types only differ in behavior, not in interface (which of course is not the general case).
But when we provide two different object types with identical data and place
them in a HashSet<Pet>
, only one of these objects survives. This emphasizes
that equals()
is not a perfectly mathematical concept but (at least
partially) a mechanical one. hashCode()
and equals()
must be defined
hand-in-hand in order to allow types to work properly in a hashed data
structure.
In the example, both the Dog
and Pig
hash to the same bucket in the
HashSet
. At this point, the HashSet
falls back to equals()
to
differentiate the objects, but equals()
also declares the objects to be the
same. The HashSet
doesn’t add the Pig
because it’s already got an
identical object.
We can still make the example work by forcing uniqueness on otherwise identical
objects. Here, each Pet
already has a unique id
so you can either uncomment
line [1] in equals()
or switch to line [2] in hashCode()
. In the
canonical form you would do both, to involve all “unchanging” fields in both
operations (“unchanging” so that the equals()
and hashCode()
don’t produce
different values between storing and retrieving in a hashed data structure. I
put “unchanging” in quotes because you must evaluate whether modification might
happen).
Side note: in hashCode()
, if you are only working with a single field, use
Objects.hashCode()
and if you are using multiple fields use Objects.hash()
.
We can also solve the issue by following the standard form and defining
equals()
in the subclasses (but still not including the unique id
):
// SubtypeEquality2.java
import java.util.*;
class Dog2 extends Pet {
public Dog2(String name, Size size) {
super(name, size);
}
@Override
public boolean equals(Object rval) {
return rval instanceof Dog2 &&
super.equals(rval);
}
}
class Pig2 extends Pet {
public Pig2(String name, Size size) {
super(name, size);
}
@Override
public boolean equals(Object rval) {
return rval instanceof Pig2 &&
super.equals(rval);
}
}
public class SubtypeEquality2 {
public static void main(String[] args) {
Set<Pet> pets = new HashSet<>();
pets.add(new Dog2("Ralph", Size.MEDIUM));
pets.add(new Pig2("Ralph", Size.MEDIUM));
pets.forEach(System.out::println);
}
}
/* Output:
Dog2[0]: Ralph MEDIUM a752aeee
Pig2[1]: Ralph MEDIUM a752aeee
*/
Notice that the hashCode()
s are identical, but because the objects are no
longer equals()
, both now appear in the HashSet
. Also, super.equals()
means we don’t need access to the private
fields in the base class.
One way to look at this is to say that Java separates substitutability from the
definition of equals()
and hashCode()
. We can still place Dog
s and Pig
s
into a Set<Pet>
regardless of how equals()
and hashCode()
are defined,
but the objects won’t behave correctly in hashed data structures unless those
methods are defined with hashed structures in mind. Unfortunately, equals()
is not only used in conjunction with hashCode()
. This complicates things when
you try to avoid defining it for specific classes, and it’s why it’s worth
following the canonical form. However, this is further complicated because
there are times when you don’t need to define either method.


A new job position has just been posted on the N.U.E Offshore Resources Careers Portal
Hurry now and apply.
Visit link below to see latest job positions
careers.nueoffshore.com
Cheers!!


Loading locations...
Elect Fredericka Phillips for Harris County Judge
Your browser does not support the video tag.
Choose your county...
Choose your candidate...
Email
Phone
Facebook
Twitter
Name