Spring Batch as Wildfly Module

posted by Roberto Cortez on

(TL;DR – Get me to the code)

For a long time, the Java EE specification was lacking a Batch Processing API. Today, this is an essential necessity for enterprise applications. This was finally fixed with the JSR-352 Batch Applications for the Java Platform now available in Java EE 7. The JSR-352 got it’s inspiration from the Spring Batch counterpart. Both cover the same concepts, although the resulting API’s are a bit different.

Since the Spring team also collaborated in the JSR-352, it was only a matter of time for them to provide an implementation based on Spring Batch. The latest major version of Spring Batch (version 3), now supports the JSR-352.

I’m a Spring Batch user for many years and I’ve always enjoyed that the technology had a interesting set of built-in readers and writers. These allowed you to perform the most common operations required by batch processing. Do you need to read data from a database? You could use JdbcCursorItemReader, how about writing data in a fixed format? Use FlatFileItemWriter, and so on.

Unfortunately, JSR-352 implementations do not have the amount of readers and writers available in Spring Batch. We have to remember that JSR-352 is very recent and didn’t have time to catch up. jBeret, the Wildfly implementation for JSR-352 already provides a few custom readers and writers.

What’s the point?

I was hoping that with the latest release, all the readers and writers from the original Spring Batch would be available as well. This is not the case yet, since it would take a lot of work, but there are plans to make them available in future versions. This would allow us to migrate native Spring Batch applications into JSR-352. We still have the issue of the implementation vendor lock-in, but it may be interesting in some cases.

Motivation

I’m one of the main test contributors for the Java EE Samples in the JSR-352 specification. I wanted to find out if the tests I’ve implemented have the same behaviour using the Spring Batch implementation. How can we do that?

Code

I think this exercise is not only interesting because of the original motivation, but it’s also useful to learn about modules and class loading on Wildfly. First we need to decide how are we going to deploy the needed Spring Batch dependencies. We could deploy them directly with the application, or use a Wildfly module. Modules have the advantage to be bundled directly into the application server and can be reused by all deployed applications.

Adding Wildfly Module with Maven

With a bit of work it’s possible to add the module automatically with the Wildfly Maven Plugin and the CLI (command line). Let’s start to create two files that represent the CLI commands that we need to create and remove the module:

wildfly-add-spring-batch.cli

The module --name is important. We’re going to need it to reference it in our application. The --resources is a pain, since you need to indicate a full classpath to all the required module dependencies, but we’re generating the paths in the next few steps.

wildfly-remove-spring-batch.cli

Note wildfly.module.classpath. This property will hold the complete classpath for the required Spring Batch dependencies. We can generate it with Maven Dependency plugin:

This is going to pick all dependencies (including transitive), exclude javax (since they are already present in Wildfly) and exclude test scope dependencies. We need the following dependencies for Spring Batch:

Now, we need to replace the property in the file. Let’s use Maven Resources plugin:

This will filter the configured files and replace the property wildfly.module.classpath with the value we generated previously. This is a classpath pointing to the dependencies in your local Maven repository. Now with Wildfly Maven Plugin we can execute this script (you need to have Wildfly running):

And these profiles:

(For the full pom.xml contents, check here)

We can add the module by executing:
mvn process-resources wildfly:execute-commands -P install-spring-batch.

Or remove the module by executing:
mvn wildfly:execute-commands -P remove-spring-batch.

This strategy works for any module that you want to create into Wildfly. Think about adding a JDBC driver. You usually use a module to add it into the server, but all the documentation I’ve found about this is always a manual process. This works great for CI builds, so you can have everything you need to setup your environment.

Use Spring-Batch

Ok, I have my module there, but how can I instruct Wildfly to use it instead of jBeret? We need to add a the following file in META-INF folder of our application:

jboss-deployment-structure.xml

Since the JSR-352 uses a Service Loader to load the implementation, the only possible outcome would be to load the service specified in org.springframework.batch module. Your batch code will now run with the Spring Batch implementation.

Testing

The github repository code, has Arquillian sample tests that demonstrate the behaviour. Check the Resources section below.

Resources

You can clone a full working copy from my github repository. You can find instructions there to deploy it.

Wildfly – Spring Batch

Since I may modify the code in the future, you can download the original source of this post from the release 1.0. In alternative, clone the repo, and checkout the tag from release 1.0 with the following command: git checkout 1.0.

Future

I’ve still need to apply this to the Java EE Samples. It’s on my TODO list.

Share with others!
  • Twitter
  • Facebook
  • LinkedIn
  • Google Plus
  • Reddit
  • Add to favorites
  • Email
  • RSS

Comments ( 15 )

  1. ReplyAbhijith

    Awesome. Just what I was looking for.

  2. ReplyAnu

    I have one question, if a project is already using spring web-mvc of version X and adding this spring-batch of version Y ( version X > Y => X has new classes not available in Y ) from wildfly module, then the project deployment can fail as it tries to load both spring versions in same deployment class loader. Do you have any idea to fix this ?

    • ReplyRoberto Cortez

      Hi Anu,

      Thank you for your question.

      The answer may vary, depending on your setup. I’m assuming that you’re using spring mvc for some web application to call a spring batch job. You could place the spring batch job into another WAR/EAR and expose an EJB to perform the call. WARs and EARs have separate classloaders, so it should fix your problem.

      Cheers,

  3. Replysid anyone

    Thanks for the good example. This runs in some minutes at the wildfly server. But I have a problem. We build a very big EAR with some JARs and WARs.

    With this example I have created a JMX MBean in a separate WAR with an operation to start the job. The jobs are defined in this WAR. Both WARs are packed in an EAR an deployed on Wildfly.

    If I execute the MBean operation then an exeption is throw because org.jberet is used:
    Problem invoking startJob: javax.management.RuntimeErrorException: java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated

    I think that all files (beans.xml, jboss-deployment-structure.xml, batch.xml, batchlet-job.xml) at the correct place.

    com.cortez.wildfly.batch.ear
    META-INF
    jboss-deployment-structure.xml
    batch.xml
    batch-jobs\batchlet-job.xml
    MANIFEST.MF
    WEB-INF
    beans.xml
    com.cortez.wildfly.batch.jmx.war
    META-INF
    MANIFEST.MF
    WEB-INF
    web.xml
    classes
    com\cortez\wildfly\batch\batchlet\ApplicationMBeanLifeCycleListener.class
    com\cortez\wildfly\batch\batchlet\MyBatchletService.class
    com\cortez\wildfly\batch\batchlet\MyBatchletServiceMBean.class
    com.cortez.wildfly.batch.war.war
    META-INF
    MANIFEST.MF
    WEB-INF
    classes
    com\cortez\wildfly\batch\batchlet\MyBatchlet.class

    Do you have any idea to fix this? Thanks.

    • ReplyRoberto Cortez

      Hi sid,

      Usually, war files are class loader isolated, so they don’t “see” each other. That would be my first guest, but it’s hard to figure out, just with this information.

      Do you have a chance to fork my project and push your changes so I can have a look?

      Cheers,
      Roberto

      • Replysid anyone

        Hello Roberto, I wrote you a mail with my example. Thanks for your reply.

        • ReplyRoberto Cortez

          Hi Sid,

          Thank you. It’s a fun problem. You should have a solution in your email. I guess I would have to write a blog post about it :)

          Cheers,
          Roberto

  4. Replysid anyone

    I think I’ve found a solution. In the EAR I must include the file src/main/application/META-INF/jboss-deployment-structure.xml with

    <?xml version=”1.0″ encoding=”UTF-8″?>
    <jboss-deployment-structure>
     <sub-deployment name=”…”>
      <exclusions>
       <module name=”org.wildfly.jberet” />
       <module name=”org.jberet.jberet-core” />
      </exclusions>
      <dependencies>
       <module name=”org.springframework.batch” services=”import” meta-inf=”import” />
      </dependencies>
     </sub-deployment>
    </jboss-deployment-structure>

    Now the BatchRuntime with org.springframework.batch… is used.

    • ReplyRoberto Cortez

      Hi Sid,

      Great! Thanks for posting the solution :)

      Cheers,
      Roberto

  5. Replyjamezp

    I’d just like to add that in new versions of WildFly either the batch-jberet subsystem needs to be removed or excluded from deployments.

    Note in WildFly 10 the subsystem was renamed from batch to batch-jberet. Versions earlier than 10 will need to exclude the batch subsystem.

    • Replyjamezp

      Sorry the XML didn’t seem to come through. I’ll try one more time

      <jboss-deployment-structure>
      <deployment>
      <exclude-subsystems>
      <subsystem name="batch-jberet"/>
      </exclude-subsystems>
      </deployment>
      </jboss-deployment-structure>

      • ReplyRoberto Cortez

        Hi Mauri,

        Thank you for your note. In fact, I haven’t tried the project with Wilfly 10, so I’m glad you did and submitted a patch.

        Cheers,
        Roberto

  6. ReplyRaja

    Hi

    When I running the test with wildfly 10.1.0.Final I am getting the below exception.

    23:10:24,621 WARN [org.jboss.modules] (Weld Thread Pool — 1) Failed to define class com.cortez.wildfly.batch.batchlet.MyBatchlet in Module “deployment.6bdf5a94-4922-4034-b522-8953db21ccfb.war:main” from Service Module Loader: java.lang.NoClassDefFoundError: Failed to link com/cortez/wildfly/batch/batchlet/MyBatchlet (Module “deployment.6bdf5a94-4922-4034-b522-8953db21ccfb.war:main” from Service Module Loader): javax/batch/api/AbstractBatchlet

    Any suggestion will be appreciated.

    Thanks

    Raja

  7. ReplyRaja

    I saw this log also in the console

    23:10:24,637 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool — 1) WELD-000119: Not generating any bean definitions from com.cortez.wildfly.batch.batchlet.MyBatchlet because of underlying class loading error: Type Failed to link com.cortez.wildfly.batch.batchlet.MyBatchlet (Module “deployment.6bdf5a94-4922-4034-b522-8953db21ccfb.war:main” from Service Module Loader): javax.batch.api.AbstractBatchlet not found. If this is unexpected, enable DEBUG logging to see the full error.

Leave a reply

Your email address will not be published.

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>