Yearly archives "2014"

Devoxx 2014 – Infinite Possibilities

posted by Roberto Cortez on

Devoxx BE 2014This year, Devoxx Belgium was held between 10 and 14 November at its usual place, Antwerp – Metropolis Business Center. This was my second time at Devoxx BE and I’ve enjoyed my time there. Unfortunately, none of my submitted sessions were selected for this event. It’s very hard to get im there, since there are so many good submissions. Check the program here.

Announcements

Devoxx is going to Poland. The popular 33rd Degree Conference is rebranding to Devoxx.

Parleys has a new look and it’s now possible to enroll in online courses by recognised experts from across the tech sector.

A new knowledge sharing platform was revealed: Voxxed.com. Here you should be able to find the most recent news about Java and JVM technologies. When the platform was being demoed, the server crashed with everyone in the audience trying to access it. A funny moment, but the platform is stable now, and had no problems accessing it afterwards.

Voxxed

A new brand of smaller conferences was launched as Voxxed Days. These are one day tech events organised by local community groups and supported by the Voxxed team. Check the schedule. Voxxed Days are going to be in Vienna, Ticino, Istanbul and Berlin. Maybe we can bring them to Portugal too!

Sessions

I’ve spent most of my time in the Hackergarten, but I’ve also attended a few sessions. I recommend to have a look into these:

All the sessions will be on Parleys. So keep an eye on it.

There are a lot of sessions dedicated or related to Docker. It seems to be the next big thing. It would be interesting to see if Docker is going to be threat to multi-platform Java and open the path to other technologies. I don’t believe in it, since Java evolved way more than that, but let’s see what is going to happen.

We also got to see the very last podcast of The Java Posse. Thank you for all the great content produced for the Java community since 2005.

Java Posse Devoxx

Interesting Facts

Attendees of Devoxx vote on certain topics on whiteboards. Check this great post: Devoxx 2014 – Whiteboard votes by Stephen Colebourne with these year hot topics.

All results should be looked carefully, since these represent a very small number of developers at a top tech conference. Still, it’s interesting to see a very good adoption of Java 8 and IntelliJ IDEA being the number one IDE these days. What made me wonder was the huge amount of web frameworks or techniques (I’ve counted 27!) to build web applications. Diversity is good, but anyone else feels that something is really wrong here?

Devoxx Webapps

Final Words

It was great to attend Devoxx and to hang out live with the persons you usually only interact online. If you have never been to a conference, you should definitely consider attending Devoxx. Probably the reference conference for Europe.

(Please serve better food next year)!

I’ve travelled next to the Java2Days conference in Sofia – Bulgaria. So expect a post about that one too.

Share with others!
  • Twitter
  • Facebook
  • LinkedIn
  • Google Plus
  • Reddit
  • Add to favorites
  • Email
  • RSS

JPA Entity Graphs

posted by Roberto Cortez on

One of the latest features in JPA 2.1 is the ability to specify fetch plans using Entity Graphs. This is useful since it allows you to customize the data that is retrieved with a query or find operation. When working with mid to large size applications is common to display data from the same entity in different and many ways. In other cases, you just want to select a smallest set of information to optimize the performance of your application.

You don’t have many mechanisms to control what is loaded or not in a JPA Entity. You could use EAGER / LAZY fetching, but these definitions are pretty much static. You were unable to change their behaviour when retrieving data, meaning that you were stuck with what was defined in the entity. Changing these in mid development is a nightmare, since it can cause queries to behave unexpectedly. Another way to control loading is to write specific JPQL queries. You usually end up with very similar queries and the following methods: findEntityWithX, findEntityWithY, findEntityWithXandY, and so on.

Before JPA 2.1, the implementations already supported a non standard way to load data similar to Entity Graphs. You have Hibernate Fetch Profiles, OpenJPA Fetch Groups and EclipseLink Fetch Groups. It was logical to have this kind of behaviour in the specification. It allows you a much finer and detail control on what you need to load using a standard API.

Example

Consider the following Entity Graph:

Movie Entity Graph

(Probably the relationships should be N to N, but lets keep it simple).

And the Movie Entity has the following definition:

Looking closer to the entity, we can see that we have three 1 to N relationships and movieDirectors is set to be Eagerly loaded. The other relationships are set to the default Lazy loading strategy. If we want to change this behaviour, we can define different loading models by using the annotation @NamedEntityGraph. Just set a name to identify it and then use the @NamedAttributeNode to specify which attributes of the root entity that you want to load. For relationships you need to set a name to the subgraph and then use @NamedSubgraph. In detail:

Annotations

This defines an Entity Graph with name movieWithActors and specifies that the relationship movieActors should be loaded.

This defines an Entity Graph with name movieWithActorsAndAwards and specifies that the relationship movieActors should be loaded. Additionally, it also specifies that the relationship movieActors should load the movieActorAwards.

Note that we don’t specify the id attribute in the Entity Graph. This is because primary keys are always fetched regardless of what’s being specified. This is also true for version attributes.

Hints

To use the Entity Graphs defined in a query, you need to set them as an hint. You can use two hint properties and these also influences the way the data is loaded.

You can use javax.persistence.fetchgraph and this hint will treat all the specified attributes in the Entity Graph as FetchType.EAGER. Attributes that are not specified are treated as FetchType.LAZY.

The other property hint is javax.persistence.loadgraph. This will treat all the specified attributes in the Entity Graph as FetchType.EAGER. Attributes that are not specified are treated to their specified or default FetchType.

For more information, please refer to the sections 3.7.4.1 – Fetch Graph Semantics and 3.7.4.2 – Load Graph Semantics of the JPA 2.1 specification.

To simplify, and based on our example when applying the Entity Graph movieWithActors:

Default / Specifiedjavax.persistence.fetchgraphjavax.persistence.loadgraph
movieActorsLAZYEAGEREAGER
movieDirectorsEAGERLAZYEAGER
movieAwardsLAZYLAZYLAZY

In theory, this should be how the different relationships are fetched. In practice, it may not work this way, because the JPA 2.1 specification also states that the JPA provider can always fetch extra state beyond the one specified in the Entity Graph. This is because the provider can optimize which data to fetch and end up loading much more stuff. You need to check your provider behaviour. For instance Hibernate always fetch everything that is specified as EAGER even when using the javax.persistence.fetchgraph hint. Check the issue here.

Query

Performing the query is easy. You do it as you would normally do, but just call setHint on the Query object:

To get the Entity Graph you want to use on your query, you need to call the getEntityGraph method on the EntityManager and pass the name. Then use the reference in the hint. Hint must be either javax.persistence.fetchgraph or javax.persistence.loadgraph.

Programmatic

Annotations may become verbose, especially if you have big graphs or many Entity Graphs. Instead of using annotations, you can programmatically define Entity Graphs. Let’s see how:

Start by adding a static meta model Entity Class:

This is not really needed, you can reference the attributes by their string names, but this will give you type safety.

This Entity Graph specifies that all relationships of the Entity must be loaded. You can now adjust to your own use cases.

Resources

You can find this sample code in the Java EE samples at Github. Check it here.

Extra Note: currently there is a bug in EclipseLink / Glassfish that prevents javax.persistence.loadgraph hint from working properly. Check the issue here.

Conclusion

Entity Graphs filled a gap missing in the JPA specification. They are an extra mechanism that helps you to query for what you really need. They also help you to improve the performance of your application. But be smart when using them. There might be a better way.

Share with others!
  • Twitter
  • Facebook
  • LinkedIn
  • Google Plus
  • Reddit
  • Add to favorites
  • Email
  • RSS

Development Horror Story – Mail Bomb

posted by Roberto Cortez on

Based on my session idea at JavaOne about things that went terrible wrong in our development careers, I thought about writing a few of these stories. I’ll start with one of my favourites ones: Crashing a customer’s mail server after generating more than 23 Million emails! Yes, that’s right, 23 Millions!

Email Bomb

History

A few years ago, I’ve joined a project that was being developed for several months, but had no production release yet. Actually, the project was scheduled to replace an existing application in the upcoming weeks. My first task in the project was to figure out what was needed to deploy the application in a production environment and replace the old application.

This application had a considerable amount of users (around 50 k), but not all of them were active. The new application had a new feature to exclude the users that didn’t log into the application for the last few months. This was implemented as a timer (executed daily) and a email notification was sent to that user warning him that he was excluded from the application.

The Problem

The release was installed on a Friday (yes, Friday!), and everyone went for a rest. Monday morning, all hell broke loose! The customer mail server was down, and nobody had any idea why.

The first reports indicated that the mail server was out of disk space, because it had around 2 Million emails pending delivery and a lot more incoming. What the hell happened?

The Cause

Even with the server down, support was able to show us a copy of an email stuck in the server. It was consistent with the email sent when a user was excluded. It didn’t make any sense, because we counted the number of users to be excluded and they were around 28 k, so only 28 k emails should have been sent. Even if all users were excluded the number could not be higher than 50 k (the total number of users).

Invalid Email

Looking into the code, we found out a bug that would cause the user to not be excluded if he had an invalid email. As a consequence these users were caught every time that the timer executed. From the total 28 k users to be excluded, around 26 k had invalid emails. From Friday to Monday, we count 3 executions * 26 k users, so 78k k emails. Ok, so now we have an email increase, but not close enough to the reported numbers.

Timer Bug

Actually the timer also had a bug. It was not scheduled to be executed daily, but every 8 hours. Let’s adjust the numbers: 3 days * 3 executions a day * 26 k users, brings the total to 234 k emails. A considerable increase but still far from a big number.

Additional Node

The operations installed the application in a second node, and the timer was executed in both. So a double increase. Let’s update: 2 * 234 k emails, brings the total to 468 k emails.

No-reply Address

Since the emails were automated, you usually set up a no-reply email as the email sender. Now the problem was that the domain for the no-reply address was invalid. Combining this with the users invalid emails, the mail server entered in a loop state. Each invalid user email generated an error email sent to the no-reply address, which was invalid as well and this caused a returned email again to the server. The loop end when the Maximum hop count is exceeded. In this case it was 50. Now everything starts to make sense! Let’s update the numbers:

26 k users * 3 days * 3 executions * 2 servers * 50 hops for a grand total of 23.4 Million emails!

Aftermath

The customer lost all their email from Friday to Monday, but it was possible to recover the mail server. The problems were fixed and it never happened again. I remember those days, to be very stressful, but today all of us involved, laugh about it!

Remember: always check the no-reply address!

Share with others!
  • Twitter
  • Facebook
  • LinkedIn
  • Google Plus
  • Reddit
  • Add to favorites
  • Email
  • RSS

Sixth Coimbra JUG Meeting – Batch Processing in the Real World

posted by Roberto Cortez on

Last Thursday, 30 October 2014, the sixth meeting of Coimbra JUG was held on the Department of Informatics Engineering of the University of Coimbra, in Portugal. The attendance was good, we had around 25 people to listen to my talk about the Java EE Batch. This is the same session I have presented at JavaOne.

Coimbra JUG Meeting 6 Audience

No one in the audience was using Java EE Batch, and only a couple were using Spring Batch technologies. By coincidence a few old colleagues of mine, working in a project where I introduced the technology. The attendees seemed very curious and interested to learn about it and the questions were great. A lot of interactions and discussions were generated during the session. A funny thing happened toward the end: there was a power failure and we had to finished the session on the dark! We were lucky, since the session was almost done. Discussions about the topic (and others), bounced to the dinner. We had around ten enthusiasts, for our biggest dinner ever!

As always, we had surprises for the attendees: beer and chocolates, if you participated in the discussion. IntelliJ sponsored our event, by offering a free license to raffle among the attendees. Congratulations to Décio Sousa for winning the license. Develop with pleasure!

Here are the materials for the session:

Enjoy!

A few additional notes:

I would like to welcome Bruno Baptista to the Coimbra JUG Organization. He is going to help me running the JUG.

Coimbra JUG is almost 1 year old! Let’s see if we can pull something interesting to commemorate!

Share with others!
  • Twitter
  • Facebook
  • LinkedIn
  • Google Plus
  • Reddit
  • Add to favorites
  • Email
  • RSS

Configure JBoss / Wildfly Datasource with Maven

posted by Roberto Cortez on

Most Java EE applications use database access in their business logic, so developers are often faced with the need to configure drivers and database connection properties in the application server. In this post, we are going to automate that task for JBoss / Wildfly and a Postgre database using Maven. The work is based on my World of Warcraft Auctions Batch application from the previous post.

Maven Configuration

Let’s start by adding the following to our pom.xml:

We are going to use the Wildfly Maven Plugin to execute scripts with commands in the application server. Note that we also added a dependency to the Postgre driver. This is for Maven to download the dependency, because we are going to need it later to add it to the server. There is also a ${cli.file} property that is going to be assigned to a profile. This is to indicate which script we want to execute.

Let’s also add the following to the pom.xml:

With the Resources Maven Plugin we are going to filter the script files contained in the src/main/resources/scripts and replace them with the properties contained in ${basedir}/src/main/resources/configuration.properties file.

Finally lets add a few Maven profiles to the pom.xml, with the scripts that we want to run:

Wildfly Script Files

Add Driver

The scripts with the commands to add a Driver:

wildfly-install-postgre-driver.cli

Database drivers are added to Wildfly as a module. In this was, the driver is widely available to all the applications deployed in the server. With ${settings.localRepository} we are pointing into the database driver jar downloaded to your local Maven repository. Remember the dependency that we added into the Wildfly Maven Plugin? It’s to download the driver when you run the plugin and add it to the server. Now, to run the script we execute (you need to have the application server running):

mvn process-resources wildfly:execute-commands -P "install-driver"

The process-resources lifecycle is needed to replace the properties in the script file. In my case ${settings.localRepository} is replaced by /Users/radcortez/.m3/repository/. Check the target/scripts folder. After running the command, you should see the following output in the Maven log:

And on the server:

wildfly-remove-postgre-driver.cli

This script is to remove the driver from the application server. Execute mvn wildfly:execute-commands -P "remove-driver". You don’t need process-resources if you already executed the command before, unless you change the scripts.

Add Datasource

wow-auctions-install.cli
The scripts with the commands to add a Datasource:

We also need a a file to define the properties:

configuration.properties

Default Java EE 7 Datasource

Java EE 7, specifies that the container should provide a default Datasource. Instead of defining a Datasource with the JNDI name java:/datasources/WowAuctionsDS in the application, we are going to point our newly created datasource to the default one with /subsystem=ee/service=default-bindings:write-attribute(name="datasource", value="${datasource.jndi}"). In this way, we don’t need to change anything in the application. Execute the script with mvn wildfly:execute-commands -P "install-wow-auctions". You should get the following Maven output:

And on the server:

wow-auctions-remove.cli

This is the script to remove the Datasource and revert the Java EE 7 default Datasource. Run it by executing mvn wildfly:execute-commands -P "remove-wow-auctions"

Conclusion

This post demonstrated how to automate add / remove Drivers to Wildfly instances and also add / remove Datasources. This is useful if you want to switch between databases or if you’re configuring a server from the ground up. Think about CI environments. These scripts are also easily adjustable to other drivers.

You can get the code from the WoW Auctions Github repo, which uses this setup. Enjoy!

Share with others!
  • Twitter
  • Facebook
  • LinkedIn
  • Google Plus
  • Reddit
  • Add to favorites
  • Email
  • RSS