Yearly archives "2015"

Introducing Décio Sousa

posted by Roberto Cortez on
tags:

Hi everyone! Until now, this blog has been written entirely by me, but today we have a new author: Décio Sousa. We have been working together for several years and I consider him one of my closest friends. For some time, I have been trying to involve other people in writing blog posts. Décio accepted the challenge and here is the result: REST API Evolution. An interesting post on how to evolve REST API’s, based on his current project.

Décio and Roberto

I would like to welcome Décio and to thank him for taking the time to write the post. I’m sure we are going to see more posts from him in the future.

I moved to Git SVN

posted by Roberto Cortez on
tags: ,

For the last couple of years, I’ve been working with a company that uses Subversion. Yes, you are probably pointing now that we should have migrated to Git by now. Let’s not discuss that. Until now, Subversion didn’t get much in my way while performing my development tasks, but that slowly started to change. Probably a lot of people went through this as well.

My SVN Problems

My main problems with Subversion at the moment are:

  • Working with multiple Subversion branches at the same time. I usually checkout each branch that I need in a separate folder. Yes, I could use svn switch, but it’s harder to isolate new feature development from bug fixing with only one source.
  • Our Subversion repository is quite big now. You feel the pain (slowness) when comparing files in the history or annotating a file.
  • Multiple commits relating to the same feature. I like to commit the several stages of a feature development. This helps to establish baselines on what’s working and what’s not in a timeline. But it’s a pain to look into the multiple commits of that feature when you are searching the history.

Solution

I’m also a Git user, and I know that Git solves these problems for me. Branches are cheap to create and you can stash your changes for easy switching between branches. Since you have a local copy of the repository, everything is faster. Using squash will combine all your feature development commits into a single commit for a cleaner history.

Instead of nagging the organization to move to Git (I’m doing that anyway), you can jump right away to Git, by using git svn. It ships with the standard Git installation and allows you to have a bidirectional connection between Git and Subversion.

This isn’t something new. It’s been around for a few years, but this is my first time using it. I have to say that it was not easy to setup thing the way I wanted. So I’m writing this post to remind myself of the steps I have followed in case I need them in the future. Of course, I also hope that these could help other people experiencing the same problems.

Setup

You’re mostly going to use the command:

git svn

Assuming the following information:

SVN Repository URLhttp://svn-repo/myProject
SVN Trunk URLhttp://svn-repo/myProject/trunk
SVN Branch URLhttp://svn-repo/myProject/branches

Init the Repository

Let’s start by creating our local repository. Type the following command:

git svn init --trunk=trunk http://svn-repo/myProject myProject

This will initialize a local Git repository. This command will not checkout anything yet. My Subversion repository follows a standard directory layout and I could use the -s argument instead of manually indicating the trunk. I choose to manually indicate the trunk with --trunk argument because I only want to check out the trunk. If you use the -s everything sitting on branches and tags will be checked out. In my case I want to control exactly which branches , or tags I’m checking out.

If we want to include branches we can type:

git svn init --trunk=trunk --branches=branches http://svn-repo/myProject myProject

Better yet, if we only want the branch B1 and the brach B2 we can do it like this:

git svn init --trunk=trunk --branches={B1,B2} http://svn-repo/myProject myProject

Now look into the .git folder and file config. It should look like this:

You can always change the settings by manually editing this file.

When fetching specific branches there is a trick here. Notice that the generated file has a * after the branches definition of {B1,B2}. This means, that git svn will fetch all subdirectories of the branches folder B1 and B2 and track each individual subdirectory as a remote branch. It’s a bit weird. Maybe I’m doing the original command wrong, but I had to manually remove the * to track the branches properly. Make sure that you have this in the file:

This Stackoverflow question: How do I tell git-svn about a remote branch created after I fetched the repo? explains in a more detailed manner how to deal with branches.

Fetch the Code

To actually fetch the code from the Subversion repository type:

git svn fetch --no-follow-parent

In same cases, Git can create additional branches with the format {B1}@-{0-9}. You usually don’t need this and you can prevent their creating by adding the --no-follow-parent parameter. For a full explanation please check this Stackoverflow question: git-svn clone | spurious branches.

By the way this operation can be VERY VERY SLOW, depending on the size of your repository, it can take several hours to complete. If you are short on time, execute the commands before you go to sleep and you should have them ready when you wake up. To speed it up, you can use the parameter --revision and specify a Subversion revision number. The fetch will only be performed from that point forward. The downside is that you don’t have the historical data before the specified revision.

Update the Code

When you need to update your Git repository with the Subversion one, execute:

git svn rebase

Push the Code

You don’t use push to send your local changes to the Subversion repository, instead use:

git svn dcommit

Final Thoughts

You can now enjoy all the benefits of using Git even if you are stuck with a Subversion repository. There are also a few limitations, but you can work around them. You cannot directly map multiple Subversion repositories to a single Git repository. This may be relevant depending on your Subversion structure. Also, committing your changes to Subversion may be a bit slow.

Anyway, I’m happy with the change. I feel that it increased my productivity, but that’s something you have to figure out by yourself. Just give it a try and see if it works for you. If not, you can always use your old Subversion repository.

Jfokus 2015 and Voxxed Days Vienna

posted by Roberto Cortez on

I kicked off my conference year by attending during the last week Jfokus 2015 and the first edition ever of Voxxed Days held in Vienna. I was scheduled as a speaker for both conferences with my sessions about Java EE 7 Batch Processing in the Real World and The 5 People in your Organization that grow Legacy Code.

Jfokus

Jfokus LogoThis was Jfokus 9th edition, so we can expect a great celebration next year for the 10th anniversary edition. We are still one year away, but I already scheduled it in my calendar! The conference was 3 days long, with the first day dedicated to Tutorials and the next couple of days to conference sessions. The numbers are impressive. It had over 1700 attendees, making it one of the largest Java conferences in Europe.

Sessions

Docker had been a hot topic over the last year. So, naturally we had a few sessions dedicated to Docker. I do recommend checking out the Docker Workshop from Ken Sipe. You can find it here. It also includes how to scale Docker using Apache Mesos. Apache Mesos is a distributed system kernel that abstracts CPU, memory, storage, and other resources away from machines so you can program against your datacenter like it’s a single pool of resources. Unfortunately, I think this tutorial session was not recorded. Only a few rooms had their sessions recorded.

These are my top 3 sessions (from the ones I have attended):

My Session

I had my session about Java EE Batch Processing in the Real World as a Lightning Talk. It was a bit hard to do it in only 15 minutes, since the session was originally planed for a full conference session of 50 minutes. Anyway, I was able to demo everything I wanted.

I also had the privilege to be a guest to the Live Nighthacking stream with Stephen Chin, where I talk about Java EE Batch in much more detail. Check it out:

And here are some slides. I didn’t use them all the Lighting session, since they are from my full session.

Jfokus Demogrounds

Voxxed Days Vienna

Voxxed Vienna LogoA new brand of smaller conferences was launched as Voxxed Days. These are one day tech events organised by local community groups and supported by the Voxxed and Devoxx team. I was happy enough to be part of the first edition ever and to be a speaker of course! We got the usual cinema like venue, which is to be expected from the Devoxx brand. There were probably around 200 attendees, or maybe a little more.

Sessions

Since this was a one day only conference, there were not many sessions, but we had 4 full tracks worth of content to choose from. I recommend to check out Monadic Java by Mario Fusco with a very good way to explain Monads in Java to newbies. Also, Coding Culture by Sven Peters is a must. You will hear real life stories about how Atlassian evolved as a company and how they create awesome stuff. These sessions were recorded and should be available on Parleys very soon.

My Session

I had the session about The 5 People in your Organization that grow Legacy Code. I have presented the same session for the first time in Java One and the recording was released a few days before I delivered the session in Vienna. This was great, since it gave the change to check some of the mistakes I made and correct a few things. This one was also recorded. Let’s see if I improved a little bit. Thank you to Sven Peters, which provided me with a few pointers to improve my presentation stance.

Anyway, here is the full video (from JavaOne):

Final Words

It was an awesome week, but also very consuming. I was very tired at the end, but it was great to hook up with old friends. Thanks to Mattias Karlsson and Grzegorz Duda for having me in Jfokus 2015 and Voxxed Days Vienna.

I als have to mention Paulo Grácio. We worked together for 6 years and he was a mentor to me in the early stages of my career. Paulo is now working in Stockholm and we are far from each other, but I’m looking forward to work together again. Thanks for the hospitality!

Roberto and Paulo

Development Horror Story – Release Nightmare

posted by Roberto Cortez on

Everyone has good stories about releases that went wrong, right? I’m no exception and I have a few good ones under my development career. These are usually very stressful at the time, but now me and my teammates can’t talk about these stories without laughing.

Never Happened QA

History

I think this happened around 2009. Me and my team had to maintain a medium to large legacy web application with around 500 k lines of code. This application was developed by another company, so we didn’t have the code. Since we were in charge now and needed the code to maintain it, they handed us the code in a zip file (first pointer that something was wrong)!

Their release process was peculiar to say the least. I’m pretty sure there are worst release procedures out there. This one consisted in copying the changed files (*.class, *.jsp, *.html, etc) to an exploded war folder on a Tomcat server. We also had three environments (QA, PRE, PROD) with different application versions and no idea which files were deployed on each. They also had a ticket management application with attached compiled files, ready to be deployed and no idea of the original sources. What could possibly go wrong here?

The Problem

Our team was able to make changes required by the customer and push them to PROD servers. We have done it a few times successfully, even with all the handicaps. Everything was looking good until we got another request for additional changes. These changes were only a few improvements in the log messages of a batch process. The batch purpose was to copy files sent to the application with financial data input to insert into a database. I guess that I don’t have to state the obvious: this data was critical to calculate financial movements with direct impact on the amounts paid by the application users.

After our team made the changes and perform the release, all hell went loose. Files were not being copied to the correct locations. Several data duplicated in the database and the file system. Financial transactions with incorrect amounts. You name it. A complete nightmare. But why? The only change was a few improvements in the log messages.

The Cause

The problem was not exactly related with the changed code. Look at the following files:

And:

This is not the real code, but for the problem purposes it was laid out like this. Don’t ask me about the why it was like this. We got it in the zip file, remember?

So we have here a variable which sets the expected Operating System and then the logic to copy the file is dependant on this. The server was running on a Unix box so the variable value was Unix. Unfortunately, all the developers were working on Windows boxes. I said unfortunately, because if the developer that implemented the changes was using Unix, everything would be fine.

Anyway, the developer changed the variable to Windows so he could proceed with some tests. Everything was fine, so he performs the release. He copied the resulting BatchProcess.class into the server. He didn’t bother about the BatchConfiguration, since the one on the server was configured to Unix right?

Maybe you already spotted the problem. If you haven’t, try the following:

  • Copy and build the code.
  • Execute it. Check the output, you should get Windows.
  • Copy the resulting BatchProcess.class to an empty directory.
  • Execute this one again. Use command line java BatchProcess

What happened? You got the output Windows, right?. Wait! We didn’t have the BatchConfiguration.class file in the executing directory. How is that possible? Shouldn’t we need this file there? Shouldn’t we get an error?

When you build the code, the java compiler will inline the BatchConfiguration.OS variable. This means that the compiler will replace the variable expression in the if statement with the actual variable value. It’s like having if ("Windows".equals("Windows"))

Try executing javap -c BatchProcess. This will show you a bytecode representation of the class file:

You can confirm that all the variables are replaced with their constant values.

Now, returning to our problem. The .class file that was copied to the PROD servers had the Windows value set in. This messed everything in the execution runtime that handled the input files with the financial data. This was the cause of the problems I’ve described earlier.

Aftermath

Fixing the original problem was easy. Fixing the problems caused by the release was painful. It involved many people, many hours, pizza, loads of SQL queries, shell scripts and so on. Even our CEO came to help us. We called this the mUtils problem, since it was the original java class name with the code.

Yes, we migrated the code to something manageable. It’s now on a VCS with a tag for every release and version.

My most useful IntelliJ IDEA keyboard shortcuts

posted by Roberto Cortez on
tags: ,

Are you looking for ways to be more productive? It shouldn’t be a secret that performing actions using the keyboard instead of the mouse will save you time. If you think only about a single action it’s not a big deal. What if you use the same action multiple times a day? If you add up all these actions, they can have a great impact on your productivity.

Starcraft APM

I’m more or less used to drive most of my actions with keyboard shortcuts. When I was younger, I played semi-professionally Real Time Strategy computer games, including Starcraft and Warcraft III. Starcraft, popularized the term APM (Actions per Minute), which counted the number of actions that a player performed per minute. By using tools, it was possible to record APMs and tell if players were using mouse actions or a keyboard and mouse action combination. Usually, players with a keyboard and mouse combination gameplay had a better chance of winning games than the one that just clicked.

So, what does this have to do with code and IntelliJ? Well, I believe that you can increase your code development productivity by learning and using the keyboard shortcuts to perform the desired actions. You can check the keyboard shortcuts on IntelliJ and you can also check the Productivity Guide which monitors your most used actions. This information is very useful, but it may be a little difficult to change your habits right away. To help you with this, I will describe my most used shortcuts in IntelliJ. You can start familiarize yourself with these and slowly introduce additional shortcuts.

ShortcutDescription
CTRL + W / CMD + W Syntax Aware Selection

This allows you to select code with context. Awesome when you need to select large blocks or just specific parts of a piece of code. If you have this code:

files.getFiles().forEach(auctionFile -> createAuctionFile(realm, auctionFile));

And place the cursor in the auctionFile and use the shortcut, it will select auctionFile. Press it again and the selection will expand to auctionFile -> createAuctionFile(realm, auctionFile). If you press it again, now the selection will expand to files.getFiles().forEach(auctionFile -> createAuctionFile(realm, auctionFile)). Pressing a final time, you get the full piece of code selected.

If you combine it with SHIFT, you can unselect by context as well.

CTRL + E / CMD + E Recent Viewed Files

This will show you a popup with all the recent files that you have opened in the IDE. If you start typing, you can filter the files.

IDEA Recent Files
CTRL + SHIFT + E / CMD + SHIFT + E Recent Edited Files

Same as Recent Viewed Files, but only shows you the files that you’ve actually changed.
CTRL + B / CMD + B Go to Declaration

If you place the cursor in a class, method or variable and use the shortcut you will immediately jump to the declaration of the element.
CTRL + SHIFT + ENTER / CMD + SHIFT + ENTER Complete Statement

This will try to complete your current statement. How? By adding curly braces, or semicolon and line change. For instance, if you have the following statement:

System.out.print()
Press the shortcut once to add an ending semi-colon. Press it again to add a new line and to position the cursor aligned with the last line.

Another example:
if (condition == true)
Press the shortcut to add opening and closing curly braces, and place the cursor inside the if body with additional indentation.

CTRL + N / CMD + N Go to Class

This one allows you to search by name for a Java file in your project. If you combine it with SHIFT, it searches any file. Adding ALT on top of that it searches for symbols. In the search area, you can use CamelHumps (type the capital letters only of the class name) notation to filter files.

IDEA Go to Class
CTRL + SHIFT + SPACE / CMD + SHIFT + SPACE Smart Type Completion

I didn’t mention it before, but I guess you are familiar with auto complete via CTRL + SPACE or CMD + SPACE. If you add a SHIFT you get the smart completion. This means that the IDE will try to match expected types that suit the current context and filter all the other options.
CTRL + ALT + ← / CMD + ALT + ← Navigate Back

This allows you to navigate back like a browser action. It remembers where your cursor was positioned and navigates back even to other files.
CTRL + ALT + → / CMD + ALT + → Navigate Forward

It’s like Navigate Back but goes Forward. Duh!
CTRL + SHIFT + F7 / CMD + SHIFT + F7 Highlight Usages

Place the cursor in a element and after pressing the cursor the IDE will highlight all the occurrences of the selected element.

There are many more keyboard shortcuts. Almost every action has an equivalent shortcut. It’s hard to learn them all, it takes time and practice. I still learn new things every week, and if for some reason I don’t code as much for a few days, I forget about the new shortcuts I’ve learned. It’s practice, practice, practice! Try to learn a few and master them instead of trying to do everything in one go. It’s easier!

An IntelliJ plugin exists to tell you with shortcuts you should use if you use the mouse. Its Key Promoter, but unfortunately it seems it’s not maintained anymore. Maybe I can update it for the latests IntelliJ versions. I would also like to see in the Productivity Guide a count of actions performed by shortcuts or mouse. If I find some free time, maybe I can do it too.

Hope you enjoyed it.