Broken special chars in Spring Roo internationalization?

This is the first post in an upcoming series of posts on Spring Roo (see footer for details an Roo).

Problem: Special chars and Umlauts are not correctly displayed in my Roo-Application

[german version]
There are often questions and support requests on the above named topic in Spring Roo-Forum. Mostly questioners write that german "Umlauts" like ä, ö, ü and ß or other special chars in cyrillic are not correctly display or saved to the database. With the following post I'd like to review the most common sources of the problem and come up with solutions.

Encoding of the HTML/ JSP page

Roo generated JSPs are always delivered with UTF-8 encoding. In case you write your additional JSPs by hand, you should assure these pages are also delivered to the browser in UTF-8 encoding so the form data will be correctly send on submit.
This can be achieved by specifying the following line of code at the beginning of each JSP:

<jsp:directive.page contenttype="text/html;charset=UTF-8"
   pageencoding="UTF-8"/>

To show you an example where to exactly put this, the "default.jspx", which serves a frame for all other JSP pages via Apache Tiles, contains the following lines to preserve UTF-8 encoding in the resulting HTML:

<html xmlns:jsp="http://java.sun.com/JSP/Page" xmlns:c="http://java.sun.com/jsp/jstl/core" xmlns:tiles="http://tiles.apache.org/tags-tiles" xmlns:spring="http://www.springframework.org/tags" xmlns:util="urn:jsptagdir:/WEB-INF/tags/util" >  
 
 <jsp:output doctype-root-element="HTML" doctype-system="about:legacy-compat" />
 
 <jsp:directive.page contentType="text/html;charset=UTF-8" />  
  
 <head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

<!-- ... -->

To check whether your page is delivered in UTF-8 encoding, open it in your browser and have a look at menu "View > Character Encoding".

CharacterEncodingFilter in web.xml file

This is the most common point of failure. We're talking about the order in which the <filter-mapping>-Tags occur in your "src/main/webapp/WEB-INF/web.xml". These tags define in which order the filters are executed on each HTTP-Request.
Roo generates the correct order since Version 1.1.1! Previous versions are still buggy.

The correct order looks like the following:

<filter-mapping>
    <filter-name>CharacterEncodingFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>Spring OpenEntityManagerInViewFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>HttpMethodFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>springSecurityFilterChain</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

In case you want to change the order it is important to understand at least a minimum of the inner workings of these filters:

CharacterEncodingFilter - Sets the encoding on the HTTP-Request in case the Browser didn't send the encoding of the HMTL-page/ form
OpenEntityManagerInViewFilter - only for Hibernate, sets the EntityManager to a ThreadLocal (see discussion on Pros and Cons)
HttpMethodFilter - Necessary to really perform e.g. HTTP PUT or HTTP DELETE requests!
springSecurityFilterChain - has to be run AFTER HttpMethodFilter in case you check HTTP-Method in Spring Security
... other filters

To avoid problems with the encoding of your special chars, "CharacterEncodingFilter" has to be run before any other filter has worked on the request data!
For this reason the general advice is: the <filter-mapping> for CharacterEncodingFilter has to be at the first position in your "web.xml" (See ROO-1698 for an explanation - the charset CANNOT be set to the request after the InputStream has been read)!

Database connection

The last point is to assure that your application communicates with the database via unicode with a default encoding of UTF-8.

As Roo generates the correct connection string for MySQL since "the early days" there haven't been many bugs with this lately.
The connection string can be configured in "src/main/resources/META-INF/spring/database.properties" and looks like the following for MySQL:
database.url=jdbc:mysql://[YOUR_DB_SERVER]:3306/[YOUR_DB_NAME]?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8

Conclusion

Since version 1.1.1 Spring Roo provides a (by now) bug free support for special chars and umlauts, which as been tested with various languages worldwide.

If you have any questions on Spring Roo or the topic named above you can find competent assistance in the Roo-Forum.

References/ Links used in the text

ROO-JIRA issues

Threads in ROO-Forum

Other

Spring Roo is a Code-Generation-Tool for Software development in Java. It is a lightweight developer tool that makes it fast and easy to deliver instant results. Best of all, you code 100% in Java and get to reuse all your existing Java knowledge, skills and experience.

There are already many Add-Ons available to customize Roo to your special needs. In case you don't find an Add-On that fits your needs, you can easily create your own Add-On in just a few steps.