Saturday, May 24, 2014

How to implement Jackson filters to filter the REST JSON response

Background

We are building an application that supports REST calls to our API. We use JBoss AS 7.1.1 and its default framework 'RestEasy' and Jackson v1.9 JSON processor. Since it is a REST application, we may need to provide provision for the clients to filter the JSON response. As other RESTFul service application does, we planned to have a query parameter called "fields" to do this. For instance, the URL that the client has to submit will be,https:///rest/product/123?fields=id,name

The above GET request URL should return the JSON object of type "product" matching id=123. But note that the response JSON object should contain only the "id" and "name" fields in it as follows: {"id": "123", "name":"xyz"}

Implementation

To do this, we may need to fetch the value of query parameter "fields" from HttpRequest. This could be achieved using javax.ws.rs annotation @QueryParam as given below.


@GET
@Produces (MediaType. APPLICATION_JSON) 
@Path( "/{id:" + ID_PATTERN + "}" ) 
public String getProduct(@PathParam ("id" ) String id, 
        @QueryParam ("fields" ) String fields) { 
    String productJson = null ; 
    final ProductModel model = productService.read(id);         
    try { 
        productJson = filterJsonFields(model, ProductModel.class.getName(), fields); 
    } catch (IOException e) { 
        throw new ApplicationRuntimeException( 
                e.getCause() + ": " + e.getLocalizedMessage()) 
                .setWebStatus(Status. INTERNAL_SERVER_ERROR ); 
    } 
  return productJson ; 
}


After fetching the whole Product object from the database, we could filter the fields of the object while serializing the Java object to JSON object, which will be returned as the response. This filtering mechanism can be done by many ways using Jackson processor. But I know two ways of doing this.
  1. Using @JsonFilter
  2. Overiding the JacksonAnnotationIntrospector

1. Using @JsonFilter

This is the simple way to filter the response object. For this, we need to declare the @JsonFilter annotation on top of our model bean classes. 

@JsonFilter(com.test.ProductModel)  
public class ProductModel implements IModel {  
    private String id ;  
    private String name ;  
    ....  
    ....  
    ...getter and setter methods...
}


Filtering mechanism

 public static String filterJsonFields(IModel model,  
  String jsonFilter, String fieldsQueryParam) throws IOException { 
          
    String jsonResult = null ;  
    SimpleBeanPropertyFilter filter = SimpleBeanPropertyFilter.  
                     serializeAllExcept(Collections.<String> emptySet());  
    Set<String> filterProps = new HashSet<String>();      
    if (fieldsQueryParam != null) {  
        StringTokenizer st = new StringTokenizer(fieldsQueryParam, "," );  
        while (st.hasMoreTokens()) {  
            filterProps.add(st.nextToken());  
        }  
        filter = SimpleBeanPropertyFilter.filterOutAllExcept(filterProps);  
    }         
    ObjectMapper mapper = new ObjectMapper();  
    FilterProvider fProvider = new SimpleFilterProvider().addFilter(jsonFilter, filter)  
                        // setFailOnUnknownId: Ignore filtering the reference member fields  
                        // Refer: https://jira.codehaus.org/browse/JACKSON-650  
                        .setFailOnUnknownId( false );  
    try {  
        jsonResult = mapper.writer(fProvider).writeValueAsString(model);  
    } catch (IOException e) {  
        throw e;  
    }  
    return jsonResult;  
}   
   

The "jsonFilter" method parameter is the class name of the model bean that is passed from the caller method (getProduct()).

Caveat: But this didn't work well with JBoss's RestEasy.


2. Overriding the JacksonAnnotationIntrospector

In this method, we need to extend the JacksonAnnotationIntrospector and override the method "findFilterId". During serialization, Jackson calls the JacksonAnnotationIntrospector.findFilterId() to identify the filter of the model bean. In our case, we are using the name of the model bean class as its filter name. 

import org.codehaus.jackson.map.introspect.AnnotatedClass;  
import org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector;  
   
public class MyFilteringIntrospector extends JacksonAnnotationIntrospector {  
       
    @Override  
    public Object findFilterId(AnnotatedClass ac) {  
        Object id = super .findFilterId(ac); 
        if (id == null ) {  
            // the name of the class will be used as its filter name  
            id = ac.getName();  
        }  
        return id;  
    }  
} 

We may need to tell Jackson to use our custom AnnotationIntrospector and not JacksonAnnotationIntrospector. For this, 

AnnotationIntrospector myFilterIntrospector = new MyFilteringIntrospector();
mapper.setAnnotationIntrospector(myFilterIntrospector );

Hence the filterJsonFields() method is updated with the above lines of code.


Filtering mechanism

 public static String filterJsonFields(IModel model,  
           String jsonFilter, String fieldsQueryParam) throws IOException {  
   
    String jsonResult = null ;  
    SimpleBeanPropertyFilter filter = SimpleBeanPropertyFilter.  
                    serializeAllExcept(Collections.<String> emptySet());  
    Set<String> filterProps = new HashSet<String>();      
    if (fieldsQueryParam != null) {  
        StringTokenizer st = new StringTokenizer(fieldsQueryParam, ",");  
        while (st.hasMoreTokens()) {  
            filterProps.add(st.nextToken());  
        }  
        filter = SimpleBeanPropertyFilter.filterOutAllExcept(filterProps);  
    }       
    ObjectMapper mapper = new ObjectMapper();  
    AnnotationIntrospector myFilterIntrospector = new MyFilteringIntrospector();  
    mapper.setAnnotationIntrospector(myFilterIntrospector);  
    FilterProvider fProvider = new SimpleFilterProvider().addFilter(jsonFilter, filter)  
                  // setFailOnUnknownId: Ignore filtering the reference member fields  
                  // Refer: https://jira.codehaus.org/browse/JACKSON-650  
                  .setFailOnUnknownId( false );  
    try {  
        jsonResult = mapper.writer(fProvider).writeValueAsString(model);  
    } catch (IOException e) {  
        throw e;  
    }  
    return jsonResult;  
}   

Friday, May 23, 2014

How to enable RestEasy in JBoss AS 7.1 to use Jackson processor

Problem 

The environment used is JBoss AS 7.1 which owns RestEasy (a framework to build RESTful webservices). JBoss by default uses RestEasy framework. I added the Jackson annotation @JsonSerialize (include=JsonSerialize.Inclusion.NON_NULL) on top of my REST model classes to filter the NULL fields from the JSON response during serialization. But, RestEasy ignored this Jackson annotation.

Solution

With reference to some forums in internet, I came to know the following. To enable Jackson processor for RestEasy, the below maven dependency and configuration need to be done. If not, RestEasy may use Jettison processor.

1. Maven Jackson dependency
<groupId>org.jboss.resteasy</groupId>  
      <artifactId>resteasy-jaxrs</artifactId>  
      <version>${version.jboss.resteasy}</version>  
      <scope>provided</scope>  
 </dependency>  
   
 <dependency>  
      <groupId>org.jboss.resteasy</groupId>  
      <artifactId>resteasy-jackson-provider</artifactId>  
      <version>${version.jboss.resteasy}</version>  
      <scope>provided</scope>  
 </dependency>  
2. Create a file under named "jboss-deployment-structure.xml" under the WEB-INF folder and include the jackson modules in the dependencies.
<? xml version= "1.0" encoding = "UTF-8"?>  
< jboss-deployment-structure>  
   <deployment >  
     < dependencies>  
       <!-- added the below jackson modules to enable JBoss RestEasy to  
       use Jackson -->  
       < module name= "org.codehaus.jackson.jackson-core-asl" />  
       < module name= "org.codehaus.jackson.jackson-mapper-asl" />  
     </ dependencies>  
   </deployment >  
 </ jboss-deployment-structure>  

Monday, September 23, 2013

Liferay: Create a quartz job scheduler

In two ways a job can be scheduled in Liferay.
  1. Configure  in liferay-portlet.xml
  2. Implement it in code.
1. Configure  in liferay-portlet.xml
  1. Override the property 'scheduler.enabled = true' in portal-ext.properties.
  2. Create a new dummy portlet in liferay-portlet.xml as follows,
     <portlet>  
         <portlet-name>job</portlet-name>  
         <scheduler-entry>  
             <scheduler-event-listener-class>com.madrone.portal.job.PortalJob</scheduler-event-listener-class>  
             <trigger>  
                 <simple>  
                     <simple-trigger-value>5</simple-trigger-value>  
                     <time-unit>minute</time-unit>  
                 </simple>  
                 <!-- cron>  
                     <property-key>0 0 0 * * ?</property-key>  
                 </cron-->  
             </trigger>  
         </scheduler-entry>  
     </portlet>  
    
  3. Add this portlet in portlet.xml
     <portlet>  
          <portlet-name> job</portlet-name>  
          <display-name>Job</display-name>  
          <portlet-class>  
              com.madrone.portal.job.PortalJob  
          </portlet-class>  
          <supports>  
               <mime-type>text/html</mime-type>  
               <portlet-mode>view</portlet-mode>  
          </supports>  
          <security-role-ref>  
               <role-name>administrator</role-name>  
          </security-role-ref>  
          <security-role-ref>  
               <role-name>guest</role-name>  
          </security-role-ref>  
          <security-role-ref>  
               <role-name>power-user</role-name>  
          </security-role-ref>  
          <security-role-ref>  
               <role-name>user</role-name>  
          </security-role-ref>  
     </portlet>  
    
  4. Create a job listener
     public class PortalJob implements MessageListener {  
         private static final Log LOG = LogFactory.getLog(PortalJob.class);  
        
         @Override  
       public void receive(Message arg0) {  
         LOG.info("PortalJob: Starts" );  
         // your code   
         LOG.info("PortalJob: Stops" );         
       }  
     }  
    
2. Implement it in code. 
  1. Override the property 'scheduler.enabled = true' in portal-ext.properties.
  2. Create an action class which is run during server startup
     public class PortalStartupAction extends SimpleAction {  
       private static final Log LOG = LogFactory.getLog(PortalStartupAction .class );  
       
       @Override  
       public void run(String[] arg0) throws ActionException {  
         LOG.info("Preparing to Schedule the job: PortalJob" );  
         Thread thread = Thread. currentThread();  
         try {  
                 SchedulerEngineUtil. schedule(PortalJob.SCHEDULER_ENTRY, thread.getContextClassLoader());  
         } catch (SchedulerException e) {  
           throw new ActionException(e);  
         }     
         LOG.info("PortalJob scheduled successfully." );  
       }  
     }  
    
  3. Configure the above action class in portal-ext.properties
     #  
     # Application startup event that runs once for every web site instance of  
     # the portal that initializes.  
     #  
     application.startup.events=com.madrone.portal.util.PortalStartupAction  
    
  4. Create a job
     public class PortalJob implements MessageListener {  
         private static final Log LOG = LogFactory.getLog(PortalJob.class);  
       
         @SuppressWarnings("serial" )  
         public static final SchedulerEntry SCHEDULER_ENTRY = new SchedulerEntryImpl() {  
              {  
              setDescription( "Job that get triggered every 5 minutes");  
                
              setTriggerType(TriggerType. SIMPLE);  
              setTimeUnit(TimeUnit. MINUTE);  
              setTriggerValue( "5");  
                
              //setTriggerType(TriggerType.CRON);  
              //setTriggerValue("0 0 0 * * ?"); // Fire at 12:00 AM every midnight  
              // Reference: http://www.quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger  
              setEventListenerClass(PortalJob.class.getName());  
              }  
         };  
       
         @Override  
         public void receive(Message arg0) {  
             LOG.info("CRPortalERPJob: Starts" );  
                 // your code  
           LOG.info("CRPortalERPJob: Stops" );         
       }  
     }  
    

Wednesday, September 4, 2013

Observer pattern

Java has two utility classes to serve Observer pattern - Observer and Observable. 
  • The aim of this pattern is to notify the objects of Observer class, when a change is done to the objects of Observable class.
  • Observer class should implement Observer interface and override the method 'update'.
  • Observable class should extend Observable class. The key method of this class is 'setChanged()'. This marks the Observable object being changed, only then the notifyObservers() will notify the observer objects that a change was done to the observable object. The notifyObservors() makes call to the Observor's update() method.
  • How do the observable object knows its observers? The observers should be added to the observable objects using the methods addObserver() of Observable class. An observer can be added or removed from the observable object.
 
1:  import java.util.ArrayList;  
2:  import java.util.List;  
3:  import java.util.Observable;  
4:  import java.util.Observer;  
5:    
6:  public class ObserverPattern {  
7:        
8:      public static void main(String []args) {          
9:          Stock s = Stock.getInstance();  
10:          s.setName("Madrone");  
11:          s.setNumShares(10);  
12:            
13:          // Observers  
14:          List<Buyer> buyerList = new ArrayList<Buyer>();  
15:          buyerList.add(new Buyer("A"));  
16:          buyerList.add(new Buyer("B"));  
17:            
18:          for(Buyer b : buyerList) {  
19:              s.addObserver(b);  
20:          }  
21:            
22:          for(int i=0; i<10; i++) {  
23:              s.setNumShares(s.getNumShares() + 1); // Observed object is modified   
24:              s.notifyObservers();  
25:          }      
26:            
27:          System.out.println("Shares available: " + s.getNumShares());  
28:          for(Buyer b : buyerList) {  
29:              System.out.println(b.getName() + " : " + b.getNumShares());  
30:          }  
31:      }  
32:  }  
33:    
34:  class Buyer implements Observer {  
35:        
36:      private String name;  
37:      private int numShares;  
38:        
39:      Buyer(String name) {  
40:          this.name = name;  
41:      }  
42:    
43:      @Override  
44:      public void update(Observable observable, Object ob) {          
45:          Stock s = (Stock) observable;  
46:          if(s.getNumShares() > 0) {  
47:              buyAShare();  
48:              s.setNumShares(s.getNumShares() - 1);  
49:          }      
50:      }  
51:        
52:      private void buyAShare() {  
53:          this.setNumShares(this.getNumShares() + 1);  
54:      }  
55:        
56:      public String getName() {  
57:          return name;  
58:      }  
59:      public void setName(String name) {  
60:          this.name = name;  
61:      }  
62:      public int getNumShares() {  
63:          return numShares;  
64:      }  
65:      public void setNumShares(int numShares) {  
66:          this.numShares = numShares;  
67:      }  
68:  }  
69:    
70:  // Singleton class  
71:  class Stock extends Observable {  
72:        
73:      private String name;  
74:      private int numShares;  
75:        
76:      public static final Stock INSTANCE = new Stock();  
77:      private Stock() {}  
78:            
79:      public static Stock getInstance() {  
80:          return INSTANCE;      
81:      }  
82:        
83:      public String getName() {  
84:          return name;  
85:      }  
86:      public void setName(String name) {  
87:          this.name = name;  
88:      }  
89:      public int getNumShares() {  
90:          return numShares;  
91:      }  
92:      public void setNumShares(int numShares) {  
93:          this.numShares = numShares;  
94:          setChanged();  
95:      }  
96:  }  

Wednesday, July 10, 2013

Oracle jdbc driver was unable to connect the database server

Today we faced a problem that a scheduled job in our production server was unable to connect to Oracle database server. The error reported was,

java.sql.SQLRecoverableException: No more data to read from socket
        at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1157)
        at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290)
        at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
        at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
        at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:204)
        at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:1041)
        at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
        at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
        at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3665)
        at oracle.jdbc.driver.OracleCallableStatement.executeUpdate(OracleCallableStatement.java:4739)
        at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1352)
        at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
        at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)

We found some details in the oracle forums that the cause for this could be due to the driver version incompatibility with the Oracle database version. But on analyzing the logs, we found that the job was failing only for the past two days. After checking this with the Oracle database team, we were informed that there was no change or upgrade done to the oracle database, but they have changed the infrastructure of the servers, which means the IP address of the database server was changed. But this should not affect us since we have configured the server url with the host name and not the IP address.

On further analysis, we found that this was due to the dns cache which was enabled (networkaddress.cache.ttl=-1) in the jvm (where the production tomcat was running) in  ~\jre\lib\security\java.security

'networkaddress.cache.ttl' is a security property which works for the below settings:

# any negative value: caching forever   (default)
# any positive value: the number of seconds to cache an address for
# zero: do not cache   
 
It is also advised that in the comments that setting this to anything other than the default value can have serious security implications. Do not set it unless you are sure you are not exposed to DNS spoofing attack.

Back to the problem - Since the dns caching was enabled, the old IP address was cached which was the cause for this problem. As a fix to this problem, we restarted the tomcat servers to flush the jvm dns cache.

Friday, July 5, 2013

Basic Git commands

1. Create a Git repository and push it to remote

Open the Git bash (Unix bash prompt) for the respective folder and execute the following commands

git init
git add *
git commit -m "initial commit message"
git remote add origin https://github.com/xxxxx/xxxx.git
git push -u origin master
 

2. Clone repository

 git clone https://github.com/xxxxx/xxxx.git

3. Create new branch

git branch branch-name
git checkout branch-name

or

git checkout -b branch-name

4. Delete a local branch

git branch -d branch-name

Delete a Remote branch 

git push origin --delete branch-name

Tips to configure Git in Windows

We were facing the problem with 'EGit- Synchronize workspace' which showed all the files as changed, even if we haven't changed the content of those files. This happened only when we clone/ pull the repository using Git bash (Unix bash prompt). As we all know the reason for this problem - Windows uses both a carriage-return character and a linefeed character for newlines in its files, whereas Linux/ Mac use only the linefeed character.
Git Hub provides some configurations to override these cross-platform issues.
Do the following before cloning the remote repository, I have the folder structure as C:\Git\Project\, where I would clone the repository inside this folder.
  1. Open Git bash at C:\Git\Project\
  2. Enter, git config --system core.autocrlf true and click enter
  3. Enter, git config --system core.whitespace cr-at-eol and click enter
  4. Then clone the repository as usual, git clone 'https://github.com/xxxxx/xxxx.git' 
  5. Then open the local repository in your Eclipse and import projects. 
  6. Right click on your projects > Team > Share to Git
  7. Right click on your projects > Team > Synchronize workspace
  8. In your Team Synchronize perspective, check the list of modified files. No files will be shown.

To know more about the commands that we used in step 2 and 3, refer http://git-scm.com/book/en/Customizing-Git-Git-Configuration