Tuesday, September 23, 2014

Authentication using Apache Shiro

Introduction

Authentication is the process of identity verification. A user should provide some information as an identity that the system understands so as to allow the user to login the system. This document covers one of Shiro's authentication mechanism that can be used for a java RESTful application.

Shiro

Apache Shiro is a java security framework that offers solution for authentication, authorization and session management for an application. Since  RESTful application are stateless, we will not deal with sessions. For every request URL, the user is authenticated. Four basic terms that Shiro uses are: 
(1) Subject: refers to the user who is using the system. It can be human-being, another application, cron job etc.
(2) Principals: refers to attributes that identify the subject, like username first name, last name, ssn
(3) Credentials: refers to secret data like password, certificates etc.
(4) Realms: this is Shiro's security specific DAO which talks to the application's back end datasource or another DAO.

Shiro configuration file (shiro.ini)

During the application deployment, Shiro loads its configuration file (shiro.ini). They call it as the initialization file where the Realm, Credential matcher, authentication filter and few more settings that are to be used in the application are configured.

shiro.ini
 [main]  
 authc.loginUrl = /web/login.jsp  
 customRealm = com.app.security.shiro.realm.CustomRealm  
 credentialsMatcher = com.app.security.shiro.checker.CustomCredentialsMatcher  
 credentialsMatcher.hashAlgorithmName = SHA-512  
 credentialsMatcher.hashIterations = 1024  
 # base64 encoding, not hex since it requires less space than hex  
 credentialsMatcher.storedCredentialsHexEncoded = false  
 customRealm.credentialsMatcher = $credentialsMatcher  
 # customRealm.dataSourceName should be the same that is used in "persistence.xml"  
 customRealm.dataSourceName = java:jboss/datasources/appDS  
 securityManager.realms = $customRealm  
 [urls]  
 # authc means the default FormAuthenticationFilter used for FORM level login  
 /web/login.jsp = authc  
 # REST URL authentication. authcBasic means the default BasicHttpAuthenticationFilter  
 # "rest": HttpMethodPermissionFilter, to be used in future for authorization  
 /rest/** = authcBasic, rest  

Shiro Filter

Shiro provides a request filter called ShiroFilter which needs to be added in our web.xml such that all the request to the application pass through this filter before being processed by the server. The Shiro filter internally forwards the request to the Authentication filter that we have configured in shiro.ini file. For our application, we used 'BasicHttpAuthenticationFilter' (which is marked as 'authcBasic' in shiro.ini). This filter internally uses the Realm DAO and the CredentialsMatcher to validate the credentials for authentication. If the user is successfully validated, then the request is forwarded to the application's REST layer to process further. 

web.xml
 <?xml version="1.0" encoding="UTF-8"?>  
 <web-app xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">  
   <listener>  
     <listener-class>org.apache.shiro.web.env.EnvironmentLoaderListener</listener-class>  
   </listener>  
   <filter>  
     <filter-name>ShiroFilter</filter-name>  
     <filter-class>org.apache.shiro.web.servlet.ShiroFilter</filter-class>  
   </filter>  
   <filter-mapping>  
     <filter-name>ShiroFilter</filter-name>  
     <url-pattern>/*</url-pattern>  
   </filter-mapping>  
   <session-config>  
    <session-timeout>30</session-timeout>  
   </session-config>  
 </web-app>  

Realm and CredentialMatcher

The principal (username) and credential (password) are retrieved from the "Authorization" URL header attribute. They are base64 encoded and are given in the following format, "Basic YWRtaW46YWRtaW4=" where 'YWRtaW46YWRtaW4=' is the base 64 encode of username:password.
The ShiroFilter internally constructs the AuthenticationToken using the username & password. The BasicAuthenticationFilter internally validates this token with the user authentication information in the application database. This is done using CustomRealm and CustomCredentialsMatcher.
CustomRealm is a custom Realm which has its own implementation to fetch the authentication information of the respective user from the application database. We use just plain JDBC to fetch the user record matching the username in the Authentication token. Plain JDBC has been used due to a Shiro limitation which is given below. Finally the AuthenticationToken and the user info fetched from database are passed to the CustomCredentialsMatcher for validation.
Shiro limitation: Apache Shiro is unaware of CDI injection in Realm. The injected UserDAO was not instantiated during CustomRealm object instantiation. There are a couple of tickets in Shiro's queue for this. Refer: https://issues.apache.org/jira/browse/SHIRO-337, https://issues.apache.org/jira/browse/SHIRO-422. Hence as a workaround, currently CustomRealm extends JdbcRealm and we fetch the user object matching username through direct JDBC queries. In future if Shiro resolves the above tickets, we can remove the plain JDBC stuff and enable the injected UserDAO code. 

CustomRealm.java
 public class CustomRealm extends JdbcRealm { //JdbcRealm extends AuthorizingRealm and AuthenticatingRealm   
   private static final Logger log = Logger.getLogger(CustomRealm.class);  
   protected static final String DEFAULT_AUTHENTICATION_QUERY = "SELECT user_name, password, password_salt "  
       + "FROM user WHERE user_name = ?";   
   // @Inject protected UserDAO userDAO;  
   public CustomRealm() {  
     super();  
     setName("CustomRealm");  
     CustomCredentialsMatcher credentialMatcher = new CustomCredentialsMatcher();  
     setCredentialsMatcher(credentialMatcher);  
     log.info("CustomRealm instantiated!");      
   }  
   protected String dataSourceName; // java:jboss/datasources/appDS - configured in shiro.ini  
   public String getDataSourceName() {   
     return dataSourceName; // java:jboss/datasources/appDS  
   }   
   public void setDataSourceName(String dataSourceName) {   
     this.dataSourceName = dataSourceName;   
     this.dataSource = getDataSourceFromJNDI(dataSourceName); // setting datasource to JdbcRealm.datasource  
   }   
   private DataSource getDataSourceFromJNDI(String jndiDataSourceName) {   
     try {   
       InitialContext ic = new InitialContext();   
       return (DataSource) ic.lookup(jndiDataSourceName);   
     } catch (NamingException e) {   
       log.error("JNDI error while retrieving " + jndiDataSourceName, e);   
       throw new AuthorizationException(e);   
     }   
   }   
   @Override  
   protected AuthenticationInfo doGetAuthenticationInfo(AuthenticationToken token)  
       throws AuthenticationException {  
     log.info("Do Shiro doGetAuthenticationInfo...");  
     //identify account to log to  
     String username = String.valueOf(token.getPrincipal());  
     checkNotNull(username, "Null usernames are not allowed by this realm.");  
     log.info("---------------------------------------------------------------------");  
     log.info("Principal: " + String.valueOf(token.getPrincipal())  
         + " - Credentials: " + String.valueOf(token.getCredentials()));     
     log.info("---------------------------------------------------------------------");  
     String hashedPassword = null;  
     ByteSource credentialsSalt = null;  
     if("admin".equalsIgnoreCase(username)) {  
       hashedPassword = "admin";  
     } else {  
       IUser user = null;  
       // user = userDAO.findByUserName(username);    
       Connection conn = null;  
       try {  
         conn = dataSource.getConnection();  
         user = getUserByUserName(conn, username);  
       } catch (SQLException e) {  
         throw new AuthenticationException("Unable to fetch user by username ["   
             + username + "].", e);  
       } finally {  
         JdbcUtils.closeConnection(conn);  
       }  
       checkNotNull(user, "No account found for user [" + username + "]");  
       hashedPassword = user.getPassword();  
       credentialsSalt = new SimpleByteSource(Base64.decode(user.getPasswordSalt()));  
     }  
     // SimpleAuthenticationInfo implements SaltedAuthenticationInfo  
     return new SimpleAuthenticationInfo(token.getPrincipal(), hashedPassword, credentialsSalt, getName());  
   }  
   @Override  
   protected AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection arg0) {  
     // TODO Auto-generated method stub  
     return null;  
   }  
   private IUser getUserByUserName(Connection conn, String username)   
       throws SQLException {   
     PreparedStatement ps = null;   
     ResultSet rs = null;   
     IUser user = null;  
     try {   
       ps = conn.prepareStatement(DEFAULT_AUTHENTICATION_QUERY);   
       ps.setString(1, username);   
       rs = ps.executeQuery();   
       // Loop over results - although we are only expecting one result,   
       // since usernames should be unique   
       boolean foundResult = false;   
       while (rs.next()) {   
         // Check to ensure only one row is processed   
         if (foundResult) {   
           throw new AuthenticationException(   
               "More than one user row found for user ["   
                   + username + "]. Usernames must be unique."); // TODO work on this warning  
         }   
         user = new User();  
         user.setUserName(rs.getString("user_name"));  
         user.setPassword(rs.getString("password"));  
         user.setPasswordSalt(rs.getString("password_salt"));  
         foundResult = true;   
       }   
     } finally {   
       JdbcUtils.closeResultSet(rs);   
       JdbcUtils.closeStatement(ps);   
     }   
     return user;   
   }   
   private void checkNotNull(Object reference, String message) {  
     if (reference == null) {  
       throw new AuthenticationException(message);  
     }  
   }  
 }  

CustomCredentialsMatcher.java
 public class CustomCredentialsMatcher extends HashedCredentialsMatcher {  
      
   private static final Logger log = Logger.getLogger(CustomCredentialsMatcher.class);  
      private static final SHIRO_CREDENTIAL_HASH_INTERATION = 1024;  
      
   public CustomCredentialsMatcher() {  
     super();  
     log.info("CustomCredentialsMatcher instantiated!");  
     setHashAlgorithmName(Sha512Hash.ALGORITHM_NAME);  
     setHashIterations(SHIRO_CREDENTIAL_HASH_INTERATION);  
     setStoredCredentialsHexEncoded(false); // Base64-encode, not hex-encode  
   }  

   @Override  
   public boolean doCredentialsMatch(AuthenticationToken token, AuthenticationInfo info) {  
        
     log.info("Shiro doCredentialsMatch...");  
        
     log.info("AuthenticationToken.credentials: " + new String((char[]) token.getCredentials()));  
     log.info("AuthenticationToken.principals: " + token.getPrincipal());  
     log.info("AuthenticationInfo.credentials: " + info.getCredentials());  
     log.info("AuthenticationInfoprincipals: " + info.getPrincipals());  
     if("admin".equalsIgnoreCase(String.valueOf(token.getPrincipal()))) {  
       return true;  
     } else {  
       // verifies by comparing the credential in token and info  
       return super.doCredentialsMatch(token, info);  
     }   
   }  
 }  

Schema design: User (user)
  • user_name : username to login
  • password  : password to login
  • passwordSalt : a secure random number used as a salt while encrypting/decrypting the password.
It is insecure to store the password as plain string in the database. Hence we encrypt the password using Shiro's Sha512Hash algorithm, also a password salt is used for this to make it more secure. Note that the hashing algorithm (Sha512Hash) used for this password crypt is configured in shiro.ini.
UserService.java
 public class UserService {  
     @Inject UserDAO userDAO;  
       
     public void createUser(UserBean userBean) {  
         String id = // generate a uuid  
         userBean.setId(id);           
         if(userBean.getPassword != null) {  
             // We'll use a Random Number Generator to generate salts. This is much more secure  
             // than using a username as a salt or not having a salt at all.  
             RandomNumberGenerator rng = new SecureRandomNumberGenerator();  
             Object salt = rng.nextBytes();  
             //hash the plain-text password with the random salt and multiple  
             //iterations and then Base64-encode the value (requires less space than Hex)  
             String final SHIRO_CREDENTIAL_HASH_INTERATION = 1024;  
             String hashedPasswordBase64 = new Sha512Hash(this.password, salt,  
                     SHIRO_CREDENTIAL_HASH_INTERATION).toBase64();  
             userBean.setPassword(hashedPasswordBase64);  
             userBean.setPasswordSalt(salt.toString());              
         } else {  
             throw new ApplicationRuntimeException("password is null");  
         }  
         userDAO.createUser(userBean);  
     }  
 }      

Saturday, May 24, 2014

How to implement Jackson filters to filter the REST JSON response

Background

We are building an application that supports REST calls to our API. We use JBoss AS 7.1.1 and its default framework 'RestEasy' and Jackson v1.9 JSON processor. Since it is a REST application, we may need to provide provision for the clients to filter the JSON response. As other RESTFul service application does, we planned to have a query parameter called "fields" to do this. For instance, the URL that the client has to submit will be,https:///rest/product/123?fields=id,name

The above GET request URL should return the JSON object of type "product" matching id=123. But note that the response JSON object should contain only the "id" and "name" fields in it as follows: {"id": "123", "name":"xyz"}

Implementation

To do this, we may need to fetch the value of query parameter "fields" from HttpRequest. This could be achieved using javax.ws.rs annotation @QueryParam as given below.


@GET
@Produces (MediaType. APPLICATION_JSON) 
@Path( "/{id:" + ID_PATTERN + "}" ) 
public String getProduct(@PathParam ("id" ) String id, 
        @QueryParam ("fields" ) String fields) { 
    String productJson = null ; 
    final ProductModel model = productService.read(id);         
    try { 
        productJson = filterJsonFields(model, ProductModel.class.getName(), fields); 
    } catch (IOException e) { 
        throw new ApplicationRuntimeException( 
                e.getCause() + ": " + e.getLocalizedMessage()) 
                .setWebStatus(Status. INTERNAL_SERVER_ERROR ); 
    } 
  return productJson ; 
}


After fetching the whole Product object from the database, we could filter the fields of the object while serializing the Java object to JSON object, which will be returned as the response. This filtering mechanism can be done by many ways using Jackson processor. But I know two ways of doing this.
  1. Using @JsonFilter
  2. Overiding the JacksonAnnotationIntrospector

1. Using @JsonFilter

This is the simple way to filter the response object. For this, we need to declare the @JsonFilter annotation on top of our model bean classes. 

@JsonFilter(com.test.ProductModel)  
public class ProductModel implements IModel {  
    private String id ;  
    private String name ;  
    ....  
    ....  
    ...getter and setter methods...
}


Filtering mechanism

 public static String filterJsonFields(IModel model,  
  String jsonFilter, String fieldsQueryParam) throws IOException { 
          
    String jsonResult = null ;  
    SimpleBeanPropertyFilter filter = SimpleBeanPropertyFilter.  
                     serializeAllExcept(Collections.<String> emptySet());  
    Set<String> filterProps = new HashSet<String>();      
    if (fieldsQueryParam != null) {  
        StringTokenizer st = new StringTokenizer(fieldsQueryParam, "," );  
        while (st.hasMoreTokens()) {  
            filterProps.add(st.nextToken());  
        }  
        filter = SimpleBeanPropertyFilter.filterOutAllExcept(filterProps);  
    }         
    ObjectMapper mapper = new ObjectMapper();  
    FilterProvider fProvider = new SimpleFilterProvider().addFilter(jsonFilter, filter)  
                        // setFailOnUnknownId: Ignore filtering the reference member fields  
                        // Refer: https://jira.codehaus.org/browse/JACKSON-650  
                        .setFailOnUnknownId( false );  
    try {  
        jsonResult = mapper.writer(fProvider).writeValueAsString(model);  
    } catch (IOException e) {  
        throw e;  
    }  
    return jsonResult;  
}   
   

The "jsonFilter" method parameter is the class name of the model bean that is passed from the caller method (getProduct()).

Caveat: But this didn't work well with JBoss's RestEasy.


2. Overriding the JacksonAnnotationIntrospector

In this method, we need to extend the JacksonAnnotationIntrospector and override the method "findFilterId". During serialization, Jackson calls the JacksonAnnotationIntrospector.findFilterId() to identify the filter of the model bean. In our case, we are using the name of the model bean class as its filter name. 

import org.codehaus.jackson.map.introspect.AnnotatedClass;  
import org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector;  
   
public class MyFilteringIntrospector extends JacksonAnnotationIntrospector {  
       
    @Override  
    public Object findFilterId(AnnotatedClass ac) {  
        Object id = super .findFilterId(ac); 
        if (id == null ) {  
            // the name of the class will be used as its filter name  
            id = ac.getName();  
        }  
        return id;  
    }  
} 

We may need to tell Jackson to use our custom AnnotationIntrospector and not JacksonAnnotationIntrospector. For this, 

AnnotationIntrospector myFilterIntrospector = new MyFilteringIntrospector();
mapper.setAnnotationIntrospector(myFilterIntrospector );

Hence the filterJsonFields() method is updated with the above lines of code.


Filtering mechanism

 public static String filterJsonFields(IModel model,  
           String jsonFilter, String fieldsQueryParam) throws IOException {  
   
    String jsonResult = null ;  
    SimpleBeanPropertyFilter filter = SimpleBeanPropertyFilter.  
                    serializeAllExcept(Collections.<String> emptySet());  
    Set<String> filterProps = new HashSet<String>();      
    if (fieldsQueryParam != null) {  
        StringTokenizer st = new StringTokenizer(fieldsQueryParam, ",");  
        while (st.hasMoreTokens()) {  
            filterProps.add(st.nextToken());  
        }  
        filter = SimpleBeanPropertyFilter.filterOutAllExcept(filterProps);  
    }       
    ObjectMapper mapper = new ObjectMapper();  
    AnnotationIntrospector myFilterIntrospector = new MyFilteringIntrospector();  
    mapper.setAnnotationIntrospector(myFilterIntrospector);  
    FilterProvider fProvider = new SimpleFilterProvider().addFilter(jsonFilter, filter)  
                  // setFailOnUnknownId: Ignore filtering the reference member fields  
                  // Refer: https://jira.codehaus.org/browse/JACKSON-650  
                  .setFailOnUnknownId( false );  
    try {  
        jsonResult = mapper.writer(fProvider).writeValueAsString(model);  
    } catch (IOException e) {  
        throw e;  
    }  
    return jsonResult;  
}   

Friday, May 23, 2014

How to enable RestEasy in JBoss AS 7.1 to use Jackson processor

Problem 

The environment used is JBoss AS 7.1 which owns RestEasy (a framework to build RESTful webservices). JBoss by default uses RestEasy framework. I added the Jackson annotation @JsonSerialize (include=JsonSerialize.Inclusion.NON_NULL) on top of my REST model classes to filter the NULL fields from the JSON response during serialization. But, RestEasy ignored this Jackson annotation.

Solution

With reference to some forums in internet, I came to know the following. To enable Jackson processor for RestEasy, the below maven dependency and configuration need to be done. If not, RestEasy may use Jettison processor.

1. Maven Jackson dependency
<groupId>org.jboss.resteasy</groupId>  
      <artifactId>resteasy-jaxrs</artifactId>  
      <version>${version.jboss.resteasy}</version>  
      <scope>provided</scope>  
 </dependency>  
   
 <dependency>  
      <groupId>org.jboss.resteasy</groupId>  
      <artifactId>resteasy-jackson-provider</artifactId>  
      <version>${version.jboss.resteasy}</version>  
      <scope>provided</scope>  
 </dependency>  
2. Create a file under named "jboss-deployment-structure.xml" under the WEB-INF folder and include the jackson modules in the dependencies.
<? xml version= "1.0" encoding = "UTF-8"?>  
< jboss-deployment-structure>  
   <deployment >  
     < dependencies>  
       <!-- added the below jackson modules to enable JBoss RestEasy to  
       use Jackson -->  
       < module name= "org.codehaus.jackson.jackson-core-asl" />  
       < module name= "org.codehaus.jackson.jackson-mapper-asl" />  
     </ dependencies>  
   </deployment >  
 </ jboss-deployment-structure>