Showing posts with label java. Show all posts
Showing posts with label java. Show all posts

Tuesday, September 23, 2014

Authentication using Apache Shiro

Introduction

Authentication is the process of identity verification. A user should provide some information as an identity that the system understands so as to allow the user to login the system. This document covers one of Shiro's authentication mechanism that can be used for a java RESTful application.

Shiro

Apache Shiro is a java security framework that offers solution for authentication, authorization and session management for an application. Since  RESTful application are stateless, we will not deal with sessions. For every request URL, the user is authenticated. Four basic terms that Shiro uses are: 
(1) Subject: refers to the user who is using the system. It can be human-being, another application, cron job etc.
(2) Principals: refers to attributes that identify the subject, like username first name, last name, ssn
(3) Credentials: refers to secret data like password, certificates etc.
(4) Realms: this is Shiro's security specific DAO which talks to the application's back end datasource or another DAO.

Shiro configuration file (shiro.ini)

During the application deployment, Shiro loads its configuration file (shiro.ini). They call it as the initialization file where the Realm, Credential matcher, authentication filter and few more settings that are to be used in the application are configured.

shiro.ini
 [main]  
 authc.loginUrl = /web/login.jsp  
 customRealm = com.app.security.shiro.realm.CustomRealm  
 credentialsMatcher = com.app.security.shiro.checker.CustomCredentialsMatcher  
 credentialsMatcher.hashAlgorithmName = SHA-512  
 credentialsMatcher.hashIterations = 1024  
 # base64 encoding, not hex since it requires less space than hex  
 credentialsMatcher.storedCredentialsHexEncoded = false  
 customRealm.credentialsMatcher = $credentialsMatcher  
 # customRealm.dataSourceName should be the same that is used in "persistence.xml"  
 customRealm.dataSourceName = java:jboss/datasources/appDS  
 securityManager.realms = $customRealm  
 [urls]  
 # authc means the default FormAuthenticationFilter used for FORM level login  
 /web/login.jsp = authc  
 # REST URL authentication. authcBasic means the default BasicHttpAuthenticationFilter  
 # "rest": HttpMethodPermissionFilter, to be used in future for authorization  
 /rest/** = authcBasic, rest  

Shiro Filter

Shiro provides a request filter called ShiroFilter which needs to be added in our web.xml such that all the request to the application pass through this filter before being processed by the server. The Shiro filter internally forwards the request to the Authentication filter that we have configured in shiro.ini file. For our application, we used 'BasicHttpAuthenticationFilter' (which is marked as 'authcBasic' in shiro.ini). This filter internally uses the Realm DAO and the CredentialsMatcher to validate the credentials for authentication. If the user is successfully validated, then the request is forwarded to the application's REST layer to process further. 

web.xml
 <?xml version="1.0" encoding="UTF-8"?>  
 <web-app xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">  
   <listener>  
     <listener-class>org.apache.shiro.web.env.EnvironmentLoaderListener</listener-class>  
   </listener>  
   <filter>  
     <filter-name>ShiroFilter</filter-name>  
     <filter-class>org.apache.shiro.web.servlet.ShiroFilter</filter-class>  
   </filter>  
   <filter-mapping>  
     <filter-name>ShiroFilter</filter-name>  
     <url-pattern>/*</url-pattern>  
   </filter-mapping>  
   <session-config>  
    <session-timeout>30</session-timeout>  
   </session-config>  
 </web-app>  

Realm and CredentialMatcher

The principal (username) and credential (password) are retrieved from the "Authorization" URL header attribute. They are base64 encoded and are given in the following format, "Basic YWRtaW46YWRtaW4=" where 'YWRtaW46YWRtaW4=' is the base 64 encode of username:password.
The ShiroFilter internally constructs the AuthenticationToken using the username & password. The BasicAuthenticationFilter internally validates this token with the user authentication information in the application database. This is done using CustomRealm and CustomCredentialsMatcher.
CustomRealm is a custom Realm which has its own implementation to fetch the authentication information of the respective user from the application database. We use just plain JDBC to fetch the user record matching the username in the Authentication token. Plain JDBC has been used due to a Shiro limitation which is given below. Finally the AuthenticationToken and the user info fetched from database are passed to the CustomCredentialsMatcher for validation.
Shiro limitation: Apache Shiro is unaware of CDI injection in Realm. The injected UserDAO was not instantiated during CustomRealm object instantiation. There are a couple of tickets in Shiro's queue for this. Refer: https://issues.apache.org/jira/browse/SHIRO-337, https://issues.apache.org/jira/browse/SHIRO-422. Hence as a workaround, currently CustomRealm extends JdbcRealm and we fetch the user object matching username through direct JDBC queries. In future if Shiro resolves the above tickets, we can remove the plain JDBC stuff and enable the injected UserDAO code. 

CustomRealm.java
 public class CustomRealm extends JdbcRealm { //JdbcRealm extends AuthorizingRealm and AuthenticatingRealm   
   private static final Logger log = Logger.getLogger(CustomRealm.class);  
   protected static final String DEFAULT_AUTHENTICATION_QUERY = "SELECT user_name, password, password_salt "  
       + "FROM user WHERE user_name = ?";   
   // @Inject protected UserDAO userDAO;  
   public CustomRealm() {  
     super();  
     setName("CustomRealm");  
     CustomCredentialsMatcher credentialMatcher = new CustomCredentialsMatcher();  
     setCredentialsMatcher(credentialMatcher);  
     log.info("CustomRealm instantiated!");      
   }  
   protected String dataSourceName; // java:jboss/datasources/appDS - configured in shiro.ini  
   public String getDataSourceName() {   
     return dataSourceName; // java:jboss/datasources/appDS  
   }   
   public void setDataSourceName(String dataSourceName) {   
     this.dataSourceName = dataSourceName;   
     this.dataSource = getDataSourceFromJNDI(dataSourceName); // setting datasource to JdbcRealm.datasource  
   }   
   private DataSource getDataSourceFromJNDI(String jndiDataSourceName) {   
     try {   
       InitialContext ic = new InitialContext();   
       return (DataSource) ic.lookup(jndiDataSourceName);   
     } catch (NamingException e) {   
       log.error("JNDI error while retrieving " + jndiDataSourceName, e);   
       throw new AuthorizationException(e);   
     }   
   }   
   @Override  
   protected AuthenticationInfo doGetAuthenticationInfo(AuthenticationToken token)  
       throws AuthenticationException {  
     log.info("Do Shiro doGetAuthenticationInfo...");  
     //identify account to log to  
     String username = String.valueOf(token.getPrincipal());  
     checkNotNull(username, "Null usernames are not allowed by this realm.");  
     log.info("---------------------------------------------------------------------");  
     log.info("Principal: " + String.valueOf(token.getPrincipal())  
         + " - Credentials: " + String.valueOf(token.getCredentials()));     
     log.info("---------------------------------------------------------------------");  
     String hashedPassword = null;  
     ByteSource credentialsSalt = null;  
     if("admin".equalsIgnoreCase(username)) {  
       hashedPassword = "admin";  
     } else {  
       IUser user = null;  
       // user = userDAO.findByUserName(username);    
       Connection conn = null;  
       try {  
         conn = dataSource.getConnection();  
         user = getUserByUserName(conn, username);  
       } catch (SQLException e) {  
         throw new AuthenticationException("Unable to fetch user by username ["   
             + username + "].", e);  
       } finally {  
         JdbcUtils.closeConnection(conn);  
       }  
       checkNotNull(user, "No account found for user [" + username + "]");  
       hashedPassword = user.getPassword();  
       credentialsSalt = new SimpleByteSource(Base64.decode(user.getPasswordSalt()));  
     }  
     // SimpleAuthenticationInfo implements SaltedAuthenticationInfo  
     return new SimpleAuthenticationInfo(token.getPrincipal(), hashedPassword, credentialsSalt, getName());  
   }  
   @Override  
   protected AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection arg0) {  
     // TODO Auto-generated method stub  
     return null;  
   }  
   private IUser getUserByUserName(Connection conn, String username)   
       throws SQLException {   
     PreparedStatement ps = null;   
     ResultSet rs = null;   
     IUser user = null;  
     try {   
       ps = conn.prepareStatement(DEFAULT_AUTHENTICATION_QUERY);   
       ps.setString(1, username);   
       rs = ps.executeQuery();   
       // Loop over results - although we are only expecting one result,   
       // since usernames should be unique   
       boolean foundResult = false;   
       while (rs.next()) {   
         // Check to ensure only one row is processed   
         if (foundResult) {   
           throw new AuthenticationException(   
               "More than one user row found for user ["   
                   + username + "]. Usernames must be unique."); // TODO work on this warning  
         }   
         user = new User();  
         user.setUserName(rs.getString("user_name"));  
         user.setPassword(rs.getString("password"));  
         user.setPasswordSalt(rs.getString("password_salt"));  
         foundResult = true;   
       }   
     } finally {   
       JdbcUtils.closeResultSet(rs);   
       JdbcUtils.closeStatement(ps);   
     }   
     return user;   
   }   
   private void checkNotNull(Object reference, String message) {  
     if (reference == null) {  
       throw new AuthenticationException(message);  
     }  
   }  
 }  

CustomCredentialsMatcher.java
 public class CustomCredentialsMatcher extends HashedCredentialsMatcher {  
      
   private static final Logger log = Logger.getLogger(CustomCredentialsMatcher.class);  
      private static final SHIRO_CREDENTIAL_HASH_INTERATION = 1024;  
      
   public CustomCredentialsMatcher() {  
     super();  
     log.info("CustomCredentialsMatcher instantiated!");  
     setHashAlgorithmName(Sha512Hash.ALGORITHM_NAME);  
     setHashIterations(SHIRO_CREDENTIAL_HASH_INTERATION);  
     setStoredCredentialsHexEncoded(false); // Base64-encode, not hex-encode  
   }  

   @Override  
   public boolean doCredentialsMatch(AuthenticationToken token, AuthenticationInfo info) {  
        
     log.info("Shiro doCredentialsMatch...");  
        
     log.info("AuthenticationToken.credentials: " + new String((char[]) token.getCredentials()));  
     log.info("AuthenticationToken.principals: " + token.getPrincipal());  
     log.info("AuthenticationInfo.credentials: " + info.getCredentials());  
     log.info("AuthenticationInfoprincipals: " + info.getPrincipals());  
     if("admin".equalsIgnoreCase(String.valueOf(token.getPrincipal()))) {  
       return true;  
     } else {  
       // verifies by comparing the credential in token and info  
       return super.doCredentialsMatch(token, info);  
     }   
   }  
 }  

Schema design: User (user)
  • user_name : username to login
  • password  : password to login
  • passwordSalt : a secure random number used as a salt while encrypting/decrypting the password.
It is insecure to store the password as plain string in the database. Hence we encrypt the password using Shiro's Sha512Hash algorithm, also a password salt is used for this to make it more secure. Note that the hashing algorithm (Sha512Hash) used for this password crypt is configured in shiro.ini.
UserService.java
 public class UserService {  
     @Inject UserDAO userDAO;  
       
     public void createUser(UserBean userBean) {  
         String id = // generate a uuid  
         userBean.setId(id);           
         if(userBean.getPassword != null) {  
             // We'll use a Random Number Generator to generate salts. This is much more secure  
             // than using a username as a salt or not having a salt at all.  
             RandomNumberGenerator rng = new SecureRandomNumberGenerator();  
             Object salt = rng.nextBytes();  
             //hash the plain-text password with the random salt and multiple  
             //iterations and then Base64-encode the value (requires less space than Hex)  
             String final SHIRO_CREDENTIAL_HASH_INTERATION = 1024;  
             String hashedPasswordBase64 = new Sha512Hash(this.password, salt,  
                     SHIRO_CREDENTIAL_HASH_INTERATION).toBase64();  
             userBean.setPassword(hashedPasswordBase64);  
             userBean.setPasswordSalt(salt.toString());              
         } else {  
             throw new ApplicationRuntimeException("password is null");  
         }  
         userDAO.createUser(userBean);  
     }  
 }      

Wednesday, September 4, 2013

Observer pattern

Java has two utility classes to serve Observer pattern - Observer and Observable. 
  • The aim of this pattern is to notify the objects of Observer class, when a change is done to the objects of Observable class.
  • Observer class should implement Observer interface and override the method 'update'.
  • Observable class should extend Observable class. The key method of this class is 'setChanged()'. This marks the Observable object being changed, only then the notifyObservers() will notify the observer objects that a change was done to the observable object. The notifyObservors() makes call to the Observor's update() method.
  • How do the observable object knows its observers? The observers should be added to the observable objects using the methods addObserver() of Observable class. An observer can be added or removed from the observable object.
 
1:  import java.util.ArrayList;  
2:  import java.util.List;  
3:  import java.util.Observable;  
4:  import java.util.Observer;  
5:    
6:  public class ObserverPattern {  
7:        
8:      public static void main(String []args) {          
9:          Stock s = Stock.getInstance();  
10:          s.setName("Madrone");  
11:          s.setNumShares(10);  
12:            
13:          // Observers  
14:          List<Buyer> buyerList = new ArrayList<Buyer>();  
15:          buyerList.add(new Buyer("A"));  
16:          buyerList.add(new Buyer("B"));  
17:            
18:          for(Buyer b : buyerList) {  
19:              s.addObserver(b);  
20:          }  
21:            
22:          for(int i=0; i<10; i++) {  
23:              s.setNumShares(s.getNumShares() + 1); // Observed object is modified   
24:              s.notifyObservers();  
25:          }      
26:            
27:          System.out.println("Shares available: " + s.getNumShares());  
28:          for(Buyer b : buyerList) {  
29:              System.out.println(b.getName() + " : " + b.getNumShares());  
30:          }  
31:      }  
32:  }  
33:    
34:  class Buyer implements Observer {  
35:        
36:      private String name;  
37:      private int numShares;  
38:        
39:      Buyer(String name) {  
40:          this.name = name;  
41:      }  
42:    
43:      @Override  
44:      public void update(Observable observable, Object ob) {          
45:          Stock s = (Stock) observable;  
46:          if(s.getNumShares() > 0) {  
47:              buyAShare();  
48:              s.setNumShares(s.getNumShares() - 1);  
49:          }      
50:      }  
51:        
52:      private void buyAShare() {  
53:          this.setNumShares(this.getNumShares() + 1);  
54:      }  
55:        
56:      public String getName() {  
57:          return name;  
58:      }  
59:      public void setName(String name) {  
60:          this.name = name;  
61:      }  
62:      public int getNumShares() {  
63:          return numShares;  
64:      }  
65:      public void setNumShares(int numShares) {  
66:          this.numShares = numShares;  
67:      }  
68:  }  
69:    
70:  // Singleton class  
71:  class Stock extends Observable {  
72:        
73:      private String name;  
74:      private int numShares;  
75:        
76:      public static final Stock INSTANCE = new Stock();  
77:      private Stock() {}  
78:            
79:      public static Stock getInstance() {  
80:          return INSTANCE;      
81:      }  
82:        
83:      public String getName() {  
84:          return name;  
85:      }  
86:      public void setName(String name) {  
87:          this.name = name;  
88:      }  
89:      public int getNumShares() {  
90:          return numShares;  
91:      }  
92:      public void setNumShares(int numShares) {  
93:          this.numShares = numShares;  
94:          setChanged();  
95:      }  
96:  }  

Thursday, July 4, 2013

Memory leak and performance analysis

Tools:

JVM Profiler - We have a plugin for it which could be installed in Eclipse.
MAT - Memory Analyzer Tool for analyzing the heap dump.



JVM monitor plugin for Eclipse


http://www.jvmmonitor.org/doc/index.html

http://help.eclipse.org/helios/index.jsp?topic=%2Forg.eclipse.tptp.platform.doc.user%2Fconcepts%2Fceproftl.xhtml

Note:
CW-Tomcat is running with java 32 bit and Eclipse also should be 32 bit.


Steps:
  1. Installed Eclipse helios 32 bit for windows
  2. Installed Liferay Plugin, EGit plugins for Eclipse
    1. Through Install New Software -  http://releases.liferay.com/tools/ide/eclipse/helios/stable/
    2. Through Install New Software -  http://download.eclipse.org/egit/updates-2.1
  3. Installed JVM Monitor plugin for Eclipse
    1. Through Eclipse Marketplace - Search JVM monitor and Click Install
    2. Through Preferences > Java > Monitor > Tools, Set JDK Root directory as C:\....\Softwares\jdk1.6.0_21
  4. Launch eclipse helios 32 bit from command prompt making Eclipse to use the same jdk that is set for JVM monitor, C:\...\Softwares\eclipse-jee-helios-SR2-win32\eclipse>eclipse -vm "C:\Mahesh\Softwares\jdk1.6.0_21\bin\javaw"

Monitoring JVM on remote host

To monitor JVM on remote host:

  1. Add the following configurations to JAVA_OPTS in ..../WAR/tomcat_folder/bin/setenv.sh
        -Dcom.sun.management.jmxremote.port=9876
        -Dcom.sun.management.jmxremote.ssl=false
        -Dcom.sun.management.jmxremote.authenticate=false
  2. Press the button Add JVM Connection on Remote Host on JVM Explorer to open New JVM Connection dialog. 
    1. Enter the ip address of the remote server
    2. Enter the port: 9876
Note: Sometimes we may not be able to connect to remote server even after setting the above three JVM arguments. In that case, add the below argument as well.        
              -Djava.rmi.server.hostname= mention the host-name here


How to capture heap dump of JVM running in Remote host

Since 'JVM monitor' does not support viewing the Memory details when monitoring remote host, we may need to capture the heap dump for analyzing the heap memory usage. Below are the steps to capture the heap dump (Ref: http://www.jvmmonitor.org/doc/, under section "How can I enable the BCI mode of CPU profiler on remote host?")
  1. In the Eclipse JVM monitoring view, select the server to monitor. In the Properties screen, you will find Timeline, Threads, Memory, CPU, MBeans and Overview. Select 'Memory' tab.
  2. Click on the 'heap dump' symbol in the top-right corner to capture the heap dump.
  3. A dialog box will be shown with the path where the dump file should be saved in the remote server (eg. /home/liferay\32323432323.hprof). And it may not allow you to select the 'Transfer ....' checkbox or to click on OK button to initiate the dump. An info could be displayed saying that ' agent is not loaded'.
  4. Now follow the instructions given in  http://www.jvmmonitor.org/doc/, under section "How can I enable the BCI mode of CPU profiler on remote host?"
  5. Copy the jvmmonitor-agent.jar from ~..\eclipse-jee-helios-SR2-win32\plugins\
    org.jvmmonitor.core_3.8.1.201302030008\lib\jvmmonitor-agent.jar
    and drop it
    into '/home/liferay' directory of Staging server (remote server) using WinSCP tool.
  6. Add the following configurations to JAVA_OPTS in  .../WAR/tomcat_folder/bin/setenv.sh
                     -javaagent:/home/liferay/jvmmonitor-agent.jar     

     7. Done. Now we are ready to dump the file. The heap dump .hprof file will be saved at '/home/liferay/'

How to use JVM Profiler:




Suggestions from different sites:

It is definitely advisable to have on the -XX:+HeapDumpOnOutOfMemoryError setting.   What is dumped is a binary .hprof file.   The setting for the file location shuold be:
-XX:HeapDumpPath=/niku/logs/heapdump.hprof


Setup for this in setenv.sh
-XX:HeapDumpOnOutOfMemoryError –XX:HeapDumpPath=d:\heapDumps




 How to open or analyze the heap dump .hprof files?

Some of the tools suggested in net are,
  1. MAT - Eclipse Memory Analyzer Tool - (Free) - http://www.eclipse.org/mat/downloads.php
  2. Visual VM - https://visualvm.dev.java.net/
  3. HAT -  Java Heap Analysis Tool - http://docs.oracle.com/javase/6/docs/technotes/tools/share/jhat.html
  4. YourKit Java Profiler
 I used MAT to analyze the heap dump (.hprof file).


MAT for Windows 32 bit was unable to open large dump files. (I tried with a .hprof file of  ~2.2 GB). Also the max heap size of MAT-32 Bit was by default 1GB, which cannot be tweaked. As per suggestion from an Eclipse forum - installed MAT for Windows 64 bit which did the job and allowed to tweak the max heap size. Have configured the vm argument -Xmx to 3GB in MemoryAnalyzer.ini file. Then the heap dump loaded in MAT without errors.