Identifying the skills gap for a Software Developer

This April I had to create a Individual Development Plan (IDP) for me as part of the regular official procedures. One of step was to identify the gaps in you compared to the ideal position you want to reach. Thinking more in this line I have created the below table which contains ways to identify the specific areas of development for a developer. 

Guide to reading the table:-

Ask yourself the questions in the column (D). If your answer is "Yes" to any of the questions then you needs to consider the action plans listed in column (E).

A. Sl No	B. Section	C	D. These things happens with You	E. Your Action Plans
1	Understanding what to do	What to do? (40%)	1. You have missed some of the requirements. 2. You hear others say "This feature was not supposed to work like this" 3. Your completed work gets re-opened during QA or User Testing.	-Improve your domain knowledge. -Ask more questions to your PO so that you can impove your understanding of the requirements. -Push for improved requirements documentation. -Spend more time in testing your features. -Listen to sprint demos to get the overview of all the new features added.
2	Knowledge of Frameworks, Design patterns, practices and principles	How to do it - Your skills to do it (20 - 30 %)	1. You don't know where to start with when you have to implement a new feature 2. You don't know if a similar functionality already exists in the application or not 3. You don't completely understand the frameworks in the application and how they are used	-Pair program with an experienced developer to learn how he approaches a problem. -Learn more about the frameworks used in your app. -Try creating sample applications using them. -Identify the patterns and principles used in your app and try to use them.
3	Problem solving, Analytical, Debugging skills	How to do it - Your Ability to do it (10 - 20 %)	1. You face difficulties when it comes to writing algorithms 2. You are weak in debugging and finding issues in the code	-See if you can apply some known patterns to solve the problem.
4	Communicating with your code	How well you did it? How easily somebody can understand how ? (15%)	1. Your code is not up to the standards or frequently ignores code quality. 2. You don't have enough code coverage 3. You can't write a quality documentation	-Use tools like sonar to asses the quality of your code. -Spend more time in refactoring and improving the code quality.
5	Communicating about your work	How well can you communicate about your work (5%)	1. Your don't follow the process in the team. 2. Your check-in comments are not useful. 3. Your team don't know what you are working on.	-Understand and adhere to the team policies. If you feel that there is somethings wrong, communicate and get it clarified.

This is the first draft of the version I have created. Try to apply this to you or your team and let me know your feedback. I hope I can expand each area by writing more in the future.

EXIN Cloud Computing Foundation Exam Review

Recently I have attended a workshop on EXIN Cloud Computing Foundation course and cleared the certification exam. This post is bit about topics covered in the exam and my experience learning those topics.








The principles of Cloud Computing. This chapter deals with definitions, types of clouds (Public, Private and Hybrid) and cloud services (IAAS, PAAS, SAAS).
Most contents in the section are from the The NIST Definition of Cloud Computing paper. Other topics include The Evolution Toward Cloud Computing, Cloud Computing Architectures and Benefits and Limitations of Cloud Computing. The part about Virtualization and its role in the raise of Could computing was quite interesting for me.

Using the Cloud. This part is about accessing the cloud and mobility in the cloud.
This module covers the topics Overview of Accessing the Cloud, How Cloud Computing Can Support Business Processes and Service Providers Using the Cloud.

Security and Compliance. Is about the risks of cloud computing and the measures you can take
This module covers the paper Top Threats to Cloud Computing prepared by the Cloud Security Alliance under the Security Risks and Mitigating Measures title. Managing Identity and Privacy section deals with Triple-A authentication and various aspects of identity management.

Implementing and managing Cloud Computing. You learn about local cloud networks and how to support the use of cloud computing
This module includes the topics Building a Local Cloud Environment and Managing Cloud Services. There is a lot of focus on managing cloud services and related governance frameworks.

Evaluation of Cloud Computing. Examples of the subjects here are cost aspects, (dis)advantages and SLA’s.
This module speaks about the business case for cloud computing. For example cost implications to an organization evaluating the cloud services in terms of capex and opex. Forming the Service level requirements and agreements.

The text in italics are taken from the official exam page.
Written with StackEdit.

Integration Testing for Spring Applications with JNDI Connection Pools

We all know we need to use connection pools where ever we connect to a database. All of the modern drivers using JDBC type 4 supports it. In this post we will have look at an overview of connection pooling in spring applications and how to deal with same context in a non JEE enviorements (like tests). 

Most examples of connecting to database in spring is done using DriverManagerDataSource. If you don't read the documentation properly then you are going to miss a very important point.

NOTE: This class is not an actual connection pool; it does not actually pool Connections. It just serves as simple replacement for a full-blown connection pool, implementing the same standard interface, but creating new Connections on every call.

Useful for test or standalone environments outside of a J2EE container, either as a DataSource bean in a corresponding ApplicationContext or in conjunction with a simple JNDI environment. Pool-assuming Connection.close() calls will simply close the Connection, so any DataSource-aware persistence code should work.

Yes, by default the spring applications does not use pooled connections. There are two ways to implement the connection pooling. Depending on who is managing the pool. If you are running in a JEE environment, then it is prefered use the container for it. In a non-JEE setup there are libraries which will help the application to manage the connection pools. Lets discuss them in bit detail below.

1. Server (Container) managed connection pool (Using JNDI)

When the application connects to the database server, establishing the physical actual connection takes much more than the execution of the scripts. Connection pooling is a technique that was pioneered by database vendors to allow multiple clients to share a cached set of connection objects that provide access to a database resource. The JavaWorld article gives a good overview about this.

In a J2EE container, it is recommended to use a JNDI DataSource provided by the container. Such a DataSource can be exposed as a DataSource bean in a Spring ApplicationContext via JndiObjectFactoryBean, for seamless switching to and from a local DataSource bean like this class.

The below articles helped me in setting up the data source in JBoss AS.

1. DebaJava Post

2. JBoss Installation Guide

3. JBoss Wiki 

Next step is to use these connections created by the server from the application. As mentioned in the documentation you can use the JndiObjectFactoryBean for this. It is as simple as below

If you want to write any tests using springs "SpringJUnit4ClassRunner" it can't load the context becuase the JNDI resource will not be available.

For tests, you can then either set up a mock JNDI environment through Spring's SimpleNamingContextBuilder, or switch the bean definition to a local DataSource (which is simpler and thus recommended). 

As I was looking for a good solutions to this problem (I did not want a separate context for tests) this SO answer helped me. It sort of uses the various tips given in the Javadoc to good effect. The issue with the above solution is the repetition of code to create the JNDI connections. I have solved it using a customized runner SpringWithJNDIRunner. This class adds the JNDI capabilities to the SpringJUnit4ClassRunner. It reads the data source from "test-datasource.xml" file in the class path and binds it to the JNDI resource with name "java:/my-ds". After the execution of this code the JNDI resource is available for the spring container to consume.

To use this runner you just need to use the annotation @RunWith(SpringWithJNDIRunner.class) in your test. This class extends SpringJUnit4ClassRunner beacuse a there can only be one class in the @RunWith annotation. The JNDI is created only once is a test cycle. This class provides a clean solution to the problem.

2. Application managed connection pool

If you need a "real" connection pool outside of a J2EE container, consider Apache's Jakarta Commons DBCP or C3P0. Commons DBCP's BasicDataSource and C3P0's ComboPooledDataSource are full connection pool beans, supporting the same basic properties as this class plus specific settings (such as minimal/maximal pool size etc).

Below user guides can help you configure this.

1. Spring Docs

2. C3P0 Userguide

3. DBCP Userguide

The below articles speaks about the general guidelines and best practices in configuring the connection pools.

1. SO question on Spring JDBC Connection pools

2. Connection pool max size in MS SQL Server 2008
3. How to decide the max number of connections
4. Monitoring the number of active connections in SQL Server 2008

Note:- All the text in italics are copied from the spring documentation of the DriverManagerDataSource.

TalkNotes - The story of SonarQube told to a DevOps Engineer

This week I spoke at Bangalore DevOps meetup on the topic "The story of SonarQube told to a DevOps Engineer". I have started writing TalkNotes inspired from Martin Fowler.Unlike his detailed article my posts aims to help the audience better understand my slides. SonarQube is a open source code quality management platform. It was a 30 mins talk focused at the need, setup, CI Infrastructure and administration of the SonarQube to the DevOps community.

The story of SonarQube told to a DevOps Engineer from Manu Pk

I have started the talk with one of my favorite subject Technical Debt. We have also looked at some of the parameters which determines the quality are coding standards breach, duplication, lack of unit tests, bad distribution of complexity, Spaghetti Design etc... I have spoke about this in more detail at previous post. There are various existing tools that helps reduce the technical debt by improve the code quality. What was missing was a easier way of tracking these code rule violations. For examples I need to know how much debt was introduced or was cleaned up? As a developer how do you quantify improvement the which a particular code refactoring has brought to the team etc..

This is where Sonar come to your help. Sonar's rich feature set allows you to do these and more. Currently it can run the quality analysis on more than 20 languages including Java, C#, C/C++, PL/SQL, Javascript, PHP, Web, XML etc.... It stores the analysis results and the data is displayed through various dashboards. Further slides discusses the sonar platform overview and installation.

The below diagram shows the CI environment including SonarQube. The Hudson plugin for SonarQube can be configured by following the wiki.

Image Idea from this blog.

The best part of sonar is its documentation. This was the most comprehensive documentation I have read about any open source product. You just need their wiki page to get 99% of answers. Now that I have it configured hope to write more about it in the coming moths.

4 simple steps to migrate legacy projects from Ant to Maven

For some time we were thinking about migrating our build to maven from ant. It happened last month and was actually simpler than what we have anticipated. From my experience, here is a brief about the steps we have followed. Our application is a  is a enterprise web application build with multiple frameworks and technologies and is deployed as a single WAR.

1. Create maven project directory structure.

As told in the Maven user guide create the below directory structure. We have done it under a new folder for the project.

2. Move the files/folders keeping the SCM logs. 

Even though the folder structure is new the source files will be old ones! We want to keep the SCM logs while moving them to new locations. Remember to commit the folders created in step 1 before you start moving your files. If you use SVN, see this user guide or SO question on how to do this. Move the java source, unit/integration test and configuration resources to appropriate folders.

3. Create the POM and add dependencies

Most critical part in the migration is adding the dependencies in the POM. Start by adding the dependencies for the frameworks used in your application. Make sure you are adding the right version of the jars. You can find the version of the jar by reading the MANIFEST.MF file inside the META-INF folder of the jar. This will help if the versions are missing from the file name.

Any third party jars can be added to the maven repository as told here. If you are using very old versions of jar files some of them may not be available in maven repository.Here you can either try upgrading to newer versions or prepare local install as told before. Once you have added all the dependencies try building the application. Watch out for any major issues.

4. Make sure you haven't changed much in the WAR

Maven is a build tool. This means your WAR should not change. So, in the last step we compare both versions and make sure they are the same. Make sure you are on top of all the differences. Also, compare the jar files generated by maven and your existing files, Sync them by 

     - Adding <exclusions> to remove the unwanted jars

     - Add the dependencies for the missing jars

This can be a tiring tasks depending on the number of jars you have in your lib. But make sure that you have each one covered and know that why they exists in your app.

May  be this is a late post, most applications might have already been migrated by now. Anyways, better late than never! According to many experts Gradle is also a good choice as a build tool for your new project.

Why is tomcat a Webserver and not an Application Server

Many application developers do not focus much on the infrastructure on which their code runs. When it comes to web applications there are common confusions like what is the difference between webserver and applications server or when to go for a EAR vs WAR file deployment etc...

There are many good answers that differentiate between web servers and applications servers like this one. Most of the times the terms Web Server and Application server are used interchangeably. This article explains the working of a typical web server. Typically we get confused with the example of Tomcat Server (an example for a web server) having the capability to run the enterprise applications. So, tomcat is a web server or an application server? Let me tell you how I convinced my self regarding this.

Some time back I was struck with the question What's the difference between JPA and Hibernate on stack overflow. I did answer it, but one of the comment lead me to a more detailed understanding of the JavaEE spec and certified servers. If you can understand this then differentiating between the web server and application server is easy. During my investigations I got this article, which discusses the advantages of both.

A more detailed look in to the meaning JavaEE specification will throw some light in to our discussions. As we know specifications are set of rules. Simply put they contain the interface. Any JavaEE servers which needs to comply to spec needs to have the implementation of these interfaces. You can find the certified JavaEE servers list here. If you are deploying your enterprise applications (means you have JPA, EJB or some technology which is part of Java EE) to the a server which comply to JavaEE then the lib need not contain the API implementation jars. But these are needed if you are using a web server like tomcat for deployment.

For example, if you use JPA in your applications and deploying it to the  Jboss AS 7, then you need any additional jars in the lib. But the same application you want to deploy to the tomcat server then you need to have additional jars to lib that implements the JPA spec may be eclipselink or Hibernate. This is what makes JBoss AS 7 an application server and tomcat a web server. Another key difference is that we can not deploy an EAR file to tomcat, it could only handle WAR files.