Karmasphere, a bigdata intelligence software company for Hadoop released a survey on Hadoop based development and usage in the industry conducted in the week of 30 Aug’2010.
Here are the excerpts and key points from the survey
Top 3 reasons of using Hadoop
- Mine data for more improved business intelligence
- Reducing the data analysis cost
- Log analysis
Key Challenges of using and deploying Hadoop
- Steep Learning Curve
- Hiring qualified people
- Products & tools availability
- Getting started and access to suitable Educational resources
Technology & Software development aspects
- Java is the primary language
- Growth expected in the use of Hive/SQL
- Constant usage of Pig to remain
- Debugging & monitoring Hadoop jobs is a challenge
Anticipated Growth of Hadoop
- Growth in the number of firms using Hadoop is likely to drive the growth of developers from 50 to 60% in a year.
In my viewpoint modern hadoop software development tools and support from companies such as Cloudera and Karmasphere commercialising hadoop in a streamlines manner is likely to drive the usage of growth.
New Opportunities beckon developers and IT service providers
Given the unprecedented explosion of data sweeping across commercial and new generation internet establishments, deriving intelligence from ‘big data’ for gaining competitive intelligence and advantages will become more and more important for businesses and this emerging trend in my opinion may drive the usage of hadoop significantly in the coming years given the fact it is the only ‘big data’ processor that has got enough traction in the industry.
With such increasing demand to apply Hadoop for critical parts of IT enabled businesses and with less qualified and experienced human resources in the market combined with some of the other related challenges cited above, the premium for developers and IT services companies providing Hadoop consulting ,development and implementation is likely to be high in the coming years and hence may offer significant job and business opportunities in the market. In my opinion, this picture presents significantly new opportunities.
If you combine the above with your own market intelligence or experiences with specific industry areas where bigdata exists , for instance in the Pharma & Life sciences , Financial services and eBusiness Industry verticals and if you intimately know cloud computing and services like Amazon AWS , You will have an edge over others on leveraging Hadoop opportunities better!