Cloud Computing will Commoditise Supercomputing :: Opportunities Galore?!

Amidst searching  for a new job, I have been diving deep in to the emerging world of High performance (Super)Computing on the Cloud. It’s been fascinating world and the impression I have now is that Cloud computing will make super computing a commodity in the not too distant future.

Cloud computing is really disruptive and will bring revolution of sorts from this perspective to enable and cater to the growing super computing demands in the fields of Science,Engineering ,Biology and Information Technology. It is already happening infact.

Today,One can deploy a Cloud computing system that can process at Teraflops scale. Amazon web services cloud was ranked 231 in the Top 500 list of Elite for its ability to process at speeds of 41.82 teraflops.

The Economics of Super computing for this  very interesting and inviting. One can run a complex Fluid dynamics CAE simulation on a 100 node system for say $200 an hour or a 10,000 core system for $2000 an hour. Even in the recent past ,say 1-2 years ago, Companies especially small and mid size ones, couldn’t even imagine about accessing or leasing a 1000 core computing cluster to do Scientific or Engineering computations.

Today, there is  drastic change in this scenario.One of the things Cloud computing is big and disruptive is just because of the paradigm shift  it is bringing about in this realm.

What is driving this paradigm shift :: Super Computing on the Cloud 

* The Availability of High powered computers,storage and networks on the Cloud and several players evolving in the cloud Eco-system with competing technologies and services,

* The on-demand nature/model of the cloud  with lower costs

* The availability of numerous and proven complex scientific processing software on a open source model (e.g. BLAST,OpenFOAM etc)

* The Availability of proven and complex open-source  software meant for enabling supercomputing models. The existence  of proven reference models.

* Availability of Innovative ,Opensource Cloud management software (e.g. OpenNebula)

* Some of the Vendors of large scale Clustering software offering their service on the cloud

* Initiatives by companies in simplifying and democratizing supercomputing as a service (very few companies ,but this is a very interesting )

* Needless to mention the huge need/demand to research, engineer and develop complex products in the areas such as Fundamental  research in Science, Life sciences, Aerospace,Automobiles,IT and Telecommunications so on on…

The very interesting fact here to be noted  is the open, democratization initiatives by a very few ,innovative companies to “really offer an end user everything” that is required to do a meaningful supercomputing effortlessly. They offer this in an on demand model, be it to an individual or to a Fortune 500 organization.

For instance an engineer or a scientist or a small upstart company with the right domain knowledge and resources may be in a position to afford and use such a facility to research discover and develop revolutionary products and can compete with a large  MNC . Also, revolutionary changes could be brought about from the real possibilities of drastically slashing the time for ideation to implementation (i.e : Shortened Time to discover and implement) .This is  a very interesting and disruptive phenomenon of the cloud.

Imagine the paradigm shift this could bring about! For instance, a Pharma Scientist with no upfront infrastructure and investments may leverage the cloud based supercomputing to discover and develop the formula for making antibiotic drugs for a complex disease for say at $ 50,000 or less. Such a person may have the potential to productize and commercialize his Intellectual properly for several million dollors and save the lifes of millions of people. There is clearly a potential for  mass impact!!

This is a small glimpse of what Cloud computing in the future will bring about. One of the many scenarios that will drive the large scale adoption of cloud (super)computing . Hence my viewpoint that not too distant in the future, the possibility of Super computing to get commoditised.

In essence the global availability combined with growing power and capacities of Cloud computing services in the form of hardware , software and easy to use solutions available and accessible to anyone in the world on a open (democratic),low cost model has significant potential to commoditise super computing in the years to come. The biggest beneficiaries of this will be the Cloud service providers (Amazon,RackSpace,Go Grid ,Microsoft Azure etc) . (Good to invest and hold some stocks in such select companies on a  long term?!)

Potential and Opportunities :- The emerging “Cloud-Supercomputing” road to Commoditisation  presents very interesting commercial possibilities and opportunities to individuals and organisations. In the context of IT & Software development which is my profession, there are significant opportunities to leverage this at this point of time, when things are at incipient stages. Here are a few possibilities and Scenarios

Sample Scenario -1 (An Easy to use, Supercomputing Enablement on the Cloud)

* An automotive company wants to develop next generation car models faster than its comptetitors through a large scale Aero and Fluid dynamics simulation and trials

* It cannot afford to buy lease or rent “Boutique” super computing facilities or may have constraints in buying expensive commercial software.

* Its key challenges include inhouse IT resources and capabilities to enable this and ofcourse budgets

Opportunity :: An Upstart Software company or a large IT services company integrates a number of available opensource and/or commercial software to form a complete solution on the cloud at a supercomputing scale, that facilitates the Automotive firm to run their Simulations seamlessly at low costs without upfront investments,overheads and liabilities!

The opportunities are in providing System and Application level software frameworks and solutions for Automatic provisioning,deployment ,control and monitoring .

(The beauty here is you as the enabler and service provider also donot incur upfront costs and overheads)

Also, if you try to be open and democratic in your model of offering, you could better cater to the demands and reap the benefits of Super computing demands on the cloud .

Sample Scenario -2 ( Economic Optimisation of Super Computing the Cloud) A Bio-Sciences company wanting to develop a new drug intends to run BLAST (Basic Alignment and Search Tool) on the cloud ,say that may potentially demand several hundred or thousands of cores during a peak BLAST operation.

One of the challenges associated with running BLAST is, you dont know how long the query and search operation will run to determine certain bio patterns and out comes. Will it run for 2 days ? Will my cloud cost me a Bomb ? This means the end user doesnot have a clear picture on what is the computing capacity needed to run it or what is going to be the cost of running it , if all they want is to run it in a few hours or days in a predictable or deterministic fashion.

Opportunity :- (Software based HPC Optimisation on the Cloud)

* Develop a Software that does an intelligent auto-scaling for specialised super computing use-case scenarios and related software tools on the cloud

* Implement intelligent algorithms (existing or that you may develop) in your software that can estimate the runtimes and capacities in advance for specific simulation tools such as BLAST and accordingly implement the auto- scaling of computing elements on a cloud. Provide real time visibility though a dynamic software interface.

* Implement this software on a SAAS model on different cloud platforms/services. Opportunities for Scientists , Engineers , Academic Researchers and Small to medium size organisations

Concluding remarks

The notion of Supercomputing on the Cloud is already a reality. Reality to a certain degree from the real supercomputing forms of usage and performance benchmarks. Nonethless, Sooner or later ,one may be able to sign up and deploy Supercomputers on an unimaginable scale on the cloud .

This paradigm shift and the emerging landscape is going to be an exciting period for Cloud services providers,IT practitioners and End users to leverage this significant opportunity .

The progression of computing needs in this 21st Century combined with the open,democratic, globalized,low cost structure of the Clouds will fuel the “Commoditisation” of Super Computing. One day, I have a dream to  gift a super computer for the Science Lab  of my school located in a small town in South India and I know now this is going to be possible!!

Finally , This is just a post to share my thoughts on what is the envisaged big picture. The technical aspects of to how Cloud computing enables Super computing , what are its various ingredients,approach and  how to leverage them for commercially to build Software products,solutions or services or How to run a large scale CAE simulation is beyond the scope of discussion for this.

If you are interested in understanding it further in your commercial initiatives or leveraging this , you may contact me at nagarajansankar[at]gmail[dot]com

I am currently involved in researching and trying out cloud based super computing for specific identified commercial models and usecases. I will be happy to collaborate with like minded people or organisations with initiatives on this front.

Advertisements

About Cloudshoring

Cloudshoring is a popular blog on Cloud computing and Big data
This entry was posted in Cloud Computing, Uncategorized and tagged , , , , , , , , , , , . Bookmark the permalink.

3 Responses to Cloud Computing will Commoditise Supercomputing :: Opportunities Galore?!

  1. cloudnumbers.com – cloud super computing for everyone!
    Grate blog posts about possibilities and scenarios for supercomputing based on cloud computing. cloudnumbers.com (http://www.cloudnumbers.com) provides researchers and companies with the resources to perform high performance calculations in the cloud. We are a international high-tech startup and may invite you to register and test your application on a computer cluster in the cloud for free: http://my.cloudnumbers.com/register
    We are looking forward to get your feedback and consumer insights. Take the chance and have an impact to the development of a new cloud computing calculation platform.

  2. Adam Haworth says:

    Cloud computing is slowly becoming the norm, its just a matter of time until all we will need is a monitor. The only major draw back at the moment is the lack of fibre optic broadband, the speeds are too slow for the services available to be fully optimised.

  3. JohannQ says:

    I dont know if drug development is the best example for opportunities here. Sure, biotech / pharma companies will save some money here – but the biggest cost-factor and obstacle will remain the FDA, in any case, I’m afraid.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s