Clouds viable for large scale scientific work?

Clouds viable for large scale scientific work?

Clouds viable for large scale scientific work?

Here at ISC Cloud 2011 most of the focus until the very end of the conference has centered on how to leverage clouds for HPC applications and needs. I’m impressed with the abilities (as I always am) of the research and academic world to leverage any new technologies which might be available. I was looking to see more of the HPC-backing technologies brought to bear on the cloud infrastructures, but sadly that wasn’t the focus here.

Not to despair though, here at the end are some analyses of the viability of existing (mostly public and hybrid) clouds for certain workloads. As you see from the slide, there are big gaps in what the clouds can actually accomplish as they continue to aim at the enterprise datacenter application set. Since I’m involved in very large problems with scaling and data manipulation, I echo this opinion but have some proposals of how to help.

So in a nutshell: Are clouds viable for large scale scientific work?

Not yet…

But stay tuned to the blog … change is coming …

  • Facebook
  • Twitter
  • Delicious
  • Digg
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS

SGI Evolution

As we enter the many core era, I was asked to attend a meeting with SGIand Intel to discuss the outlook for the future and provide input on the upcoming changes in the marketplace and how technical computing systems will bring to bear on the unique challenges in our industries.

I like the focus of changing HPC to “Technical Computing” and that the needs of the cloud and virtualization are beginning to drive the companies moving forward. I’ve been working for some time to drag most vendors in the HPC space to understand just what and where those challenges really are. Certainly the focus and push from the industry at SC10 echoed these needs and I think both SGI and intel have heard the message.

“Big Data” is seen by the executive staff at SGI as an intersection between business applications and technical computing applications. The idea is to bring together structured data from the traditional database and data warehousing markets to the unstructured data sets common in hadoop and similar applications. This is a strong play, and one that I think has been a long time coming, but one which requires an enormous integration understanding. This is unfortunately, where things break down. While I agree that the business application and technical computing sides both generate enormous amounts of data, and that one of the keys to keeping technology companies moving forward is the means by which they can capitalize on these data mines, I do not see a great deal of comprehension on how to integrate the two. Closed platform computing remains just that: closed. And open platforms, while remaining open, make little inroads towards their closed colleagues. Instead they spend a lot of time reinventing closed wheels.

Systems designers need to look beyond what their vendors are suggesting and take a second order approach to the overall goals of the organization. I am not suggesting a near religious stance on either open or closed, or even a strong commitment to best-of-breed point solutions. Rather, I think that work needs to factor in cost, availability, scalability, longevity, and human resources in a way that creates long term reusable solutions for companies. I had a long conversation with SGI after the meetings, and I think that there is some desire to move in this direction for their customers, but I remain unconvinced that they will get far beyond their own systemic walls. If some of their products like ICE and UV are any indication, the folks doing the design, documentation, and software for these platforms have a very set mind on the usage of those systems. Woe to the company not in their model.

Intel is more pragmatic, which makes sense based on their position in the market. They are willing to embrace everything from radical change to staunch tradition within an IT department. I don’t detect the depth of understanding for any particular set or stance in the technology world from them however, making them a necessary vendor, but almost ancillary to any decisioning by architects in these markets.

  • Facebook
  • Twitter
  • Delicious
  • Digg
  • StumbleUpon
  • Add to favorites
  • Email
  • RSS