Venturing into the Computing Cloud

Starting this summer, researchers who need super-computing power will have more options for crunching big data sets, including access to a private Duke "cloud."

"We wanted to be in a position where our researchers have a menu of options, and they can choose what's most suitable for how they do their computing," said Jim Siedow, Duke's vice provost for research. "One of those options should be the cloud."

Computing "in the cloud" allows users to access hardware and infrastructure on demand over the Internet, paying only for what they use.

"We have a large pool of hardware, and with that, we can provide options similar to those available through public cloud services like Amazon, but at significantly reduced rates," said Julian Lombardi, assistant vice president for the Office of Information Technology (OIT). "That's an option researchers didn't have before."

In the past year, more than 700 researchers from a range of disciplines have used the Duke Shared Cluster Resource, a high-performance network of about 670 Linux machines maintained by OIT. Usage has been especially high in disciplines such as computational biology and genomics.

With the cluster, researchers pool existing resources and share access to greater computing power than they could afford on their own, without worrying about the "care and feeding" of the equipment, since Duke provides the physical space, systems administration, programming assistance, and power and cooling for the machines.

Researchers will still be able to purchase machines to contribute to the cluster - what Lombardi described as the "condo" model, where those machines are added to the cluster and excess cycles shared with other users.

But the new model, approved by the provost this spring and administered by Duke's Scalable Computing Support Center (SCSC), will include two new options:

• A private Duke "cloud," where researchers can purchase high-priority cycles as needed from a pool of machines and pay per CPU hour (similar to the pricing model for Amazon's cloud).

• An economy cloud option - or what Lombardi jokingly calls a "fog" - where researchers can purchase low-priority cycles only, for a lower per-CPU rate.

While the number of machines in the cluster has stayed steady over the past few years, old machines have been replaced with newer ones that provide exponentially more computational power, said John Pormann, director of the SCSC. The cluster now has more than 4,300 CPU-cores.

The SCSC also will expand its staff to include a second high-performance computing analyst and a visualization consultant to help researchers in a widening range of disciplines explore how high-performance computing can be used in addressing new research questions.

"Strategically, this is a sustainable model. We can keep an eye on how our needs evolve as Duke grows and more fields become computational, and we can base our growth on what our researchers actually need," said Alvin Lebeck, a computer science professor and chair of the Research Computing Advisory Committee.

The new services will be available July 1.