CAE on the cloud

By Evan Yares, Senior Editor

Will the availability of the high-performance cloud computing help drive use of CAE tools?

The biggest problem with Computer Aided Engineering (CAE) is that too few people use it. There are several good reasons, but one that’s significant is money. CAE software licenses, and the big computers on which to run it, can be expensive.

It may be that the answer to the money problem with CAE lies in the cloud. Or, to be precise, in cloud computing.


Image of NASA X-34 hypersonic re-entry simulation, showing a high angle of attack, with adaptive meshing for off-body shock capture. Created with CD-adapco STAR-CCM+ software by Matt Knapp of TLG Aerospace.

From a technical perspective, it’s not that much of a challenge for CAE vendors to support cloud computing. Most major CAE systems have supported high-performance computing (HPC) clusters for quite a long time. Clusters and clouds vary only in the details of their implementation. (Clusters use fast computers with fast interconnections. Clouds include software to control the provisioning of the systems.)

CAE vendors approach cloud support several different ways, mostly varying in licensing terms, and scalability. Following are some interesting examples.

Free CAE, cheap computing
One of the most compelling options for cloud CAE is the combination of OpenFOAM and Amazon EC2.

OpenFOAM is a well-known free open-source (GNU General Public License) system for the solution of continuum mechanics problems. It has an extensive range of features to solve anything from complex fluid flows involving chemical reactions, turbulence and heat transfer, to solid dynamics and electromagnetics. Included with OpenFOAM are more than 80 solver applications, and more than 170 utility applications for pre- and post-processing.


In simulations conducted at the Ohio Supercomputer Center, airflow under the Buckeye Bullet battery-powered land speed vehicle traveling at 300 mph without wind deflectors installed in front of the wheel wells (above) is compared with the airflow under the vehicle with wind deflectors installed (below). CFD simulation by OpenFOAM, with post-processing by Paraview.

One thing that distinguishes OpenFOAM from other open-source CAE applications is that it’s supported by both a non-profit organization (the OpenFOAM Foundation), and a commercial organization (ESI Group.) As a result, the software is not only updated regularly, it’s also well supported. ESI offers training courses for OpenFOAM around the world.

OpenFOAM is designed for parallelism from the ground up, providing good performance with both shared memory (as with multicore CPUs) and distributed memory (as with clusters and massively-parallel computers.) It scales well beyond at least 1000 cores.

Most parallel-optimized CFD systems, OpenFOAM included, provide the best performance when run on compute clusters with large amounts of RAM, and fast I/O performance. Typical public clouds (such as those from Amazon, Rackspace, and Google) use standard Gigabit Ethernet to connect their nodes. This might be plenty fast for most applications, but it can be a bottleneck when running CAE software.


In a simulation run at the Ohio Supercomputer Center, an isolated view of a wheel on the Buckeye Bullet battery-powered land speed vehicle illustrates airflow within the wheel well with the vehicle traveling at 300 mph. CFD simulation by OpenFOAM, with post-processing by Paraview.

A couple of years ago, Amazon introduced a special form of cloud instance in their Elastic Compute Cloud (EC2), designed specifically to support high performance computing (HPC.) The “Cluster Compute Eight Extra Large Instance” uses two Intel Xeon E5-2670 eight-core “Sandy Bridge” processors, with 60.5 GB of memory, and 10 Gigabit Ethernet I/O. You can use as many of these instances as you want (based on availability), at $2.40 per hour each.

How fast can Amazon EC2 run? As of last June, an Amazon EC2 cluster, using 17024 cores and 65968 GB of memory, was rated the 72nd fastest supercomputer in the world, achieving 354.1 TeraFLOPS performance.

Though, for most CAE runs, it might not make sense to provision 1064 computer instances—which is what it takes to get 17024 cores—the important point is that you could, if you needed to. (Though, to be fair, if you’re regularly running massively large simulations, you should probably consider the economics of buying your own cloud/cluster. A nice 1000 computer cluster, with all the whistles and bells, ought to set you back about $10 million or so.)

Pay per use CAE
For those who prefer a fully supported solution, there are various options, both in cloud services and software.

While mainstream cloud services companies, and Amazon in particular, have the capability of supporting CAE applications, it’s not an area where they have significant in-house expertise. For that, you need to look for companies that specialize in HPC cloud computing.

One example of such a company is Sabalcore, which provides hosting, and support for OpenFOAM, CD-Adapco’s STAR-CCM+, and ANSYS HPC (including Fluent and CFX.) Sabalcore’s HPC On-Demand services run on Linux clusters specifically designed for high-performance computing. The nodes are “bare metal” (with no overhead from virtualization), and are connected by InfiniBand (a switched fabric communications link commonly used in HPC.) Costs are between $0.20 to $0.29 per processor core-hour.

STAR-CCM+ is a good example of a product with a particularly cloud-friendly licensing scheme. The STAR-CCM+/Power-on-Demand product allows access by-the-hour, on an unlimited number of cores. Each license allows access to unlimited computing resources, either on in-house clusters or on cloud computing services, with an uncounted number of sessions. Power-on-Demand essentially gives users an unlimited simulation burst capacity.

CAE as a service
Traditional CAE vendors may support cloud computing, but, generally, on terms not too different from what they normally offer. Autodesk has decided to offer CAE, running on the cloud, as a service.

Autodesk has three lines of CAE products: mechanical (stress, vibration and motion), CFD (fluid flow and thermal), and MoldFlow (the very well-known plastic injection molding simulation tool.) All of these are available in Simulation 360. (The “360” is Autodesk’s branding for its cloud-based software as a service products.)

The Simulation 360 products are almost exactly the same as their desktop equivalents. They differ primarily in that their solvers run on the cloud, instead of on the local computer. With Simulation 360, when you press the “solve” button, the preprocessed mesh is sent, over the internet, to the Autodesk cloud, where it is processed. A unified job manager provides real-time feedback on the status of the simulation. When it is finished, the results are downloaded to the local workstation.


Gaming system showing air flow and thermal simulation results using Autodesk Simulation CFD.

Autodesk Simulation 360 is offered on an annual subscription basis, with several options:

• Mechanical and CFD, including up to 120 jobs, for $3600 per year
• Mechanical and CFD, including unlimited jobs, for $7200 per year
• Mechanical, CFD and MoldFlow, including up to 120 jobs, for $10,000 per year
• 10 job capacity pack, for $100

In its initial release, Autodesk is offering a standard level of solver performance. When a job is sent up to the cloud, it runs on the equivalent of a single high-performance 8-core computer. This isn’t because Autodesk’s solvers lack the capability of running on clusters. It’s just that this is the first release of Simulation 360, and Autodesk wants to be conservative. In the future, it’s very likely that Autodesk will offer the option of running the solvers on large clusters.


Desktop printer highlighting internal mechanical stress simulation results with Autodesk Simulation Mechanical.

One thing that distinguishes Autodesk Simulation 360 from other cloud-based CAE options is its particular suitability for design engineers (in addition to analysts.) Autodesk is, first and foremost, a CAD company. It’s natural that its CAE offerings would be tuned to the needs of CAD users.

How does the cloud change things?
CAE on the cloud is not a panacea. There are at least two legitimate arguments against it. The first is one of data security. Some companies have policies that will simply not allow their engineering data outside their firewall. Even though cloud computing providers generally have very good security policies and practices. The second is that data transfer to and from the cloud can be a problem with large simulations. Some large CFD analyses can generate massive amounts of data. Many gigabytes.

These issues are often surmountable—but it’s important to know they exist.

For most CAE programs, the benefit of running on the cloud comes down to scalable performance, without needing to buy and manage a compute cluster. It changes the economics of the compute environment, but not of the software. For Autodesk Simulation 360, the benefit is different: it provides easy accessibility of simulation tools, at a low subscription price, with no need to buy high-end computers to run it on.

Autodesk Simulation 360 is worth watching, in that it may help promote use of CAE tools among a new class of users. To the extent that it’s successful, it’s likely to inspire imitators.





>>Reprint info<<


  1. No mention of Penguin Computing. Penguin has had an HPC Cloud Service for the past 2 years and CAE in the cloud is our fastest growing sector. We are partners with LSTC, Ansys and CD Adapco and customers run LS Dyna, Fluent and StarCCM+ on our HPC Cloud. Full HPC support is included in our service. And our pricing can be less than what was mentioned above. We also just installed a Top 500 Cluster and a Fortune 50 manufacturer. – Catherine Pringle, Penguin

Speak Your Mind