21

I am not aware of any published Mma benchmarks for the new R2-D2 2014 Mac Pro, so I thought it might be of interest to other Mac fans to see some Mathematica 9 benchmarks.

My machine arrived on Friday 24 Jan (fortuitously, on the exact 30th anniversary of the launch of the Mac). It is easily the coolest Mac ‘desktop’ I have ever owned, and I have owned most of them over the years, including the magnificently silent Cube. The Cube, however, was always underpowered for its time, whereas R2-D2 feels like a little nuclear hotpot out of … well .. Star Wars.

The comparison here is:

A. Mac Pro 2008 ............... 8 cores @ 2.8 GHz

B. Mac Pro R2-D2 2014: ... 6 cores @ 3.5Ghz

Although the new R2-D2 Mac Pro (B) has fewer cores than my old Mac Pro (A), … on the new R2-D2 Mac Pro, each core can run two threads. So, for example, if I evaluate:

 LaunchKernels[]

… 12 kernels start up on the new R2-D2 under Mma 9, compared to only 8 on my old Mac Pro.

There are thus 2 obvious factors to give a speed boost for the new machine (even if other things were equal):

  • The increase in the clock rate from 2.8 GHz to 3.5 GHz (i.e. 25%)

  • The effective increase in the number of kernels/processes from 8 to 12 (i.e. 50%), which would help those calculations that are ‘massively’ parallel.

Mathematica 9 Benchmark package

Running Mathematica 9’s in-built:

Needs["Benchmarking`"]

Benchmark[]

… test (from a fresh start-up) yields:

  • 2008 Mac Pro:

    BenchmarkResult" -> 0.648, "TotalTime" -> 21.36 seconds

  • 2014 R2-D2 Mac Pro:

    BenchmarkResult" -> 1.662, "TotalTime" -> 8.33 seconds

The improvement seems quite impressive ... and a little better than I was expecting.

However, I am not convinced that the Mathematica 9 Benchmark test is at all a realistic measure of actual performance one is likely to see. My impression is that the computations in the test suite are unnaturally well-disposed to massively parallel computations, which is unfortunately not what happens in real-life day to day calculations (well, not mine).

mathStatica benchmark tests in Mathematica 9

We have a standard set of benchmark tests we use for comparing performance under different numbers of kernels running mathStatica with Mathematica 9 (here computed on the old 2008 Mac Pro) as per:

Unlike the Mma Benchmark suite, the mathStatica test suite calculations have a far greater emphasis on symbolic algebra calculations … in particular integration … and typically problems that involve solving multiple numbers of integrals simultaneously (so, one sees benefits from parallel computation, but as in the real world, sometimes these benefits max out at 2, 4 or 8 integrals/processes).

Here are the results for the 10 mathStatica test examples:

Timings in seconds

The above table provides a better representation of the types of symbolic problems I tend to solve using Mathematica. The resulting performance improvement … given 6 years of hardware technology advancement … is, I think, on balance, frankly somewhat disappointing. For these calculations, the performance improvements mostly fall into the 15% to 30% faster range … which is really just the faster clock speed at work. [ For these types of problems, one obtains clear parallel advantages usually for up to about 6 or 8 processes, but the range of problems that benefit from additional processes gets quite restrictive. ]

In brief, the old 'version' of Moore’s Law that provided doubling of performance every 18 months appears to be sadly over.

Summary

The Mathematica 9 benchmark suites suggest the new R2-D2 2014 Mac Pro is more than twice as fast as the old 2008 Mac Pro, but these results appear to be unrealistic and rather wishful, unless you live in a massively parallel universe. The improvements for symbolic performance seem a little disappointing, given the 6 years of hardware technology advances that separate the 2 models. Nevertheless, I must say that I am delighted with the new machine. The real performance improvements are not so much on the CPU, but in the way applications snappily open up (super fast SSD replacing hard drive: about a dozen times faster for large file copies), and the fast GPUs (which, unfortunately, Mma does not seem to take much practical advantage of). It is also dramatically quieter than the old Mac Pro … but alas definitely not silent, contrary to the claims made on many review sites … the single fan is most definitely audible. Perhaps I have a quieter working environment than the usual, or am more sensitive to such things.

Now to find my light sabre. MTFBWY.


Notes:

Mac Pro 2014 -- 6 cores @ 3.5 GHz with 32gig RAM, dual D300 GPUs, 1TB SSD

{"MachineName" -> "r2d2", "System" -> "Mac OS X x86 (64-bit)", "BenchmarkName" -> "MathematicaMark9", "FullVersionNumber" -> "9.0.1", "Date" -> "January 29, 2014", "BenchmarkResult" -> 1.662, "TotalTime" -> 8.329, "Results" -> {{"Data Fitting", 0.359}, {"Digits of Pi", 0.57}, {"Discrete Fourier Transform", 0.621}, {"Eigenvalues of a Matrix", 0.481}, {"Elementary Functions", 0.336}, {"Gamma Function", 0.449}, {"Large Integer Multiplication", 0.436}, {"Matrix Arithmetic", 0.771}, {"Matrix Multiplication", 0.508}, {"Matrix Transpose", 0.666}, {"Numerical Integration", 0.657}, {"Polynomial Expansion", 0.097}, {"Random Number Sort", 1.19}, {"Singular Value Decomposition", 0.552}, {"Solving a Linear System", 0.636}}}

  • If you could upload your mma benchmark results they could be added here. – Sjoerd C. de Vries Jan 27 '14 at 23:02
  • @SjoerdC.deVries Hi Sjoerd ... I've added the benchmark output above ... should do the trick. Please do add them to the table! – wolfies Jan 28 '14 at 16:50
  • I recently purchased a maxed-out iMac and ran the same benchmark test: 8.33 seconds, identical to what you have above. Given it has less cores, this leads me to believe that Benchmark[] is not parallelized. So I don't really know that Benchmark[] is terribly useful for serious computer-to-computer comparison. – Guillochon Jan 29 '14 at 15:50
  • @Guillochon Your result with the maxed out iMac does not surprise me ... in fact, in the Macworld app tests, the maxed out iMac often outperforms even the 8 core r2-d2 Mac Pro in CPU tests. See: macworld.com/article/2082568/…. In part, this is because the iMac has the newer generation Haswell chips; and in part because with Turboboost, the iMac has the higher clock rate (up to 3.9Ghz). I think the latter are making up for the lower number of cores on the iMac. – wolfies Jan 29 '14 at 16:55
  • Monitoring my CPU usage when the benchmark is running, it never exceeds 25% load. I don't believe the benchmark is running in parallel. Correction: I didn't have the monitor running at fine enough granularity, it does seem to run some fraction of the routines in parallel, leading to brief spikes in CPU usage. – Guillochon Jan 29 '14 at 17:10
  • @wolfies Updated! You're top of the heap for now. – Sjoerd C. de Vries Jan 29 '14 at 18:09
  • 2
    Unfortunately Moore's law predict the increasing of the number of transistors on integrated circuits instead of the performance. The performance grows very slowly, especially for sequential algorithms. – ybeltukov Jan 31 '14 at 21:10
  • @ybeitukov Yes yes - that is of course why the word 'version' in ... 'version' of Moore's law ... above was carefully placed in 'quote marks', because it refers to the vernacular interpretation of Moore's Law. The latter used to apply to performance too ... but no longer. – wolfies Feb 1 '14 at 6:28
  • Some parts of the standard benchmark are parallelized; others are not. The linear algebra elements for instance are parallelized using the Intel MKL. These will be considerably faster on the new processor due to the AVX implementation. I do agree with the premise though that most things we would like to do with Mathematica are either inherently difficult to parallelize or simply lack a parallel implementation at present. The main conclusion is IMO really that it is not possible to have a meaningful benchmark without considering the application of which it is supposed to be representative. – Oleksandr R. Feb 2 '14 at 16:17

You must log in to answer this question.

Browse other questions tagged .