cruncher - A Multi-Threaded Pi Program (2024)

	From a high-school project that went a little too far... By Alexander J. Yee

(Last updated: August 7, 2024)

The first scalable multi-threaded Pi-benchmark for multi-core systems...

How fast can your computer compute Pi?

y-cruncher is a program that can compute Pi and other constants to trillions of digits.

It is the first of its kind that is multi-threaded and scalable to multi-core systems. Ever since its launch in 2009, it has become a common benchmarking and stress-testing application for overclockers and hardware enthusiasts.

y-cruncher has been used to set several world records for the most digits of Pi ever computed.

• 202 trillion digits - May 2024 (Jordan Ranous, Kevin O’Brien, and Brian Beeler - StorageReview)
• 105 trillion digits - February 2024 (Jordan Ranous, Kevin O’Brien, and Brian Beeler - StorageReview)
• 100 trillion digits - June 2022 (Emma Haruka Iwao)
• 62.8 trillion digits - August 2021 (UAS Grisons)
• 50 trillion digits - January 2020 (Timothy Mullican)
• 31.4 trillion digits - January 2019 (Emma Haruka Iwao)
• 22.4 trillion digits - November 2016 (Peter Trueb)
• 13.3 trillion digits - October 2014 (Sandon Van Ness "houkouonchi")
• 12.1 trillion digits - December 2013 (Shigeru Kondo)
• 10 trillion digits - October 2011 (Shigeru Kondo)
• 5 trillion digits - August 2010 (Shigeru Kondo)

Current Release:

Windows: Version0.8.5 Build 9543(Released: July 21, 2024)

Linux: Version0.8.5 Build 9543(Released: July 21, 2024)

Official Mersenneforum Subforum.

Official HWBOT forum thread.

News:

Zen5's AVX512 Teardown: (August 7, 2024) - permalink

AMD's Zen5 has officially launched. That means another architectural teardown of the AVX512!

Zen5 Optimizations: (July 20, 2024) - permalink

Optimizations for AMD's upcoming Zen5 processors has been released for y-cruncher v0.8.5. The latest build (9542) will a new binary "24-ZN5 ~ Komari".

At this time, I cannot say much other than the following:

• The speedup of "24-ZN5" over "22-ZN4" is greater for single-threaded computations than multi-threaded computations.
• The BBP program has not changed. Both "22-ZN4" and "24-ZN5" have the same BBP code.
• None of the existing binaries have changed other than a minor bug fix. So existing benchmarks done with them will not be invalided by this update.

So if you are hardware reviewer who will be using y-cruncher for a Zen5 review, you will want to include single-threaded and BBP benchmarks. Furthermore, I only expect speedups to happen on the Granite Ridge desktop CPUs. I do not expect any speedup on the Strix Point laptop CPUs due to their cut-down AVX512.

Older News

Records Set by y-cruncher:

y-cruncher has been used to set a number of world record sized computations.

Blue: Current World Record

Green: Former World Record

Red: Unverified computation. Does not qualify as a world record until verified using an alternate formula.

See the complete list including other notably large computations. If you want to set a record yourself, the rules are in that link.

Features:

The main computational features of y-cruncher are:

• Able to compute Pi and other constants to trillions of digits.
• Two algorithms are available for most constants. One for computation and one for verification.
• Multi-Threaded - Multi-threading can be used to fully utilize modern multi-core processors without significantly increasing memory usage.
• Vectorized - Able to fully utilize the SIMD capabilities for most processors. (SSE, AVX, AVX512, etc...)
• Swap Space management for large computations that require more memory than there is available.
• Multi-Hard Drive - Multiple hard drives can be used for faster disk swapping.
• Semi-Fault Tolerant - Able to detect and correct for minor errors that may be caused by hardware instability or software bugs.

Download:

Latest Releases: (July 21, 2024)

Downloading any of these files constitutes as acceptance of the license agreement.

OS	Download Link	Size
Windows	y-cruncher v0.8.5.9543.zip	42.9 MB
Linux (Static)	y-cruncher v0.8.5.9543-static.tar.xz	33.7 MB
Linux (Dynamic)	y-cruncher v0.8.5.9543-dynamic.tar.xz	26.7 MB

Downloads can also be found on GitHub. Use this if you prefer HTTPS.

The Linux version comes in both statically and dynamically linked versions. The static version should work on most Linux distributions, but lacks TBB and NUMA binding. The dynamic version supports all features, but is less portable due to the DLL dependency hell.

HWBOT submission is back with this release. So I expect the leaderboards to be rewritten soon.

System Requirements:

Windows:

• Windows 7 or later.
• The HWBOT submitter requires the Java 8 Runtime.

Linux:

• 64-bit Linux is required. There is no support for 32-bit.
• The dynamic version has been tested on Ubuntu 24.04.

All Systems:

• An x86 or x64 processor.

Very old systems that don't meet these requirements may be able to run older versions of y-cruncher. Support goes all the way back to even before Windows XP.

Version History:

• y-cruncher
• HWBOT Submitter

Other Downloads (for C++ programmers):

• Source code for the Digit Viewer
• Mini-Pi: A tiny self-contained Pi program

Advanced Documentation:

• Multi-Threading
• Memory Allocation
• Swap Mode
• Custom Formulas

Benchmarks:

Comparison Chart: (Last updated: July 8, 2024)

Computations of Pi to various sizes. All times in seconds. All computations done entirely in ram.

The timings include the time needed to convert the digits to decimal representation, but not the time needed to write out the digits to disk.

Blue: Benchmarks are up-to-date with the latest version of y-cruncher.

Green: Benchmarks were done with an old version of y-cruncher that is comparable in performance with the current release.

Red: Benchmarks are significantly out-of-date due to being run with an old version of y-cruncher that is no longer comparable with the current release.

Purple: Benchmarks are from unreleased internal builds that are not speed comparable with the current release.

Laptops + Low-Power:

Mainstream Desktops:

High-End Desktops:

Multi-Processor Workstation/Servers:

Due to high core count and the effect of NUMA (Non-Uniform Memory Access), performance on multi-processor systems are extremely sensitive to various settings. Therefore, these benchmarks may not be entirely representative of what the hardware is capable of.

Fastest Times:

The full chart of rankings for each size can be found here:

• Rankings: 25m
• Rankings: 50m
• Rankings: 100m
• Rankings: 250m
• Rankings: 500m
• Rankings: 1b
• Rankings: 2.5b
• Rankings: 5b
• Rankings: 10b
• Rankings: 25b
• Rankings: 50b
• Rankings: 100b
• Rankings: 250b
• Rankings: 500b
• Rankings: 1t

These fastest times may include unreleased betas.

Got a faster time? Let me know: a-yee@u.northwestern.edu

Note that I usually do not respond to these emails. I simply put them into the charts which I update periodically (typically within 2 weeks).

Performance Tips:

Memory Bandwidth:

Because of the memory-intensive nature of computing Pi and other constants, y-cruncher needs a lot of memory bandwidth to perform well. In fact, the program has been noticably memory bound on nearly all high-end desktops since 2012 as well as the majority of multi-socket systems since at least 2006.

Recommendations:

• Make sure all memory channels are populated. This is by far the most important since bandwidth scales almost linearly with the # of channels.
• Run your memory at as high a frequency as possible to maximize bandwidth.
• Memory timings are usually less important. Long memory latencies are hidden away fairly well by Hyperthreading.
• On Skylake X processors, L3 cache bandwidth is also a bottleneck. So overclock the cache as much as possible.

Don't be surprised if y-cruncher exposes instabilities that other applications and stress-tests do not. y-cruncher is unusual in that it simultaneously places a heavy load on both the CPU and the entire memory subsystem.

Parallel Performance:

y-cruncher has a lot of settings for tuning parallel performance. By default, it makes a best effort to analyze the hardware and pick the best settings. But because of the virtually unlimited combinations of processor topologies, it's difficult for y-cruncher to optimally pick the best settings for everything. So sometimes the best performance can only be achieved with manual settings.

• Try both the Push Pool and Cilk Plus frameworks. While the Push Pool is faster in most cases, Cilk Plus may be better for extremely small computations as well as on machines with many (> 64) cores.*
• Experiment with larger task decomposition sizes. This may alleviate problems with load-imbalance.*
• On Windows, if the system has more than 64 logical cores, make sure node-interleaving is disabled in the BIOS. Otherwise, it would lead to imbalanced processor groups which will lead to load-imbalance.

*These are advanced settings that cannot be changed if you're using the benchmark option in the console UI. To change them, you will need to either run benchmark mode from the command line or use the custom compute menu.

Load imbalance is a faily common problem in y-cruncher. The usual causes are:

1. The number of logical cores is not a power-of-two.
2. The cores are not hom*ogenous. Common reasons include:
   • The cores are clocked at different speeds.
   • The cores have access to different amounts of memory bandwidth due an imbalanced NUMA topology.
   • The cores are different generation cores hidden behind a virtual machine.
3. CPU-intensive background processes are interfering with y-cruncher's ability to use all the hardware. This applies to all forms of system jitter.

Large Pages:

Large pages used to not matter in the past, but they do now in the post-Spectre/Meltdown world. Mitigations for the Meltdown vulnerability can have a noticeable performance drop for y-cruncher (up to 5% has been observed). It turns out that turning on large pages can mitigate the penalty for this mitigation. (pun intended)

Refer to the memory allocation guide on how to turn on large pages.

Swap Mode:

This is probably one of the most complicated features in y-cruncher.

• Read the guide so you know how to use it.
• Depending on the CPU capability of your system, chances are you will either need multiple NVMe SSDs or many hard drives to avoid bottlenecking on disk I/O.
• Don't use hardware or software RAID. y-cruncher usually does a better job if you let it manage each drive separately.
• Don't use SSDs if you care about their lifespan. y-cruncher can and will destroy SSDs if you sustain it long enough.

Known Issues:

Everything in this section is in the process of being re-verified and moved to: https://github.com/Mysticial/y-cruncher/issues

Performance Issues:

• Swap computations on the latest Ubuntu (15.10) and possibly everything else with the same kernel version have very poor performance in swap mode. This is because the OS does excessive and unnecessary disk swapping to the pagefile. The solution is to disable the swap file so that no paging is possible. It may also suffice to set the "swappiness" value to zero. y-cruncher will also attempt to lock pages in memory to prevent the OS from shooting itself with paging.

Algorithms and Developments:

• Algorithms and Internals
• Current and Future Developments
• Technology Wishlist

FAQ:

Pi and other Constants:

• Where can I download the digits for Pi and other constants that are featured in the various record computations?
• Can you search for XXX in the digits of Pi for me?
• Can you add support for more constants? I want to compute, Khinchin, Glaisher, etc...
• I have found an amazing new algorithm for X that's better than the rest!

Program Usage:

• What's with the warning, "Unable to acquire the permission, "SeLockMemoryPrivilege". Large pages and page locking may not be possible."?
• What's the difference between "Total Computation Time" and "Total Time"? Which is relevant for benchmarks?
• Why does y-cruncher need administrator privileges in Windows to run Swap Mode computations?
• Why is the performance so poor for small computations? The program only gets xx% CPU utilization on my xx core machine for small sizes!!!
• I've found a bug! How do I report it? Can you fix it?

Hardware and Overclocking:

• My computer is completely stable. But I keep getting errors such as, "Redundancy Check Failed: Coefficient is too large".
• Why is y-cruncher so much slower on AMD processors than Intel processors?
• Why is y-cruncher faster on AMD processors than Intel processors?

Academia:

• Is there a version that can use the GPU?
• Why can't you use distributed computing to set records?
• Is there a distributed version that performs better on NUMA and HPC clusters?
• Why have recent Pi records used desktops instead of supercomputers?
• Why do desktops use the Chudnovsky formula while supercomputers use the AGM? Isn't the AGM faster?

Programming:

• What is the technical explanation for the notorious, "Coefficient is too large" error?
• Is there a publicly available library for the multi-threaded arithmetic that y-cruncher uses?
• Is y-cruncher open-sourced?

Other:

• What about support for other platforms? Mac, ARM, etc...

Links:

Here's some interesting sites dedicated to the computation of Pi and other constants:

• Mathematical Constants and Computation
• Shigeru Kondo's Pi pages (dead)
• Stu's Pi page (dead)

Questions or Comments

Contact me via e-mail. I'm pretty good with responding unless it gets caught in my school's junk mail filter.

You can also find me on Twitter as @Mysticial.