Making more informed Linux hardware choices, courtesy of Phoronix.

More information about version 3.0 of the Phoronix Test Suite and OpenBenchmarking.org from SCaLE, the Southern California Linux Expo. The accompanying Phoronix article is here.


Slides available here:

Making more informed Linux hardware Choices - Phoronix

OpenBenchmarking.org and PTS 3.0 - new advances in benchmarking

I'm really pleased to see the launch of OpenBenchmarking.org. Read the welcome note from Michael Larabel, the Founder of Phoronix Media, Lead Developer Of The Phoronix Test Suite & OpenBenchmarking.org:

WELCOME TO OPENBENCHMARKING.ORG
26 FEBRUARY 2011

Whether you consider yourself a Linux hardware enthusiast, the normal Windows user wishing to transition to Linux, or the procurement officer of a Fortune 500 company, welcome to OpenBenchmarking.org. OpenBenchmarking.org is an open, collaborative testing platform for benchmarking and performance analysis. Regardless of operating system, there is nothing similar, and this is the largest public Linux hardware database, but it's much more than that; it's an adaptive, discovery platform with a focus on repeatable benchmarks, global comparisons, crowd-sourced / community testing, and a high level of interactivity. There are thousands of computer components being tracked in a variety of configurations and thousands of test results accumulating from different platforms whether they be desktops, servers, or even mobile devices.

There are a number of unique concepts demonstrated by OpenBenchmarking.org, including crowd-sourced benchmarking, the ease to which benchmarks can be repeated, results can be stored publicly or privately and shared throughout the world, and unique features for performance classifications and advanced hardware/software searches. OpenBenchmarking.org is designed to be very extensible to meet the needs of both individuals and software projects / organizations.

Arriving at this same time is the general availability of Phoronix Test Suite 3.0-Iveland. This release features integration with OpenBenchmarking.org for uploading and downloading test results, seamlessly obtaining updated/new test profiles and test suites, logging into an OpenBenchmarking.org user accounts, MIME OpenBenchmarking-format launching capabilities, support for attaching system logs and other hardware/software statistics, and other functionality. In addition, Phoronix Test Suite 3.0 offers enhanced result graphs, support for external result notification hooks, support for nesting the Phoronix Test Suite within Wine environments, recognition of new code compilers, client support for new embedded operating environments, and over 200 other changes since the release of Phoronix Test Suite 2.8 in 2010.

In exploring OpenBenchmarking.org, here are a few real-world examples to show off some of the areas of OpenBenchmarking.org:

Sample Searches: Radeon HD 4850, Intel Core i7 970, Ubuntu 10.10, GPU, AMD
Sample Test Results: A Proper Graphics Comparison, A Simple Test
Sample Test Profile Pages: Phoronix Test Suite, Nexuiz
Sample User Pages: Michael Larabel, Phoronix Test Suite

Over the coming days and weeks, OpenBenchmarking.org will continue to be tuned and a number of new features will continue to be introduced. You can view this first implementation right now as a "beta", but any and all feedback regarding OpenBenchmarking.org is welcome and much appreciated. In particular, there are major refinements coming very soon to the product search functionality, support for complex searches and sub-queries, enabling the upload of user test profiles and suites, further advancements to OpenBenchmarking.org Performance Classifications (OPC), support for embedding rich-format graphs externally, and much more.

Further in the pipeline is an external API, support for running Phoromatic over the OpenBenchmarking.org platform, better support for peripheral devices, the official roll-out of Phoronix Certification & Qualification Suites (PCQS), OpenBenchmarking.org Professional, and much more.

Please try out OpenBenchmarking.org and any questions or comments are much appreciated.

Michael Larabel
Founder of Phoronix Media, Lead Developer Of The Phoronix Test Suite & OpenBenchmarking.org

Phoronix releases Phoromatic 1.0!

Phoronix, ever on the leading edge of cross-platform benchmarking, announced today that they are releasing version 1.0 of Phoromatic, an innovative remote test management system used in conjunction with the Phoronix Test Suite.

Among the new features in Phoromatic 1.0:

• Generation of composite graphs (showing geometric mean, harmonic mean, and aggregate sum for all test data on a particular day).
• Narrowing down the graphs that are visible
• Viewing information about Phoromatic test systems in real-time
• Creation of an automatic RSS feed for test results
• Feature rich results tables
• Integrated system log viewers (such as this one).

Along with the announcement of Phoromatic 1.0, Phoronix has announced a partnership with Canonical, the company behind Ubuntu Linux, to provide daily updates on the performance of the most recent daily builds of Ubuntu packages (10.04 as of now). It is called the Phoromatic Ubuntu Tracker. Daily Ubuntu benchmarks are provided on three systems- a Core 2 Duo Mac Mini, as well as two Atom 330 NetTops. This automated testing and benchmarking will provide much-needed insight into the development process, as there will be a great deal of test data available when changes are made to packages in the Operating System.

To reinforce Phoromatic's focus on cross-platform benchmarking, they mention that this benchmarking is available on Linux, any of the BSD-based OSes, Mac OSX, and OpenSolaris. They are also working on support for Windows Operating Systems, as well.

For companies that are looking to perform internal benchmark testing, Phoronix offers PTS Commercial, which offers enterprise-level support to the benchmarking process.

For those of you with an eye towards the horizon, Phoronix is working feverishly on Phoronix Test Suite version 2.6, code-named "Lyngen", and plan to release version 2.6 in May of 2010.

Happy Benchmarking, everyone!

PTS Desktop Live 2010.1 released!

Phoronix announced today that they have released the first major update to the Phoronix Test Suite's Live CD/DVD. The new version is PTS Desktop Live 2010.1, codenamed "Anzhofen". It is based on Ubuntu Lucid Lynx 10.04 and integrates the Phoronix Test Suite 2.4.1 in a live CD or USB stick.

This release includes files for 43 of the nearly 130 tests, so you can do some testing even without an internet connection. If you have internet access, you can download and run additional tests. PTS Desktop Live 2010.1 also includes integration with Phoronix Global (an online benchmark publishing site) and Phoromatic (remote benchmark administration). Documentation for the PTS Desktop Live project is available here.

Phoronix releases Phoronix Test Suite 2.4!

The fine people over at Phoronix have recently announced the release of version 2.4 of the Phoronix Test Suite.

Among the new features:

• New Operating System support -- PTS 2.4 now has enhanced compatibilty for OSes such as Palm WebOS, OpenSolaris FreeBSD 8.0, OpenBSD, NetBSD, as well as initial OS support for Windows 7. Imagine running the same benchmark on your Palm Pre, OpenBSD server, and Win 7 home machine!
• Actual image quality comparisons -- PTS 2.4 is introducing tests that provide tools for examining rendered images on a pixel by pixel basis to look for rendering or artifacting differences. Initial tests are available for Nexuiz and Enemy Territory: Quake Wars, though they are looking to expand support in future versions.

Another announcement that was made is regarding the Live DVD project of the Phoronix Test Suite, called PTS Desktop Live. It appears that a new version of the Live DVD will be coming this month, followed by the first release of a testing suite specifically designed for netbooks, called PTS Netbook Live. They're still light on the details for PTS Netbook Live, but keep your eyes peeled as developments continue.

Phoronix releases version 2.2 of Phoronix Test Suite!

Phoronix recently announced on their website that version 2.2 "Bardu" of the Phoronix Test Suite has been released. The official release contains quite a few fixes and new features, as the changelog shows. The PTS Live DVD release based on Phoronix Test Suite version 2.2 is expected in December of this year.
The benchmarking suite is currently available for the Linux, OpenSolaris, BSD, and Mac OS X platforms.
The downloads page is located here.
Happy Benchmarking, everyone!

EDIT:: I've been having issues running the GUI for the Phoronix Test Suite. The error message that I get in the command line after running "phoronix-test-suite gui" is :
Class 'GtkWindow' not found in /usr/share/phoronix-test-suite/pts-core/objects/gtk/pts_gtk_window.php on line 23
Apparently it's an issue with the lack of a compiled PHP GTK package ( or at least the forum posts say so.)
Command line running of PTS 2.2 should still work just fine.

Phoronix Releases PTS Desktop Live 2009.3 - Benchmarking Live-DVD

Good news for all of you benchmarking junkies out there. Phoronix has recently released PTS Desktop Live 2009.3, a live-DVD version of the Phoronix Test Suite for use in benchmarking Linux, BSD, OpenSolaris, and Mac OSX systems. Along with the live-DVD, they have also announced version 2.0 of the Phoronix Test Suite. A link to the changelog is here.

To provide a little history about the Phoronix testing suite, Phoronix has been on the forefront of Linux benchmarking for several years. The original intent of the testing suite was to provide a uniform set of benchmarking tools for the Linux/BSD/OpenSolaris/OSX crowd, as the tools that were previously available, such as glxgears, were not as informative as Phoronix would have liked. Since no tools existed, they decided to make their own internal tools in 2004. Eventually they realized that the public had the same needs, so they decided to release a public version of the testing suite, as well. Their initial public release (0.2.0) of the Phoronix testing suite was released on April 2, 2008, and they have been providing a steady stream of updates to the testing suite since then. I had used early versions of this testing suite, and I had found it a little clunky and difficult to manage on an installed Linux system, so this live-DVD provides a real landmark in the usability of these benchmarking tools. Also, since every system will be running the live DVD, it provides a certain measure of standardization to the testing process, as the Live DVD makes the test environment standardized among different systems.

System Requirements of the PTS Live DVD are:

- Intel / AMD x86_64 CPU
- 2GB+ of RAM Recommended
- Internet Connection Recommended
- ATI, NVIDIA, or Intel Graphics

A link to the download is here. Hosting is being provided by Softpedia at the moment. The MD5 is 93a7a9cc202b146162c211d41bbeaa99.


Phoronix is not resting on its laurels, either. They are planning on releasing the Phoronix Certification & Qualification Suite (PCQS), a stress test for use in validating system hardware on Linux system, sometimes later this year. It will be another useful tool for Linux system administrators.

Another tool that is in beta testing with Phoronix is Phoromatic, a remote management tool for the Phoronix Testing Suite. Starting with version 2.0 of the Phoronix Testing Suite, remote management will be available through an account on phroromatic.com (or via a LAN version of the phoromatic software). No news on the intergration with the live DVD, but I'm thinking there is a good chance it will be able to do that, too. The following description comes directly from the site:

Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name Phoromatic) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.

Phoromatic is able to communicate with any system running Phoronix Test Suite 2.0 or newer whether it be running Linux, Mac OS X, BSD, or OpenSolaris. Creating a single-user account at Phoromatic.com is free, easy, and can then connect to one or even dozens of test systems running the Phoronix Test Suite. Phoromatic can also be installed on a LAN / intranet for professional support customers of the Phoronix Test Suite.