Sort Benchmark Home Page

New: We have finalized the rules for 2010. Our main changes are:

Compression: To maintain the spirit of the benchmark, compression of sort data in the temp files or across the network is not allowed.
Input: Entrants must use a new input generator gensort that creates completely pseudo-random records for the benchmarks that require binary records.
GraySort: A Daytona sort must use the same degree of replication for both the input and output files.
JouleSort: A new 100TB category is introduced.
PennySort: Every year, we receive submissions with price quotes that include discounts. We have clarified that discounts of any kind and refurbised equipment are not allowed. See FAQ.
Deadlines: We are reviewing entries submitted prior to Jan 1, and moved this year's deadline to May 15, 2010.

Please see process changes below and the FAQ for more details.

We will be posting the mid-year (pre Jan 1, 2010) results shortly. Thanks to all contestants for bearing through all the new rule changes.

Background

Until 2007, the sort benchmarks were primary defined, sponsored and administered by Jim Gray. Following Jim's disappearance at sea in January 2007, the sort benchmarks have been continued by a committee of past colleagues and sort benchmark winners. The Sort Benchmark committee members include:

Chris Nyberg of Ordinal Technology
Mehul Shah of Hewlett-Packard
Naga Govindaraju of Microsoft

Year 2009 Results
	Daytona	Indy
Gray	2009, 0.578 TB/min Hadoop 100 TB in 173 minutes 3452 nodes x (2 Quadcore Xeons, 8 GB memory, 4 SATA) Owen O'Malley and Arun Murthy Yahoo Inc.	2009, 0.564 TB/min DEMSort 1,000,029,388,800 records in 10,628 seconds 195 nodes x (2 Quadcore processors, 16 GB memory, 4x250GB disks) 288-port InfiniBand 4xDDR switch Mirko Rahn, Peter Sanders, Johannes Singler and Tim Kieritz Karlsruhe Institute of Technology, Germany
Penny	2009, 223 GB psort 2.6 Ghz AMD Athlon LE 1640, 4 GB RAM, 5x160 GB 7200 RPM SATA, Linux Paolo Bertasi, Marco Bressan and Enoch Peserico Univ. Padova, Italy	2-way tie: 2009, 246 GB OzSort 2.7 Ghz AMD Kuma X2 7750+, 4GB RAM, 5x160 GB 7200 RPM SATA, Linux Nikolas Askitis and Ranjan Sinha Univ. Melbourne, Australia 2009, 248 GB psort 2.6 Ghz AMD Athlon LE 1640, 4 GB RAM, 5x160 GB 7200 RPM SATA, Linux Paolo Bertasi, Marco Bressan and Enoch Peserico Univ. Padova, Italy
Minute	2009, 500 GB Hadoop 1406 nodes x (2 Quadcore Xeons, 8 GB memory, 4 SATA) Owen O'Malley and Arun Murthy Yahoo Inc.	2009, 955 GB DEMSort 195 nodes x (2 Quadcore processors, 16 GB memory, 4x250GB disks) 288-port InfiniBand 4xDDR switch Mirko Rahn, Peter Sanders, Johannes Singler and Tim Kieritz Karlsruhe Institute of Technology, Germany
Joule 10⁸ recs	2007, 8.6 kJoules CoolSort 11,600 records sorted / joule Mobile Core 2 Duo, 13 SATA laptop disks, Nsort Suzanne Rivoire (Stanford), Mehul A. Shah (HP Labs), Partha Ranganathan (HP Labs), Christos Kozyrakis (Stanford)
Joule 10⁹ recs	2007, 88 kJoules CoolSort 11,300 records sorted / joule Mobile Core 2 Duo, 13 SATA laptop disks, Nsort Suzanne Rivoire (Stanford), Mehul A. Shah (HP Labs), Partha Ranganathan (HP Labs), Christos Kozyrakis (Stanford)	2009, 87 kJoules OzSort 11,600 records sorted / joule 2.6 Ghz AMD Athlon LE-1640, 4GB RAM, 7x160 GB 7200 RPM SATA, Linux Nikolas Askitis and Ranjan Sinha Univ. Melbourne, Australia
Joule 10¹⁰ recs	2007, 2920 kJoules CoolSort 3,425 records sorted / joule Mobile Core 2 Duo, 13 SATA laptop disks, Nsort Suzanne Rivoire (Stanford), Mehul A. Shah (HP Labs), Partha Ranganathan (HP Labs), Christos Kozyrakis (Stanford)

Common Rules

All the sort benchmarks share the following ground rules:

Must sort to and from operating system files on secondary storage.
No raw disk usage allowed since we are trying to test the IO subsystem.
File or device striping (RAID 0) are allowed (encouraged) to get bandwidth. If file striping is used then the concatenated files must form a sorted file.
The output file must be created as part of the sort.
Time includes the launching of the sort program.
The sort input records must be 100 bytes in length, with the first 10 bytes being a random key.
Use the gensort record generator to create the input records.
The sort output file must be validated for correct key order and checksum.

Sort Benchmarks

GraySort	Metric: Sort rate (TBs / minute) achieved while sorting a very large amount of data (currently 100 TB minimum).
PennySort	Metric: Amount of data that can be sorted for a penny's worth of system time. Originally defined in AlphaSort paper.
MinuteSort	Metric: Amount of data that can be sorted in 60.00 seconds or less. Originally defined in AlphaSort paper.
JouleSort	Metric: Amount of energy required to sort either 10⁸, 10⁹ or 10¹⁰ records (10 GB, 100 GB or 1 TB). Originally defined in JouleSort paper.
TeraByte Sort	Metric: Elapsed time to sort 10¹² bytes of data. The TeraByte benchmark is now deprecated because it became essentially the same as MinuteSort.
Datamation Sort	Metric: Amount of time to sort one million records (100 MB). This is the original sort benchmark, defined in A Measure of Transaction Processing Power With 25 others Datamation, V 31.7, April 1985, pp 112-118. Originally, winners took 1 hour, now 1 second! So the benchmark is deprecated.

Benchmark Categories

For each sort benchmark, there are two categories:

Daytona (stock car)	Indy (formula 1)
Sort code must be general purpose.	Need only sort 100-byte records with 10-byte keys.

Complete Rules

For detailed rules, see the frequently asked questions.

Process

Medals are awarded each year at ACM SIGMOD.
Entrants can contact us to get their result "certified" by May 15.
Entries must include a document describing the algorithm and the hardware in enough detail so that others could reproduce the result.

New Entries

The submission deadline for the 2010 contest is 15 May 2010.

We will accept early entries for the following year before 15 November 2010 if the new result exceeds the current record by 50% (1.5 times the rate or size). Approved early entries will be published on this site. From 15 November 2010 to 1 April 2011, any entries exceeding the previously published record (including early entries) will be accepted.

Past Winners
	Daytona	Indy
Gray	-	-
Penny	2008, 181 GB 1,812 M records in 2,408 seconds psort 2.4 Ghz AMD Athlon 64, 2 GB RAM, 4x160GB SATA disks Linux Paolo Bertasi, Marco Bressan and Enoch Peserico Univ. Padova, Italy 2007, 39 GB TokuMergeSort 330$ system 2 Ghz AMD Athlon 4200+, 512 MB RAM, 2x80GB SATA disks Bradley C. Kuszmaul , MIT 2006, 34 GB Bytes-Split-Index Sort (BSIS) $760 system 1.8 GHz AMD, 1 GB RAM, 4x80GB SATA disks, WindowsXP Xing Huang and BinHeng Song School of Software, Tsinghua U. , Beijing, China Bo Huang Math & CS, Hunan U. of Technology , Zhuzhou, China 2005, 15 GB PostManSort 979 sec on a $951 Wintel 2 SATA Robert Ramey 2004, 10 GB THsort (105 million records) 1098 seconds on a $857 Linux/AMD Peng Liu, Yao Shi, Li Zhang, Kuo Zhang, Tian Wang, ZunChong Tian, Hao Wang, Xiaoge Wang Tsinghua University, Beijing, China 2000, 4.5 GB HMsort Brad Helmkamp, Keith McCready Stenograph LLC 1999, 2.6 GB HMsort Brad Helmkamp, Keith McCready Stenograph LLC 1998, 1.3 GB PostmanSort/NT Robert Ramey Software	2008, 190 GB psort 2.4 Ghz AMD Athlon 64, 2 GB RAM, 4x160GB SATA disks, Linux Paolo Bertasi, Marco Bressan and Enoch Peserico Univ. Padova, Italy 2006, 59 GB GpuTeraSort 3 GHz Pentium IV, 2 GB RAM, 7800GT Nvidia graphics card, 9x80GB SATA disks (4 data and 5 runs) WindowsXP Naga Govindaraju, Ritesh Kumar, Dinesh Manocha, Jim Gray U. North Carolina at Chapel Hill, USA 2003, 43 GB SheenkSort Linux/AMD system Lei Yang, Hui Huang, Zheng Wan, Tao Song Tsinghua University, Beijing, China 2001, 12 GB DMsort Araron Darling, Alex Mohr, U. Wisconsin, Madison 2000, 4.5 GB HMsort Brad Helmkamp, Keith McCready Stenograph LLC 1999, 2.6 GB HMsort Brad Helmkamp, Keith McCready Stenograph LLC 1998, 1.5 GB NT Sort Microsoft
Minute	2007, 214 GB TokuSampleSort tx2500 disk cluster 400 nodes x (2 processors, 6-disk RAID, 8 GB memory) Bradley C. Kuszmaul, MIT 2006, 40 GB NeoSort Windows, Fujitsu 32 Itanium2, 128 SAN disks Chris Nyberg, Charles Koester Ordinal Technology Corp 2004, 34 GB Nsort Windows, 32 Itanium2, 2,350 disks Chris Nyberg, Charles Koester Ordinal 2000, 12 GB Nsort SGI 32 cpu Origin IRIX 1998, 5.8 GB Nsort SGI 32 cpu Origin IRIX 1997, 3.5 GB Nsort IRIX Challenge Ordinal Technology Corp 1995, 1.1 GB AlphaSort Nyberg	2007, 264 GB TokuSampleSort tx2500 disk cluster 400 nodes x (2 processors, 6-disk RAID, 8 GB memory) Bradley C. Kuszmaul, MIT 2005, 125 GB SCS Linux, 80 Itanium2, 2,520 SAN disks Jim Wyllie, IBM Almaden Research 2004, 32 GB Nsort 32 x Itanium2 WinServer Chris Nyberg, Charles Koester Ordinal Technology 2000, 21.8 GB NOW+HPVMsort 64 nodes WinNT Luis Rivera, Xianan Zhang, Andrew Chien UCSD 1999, 10,3 GB NOW+MPI HPVMsort Luis Rivera, UIUC Andrew Chien, UCSD 1998, 8.41 GB NowSort 95 UltraSparc + MyrinetSolaris UC Berkeley 1997, 3.5 GB Nsort SGI/Nyberg,Koester Nsort/Irix/Challenge 1995, 1.08 GB AlphaSort Nyberg
Joule 10⁸ recs	-	-
Joule 10⁹ recs	-	-
Joule 10¹⁰ recs	-	-
TeraByte (deprecated)	2008, 3.48 minutes Hadoop 910 nodes x (4 dual-core processors, 4 disks, 8 GB memory) Owen OMalley, Yahoo 2007, 4.95 min TokuSampleSort tx2500 disk cluster 400 nodes x (2 processors, 6-disk RAID, 8 GB memory) Bradley C. Kuszmaul , MIT 2004, 33 min Nsort Windows, 32 Itanium2, 2,350 SAN disks Chris Nyberg, Charles Koester Ordinal Technology 2000, 49 min Tandem FastSort 68x2 Compaq Tandem Sandia Daivd Cossock , Sam Fineberg, Pankaj Mehra , John Peck Tandem 1998, 151 min Nsort SGI 32x Origin 2000 Chris Nyberg, Charles Koester Ordinal Technology	2007, 3.28 minutes TokuSampleSort tx2500 disk cluster 400 nodes x (2 processors, 6-disk RAID, 8 GB memory) Bradley C. Kuszmaul , MIT 2005, 7.25 min SCS Linux, 80 Itanium2, 2,520 SAN disks Jim Wyllie , IBM Almaden Research 2000, 18 min SPsort 1952 SP cluster 2168 disks Jim Wyllie , IBM Almaden Research 1998, 151 min Nsort SGI 32x Origin 2000 Chris Nyberg, Charles Koester Ordinal Technology
Datamation (deprecated)		2001, .44 sec Datamation 2001: A Sorting Odyssey NOW-sort on 32 Linux PCs (2xP3(550 MHz), 1 GB, 5x9GB disks). Florentina Popovici, John Bent, Brian Forney, Andrea Arpaci Dusseau, Remzi Arpaci Dusseau 2000, .998 sec Mitsubishi DIAPRISM Hardware Sorter HP 4 x 550MHz Xeon PC server + 32 SCSI disks, Windows NT4 Shinsuke Azuma, Takao Sakuma, Tetsuya Takeo, Takaaki Ando, Kenji Shirai Mitsubishi Electric Corp. 1999, 1.18 sec Millennium Sort 16x2 Dell NT Myrinet Phillip Buonadonna, Spencer Low, Josh Coates UC Berkeley 1997, 2.4 sec NowSort Arpaci-Dusseau UC Berkeley 1996, 4.2 sec Nsort SGI Challenge Chris Nyberg, Charles Koester Ordinal Technology 1994, 7 sec AlphaSort Nyberg, DEC 1993, 9 sec AlphaSort Nyberg, DEC 1988, 28 sec Cray1 Weinberger 1987, 980 sec Tandem Tsukerman

This page is maintained by Chris Nyberg (chris dot nyberg at ordinal dot com) and Mehul Shah (mehul dot shah at hp dot com).