Bonnie - File System Benchmark

Bonnie - File System Benchmarks

Below are tables of results from a file system benchmark called bonnie. I usually use my own benchmark tool to determine the read and write performance in MBytes/sec because it more closely resembles how we use a file system. However, many people have asked that I provide the results from bonnie. To help ensure that the hardware plays a role in the results, I use file sizes of 1024 MBytes or more. A value greater than the total amount of RAM in the system ensures that data will be flushed out to disk and not just stored in file system buffers or cache.

Although the bonnie benchmark tests were run on idle systems, cache1 - cache10, mss3 - mss6, and mssdb1 had been in use and the file systems contained files which could not be removed. Therefore, the results between two identical systems may be different because of the tacks used to store the test file (inner vs. outer tracks). All other systems were tested with newly created file systems.

Also, note that the tables are grouped by RAID level and the type of RAID controller or disk drive used.

RAID 0 - SCSI Disk Systems

				Sequential Output						Sequential Input				Random Seeks
				Per Char		Block		Rewrite		Per Char		Block		Random Seeks
Machine	Year	MB	FS	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	/Sec	%CPU
cache1	2001	1024	Ext2	10027	99.3	73212	68.2	17708	30.1	8849	82.5	62443	37.6	551.1	8.1
cache2	2001	1024	Ext2	10146	99.9	71577	65.9	17835	30.8	8926	82.6	63282	36.3	526.1	8.8
cache3	2001	1024	Ext2	10792	99.7	66572	60.3	15849	26.7	9894	85.5	56909	31.1	509.2	6.6
cache4	2001	1024	Ext2	10875	99.9	74694	70.4	18300	30.6	10083	86.4	67127	34.9	554.3	7.6
cache5	2001	1024	Ext2	10876	99.9	74696	69.5	20240	33.2	9996	85.9	59739	37.2	629.8	9.0
cache6	2001	1024	Ext2	10885	99.9	78616	75.9	21809	36.7	10193	87.7	65874	38.7	609.3	7.6
cache7	2001	1024	Ext2	10825	99.2	67694	63.3	18747	31.4	10228	87.9	59150	33.5	534.9	8.8
cache8	2001	1024	Ext2	10741	99.7	78877	87.7	19786	37.9	10085	87.2	64227	36.6	546.1	8.9
cache9	2001	1024	Ext2	10376	98.2	70065	71.3	19614	35.7	9468	84.3	59726	37.9	560.4	9.0

RAID 0 - IDE Disk Systems

				Sequential Output						Sequential Input				Random Seeks
				Per Char		Block		Rewrite		Per Char		Block		Random Seeks
Machine	Year	MB	FS	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	/Sec	%CPU
cache10	2001	1024	Ext2	11475	99.9	72262	97.6	19709	27.6	10748	88.4	59041	31.8	547.6	8.4
cache11	2001	1024	Ext2	13178	99.8	87447	68.3	29877	23.7	12304	97.6	88335	35.6	518.8	6.0
cache12	2001	1024	Ext2	13177	99.9	76210	64.8	29770	23.6	12315	97.8	87837	35.5	549.0	6.5
cache13	2001	1024	Ext2	13224	99.9	96014	58.5	29140	24.0	12140	95.5	88473	37.0	478.0	6.7
cache14	2001	1024	Ext2	13212	100.0	94530	59.7	31432	25.6	12186	96.1	88640	37.1	483.5	4.7

RAID 5 - SCSI Disk Systems

				Sequential Output						Sequential Input				Random Seeks
				Per Char		Block		Rewrite		Per Char		Block		Random Seeks
Machine	Year	MB	FS	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	/Sec	%CPU
mssdb1	2001	1024	Ext2	10951	89.1	19404	15.0	10314	16.6	12837	97.0	51200	19.0	643.6	7.2
mss3	2001	1024	Ext2	15593	98.7	19306	25.4	9871	11.5	11236	67.5	64185	26.2	652.0	8.6
mss4	2001	1024	Ext2	15587	99.8	15587	21.3	8689	10.1	11617	71.4	55048	22.4	518.5	8.9
mss5	2001	1024	Ext2	14186	100.0	19872	44.5	10454	7.0	9379	69.4	67459	27.1	556.6	6.4
mss7	2001	1024	Ext2	14215	100.0	20722	40.7	11577	7.6	10665	78.7	71043	23.1	586.6	7.0
mss8	2001	1024	Ext2	14231	99.9	19850	32.0	12935	8.8	11376	83.0	72199	23.3	726.5	9.4
fs11	2001	1024	Ext2	14193	99.9	14087	42.7	13319	10.0	8581	63.1	51734	18.3	643.3	8.7
fs14	2001	1024	Ext2	14207	100.0	15377	24.6	12989	9.3	7991	58.4	51488	16.4	667.9	8.0
mss12	2002	1024	Ext3	14452	92.6	14325	11.2	12192	8.5	13508	81.9	73031	19.8	673.4	4.5

RAID 5 - IDE Disk Systems

				Sequential Output						Sequential Input				Random Seeks
				Per Char		Block		Rewrite		Per Char		Block		Random Seeks
Machine	Year	MB	FS	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	/Sec	%CPU
test	2002	1024	Ext2	14175	100.0	51199	28.8	21130	14.7	12106	89.1	42002	16.4	421.8	4.4
cache22 (1)	2002	1024	Ext2	11933	86.8	58816	33.4	15428	16.8	12222	90.4	81446	32.2	4064.4	33.5
cache22 (1)	2002	1024	Ext3	11792	94.4	22390	21.5	16552	16.6	10676	80.0	90583	29.3	3681.6	35.9
cache22 (1)	2002	1024	Ext3 wbk	10031	81.0	18475	18.6	12904	15.6	11480	86.5	91204	36.4	4532.2	38.5
cache22 (1)	2002	1024	XFS	12621	95.9	56078	37.6	37624	34.6	12865	96.1	96706	42.4	1056.1	9.2
cache22 (1)	2002	2047	Ext2	11644	86.0	50326	29.5	14394	14.8	13007	95.9	82838	33.8	557.6	7.4
cache22 (1)	2002	2047	Ext3	11679	94.0	19321	18.5	14981	14.7	11988	89.6	91210	36.3	421.5	5.2
cache22 (1)	2002	2047	Ext3 wbk	9092	74.0	17797	18.1	12449	14.8	12037	90.2	91349	34.8	574.3	4.9
cache22 (1)	2002	2047	XFS	11793	90.7	53197	33.8	36510	34.3	13174	98.3	94895	41.1	438.8	5.8
fs16 (1)	2002	2047	XFS	23444	100.0	51995	22.0	36449	15.6	22806	90.5	148400	28.3	1423.3	29.5
fs16 (2)	2002	2047	XFS	23312	99.4	59281	24.7	42362	19.1	22843	90.2	140556	22.1	1643.6	6.6

RAID 50 - IDE Disk Systems

				Sequential Output						Sequential Input				Random Seeks
				Per Char		Block		Rewrite		Per Char		Block		Random Seeks
Machine	Year	MB	FS	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	K/Sec	%CPU	/Sec	%CPU
cache22 (2)	2002	1024	Ext2	13203	97.1	88470	56.8	29341	29.5	13096	97.4	198582	91.1	3113.9	21.0
cache22 (2)	2002	1024	Ext3	12199	98.8	45191	45.3	31390	31.8	11656	88.7	175081	92.3	3796.5	33.2
cache22 (2)	2002	1024	XFS	12631	96.5	67111	42.1	60971	51.5	12680	97.2	177265	97.0	1654.2	15.3
cache22 (2)	2002	2047	Ext2	12825	94.3	85652	54.8	25515	26.7	13140	97.8	188150	95.1	640.2	6.7
cache22 (2)	2002	2047	Ext3	12243	98.7	38757	39.6	25660	27.2	12679	95.9	173684	94.8	547.3	4.7
cache22 (2)	2002	2047	XFS	12125	93.2	64319	40.7	60649	54.6	12655	97.5	175708	96.5	501.4	4.5

cache1, cache2 (SCSI Disks)

Processors - Dual Intel Pentium III 650 MHz
Memory - 512 MBytes 100 MHz SDRAM ECC
Motherboard - ASUS P2B-D
RAID Controller - Mylex eXtremeRAID 1100 with 32 MBytes cache
Disk Drives - Seagate ST150176LW (Qty. 8)
RAID Configuration - RAID-0 (2 disks on channel-0 and 6 disks on channel-1)
File System = Ext2
OS - Red Hat 6.2, Linux Kernel 2.2.18pre11-va1.1

cache3, cache4 (SCSI Disks)

Processors - Dual Intel Pentium III 700 MHz
Memory - 512 MBytes 100 MHz SDRAM ECC
Motherboard - ASUS P2B-D
RAID Controller - Mylex eXtremeRAID 1100 with 64 MBytes cache
Disk Drives - Seagate ST150176LW (Qty. 8)
RAID Configuration - RAID-0 (4 disks on channel-0 and 4 disks on channel-1)
File System = Ext2
OS - Red Hat 6.2, Linux Kernel 2.2.18pre11-va1.1

cache5 - cache9 (SCSI Disks)

Processors - Dual Intel Pentium III 700 MHz
Memory - 512 MBytes 100 MHz SDRAM ECC
Motherboard - ASUS P2B-D
RAID Controller - Mylex eXtremeRAID 1100 with 32 MBytes cache
Disk Drives - Seagate ST173404LW (Qty. 6)
RAID Configuration - RAID-0 (3 disks on channel-0 and 3 disks on channel-1)
File System = Ext2
OS - Red Hat 6.2, Linux Kernel 2.2.18pre11-va1.1

cache10 (IDE Disks)

Processors - Dual Intel Pentium III 733 MHz
Memory - 512 MBytes 133 MHz SDRAM ECC
Motherboard - ASUS CUR-DLS
RAID Controller - 3ware Escalade 6800
Disk Drives - IBM DTLA-307075 (Qty. 6)
RAID Configuration - RAID-0
File System = Ext2
OS - Red Hat 6.2, Linux Kernel 2.2.18pre11-va1.1

cache11 - cache20 (IDE Disks)

Processors - Dual Intel Pentium III 933 MHz
Memory - 512 MBytes 133 MHz SDRAM ECC
Motherboard - Intel STL2
RAID Controller - 3ware Escalade 6800
Disk Drives - IBM DTLA-307075 (Qty. 6)
RAID Configuration - RAID-0
File System = Ext2
OS - Red Hat 7.1, Linux Kernel 2.4.2-2

cache21 - cache22 (IDE Disks)

Processors - Dual Intel Pentium III 1 GHz
Memory - 1024 MBytes 133 MHz SDRAM ECC
Motherboard - Intel STL2
RAID Controller - 3ware Escalade 7850
Disk Drives - IBM DTLA-307075 (Qty. 12)
RAID Configuration - RAID-5 (6 disks)
File System = Ext2, Ext3, XFS
OS - Red Hat 7.3, Linux Kernel 2.4.18-lkpc-4
(1) File system on one 6-disk RAID-5 array.
(2) File system on a RAID-0 stripe over two 6-disk RAID-5 arrays.

mssdb1 (SCSI Disks)

Processors - Dual Intel Pentium III 800 MHz
Memory - 512 MBytes 100 MHz SDRAM ECC
Motherboard - ASUS P2B-D
RAID Controller - Mylex eXtremeRAID 1100 with 64 MBytes cache
Disk Drives - IBM DDYS-T18350M (Qty. 4)
RAID Configuration - RAID-5 (4 disks on channel-0)
File System = Ext2
OS - Red Hat 6.2, Linux Kernel 2.2.17-14

mss3 (SCSI Disks)

Processors - Dual Intel Pentium III 1 GHz
Memory - 512 MBytes 133 MHz SDRAM ECC
Motherboard - Intel STL2
RAID Controller - Mylex eXtremeRAID 2000 with 64 MBytes cache
Disk Drives - Seagate ST336705LC (Qty. 10)
RAID Configuration - RAID-5 (5 disks on channel-0 and 5 disks on channel-1)
File System = Ext2
OS - Red Hat 6.2, Linux Kernel 2.2.18pre11-va2.1

mss4 - mss7 (SCSI Disks)

Processors - Dual Intel Pentium III 1 GHz
Memory - 512 MBytes 133 MHz SDRAM ECC
Motherboard - Intel STL2
RAID Controller - Mylex eXtremeRAID 2000 with 64 MBytes cache
Disk Drives - Seagate ST336705LC (Qty. 5)
RAID Configuration - RAID-5 (5 disks on channel-0)
File System = Ext2
OS - Red Hat 7.1, Linux Kernel 2.4.2-2

mss8 - mss10 (SCSI Disks)

Processors - Dual Intel Pentium III 1 GHz
Memory - 512 MBytes 133 MHz SDRAM ECC
Motherboard - Intel STL2
RAID Controller - Mylex eXtremeRAID 2000 with 64 MBytes cache
Disk Drives - Seagate ST373405LC (Qty. 6)
RAID Configuration - RAID-5 (4 disks on channel-0 and 2 disks on channel-1)
File System = Ext2
OS - Red Hat 7.1, Linux Kernel 2.4.2-2

mss11 - mss12 (SCSI Disks)

Processors - Dual Intel Pentium III 1.26 GHz
Memory - 512 MBytes 133 MHz SDRAM ECC
Motherboard - Intel SDS2
RAID Controller - Mylex eXtremeRAID 2000 with 64 MBytes cache
Disk Drives - Seagate ST373405LC (Qty. 7)
RAID Configuration - RAID-5 (4 disks on channel-0 and 3 disks on channel-1)
File System = Ext3
OS - Red Hat 7.2, Linux Kernel 2.4.18-lkpc-4

fs11 - fs14 (SCSI Disks)

Processors - Dual Intel Pentium III 1 GHz
Memory - 512 MBytes 133 MHz SDRAM ECC
Motherboard - Intel STL2
RAID Controller - Mylex eXtremeRAID 2000 with 64 MBytes cache
Disk Drives - Seagate ST173404LC (Qty. 8)
RAID Configuration - RAID-5 (8 disks on channel-3)
File System = Ext2
OS - Red Hat 7.1, Linux Kernel 2.4.2-2

test (IDE Disks)

Processors - Dual Intel Pentium III 1 GHz
Memory - 512 MBytes 133 MHz SDRAM ECC
Motherboard - Intel STL2
RAID Controller - external ATA RAID system with an Ultra160 SCSI host connection
Disk Drives - IBM DTLA-307075 (Qty. 8)
RAID Configuration - RAID-5 (7 disks)
File System = Ext2
OS - Red Hat 7.1, Linux Kernel 2.4.2-2

fs16 - fs19 (IDE Disks)

Processors - Dual Intel Pentium 4 Xeon 2.0GHz GHz
Memory - 2 GBytes DDR200 ECC
Motherboard - SuperMicro P4DPE
RAID Controller - 3ware Escalade 7850
Disk Drives - Maxtor 4G160J8 (Qty. 16)
RAID Configuration - RAID-5 (7 disks)
File System = XFS
OS - Red Hat 7.3, Linux Kernel 2.4.19-xfs
(1) Hyper-Threading turned on.
(2) Hyper-Threading turned off.

Contact Information: Andy Kowalski, kowalski@jlab.org

This document is maintained by {helpdesk@jlab.org}