Merge lp:~deeptik/lava-test/tiobench into lp:lava-test/0.0

Proposed by Deepti B. Kalakeri
Status: Merged
Merged at revision: 52
Proposed branch: lp:~deeptik/lava-test/tiobench
Merge into: lp:lava-test/0.0
Diff against target: 73 lines (+69/-0)
1 file modified
abrek/test_definitions/tiobench.py (+69/-0)
To merge this branch: bzr merge lp:~deeptik/lava-test/tiobench
Reviewer Review Type Date Requested Status
Paul Larson (community) Approve
Review via email: mp+37501@code.launchpad.net

Description of the change

This patch automates the installation, execution, and parsing of tiobench IO performance test suite. This patch has been tested on Manderick for installation/execution and parsing of results.

The test first run through with a 4KB block size and then again with a 8KB block size with 10 threads in parallel for 2 iterations. This is just an example.

Please suggest an appropriate values to be used for the parameters which would align to the Linaro testing.

To post a comment you must log in.
Revision history for this message
Paul Larson (pwlars) wrote :

37 + "./tiobench.pl --block=4096 --block=8192 --threads=10 "\
38 + " --size=%s --numruns=2" % (VERSION, FILE_SIZE)]
Is there a reason why --block is specified twice?
Also, we need to scale this WAAAAAY back. Most of the devices we use are booted off a small SD card, so space is very limited. Also, 10 threads on a single-core, low power device is about 8 or 9 too many.

review: Needs Fixing
Revision history for this message
Deepti B. Kalakeri (deeptik) wrote :

> 37 + "./tiobench.pl --block=4096 --block=8192 --threads=10 "\
> 38 + " --size=%s --numruns=2" % (VERSION, FILE_SIZE)]
> Is there a reason why --block is specified twice?
> Also, we need to scale this WAAAAAY back. Most of the devices we use are

I did not get with point "we need to scale this WAAAAAY back." Can you elaborate.?

> booted off a small SD card, so space is very limited. Also, 10 threads on a
> single-core, low power device is about 8 or 9 too many.

This is just an example of how the tiobench can be used. I basically written the tiobench to use the (disk) block size of 4096 and then 8192.
So in case if Linaro how much block size should be used ?

Revision history for this message
Paul Larson (pwlars) wrote :

On Tue, 2010-10-05 at 17:44 +0000, Deepti B. Kalakeri wrote:
> > 37 + "./tiobench.pl --block=4096 --block=8192 --threads=10 "\
> > 38 + " --size=%s --numruns=2" % (VERSION, FILE_SIZE)]
> > Is there a reason why --block is specified twice?
> > Also, we need to scale this WAAAAAY back. Most of the devices we use are
>
> I did not get with point "we need to scale this WAAAAAY back." Can you elaborate.?
>
> > booted off a small SD card, so space is very limited. Also, 10 threads on a
> > single-core, low power device is about 8 or 9 too many.
>
> This is just an example of how the tiobench can be used. I basically written the tiobench to use the (disk) block size of 4096 and then 8192.
> So in case if Linaro how much block size should be used ?
ok, I see from reading the help that it seems you can specify --block
more than once with multiple runs and it will do both. I was thinking
maybe one of them was accidental. I don't have a problem with the block
size if you feel it is appropriate. As mentioned before, the two things
that seem to be causing problems here are the number of threads, and the
--size. As a point of reference, I have about 200M free on my sd card,
but there's no guarantee that every machine will. I don't know that
there's an easy way to tell it not to try to overconsume space, if not
then we just have to make a reasonable guess. I would not go over 2
threads.

lp:~deeptik/lava-test/tiobench updated
42. By Deepti B. Kalakeri

Addressing review comments to the tiobench IO performance test suite

Revision history for this message
Paul Larson (pwlars) wrote :

I ran this on my beagle last night, and the test seemed to run fine but:
"test_results": [],
Likely something in the regexp, but I haven't had a chance to go through it yet. I'll try to take a look today if I get a chance.

review: Needs Fixing
Revision history for this message
Paul Larson (pwlars) wrote :

Found the problem, you're regexp for the 2nd match line is too strict. Given that there's nothing else I see in the subsequent output that could get a false positive for it, I'd consider just having it match \d+\.\d+\.\d+\S+

2.6.35-1006-linaro-omap 512 4096 2 8.87 20.57% 1.747 1698.33 0.00000 0.00000 43

62 + test_id = ('%s_%s[kernel]_%s[filesize]_%s[blksize]') \
63 + % (test_case_id, kernel, filesize, blks_size)
I don't think I'd include the kernel in the test_case_id. The other parts of it are fine though I think.

64 + results['test_case_id'] = test_id
Let's match to a variable name other than test_case_id earlier on so that we don't have to do this.

Your test_case_id's currently come out looking something like this:
"SequentialReads_2.6.35-1006-linaro-omap[kernel]_512[filesize]_4096[blksize]"
This contains illegal characters, notably the []'s. The test_case_id pattern that it compares against for determining legality is:
"^([a-zA-Z0-9_-]+)(\.[a-zA-Z0-9_-]+)*$"

lp:~deeptik/lava-test/tiobench updated
43. By Deepti B. Kalakeri

Addressing review comments to the tiobench IO performance test suite

Revision history for this message
Deepti B. Kalakeri (deeptik) wrote :

The merge has the following changes:

1) Removed the parsing of the kernel information
2) Changed the name of the group in pattern1 from test_case_id to test_id
3) Removed the [] in the final test_case_id
4) Associated the unit information as part of the final test_case_id to make it more meaningful

Revision history for this message
Paul Larson (pwlars) wrote :

Approved, merging

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'abrek/test_definitions/tiobench.py'
2--- abrek/test_definitions/tiobench.py 1970-01-01 00:00:00 +0000
3+++ abrek/test_definitions/tiobench.py 2010-10-13 18:49:54 +0000
4@@ -0,0 +1,69 @@
5+# Copyright (c) 2010 Linaro
6+#
7+# This program is free software: you can redistribute it and/or modify
8+# it under the terms of the GNU General Public License as published by
9+# the Free Software Foundation, either version 3 of the License, or
10+# (at your option) any later version.
11+#
12+# This program is distributed in the hope that it will be useful,
13+# but WITHOUT ANY WARRANTY; without even the implied warranty of
14+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15+# GNU General Public License for more details.
16+#
17+# You should have received a copy of the GNU General Public License
18+# along with this program. If not, see <http://www.gnu.org/licenses/>.
19+#
20+"""
21+ This script automates the installation, execution, and
22+ results parsing for the tiobench test suite.
23+ tiobench is a multi-threaded I/O benchmark. It is used to measure
24+ file system performance in four basic operations: sequential read,
25+ random read, sequential write, and random write.
26+"""
27+import re
28+import abrek.testdef
29+
30+VERSION="0.3.3"
31+URL="http://prdownloads.sourceforge.net/tiobench/tiobench-%s.tar.gz" %(VERSION)
32+MD5="bf485bf820e693c79e6bd2a38702a128"
33+INSTALLSTEPS = ['tar -zxvf tiobench-%s.tar.gz' % VERSION,
34+ 'cd tiobench-%s && make' % VERSION]
35+RUNSTEPS = ["cd tiobench-%s && "\
36+ "./tiobench.pl --block=4096 --block=8192 --threads=2 "\
37+ "--numruns=2" % (VERSION)]
38+
39+
40+class TIObenchTestParser(abrek.testdef.AbrekTestParser):
41+ def parse(self):
42+ # Pattern to match the test case name
43+ pattern1="(?P<test_id>^(Sequential|Random) (Writes|Reads))"
44+ # Pattern to match the parameter details and measurement
45+ pattern2=".*?(?P<file_size>\d+)\s+(?P<blks_size>\d+)\s+.*? "\
46+ "(?P<measurement>((\d|#)+\.?\d*))"
47+ filename = "testoutput.log"
48+ pat1 = re.compile(pattern1)
49+ pat2 = re.compile(pattern2)
50+ tc_id = None
51+ with open(filename) as fd:
52+ for line in fd:
53+ match1 = pat1.match(line)
54+ match2 = pat2.search(line)
55+ if match1:
56+ tc_id = match1.group('test_id').replace(" ", "")
57+ if match2 and tc_id != None:
58+ results = match2.groupdict()
59+ blks_size = results.pop('blks_size')
60+ filesize = results.pop('file_size')
61+ results['test_case_id'] = ('%s_%sMBfilesize_%sbytesblksize') \
62+ % (tc_id, filesize, blks_size)
63+ self.results['test_results'].append(results)
64+
65+ if self.appendall:
66+ self.appendtoall(self.appendall)
67+
68+tiobench_inst = abrek.testdef.AbrekTestInstaller(INSTALLSTEPS, url=URL, md5=MD5)
69+tiobench_run = abrek.testdef.AbrekTestRunner(RUNSTEPS)
70+parse = TIObenchTestParser(appendall={'units':'MB/s', 'result':'pass'})
71+testobj = abrek.testdef.AbrekTest(testname="tiobench", version=VERSION,
72+ installer=tiobench_inst, runner=tiobench_run,
73+ parser=parse)

Subscribers

People subscribed via source and target branches