| Author |
Message |
|
|
Post subject: [devel] config for performance - user benchmark proposal
 Posted: Jun 07, 2008 - 11:31 PM
|
|
Joined: Jan 10, 2008
Posts: 225
Status: Offline
|
|
Just found at "http://svn.berlios.de/svnroot/repos/fullstory/linux-sidux-2.6/trunk/debian/changelog"
Code:
linux-sidux-2.6 (2.6.25-28) UNRELEASED; urgency=low
* NOT RELEASED YET
...
* several configuration changes:
- switch to performance cpufreq governor.
Stefan, why do you want to switch to performance governor for default binary sidux - kernel? As ever more people using laptops this seems to me a bad idea. I just have read somewhere in a foreign forum that "ondemand" is the most energy efficent governor today. Because i think cpufreq "conservative" is just the same as "ondemand" but with slower ups and downs, that's my governor for a centrino notebook here.
A suggestion for better performance evaluation:
Why not having a performance testing script with some
Quote:
"time startsome>timelog; time compile some>>timelog;cat kernelconfig>>timelog; email toyou"
Would that help you to evaluate the best performance kernel-config? |
Last edited by ralul on Jun 09, 2008 - 04:10 AM; edited 1 time in total
|
| |
|
|
|
 |
|
|
|
Post subject: Re: [devel] slh-kernel config for performance
Posted: Jun 08, 2008 - 12:42 AM
|
|
Joined: Nov 25, 2006
Posts: 2571
Status: Offline
|
|
|
ralul wrote:
Stefan, why do you want to switch to performance governor for default binary sidux - kernel? As ever more people using laptops this seems to me a bad idea.
As of yet, the change in question hasn't reached any "binary" kernel image and is only present in an intermediate development version control system. Furthermore the perception of imposing a battery penalty on notebook users is not entirely correct, as the userspace powersaving governor can (and does) override this default setting, which only gets active in the absence of any userspace daemon (e.g., it does not affect sidux).
I'm stressing the relevance of that configuration change being an intermediate development snapshot, because we (or in this case I am) are chasing a significant and reproducable, but strange and unexpected kernel performance regression, which means svn will get to see a number (albeit for practical reasons much less than the actual number of internal regession tests) of "unconventional" changes (as the more conventional and reasonable have already been tried without improvements) that are not expected to appear in released kernel image, after all svn is supposed to be a development tool and not (necessarily) an enduser representation.
ralul wrote:
A suggestion for better performance evaluation:
Why not having a performance testing script with some
Code:
"time startsome>timelog; time compile some>>timelog;cat kernelconfig>>timelog; email toyou"
Would that help you to evaluate the best performance kernel-config?
So lets take a look at this approach, to be of any use, we'd need before and after benchmark results to quantify the effects, these benchmarks would have to happen under exactly the same circumstances (same system, cold disk caches, close to identical workload (no, watching DVB-X or MP3 at the same time doesn't work), reproducable benchmark settings or even benchmarking something that can be benchmarked in the first place - and soon it gets way to complex for a wider audience, not that I expect most users to want (or be able) to compile kernels or change >4000 configration options in the first place.
But if you have certain configuration changes or patches in mind, don't hesitate to report and discuss them and their intended effects on a individual basis. We are always open to suggestions, as long as they come under a compatible (the upstream kernel is GPLv2) and dfsg-free license and don't impose further risks or disadvantages to our users, additionally the divergence from upstream should be reasonable and well defended, e.g. extensive ACPI patches changing temperature barriers or voltage levels for CPUs have pretty much zero chances for acceptance (unless they're already blessed by upstream ACPI maintainers and already staging in the ACPI git). |
|
|
| |
|
|
|
|
|
|
|
Post subject: RE: Re: [devel] slh-kernel config for performance
Posted: Jun 08, 2008 - 03:10 AM
|
|
Joined: Dec 01, 2006
Posts: 751
Status: Offline
|
|
Who exactly is reporting these performance regressions? As you stated earlier, quantifying consistent and interpretable benchmarks is already difficult and the chance that these reporters spent time to follow some standard (common sense) benchmark guidelines is a little bleak.
But I'm not here to judge anyone's honesty and bathroom practices... |
_________________
"Cool was never cool until the cool guys at Cool industries developed a cool new product: Cool."
|
| |
|
|
|
|
|
|
|
Post subject: RE: Re: [devel] slh-kernel config for performance
Posted: Jun 08, 2008 - 11:30 AM
|
|
Joined: Dec 19, 2006
Posts: 1030
Status: Offline
|
|
| afaik, slh is experiencing a regression on his own system(s). His comparison point is the current Debian kernel, his quick 'n' dirty benchmark is build time of the same package. |
|
|
| |
|
|
|
|
|
|
|
Post subject: Re: [devel] slh-kernel config for performance
Posted: Jun 09, 2008 - 03:47 AM
|
|
Joined: Jan 10, 2008
Posts: 225
Status: Offline
|
|
|
slh wrote:
... and benchmarks are everything but easy to interpret and even after that, it doesn't actually mean that the results could be generalized...
So lets take a look at this approach, to be of any use, we'd need before and after benchmark results to quantify the effects, these benchmarks would have to happen under exactly the same circumstances (same system, cold disk caches, close to identical workload ...
I think of a benchmark of many systems but with themselfes and only if circumstances are the same. For example following script (named bsbenchmark - something like background-small-sidux-benchmark)
Code:
#!/bin/bash
dmesg | fgrep '0.000000] Linux' >/dev/null || ( echo bsbenchmark needs timestamps configured in kernel - $0 failure && exit 0 )
#
workdir=/var/log/bsbenchmark
benchdate=000
benchbefore=000
machine=000
kernelv=$(uname -r)
#
# if not exists workdir exit
test -d $workdir || mkdir $workdir || ( echo $0 needs a workdir $workdir && exit 1 )
#
# make a datetime with ten minute interval for later exit if bsbench started accidently in less than 10 minutes
declare -i bdate=`date +%M`
let bdate=$bdate/10
benchdate=$(date +%Y-%m-%d-%H)$bdate
### for testing now - delete following line later #####
benchdate=$(date +%Y-%m-%d-%H%M%S)
#
# make pc identity string for later compare bench result of same machines over time
machine=$(ifconfig|grep Ethernet |sort |sha1sum - |sed -e's/ -//')
#
cd $workdir || exit 2
#
# get last benchmark for later diff
for i in 20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]-*
do
test $i '>' $benchbefore && benchbefore=$i
done
#
# exit if bsbenchmark has already worked in between 10 minutes - otherwise make benchdir and cd
test -d $benchdate && echo $0 has benchmarked this machine short before && exit 3
mkdir $benchdate && cd $benchdate || exit 4
#
if [ -f /proc/config.gz ]
then
zcat /proc/config.gz|sort -r|sed -e'/^#/d' -e'/^$/d' -e's/CONFIG_//' >>${machine}_${benchdate}_${kernelv}.kconf
elif [ -f /boot/config-${kernelv} ]
then
cat /boot/config-${kernelv}|sort -r|sed -e'/^#/d' -e'/^$/d' -e's/CONFIG_//' >>${machine}_${benchdate}_${kernelv}.kconf
else
echo no kernel config found on this system - $0 failure && exit 5
fi
#
#
dmesg >>${machine}_${benchdate}_${kernelv}.dmesg
ls -l --time-style=+%Y-%m-%d-%H%M /var/lib/dpkg/info/*.list | \
sed -e's#\/var\/lib\/dpkg\/info\/##' -e's/^.* 20//' -e's/.list$//' >${machine}_${benchdate}_${kernelv}.plist
ls -l --time-style=+%Y-%m-%d-%H%M /etc/rcS.d | sed -e's#\/etc\/##' -e's/^.* 20//' >${machine}_${benchdate}_${kernelv}.slist
ls -l --time-style=+%Y-%m-%d-%H%M /etc/rc3.d | sed -e's#\/etc\/##' -e's/^.* 20//' >>${machine}_${benchdate}_${kernelv}.slist
ls -l --time-style=+%Y-%m-%d-%H%M /etc/rc5.d | sed -e's#\/etc\/##' -e's/^.* 20//' >>${machine}_${benchdate}_${kernelv}.slist
if [ $benchbefore '>' 000 ]
then
# this line is not enough - some markers needed - i.e. X has reached login form
diff ../${benchbefore}/${machine}*.dmesg ${machine}_${benchdate}_${kernelv}.dmesg >${machine}_${benchdate}_${kernelv}.dmesg_diff
# to be included some lmbench and cpuinfo for differenciating some architectures.
diff ../${benchbefore}/${machine}*.kconf ${machine}_${benchdate}_${kernelv}.kconf >${machine}_${benchdate}_${kernelv}.kconf_diff
diff ../${benchbefore}/${machine}*.plist ${machine}_${benchdate}_${kernelv}.plist >${machine}_${benchdate}_${kernelv}.plist_diff
diff ../${benchbefore}/${machine}*.slist ${machine}_${benchdate}_${kernelv}.slist >${machine}_${benchdate}_${kernelv}.slist_diff
fi
cd $workdir && rm -r -f $benchbefore
echo Now $0 would send diffs to sidux.com
# Think of not sending when slist_diff>0 (user has configured starters)
# But packagelist_diff would be interesting - if some upgrades behave strange
# all people would see performance degrading. This way a benchmark
# like this would be benchmarking debian sid as a whole.
could give you, when statistically analyzed from hundreds of people, a strong hint of how a kernel config changes behave for us consumers. From the normal user perspective the most suffer of waiting is starting up the system and the apps.
Besides:
Wouldn't it be a nice service for sidux users to have a curve of their computer how it behaves over the years. They only had to input their machine's sha1-identity number which this scrpit generates from ifconfig.
I just have an idea of real reallife benchmarking: There had to be an extra repository benchmark-packages,
where are some heavy used apps with compiled in markers, they write into syslog like: time firefox is starting / time firefox ready
When data from hundreds of users is analyzed it should be much clearer what developments are really better. |
|
|
| |
|
|
|
|
|
|
|
