Monday, April 23, 2012

BLD-3.4-rc4

 BLD for Linux kernel 3.4-rc4.

 On previous release, I was shouting that about O(1) cpu picking technique at sched_fork() and sched_exec(). But, moronically I've released a version where bld_select_task_rq() wasn't called at sched_exec! Actually, I messed it up. The whole development thing is something, I often make small changes and looks to see, whether it makes any impact,and while doing these sort of things, I forget even the most obvious things...

 Now, some words regarding BLD. As I was saying, BLD is a load distribution technique and it's exactly what it does. It's not a *scheduler* but at phoronix.com I've seen they claimed "Yet another scheduler..." (something like that). NO! Again, it's a load distribution technique, it's a part of a scheduler, it tries to distribute load evenly as if load balancer isn't needed (which, is a second line of defense). But, whether a load balancer (which runs at idle context) will be really needed or not, it's a matter of extensive experimentation. The thing that goes through my mind is, if we can distribute load properly we don't need a load balancer, cause load balancer is something we need if there's any imbalance. BLD tries to make sure that, there's no imbalance on the system and this technique will be applicable to any per CPU runqueue scheduler.

 Now, as stated previously it doesn't depend on scheduler tick, so you can expect longest tickless idle, which is good from power savings point of view.

 So, here is a version against Linux Kernel 3.4-rc4, patch will work also on 3.4-rc3, 3.4-rc2 and could be downloaded from:

http://code.google.com/p/bld/downloads/list

Thanks,
Rakib Mullick.

34 comments:

  1. currently testing on 3.4 kernel, it's also included in the geek-sources where I first saw the patch (https://github.com/init6/init_6/tree/master/sys-kernel/geek-sources)

    thanks for creating it & making it available for 3.4 :)

    ReplyDelete
    Replies
    1. Let me know how it works for you on 3.4 kernel and thanks for letting know that it's been added on geek-source, I never knew that.

      Personally, I test BLD for all the -rc released kernel. Since there wasn't much feedback, so I didn't make it available, but I will, if there some users :)

      Thanks,
      Rakib

      Delete
  2. I'm testing the 3.9-rc4 version since yesterday. Looks promising :)

    ReplyDelete
    Replies
    1. Wow, thanks! Looking forward to get an update!

      Delete
    2. So far, so good. Workload seems to be more balanced with BLD, will keep testing. Could you update BLD patch for kernel 3.10-rc ?

      Delete
    3. It'd be nice if you let me know which sort of workload you've used and how BLD reacts, how was the performance etc. And, I've uploaded BLD patch for kernel version 3.10-rc5, please checkout at code.google.com/p/bld.

      Thanks,
      Rakib.

      Delete
    4. HM.. Daily desktop usage, multimedia converting, gaming, multi-thread (-j8 @ Intel I7) source compilation. Imho BLD works very well, comparable to BFS or even better (BFS is buggy). Do you need me to run any specific benchmark or something?

      Delete
    5. Really good to know that BLD works very well for you. I ran BFS few times, but never tried to compare with other. One thing needs to be clear that, BLD is about load balancing it doesn't deal with task prioritization or task enqueue/dequeue, it uses CFS's scheduling policy but not CFS's load balancing technique. So, IMHO comparing BLD with BFS might not be fair.

      There are few benchmark for measuring scheduler performance like. kernel compilation, hackbench, sysbench oltp, netperf etc. So, if you've interest and time, you can checkout these tools, how they perform against mainline kernel. Along with these benchmarks, it'd be great if you could just observe the power efficiency (if you're using laptop) of BLD.

      Thanks,
      Rakib

      Delete
  3. Hi,

    Your 3.10 patch doesn't work for final kernel version.

    ReplyDelete
    Replies
    1. You mean 3.10-rc7 ? Or some other problem?

      Delete
  4. No -rc, the final version.

    -----------------------------------------
    Kernel/3.10/linux-3.10$ patch -p1 < bld-3.10-rc5.patch
    patching file init/Kconfig
    patching file kernel/sched/bld.h
    patching file kernel/sched/core.c
    Hunk #7 succeeded at 774 (offset 12 lines).
    Hunk #8 succeeded at 782 (offset 12 lines).
    Hunk #9 succeeded at 1258 (offset 12 lines).
    Hunk #10 FAILED at 1408.
    Hunk #11 FAILED at 1432.
    Hunk #12 succeeded at 1468 (offset 13 lines).
    Hunk #13 succeeded at 1733 (offset 13 lines).
    Hunk #14 succeeded at 2666 (offset 13 lines).
    Hunk #15 succeeded at 2766 (offset 13 lines).
    Hunk #16 succeeded at 3015 (offset 13 lines).
    Hunk #17 succeeded at 7063 (offset 13 lines).
    Hunk #18 succeeded at 7111 (offset 13 lines).
    2 out of 18 hunks FAILED -- saving rejects to file kernel/sched/core.c.rej
    patching file kernel/sched/fair.c
    patching file kernel/sched/rt.c
    patching file kernel/sched/sched.h
    ---------------------------------------

    ReplyDelete
    Replies
    1. Hmm. In between rc5 and rc7 release there was some changes in the scheduler code. Please wait, this might be the last rc release before 3.11 stable release, I'll release a patch for the stable release.

      Thanks,
      Rakib

      Delete
  5. There is already 3.10 (not 3.11) stable available for download :) and this stable release is what I am talking about :)

    ReplyDelete
  6. Well, I was a bit late pulling the latest kernel, so missed it. Anyway, I've uploaded BLD-3.10.0 for Linux 3.10.0 stable version. From now on, I'll try to release only for stable kernel versions not for -rc.

    Also don't forget to share your experience of BLD. Thanks for your continuing interest on BLD.

    Thanks,
    Rakib

    ReplyDelete
  7. Still doesn't work

    ------------------------
    /usr/src/linux-3.10$ sudo patch -p1 < BLD-3.10.0.patch
    patching file init/Kconfig
    patching file kernel/sched/bld.h
    patching file kernel/sched/core.c
    Hunk #10 succeeded at 1421 (offset -2 lines).
    Hunk #11 succeeded at 1450 (offset -2 lines).
    Hunk #12 succeeded at 1477 (offset -2 lines).
    Hunk #13 succeeded at 1742 (offset 7 lines).
    Hunk #14 succeeded at 2675 (offset 574 lines).
    Hunk #15 succeeded at 2775 (offset 574 lines).
    Hunk #16 succeeded at 3024 (offset 574 lines).
    Hunk #17 succeeded at 7072 (offset 571 lines).
    Hunk #18 succeeded at 7120 (offset 571 lines).
    patching file kernel/sched/fair.c
    Hunk #1 succeeded at 2993 (offset -46 lines).
    Hunk #2 succeeded at 3415 with fuzz 1 (offset -46 lines).
    Hunk #3 succeeded at 5364 (offset -37 lines).
    Hunk #4 succeeded at 5410 (offset -37 lines).
    Hunk #5 succeeded at 5428 (offset -36 lines).
    Hunk #6 succeeded at 5474 (offset -35 lines).
    Hunk #7 succeeded at 5487 (offset -35 lines).
    Hunk #8 succeeded at 5501 (offset -35 lines).
    Hunk #9 succeeded at 5731 (offset -35 lines).
    Hunk #10 FAILED at 6193.
    Hunk #11 succeeded at 6194 (offset -37 lines).
    Hunk #12 succeeded at 6204 (offset -37 lines).
    1 out of 12 hunks FAILED -- saving rejects to file kernel/sched/fair.c.rej
    patching file kernel/sched/rt.c
    Hunk #1 succeeded at 1238 (offset 70 lines).
    Hunk #2 succeeded at 1296 (offset 70 lines).
    Hunk #3 succeeded at 2068 (offset 90 lines).
    patching file kernel/sched/sched.h
    Hunk #1 succeeded at 521 (offset -3 lines).
    ----------------------

    ReplyDelete
    Replies
    1. Do you use git to pull new version of kernel? Where's the current HEAD points to? On my git tree it points to:

      commit "ddcf6600b133697adbafd96e080818bdc0dfd028"

      Delete
  8. I'm applying your patch against 3.10 mainline/stable, not linux-next.

    ReplyDelete
  9. Anyway... patching against linux-next doesn't work neither.

    ReplyDelete
    Replies
    1. Ok, a new patch for 3.10 is created, check it. Hopefully, this will work properly. And I've removed the previous patch, BLD-3.10.0.

      Thanks,
      Rakib.

      Delete
    2. O, it compiles fine now. I'll test it and let you know about my experience.

      Delete
  10. Unfortunately I'm experiencing huge performance impact on 3.10 with you lastest patch. The system becames very unresponsive under heavy workload like make -j(number_of_cores). Keyboard and mouse input is very laggy and system is unusable. :/

    ReplyDelete
    Replies
    1. You mean while running make, system becomes less responsive? Does it happen with only BLD patch applied? How about without applying patch?

      Delete
  11. I mean, while running make system becomes VERY unresponsive and I'm not experiencing this issue with vanilla kernel. This issue happens also with compiler reniced to 20 (lowest priority), which shouldn't happen at all.

    ReplyDelete
    Replies
    1. Why this shouldn't happen at all? BLD's primary concern is to reduce imbalance and improve performance by reducing latencies. BLD don't deal with any sort of process prioritization or task enqueue/dequeue technique but placing tasks properly. But, yes if it somehow screws vanilla kernel, then it needs to be addressed too.

      So, it'd be nice if you please sent me the config you're using (over email?), system you're using? Will try to reproduce on my system, otherwise won't be able to address the issue i guess.

      Delete
  12. My system is 3.10.1 SMP PREEMPT x86_64 GNU/Linux (Intel I7 3630QM / 16GB RAM / SSD) and my config is here http://paste.kde.org/pfe6c3730/

    ReplyDelete
    Replies
    1. The config you've sent it was for 3.10. I've ran the config, with both bld enable and disable. It's true that on that particular config system's interactivity gets lower, particularly when running make and trying do something else, but with vanilla kernel I've also seen the problem, yes vanilla kernel works slight better, not as much as with bld (my system config was different than yours, I ran it over core i3, 4G raml). And, now I'm trying to figure out the problem. Will let know when gets fixed or any improvement, if I can. Thanks!

      Delete
  13. Replies
    1. Not yet. The issue is very subtle. Would've been nice if it could be found exactly from which version this starts to trigger. One of the weird thing about that config is, when i use run top it shows gnome-terminal consumes 60% - 70% of cpu, which usually doesn't happen with other configs. I think it's an indication of CFS's cpu sharing technique or cfs bandwidth controlling ? Can't tell exactly, but also true for vanilla kernel too.

      Delete
  14. mhm... I don't use gnome and any of my processes consume >1% on idle system.

    ReplyDelete
    Replies
    1. Hmm. That seems like we experiencing different issues :-/ Do you use this config for every releases ? Or you just tried this config only for 3.10 version, and hit the problem ?

      Delete
    2. I use this config for every release with minor changes. 3.11 atm.

      Delete
  15. Just tried your patch against 3.11.2 kernel, but no luck. Launching any cpu intensive process makes whole kde/kwin very laggy and choppy animations.

    ReplyDelete
    Replies
    1. Are you using the previous config you gave? Have you properly configured the kernel with graphics support. That laggy, choppy animation _might be_ due to lack of graphics support.

      Delete
    2. This is not a problem with my configuration. Same config, drivers installed. Vanilla kernel (cfs) works fine, even bfs does.

      Delete