Kernel Traffic #287 For 1 Jan 2005 By Zack Brown Table Of Contents * Standard Format * Text Format * XML Source * Czech Translation * Mailing List Stats For This Week * Threads Covered 1. 9 Nov 2004 - 23 Nov 2004 (44 Linux 2.6.10-rc1-mm4 Released posts) 2. 11 Nov 2004 - 24 Nov 2004 (57 Out-Of-Memory Killer: Hunting For The posts) Proper Layer 3. 14 Nov 2004 - 24 Nov 2004 (87 Linux 2.6.10-rc2 Released posts) 4. 16 Nov 2004 - 22 Nov 2004 (23 Linux 2.6.10-rc2-mm1 Released posts) 5. 16 Nov 2004 - 24 Nov 2004 (18 Linux 2.4.28 Released posts) 6. 17 Nov 2004 - 18 Nov 2004 (9 Status Of OSS Deprecation posts) 7. 17 Nov 2004 - 18 Nov 2004 (9 Xen 2.0 Updates posts) 8. 18 Nov 2004 - 21 Nov 2004 (30 Linux 2.6.10-rc2-mm2 Released posts) 9. 18 Nov 2004 - 19 Nov 2004 (6 Kprobes Wrapper To Define posts) jprobe.entry 10. 18 Nov 2004 (1 LKST 2.2.0 For Linux 2.6.9 Released post) 11. 18 Nov 2004 - 19 Nov 2004 (4 Linux 2.6.9-ac10 Released posts) 12. 18 Nov 2004 - 19 Nov 2004 (6 Intel Thermal Monitor For x86_64 posts) Updated 13. 18 Nov 2004 (2 Possible GPL Violation By Silicon posts) Laboratories 14. 18 Nov 2004 - 19 Nov 2004 (5 New Smaller /sbin/hotplug Tool posts) 15. 18 Nov 2004 - 19 Nov 2004 (6 Documenting ioctls posts) 16. 19 Nov 2004 - 20 Nov 2004 (5 Maintainership Of Orphaned Code posts) 17. 20 Nov 2004 - 23 Nov 2004 (17 Some Discussion Of Compiler posts) Extensions 18. 24 Nov 2004 (1 kernel.org Hardware Troubles post) Mailing List Stats For This Week We looked at 2099 posts in 12934K. There were 438 different contributors. 260 posted more than once. 162 posted last week too. The top posters of the week were: * 89 posts in 755K by Adrian Bunk * 82 posts in 476K by Andrew Morton * 63 posts in 281K by Greg KH * 54 posts in 974K by Roland Dreier * 50 posts in 280K by Linus Torvalds * Full Stats 1. Linux 2.6.10-rc1-mm4 Released 9 Nov 2004 - 23 Nov 2004 (44 posts) Archive Link: "2.6.10-rc1-mm4" Topics: Framebuffer, Kernel Release Announcement People: Andrew Morton Andrew Morton announced Linux 2.6.10-rc1-mm4, saying: ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.10-rc1/ 2.6.10-rc1-mm4/ * The 4-level-pagetable code is hopefully finalised. * Various architecture updates. * Added the new FRV architecture. * v4l updates, fbdev updates. * A process matter: I'm now tracking any regressions since 2.6.9, with the intention that we not release 2.6.10 until they're all fixed. (Where "tracking" means shoving them into a folder called "bugs"). So if anyone is aware of any post-2.6.9 regressions, please make sure that I have a copy of the email. I'm less interested in bugs which existed prior to 2.6.9, as they're presumably minor and as long as we don't actually introduce regressions then we'll make decent progress across kernel releases. 2. Out-Of-Memory Killer: Hunting For The Proper Layer 11 Nov 2004 - 24 Nov 2004 (57 posts) Archive Link: "[PATCH] fix spurious OOM kills" Topics: Big Memory Support, OOM Killer People: Marcelo Tosatti, Andrea Arcangeli, Andrew Morton Marcelo Tosatti said: This is an improved version of OOM-kill-from-kswapd patch. I believe triggering the OOM killer from task reclaim context is broken because the chances that it happens increases as the amount of tasks inside reclaim increases - and that approach ignores efforts being done by kswapd, who is the main entity responsible for freeing pages. There have been a few problems pointed out by others (Andrea, Nick) on the last patch - this one solves them. First, Andrea noted that if progress had been made in the high zone, the OOM killer would not be triggered. Now it conditions the triggering on "DMA+Normal" reclaiming success. Nick noted that the last patch would do wrong on the NUMA case (because of per-node kswapd) - its not a problem now because we will only kill if there is any task not being able to allocate and free pages. The memory allocation fallback to other nodes will prevent that from happening. Another drawback from the last patch was that it disable the "all_unreclaimable" logic which attempts to avoid scanning storms - that way kswapd was able to detect OOM condition. What it does now is to disable the all_unreclaimable logic after 5 seconds that its been set. This is enough time for the system to complete IO of (at least some) pages which have been written-out for reclaiming purposes. After that period (which can be sysctl'ed BTW), it then performs a full scan. In case no progress has been made, and both DMA and normal zones are below the pages_min watermark, the OOM killer is triggered. It looks very reliable in my testing - but I need others to test it as well (Martin and Thomas especially who have good test cases). Andrea Arcangeli felt it was wrong to have kswapd kill anything. He said, "kswapd is an async helper like pdflush and it has no knowledge on the caller (it cannot know if the caller is ok with the memory currently available in the freelists, before triggering the oom)." He went on to say that his own plan was to move the OOM killing code out of vmscan.c, which he said could not know which task was proper to kill, and into page_alloc.c, which could. He said, "vmscan.c can only know if something is still freeable, but if something isn't freeable it doesn't mean that we've to kill anything (for example if a task exited or some dma or normal-zone or highmem memory was released by another task while we were paging waiting for I/O). Every allocation is different and page_alloc.c is the only one who knows what has to be done for every single allocation." Marcelo replied he had asked Andrea three times already whether Andrea had an idea of how exactly the proper method would work, and received no concrete answer each time. Andrea replied that his idea of moving the OOM killer into page_alloc.c was his proper method, but that he wouldn't have time to work on it for at least a week. Marcelo wished him luck. Apparently this means Andrea will get to complete his work before Marcelo decides on any final solution for 2.4. Elsewhere, Chris Ross tested Marcelo's original patch, but found that it would go haywire under certain circumstances. Andrea said he'd get his patch out as soon as possible, but that in the mean time debugging Marcelo's patch would also help clarify other problems in that area of the kernel. The discussion continued, with Andrew Morton and others joining in; Andrea did not post his improvement during that discussion, however. 3. Linux 2.6.10-rc2 Released 14 Nov 2004 - 24 Nov 2004 (87 posts) Archive Link: "Linux 2.6.10-rc2" Topics: Disks: IDE, FS: NTFS, Framebuffer, I2C, Kernel Release Announcement, PCI, Sound: ALSA, USB People: Linus Torvalds Linus Torvalds announced Linux 2.6.10-rc2, saying: Ok, the -rc2 changes are almost as big as the -rc1 changes, and we should now calm down, so I do not want to see anything but bug-fixes until 2.6.10 is released. Otherwise we'll never get there. A lot of driver updates, many of them of the small and trivial kind, others less so. USB, ALSA, fbdev, IDE, i2c, v4l, you name it. With a sprinking of core device model and PCI updates thrown in for good measure. Also, a number of architecture updates: ppc64, m68k, uml, parisc, arm. And md, NTFS, and Documentation updates too. The diffstat shows more of the story - not so much single really big changes, more of a fairly wideranging thing. 4. Linux 2.6.10-rc2-mm1 Released 16 Nov 2004 - 22 Nov 2004 (23 posts) Archive Link: "2.6.10-rc2-mm1" Topics: Kernel Release Announcement People: Andrew Morton, Dave Airlie, Lee Revell Andrew Morton announced Linux 2.6.10-rc2-mm1, with various fixes, but nothing major. Lee Revell noticed that VIA DRM had existed in 2.6.9-mm1, but was now gone. He asked why, and Andrew replied, "Actually I haven't been updating that for a while, because of ghastly conflicts upstream. Then it disappeared altogether due to administrative error. I'll see if I can resurrect it." Dave Airlie also explained, "I asked Andrew to kill it, I wasn't happy with it security wise still, resurrecting it could be messy as the tree isn't converted over to the core/library split, I'll probably pick it back up once Linus merges the current diffs after 2.6.10 is released... VIA DRM still only is useful for 2D HwMC stuff for non-root users, having to make a user run 3d apps as root is probably worse than having an in-secure DRM, so I'm still waiting for the VIA/ unichrome people to see what they can do with it..." Lee replied, "FWIW almost no one uses the DRM for 3D on this card. Many people do use the 2D HwMC stuff though." And Dave said, "I might merge up a DRM with the 3D part disabled, so the 2D stuff works after 2.6.10 comes out, Alan wanted me to merge up all the outstanding drms that have security issues and disable them for non-root but someone pointed out the issues with making people run untrusted things as root to be able to use them at all..." 5. Linux 2.4.28 Released 16 Nov 2004 - 24 Nov 2004 (18 posts) Archive Link: "Linux 2.4.28-rc4" Topics: Power Management: ACPI People: Marcelo Tosatti, Chris Wright, Barry K. Nathan, Ozkan Sezer Marcelo Tosatti announced Linux 2.4.28-rc4, saying: Here goes the fourth release candidate of 2.4.28. A few small problems showed up in time for another -rc. Missing exported symbols in the networking area, an ACPI poweroff bugfix, aic7xxx compile fix with -Werror, a binfmt_elf underflow enhancement, an SCTP fix, and a couple TG3 fixes. This will become v2.4.28 final if nothing _really_ bad shows up. Thanks to everybody who has been contributing to make this happen. Ozkan Sezer pointed out some security fixes that had been recently posted; but Barry K. Nathan said he'd discussed them with Marcelo, and it was too late to include these in 2.4.28; Marcelo confirmed this, saying the patches would make it into 2.4.29-preX. Massimo Cetra asked why this decision had been made, if the security holes could be exploited. It seemed to him the most important thing was to close the holes before releasing 2.4.28 if possible. But Chris Wright pointed out that 2.4.28 had already been published by that time (Marcelo announced it as unchanged from -rc4); and Barry K. Nathan remarked that as far as he'd been able to determine, at the worst, an exploit would crash the program attempting to perpetrate the exploit - not such a terrible calamity. 6. Status Of OSS Deprecation 17 Nov 2004 - 18 Nov 2004 (9 posts) Archive Link: "[patch 2.6.10-rc2] oss: AC97 quirk facility" Topics: Sound: ALSA, Sound: OSS, Sound: i810, Version Control People: John W. Linville, Jeff Garzik, Jan Engelhardt, Lee Revell, Alan Cox, Andrew Morton John W. Linville posted an OSS audio fix, "stolen shamelessly from ALSA" . Andrew Morton asked why not just use ALSA instead of OSS, and Jeff Garzik replied, "Until we actually remove the OSS drivers, it's sorta silly to leave them broken." Jan Engelhardt replied, "It's just as silly to fix something we're removing anyway." Alan Cox pointed out that a lot of folks still used OSS, and Jeff added, "Until it's gone, the current users would prefer not-broken to broken." Jan said it would be better to leave it broken, as an incentive to switch to ALSA; but Jeff said, "i810 audio still locks up in ALSA ATM..." At this point Lee Revell remarked, "Fixed in ALSA CVS on Tuesday. This fix needs to go in 2.6.10." 7. Xen 2.0 Updates 17 Nov 2004 - 18 Nov 2004 (9 posts) Archive Link: "Xen 2.0 VMM patches" Topics: BSD: FreeBSD, BSD: NetBSD People: Ian Pratt, Andi Kleen Ian Pratt said: The Xen team would like to start submitting upstream the patches required to enable Linux to run over the Xen 2.0 Virtual Machine Monitor. For more information about Xen see: http://xen.sf.net There are three main classes of patch required: core : small patches that provide extra hooks for arch-xen arch-xen : large patch to add arch/xen and include/asm-xen xen-drivers : patch to add virtual block, network and console drivers We're proposing to submit the 'core' patches first, as these probably require closest inspection (the others are all new files in new directories, so can't break existing architectures). There is already a significant user base and lively developer community backing the Xen project, so we're confident we will have the resources to maintain arch-xen on Linux 2.6 going forward (as well as Linux 2.4/NetBSD/FreeBSD/etc). Let us know what you think of the patches ;-) Andi Kleen thought it didn't seem quite necessary to have a whole Xen port, paralleling the i386 tree. But Ian said: We've experimented with doing thing the other way round, having xen a sub architecture of i386, but it was _really_ messy. I firmly believe that having a separate arch/xen is the best approach for the moment. In the future, it might make sense to merge arch xen into i386, but to do this cleanly would require significant restructuring of i386. I think that's something we could move toward after everyone's gotten comfortable with having arch xen in the tree. The fact that arch xen is self contained actually makes it easier for us to maintain in some respects. We've been tracking 2.6 releases for some time without too much difficulty. Andi replied: 2.6 has been relatively easy for now (because it was supposed to be a "stable kernel"), but I suspect it'll get worse again over time. e.g. in 2.5 it was really bad for long times. Essentially you will need to commit significant man power to this. Also it's quite hard to always catch all the changes that get done to i386. Overall I think it's a bad idea to have four different x86 like architectures in the tree. Especially since there will be likely more hypervisors over time. i386 and x86-64 make some sense because 64bit is a natural boundary, but extending it elsewhere doesn't scale very well. Ian said, "Over time, I hope we can merge the i386 xen port in with the native architecture, but its going to require significant restructuring of the native architecture to do it cleanly. In the meantime, I think we just have to bite the bullet and maintain a separate architecture. I believe we have resources to do this." 8. Linux 2.6.10-rc2-mm2 Released 18 Nov 2004 - 21 Nov 2004 (30 posts) Archive Link: "2.6.10-rc2-mm2" Topics: Kernel Release Announcement People: Andrew Morton Andrew Morton announced Linux version 2.6.10-rc2-mm2, saying: ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.10-rc2/ 2.6.10-rc2-mm2/ * Lots of small bugfixes. Some against patches in -mm, some against Linus's tree. * There's a patch here which should address the oom-killings which a few people have reported. 9. Kprobes Wrapper To Define jprobe.entry 18 Nov 2004 - 19 Nov 2004 (6 posts) Archive Link: "[PATCH] Kprobes: wrapper to define jprobe.entry" Topics: Executable File Format People: Ananth N. Mavinakayanahalli, Andrew Morton, David S. Miller Ananth N. Mavinakayanahalli said: Here is a patch that adds a wrapper for defining jprobe.entry to make it easy to handle the three dword function descriptors defined by the PowerPC ELF ABI. Current patch against 2.6.10-rc2-mm1 + kprobes patch for ppc64. Changes for adding this wrapper for x86, ppc64 (tested) and x86_64 (untested) below. The earlier method of defining jprobe.entry will continue to work. He posted some pseudocode to use jprobes with this patch, adding, "I am not aware of the semantics for sparc64 for making this change." Andrew Morton said he hadn't seen the kprobes patch for ppc64 yet, and said, "what should I do with this? I'm inclined to drop it until the x86_64 part has been tested and Dave has had a go at the sparc64 version." David S. Miller remarked, "Yes, now that we have kprobe support on 4 platforms, it is important that anyone who changes public parts of this interface do the necessary per-platform fixups necessary to coincide with such changes. I think the person changing the data type should be the one fixing up sparc64 :-)" Ananth also replied to Andrew, saying, "I have now tested the patch succesfully on x86_64 and updated it for sparc64 too (Dave says the change looks good)." Andrew asked how much review and testing Ananth's kprobes-for-ppc64 patch had received so far, and Ananth replied, "The patch was earlier posted on the PPC64 mailing list for comments and Paul had reviewed it. I had to update the patch to take care of the base kprobe changes that were made to accomodate the x86_64 port. The patch is tested on POWER3 (uni) and POWER4 (lpar)." 10. LKST 2.2.0 For Linux 2.6.9 Released 18 Nov 2004 (1 post) Archive Link: "[ANNOUNCE/LKST] LKST 2.2.0 for linux-2.6.9 is released." People: Masami Hiramatsu Masami Hiramatsu said: We are pleased to announce releasing new version of Linux Kernel State Tracer. The Linux Kernel State Tracer(a.k.a. LKST) version 2.2.0 has been released. This version can be applied to linux-2.6.9. And LKST 2.2.0 supports IA32, IA64 and x86-64 platforms. LKST is a tool that supports to analyze of fault and evaluate for kernel. Especially it is usuful for analyzing the unanticipated fault of kernal. The latest version of the LKST newly supports the x86-64 architecture. For more changes, see Changelog-2.2.0.txt