Kernel Traffic #212 For 6�Apr�2003

OK, thanks. So with 48 disks you've lost five megabytes to blkdev_requests and deadline_drq objects. With 4000 disks, you're toast. That's enough request structures to put 200 gigabytes of memory under I/O ;)

We need to make the request structures dymanically allocated for other reasons (which I cannot immediately remember) but it didn't happen. I guess we have some motivation now.

Here's a patch that makes the request allocation (and io scheduler private data) dynamic, with upper and lower bounds of 4 and 256 respectively. The numbers are a bit random - the 4 will allow us to make progress, but it might be a smidgen too low. Perhaps 8 would be good. 256 is twice as much as before, but that should be alright as long as the io scheduler copes. BLKDEV_MAX_RQ and BLKDEV_MIN_RQ control these two variables.

We loose the old batching functionality, for now. I can resurrect that if needed. It's a rough fit with the mempool, it doesn't _quite_ fit our needs here. I'll probably end up doing a specialised block pool scheme for this.

Hasn't been tested all that much, it boots though :-)

I've created very preliminary patches to add PPC Performance Monitor support to oprofile cvs and linux 2.4.20-ben8:

http://bryanr.org/linux/oprofile/

Sadly, this approach seems very unstable on a 750CX imac (PVR 0008 2214). The box freezes hard and requires a cold reboot after just a few minutes of profiling. As benh hinted in a previous thread, I suspect there's an undocumented erratum for this CPU related to decrementer + pm use. If anyone has contacts in IBM, further info would be helpful to rule out software error and possibly to obtain a workaround...

Assuming lots of PPC perfmon hardware is effectively useless with the decrementer, some solutions are:

1. use the pm irq for all timer-related stuff in Linux, and turn off the decrementer completely. May mean we need to disable idle loop HLTs, hurting thermal dissipation / power consumption--I believe the PMC cycle counters stop incrementing inside power_save_6xx().
2. use the decrementer for profiling. we'd forfeits the perfmon's ability to sample when MSR[EE]=0 (irqs disabled), taking a big chunk out of oprofile's appeal imo.
3. hybrid approach. when not using oprofile, the kernel runs as it does now via the decrementer. when profiling, we switch everything over to the pm system, disabling the power_save stuff. I suspect this is the best approach on broken silicon, though there may be some minor nastiness with swapping between different exception mechanisms for timer_interrupt()

I'm going to try the hybrid approach and will post new patches soon.

My work is commercially sponsored by Ixia and therefore focuses on 2.4 and the IBM PPC750FX/CXe. However, I'm happy to discuss a 2.5 port, support for other chips, an eventual upstream merge, and any other issues related to bringing up oprofile on the PPC.

Well, since BK has made it so trivial for me to merge with Greg, the thing is already integrated into my tree, and as a result the patches should already have been sent out on the patch lists by the robots, and the snapshots will follow shortly as the automation decides to kick in.

In short, give BK the credit it deserves. You get all the information you want, and the fact that you depend on and force yourself to use inferior tools is not the fault of BK.

In other words: _despite_ your luddite ways you actually have more information available to you than you would have had without BK.

So stop whining about BK. Put up or shut up - you get timely non-BK snapshots, and the fact that others see the value of their tools in the things they do for them shouldn't be an issue for _you_.

Stay in the stone age if you wish, but don't expect your stone-age muscle-propellered log car to go as fast as the rocket of the future. And don't complain to us who don't want to expend energy on stuff that shouldn't need it. We've got better tools.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.5/2.5.66/2.5.66-mm1/

• The anticipatory scheduler is in wrapup mode now. It is pretty much in its final form.

• The ext2 locking changes have been significantly redone.

  The per-blockgroup data structures had to go. For a 4TB filesystem we cannot even kmalloc that many pointers, let alone data structures.

  So the per-blockgroup spinlocking has been replaced with hashed spinlocking and the per-blockgroup accounting has been removed. A "per-cpu counter" thing has been invented to amortise the locking cost of the filesystem-wide counters.

• ext3 is now using spinlocking in its block allocator rather than a filesystem-wide semaphore.

  It is stability-tested but I have not yet performance tested this closely. It does appear to have improved the context switch problem (and the file fragmentation problem which the context switch problem causes). But there's a way to go here.

struct umsdos_ioctl has twice dev_t followed by padding. Probably these should become unsigned longs. I'll send a patch later tonight.

Is it used anywhere? That requires detective work. It is used by the utilities udosctl (a useless demo utility), umssync and umssetup. I do not know of any others. No doubt people will tell me what I overlooked. Less conservative people will tell me that umsdos has to be killed entirely.

It is with delight that I write to announce the first release of the port of Software Suspend for 2.4 to 2.5. This version has all the functionality of the 2.4 version beta19-17 - the current development version for 2.4. This includes the following enhancements over the version currently included in the 2.5 kernel:

• HighMem support
• Multiple swap partiton support
• Ability to cancel a suspend by pressing ALT.
• Ability to save what is close a perfect image image of RAM (resulting in a very fast, responsive system on resume - assumes enough swap available to store your full image)
• Extensive debugging capabilities
• Fast and reliable (extensive testing done under 2.4).

There are issues still to be dealt with, but these should not in any way interfere with testing at this stage. They are:

• 2 page flags currently used: to be converted to dynamically allocated bitmaps
• Interaction with S3 support to be tested and worked on (some common files affected, S3 support not tested).
• Code cleanups still to be done in some places.

You can find the patch against 2.5.66 on www.sourceforge.net/projects/swsusp. It's in the swsusp-devel section at the bottom of the list.

I will not take the code which still relies on the page flag bits. I look forward to your dynamic bitmap implementation.

As for the current patch, I was unable to test its actual functionality, which I will get to in a moment.

First, I request that you please name the patches something sensible that won't collide with anything else, like swsusp-2.5.66-<n>.diff, not 'patch-2.5.66-<n>.diff'.

Also, please make it explicitly clear which patch(es) are needed for download. On the sourceforge site, both -01 and -02 are highlighted as current, though they appear to different versions of the same thing.

I'm glad to hear that you have completed the full port, but many people appreciate incremental patches, especially if the cumulative changes touch multiple parts of the kernel. Please consider breaking the one large patch into multiple, easily digestible, chunks.

Would you mind if I removed TIOCTTYGSTRUCT?

I suppose you don't need it any longer, and otherwise could easily add some debugging stuff again when needed. This ioctl exports lots of kernel-internal stuff that userspace has no business looking at. The direct reason I ask is that it also exports a kdev_t, and the meaning of that will change.

This version of modutils is almost identical to 2.4.23. The changes affect architectures that have function descriptors, i.e. ia64, ppc64, hppa, hppa64. It also adds support for combined s390/s390x utilities.

For historical reasons, insmod and depmod treat modules with neither EXPORT_SYMBOL() nor EXPORT_NO_SYMBOLS() as exporting everything. This provides backwards compatibility with 2.0 kernels and some 2.2 modules. No new code should be relying on this behaviour and the feature has been removed in 2.5 kernels. Unfortunately some developers are still relying on this default behaviour, even for new code.

When an architecture has function descriptors and uses EXPORT_SYMBOL() on a function, gcc generates a function descriptor and ksymtab contains the address of that descriptor. Without an explicit EXPORT_SYMBOL(), gcc does not generate a function descriptor and the exported symbol points to the start of the function body. Any attempt to call to that function tries to use the start of the function code as a descriptor and breaks spectacularly.

To prevent this kernel breakage, I am making an incompatible change to modutils. It only affects ia64, ppc64, hppa and hppa64 users, and only if they are relying on the deprecated feature of all symbols being exported.

Users on these architectures must ensure that their modules still resolve and add EXPORT_SYMBOL() where necessary before doing a permanent upgrade to modutils 2.4.25. The simplest way to check is to build (but not install) modutils-2.4.25 then

./depmod/depmod -nae > /dev/null

Any unresolved references that did not occur with modutils 2.4.23 need an explicit EXPORT_SYMBOL(). If this is too much bother, stay on modutils 2.4.23 and risk the kernel breakage.

Other architectures can safely upgrade to 2.4.25 with no change, or they can stay on 2.4.23.

If anybody fancies a janatorial task, configure modutils 2.4.25 with CFLAGS="-O2 -Wall -DHAS_FUNCTION_DESCRIPTORS" ./configure, build it then run ./depmod/depmod -nae > /dev/null. You can do that on any architecture to find kernel modules that still rely on exporting all symbols.

No, I am not going to fudge modutils 2.4 to allow the continued default export of data symbols but not text symbols on architectures with function descriptors. It is too much extra work just to allow the continued use of a deprecated feature that has already been removed in 2.5 kernels.

WHAT

This is a kernel-mode API, and an initial set of drivers for it, that helps Linux 2.4 and 2.5 kernels support intelligent "USB Device" (peripheral) hardware.

The code is ready for more general use by the Linux community, including development of new drivers. It supports network connections over USB "out of the box", using the NetChip 2280 USB 2.0 high speed controller, and is now being used with high speed USB devices running under Linux.

(Note that such an API is on the 2.5 wishlist. This is the first such API that was designed from the ground up to work with the existing host side Linux-USB stack, and to support USB 2.0 high speed devices.)

WHERE

http://kernel.bkbits.net/~david-b/

Temporary web page with info, including:

• gadget.pdf ... kerneldoc, in PDF format
• gadget25-0331.patch ... patch for 2.5
• gadget24-0331.patch ... patch for 2.4

bk://kernel.bkbits.net/david-b/gadget-2.5
bk://kernel.bkbits.net/david-b/gadget-2.4

BitKeeper repositories

HELPING

There are lots of opportunities to write drivers here, both for dozens more USB "class" specifications and, if you want to get down'n'dirty, for USB device controller hardware.

Please discuss this on the [email protected] mailing list, unless/until a new list gets set up.

DETAILS

Since talking about a "USB Device Driver" becomes ambiguous when both sides of the protocol stack can run Linux, Linux-USB developers have chosen new terminology. A "USB Device Driver" is what current Linux kernels have: a Host-side driver. A Device-side driver is instead called a "USB Gadget Driver" ... that's why the new name.

The API is straightforward and thin, just one new header file to shape how "gadget" drivers talk to the underlying controller hardware. There is no "mid-layer" requirement, and all policy for device configuration and management goes above this API. I/O involves just submitting an asynchronous request to the relevant endpoint (like URBs but simpler).

There are currently two USB device controller drivers available implementing that API.

• The "net2280" driver supports the NetChip 2280 controller, which is a PCI device that supports USB 2.0 high speed transfers. (PCI card versions are available for development.)

• There's a "dummy_hcd" which provides partial emulation (bulk and control transfers) of a controller so that you can do some stages of development without real hardware. (Maybe with debug assistance from UML.) 2.5 only for now (got patch?).

  This can emulate three kinds of hardware that are of interest to the Linux community: the net2280 (as above), the sa1100 (found in older PDAs), and the pxa25x (found in newer PDAs).

There are currently two gadget drivers using that API:

• "Gadget Zero" helps development and testing. One configuration sinks or sources data; the other one loops data sent out from the host back in to that host. Almost any USB controller can support this driver. You can start a new driver from this one by "clone and modify" (using new vendor and product IDs).

• There's a CDC Ethernet gadget driver, letting you use USB as a network link. It talks to the standard Linux "CDCEther" (2.4) or "cdc-ether" (2.5) drivers, or corresponding class drivers that are found on other operating systems. (It's analagous to the ARM Linux sa1100 "usb-eth" driver; but more standard.)

  You'll want some recent patches to those drivers, or the latest 2.5 "usbnet" for performance. (18+ Mbyte/sec at high speed, using TTCP in one direction, vs 4+ MByte/sec; no tuning yet.)

Both of those gadget drivers have compile-time configuration support to let them work with net2280, pxa25x, or sa1100 usb drivers. Each controller has slightly different endpoint capabilities; gadget drivers must choose endpoints and configurations accordingly, and there's no point in trying to do that at run time.

The Linux Security Modules project provides a lightweight, general purpose framework for access control. The LSM interface enables developing security policies as loadable kernel modules. See http://lsm.immunix.org for more information.

2.5.66-lsm1 patch released. This is a rebase up to 2.5.66 as well as some interface and module updates. Out of tree projects will want to resync with interface changes.

Full lsm-2.5 patch (LSM + all modules) is available at:
http://lsm.immunix.org/patches/2.5/2.5.66/patch-2.5.66-lsm1.gz

The whole ChangeLog for this release is at:
http://lsm.immunix.org/patches/2.5/2.5.66/ChangeLog-2.5.66-lsm1

The LSM 2.5 BK tree can be pulled from:
bk://lsm.bkbits.net/lsm-2.5

2.5.66-lsm1
 - merge with 2.5.59-66                                 (me)
 - restore file permission hooks to sendfile            (Stephen Smalley)
 - security.h inclusion in network files                (Stephen Smalley)
 - cleanup init[open]_private_file                      (Stephen Smalley)
 - syslog, sysctl cleanups                              (Stephen Smalley)
 - add CONFIG_SECURITY_NETWORK                          (Stephen Smalley)
 - cleanup for newer skb allocation                     (me)
 - SELinux:                                             (Stephen Smalley)
   - labelled network fixes
   - ptrace fixes, drop support for exec_permission_lite
   - minor fixes
   - use kernel SID in reparent_to_init
 - drop task_kmod_set_label hook                        (me)
 - drop explicit exec_permission_lite hook              (me)
 - drop exta call to security_sock_rcv_skb hook         (me)
 - fix setfs[ug]id return values                        (Jakub Jelinek)

I would be glad to hear comments on this new release, available at

http://sourceforge.net/projects/gujin

http://freshmeat.net/projects/gujin

Gujin is a GPL bootloader rewritten from scratch, it can now be installed on a hard disk (in a small FAT 12/16 partition) or on a floppy.

It is still not perfect, but can still:

• boot any kernel of any uncompressed size in an E2FS, E3FS or FAT12/16/32 partition using BIOS, EBIOS _or_ IDE CHS/LBA/LBA48. Libraries for E*FS reading and gzip decompression are rewritten from scratch for size optimisation - no known bug after extensive testing.
• boot Linux in whatever graphic mode supported by VESA (I use usually 1280x1024 24 bpp) staying in real mode. Use of mouse or joystick to select the kernel to boot.
• boot from BIOS startup _or_ starting from DOS.
• recognise most of QWERTY AZERTY and QWERTZ keyboards to change the command line and provide more input -:).
• boot strange setups like this-other-operating-system in extended partitions and automagic management of partition hiding.

There is a lot of stuff to code and test still, for instance the default function to get Linux i386 real-mode parameter can be overloaded by code present at end of the compressed kernel (linux_param_realfct_size in vmlinuz.* shall use a function like the one in linuxparam.c which would be concatenated at the end of linux.kgz) and processor restrictions in the comment field of the GZIPed kernel has also to be tested (get the Gujin loader to complain if the kernel cannot be run on the current processor, using all the CPUID flags).

Also, the late relocation address of the kernel can be modified (because the kernel can now be of any size, it takes DMA-able memory below 16 Mb, precious for the sound buffers, so the default loading address shall be moved to 16 Mbytes and over for up to date systems)

If interrested, you should begin by downloading the install.tgz package because the compiler used is GCC-2.95.3/4 or GCC-3.0.4 due to compiler bugs. Read the doc first !

I am currently writing the ANSI font download (to not use the PC fonts) and will next support more keyboards. My days just have 24 hours and have a limited time to play with Gujin, so I would appreciate some help.

Yet another release of the VM docs and I hope this is the last major update to it. The swapping and page replacement chapters are the two most notable changes with the usual cleanups and embellishments elsewhere. I am tentatively saying it is now fully accurate now though and the final few technical errors should have been shaken out of it. If I'm wrong, feel free to point it out and laugh a bit.

Arguably, what is more important is that I've written a set of acknowledgments where I tried to compile a list of everyone that gave me a hand. Thanks to anyone who sent me technical corrections, grammar corrections and the odd word of encouragement, it is much appreciated. If I missed anyone, send me an indignant email.

I am aiming to leave this pretty much as it is for the next two weeks and if nothing major happens, it'll be rubber stamped, finalised and I'll make the TeX source publicly available (hopefully on http://www.linux-mm.org or somewhere else that doesn't depend on my website existing) and start working on something else

Understanding the Linux Virtual Memory Manager

PDF: http://www.csn.ul.ie/%7Emel/projects/vm/guide/pdf/understand.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/understand/
Text: http://www.csn.ul.ie/%7Emel/projects/vm/guide/text/understand.txt

Code Commentary

PDF: http://www.csn.ul.ie/%7Emel/projects/vm/guide/pdf/code.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/code
Text: http://www.csn.ul.ie/%7Emel/projects/vm/guide/text/code.txt

As usual, comments and feedback welcome

http://sourceforge.net/projects/libivykis/

libivykis is a thin wrapper over various OS'es implementation of I/O readiness notification facilities (such as poll(2), kqueue(2), epoll_create(2)), and is mainly intended for writing portable high-performance network servers.

It has so far been used to implement a streaming video server and proxy servers for various protocols.

This is the first public release.

Hi, I was moving machines around and for the last day and a half or so the CVS trees weren't getting updated. They should be up to date as of now.

Do we need a cvs-updates mailing list which gets notified of updates or do you not care? I'd be interested in knowing how many people are using the CVS trees.

One other comment: there was concern that the incremental update would not be as good as a one pass conversion. So far, the one pass and the incremental results are identical, no exceptions.