Kernel Traffic #192 For 18�Nov�2002

Ok, I'm the one to blame for that patch.

It was intended to fix some problems where the console subsystem would call fbcon_blank after the fbdev HW was put to suspend, thus crashing the system.

This should really be fixed in the low level fb drivers to ignore blanking when they are asleep though. This is a kind of hack as 2.4 lacks a proper model for ordering power management requests

Marcelo, feel free to delete those 4 lines (but not the whole patch), just the 4 lines in fbcon_blank, I'll make sure the various drivers are made safe.

Along with the new fbdev api I also have rewritten the console layer. The goals are:

1. The idea here was to move alot of the basic functionaly present in alot of low level drivers into the the higher layers thus making the low level drivers smaller and cleaner. A good example is using the /dev/fb interface to resize a VC. That is just plain dumb.
2. The second goal was to seperate out the terminal emulation to allow for a light weight printk. Also the idea was to make the VT console system modular. On embedded devices then we can insmod the VT console system. This is partially done.
3. Multi-desktop systems. Already done this. The current code in BK doesn't support this just yet as I have a few bug to beat out for single headed systems. It will take about one more week to get this ready.

I doubt this code will go into 2.5.X but it is avaiable for anyone to play with it.

bk://linuxconsole.bkbits.net

BTW I will make patches avaiable as soon as 2.5.45 comes out.

Reasons to apply:

• will more than double linux filesystem performance (see http://www.namesys.com/v4/fast_reiser4.html), this was measured for reading and writing the linux source code tree
• applying will allow vm and vfs changes to be tested and benchmarked on what will be the fastest linux fs by a factor of two when 2.6.0 ships
• performs all fs operations as an atomic transaction, so that, for instance, write() and truncate() system calls either happen entirely or not at all
• creates necessary infrastructure for an atomic fs transaction API (not yet included in Reiser4).
• scalable by design to arbitrarily large numbers of CPUs (use per node locks rather than system wide locks)
• eliminates fixed size journal area
• creates a complete plugin based infrastructure. This will allow folding in such features as constraints and inheritance as easily coded plugins. It will make it possible to implement new security attributes as just files with new plugins.
• First installment of an effective competitor to the Microsoft OFS project. No other Linux FS is even trying to provide an alternative to OFS.

We are quite excited over having combined such dramatic performance increases with atomic transactions, even better packing of small files, and a plugin infrastructure. This functionality that has killed performance in other filesystems. (BeFS for instance was forced to abandon important parts of its original vision for performance reasons.)

You once told me that you agreed that filesystems should have until 6 weeks after VM/VFS stabilizes. I regret that I have the need to remind you of that. Reiser4 could not be ready earlier. The changes we need in the core code are all fairly trivial, exporting functions and the like, I'll let you read the details yourself. I hope that my fellow tribesman will look at the wooly mammoth on my shoulders as I come back from the hunt, forgive me for being late for dinner, think a thought for the poor hungry MS tribe, and help me make a roasting spit.;-)

We circulated all of the changes we needed in the core something like two weeks ago, nobody objected, and Andrew Morton actually read through them and ok'd them. Viro and Hellwig of course didn't read them on the first posting, and then waited until today to find something to object to, and complained there wasn't enough time left in today. (Being just as helpful to our integration as with V3....) We will be happy to fix things in the manner the discussion leads to as soon as the discussion resolves, it seems to be still in progress as I write.

Reiser4 is clearly labeled as EXPERIMENTAL with notes that it should only be used by developers, benchmarkers, and testers for now. It passes fsx and dbench, it passes mongo.pl for ump, it crashes for mongo.pl smp. We expect it to be suitable for removal of the EXPERIMENTAL label before 2.6.0 ships (when it is suitable to remove it from the rest of the kernel. ;-) )

I'd like to offer you a seminar on Reiser4 if you have time. I am in the US/bayarea for Halloween and next month. (My kids get to try their first Halloween today. I hope your kids have fun too.)

I won't send you the other Nikita patches emails as I see you are already reading them. Please consider Nikita to be authorized as the official maintainer of Reiser4 for the next month (until my return to Moscow).

I didn't read the reiser papers yet, but from Hans' description it sounds like reiser4 gives all the guarantees ext3 does with ordered writes, _and_ they get good performance.

(In fact, from the description it sounds like it gives _more_ guarantees than even ext3 with ordered writes, in that it gives transactional behaviour for arbitrary writes. Maybe I should read the paper).

Reiser4 uses "wandered logs" that are similar to phase-tree or things that are called "shadows" or "side files" in the data bases world.

Idea is that most blocks with file system data (and meta-data) are accessed by first reading their block number from some other "parent" block (like indirect block in ext2). Now, if block is modified during transaction *and* its parent block is also dirty, one can avoid writing copy of block into the journal by:

• allocating new block number ("wandered block")
• storing modified content in the newly allocated wandered block
• updating parent block to point to the new location

Old block is now unreachable from the parent, and if its block number is stored somewhere in the journal one can use it for recovery.

Reiser4 balanced tree lends itself nicely into this model, of course.

Usual problem with such techniques is that they tend to destroy packing due to frequent relocations. But in reality this can be used exactly for the purpose of improving packing, if allocation of wandered blocks if delayed for sufficiently long time (like until transaction commit).

Ok, our dear master.kernel.org seems to be back from the dead, which means that I can punish it some more and upload stuff to it again.

2.5.46 merges some more stuff (yeah, I still have stuff in my mailbox, but I'm calmer now that I don't forsee any more huge issues), and is pretty much all over the map. Merges with Alan, Al and Andrew, and m68k stuff from Geert. Sysfs from Pat, and ext2/3 updates from Ted.

we would like to announce a significant change in direction for the Enterprise Volume Management System project.

As many of you may know by now, the 2.5 kernel feature freeze has come and gone, and it seems clear that the EVMS kernel driver is not going to be included. With this in mind, we have decided to rework the EVMS user-space administration tools (the Engine) to work with existing drivers currently in the kernel, including (but not necessarily limited to) device mapper and MD.

Why make this change? With EVMS being passed over for inclusion in 2.5, the future of the EVMS kernel driver becomes very uncertain. We could obviously continue working on it and keep it up-to-date as a patch against the latest kernels. Numerous helpful comments and changes were suggested during the review of the code last month on the kernel mailing list. We could spend the time to make many of the desired fixes, including some architectural and interface changes. However, the one issue that has not been addressed at length is EVMS's in-kernel volume discovery mechanism. We believe that even if the other changes are made, this will eventually become an issue at a later time. Moving discovery to user-space is certainly a possibility. However, at that point, it would become difficult to differentiate the EVMS driver from the device mapper driver, since they would be performing very similar tasks.

In addition, there would be no need to maintain duplicate MD kernel code in order to provide compatibility with existing software RAID devices. Obviously this duplication has been a significant issue, but it was an unfortunate necessity in order for MD devices to be discovered within the current EVMS kernel framework. With discovery moving to user-space, the EVMS tools can simply be rewritten to communicate with the existing MD driver in the kernel. This approach allows MD to be used directly, without requiring it to be immediately ported to device mapper. However, if the decision is made in the future to make that port, then the EVMS tools should only become simpler.

We will also emphasize that this change has not been made suddenly or without a great deal of thought. We have been contemplating this possibility since shortly after the Ottawa Linux Symposium in July. However, we continued to develop the EVMS kernel driver because of input from our users. We wanted to go ahead and submit the driver and get the opinion of the full community before making this decision. In the last few weeks it has become clear that the current EVMS approach is not what the kernel community was looking for, so we have spent that time determining the feasibility and consequences of making this switch. We have come up with a good initial plan, and everyone involved now agrees that this is the best course of action.

So how will this switch affect the EVMS users? Ideally, we want the users' experience with EVMS to remain completely unchanged. Based on our current plans, the user interfaces will not have to change at all, since we don't see any major changes to the Engine's external application interface. The plan is to provide the same, single, coherent method for performing all volume management tasks. This change will be almost transparent for most users. The same features, plugins, and capabilities will be supported.

There will, of course, be some minor changes. Specifically, installing EVMS will be slightly different. It will involve different kernel options than you are used to with the current version. In the 2.5 kernel, all of the major components are already present, so little, if any, kernel patching should be necessary. Since device mapper has not yet been included in the main 2.4 kernel, 2.4 users will still require kernel patches. In addition, some functionality still does not exist in any of the available drivers. Specifically, we may provide extra device mapper modules for features like bad block relocation. The installation of the EVMS engine tools, on the other hand, should not change significantly from the current method.

The other major difference will be due to the move to user-space discovery. First of all, why make this switch? The most obvious reason is that the kernel drivers become much simpler, and the only things they need to provide is I/O handling and a method for activating the volumes. While disk partitioning and software RAID still perform discovery in the kernel, the trend seems to be to move these tasks to user-space. It is likely at some point in the future that partitioning and MD will also be moved out of the kernel as well. However, the drawback to making this switch is losing automatic boot-time volume discovery. Activating EVMS volumes will now require a call to a user-space utility, which will need to be added to the system's init scripts in order to activate the volumes on each boot.

In addition, this switch complicates having the root filesystem on an EVMS volume. Currently there is a lot of work being done on adding initramfs to the 2.5 kernel, which will provide a pre-root-fs user-space. This new system should provide a simple method for adding tasks to run during this early user-space, and those who wish to use root-on-EVMS will just need to add the EVMS tools to their initramfs. For 2.4 users, this means using an initial ramdisk (initrd) to provide this same pre-root user-space. Initrd setup is certainly awkward and often distribution- specific. But we will do our best to provide adequate instructions and assistance to those who need help in that situation.

Looking ahead, we *will* continue to *fully* support the 1.2.0 version of EVMS on 2.4 kernels, and possibly release a 1.2.1 version with some recent bug fixes. We will also make a reasonable effort to maintain the current EVMS kernel driver on 2.5. It will not go through any other major changes, but we will try to keep it up-to-date and working with the latest 2.5 releases, until the new EVMS tools are complete. At that point, the 2.5 EVMS driver will be dropped. Also, the new enhancements we have been working on recently, such as clustering and volume move, will only be developed under the new Engine model, and will not be available for the current 1.2.x code base.

So how long will this take? Currently, we are estimating that we can have the user-space volume activation framework working, along with initial support for most of the plugins, by early 2003. Certain features, such as BBR and Snapshotting, may take longer to work out the details of their operation. We will soon open a new CVS tree to hold the new Engine code, leaving the old trees as a repository for bug fixes to the 1.2.x version.

In summary, we feel that this decision is the best way to support our users for the long term. We want to provide EVMS on current and future kernels, and we feel this change provides the best method for achieving that. At the same time, this addresses all of the concerns voiced by the kernel community. If anyone has any questions or concerns about this decision, please email us or the EVMS mailing list at [email protected]. We will be happy to answer any questions or discuss these changes in more detail.

I think that's a very good move for EVMS in the long term. You will be able to provide the users what they want (an easy to user and integrated volume managment solution) without having the pain of maintaining a large base of kernel-level code.

Of course there will be some hassle for you now, like adding DM plugins for higher raid levels, etc.. But in the end I guess it will hope both EVMS and Linux. EVMS by reducing the scope of the project without reducing it's functionality, the Linux by having a modular and leight-weight in kernel volume-managment solution with many eyes looking at it, using it and improving it. I guess you will find a bunch of suboptimal things in DM very soon and I hope you will help the kernel community in fixing it. This of course means also that DM will hopefully get an integral part of the kernel, not an Sistina Project like LVM1.

I (and I guess many other kernel developers interested in storage handling) will look forward to the comments who the current kernel-level storage managment facilities are useable by an unified userland engine handling it, and I guess mthere will be many obvious improvement.

I'm also looking forward to IBM's opensource cluster volumemanagment integration as competitions to Sistina's propritary addons.

I wish you all luck with your new direction and expect me and other kernel developers in that area to help you wherever we can.

This decision (to move the bulk of EVMS code to userland and isolate the changes needed in the kernel) *definitely* means less work in the long run - for EVMS people in the first place.

Userland code is easier to write. There one has full runtime environment and that alone means a lot. There one has no 8Kb limit on the stack size. There one has memory protection. And there code doesn't have to do anything about the changes of kernel internals. It's also easier to debug - for very obvious reasons.

The goal is to provide functionality, not to put it in the kernel - the latter always means harder life. It is the last resort measure ("we have no way to do that in userland with acceptable performance and correctness, damn, time to deal with the kernel side") and finding a way to make do with more compact kernel part (ideally - already maintained by somebody else ;-) is always good news.

And I seriously doubt that work thus far put into EVMS goes down the drain from move to userland - they would have to be absolutely incompetent for that to be the case and I don't see what allows you to accuse them in that.

What that decision does mean is serious one-time effort that makes life easier once it's done. And that had taken real courage - my applause to them (and not only mine, while we are at). What they had done was pretty amazing and my respect to the team that had chosen to do the right thing, had been able to defend that decision and to their management that had allowed that has just gone _way_ up. Bravo, folks. And best luck - seriously.

I respect very few people. These I _do_ respect. A lot.

Sistina has a new website, it took me quite a few clicks to find where they'd hidden stuff:

http://www.sistina.com/products_lvm_download.htm

or

ftp://ftp.sistina.com/pub/LVM2/

I consider LVM2 to be more bug free than LVM1, the only thing holding people back is the fact that they will have to patch the kernel to get dm. Then upgrading from LVM1 is simply a question of building libdevmapper and the LVM2 tools and using them. There will be a new release next week that will a bit more documentation.

I felt like making some coasters tonight so I figured it was a good time to give ide-cd a chance with cdrecord. For the most part, it worked great! I was stunned, in fact, by how smoothly it went after I upgraded my cdtools. I was running SMP + preempt and it wrote smoothly at 12x with < 2% CPU usage the whole time. Buffer capacity never dropped below 98%.

I was almost disappointed at not making any coasters so I decided to stress it a bit. I tried running 'make -j10 bzImage' on the 2.5.46 kernel tree while it was writing and that also worked! Buffer never dropped below 96%. (This machine is 2x Xeon 450, 256 MB RAM, BTW.)

Still without coaster I tried one more thing... 'dd if=/dev/zero of=foo bs=1M' in parallel with another burn. That one did it in. ;) I'm running ext3 and the writeout load totally killed burn, which isn't surprising. I was asking for it, I know. What happened when the cdrecord buffer underran was surprising, though: oops below. Very repeatable. Can supply copious hw details if it helps.

Hard disk is sdc on onboard AIC7xxx. Writer is hdc, the only device on the secondary onboard IDE channel. All other disks (IDE & SCSI) were idle during the test.

Source image for the burn was on sdc3 and a directory on the same partition was also the target for the 'dd'.

url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.46/2.5.46-mm1/

It wasn't clear whether it was useful or desirable to keep these patchsets turning over. But it will be helpful to keep them as a marshalling point for people to see what is queued up, to get some additional testing and stabilisation and for people to sync up against. And also to keep things like shared pagetables and dcache-rcu under test.

2.5.46-mm1 includes various fixes to things, Bill's hugetlb rework, dcache-rcu and shared pagetables.

Also the patches which make the address_space's private_lock and page_lock irq-safe. So Badari can run set_page_dirty() from interrupts...

This document explains some of the new functionality to be found in the 2.5 Linux kernel, some pitfalls you may encounter, and also points out some new features which could really use testing. Note, that "contact [email protected]" below also implies that you should also cc: [email protected].

Latest version of this document can always be found at http://www.codemonkey.org.uk/post-halloween-2.5.txt

Thanks to Andrew Morton, Alan Cox, Alan Willis and others for valuable feedback.

in order to make it easier to keep up with the main Linux tree I've set up a bitkeeper repository with our NUMA scheduler at

bk://numa-ef.bkbits.net/numa-sched

(Web view: http://numa-ef.bkbits.net/) This used to contain my node affine NUMA scheduler, I'll add extra trees when the additional patches for that are tested on top of our NUMA scheduler.

Is it ok for you to have it this way or would you prefer having the core and the initial load balancer separate?

The tree is currently in sync with bk://linux.bkbits.net/linux-2.5 and I'll try to keep so.

The following is an experimental patch adjusts the way we allocate resouces for PCMCIA cards in cardbus devices.

When a PCMCIA driver wishes to claim resources for a card, the original code used to use an in-kernel structured list to find a free area of memory, mostly ignoring the Linux resource subsystem (although it did make use of the resource subsystem, it's racy.)

The following patch changes this. We instead use the PCI resource code to allocate a resource of the requested size, alignment, and offset from the parent bus of the cardbus bridge.

Why do we need the size, alignment and offset?

Some PCMCIA devices have specific requirements for IO addresses; some cards only decode 10 bits, and expect these 10 bits to be one of a selection of addresses. This means that the top 6 bits are ignored, and can be any value. However, the lower 10 bits must be one of a fixed set of values, eg 0x3f8, 0x3e8, 0x2f8, 0x2e8.

Hence, 0x3f8, 0x7f8, 0xbf8 etc are all possible values, which obviously gives you an alignment of (1 << 10) and an offset of 0x3f8.

If there is no entry in the MAINTAINERS file, then it is Linus. He did the Alpha port. The first foray of Linux outside of the Intel architecture was to the Alpha processor. The Alpha system came from the laboratories of the Digital Equipment Corp. An engineer from Digital (now HP) arranged for a loan of an Alpha server to Linus Torvalds for him to begin a port of Linux. This act of beneficence greatly accelerated the migration of Linux to other platforms. Linus is still the MAINTAINER for Alpha to this day. He still has his loaner box from Digital.

There are still a few of us that work for the Alpha Processor Group at HP that work on maintaining the kernel. I have tossed your patch into the directory that holds all kinds of miscellaneous alpha bits for the next time one of us looks at the kernel. Since people that work on the Alpha Architecture are greatly outnumbered by the people that work on other architectures, usually by the time we have a stable working kernel for Alpha, the kernel.org kernels are usually many versions ahead.

Weeeellll.... If you want to go by the "if there is no listing in MAINTAINERS" rule, sure :)

Richard Henderson can be considered the current alphalinux kernel maintainer for 2.4.x and 2.5.x, though he gets help from Ivan Kokshaysky and Jay Estabrook, and a tiny bit of help from me too.

So at the very least, please make sure alpha kernel patches get CC'd to [email protected] and [email protected].

Some very long, and pointless, arguments can be had about which was _first_. Lets talk about something more interesting - which ones are still usable :) I do recall that Motorola Computer Group was very very unhappy that an Alpha from DEC arrived for Linus before their PowerStack. Maybe you can compare who shipped a machine to Linus first, then compare which was powered on and so on.

I believe Linus only re-did the Alpha port. It was first done inside DEC. They should get credit for getting on the Linux bandwagon early on.

The Linux Test Project test suite LTP-20021107.tgz has been released. Visit our website (http://ltp.sourceforge.net) to download the latest version of the testsuite, and for information on test results on pre-releases, release candidates & stable releases of the kernel. There is also a list of test cases that are expected to fail, please find the list at (http://ltp.sourceforge.net/expected-errors.php)

We encourage the community to post results, patches, or new tests on our mailing list, and to use the CVS bug tracking facility to report problems that you might encounter. More details available at our web-site.

Change Log
------------
- Added "setdomainname01", "setdomainname02",     ( Saji Kumar )
  and "setdomainname03" to "syscalls" runtest file
- Added "sethostname01", "sethostname02",         ( Suresh Babu ) 
  and "sethostname03" to "syscalls" runtest file
- Fixed bug introduced in "fsstress.c"     ( Andi Kleen, Andrew Morton )
- Fix "chdir03.c" to remove unintentional \n in   ( Paul Larson )
  the directory name
- Added code to remove the tmp test dir           ( Robbie Williamson )
  in "fcntl11.c"
- fix for "shmctl01.c" to get rid of the shmdt    ( Manfred Spraul )
  failures in "shmctl01"
- Fix for "readdir01" slightly incorrect errno    ( Paul Larson )
  handling
- Back out "readv01", "readv02" changes to        ( Paul Larson )
  expect EINVAL when count==0.  Kernel is going
  to keep the old behaviour.
- Fix for "waitpid02". uses undefined div by      ( Paul Larson )
  0 behaviour
- Revert "writev01.c" back to not expect EINVAL   ( Paul Larson )
  when count==0
- Fix for "mc_commo". Changed a 'ps -ef' command  ( Robbie Williamson )
  to 'ps -ewf' to ensure that a grep finds the
  info it needs.
- Fix in mc_member. Corrected typo causing false  ( Robbie Williamson )
  pass. Found by Li Ge
- Fix in "tcpdump01". Removed erroneous INTERFACE ( Robbie Williamson )
  declaration.
- Fix tools/ltprun to use the new runalltests     ( William Jay Huie )
  semantics

This includes the first noticeable vmstat change. Now you get IO-wait time separate from idle time if you run Linux 2.5.41 or later.

There's a Solaris-compatible pmap as well, lacking only the per-vma resident memory stats that Linux doesn't supply. (hint, hint... though I don't know where they'd fit)

http://procps.sf.net/
http://procps.sf.net/procps-3.1.0.tar.gz

------------- recent changes -------------

procps-3.0.5 --> procps-3.1.0

vmstat displays IO-wait time instead of bogus "w"
can build w/o shared library (set SHARED=0)
when IO-wait hidden, count as idle, not as sys
pmap command added (like Sun has)
do not crash GNU make 3.79
top slightly faster

procps-3.0.4 --> procps-3.0.5

top tolerates super-wide displays
better (?) RPM generation
XConsole and top.desktop removed
old build system removed
code cleanup
pgrep and pkill get "-o" (oldest matching process)
had vmstat "bi" and "bo" output interchanged on 2.5.xx
fix man page tbl directives
top man page cleaned up

As part of the "Better SCM" site (that is still under construction), I wrote a few essays about BitKeeper:

http://better-scm.berlios.de/bk/

I am a former user of BitKeeper and the bkbits.net service who used it for maintaining the code of two of my pet projects. The articles include:

An analysis of the suitability of BitKeeper for Free Software Developers - analyzes the product itself, the support given by BitMover, the BKBits.net service and the license. Contains some tips for new users.

Why a change of the BitKeeper licensing would be a good idea (from BitMover's POV) - a rational analysis why a change in BitMover's strategy can yield them a greater advantage in the long run. (note that I do not recommend completely Open-Sourcing it)

There's also an old and slightly deprecated article entitled "GPLing BK" which gives several advantages that a more liberal BitKeeper license would give BitMover. (it is deprecated because I no longer thing a complete freeing would be a good idea at this point).

url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.46/2.5.46-mm2/

There's some work here to address some of the whacky corner cases which have traditionally knocked the VM over. Some are fixed, some still need work. My latest party trick is to run a 4G highmem machine with 800 megabytes of ZONE_NORMAL under mlock. Can't say that it completely works yet...

Of note in -mm2 is a patch from Chris Mason which teaches reiserfs to use the mpage code for reads - it should show a nice reduction in CPU load under reiserfs reads.

And Jens's rbtree-based insertion code for the request queue. Which means that the queues can be grown a *lot* if people want to play with that. The VM should be able to cope with it fine.

And a new set of block a_ops which never, ever, ever attach buffer_heads to file pagecache. Implemented for ext2 - use `mount -o nobh'.

And several VM tuning and stability tweaks.

I apologize for not being able to get a report out last week; I'm in the middle of a career change (anyone need a system administrator?). I've (mostly) kept up with traffic, but there are a lot of older items I've not been able to close out.

The latest version of this list is kept at: http://members.cox.net/tmolina/kernprobs/status.html

It will remain there until mid-December when I move and lose Cox internet access. I'll be moving to [email protected] which is a dialup with no web page included (They are Linux-friendly though - they don't require special software). I haven't decided yet whether to get a domain and a web-hosting service, although there are some out there which don't cost much for minimal storage.

I'm hoping most of the older items can be closed out or eliminated. I'm sending a bcc to each with a report on the list.