The US Government, in its wisdom, has decreed that crypto hooks aren't allowed to be exported either.

Presumably that's because without the hooks, we foreigners aren't even clever enough to link in the existing crypto we've already smuggled from the US, so that rule will stop us from using crypto just as effectively.

However, I believe you're allowed to export hooks if they're for compression. Apparently we're not clever enough to plug crypto modules into compression hooks either :)

I do not believe in raw IO - even for streaming audio it's just too common for the data to have been available in the cache, and by using raw IO you (for absolutely no good reason) just made the machine do more IO than it should have.

There are very specific cases where the application knows that its dataset is larger than physical memory, but those tend to be limited to quite large problems. And they're getting larger.

But having a better way to decide what to throw out - that I am a strong believer in.

I think it's because mark_dirty_buffer doesn't enforce a limit in the grow of dirty buffers in the system.

Baically the buffer code checks if there are too much dirty buffers only for the data writes (because data writes usually pass through block_write_partial_page or the other equivalent filesystem-helper functions in 2.3.x).

So if your filesystem writes 120Mbyte of metadata before the first data write, then you can fill the buffer cache with 120mbyte of dirty buffers without blocking the application waiting for some write-I/O completation. This unlimited grow of unfreeable memory in cache may lead to the VM subsystem to be not able to recycle the cache in time and so the tasks that needs memory will be killed (as happened in the early 2.2.x).

Well, the ext2 folks actually didn't have any input on that side at all.

It's just that the people who DID have input on the new VFS layer (really just Ingo and me, although some others at least saw what happened) only used ext2 (and in my case NFS - NFS to some degree was actually the first "real" page cache client).

The page cache design was actually done with the notion of "let's not care how filesystems do this now, let's just care about how we _want_ it done", and the new code should be pretty easy to use for new filesystems. It can be a real bitch to integrate old filesystems to, though, although there was a lot of work to make it easier for the common cases.

I've updated my x86 performance-monitoring counters patch, which provides user-space access to the performance-monitoring counters (PMCs) in Intel P5/P6, Cyrix, and WinChip processors. The current version is release 0.5 for kernel 2.3.15.

The current release provides

1. virtual PMCs and TSCs that are saved and resumed automatically as processes are,
2. hardware support for Intel, Cyrix, and IDT WinChip [the AMD K7 may be supported if I can find the appropriate documentation], and
3. a "remote-control" feature which allows "monitor" processes to control and sample the PMCs of other processes. The code should be SMP-safe, although I have not been able to test it on an SMP machine.

(Item (3) is the main change since the previously announced release 0.2 from June.)

Umm, oops. <blush>

TSX-11 has been in <<media res>> most of the summer, since it's been in between a (half-finished) computer upgrade, and I (and the other archive team members) have been consumed with other activities. In my case, it was due to my changing jobs. I've updated the watchdog deamon, and I will be working on processing the rest of the incoming queue this week.

In the meantime, if you don't get an answer sent to the archive e-mail address, please try sending mail directly to me (tytso@mit.edu). I'll make sure it gets dealt with.

I should have the hardware transition finished this month, and things should be better after that. I apologize for the disruption and the inconvenience.

In the meantime, of course, if you choose to change the primary location of the watchdog deamon to another FTP server, I really can't blame you. Again, my apologies.

1. Updated Documentation/Changes.
2. Added pcibios_assign_resource() to arch-dependent code. This function should handle address assignment when a new device (or, more frequently, a new [read: misconfigured] region on an old device) is found.
3. Changed i386 bios32.c to use pcibios_assign_resource() for all allocations.
4. Distinguish between pci_dev->name (only bus/slot/function, used for PCI subsystem initialization messages) and pci_dev->full_name (used for resource management stuff).
5. Added pci_find_capability which is a library function to be used by drivers whenever they need to walk the PCI capability lists.
6. Removed pci_dev->master flag.
7. pci_scan_bus: If the bus already exists, don't attempt to scan it again (yes, there really exist machines with a single bus behind two different bridges [a host bridge and a fake PCI-to-PCI bridge], argh).
8. /proc/pci works again.
9. Modified the /proc/pci output format. I've left out several non-interesting status bits like devsel timing, but I tried to keep as close to the original format as possible not to break programs parsing this file (ugh!).
10. Removed PCI_REGION_* macros.
11. Added fixup for broken S3 cards reporting 32M region sizes instead of 64M.
12. Changed allocation of PCI resources -- we really need to know about address ranges assigned to individual PCI buses to be sure which addresses are free and which are routed to which bus. Each bus now has four pointers to resources from which are all resources of devices on this bus allocated (see pci_find_parent_resource() for a matching algorithm). As usually, everything can be overriden in arch-specific code.
13. Removed the i448BX fixup as it was superseded by the bus resource management changes.
14. resource.c: ioport_resource should have IORESOURCE_IO flag set, iomem_resource likewise.

This is just crap.

You should NOT use bus/slot/fn in the name. For identification purposes, you can just use the numbers in the "struct pci_dev" thing directly - no need to try to continually get them into the name, because they are already available as pure numbers.

Numbers are numbers, and should be of type "int" or similar.

Text is text, and should be of type "char []".

Why do you have this need to mix the two?

For "normal" purposes when we need to report information about the device itself (as in /proc/ioports), we surely should use the full name which contains only vendor and device.

For "special" purposes like error messages printed by PCI init code or by device probing in drivers, it's more important to let the user know the bus and the slot, not the real name of the device. Hence, you can think of pci_dev->name as of name of the _slot_ and pci_dev->full_name as of name of the _device_, but we really need both of them.

the thing I object to is to saving that special string away, when the information exists there as-is. I really don't see the advantage of

        printk("%s", dev->name);

over the much clearer

        printk("%d:%d:%d", BUS(dev), SLOT(dev), FN(dev))

Sure, the latter is slightly longer, but at least it's obvious what it will actually print out - it's clear that now we're printing out the _position_ of the card rather than it's "name".

The PCI saga continues ... there is a small patch against 2.3.16:

1. pci_dev->slot_name defined and used (I hope this variant is not confusing anymore)
2. Use PCI BIOS IRQ routing table (if available) to find all the peer buses. This should be a way more reliable than the peer bridge magic we were using before (and still are using if there is no routing table).

The remaining things I'd like to solve before 2.4 (except for bug fixes, of course :)) :

1. Introduce some mechanism for manual setting of the kernel view of IRQ's, so that people having motherboards with broken BIOSes can fix the things manually.
2. Write a script creating devlist.h from the pci.ids file.
3. Write helper functions for allocation/freeing of PCI resources. As far as I remember, we didn't agree yet whether there should be a single function allocating all the resources (and drivers with special needs not using it) or a function for allocating a single resource (called multiple times by each driver). I prefer the first solution as it can be implemented as a `pci_enable_device()' type function which could do things like waking up powered down devices as well.

I expect this to blow up for a number of people - how certain are you that all BIOSes really do this right? I'd be surprised if there weren't problems with pointers to >64kB segments etc on various architectures, resulting in nonbootable systems.

As far as I know, the information should be in the ACPI tables, and those should be findable without any BIOS calls.

Quite frankly, I'm not willing to start to use more BIOS calls that are likely to be broken on some (unlikely) machines just to avoid problems on other (unlikely) machines. EVERY time we've had a BIOS interface, we've had trouble on some machines. We don't want to go down that path.

A few longstanding, but important bugs have been fixed recently in the NFSv3 patches for linux. I'd therefore like to point out a few of these recent (past 3 weeks) fixes and cleanups:

1. NFS over TCP now works as expected. Previous versions were a tad unreliable, with transfer speeds being a factor 10 or so lower than the UDP (when the thing wasn't hanging 8-(). Now tests indicate a reduction in speed is of the order 20% on a local network, which is more in line with expectations. I'd be very interested in hearing other people's experiences here.

2. Problem of waiting on socket buffer memory fixed. If a socket buffer runs out of write buffer memory (there's only 64k per socket), the code will now sleep (or send off requests to other NFS partitions) until there is enough free memory.

   This problem was the cause of a certain amount of network storms against Solaris machines. The latter prefer 32k wsizes (+ RPC call header), meaning that the socket could only buffer 1 request at a time. This lead to the transmission looping and sending off lots of headerless UDP fragments.

3. Spinlocking for improved SMP safeness. In principle, the kernel lock should be held whenever we're in the RPCIOD code, or the NFS code. There was, however, some indication of corruption of wait queues on SMP-machines. Possibly due to manipulations of the queue in interrupts and/or bottom_halves?

1. A long-standing memory leak involving large reads has been fixed (if the read failed, the allocated pages beyond 4k were sometimes not being freed).
2. Renaming of directories should now be fixed again.

3. New stale inode detection. Is much more relaxed. This means that we avoid races with stale file handles. It should hopefully also work better than the old code against named sockets.

   In addition, the NFSv3 inode allocation code has been rewritten for greater clarity.

4. The updating of 'atime' should hopefully be a bit more consistent. If we've been reading the file from the cache, then the cached value of atime won't get set back again by the next write/getattr/whatever statement that happens to return the server's idea of what atime should be.

The current version of the NFSv3 patches is 0.11.6. It should patch cleanly against stock linux-2.2.10, 2.2.11 and 2.2.12. You may pick up the latest NFSv3 patches at: http://www.fys.uio.no/~trondmy/src/linux-2.2.12-nfsv3.dif.bz2

Please note that in order to mount NFSv3 partitions, you will need a patched version of the 'mount' command. A Redhat 6.0 RPM can be found at http://www.fys.uio.no/~trondmy/src/nfsv3-mount/mount-2.9o-1.2.i386.rpm. The source RPM can be found in the same directory. In addition, there is a patch against stock 'mount-2.9o'.

Finally: please note the existence of a TODO list which should (hopefully) be updated regularly with known bugs/missing features etc. Please find the latter at: http://www.fys.uio.no/~trondmy/src/TODO

Logic and Reason have prevailed: Macmillan has agreed to publish our Kernel Programming Book under the OPL, and I have agreed to use docbook as the authoring process to ease later migration of all sections of the book back into the LDP.

This is not a "module programming" book; Alessandro is much better at that than I could ever hope to be, and that is the way it should be. "Professional Linux Kernel Programming" is a guide for people who need to get under the hood and leverage the freedoms of the GPL in employing Linux for specialized applications. There will be material on driver programming, but this is not the primary focus of this book. The book will also focus on the 2.2/2.3 kernels and while we hope to also include whatever is known about 2.4/2.5, there is very little need to support 2.0/2.1 (Beck & al have already done this admirably)

We are looking at a Dec 31 deadline for all submissions, and we are currently seeking contributing authors for sections including

• the network layer and the network devices
• SMP, IPC and Memory management
• salvaging investments in NT drivers
• kernel programming tools, debugging and 'best practices'
• the sound systems

When we are done, we hope to leave behind a complete opus of papers in the LPD to cover programming issues in the entire kernel. We have a table of contents online at http://members.xoom.com/teledynamics/book and invite your comments.

Anyone interested in participating in this project is invited to contact me directly at garym@canada.com or by phone at 519-422-2723

At risk of turning linux-kernel into a me-too-fest I would very much like to see the knfsd patches go into 2.2.next (and also 2.3.next).

Alan has put in a lot of hard work to move 2.2 towards stability and I, for one, am more than a little disturbed to see that impeded by a few whiners who can't be bothered to upgrade one small support package.

At some stage, the distributors are going to have to bite the bullet and issue kernel updates. There were simply too many bugs fixed between 2.2.5(+bits) and 2.2.13pre for Red Hat (for example) to sit on these kernels until 6.1 (I hope).

So they're left with a choice of:

1. 2.2.something (say 13) as-is. knfsd doesn't work.
2. 2.2.13 + the knfsd patches that shipped in their original kernel. knfsd works, but not as well as it might.
3. 2.2.13 + the new knfsd patches. They'll also have to issue a knfsd update. Everything works as well as is possible with the software currently available.

Option 3 is the only viable solution as far as I'm concerned.

Perhaps if 2.2.13 proves solid, we can leave the old-tool-people there, and push the knfsd patches into 2.2.14. That way, nobody needs to feel left out.

Well Alan, with RAID the upgrade was pretty dangerous - you had to mess around with converting important, critical config files by hand (mdtab -> raidtab) and people feared they could lose their data. At least I did. And if you created a new style RAID partition there was no way to go back to a kernel < 2.2.12.

All of those facts do not apply to the kernel NFS server upgrade. It's just a tools upgrade, there is _no_ chance of data loss at all. If you want to go back to 2.2.12, downgrade the tools, be done.

Another factor which people have to keep in mind is how many people are using RAID heavily in 2.2.x and not using the new RAID stuff.

And one cannot judge this simply by the loudness or number of the people who complain (unhappy people are loud, happy people are mostly silent). My personal judgement says there are more people slaving away at adding the RAID patches to the standard kernel than those who need to go through the RAID upgrade process.

more progress on the OSS/maestro front. Output is feeling much better now. It doesn't leak memory and doesn't jitter when you don't feed it quickly enough. Happiness.

Its still quite sick on some chip/codec pairs, this is annoying me to no end. I'd appreciate if people could send me blindingly specific reports on hardware success/failure. Its expected to work on most simple maestro 1/2 single codec setups, but when you start getting into 2e multi codec docking nuttiness the forecast gets grim.

recording is still hopelessly broken and there are still weird mixer interface barfos, both of which I intend to address next.

evil mmap() hacks may happen eventually also.

This one has been there for a while, and I finally sat down just now to track it down.

What I see is that the generic cdrom driver is passing down packet commands with a buffer length which is negative. I see that some of the cgc command building routines are setting negative buffer lengths on purpose.

This is illegal and is confusing SCSI drivers quite badly, because the scsi command will have SCpnt->request_bufflen < 0 and this will be given to the controller for the DMA request.

I don't know what the intentions were here, but this does need to be fixed somehow so I leave it to Jens to figure out the correct fix.

Actually there are a couple of reasons it is relevant to Linux.

There is already code out there to run UDI drivers on a Linux system. If you look, one of the demonstration systems was a Linux system. Intel did the port.

Intel has also made statements to the effect of reference drivers for it's hardware are most likely going to be UDI. If this happens, you are going to want UDI support just so you can bring up new Intel hardware fast.

With some work on the UDI interface, you should be able to make it so all UDI drivers are RTLinux friendly. Also, you could change the mutex/semaphore/locking code as many times as your little hearts desire without having to rewrite every UDI driver. The UDI interface has all of the mutex and timed waits happening outside the driver, in the UDI support routines.

I would have thought that people would rather spend their time working on improving the operating system rather than rewrite an ethernet or SCSI device driver for the umpteenth time. That is what UDI is intended to allow you to do, focus on the OS and use standard drivers.

I've read the UDI spec in detail. Its about fit for BIOS loaders but thats it. If intel provide source it will be worth porting their drivers to the OS properly. If intel don't provide source we don't care anyway.

You simply cannot express stuff like the linux parport sharing in UDI, its got no equivalence. Out goes parallel devices. You can't portably express the Linux tty layer, out go tty devices. It's too slow for serious networking and it can't properly express our scsi stuff and make good use of it.

So what are you going to do with it. Joysticks ?

As some of you might already know, there are now (at least :-) ) two concurrent approaches to implementing ACPI. I do not think that concurrently developing them is a good idea, and I hope you all agree on this. I will summarize the advantages and disadvantages of these solutions briefly:

The "classic" ACPI4Linux patch: This is a rather huge patchset which already does some things that do not require the AML VM to work. Its greatest strength is that it contains much useful code, its greatest weakness size and complexity. I believe that following this track will need enlarging the kernel by about 300k before we can even dare to activate any real features.

A modular approach: This is a solution that is entirely module-based as opposed to the ACPI4Linux patch which is not modularisable. It should work on 95% of all PCs without patching the kernel, and on 100% with a 2k sized patch to memory management. The current concept leaves the effective work to a userspace daemon, the module itself just registers the interrupts and reaches the events down to userspace. Its greatest strength is simplicity, its greatest weakness the fact that ACPI devices need to be initialized by an ACPI enumerator to work correctly, which would require e.g. the IDE driver to reinitialize the device while the machine is already up and running, a thing I would not like.

Both patches will require greater changes to other drivers, such as IDE and USB later on.

I, personally, would vote for the ACPI4Linux patch because testing is easier if you play with the hardware at a stage where you cannot do much damage, but other people, including Max Berger, who is the "kernel guy" of the project (I'm more into userland code) think different on this issue [Max, Andy, I think it would be good to post something more about the module here].

When you have the kernel lock you can guarantee that accesses to anything else protected by the kernel lock can't happen.

So, generally you:

1. Acquire the kernel lock You can now guarantee that no other access to code/data protected by the kernel lock can happen.
2. Mess with shared lists / page tables / etc
3. Let go of the lock

The big kernel lock used to protect lots of stuff. Ie, acquiring the lock used to stop a whole lot of stuff running.

These days, the big lock is being replaced with more localised locks, to improve the granularity of access. Hence, you can have 1 CPU doing one kernel task, and another CPU doing something unrelated, provided they are protected by different locks.

Actually the kernel lock is different to other spinlocks.

When a task holding the kernel lock blocks, i.e. calls schedule(), the lock is dropped. When the task is run again, the lock is reacquired.

Ordinary spinlocks are different. You must never call schedule() while holding an ordinary spinlock. This means no wait_on_blah() or down() calls in the critical region, and no non-atomic memory allocation. These rules don't apply to semaphores used as mutexes.

I've been trying to debug a high performance fibrechannel HA under Linux. I'm now in a situation where its 50/50 whether the bugs remaining are in the scsi layer or the adapter. The scsi code however is so hard to read its going to be quicker to clean it up and fix bits than to try and debug it further. Is anyone else trying to clean up the scsi mess currently?

I'm not going to write some new scsi layer, sorry someone crazier can do that just to clean up the cruft, including stuff like banging it all through indent and extracting common code into functions, moving long complex conditional code into functions etc.

Unfortunately, it seems difficult to fix it properly:

1. affected files: several places outside VFS access i_sem directly: [at least]
• fs/nfsd/vfs.c
• arch/*/kernel/sys_*.c
• drivers/char/tty_io.c
2. different types of files have completely different synchronzation requirements, so I think flags should be added to VFS which sync calls are required.
• pipes: nothing, even parallel read()/write() calls to the same filp must not block in VFS [the pipe could use O_NONBLOCK] linux-2.2 uses a dirty hack: fs/pipe.c knows that sys_write() called down(i_sem), and pipe.c calls up().
• e.g. raw-io: everything can run parallel _except_ parallel sys_read()/sys_write() calls to the same filp [flip->f_pos!]. No restrictions on parallel sys_pread()/sys_pwrite(). Note that this cannot be implemented in raw.c, because raw.c does not know whether sys_read() or sys_pread() was called.
• normal files: parallel, non-overlapping read/write calls are possible[ overlapping writes violate NfsV2]. Truncate can run in parallel with read()/write() if these calls don't access the area between [old-EOF, new-EOF]. Even Windows 95 allow this level or parallelity for some files, this should be possible under Linux as well.
• everything else: I'd call down() before calling f_ops->read()/write(), I don't know if all character devices can handle parallel read()/write() calls. If they can handle it, then they should set appopriate flags.

I've discovered a bug in the as86 assembler which was causing my attempts to make the ix86 boot code correctly handle kernels larger than 1M to play havoc with my system. Since I've now isolated the effects of the bug, I'm reporting it for those interested therein.

The problem is with as86's handling of the .org directive, and specifically with the case where the directive specifies an address lower than what would be used in the absence of the .org directive. When this occurs, as86 does one of two things, with there being no apparent pattern to its choice of which it does:

• Most of the time, it inserts lots of garbage at the point the directive occurs. I would presume it's trying to insert 4G of garbage, but my partitions aren't that large so it stops when it reaches the "Disk full" condition, producing an "Out of memory" error (yes, that is correct).
• Occasionally, it sets the location to the correct one, but in the process overwrites the first two bytes of the code with 0x5a, 0x7f and thus trashes the resulting code. There is no error message generated in this case.

Since I was tweaking the i386 boot sector when this bug hit me, I was less than amused...