Jason Gunthorpe was unable to get a loop device working over NFS, and
Matthew Wilcox pointed out that NFS didn't support the bmap() method, and he
wasn't sure it would even be possible to implement it. Jason asked if there
were some alternative to bmap() which might be used, and Alan Cox said,
You could fall back to
read/write in theory. There is an interesting project for you 8)
At first, Jason thought it would be a simple problem, but after running into some implementation difficulties, he posted again with some questions, and Alan gave him some advice.
Meanwhile, Pavel Machek replied to Alan's initial statement about being able
to fall back on read/write, saying, Really? Think
about deadlocks. Both nbd and loop are _evil_. Mount sparse file over
loopback, then write to both filessytems in heavy way to see why. THIS IS
NOT EASY.
Alan replied, Note the use of the word
"interesting". I think it is quite doable. I don't think it is at all
trivial.
Stephen C. Tweedie also replied to Pavel, saying It's not all that hard. In 2.2 we should have
all of the VM deadlocks taken care of. The main remaining problem is the
single request array, and having better per-device request management should
eliminate most of the problems there.
There followed a bit of discussion on various implementation problems.
Dan Melomedman said he'd been looking for the M1541 UDMA patch for 2.2.{5-10} for a very long time, and Andre Hedrick pointed him to http://www.dyer.vanderbilt.edu/server/udma/. EOT.
Grigory Lyahovitsky gave a link to a page in the Linux Kernel Module Programming Guide, that discussed using /proc files for module I/O. Grigory was getting linker errors:
procfs.o: unresolved symbol __put_user_X
procfs.o: unresolved symbol __get_user_X
Augusto Cesar Radtke said, Sometime ago
with a personal thread with Linus, he says to me that the functions put_user
and get_user should never be used, except when you know what really are you
doing, these functions doesn't have a range check and you need to use a
verify_area() before. So use the new 2.2.x copy_from_user and copy_to_user,
examples of utilization can be found at http://www.atnf.csiro.au/~rgooch/linux/docs/porting-to-2.2.html
But Andi Kleen replied:
You must be confusing it with 2.0 {put,get}_user or with 2.2 __get/__put_user. There it was/is true.
2.2 *_user all do full memory verification. What doesn't are the __ variants (__get_user, __put_user etc.). They do exception handling for invalid user pages, but don't check if the destination or source is a kernel address. So only use them when when you checked the range earlier with another *_user or a verify_area.
get_user(bla, ptr) is equivalent (but slightly more efficient) to copy_from_user(&bla, ptr, sizeof(bla)). Similar for put_user/copy_to_user.
Joel N. Weber II said, if I use smbmount to mount a filesystem from an NT server,
the date stamps are completely screwed up. I assume that this is because the
flavor of date stamps that NT uses don't happen to be correctly supported by
my kernel.
Alan Cox replied, Correct. 2.2.10 gets this right, as does 2.2.5 if you build
it without the WIN95 bug fix. This got discussed a fair bit recently and
Tridge submitted stuff to eliminate the current mess and do it right for all
cases
Michael H. Warfield elaborated:
This is a well known problem. It's a configuration error in compiling the kernel. The kernel was compiled with the Win95 Bug Workaround enabled. To use smbfs with Windows 98, Windows NT, or Windows 2000, this option MUST BE DISABLED. Several distros, including RedHat, have been notified of the error and are correcting it in a future release. To fix it, you must recompile your kernel with that option disabled. There is no other workaround.
ITMT, Tridge (Andrew Tridgell of the Samba Team) has revampted smbfs and smbmount to eliminate the compile time option entirely, replacing it with run time detection of broken Windows 95 systems. Expect that in a future kernel release sometime soon.
Lucas Vossberg was trying to raise the 32-groups-per-user limit on his 2.0.33 system. Frank van Maarseveen posted some patches and replied:
We've done that on linux-2.2.10-ac8 by applying two patches (attached), one for the linux kernel and one for glibc-2.0.7-17 (they are almost identical). The main library problem is the function initgroups() which appeared to use a #define for the maximum number of groups. We have only replaced /lib/libc-2.0.7.17 for this. It would be nicer if a separate small system call library existed, offering a user mode sysconf() for the _SC_NGROUPS_MAX parameter (and others). It could even try to calculate it instead of using a #define from the linux include files.
Note that when you are using NFS there is a hardcoded limit in the underlying RPC protocol of 16 groups. The first 16 groups are passed to the server for permission checking, the others are ignored. I've finished a patch for 2.2.10-ac8 which maintains a dynamic group list for RPC, assuming the server will do UNIX style permission checking. If the NFS client is not able to pass all groups then it might as well make a reasonable selection of which groups to pass.
We are happily running linux-2.2.10-ac8 with 256 groups, even on NFS. I'll first finish a similar patch for linux-2.3.9 (10?) before putting it all on the mailing list.
Robert Walsh asked if any methods of profiling the kernel were available,
other than /dev/profile which didn't have enough granularity for his
purposes and didn't provide call-graph stats, etc. Jeff Dike gave a pointer
to his user-mode
kernel page, which describes the project as "a Linux kernel ported to
its own system call interface, so it runs in user mode." He added,
I'm currently updating it to
2.3.8, and I'll bring it current with the latest development kernel when I
get it working with 2.3.8.
Andrea Arcangeli also replied to Robert, saying that he had developed a
__mcount() function to handle kernels compiled with -pg, and added,
You'll continue to use
readprofile and /proc/profile but what you'll get is the real profiling
numbers via mcount.
Andrea gave a URL to the IKD patch, and
concluded, There's the
2.2.x versions only since I had not need of it into 2.3.x (yet :). Anyway
the porting should be trivial playing a bit with rejects.
Dimitris Michailidis also replied to Robert, saying:
Back in May I posted a patch to enable kernel profiling using gprof, which should give you the call graph statistics you want. The patch basically implements mcount() to collect statistics, adds a couple of files to /proc to make the statistics available to the user land, and provides a command, kernprof, to generate gmon.out for consumption by gprof. kernprof, in addition to preparing data for gprof, also does the job of readprofile, with a number of bugs of the latter fixed.
You can get the patch from
http://linuxwww.db.erau.edu/mail_archives/linux-kernel/May_99/2383.htmlor other archives. Since there has been no response to this patch there has been no further development since the initial release, but it probably still applies cleanly against the 2.2.x kernels.
As noted in the initial post, due to bugs in gcc/egcs that cause miscompilation of programs that use -pg and regparms, to use this patch you either need a hacked version of egcs or you need to disable the FASTCALLs in the kernel (the latter is done in the IKD patch).
Let me know if you have any suggestions to improve the patch or need other assistance.
Philip Blundell pointed out that most of the code in
arch/i386/kernel/profile.c in Dimitris' patch was actually not architecture
dependent, and Dimitris replied, True. Actually, the only arch dependent part is the little
bit of assembly to save/restore the registers in mcount(), which is needed
for it to work with regparms,
to which Robert replied, Actually this would be ideal for our
situation because we're developing on the PPC and we'd need make porting the
profiling code as easy as possible. The less we have to do, the better.
Perhaps this could become a part of future kernels? I can see lots of
situations where having a handle on kernel performance would be cool without
having to apply patches to do it each time... :-) Better to just config it
in as needed.
Matthias Riese wanted to use ext2-specific mount options for mounting his root directory. Matthew Wilcox said that if it didn't matter whether the options were specified for the initial read-only mount for the filesystem check, then all that would be necessary would be to specify the options in /etc/fstab and the filesystems would be correctly remounted.
But Matthias pointed to this comment in the code, which was why he hadn't tried Matthew's suggestion:
/* * Alters the mount flags of a mounted file system. Only the mount point * is used as a reference - file system type and the device are ignored. * FS-specific mount options can't be altered by remounting. */
At this point, Stephen C. Tweedie said, That comment appears to be obsolete. We
pass the "data" field all the way from the mount syscall to the filesystem
remount method, and ext2_remount certaily reparses it, so there should be no
problem accepting fs-specific options in a remount.
Matthias reported success, and suggested that that comment be deleted from the source.
Prasong Aroonruviwat discovered FAT is still not working in the latest
development kernels, and in the course of discussion, Alexander Viro said,
One of the reasons why
FAT is badly b0rken is the damn 'text mode'. I.e. automagical translation of
\r\n to \n and back. It is *wrong*. Not to mention the fact that it screws
up on binaries and relies on the list of known extensions (barf) to do its
magic. Or that renaming such files changes their visible contents. Worse
yet, open, rename, read, close, open, read again and there you go - second
read may return something pretty different. IMO such thing has no place in
the kernel...
Other threads had similar stories.
Matthias Riese announced:
I've put some new stuff on our download page at
http://aerobee.informatik.uni-bremen.de/acl_download.htmlincluding:
ACL patch for linux-2.2.10
ACL patch for linux-2.3.10Bugfixes:
- remounting to enable ACL support works now, so also / can be mounted with ACL support
- replaced #include <config/ext2/fs/acl.h> with #include <linux/config.h>
- standard permissions are used correctly (again) if no ACL is associated with an inode
New patch for e2fsprogs, works with 1.14 & 1.15.
Chris Evans reported complete success and suggested getting the patches into the official kernel, and Giovanni Faglioni agreed.
Riley Williams reported that the IOMEGA ZipDrive drivers wouldn't compile in
2.3.10, and asked if this was related to the recent FAT breakage. Tim Waugh
replied, No, it's related to
the parport changes in 2.3.10. See my posting on the 5th of July '2.3.10-pre3:
imm.c' for a patch that makes it compile.
Matthias Welwarsky announced:
a new version of my AX.25 patches is out for download. Check
http://stud.fbi.fh-darmstadt.de/~matze/ax25.htmlor
http://www.afthd.tu-darmstadt.de/~dg2fef/ax25.htmlRelease Notes for ax25-kernel-2.3.10-1
- soundmodem transmit problem should be fixed, however it is still untested.
- Thomas Sailers EPP driver has been resurrected. I've renamed it to baycom_eppflex and made a few changes to make it compile with 2.3.10's parport interface. However I've made no attempt to run it.
- mkiss is gone. use kiss instead and report problems to me.
- changed the format of /proc/net/ax25 to the format used in 2.2.10 and 2.3.10 respectively.
- changed __initfunc() to __init wherever I found it :-)
Two versions of devfs got released this week. Richard Gooch announced:
Hi, all. Version 114 of my devfs patch is now available from: http://www.atnf.csiro.au/~rgooch/linux/kernel-patches.html The devfs FAQ is also available here.
This is against 2.3.10. Highlights of this release:
- Added support for /dev/netlink Thanks to Dennis Hou <smilax@mindmeld.yi.org>
- Return EISDIR rather than EINVAL for read(2) on directories
- Ported to kernel 2.3.10
Several days later, under the Subject: [PATCH] devfs v115 available, he announced:
Hi, all. Version 115 of my devfs patch is now available from: http://www.atnf.csiro.au/~rgooch/linux/kernel-patches.html. The devfs FAQ is also available here.
This is against 2.3.10. Highlights of this release:
- Added support for all remaining character devices Thanks to Dennis Hou <smilax@mindmeld.yi.org>
- Cleaned up netlink support
This time, there were some replies. Tim Waugh reported adding a new character device (/dev/parport0) in 2.3.10; Richard asked for a patch, and Tim sent one.
The Mandrake developer said he was thinking of adding devfs to the Mandrake kernel RPM, and wanted to know if there was any known problem, and Richard said nope.
Alexander Viro said:
Support for CRLF<->LF translation in the kernel must die. Unless somebody has damn good reasons for preserving this bogosity I'm going to exterminate the bloody thing. It is guilty in:
- breaking mmap() semantics.
- breaking lseek() semantics.
- breaking write() semantics.
- being bloody slow, painful and kludgy.
- making FAT support with the new page cache hard.
- confusing the hell of unaware victims^Wusers.
- bringing the list of magical filename extensions into the kernel.
- belonging to userland *and* being already implemented there.
If somebody has really convincing arguments for preserving the sucker - tell. Otherwise it will die.
There was much agreement, and some dissent from folks who liked the translation feature, but no overwhelming arguments against taking it out.
Patrick J. LoPresti noticed a crash during shutdown on his dual PIII/550 system, with aic7890 SCSI and eepro100 network card, running kernel 2.2.10-ac10 SMP compiled with gcc. He posted an oops, and replied to himself 5 hours later, saying:
I may have found the problem. The files I am looking at are all in arch/i386/kernel and have had no relevant updates since 2.2.10.
There appears to be a race condition between irq.c:free_irq() and io_apic.c:do_level_ioapic_IRQ().
The problem is that do_level_ioapic_IRQ() releases irq_controller_lock before passing "action" to handle_IRQ_event(). (See io_apic.c line 1127.)
The close routine of the eepro100 driver, like most drivers (?), calls free_irq(), and since the spinlock is not held during handle_IRQ_event(), there is nothing to stop free_irq() from kfree-ing the action structure *before* it is used by handle_IRQ_event(). (Yes, the driver does disable interrupts before calling free_irq(), but there is still a race condition.)
An hour and a half later he posted a patch that fixed the problem for him, though he admitted it wasn't perfect and should be gone over by the wizards. Johannes Hirche confirmed the initial problem with the same kernel and reported complete success with Patrick's patch. End Of Thread.
The FireWire development intrigue (see
I just want to let people know that IEEE 1394 development is going on rapidly. I have made a patch available at the project's homepage http://eclipt.uni-klu.ac.at/ieee1394 that reflects the current status (Code is from Andreas Bombe, Sebastien Rougeaux and me). That effectively means, my codebase was merged with Andreas', I used my university holidays also for the raw device and Sebastien came quite far in a short time with the OHCI driver.
- New core module
- Texas Instruments PCI-Lynx driver
- Adaptec AIC-5800 driver
- OHCI (Open Host Controller Interface)
- RAW 1394 I/O driver
From this website you can also download a user-space library which allows communication via the raw device.
Note that this patch compiles when applied to 2.2.10. But that's the only thing that's sure here :-)
It's still developer-only!
Johnnie Peters posted a HOWTO, saying, This
document describes how to install and use the Motorola Computer Group PKS
package for Linux, and answers some frequently asked questions.
See the l-k article for the doc.
H.J. Lu announced the Linux kernel NFS daemon 1.4.6; see the l-k article for details and URLs.
Hunting bugs, Steven N. Hirsch mixed and matched some of the various NFS patches that have been floating around, and had this complaint:
I know I've made this appeal before, but here it goes again:
Proper NFS function (i.e. client, server, locking subsystem, etc) is quite important to many of us. Would it be possible to work towards a bit more intercommunication and coordination between those chasing each other about the same sections of code? The situation is confusing the heck out of me (with many years of hacking background) - I can only begin to imagine what newbies and harried sysadmins are going through.
I know that many others share this sentiment. I _do_ appreciate the hard work, and am not trying to negate the considerable efforts you've all made. I'd just like to see some movement towards consolidation of patches.
H.J. Lu replied, The NFS is very
important to us. That is why I spend my time on it. We need a stable NFS
client and server with file locking. I have beeen running NFS tests and
benchmarks for my NFS package. That is how I found those bugs. I have to
have a working V2 NFS before touching V3 stuff. Unfortunately, NFS is not
the only project I am working on. I cannot spend much more time on it before
V2 is solid. It looks like V2 is very close. I will make a V2 NFS server
presentation at LinuxWorld Expo in August. After that, hopefully I can start
working on V3 stuff. Before that, if you want a working kernel with stable
NFS, you can try VA kernel. We have been beating it to death on our
hardwares. It seems to be very solid to us.
Paul Jakma said, Can the
patches from knfsd 1.4.6 *please please please* go into 2.2.11? I spent a
long time cursing linux for it's broken NFS because I didn't know about
these patches. I'm sure there are a lot of other people out there in the
same position. It really should be in the main tree.
Alan Cox replied, They happen
to be very high on my list. Fear not
(at which point Horst von Brand
kicked up his heels with joy)
Chris D. Faulhaber posted the URL
of a patch against 2.2.10, and said, I have added a bit of functionality to
the aha1542 driver. Previously, the aha1542 would not accept command-line
arguments via insmod. My patch gives the module the same functionality at
the LILO arguments (portbase, buson and busoff timings, and dma
speed).
Gerard Roudier complained about a jiffies wrap fix in a driver he maintains (ncr53c8xx), which he felt made things harder to understand. Ingo Molnar disagreed, saying the change was not so obscure, and Gerard said:
People who want to help debugging should also send their patches to the maintainer. They just have to look into the header of the source file. Failing to do so may lead to unnecessary burden. For example:
- I will have to waste time fixing my incremental patch series.
- I have been an idiot each time I claimed that people could apply patches for 2.2.9 to 2.2.10. This just occurred, by the way.
As you know, there is still some FS corruption problem that haven't been cleared up in 2.2.10, based on reports I see at the list. Since drivers are often suspected in that process, I had decided not to send patches except real bug fixes until we will have a real handle on the problem. I still appeared as a complete idiot, each time I have written that.
At the moment, I have been reported that a system has PCI parity problems with 2.2.10-ac10, but works under 2.2.10-ac10 using drivers from 2.2.9. The changes between these 2 versions seem not correlated with the problem, but they add useless confusion.
I have patches for both 2.2.X and 2.3.X (large) that wait for the FS corruption problem to be cleared up. Mixing useless changes does not help track such a serious problem, in my opinion.
Among the diff between 2.2.9 and 2.2.10-ac10 that involve ncr/sym drivers, only the 2 lignes that address Alpha are fixes. Other ones are not fixes at all and so should have been defferred.
Riley Williams posted a patch, and said, Just a small bugfix relating to the top level makefile in
Linux kernels: If one wishes for some reason to redirect the results of
'make modules_install' to a non-standard root, the current setup requires
one to always specify the full path to the target directory, as relative
paths get implemented relative to the wrong directory. The enclosed patch
fixes this.
Riley Williams posted a patch, and said:
Some time back, there was a thread on the subject of BogoMIPS accuracy and resolution, and I commented that I would look into the situation and explore it on the various systems available to me. I forget who I made this comment to, and the relevant emails are no longer to be found here, but here's the results anyway. All relate to ix86 class systems as I have no others available.
- The current calculation yields a result with a resolution of 8 bits, for an accuracy claimed to be slightly better than 1% but which experiment has generally shown to be between 1.3% and 1.5% of the best result. In this context, "best result" is defined as being the result at the highest resolution that remains stable.
- On 386 class systems, the resulting figure remains stable when the resolution is increased to 11 bits, and is stable on 21 of the 23 systems in this class available to me when increased to 12 bits, but increasing it beyond that point only results in random bits being added to the result.
- On 486 class systems, the resulting figure remains rock stable when the resolution is increased to 12 bits, but increasing it beyond that point only results in random bits being added.
- On the Pentium and P2 based systems available to me, the figure remains stable when the resolution is increased to 14 bits in every case, and to 15 bits on one of the Pentium class systems available to me, but additional resoultion just adds random bits to the result.
- On the sole K6-2 based system available to me, the figure remains stable when the resolution is increased to 16 bits, with random bits added beyond that point.
I would therefore suggest that the enclosed patch be applied, which increases the resolution to 12 bits, thus providing the best available accuracy on all classes of ix86 system.
Rik van Riel said:
Over the course of the last year I've become involved with quite a lot of Open Source and documentation projects. So many, in fact, that I'm no longer able to put enough time and dedication into the Linux Memory Management home page.
In order to ensure good kernel documentation in the future, I think it will be best to place the Linux-MM site under the responsibility of a team of people who all have access to the CVS tree where the docs reside.
If you want to share in the responsibility of documenting the Memory Managemen subsystem or if you want to learn about it and are willing to write down your learning experience, please contact me at <riel@nl.linux.org>
As usual, there are two relevant mailing lists:
- linux-mm@kvack.org (for the technical matters)
- kernel-doc@nl.linux.org (documentation forum)
Both of these lists are Majordomo-managed, so the usual routine applies. If you don't know how Majordomo works, just send a message to <Majordomo@nl.linux.org> with the word "help" in the BODY of the message.
Heinz Mauelshagen announced:
I'm happy to announce that Logical Volume Manager 0.7 is available at <http://linux.msede.com/lvm>.
New features include:
- loop devices can be used as physical volumes for testing or evaluation purposes
- correction of block device major in case of existing VGs with pre official one
- support to create an initial ramdisk to have the root filesystem in a logical volume
- support for volume group and logical volume names containing '.'
- /proc/partitions support regardless of Linux version
- larger stripe sizes up to 512k