[02:59:05] --- dev-zero@jabber.org has left [08:07:48] --- Simon Wilkinson has become available [08:17:43] --- jaltman has left: Disconnected [09:19:58] --- Simon Wilkinson has left [09:41:34] --- Simon Wilkinson has become available [09:42:30] Does anyone understand how DoPartialWrite (in Linux's writepages) gets the correct credentials in order to write to the fileserver, when its run from a different kernel thread (pdflush) ? [09:54:58] --- Simon Wilkinson has left [10:00:11] --- Simon Wilkinson has become available [10:37:55] Well, I don't know how it ever worked. But it would appear that the situation now is that begin_write() or page_mkwrite() should use the page private information to store per-page credentials information. To which I can only say, yuck. [10:39:00] The other option is that we use the credential information that's stashed in struct file when a file is opened. But that's difficult when a file is open by more than one user. [11:03:56] --- jaltman has become available [11:06:13] --- dev-zero@jabber.org has become available [13:52:58] --- Russ has become available [14:15:10] --- Simon Wilkinson has left [14:16:05] does Linux have the paging problem that Windows faces where a memory mapped file can remain memory mapped even after the process closes the file handle? do all paging requests for prefetching, memory mapped fetching and storing come from a system process and never from the process on whose behalf the work is being performed? [14:16:23] --- dev-zero@jabber.org has left [14:16:40] If so, the best we can do is to take a best guess at which creds should be associated with the request based upon recent history. [14:22:18] --- Russ has left: Disconnected [14:22:35] --- Russ has become available [14:50:33] --- dev-zero@jabber.org has become available [15:22:57] --- Russ has left: Disconnected [15:40:46] --- deason has become available [16:27:14] --- Simon Wilkinson has become available [16:27:48] No, and no. [16:28:18] The problem is only with writes to memory mapped pages. All of our other writes are done syncronously, by the process that initiated them. [16:29:40] Writeback (to disk cache) of dirty memory mapped pages is performed by 'pdflush' or 'flush-afs' depending on your kernel version. This doesn't have the user's credentials, which normally isn't a problem. Cache chunks get written back to the fileserver by close(), which again, runs from a process associated with the user. [16:30:03] Actually, I believe the answer to Jeff's first question is "yes". A memory-mapped file remains mapped even after the file descriptor used to set up the mapping is closed. [16:30:15] The file descriptor is closed. [16:30:24] The stuff we care about in the kernel is still active. [16:30:50] So, yes the page remains memory mapped, but we don't actually care - the 'close' has no meaning to us. [16:30:59] Yes; Jeff's first question was whether a file can still be mapped after being closed. [16:31:22] Sure. But the 'close' doesn't make it past the VFS layer. [16:31:25] Hrm. That's a good point, actually. [16:31:37] We don't see a "close" until all the fd's are closed _and_ the last mapping goes away. [16:31:46] Yup. Our close is actually 'release'/ [16:31:59] But that's a digression really. [16:32:31] Our problem is that when the cache fills beyond 2/3rds, we start trying to write chunks back to the fileserver as a byproduct of writepage() [16:32:37] And we don't have any credentials to do so with. [16:32:48] Historically, this wasn't an issue, because this code path just deadlocked. [16:32:56] well, yeah. so it's only really a problem when a dirty page is written back _and_ that results in us doing a premature writeback to the fileserver due to cache pressure. At which point we're screwed. [16:33:01] Yup. [16:33:22] historically, there wasn't much complaining about that deadlock. [16:33:43] And the pain was hidden by the fact that pdflush used to just deadlock at this point. But the newer Linux kernels, coupled with the work Marc has done on bdi support in the CM, means that we no longer deadlock. [16:33:54] The European HPC sites moan about it pretty regularly. [16:33:56] I can think of ways to work around the problem, maybe, but they all sort of suck. [16:34:28] The 'approved' way to fix it is to attach a pointer to the process's cred structure to the page when you dirty it. [16:34:57] Uh, writing to memory dirties it. I don't think we get involved there. [16:35:04] --- stevenjenkins has left [16:35:10] page_mkwrite gets called. We can hook that. [16:35:14] --- deason has left [16:35:23] --- abo has left [16:35:38] --- abo has become available [16:36:00] David Howells wrote up how to do this when he developed the new Linux 'cred' stuff. [16:36:09] --- deason has become available [16:37:06] Alternatively, we need to stash credentials (either our 'cred' structure, or the Linux one), in the vnode, probably at file open. [16:38:35] --- stevenjenkins has become available [16:38:50] I was thinking of stashing a uid (PAG) in the vnode at open. [16:39:16] We already do similar in FakeClose [16:40:11] You'd have to check your open flags before doing so, and only stash a PAG that can write. [16:40:41] Well, you'd do it on open, if opening for write, after the AccessOK check has already been done. [16:40:51] So, you know it can write. [16:40:53] Whilst this isn't a huge problem at present, I'm trying to figure out how to move write() to using writeback too, when all of this will be a huge issue. [17:02:48] --- edgester has become available [17:03:13] Hi Simon [18:02:13] I'm writing a git hook to not let me commit unless I add a LICENSE line to my commit message. Would anyone else be interested? [18:02:28] Should all commits have a LICENSE line? [18:05:49] i have no idea [18:09:33] Sounds like a policy question. IMHO, all commits should have a LICENSE IPL line. [18:10:53] if all commits should have that line, then the IPL license is implied and the message is implied and not needed [18:11:01] for a new module why would we want to force IPL? [18:11:08] if it's always IPL, why would we bother? [18:11:12] etc... [18:11:38] can you make a non-IPL commit to an IPL-license file? [18:12:15] you can make a less-restrictive but compatible commit to it the file will still be IPL unless all previous contributors agree to relicense [18:12:19] and ibm's not gonna [18:12:44] ok, I thought that was the case [18:13:58] > you can make a less-restrictive but compatible commit to it People have funny ideas about "compatible" that don't entirely work. For example, the IPL requires each Contributor to grant a patent license in certain circumstances. If you take someone else's code that they gave you under the BSD license, and commit it to openafs, _they have not made that grant_ [18:13:59] I always kick myself when I commit a change and it doesn't have a license line [18:14:55] so, BSD and IPL are not compatible? [18:16:24] That question is not phrased so as to be meaningful. To the extent that "compatibility" is a meaningful concept here, it is _not_ a reflexive relationship. [18:17:38] you lost me with "reflexive" [18:19:05] equality is reflexive. If A=B, then B=A. When people talk about "license compatibility", they are often talking about a fictional relationship which is _not_ reflexive. [18:19:20] ah [18:20:10] If I take two pieces of code, A and B, combine them in some way, and want to distribute the result (and for now, let's ignore any other things I might want to do with it, which might also be restricted), I can do so only if I comply with all of the terms of the license of A _and_ with all of the terms of the license of B. [18:20:50] hmmm [18:21:10] For example, the IPL says When the Program is made available in source code form: a) it must be made available under this Agreement; and b) a copy of this Agreement must be included with each copy of the Program. [18:21:24] I would like to see some guidelines on how the project wants to deal with these multiple licenses [18:22:54] That's somewhat viral; it means if I want to give you code I received under that license, I have to give it to you under that license. Not under "compatible terms" or some nonsense; _under that license_, because that's what it says. [18:25:13] Now, the BSD license says * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. [18:25:20] ok, that's been my understanding, but I've read all of this stuff about using multiple licenses in one file [18:26:37] If you are the sole owner of something, or have buy-in from everyone who owns it, you can distribute it under any terms you want, including something like "you may choose X or Y", which is what Perl does. [18:27:28] so ,the man pages that I made from scratch (i copied the bare skeleton) could be BSD licensed [18:27:40] and that's what I claimed [18:29:40] Sure, but you can't just say "BSD licensed"; you need to tell us who owns the copyright, and what the terms are, and you probably want the no-endorsement clause to mention your name, or your employers, and not that of the University of California. [18:31:08] Copyright 2007 Jason Edgecombe This documentation is covered by the BSD License as written in the doc/LICENSE file. This man page was written by Jason Edgecombe for OpenAFS. [18:31:34] And then, once you do that, to distribute it, we need to actually include a copy of that text in OpenAFS. Some licenses, like the GPL, are written such that each person who uses them on new software doesn't have to make modifications to tune t. The BSD license is not like that, which was part of what mde the original 4-clause version so painful. [18:32:43] Yeah, like that. [18:33:13] the doc/LICENSE file looks good to me. [18:34:51] Now, that's annoying, because it means one more license we have to distribute and one more bit of complexity that makes it harder for people to deal with than if we had a single license. Whether you can do that or not is unclear, since someone might reasonably argue that your man pages are derived works of the software they document. IANAL so I don't know if such an argument would hold water. [18:36:13] But it becomes much more painful when someone contributes a large chunk of code that is intended to be integrated into the existing code but says "LICENSE BSD", with or without actual license text. Because we _can't_ just ignore it, apply the changes anyway, and claim the result is simply covered by the IPL -- the BSD license _requires_ that we distribute a copy. [18:39:10] * edgester sighs [18:39:11] In any case, when people talk about "compatibility", they generally mean it is at least _possible_ to comply with the terms of both licenses, as is the case here. The IPL and GPL are not "compatible" in that sense, because they impose conflicting requirements on anyone wishing to distribute the combined work. [18:40:04] I knew IPL and GPL aren't compatible [18:40:25] I do agree that one license would be simpler [18:40:26] The sane thing to do is to pick one license for the entire package, and require that any contributions be made in such a way and under terms that do not require changing that license. [18:42:46] I wrote a git hook that requires that a line containing "LICENSE BSD" "LICENSE IPL" or "LICENSE MIT" exist in the commit message. It could easily be modified to be only IPL. If anyone wants it, let me know [18:46:49] So, I have two takes on this: (1) everything _should_ be submitted as a Contribution to openafs under the IPL, such that it can be incorporated without changing openafs. (2) All a hook needs to do is verify that some assertion of license is present. The decision as to whether to accept a commit may depend on the terms under which it is provided, the existing license on code it modifies, and other factors. [18:49:09] Note that there's an additional mess, in that anything anyone submits to gerrit is automatically redistributed by us, to anyone who wants it, without our having first verified that its terms permit that. I'm not sure of a good way to deal with that, though perhaps it would be sufficient to have people assert at signup time that they grant permission for verbatim redistribution of anything they submit via that interface. [18:51:44] --- Russ has become available [19:10:22] ouch [19:10:28] sheesh [19:46:04] --- jaltman has left: Replaced by new connection [19:46:05] --- jaltman has become available [19:50:43] --- edgester has left [20:15:36] --- stevenjenkins has left [20:15:44] --- stevenjenkins has become available