[00:31:30] --- Russ has left: Disconnected [00:53:56] --- Simon Wilkinson has become available [00:54:32] --- Simon Wilkinson has left [01:10:43] --- rdw has become available [01:37:46] --- abo has become available [01:37:53] --- abo has left [01:38:05] --- abo has become available [02:50:30] --- haba has left [03:48:48] --- Jeffrey Altman has left: Replaced by new connection [03:48:49] --- Jeffrey Altman has become available [05:04:31] --- haba has become available [05:47:27] --- haba has left [05:53:52] --- meffie has become available [06:37:33] --- jaltman has left: Disconnected [06:45:14] --- jaltman has become available [06:49:50] --- deason has become available [06:52:21] --- haba has become available [06:54:47] --- reuteras has left [07:56:07] --- haba has left [08:31:32] --- meffie has left [08:34:18] --- kaj has left [08:50:24] --- rdw has left [10:28:31] --- Russ has become available [10:36:16] --- Simon Wilkinson has become available [10:37:38] I will need a pthreaded aklog for rxgk, so it's on my list to convert it. Logically, it shouldn't be that hard. [10:38:15] In other news, I now have an OpenAFS with src/des removed, and using hcrypto for all of its des functions. Which means that we need to have a discussion about symbol exports at some point. [10:46:34] --- Simon Wilkinson has left [11:20:40] Awesome! [11:22:42] symbol exports: what all will be needed for rxgk? rxk5? rxkad? should we export anything that isn't? [11:23:24] I would vote for no. [11:23:42] The general rule of thumb for exports is to export the bare minimum you can get away with, since you can always export more later but unexporting something is a giant PITA. [11:26:30] i'm inclined to agree, but we have people who get all pissy when their antique software doesn't do whatever the new latest greatest thing is. [11:27:01] (in which case the answer would be "tough", but if we reasonably know something will be needed and used it wouldn't be unreasonable to export it [11:27:08] True. [11:27:45] Although in general if they really want a crypto library, they probably should just link with hcrypto directly rather than getting it from AFS. Ideally, I'd like to have any AFS software never call the hcrypto API directly, only use it via higher-level AFS calls. [11:27:58] By "AFS software" I mean "not stuff in our tree, stuff that uses our libs." [11:42:18] Outside the kernel, AFS should not build and use its own hcrypto if I already have it. [11:47:42] > Outside the kernel, AFS should not build and use its own hcrypto if I > already have it. and the one you have works. orrectly and fully. [11:47:48] otherwise, hey, support nightmare! [11:51:59] --- deason has left [11:52:16] --- deason has become available [12:02:20] Well sure, we can/should impose minimum requirements. But if we build our own, it - should not have an soname that could be the same as a non-AFS one, unless it is exactly the same as an upstream distribution - should not be installed as libhcrypto.so if another version exists, or perhaps at all - maybe should have all the symbols renamed, though this could be annoying [12:05:24] i'd call ours libafshcrypto or something, and honestly i think i'd be happy always providing it and having it shim our api on top of whatever natively exists. i'd be curious on your, russ' simon's (and other) takes on that [12:05:35] The OpenAFS 1.4.12 sync into Ubuntu lucid was approved on Saturday and completed this morning. [12:05:53] yay [12:06:22] finally i can avoid snarking about at least one debianish platform for a while [12:07:39] do we take on the burden of maintaining it if we provide our own? [12:08:35] of the shim library? well, yes [12:08:51] i don't think we should ever provide a complete hcrypto, honestly [12:10:51] oh, yes, I misunderstood. [12:12:22] --- meffie has become available [12:16:45] Debian will definitely want us to use the existing hcrypto and not our own for the underlying code implementation so that we don't have to fix security bugs in multiple packages. A shim layer would be fine. [12:17:02] There's a lot of appeal in a shim layer because it lets us replace crypto implementations if we need to, but it does add complexity. [12:17:05] for kernel we'd need our own [12:17:08] Right. [12:17:25] Kernel and userspace have very different issues there. We don't have to worry about exports for the kernel stuff so much either. [12:17:53] If the security flaw affects stuff that the kernel uses, then we have to do a separate security release and there's no way around it. [12:18:08] yup [12:28:29] A shim seems reasonable, if we don't want to commit to always requiring hcrypto. And maybe even if we do, if only to provide a layer of abstraction for code that runs in both the kernel and userspace and will potentially be using different backends in each case. [12:32:51] --- Simon Wilkinson has become available [12:33:36] --- Simon Wilkinson has left [12:34:05] --- Simon Wilkinson has become available [12:35:53] My current plan is to use hcrypto as the API [12:36:07] So libafsrpc gains a dependency on libafshcrypto [12:36:22] libafshcrypto is hcrypto with the public key stuff removed. [12:36:29] You can substitute hcrypto if you have it. [12:38:21] --- Russ has left: Disconnected [12:38:31] --- Russ has become available [12:38:57] > You can substitute hcrypto if you have it. So if hcrypto is available, we link against it. If not, we build and link against libafshcrypto. ? [12:39:05] That's the plan. [12:39:07] --- summatusmentis has left: Lost connection [12:39:24] If you are building with Kerberos, and your Kerberos is Heimdal, then you get hcrypto [12:39:39] If you are building with Globus, then you'll probably get OpenSSL, although I need to investigate that [12:39:49] Otherwise, you'll get our libafshcrypto [12:40:11] This is providing Love doesn't expunge des from hcrypto at some point in the future. [12:40:54] If that happens, we might have to provide our own DES, but use hcrypto for everything else. Which would be annoying. [12:41:53] Well, we're going to have to provide our own rfc3961 library, anyway. I was thinking about bundling it into our crypto library, but that would stop us from using Heimdal's one. [12:41:56] --- summatusmentis has become available [12:43:16] there is a benefit to always using our own hcrypto. it ensures that we always link against the crypto implementation that we have tested against. [12:43:33] Tested? Ha! [12:43:53] and it permits us to push a later version than the Heimdal or Globus on the system [12:44:26] That's very true. There are pros and cons. [12:44:33] for example, if we want a new cipher or prf or something that Love has accepted to the Heimdal repo but has not been packaged by a distro [12:44:39] It's really not the right thing to do from a Linux distribution standpoint, though. You don't want multiple copies of a particular library; it defeats the whole point of having shared libraries that you can fix independent of the applications that use them. [12:45:07] Does Linux package Heimdal hcrypto separately from Heimdal? [12:45:13] No. [12:45:17] Depends on what you mean by separate. [12:45:32] Also, there is no "Linux" [12:45:33] can I install an hcrypto package? [12:45:39] Fedora doesn't have Heimdal packages at all. [12:45:44] would Debian consider such a thing? [12:45:54] Does Heimdal build hcrypto as a separate library? [12:45:59] I was assuming that it did... [12:45:59] Yes [12:46:11] Then yes, it's a separate package on Debian (although for some reason I can't find it right now). [12:46:34] Every shared library Heimdal builds is a separate package on Debian. [12:46:37] To be a good Fedora citizen, we should really use NSS. [12:46:56] Oh, hm, I wonder if the Debian Heimdal packages are built with OpenSSL. [12:46:59] That might be why I don't see it. [12:47:44] I wonder if the IPL and SSL license combination would be acceptable to Debian ... [12:47:48] Ah, yes, they are. [12:47:48] does someone want to fund you to implement Heimdal crypto on top of NSS? If so, I will be more than supportive [12:47:52] Sure, I don't see any reason why not. [12:48:14] The additional requirements of the IPL and the SSL license are compatible so far as I can tell. [12:48:36] Cool. [12:48:47] --- deason has left [12:48:49] So, let me ask my original question in a different way. [12:48:54] Looks like I'd have to ask Brian to build hcrypto instead of using OpenSSL for Debian, which I suspect is a good idea anyway. [12:48:58] At the moment libafsrpc exports des functions. [12:49:06] I'm proposing removing all of those exports. [12:49:09] which is yuck [12:49:13] For Debian's position, see http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles [12:49:15] And making libafsrpc depend on a crypto library. [12:49:15] --- deason has become available [12:49:17] --- abo has left [12:49:42] for windows I will need to implement forwards for the DES functions in afsrpc.dll [12:49:56] I think removing our DES exports is the right thing to do. [12:49:59] --- abo has become available [12:50:47] And nobody is arguing for embedding the crypto into libafsrpc ? [12:51:32] I won't. [12:51:59] No. [12:52:16] Cool. [12:52:41] So, I'll try and figure out what to do with our current des_init and des_random mutexes, and then push this lot to github. [12:52:51] I really don't want any of us non-cryptographers responsible for maintaining a crypto implementation. [12:53:55] (I don't think we want any of this for 1.5, hence stashing it on github, rather than sticking it in gerrit) [12:54:46] please not for 1.6 [12:55:48] --- Simon Wilkinson has left [13:00:03] --- Simon Wilkinson has become available [13:17:00] --- Simon Wilkinson has left [13:18:24] --- Simon Wilkinson has become available [13:27:03] --- Simon Wilkinson has left [13:31:00] --- Simon Wilkinson has become available [13:37:04] --- Simon Wilkinson has left [13:38:26] --- summatusmentis has left [13:38:42] --- summatusmentis has become available [14:17:42] --- mdionne has become available [14:24:11] deason: regarding the ubik problem, can we reduce the window during which the write lock is required? can this be done by creating a three state lock in which there is an intermediate lock state upgradeable-read in which the holder is the only entity that can switch the lock from read to write. This might permit a transaction to begin, push the data out to the peers, but not block the reads until it is time to commit the data change. If the transaction times out, the lock would be dropped without harm and if it is dropped after the "commit" instruction arrives that changes the lock from read to write it is too late. [14:26:51] yes, but that's such a larger change to ubik I think; and doesn't entirely remove the possibility of the slowdown but may be enough [14:27:19] it would seem much harder to roll out to ubik installations, since you need cooperation of the other sites for a new lock type or whatever new message sent out [14:27:41] perhaps something could be done for old sites, though, I haven't thought through.... [14:28:10] the question is what problem are you trying to solve and for what environments? [14:28:43] I'm nervous about breaking the consistency of the data view [14:30:17] the problem is a single site outage or network problem effectively kills the whole cell for up to minutes [14:31:01] but a change to ubik needs some backwards compatibility; taking old sites down and bringing up new ones all at once isn't going to be feasible [14:32:03] certainly not but if you can determine the versions of the other peers then you can agree to a protocol that is compatible with the set [14:32:57] > I'm nervous about breaking the consistency of the data view that's why I'm asking how big of a deal it actually is; if it's possible for a switch to be added to let an administrator decide, or if it's jsut going to blow up if they do that [14:34:01] I won't say it will just blow up. The impact on a particular site is likely to be more subtle [14:35:09] as with much of afs, the assumptions are simply not documented and there isn't a list of behaviors that sites were instructed not to mess with [14:37:31] --- Simon Wilkinson has become available [14:38:07] --- Simon Wilkinson has left [14:38:13] --- Simon Wilkinson has become available [14:56:46] A problem for anyone still around ... [14:57:34] libhcrypto isn't thread safe. In particular, the random state used by the default fortuna RNG needs to be mutex protected if the library is used by more than one thread. [14:58:05] If we fix this in the library, then we can't just drop in the 'real' hcrypto. [14:58:15] Get Love to fix it upstream? [14:58:32] (Sorry to comment and run -- meeting. Back in an hour.) [14:58:46] I'm going to speak to him about it, but he's fixed it in Heimdal by making everywhere that calls RAND_*() take a mutex. [14:59:22] The problem with fixing it in the library is that you have to fix it for every possible threading implementation you might be linked with - which was MIT's problem. [15:00:16] --- deason has left [15:00:31] --- abo has left [15:01:08] --- deason has become available [15:01:20] --- abo has become available [15:04:17] weren't we already talking about a shim layer ontop of hcrypto? [15:04:50] The problem is that we don't really want a shim layer ontop, but a shim layer within. [15:05:11] We really want to just lock around the RAND() functions, rather than having to lock every crypto operation that acquires entropy. [15:06:05] If we end up using the EVP layer (which I would like to, because it means we're OpenSSL compatible, and can make use of hardware encryption engines of the fileservers), we really can't afford to lock every crypto call. [15:06:33] I need to investigate further ... [15:07:13] Does the Heimdal EVP layer implement the OpenSSL functions for allowing the application to register the locking functions to be used for various purposes? [15:08:12] If not, I would submit a patch to Love that adds the ability for the hcrypto caller to override the default locking behavior by specifying an alternate set of lock operations to be used [15:08:18] Not that I have seen, no. I was unaware of the existence of that. That sounds good. [15:09:04] OpenSSL added it when the port to Windows was done because the locking implementations on the two OSes was so great and they didn't want to be responsible [15:09:20] --- abo has left [15:09:24] It sounds like that would be useful for our case, too. [15:09:54] Although, if your RNG implementation does locking internally, you don't actually need it. The rest of hcrypto is thread safe (no global state) bar the prng. [15:09:59] --- abo has become available [15:10:15] I believe Atro's problem is that IBM changed the file format for NetRestrict in 2004 and we don't have the equivalent functionality. [15:10:39] As in, no wildcards for us? [15:10:56] we support wildcards. we don't use a 'M' to indicate that '255' is wild [15:11:01] I was wondering about that M; I don't see any way we use that [15:11:06] yeah, that [15:11:19] it should just skip over it, though, I thought [15:11:32] we use any non-space character preceding the address to indicate that the line should be ignored [15:11:57] at least that is how my quick review of the code parsed it [15:14:05] yeah, now I thought it looks like the line should be logged an error if we get non-whitespace/alnum/dot, but I thought I tried it and didn't see an error message... [15:15:14] but I need to go for a bit, can't quite look at it now [15:22:49] Interestingly enough, Heimdal only locks the crypto state whilst initialising it, not whilst reading bytes from the pool. I currently can't see how that is safe with fortuna. [15:29:23] --- andersk@mit.edu/dr-wily has left [15:29:23] --- andersk@mit.edu/dr-wily has become available [15:29:33] --- andersk@mit.edu/dr-wily has left [15:29:33] --- andersk@mit.edu/dr-wily has become available [15:29:56] --- andersk@mit.edu/dr-wily has left [15:30:00] --- andersk@mit.edu/dr-wily has become available [15:35:48] > you can agree to a protocol that is compatible with the set we're going to need to do this eventually anyway. might as well negotiate a protocol somehow. how we can do it backwards-compatibly, and allow downgrades, is the first question, ignoring what gets stored for the moment [15:44:05] --- Simon Wilkinson has left [17:51:37] --- kula has left [18:01:40] --- meffie has left [18:15:42] --- kula has become available [18:17:08] --- mdionne has left [18:17:48] --- mdionne has become available [18:20:12] --- deason has left [18:29:15] --- deason has become available [18:38:06] --- Russ has left: Disconnected [19:48:09] --- mdionne has left [19:49:26] --- Russ has become available [20:22:46] --- Born Fool has become available [21:35:59] --- Born Fool has left [21:37:38] --- tharidufernando has become available [21:46:48] --- kaj has become available [21:49:52] --- tharidufernando has left [21:50:35] --- tharidufernando has become available [21:54:05] --- tharidufernando has left [21:54:17] --- tharidufernando has become available [21:59:17] --- tharidufernando has left [22:00:26] --- tharidufernando has become available [22:04:24] --- tharidufernando has left [22:04:50] --- tharidufernando has become available [22:08:50] --- tharidufernando has left [22:09:56] --- tharidufernando has become available [22:15:16] --- tharidufernando has left [22:17:40] --- reuteras has become available [22:40:29] --- kaj has left [22:47:16] --- deason has left [23:28:02] --- kaj has become available [23:42:14] --- kaj has left [23:42:17] --- kaj has become available [23:42:28] --- Russ has left: Disconnected