[00:55:50] --- lama has become available [03:45:16] --- Simon Wilkinson has left [03:48:58] --- pod has left [04:51:29] --- pod has become available [05:58:12] --- Simon Wilkinson has become available [06:16:39] --- lama has left [06:50:00] --- lama has become available [07:13:59] --- reuteras has left [07:15:06] --- deason has become available [07:50:11] --- Simon Wilkinson has left [07:50:12] --- Simon Wilkinson has become available [07:52:04] --- Simon Wilkinson has left [07:52:13] --- Simon Wilkinson has become available [07:54:08] --- Simon Wilkinson has left [07:56:05] --- Simon Wilkinson has become available [08:05:31] deason: I'm going to try pushing a patch set that doesn't use XCFLAGS at all for LDFLAGS [08:05:54] okay, tx [08:05:58] That still doesn't answer why MT_CFLAGS contains two copies of XCFLAGS on HPUX, but if things work, I guess we care slightly less about that. [08:06:20] and to answer your question from before, I assume hpux is cleaning up stuff since the git stpes are the same as everywhere else (afaik), but no, I haven't really specifically checked [08:06:48] It's just that I couldn't see any way at all that MT_CFLAGS can include XCFLAGS, yet that's what looks like is happening from the build logs. [08:11:05] --- Simon Wilkinson has left [08:11:31] --- Simon Wilkinson has become available [08:12:33] But, also, is HPUX's CFLAGS line correct for something that is supplied just to the compiler, not to the linker? [08:13:53] the XCFLAGS in osconf.m4? no, I'm pretty sure some of that should be ldflags; I mean, it's specifying libs to link to [08:14:07] Yeah. [08:14:31] That was my feeling, too. But if -ldld should only be used for a shared link line, then I'm very confused. [08:14:37] the -D's should be there, as should the -Wp; I assume that's it [08:15:09] oh and the +DA stuff [08:17:09] --- Simon Wilkinson has left [08:17:23] --- Simon Wilkinson has become available [08:18:06] And where should -ldld go? If it goes into XLDFLAGS, then won't have the same problem that things won't build with it in the linker line? [08:22:43] Actually, it does look like we're only failing because -ldld is included twice, or because of where it is in the command line. There are other link lines that link with static libraries that include -ldld and work fine. [08:26:21] i would (and did) guess it was because -ldld is first, but i am unsure [08:26:55] I thought it was because -ldld is after -Wl,-a,archive [08:26:57] Actually, what was submitted as 4171 doesn't look like what I intended to be there. [08:28:12] the idea behind the flags, I believe, was to include "system library" stuff in *FLAGS first, and then in *FLAGS, the last thing they say is -Wl,-a,archive, so the "AFS library" stuff is all done statically [08:28:28] since, all of our libraries used to be static only [08:28:41] that's just my impression from looking at the flags, though [08:29:43] Yeah, I've ballsed up a rebase. [08:30:18] There's a load of stuff in 4172 that should have been in 4171 instead. [08:30:21] well, we should add a macro that is that, then. but yes, that does sound familiar now that i think about it [08:30:54] I'm going to get the HPUX builder to build 4172, and if that fixes it, I'll split out that patch into its correct parts. [08:31:27] is there a problem with just -Wl,-a,shared_archive? I don't know if I've just tried doing that [08:34:28] /whispers libtool [08:36:58] It looks like the MT_CFLAGS change breaks Irix, though. I just need to wait an hour for the stdio log to download to my laptop before I can diagnose it. [08:40:55] libtool can blow me [08:41:32] and the day may come when it does :) [08:41:38] Not automake, you understand. Just libtool. [08:42:37] automake is worse, but libtool's shell script spewage is ... special [08:43:21] It's less special than it used to be. [08:43:29] i know. [08:43:40] But we're going to need to do something. I'm already bored with DYLD_LIBRARY_PATH, and that's only going to get worse [08:44:43] hey let's rewrite it all to use cmake. or cook. or hey, scons! [08:44:57] and yes, DYLD_LIBRARY_PATH bit me again [08:45:27] Let's not. [08:45:55] i am tempted to ln -s ~/Source/openafs/x86_darwin_100/lib/lib*so.1 /usr/lib which i should prob not. but in a "transarc paths" world, life is very hard. [08:46:35] In relocatable transarc paths world, life is not pleasant at all. [08:46:49] yup [08:46:55] In a "make dest" world, we have no choice but to link everything statically. So the build system is going to need to support that option. [08:47:14] i have been grappling with the question for a while of what the answer is. and i have none [08:47:25] static linking is not the answer, it's a dodge [08:48:12] I think that an answer is that we remove "make dest" [08:48:12] Instead, there's a configure option which sets up the tree to build a "make dest" compatible tree [08:48:23] That would statically link everything in sight, and "make install" then installs into a dest-style tree. [08:49:03] Normal builds, however, would build a tree that expects to be installed at $prefix, and build that tree using shared libraries. [08:49:57] We'd then either use libtool, or our own gloop, to replace in-tree binaries with scripts that setup the relevant environment variables before running them out of the tree. [08:50:29] removing make dest would kinda suck. the real issue is just knowing how to link a run time library path, not make dest itself [08:51:29] The problem is that we can't handle the case where someone installs a dest tree at a different path than the one it was built for. The only way to handle that, without requiring user interaction, is to build statically. [08:52:12] And "make dest" is too late to make the decision to do static builsd. [08:52:23] shadow: re 4159: what about src/*libafsauthent? rx_Init &co are already specified in there [08:52:27] i know. well. i don't like. but i have no better answer yet [08:52:47] only one of libafsrpc and libafsauthent better specify each symbol [08:52:55] otherwise, linker fail [08:53:54] both of their .def's have rx_Init and rx_Finalize and a few other things [08:54:35] Well, that's Windows. I suspect different rules may apply :) [08:54:56] oh, def, not exp. sorry [08:55:29] def, i don't know but will let jeff comment. exp, we need, and need to bump so minor version [08:56:12] You shouldn't be able to export symbols from libafsauthent that actually exist in libafsrpc - the shared linker should throw it back at you. [08:56:53] Yeah. [08:56:57] well, ignoring windows which i know nothing of, yes, that's true. which is why i said afsrpc and not afsauthent in my comment :) [08:57:14] Although nothing in-tree uses the shared version, so nobody will notice if you forget to export the symbols entirely :) [08:57:36] russ will love us more if we don't suck [08:58:00] (which is another reason I'd like to switch our command line utilities over to shared linkage at some point) [08:58:57] on windows afsauthent.dll is re-exporting symbols from afsrpc.dll. It was something Transarc did but I don't want to break app compatibility by undoing it. Don't add new rx_ symbols to afsauthent.def [09:07:57] Hmmm. The HP/UX buildbot seems to have wedged in the middle of a git operation. [09:09:19] no, the namei builder is in a loop trying to get the commit, presumably because the branch/commit entered was wrong [09:10:14] it's annoying, but it's a buildbot thing, I think; it ignores the "can't parse rev" error [09:10:38] Should I be able to do this with a SHA1 that isn't on master yet? [09:10:39] the branch 'master' isn't going to have the commit on a gerrit change that hasn't been merged yet, you need to give the refs/changes/foo branch [09:10:45] Ah. Right. [09:10:57] I'll kill that job and try again. [09:37:13] --- pod has left [09:39:31] --- rra has become available [09:42:23] For the early 1.6 releases, I wonder if we should start requiring that every change being proposed for that release is accompanied by a bug report, and that the change fixes a significant problem. [09:42:41] I have real fears that we're changing stuff so fast that we're never going to stabilise the release. [09:43:54] we need to force people to file real bug reports. also, you don't remember 1.4.0, do you? [09:44:12] Simon Wilkinson: i noticed the yum repo definition rpm for 1.4.14 is not in /afs/openafs.org/sofware/openafs/1.4.14. do you want me to build one to put there? [09:44:14] I remember 1.4 pretty much being a joke as a stable release :) [09:44:43] meffie: If you don't mind. [09:45:09] Simon Wilkinson: ok, is the old spec somewhere? [09:45:10] The yum repo definitions aren't automatically built, so they tend to slip through the net. [09:45:35] Hmmm. Now that's an interesting question. Is the SRPM for them not on the website? [09:45:41] it's not [09:45:45] or, the openafs-repository-..src.rpm? [09:45:45] you never gave me one [09:46:03] And I bet it's not in the build space either. Hang on. [09:48:09] /afs/inf.ed.ac.uk/user/s/sxw/Public/openafs-repository-*.src.rpm [09:50:55] thanks, got it. [10:22:46] --- Simon Wilkinson has left [10:33:47] To do shared libraries with the dest tree, you want to use shared library paths relative to $ORIGIN. [10:33:55] At least on ELF systems that support it (Linux and Solaris). [10:34:25] that would work where it works [10:34:57] Well, I wouldn't try to do shared libraries at all on, for example, AIX if you can avoid it; they're so weird that you need a lot of AIX expertise to debug problems. [10:35:15] I wonder if Mac OS has something like $ORIGIN. [10:36:25] shadow@gmail.com/owl1FDC9F18: can you schlep these up to /afs/openafs.org? /afs/sinenomine.net/user/mmeffie/public/rpm/* [10:36:42] It would be great to use shared libraries for the general client programs. Would save a ton of disk space. [10:37:51] looking... [10:39:43] What we always used to do is that we'd build stuff shared for solaris and Linux and static for HP-UX, IRIX, and AIX so that we didn't have to deal with a bunch of different shared library models. That's probably too conservative, but something like that is worth considering. [10:40:28] It's a 90/10 thing; 90% of our users are using either Windows, Mac OS X, or something that does ELF. So if you bail on everything else, the number of people impacted is fairly small. [10:40:49] macos does elf [10:43:25] Oh, does it? I thought it was using some sort of COFF varient. [10:44:25] wait. mach-o. not elf. sorry [10:44:40] I was going to say; AFAIK the object format hasn't changed since... [10:45:06] sorry. my mind is elsewhere [11:51:05] --- jaltman/FrogsLeap has left: Disconnected [12:00:48] um... I'm not sure what to do for th 4159 pullup for the shlibafsrpc minor [12:00:52] <<<<<<< HEAD LIBAFSRPCMINOR=2 ||||||| LIBAFSRPCMINOR=3 ======= LIBAFSRPCMINOR=4 >>>>>>> eddcee3... RX: Avoid timing out non-kernel busy channels [12:01:45] bump to .5 [12:02:03] then we have to remember to .6 in master? [12:02:12] yeah, sadly [12:02:25] alas [12:02:34] actually, given rxgk, i suspect master should be 2.x [12:02:48] well, what has landed of the rxgk work [12:09:10] Really? Did rxgk create a backward-ABI-incompatible version of that library? [12:10:04] diff suggests i am wrong [12:10:09] so, no [12:10:49] Good. In general, we should avoid doing that. [12:11:31] well, sure [12:42:57] --- todd2 has become available [13:18:12] --- todd2 has left [13:52:20] --- pod has become available [13:57:00] --- sxw has become available [14:07:58] --- sxw has left [14:08:37] --- pod has left [14:15:19] --- sxw has become available [14:16:34] Until we stop making our internal structures public, pretty much every release we do has ABI issues [14:16:43] that too [14:17:40] Better to bump the SONAME when we don't need to than not bump it when we did need to. [14:18:19] well, we don't need major 2 yet [14:19:22] I would like to rework our API. Now that we're warning free, I think its the next Big Task [14:19:38] "rework" implies we have one [14:20:13] Hiding the internals of rx_call, rx_connection and friends is pretty high up the list. [14:20:59] Anyone relying on sonames for libafsrpc will have been very sad a long time ago. [14:21:19] meh! [14:24:21] * rra would love to see us do that reworking. [14:24:24] * rra is a big fan. [14:24:47] For bonus points, we could add Doxygen comments or something and have a documented API. [14:24:58] --- jaltman/FrogsLeap has become available [14:25:04] Of course, easy for me to say when I don't have time to work on it. ;) [14:25:27] Most of rxgk is doxygen'd [14:25:37] And has a test suite :) [14:27:05] With this, I think the trick is to start at the bottom of the stack and work upwards [14:27:49] And probably to throw libafsauthent and libafsrpc away in the process [14:28:08] in favor of? [14:28:41] fewer libraries is better. beats rx lwp rxkad sys afsutil as a prereq for prot volser vl auth [14:29:29] If you're on a platform with dependencies, you don't care. [14:29:57] And I shouldn't have to kink against vol code if ai'm writing a ptclient [14:30:18] Link, even [14:32:47] i care if my makefiles say something that doesn't exist anymore [14:33:38] Yeah, so we'd need to ship both for a bit. [14:37:30] I'd prefer us not go the route of making too many shared libraries. I think the only point at which it's useful to break something off as a separate shared library is if it pulls in some external dependency that you may not want. [14:37:39] If it's just chunks of AFS code, fewer shared libraries is better. [14:37:55] Using both MIT Kerberos and Heimdal can be a ginormous pain in the ass due to the massive numbers of shared libraries. [14:38:32] Although Heimdal is a bit better because the main type of code doesn't require adding on a bunch of extra stuff. [14:47:29] --- Simon Wilkinson has become available [14:47:58] The problem is that libafsauthent is a real mess of bits and bobs [14:48:22] "all the non-rpc parts" [14:48:28] er, rpc ayer [14:48:33] ... and the kitchen sink. [14:49:23] I'd much rather have a libafsbase, libafsrpc, and then a library for each of the afs services - libafsvl, libafspt, libafsfile [14:50:05] Ick. :/ [14:50:12] Why not just one libopenafs? [14:50:44] Because it's pretty rare that you're writing a piece of code that needs everything [14:50:52] Code that isn't loaded is free. [14:51:08] From a usage perspective, sure. But not when you're building. [14:51:21] If these are shared then the pain is less. [14:51:27] How so? My worry is that stuff like the AFS Perl extension has to get linked with 20-odd libraries. [14:51:34] Also, that means I have to generate 20-odd separate Debian packages. [14:51:51] Well, 20 may be an exaggeration. [14:51:55] But we have a *lot* of subsystems. [14:51:58] That's Debian's choice. I just need 2 RPMS for Fedora (devel, and not) [14:52:13] It's Debian's choice because handling SONAME changes is insane without it. [14:52:28] If you bundle everything together and one SONAME changes but the others don't, you're screwed. [14:52:59] See the giant mess that happened with the SONAME transition with MIT Kerberos in Debian. [14:53:05] Sure, it's Debian's choice. But I'm not sure we should let the way people package us unduly influence the way we structure our libraries. [14:53:30] I don't think that's what I'm advocating. [14:53:51] Debian's packaging is just surfacing the fact that we're creating X separate interface specifications with separate SONAMEs and separate version numbers. [14:53:56] That's complexity however you look at it. [14:54:17] If they change independently, Red Hat packaging will either be very sad or excessively strict on dependencies, because that's the only way you can handle it. [14:54:54] The way I'm looking at this is that we already have a large number of internal interfaces. [14:55:10] There's a tradeoff, I admit -- it's nice to be able to bump the interface of the file package without changing all the other SONAMEs and recompiling stuff that doesn't care about the file package. [14:55:25] Hiding that complexity by bundling them all into one package for external (and possibly internal consumption) just serves as an abstraction layer. [14:55:39] But you end up with this big table of SONAMEs mapping to AFS releases that gets kind of complicated. Heimdal has this problem right now, and it takes some amount of mental space to figure it out. [14:56:06] I completely agree with you on the need for defining the interfaces and figuring out what's public and what's private. [14:56:15] I'd expect my packaging system to handle that complexity for me. [14:56:17] I think that's somewhat orthogonal to the question of how many separate shared libraries we build. [14:56:37] Your packaging system can only handle it if you generate separate packages for every shared library. [14:57:08] Suppose libafsfile.so.1 goes to libafsfile.so.2, but nothing else changes. [14:57:16] If you have one shared library package that contains everything, what do you do? [14:57:32] everything requires the version of the package that contains libafsfile.so.2 [14:57:49] And if you have stuff on your system that still requires libafsfile.so.1, well, you lose. [14:57:50] Stuff built against libafsfile.so.1 will break if you install the package that contains libafsfile.so.2. [14:57:55] Yeah. [14:58:21] What if you want stuff that's linked against libafsfile.so.1 and stuff that's linked against libafsfile.so.2 installed at the same time? [14:58:38] While you're rebuilding, or during upgrades, for example? [14:58:54] You don't get to do that. You only do major .so bumps at upgrade times, when you do whole system upgrades as a single transaction. [14:58:54] That's why Debian breaks them all apart. So that you can co-install different revisions of libraries. [14:59:05] 1.4.14 would ship one set of libraries. you have one set of packages. new libaraies, new version. don't care [14:59:21] Whole system upgrades as a single transaction become essentially impossible with enough shared libraries and enough binaries. [14:59:28] Debian reached the pont where that's impossible about 10 years ago. [14:59:37] Well, Fedora is still managing to do that. [14:59:46] where shipping 1.4.14 is impossible? yeah, it does seem like it :) [14:59:57] I think Fedora is just accepting a bunch of breakage that Debian doesn't accept. [15:00:03] I don't think they're actually doing that. [15:00:18] You can go from Fedora 13 to Fedora 14 in a single RPM transaction. [15:00:29] Consistent state before, consistent state after. [15:00:47] --- sxw has left [15:01:00] In any case, this isn't my main object. [15:01:03] objection. [15:01:05] "every bit as broken after as when you started". i guess it's not so bad as that. fedora has treated me ok [15:01:06] Anyway, this is sort of a rathole. I think Fedora's method of doing this is completely wrong, and I believe that pretty strongly and am not going to be convinced otherwise, but I'm not sure it's worth arguing about. Anyway, reagardless, I think we're borrowing a bunch of trouble by maintaining a bunch of different interfaces that are all separately versioned, based on what I've seen with other projects that do this. [15:01:36] The reason I have for each module being an independent shared library is, ultimately, an internal one. [15:02:02] And that is that each module is separate. The public interface that it exposes to the rest of the tree should be the one that it exposes to the rest of the world. [15:02:19] I agree, that's a fairly good argument. [15:02:26] Our means of controlling interface boundaries is with shared library exports. [15:02:29] exhibitionist modules [15:02:37] Ergo, each module should be in its own shared library. [15:02:46] I suspect that's the motivating argument in Heimdal too. [15:03:11] i'm not wild about the heimdal model. [15:04:11] The problem is that unless we have a programatic way of enforcing interface boundaries then people start taking short cuts, and before you know it you're back in the same old rats nest. [15:04:20] Okay, hm. You've got a good point there; it's harder to control visibility if you don't build shared libraries inside the tree and link to them just like external clients do. [15:05:03] shlibafsauthent and shlibafsrpc are regularly broken, simply because nothing within the tree makes use of them. [15:05:16] And I can create 20-odd shared library packages or whatever is needed; Brian does for Heimdal. It's kind of annoying, but it's mostly just annoying once. My primary concern is that we're going to have libafsfile.so.1, libafsvl.so.4, libafsbase.so.12, and so forth, and people are going to get really confused about which one came with which OpenAFS. [15:05:38] They aren't going to get confused on Fedora or Debian, where the packaging system takes care of it for them. [15:05:49] They're going to get confused when they build it themselves and make install into /usr/local. [15:05:52] Well, the solution to that is to use one .soname across all of the shared libraries. We're not using automake, so we actually have the freedom to do that. [15:06:03] Ugh. Ugh ugh ugh. [15:06:10] You change one interface, you rev them all :) [15:06:13] and now i will stab myself in the face [15:06:14] Changing sonames when nothing changed about the interface is really annoying. [15:06:28] You basically gratuitously break backward compatibility for existing binaries. [15:06:42] We can't break backwards compatibility. [15:06:59] The requirements for windows pretty much guarantee that we can never do a major soname bump. [15:07:07] If you change libafsfile.so.1 to libafsfile.so.2, everything linked against libafsfile.so.1 stops working. [15:07:09] Oh. [15:07:18] So then SONAMEs never change, and what were we talking about again? [15:07:18] oh? [15:07:35] We can add new symbols, and so need new minor sonames. [15:07:51] can't remove? [15:07:56] But Jeffrey has big problems if you change the signature of, or remove, an existing symbol. [15:07:57] or change signature? [15:08:02] Adding new symbols, pah. For that, I don't care what you do. Debian will ignore your minor sonames and manage that through symbols files. [15:08:05] Yeah, and yeah. [15:08:16] Window's is for life, not just for Christmas. [15:08:21] i guess i knew sad but not nonstarter [15:08:30] window's what? [15:08:36] Okay, if we're never going to remove symbols or ever change interfaces, then I don't actually care what you do, since SONAMEs never change. [15:08:54] And at that point, yes, I understand why Fedora doesn't separate the packages apart, because they don't need to change independently. [15:09:01] However, what is Windows doing on Heimdal? [15:09:06] Love revs SONAMEs all the tme. [15:09:14] windows can also have its own libraries that i don't care about with their own dll name and whatever happens happens [15:09:43] I don't know about Windows Heimdal, you'd have to ask Jeffrey about that. [15:10:06] until recently it didn't matter. [15:10:20] But as far as he explained it to me, the OpenAFS situation on Windows is that once a code point has been allocated for a symbol, there must be something, even if its just a stub, in the symbol table for that. [15:10:55] And you can never change the signature of a symbol once you've shipped it. A different symbol means that you have to implement a new function. [15:11:11] Well, unless you change the name of hte library. [15:11:31] Yeah. [15:11:31] It's actually the same situation as Linux, just that on Linux we're a lot more cavalier about changing the name of the library (that's what an SONAME bump basically does). [15:13:17] wow. I finally caught up [15:14:56] The situation on Windows is a real mess until we drop support for Windows 2000 and pre-SP2 XP. Once that happens we can start packaging libraries as side-by-side assemblies. At that point applications will have to be built against an assembly and the abi signature will be controlled by that boundary. We won't have arbitrary dlls in the PATH that can break an application built against something older. [15:15:56] Heimdal on Windows is side-by-side. MIT is not [15:16:12] --- jaltman/FrogsLeap has left: Disconnected [15:21:33] --- deason has left [15:23:01] I guess it wouldn't be possible to use a number of libraries in tree, to enforce interface boundaries, but to ship a single library externally. [15:23:57] Yeah, it would, but it's actually pretty annoying to do that. [15:24:10] The problem is that the binaries you build in-tree have encoded dependencies on the internal shared libraries. [15:24:17] so you have to relink them all on install, which is a mess. [15:24:29] Or you end up shipping both, which is more of a mess. [15:24:29] I'd rather just ship a bunch of shared libraries than do that. [15:24:51] Yeah, "both" is definitely not the right answer; that gives us the worst of both worlds. [15:25:04] The main reason that I'd like to do all of this work is in order to improve our internal code quality [15:25:42] --- jaltman/FrogsLeap has become available [15:25:49] There are far too many places where someone has found a need to twiddle an internal setting, and so directly accesses an element in a supposedly private structure, or makes use of an internal function. [15:26:05] Yup, I'm with you on that. [15:26:06] It's only by building walls between our various modules that we're going to pick up on all of this. [15:26:11] Using shared library visibility to enforce that would be great. [15:27:59] I'll happily absorb the work of packaging in order to get that benefit. :) [15:28:42] Cool. I just need to find cycles to actually do it ... [15:28:42] can you take care of the side-by-side assemblies for windows while you are at it? :) [15:29:33] btw, I think that the rxgk deployment is the right timeframe to implement side-by-side for windows [15:29:59] my only concern is that we might have to create a backward compatible set of loose DLLs for existing apps [15:30:25] I guess the question is where rxgk fits into the OpenAFS release cycle. [15:30:38] Are we likely to ship a release from master before rxgk lands in the tree. [15:30:48] at the rate 1.6 is going? [15:31:12] At the rate 1.6 is going, we should all give up now. [15:31:42] My point about master, though, is that master is "rxgk ready" [15:32:04] It's got all of the bits you need to run with security layer 4, just not the implementation of that layer. [15:32:25] 1.7 will be cut from master at this point [15:34:31] completely off topic. if you do have any windows boxes, please run windows update asap. the hole in media player that was patched today will have exploit code in the wild real soon now [15:37:07] In rxgk news, I have now completely implemented all of draft-wilkinson-afs3-rxgk. A revised version of that document based on implementation experience will follow in the next couple of days. [15:38:40] excellent [15:41:38] There is still work outstanding to complete draft-wilkinson-afs3-rxgk-afs, but at this point I want to try and get the base document moved down the standardisation process so we can start pulling code implementing that into the OpenAFS tree. [18:11:56] --- rra has left: Disconnected [18:33:19] --- Russ has become available [20:05:27] --- deason has become available [21:31:13] --- deason has left [23:03:05] --- reuteras has become available