[00:34:41] I think the fix is to remove both the afsconfig.h and afs/param.h includes, and the inclusion of assert.h [01:30:08] Re: openafs-next. The idea here would be to serve a similar function to linux-next - it would be a rebasing repository which contains the merges of all of the git trees which are blocked waiting to merge into the tree at the next merge window. It provides an idea of what that tree would look like, as well as identify areas of conflict. [01:33:14] --- Russ has left: Disconnected [04:21:12] Given we can't use $< in ${CCRULE}, what would peoples preference be for places where the make rule needs to have more than just the .c file as a dependency? [04:21:20] We could redefine the pattern rule, but that gives us a warning. [04:21:34] Or we could do something like ${CCRULE} [04:34:50] --- Jeffrey Altman has left [04:36:53] Gerrit is just going to have a quick restart. [04:54:25] --- jaltman has left: Replaced by new connection [04:54:26] --- jaltman has become available [05:18:39] 0. [05:18:40] + [05:18:40] [05:32:55] --- sxw has left [07:05:12] --- deason has become available [07:06:12] --- abo has become available [07:27:18] --- haba has become available [07:42:39] --- matt has become available [07:51:02] --- sxw has become available [07:54:58] --- jaltman has left: Disconnected [08:14:50] kaduk: another place for those kind of things could be in gerrit itself; you can still comment on changes after they've been merged [08:17:30] Commenting on gerrit does help, but it only works if the person is still around. [08:18:19] In general, if you want to make sure that a problem doesn't get missed, mail to openafs-bugs is the best way to go. [08:27:07] Should volume locks be logged somewhere? pdc.modules 537045975 RW 2535 K On-line houting.pdc.kth.se /vicepa RWrite 537045975 ROnly 537045976 Backup 537045977 MaxQuota 5000 K Creation Fri May 16 09:43:19 2003 Copy Thu Apr 5 11:36:43 2007 Backup Tue Jun 15 00:33:22 2010 Last Update Tue Jun 15 16:39:02 2010 183 accesses in the past day (i.e., vnode references) RWrite: 537045975 ROnly: 537045976 Backup: 537045977 number of sites -> 3 server houting.pdc.kth.se partition /vicepa RW Site server houting.pdc.kth.se partition /vicepa RO Site server sculpin.pdc.kth.se partition /vicepa RO Site Volume is currently LOCKED [08:27:34] I know that it is locked now, but since when is the question. [08:28:30] And why has it not been unlocked if Tue Jun 15 00:33:22 2010 1 Volser: Clone: Recloning volume 537045975 to volume 537045977 was the last action (17h ago) [08:33:38] turning up logging on the vlserver would log it for you, but you probably don't want to do that since it pulls in a ton of ubik logging, too (or turning on vlserver audit logging) [08:34:13] only vos really knows why it wasn't unlocked (assuming it was locked as part of a release operation)... it could have failed to contact the vlserver for the last 'unlock' step, or was otherwise interrupted [08:37:06] Or someone could have just run 'vos lock', I guess... [08:50:20] --- kaj has left [08:50:25] --- reuteras has left [09:07:07] --- sxw has left [09:25:44] --- haba has left [09:45:35] --- kaj has become available [09:51:57] --- mattjsm has become available [10:06:25] --- rra has become available [10:11:52] --- mattjsm has left [10:14:26] --- kaj has left [10:32:56] --- kaj has become available [10:39:27] Russ, is it possible to integrate shell script tests with the runtests run or will they always have to output the verbose test results? [11:03:05] --- kaj has left [11:07:20] --- sxw has become available [11:41:08] --- kaj has become available [11:50:16] --- kaj has left [12:00:17] --- mmeffie has left [12:10:18] --- mmeffie has become available [12:10:21] JSund: Sorry, not sure what you mean. [12:11:13] Taking a stab at what you maybe meant: You can write test cases in shell, but they all need to follow the TAP output format. However, that output format can be just "1..1\nok 1\n" if you don't want to separate things into separate tests. It would be nicer to do individual tests in case one of them fails, though. [12:11:20] rra: runtests outputs something like "Running all tests in TESTS... Test1.....ok Test2.....ok [12:11:46] is it possible to get the same shortened output for shell tests or are they limited to the TAP format? [12:12:15] All the tests run by runtests need to output their results in TAP format. [12:13:04] * rra has to run -- back in about an hour and a half. [12:13:23] http://www.pastie.org/1005791 [12:13:50] JSund: I think you're confusing the output of the tests, with the output of runtests itself. [12:14:48] right - I would like to have runtests output the results of shell tests as well as C tests, but I'm guessing that's impossible [12:15:26] I think you should be able to just list the name of the shell test in the "TESTS" file. [12:21:18] --- mmeffie has left [12:21:21] Right, I had suffixed the shell script with .sh, which wasn't expected by runtests [12:21:33] Now it works, thanks! [12:31:19] --- mmeffie has become available [12:41:18] --- mattjsm has become available [12:41:45] matt: afsd fails with a bad system call and dumps the core. [12:42:04] is that going to be essential for the next step? or do i need to poke through and fix that first? [12:43:44] sounds like you have not loaded the kernel module [12:43:54] indeed. :) [12:44:10] so no, then. thanks [12:44:31] yeah; that's pretty much expected behavior for afsd in that case [12:45:11] afsd donates threads to the kernel by making system calls. If there's nothing in the kernel to answer those system calls, it will be sad. [12:46:20] --- mmeffie has left [12:48:33] one of these days we really should make it not work that way; most platforms these days have better ways of creating kernel threads. that said, afsd also does a bunch of other setup by making system calls, none of which is interesting or will work until the module is loaded. [12:49:22] Yeah. On Linux we actually have a horrible mess where afsd making the system call prompts the kernel module to actually create the thread. [12:49:45] (and some kernel threads get created without afsd being involved at all) [12:57:01] --- mmeffie has become available [12:57:09] --- mmeffie has left [12:57:19] --- mmeffie has become available [12:59:47] oh, the module was not loaded. yes, start with that. [12:59:52] more details in email [13:16:51] --- jaltman has become available [13:21:20] JSund: Oh, yeah, sorry, test cases have to end in .t or -t. [13:25:52] --- Jeffrey Altman has become available [13:30:35] Is there any reason that I shouldn't put the clang-analyzer results on the web somewhere? [13:31:01] <_cclausen> can't anyone run it for themselves anyway? [13:32:20] Yes. [13:32:37] <_cclausen> oh, fancy llvm.org is hosted on the uiuc campus. I had no idea [13:32:41] --- steven.jenkins has left [13:32:42] And I've been over the results, and there's nothing immediately horrible in them. [13:33:01] <_cclausen> sounds to me like they can be posted somewhere then [13:33:43] What we really need is a front end to them. So they can be annotated, and the false positives hidden. [13:35:03] like, wikitext? [13:35:24] Would it be useful to annotate the false positives in the source code directly? [13:36:07] Also, in general, are there reasonable improvements to our coding standards that we could make that would eliminate the false positives, similar to how we try to get rid of GCC warnings by making the code more obvious to the compiler even if it isn't technically wrong? [13:36:37] There are some things we can do. I've been looking at some of those. [13:37:24] Others are simply limitations in clang's path analysis - it doesn't (yet) do any intra-module analysis, so has no idea what effects a subroutine call may have. [13:37:39] It's also not too great at tracking global variables. [13:37:54] And while eliminating global variables is a fine idea, that's going to take us a while. :) [13:38:44] There's also stuff that we need to do to shut up gcc warnings, that clang flags as being unecessary (initialisations for paths that clang knows are never reached, but gcc isn't so smart) [13:38:57] Oh, that's annoying. [13:39:37] Initialized but never used is legitimate lint. Initialized but used in only some paths is just obnoxious, IMO. [13:39:52] I think we'll probably have to ignore most of clang's dead initialisation warnings. But some of them at least, are very useful to warn you that you've done "code = FunctionThatMyFail()", and yet never checked the value of code. [13:40:15] Yeah. [13:40:21] I like that sort of stuff. [13:40:41] --- Kevin Sumner has left [13:40:42] we can't tweak it so we're just told about things that are set, but not referenced in _any_ branch? [13:40:59] deason: That's exactly the warnings that we get. [13:41:04] --- Kevin Sumner has become available [13:41:05] All 583 of them. [13:41:22] Hm, then those I think are legitimate to just remove, yes? [13:41:27] I mean, any branch, even unreachable ones [13:42:02] this is reminding me of the UFSGetVolSlot oldvtix thing just a few weeks ago :) [13:42:13] Yeah. [13:42:32] rra: but then you get gcc warnings [13:42:58] The analyzer doesn't know about unreachable branches (it aggressively prunes the tree to keep it under control), so that's data we can't get from it - gcc will tell us some of it with the right flags. [13:43:12] --- abo has left [13:43:19] deason: Not for everything. [13:43:22] deason: if the variable is initialized and never used, then we should remove both the initialization *and* the variable, no? [13:43:24] For example if have: [13:43:27] --- abo has become available [13:43:30] ts = cmd_CreateSyntax(...); [13:43:38] ts = cmd_CreateSyntax(...);; [13:43:41] (and so on) [13:43:48] Oh, I see. It's not the *variable* that's never used, it's the value. [13:43:53] That's something different than what I thought you were saying. [13:43:54] I'll get a warning for each of those lines, because I'm assigning ts and never testing it. [13:43:56] rra: oh right, hah :) [13:44:25] Yeah, I suspect that most of those are legitimate as at least coding style nits, but I can see that getting rather complicated. [13:44:53] Yeah. That's why the code case is interesting ... In an if() ladder, it catches the cases that you assign, but don't test, a return value. [13:45:31] And if you really don't care, usually you can remove the assignment. [13:45:44] But the implications, like const cleanliness, could have far-reaching consequences to completely squash all the warnings. [13:45:50] (So, if you do "code=ThingThatMightFail(); if (!code) code=OtherThingThatMightFail(); return;" it'll warn you that you forgot to test the second code. [13:46:02] I don't think we can ever completely remove static analysis warnings. [13:46:32] Probably not, although we can get pretty far. [13:46:38] That's why the grown up tools like coverity have a database, and ways of saying "this warning, at a bit of code that looks like , should be ignored because " [13:46:53] I'll stick the output online, and let folk take a look. [13:47:05] I got INN, which was a medium-sized legacy code base, to compile cleanly under gcc with -Wall -W -Wendif-labels -Wpointer-arith -Wbad-function-cast -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Werror [13:47:07] --- Kevin Sumner has left [13:47:25] --- Kevin Sumner has become available [13:47:29] With no exceptions. [13:47:46] --- mmeffie has left [13:47:56] --- mmeffie has become available [13:48:13] --- kaj has become available [13:48:54] There's an important difference between compiler errors, and those that you get back from a static analyser, though. [13:50:16] > ts = cmd_CreateSyntax(...); If you're not going to us that value, consider rewriting as (void)cmd_CreateSyntax(...); [13:50:16] Are there any of those warnings we should consider turning on for OpenAFS? [13:59:11] --- mmeffie has left [13:59:29] --- mmeffie has become available [14:11:01] At Jeff's prompting, I've finally pushed the old afs3-standardisation draft into the I-D repository - draft-wilkinson-afs3-standardisation [14:12:04] Or even, draft-wilkinson-afs3-standardisation [14:20:52] Eventually I'd like to see us enable all of those warnings, but they're of varying degrees of difficulty. [14:21:02] -Wwrite-strings requires we be const-clean, for instance, which we almost certainly aren't. [14:21:29] -Wstrict-prototypes I think we could probably turn on now. -Wendif-labels is trivial. [14:22:06] -Wmissing-prototypes is a coding style thing which requires that every function either be prototyped in advance (ideally in a header file) or declared static. I like it, but I suspect it would require pretty extensive sprinkling of static through the tree. [14:22:37] I'm fairly sure we fail -Wpointer-arith and -Wbad-function-cast already unless they're in -Wall now, but they'd be worthwhile. [14:22:45] -W has a bunch of random stuff, some of which is sort of annoying. [14:23:04] So I think they're all worth considering, but the amount of pain varies. const-cleanliness in particular is rather obnoxious to do with a large code base. [14:24:18] I thought we already had strict-prototypes and pointer-arith [14:24:24] We might. [14:24:27] I haven't gone and looked. [14:28:05] Shortly I'll be sending a message to the various relevant mailing lists in an attempt to figure out whether there is a consensus on that document. If it looks to us like there is, the registrars will begin the election process. [14:28:33] I wonder where I ought to announce that. Presumably afs3-stds, openafs-devel, arla-drinkers. Where else? [14:28:34] for write-strings... yeah, I remember at one point trying to be able to pass const strings into the libafs code via libuafs... it was a nightmare [14:29:02] You pretty much have to start with your low-level libraries and systematically work your way up, adding const. [14:29:05] It took me quite a while in INN. [14:29:59] You have to do that for cases where libraries accept const data as arguments. For cases where libraries return const data, it's harder. [14:30:43] the other problem is that many of these warnings still exist because our public APIs practically force them [14:31:00] --- mmeffie has left [14:31:30] Thankfully, you can add const to public APIs without changing the ABI. [14:31:36] --- mmeffie has become available [14:31:54] If you call third-party libraries, that's another matter; INN has to cast away const in a few places, such as struct iovec's and the Berkeley DB API. [14:32:07] FWIW, IMHO, const is an abomination and we'd all be better off if it had never been added to the language. [14:32:20] I really like it once you have const discipline throughout a code base. [14:32:25] It's caught a lot of bugs for me. [14:32:33] The same goes for the Berkeley DB API, and in fact all of Berkeley DB :-) [14:32:35] Whether getting there is worth the pain is definitely a matter of opinion on which reasonable people differ. [14:33:26] The main bug that const discipline catches is memory management: being sure that things that you don't think will be freed will never be freed. [14:33:44] It ensures that you won't accidentally call free on a static string and explode. [14:37:27] --- sxw has left [14:42:22] rra: specifically I'm thinking of the unsigned/signed things... at least ubik and VL [14:42:40] Oh, yeah. [14:42:44] That's a bit more annoying. [14:43:26] the signed/unsigned thing: tears. lots of tears [14:44:10] Speaking of warts on the C language that I wish never existed: signed char. [14:44:38] Yeah. That whole thing was just wrong. [14:45:31] fwiw: if you think you care about 1.6 release planning, and you aren't on the release-team list, you should probably fix that [14:45:45] Well, actually, you sometimes want a char-sized (i.e. byte-sized) signed number, especially if you're manipulating low-level stuff. [14:46:55] I'd be okay with an explicit signed char data type (preferrably, following Java, called byte or something instead), but that char is signed by default on most platforms (and, worse, may or may not be signed depending on your platform) is just hideous. [14:47:21] int32_t :) [14:47:33] No, something called "byte" really ought to be unsigned. int8_t is a fine name, though. :-) [14:47:41] Yeah, int8_t works. :) [14:47:42] "byte" is signed? it sounds like a uint8_t [14:47:45] byte is signed. [14:47:51] All Java basic numeric types are signed. [14:47:58] Java has no unsigned basic types. [14:48:37] I thought release-team required some degree of specialness or something [14:48:55] <_cclausen> I hope not [14:48:57] actually, "byte" in C ought to be slightly magic, in that it should be unsigned but not generate the warnings that unsigned char does. [14:48:58] <_cclausen> I'm on that list :-) [14:50:42] if you submitted patches to gerrit and we took them, that's probably plenty special [14:51:11] > I thought release-team required some degree of specialness or something It sort of does; it's supposed to be people actually involved in some stage of a release. That includes at least gatekeepers and anyone who builds binary releases from time to time, plus some other people. We generally reject people who just try to subscribe to all the openafs lists (or all the lists at lists.openafs.org), but I can't think of any case where someone with a halfway legitimate interest has tried to subscribe and been rejected. [14:51:29] just subscribe. i have the moderator interface up. i'll approve it [14:52:16] <_cclausen> in theory it involves people who actually test releases as well, which I think is how I got on there [14:52:47] That said, I'm not sure release-team is the right place for this discussion. People who are not somehow involved in the actual release process should not have to be subscribed there to stay in the loop. [14:53:33] People who test things before we call them a release, yes. People who install every version post-release and submit bug reports, no [14:53:34] the discussion will be on jabber. [14:53:48] > the discussion will be on jabber. and should be announced on -devel [14:53:52] not there. but it's an occasion for a reminder [14:54:44] > not there. but it's an occasion for a reminder huh? [14:54:59] you want to fill in some missing bits, like the entire first sentence? [14:54:59] which word(s) should i explain [14:55:23] 1.6 is coming, obviously. if you belong on release team, now would be the time? [14:56:05] what should not happen where? you're arguing that informing developers that a discussion will happen about things like when 1.6 will happen, what will be in it, and what the VCS tree will look like around/after that is _not_ something that belongs on -devel? [14:56:46] actually, the people who deploy 1.6 matter far more than the people who are developers, imo, for obvious reasons [14:57:07] it's a release planning meeting, not a development planning meeting [14:57:11] Matter, yes. Belong on release-team, no. [14:57:28] I think it would be fine to also notify -devel, although I'm not sure we're going to reach more people than here plus -release. [14:57:47] Alternatively, I think we should only discuss OpenAFS 1.6 on arla-drinkers, which doesn't get enough traffic and is lonely. [14:57:54] So it's like every other release planning meeting? You're going to talk about things like whether all the known bugs are squashed in the latest RC, who's going to build what binaries, etc? [14:58:17] as opposed to? [14:58:40] really, it comes down to "would you (for some you) deploy this?" [14:58:59] otherwise, we can wank forever and continue not releasing anything [14:59:11] We're releasing master as 1.6, and so far as I know no one's proposed pulling anything that's in master before the release, so there isn't much to say about in and out stuff. [14:59:20] As opposed to "will $large_feature be in 1.6, and if not, when?" "will the tree freeze? for how long? what do I do with my patches?" [14:59:42] > would you (for some you) deploy this? that almost suggests mentioning it on -info is a good idea. [14:59:44] $large_feature: is there code now? if not, presumably not. [14:59:49] The first question is hopefully anwered by the roadmap. [15:00:09] and yeah, what he said [15:00:09] > will $large_feature be in 1.6, and if not, when I have at least two questions like this [15:00:17] > the roadmap I haven't looked at that in some time, because it always seems to be meaningless. [15:00:25] --- Kevin Sumner has left [15:00:28] --- abo has left [15:00:30] (well, not "large") [15:00:37] We should figure out what we're doing about freezing the tree. I assumed that we were branching and then master would not be frozen, although it will be hard to get too much gatekeeper attention. We probably need to send out mail explaining the bit we talked about at the workshop, namely what 1.8 will be, how 1.8 will be handled, and the fact that the next traditional release will be 1.10. [15:00:43] well, we already know that there is code for features that will not be in 1.6, and there has been a lot of communication, at least at afsbpw, about what sites should expect [15:00:44] the "if not" should be obvious now. the "when" one hopes is answered on the roadmap, and if it's not, it's not really specific to 1.6, is it? [15:00:51] --- Kevin Sumner has become available [15:01:07] --- abo has become available [15:01:09] Oh, also... > would you (for some you) deploy this? Not if it contains broken dafs. Possibly also for some other values of $large_feature [15:01:21] It had better not have broken dafs. [15:01:25] does it contain broken dafs? [15:01:26] Working dafs is the definition of 1.6. [15:02:03] Right, so if there's been "a lot of communications", but all of it in person to people who were at the workshop, then there hasn't been a lot of communications. Try telling -devel about your development plans. [15:02:15] Yeah, we need to send that part out in e-mail. [15:02:21] To -announce, actually. [15:02:35] At least the 1.6, 1.7, 1.8, 1.9, 1.10 plan. [15:02:37] andrew's patch looks like the patch i had but hadn't pushed. mine works. i assume his does too and will push it. after that, i haven't (yet) seen any other dafs issues [15:02:37] --- mmeffie has left [15:03:11] --- mattjsm has left [15:03:35] Is that a patch to fix the problem where state was corrupting each time the server went down? [15:03:49] --- mmeffie has become available [15:05:08] yes [15:05:28] (only "each time" or not depended on what type of clients you had) [15:05:40] not corrupting so much as failing to verify. invalid assumptions [15:05:43] (well, technically the state was fine but we screwed up loading it) [15:05:47] yeah [15:06:53] The remaining problems I'm aware of are problems with the salvager that have been there ~forever but that dafs just makes a bit more obvious because it runs continuously so you notice when it croaks and dies. [15:17:40] Before sending out the 1.6, 1.7, 1.8, 1.9, 1.10 plan, it would be good if there was some public discussion of it first. [15:18:02] Should I send it to -devel first and then, after a discussion period, send the results to -announce? [15:18:41] <_cclausen> any reason not to send at least the initial email to -info as well? Or do you only want developers? [15:18:56] I think it definitely needs discussed - at least the version I originally heard, where no new development could take place until the Windows redirector was integrated. [15:19:14] Yeah, that's not quite what we're doing, although that's what it sounds like without all the details. [15:19:25] I hate to send stuff to both -devel and -info, although I suppose I should just get over it. [15:19:50] <_cclausen> doesn't mailman only actually send one if you are on both lists? [15:19:53] I guess the bigger question is - how widely did you consult before you decided what "we're doing" [15:19:53] Do we expect people on -info to have objections and discussion, or is -info more for notification purposes (and hence covered later by -announce)? [15:20:21] <_cclausen> I suspect that -announce is not a super-set of people on -info [15:20:53] Because I think that presentation is important here: Is this a plan that your asking for objections to, or a proposal that you're putting forwards to stimulate discussion. [15:20:58] <_cclausen> my only concern is people only on -info [15:21:00] I think that people who are only on -info and not on -announce are going to be sad frequently for many other reasons, but I suppose in this case we can reach out special. [15:21:58] Simon: I don't think there's anything horribly controversial in what we're proposing and I don't think it's horribly likely that anyone will object, but given that the decision was primarily made by Jeff, Derrick, and I in a hotel room, it's possible that we missed something and should adjust accordingly. So I'd say that it's a plan for which we're asking for feedback and possible objections. [15:22:45] --- Kevin Sumner has left [15:23:02] --- Kevin Sumner has become available [15:23:19] --- mmeffie has left [15:23:28] I just think that what happens to master is going to affect (at least) Matt, Marcus, Andrew, Christof and Hartmut - and that they should probably all be consulted about what works best for them before anything is firmly announced. [15:23:48] --- kaj has left [15:24:06] [15:24:07] Agreed, although given that what's happening to master is basically "nothing; open for business as usual," I don't think it should be a problem. [15:24:33] yeah, that's what I heard; I don't have any objections to anything I've heard yet [15:24:55] discussion of development plans belongs on -devel but users, who presumably only read -info, ought to have some input on things like which features are important to them. [15:25:03] -announce is a fine place to announce plans once they are decided. but there needs to be discussion first. That is, unless the gatekeepers are going to pull the usual "this is how we've decided it's going to be, and too bad if you don't like it or wanted to have any input". [15:25:15] The thing is that master hasn't really been open for business for a while now. [15:25:19] Ah, there we go. Now you can have things I said some minutes ago, before !#@%$!#$@# jabber broke again [15:25:41] Something that we're going to have to plan is how to land all of the stuff that's queued up. [15:25:43] --- mmeffie has become available [15:25:55] jhutz: There should definitely be discussion first. My bad -- I was thinking about -announce in the context of "this should really reach a broader audience than just -devel" and not in the context of "that makes this all final," which wasn't my intention. [15:26:10] Simon: Agreed, but we haven't talked about that at all yet and so far as I know don't have a plan, except for the Windows redirector. [15:27:21] There also seems to be an impression that 1.4.x is dead. From where I'm standing, I don't think that's the case. In fact, I'll need to continue nurturing 1.4 for at least the next 12 months. [15:27:22] I guess Simon's worried that redirector stabilisation will become a dafs like project? Sorry if I'm misunderstanding. I thought the plan was forward looking, and was glad to see what looks like release windows to move quickly on at least a number of large features. [15:27:24] Yeah; I don't think -announce is the place to announce a discussion. Just _have_ the discussion, on the mailing list, and be done. Announce really ought to stay low-volume. [15:27:47] 1.4 is certainly not dead in my eyes. I expect to have platforms using that for quite some time. [15:27:50] 1.4 is definitely not dead for UNIX. [15:27:57] 1.4 is pretty moribund for Mac OS X. [15:28:05] 1.4 is dead for Windows. [15:28:26] matt: Yes. My worry was that at one point there was a suggestion that the redirector merge would happen on master. So no new feature work could happen until the redirector started shipping. [15:28:37] There are significant Mac OS X crash bugs in 1.4 that my understanding is are very difficult to pull up. [15:28:58] Simon: Right, yeah, we're not doing that. [15:29:02] --- Kevin Sumner has left [15:29:05] I _think_ that the plan now is that we branch 1.6, and 1.7 branches from 1.6, and the redirector work happens on the 1.7 branch. So master can continue on its merry way once 1.6 is cut. [15:29:26] --- Kevin Sumner has become available [15:29:30] Well, we are sort of doing that in that we're pulling the redirector into master and 1.7 at the same time, but I don't expect it to hold things up too horribly much. I hope not at least. [15:29:49] sxw: not 'dead', but my impression at bpw was that maybe it would only be getting security fixes and more platform support, or something like that [15:29:52] I didn't think we were freezing for the redirector merge -- I'll put it that way. [15:29:53] If I were you, I'd pull the redirector into 1.7 and then merge to master. [15:29:57] <_cclausen> doesn't hte redirector only affect windows? Or am I mis-understanding? [15:30:04] ++simon [15:30:15] Simon: Why would you do that? It seems like an inversion of our normal repository handling. [15:30:26] Because then git will handle the pain for you. [15:30:33] it needs a branch to merge on, but that branch shouldn't break acceptance of changes [15:30:33] I'm not seeing how. [15:31:10] It's the git "commit to stable, merge to master" workflow. [15:31:12] also what he said. the way we normally do things was all done for cvs, not because it was what we wanted [15:31:15] In order to merge the redirector, it has to get rebased against current master and submitted. [15:31:27] That has to happen at some point no matter what we do. [15:31:35] I don't see how commiting it to a branch first makes that any easier. [15:31:42] sort of. the redirector branch would have merge commits [15:32:04] The redirector branch will have merge commits from 1.6. [15:32:09] But not for the actual redirector work itself. [15:32:28] --- abo has left [15:32:29] Maybe I'm confused. [15:32:31] It depends if you want to isolate the redirector work from the churn on master. [15:32:43] --- mmeffie has left [15:32:44] My point there is that I don't see how that's possible. [15:32:48] which battle does redirector need to fight [15:32:50] At some point, it has to land on master. [15:32:54] master or 1.6 [15:32:55] At which point we need to deal with the churn. [15:32:56] sure [15:33:00] --- abo has become available [15:33:01] Why not deal with the churn directly? [15:33:05] Yes. But it doesn't have to land on master before it lands in 1.7, necessarily. [15:33:21] I don't want 1.7 to have code that's not on master. [15:33:31] Well, let me put it this way. [15:33:38] I don't want to release 1.7.0 with code that isn't on master. [15:33:44] Sure. [15:33:54] So you merge 1.7 to master before you release [15:34:02] I don't really care what happens temporarily in the trees, but I don't think we're doing ourselves any favors by trying to land the redirector code as one giant merge commit. [15:34:07] Doesn't that just make everything even worse? [15:34:22] We aren't landing it as a giant merge commit, though. [15:34:32] It lands on 1.7 as whatever set of commits Jeffrey has in mind. [15:34:54] Then, at some point, those commits get merged to master. They're not a single commit - they're still in the same atomic unit as earlier. [15:35:10] And all that needs reviewed is the changes contained within the merge to deal with any tree churn. [15:35:11] The 1.7 step of that work seems like pointless makework to me. [15:35:27] --- Kevin Sumner has left [15:35:28] Why don't we just review and apply the patches directly to master? [15:35:50] We're going to have to rebase them for the churn when we pull them from 1.7 anyway; it seems more straightforward to do that directly. [15:35:55] Onto a master that's also getting all manner of other changes made to it? [15:36:20] --- Kevin Sumner has become available [15:36:28] Sure -- for one thing, most of the work is only in the Windows tree, which is not getting lots of independent work unrelated to the redirector branch. [15:37:00] For another, the problem is exactly the same so far as I can tell whether we're incorporating patches that Jeffrey has in a private tree or whether we're incorporating patches that we have in 1.7. [15:37:10] You have to do the same amount of work with the same amount of churn. [15:37:25] > Why don't we just review and apply the patches directly to master? Because we should be reviewing redirector work in small chunks as it happens, so the result of such review can be taken into account when doing further redirector work, rather than all at once when it's finally ready to hit master. [15:37:30] --- mmeffie has become available [15:37:48] I'm not convinced, at all. And my experience of getting huge patches into Mozilla only makes me less convinced. [15:38:01] To me, what we're speaking of is a merge window for redirector. Let's say its going to master. Who is, noting jhutz remark, reviewing it? Or is it in fact assert(stable)? [15:38:27] If we have a merge window, how long is the window. If short, there's nothing harmful, from my point of view. [15:38:31] At the very least, Derrick and I are reviewing the non-Windows code, and I assume other people like Simon or Andrew or yourself will want to take a look as well. [15:38:33] matt: Yes. What I'm trying to suggest is that the merge window should occur somewhere where it doesn't impinge on other development. [15:39:00] I'm afraid that I completely don't understand Simon's point, and really need to have him here in person (or be there) so that he can draw it on the board for me, since for some reason it's not making any sense. [15:39:00] --- abo has left [15:39:09] I agree logically, but I'm proposing that if redirector's merge window is one or a few weeks, it's moot? [15:39:12] Because I suspect that not only will we want to avoid destabilising changes whilst the merge is actually happening, we'll also want to avoid them whilst we do the assert(stable) stage. [15:39:48] --- abo has become available [15:40:05] Here's the whole plan as we had originally laid it out, just so that we're all having discussions from the same starting point and are talking about modifying the same thing. :) [15:40:12] Ideally, the _development_ can occur somewhere visible, with review, so that pulling it into master becomes mostly mechanical, and what you're reviewing when you do that is not "what does this large patch do", but "in what ways is this large patch different from what was already reviewed as it was being developed". And the answer to the latter is just the result of whatever churn on master since the earlier review, and most of _that_ has already been reviewed too. [15:40:14] maybe instead of a whiteboard, we need some ascii art in emails :) [15:41:04] > really need to have him here in person OK; everyone come to Pittsburgh [15:41:50] I certainly will help "review" redirector. But it's not a grading excercise. We know it's existed for more than 1.5 years, seen earlier drafts, and been through stabilising efforts over the delta. [15:42:41] One of the few areas where I hope nothing's got too different than last time I looked is to do with krb5 integration related to rxk5. [15:43:08] That has a later window though, so it is out of view. [15:43:29] 1. We branch 1.6 2. We immediately create a tracking branch for 1.7 based on the 1.6 branch, which will automatically merge all 1.6 commits. 3. Jeffrey submits the current redirect branch to master in Gerrit (or an initial set of changes for it, in which case we iterate the steps below). 4. Those changes are reviewed, modified, applied, etc., as normal in master. 5. As redirector changes go into master, we pull them up to 1.7. 6. We continue until the redirector code is merged and pulled up to 1.7, and then we release 1.7.0 off the 1.7 branch. 7. Further fixes to the redirector code are made on master first and pulled up to 1.7, and we release further 1.7 releases until the code is stable. 8. We release 1.8 off the 1.7 branch and retire the 1.6 branch, since 1.8 is at that point exactly 1.6 + redirector. [15:43:50] > it's not a grading excercise. It's not, but "I've had this code for 1.5 years, so you should integrate it without the normal code review" is a laughable argument. [15:44:15] I'm not making that argument, and I agree we should all get more involved in Windows developmetn :) [15:44:40] rra: That sounds like a fine plan. Especially if you like typing git cherry-pick :) [15:44:41] I think Simon is saying that he thinks 3 and 4 are not viable. [15:45:19] Right, we knew going in that the gatekeepers are going to have to do a bunch of work to maintain three branches for a while, although since one of them is a tracking branch, it won't be as bad as it would be without Git. [15:45:19] I think that's perfectly viable. I just think that our tools provide us with simpler ways of achieving the same end result. [15:45:29] I think 3 and 4 is going to create quite a bit of churn on master, and thus work for everyone else trying to do development against master. [15:45:40] Right, so that's the part that I don't understand. [15:45:59] yeah, IFS changes aren't going to affect me if I stay away from IFS-land... [15:46:09] I agree that there will be a lot of churn and a lot of work for everyone as the redirector branch is merged, as there will be for every other large code drop. [15:46:17] I don't see why you or Simon think that it's avoidable? [15:46:19] just like src/WINNT churn doesn't make contributing DAFS code any more difficult right now [15:46:36] I'm not saying there won't be a lot of work to do for a merge. [15:46:39] Would you rather type git cherry-pick 200 times, or git merge once. [15:46:43] I'm not understanding why you think more branches will help. [15:46:45] sure, so why do large code drops that way? why not do them on a branch where they don't affect other things, at least until they're stable enough code-change-wise to merge more or less all at once? [15:47:00] Because the git merge is way, way harder than the 200 git cherry-picks. [15:47:13] The 200 git cherry-picks are 95% trivial and obvious. [15:47:19] I think the Linux kernel would disagree with you. [15:47:38] What I'd like to see us do is trivially quantify acceptable merge delay, because, if I can't get a commit for few weeks while we merge a giant feature, I don't think it's a big concern. I've had code that waited years, that was a bigger deal. [15:47:43] I was under the impression that the Linux kernel never uses non-rebased merges. [15:47:48] > Would you rather type git cherry-pick 200 times, or git merge once. It's not just that. It's also people who have to rebase every time there's a change. I suppose for _this_ big change you can argue that there really is nothing else going on that touches the same code, but I'm not 100% sure I believe you, and that certainly doesn't translate to, say, rxk5 [15:47:52] In which case they're doing cherry-picks, just with a different tool. [15:48:12] Nope - everything that flows from subsystem maintainers to Linus is merges. [15:48:14] My experience with Git is that it's *way* less painful to rebase 200 times with small changes than to rebase once with a giant code drop. [15:48:17] Look at the kernel change log. [15:48:26] We're not going to rebase, though. [15:48:36] contributors are, which I think is jhutz's point. [15:48:38] We're starting with the same history. [15:48:45] Everything that's not already in the tree has to rebase when master changes. [15:49:18] Yes, and I just don't bother. I merge until I'm ready to push, and then rebase onto a clean branch. [15:49:58] That's the same thing, though -- if you do that multiple times over the course of the redirector landing, you have a simpler problem each time than if we land the redirector as one merge and you have to absorb all that change in your working trees at once. [15:50:28] Well, it's just the same as being on holiday for a week. [15:50:56] Right -- my point, I guess, is that if you want to wait for it to all land and then do the merge or rebase once, you can do that with either merge strategy. [15:51:06] If you want to incrementally merge, you cannot do that if we land it as one merge. [15:51:29] It just seems to me like someone has given us a shiny new steam engine, and yet we're insisting on putting a man in front of it with a red flag. [15:51:51] --- abo has left [15:52:12] Well, I guess for me the steam engine part is the fact that when the redirector work lands in Gerrit, it will be as a collection of independently-reviewable patches. [15:52:18] --- mmeffie has left [15:52:26] That's the core of the new shiny. [15:52:39] --- abo has become available [15:52:54] Under CVS, we would have to do a code freeze, a giant merge, and then a huge amount of work to fix up any other pending patches. [15:53:15] I feel like incorporating it a patch at a time is the approach that takes advantage of Git over what we would have done in CVS. [15:53:20] I just want to state that to some significant extent, a large feature should be looked at as a whole. There are some separate commits in gerrit that are unhelpful to me as sequential changesets. [15:53:31] Ie, in the redirector list that just hit. [15:55:11] I don't think that list was supposed to hit - I think Jeffrey plans on presenting it in a different way. [15:55:56] That would be fine, but I and the world have no way to infer that atm. [15:56:17] The fact that they were all abandoned I think is what says that. :) [15:56:19] they were all abandoned, which is why I thought that... [15:56:54] Well, that doesn't say they'll be presented in a different way when they eventually land, I should say. But it does imply that they weren't intended for submission right now. [15:57:21] The other big advantage with git merge, before I give up on this line of reasoning, is that it ensures that you don't miss anything. [15:57:25] I didn't get the email to that effect, but I got the email announcing their landing. We're substituting a tool for communication, I think, in any case. [15:57:39] See, I don't think it should land in Gerrit _on master_ as lots of commits. Better to do the review work as it is submitted to some other place, mostly isolated from churn, so it is easy to deal with review, update the development work according to review, etc. By the time it hits master, it should already have been reviewed in significant detail, so the _only_ thing you're reviewing for the merge to master is whatever has changed in the redirector patches as a result of things that changed in master in the interim. [15:57:49] ++jhutz [15:57:50] you probably get an email if you configure gerrit to give you emails when anything happens in any changeset [15:57:53] ++jhutz too [15:58:11] That's been my argument for a while about how we should approach landing major feature changes. [15:58:20] Personally, I want to get all changes to AFS, from whatever source, as individual patches in Gerrit reviewed separately. [15:58:39] So I guess I'm one of the people you have to convince there, since it sounds like you want something that's the exact opposite of what I want. [15:58:44] I don't understand how that makes any difference.... how does this relate to "the churn on master" if nothing else on master touches the same code? [15:58:50] No. I want them all reviewed in gerrit. [15:58:56] Just not on the master branch. [15:59:22] Oh, okay, so we're still back to my same mental block on why that makes any meaningful difference. [15:59:24] Then you merge them into master when they're done - and all you have to review on the final merge is anything that comes up from the merge commit. [15:59:50] Okay, so say we've got DAFS. [16:00:16] It's a big, destabilising change. [16:00:26] Right. [16:00:40] It's much easier to review that piecemeal on a branch, taking each bit as it comes and comitting it onto that branch. [16:01:03] I agree that it's much easier to review in piecemeal. [16:01:08] --- mmeffie has become available [16:01:10] I'm not getting the "on a branch" part. [16:01:11] You can then invite testers for the branch code, knowing that those testers will _only_ be testing dafs, not dafs + whatever landed on master that day of the week. [16:01:20] Oh, okay, I get that part. [16:01:25] The review is probably going to take months. [16:01:43] You can do that review in pieces, without constantly having to rebase in order to be able to push patches. [16:01:45] --- Kevin Sumner has left [16:01:52] > as individual patches in Gerrit reviewed separately Sure, as they are committed onto a development branch. We aren't limited to running Gerrit only for master. [16:02:04] Hm, okay. [16:02:06] --- Kevin Sumner has become available [16:02:22] Yes, gerrit for merge branches would be useful. [16:02:24] So the advantage here is that you can get the entire code onto a branch before you think it's ready for master so that it can be tested independently on that branch? [16:02:25] > if nothing else on master touches the same code? That is a large and, in the general case, unfounded assumption. It _might_ be true for redirector. Maybe. Probably not. It's not true for most large changes. [16:02:44] Yes. [16:02:51] yes, but I thought this discussion was specifically for redirector [16:02:58] the 1.7/1.8 branches [16:03:05] The redirector is something of a weird case in that to some extent some of that's already happened, but not in the way that we want to advocate going forward. [16:03:12] I've moved on to trying to convince Russ that it's a good idea, in general. [16:03:20] as have I [16:03:22] I think I'm getting through my mental block now. [16:03:28] So it's not about the patch review per se. [16:03:40] Well, it reduces some of the pressure on the review process. [16:03:45] It's about being able to review the entire change as a unit after the individual patch review, without worrying about whether artifacts you're seeing are because of master churn. [16:03:54] Yes. [16:03:59] It doesn't help with the individual patch review, but it lets you review the entire change as a unit. [16:04:12] I thought russ's counterpoint to the general idea would have been that large blob DAFS-like changes should be avoided entirely... [16:04:27] deason: Yeah, but to some extent it's not clear that's possible. [16:04:39] I mean, we keep saying that, and yet we already have four more blob-like changes that we know are coming. [16:05:15] So either we're not saying it loudly enough or what we're saying is in conflict with reality, and I suspect it's the latter because some changes really do require separate development in a blob-like fashion. [16:05:18] I don't think that's really a mark against it being possible, but it's a valid point that "we shouldn't do that, but well, we did" [16:05:44] Honestly, folks, 'large blob like' changes are really 'large coherent units of change'--the problem we created was in the (non review and) merge process [16:05:45] To some extent, major architecture changes are going to be relatively large blobs - at least until they make sense. [16:05:53] DAFS is actually a good example. I'm not sure that we could have figured out how to merge DAFS without doing it at least partly as a blob. [16:06:01] Yeah. [16:06:08] no, large parts of DAFS were definitely possible as smaller chunks [16:06:18] Oh, I'm not saying we couldn't have done better than we did. [16:06:25] But I'm saying that some of the stuff in there was always going to be blob-like. [16:06:30] Like "here, have a salvage server." [16:06:34] That's sort of inherently a blob. [16:06:34] The trick is finding the unit-of-change that makes sense. [16:06:53] pieces of functionality [16:07:01] (where possible) [16:07:04] I sometimes go so far as to say that many parts of dafs are different features that really do not need to even be related, but eh [16:07:21] Also, sometiems you have to do a bunch more work to make something a separate feature that makes sense by itself without the rest of your stuff, and it's not clear that's always work well-spent. [16:07:23] You want when reviewing to see the implications of the change as a whole. You need to accomplish review and validation in a finite time that folks agree on. [16:07:46] --- mmeffie has left [16:08:11] --- Kevin Sumner has left [16:08:19] --- Kevin Sumner has become available [16:08:28] --- mmeffie has become available [16:08:28] Like for example it's not clear that it would have been worth the effort to make the salvage server something that you could run as a separate daemon outside of bos prior to merging the bos changes to support starting the salvage server with the file server. That's kind of an artificial example, but you might get the gist of what I mean from it. [16:08:43] rra: yes, but then you can make a change that makes no sense by itself; there have been many submissions like that [16:08:51] "here is a new function that nothing calls" [16:09:00] we've done it before [16:09:10] Yeah. And I think the point of both Simon and jhutz is that reviewing changes that make no sense by themselves isn't really review. [16:09:12] andrew: why do you wish to do this? it is not making life easier [16:09:18] Or better still "here is a new function that nothing calls, but look it's got all these tests that prove that it works" [16:10:10] The problem is that you can get into a situation where you land that function that nothing calls, and then in reviewing the thing that calls it, you find that you needed it to behave differently. But you've already landed that function in master and now someone else used it because it was there, and now you can't easily change it, and everyone is sad. [16:10:28] you've driven me to drink (not really, but i'm near a decent liquor store, so i am getting some stuff to take home i can't get in PA) back later [16:11:11] Okay, so I think I'm convinced that we should set up a separate merge branch managed by Gerrit, commit the redirector to that, and then review and test it there. [16:11:12] rra: then don't merge the function until you're ready to merge both [16:11:22] A point I've tried to make, but which has not been taken up, is finite time for review and merge. [16:11:32] I would lean against calling that 1.7. [16:11:51] But maybe we should. [16:11:52] Hrm. [16:12:12] > finite time for review and merge. You mean "you must review my thing by time X, and if you don't, then I get to have it committed without further review" ? No thanks. [16:12:14] Maybe there are no real drawbacks to using the 1.7 tracking branch for that. [16:12:41] matt: for submitters, no, but reviewing things over a certain size is.... difficult [16:12:43] So we'd create the tracking branch, submit the redirector work to it, review it there, and once it's all reviewed and landed, merge to master and review that? [16:13:07] or maybe i will not leave just yet. anyway, the advantage of calling it 1.7 is we have something we can number it if we want releases people can test to prove it [16:13:09] And then from that point on, further incremental changes go to master first and cherry-pick to 1.7 like we would always do for a stable branch. [16:13:24] jhutz: No, that's not what I'm talking about. [16:13:44] jhutz: "if it's not reviewed by time X it doesn't go in to release Y" [16:14:16] deason: right, that's what I -am- talking about; and I'm talking about how we can make plans that have dates, and -we make the dates- [16:14:20] --- Kevin Sumner has left [16:14:20] or s/reviewed/working/, or whatever... eh, I'll let matt speak for himself [16:14:29] --- mmeffie has left [16:14:36] deason: no, that's fine [16:14:50] --- Kevin Sumner has become available [16:15:54] --- mmeffie has become available [16:16:47] Personally, I think date based releases make a lot of sense for open source projects. [16:17:08] +above a certain size. [16:17:12] But OpenAFS is above that size. [16:17:22] <_cclausen> I think the Ubuntu release schedule, even with its problems [16:17:24] <_cclausen> err [16:17:28] <_cclausen> I like* [16:18:30] The big advantage is that it stops you from holding off for "just another bug" [16:18:33] It makes sense to say, redirector has to do go in next, and then X, and then Y. But if hypothetically, we can't stabilise or merge redirector, we would indeed move on. [16:19:24] That's another huge advantage for Simon's branch model: if we can't stabilize a large change, much better to find that out on a branch. [16:19:27] Yeah. Which is why doing those merges out of the main development stream helps. Because backing out suddenly is more feasible. [16:19:34] If it's in master and we can't stabilize it, life sucks. [16:19:44] (see dafs for details) [16:20:02] well, we can stabilize it. just, like, about a year later than ideal. or more [16:21:23] rra: yes, that's true (branch model) [16:23:24] --- JSund has left [16:23:26] --- JSund has become available [16:23:37] --- deason has left [16:25:49] WRT Matt's comment about getting reviews accomplished in a timely fashion. One of the best solutions to that is finding your reviewer(s) before writing the code. [16:26:17] If nobody steps up to say that they're prepared to review it _before you write any code_, it's a pretty good indication that you'll have a tricky time getting it into the tree. [16:31:12] I think we all have some collective interest in forward progress, especially on things people say they agree are on a roadmap. Other personal projects, sure, whatever. [16:35:30] Put another way, I have confidence that at least one person will review any plausible-sounding thing I write, and it's Derrick. It can't all fall on him... [16:36:38] --- deason has become available [16:40:04] Okay, I just sent mail to -devel and -info. [16:41:35] In fact, I don't think what I myself just said makes sense. I was only talking originally about the need to get review on major features, not any random thing. [16:47:54] --- matt has left [16:47:56] --- mattjsm has become available [16:51:09] --- matt has become available [16:51:47] --- mattjsm has left [16:55:44] --- matt has left [17:08:50] > release-team So, is that something I want to be on? [17:13:39] Probably eventually. You're planning on building binaries for your favorite platform, right? [17:14:13] That is the plan, yes. [17:14:26] (I even have something vaguely resembling dedicated hardware.) [17:38:54] --- kula has left [17:52:45] --- mdionne has become available [18:02:42] --- rra has left: Disconnected [18:21:33] --- Russ has become available [18:53:52] Is there any reason not to rip the remaining Digital UNIX support out of the tree? We haven't supported it for a while, right? [18:54:52] Or did we only unsupport the client and want to keep supporting servers? [18:55:02] we still support the servers. [18:56:00] Okay, I'll just yank the client stuff that's left, then. [18:57:20] Well, the kernel stuff. [18:57:46] I'll leave the other bits. I suppose I could yank the afsd stuff, but I'm not sure what does or doesn't get built when there's no kernel build, and I don't want to gratuitously break the build. [19:00:41] Do we still support db servers on SunOS? [19:00:52] Also, do we still care about src/afs/SUNOS? [19:01:11] dbservers on sunos: i am probably the last consumer :) [19:02:56] Oh, never mind on src/afs/SUNOS. [19:03:00] I didn't actually look in it. :) [19:03:43] One of these days, Derrick, you'll upgrade your dbservers to BeOS and we'll have to throw some sort of party. :) [19:03:54] Or OS/2. I think they should run OS/2. [19:04:09] there's a nonzero temptation to run one on an iphone. because i can. [19:04:22] I think that would be awesome. [19:08:43] --- mdionne has left [19:14:40] Try to explain to the app store reviewers what it does [19:20:48] "It lets you locate files on the Internet. You know, like BitTorrent." [19:21:59] "But it's entirely legal" [19:22:14] Alternately: "It's the rendezvous server for a crowd-sourced cloud storage mesh network." [19:25:14] No; they'll probably reject a "server" [19:31:23] why? other software in the app store has servers [19:31:40] like, something has a web server to let you upload files to the app. [19:31:55] huh. O [19:31:58] OK, that is [19:34:17] so, you need a vlserver, a fileserver, and hartmut's disk server. then you can have a volume distributed across a bunch of phones, with each user storing a few files [19:34:24] --- shadow@gmail.com/owl55758F1A has left [19:35:12] though we'd need much better roaming fileserver support [19:39:56] --- shadow@gmail.com/owl1EA1D463 has become available [19:54:58] --- dwbotsch has left [19:55:36] --- dwbotsch has become available [19:59:36] --- phalenor has left [20:03:48] --- phalenor has become available [20:29:54] --- phalenor has left [20:39:55] --- phalenor has become available [22:06:43] --- deason has left [22:09:51] --- reuteras has become available [22:28:14] --- kaj has become available [22:35:47] --- haba has become available [23:11:16] --- kula has become available [23:24:42] --- kaj has left [23:25:37] --- Russ has left: Disconnected