Home
release-team@conference.openafs.org
Friday, January 17, 2020< ^ >
meffie has set the subject to: openafs release team
Room Configuration
Room Occupants

GMT+0
[12:03:31] mbarbosa joins the room
[14:32:48] meffie joins the room
[16:57:47] wiesand joins the room
[16:59:28] <wiesand> Hay, we've got a topic
[16:59:53] <meffie> good afternoon!
[17:00:09] <meffie> i just did: /topic <something>
[17:00:16] <wiesand> Good evening (it's been completely dark ehere for a while)
[17:00:26] <mvita> hello
[17:00:33] <wiesand> Hello Mark
[17:00:51] <kaduk@jabber.openafs.org/barnowl> greetings
[17:00:56] <wiesand> Hi Ben
[17:01:44] <wiesand> Disclaimer: I probably shouldn't communicate at all this evening. Will do anyway, but please take it with a grain of salt.
[17:02:18] <kaduk@jabber.openafs.org/barnowl> Is this a "get well soon" situation?
[17:02:40] <wiesand> Partly, yes.
[17:03:04] <kaduk@jabber.openafs.org/barnowl> In that case ... get well soon!
[17:03:14] <wiesand> Another part is that my batteries are simply depleted for this week.
[17:03:36] <wiesand> Thanks.
[17:04:00] <kaduk@jabber.openafs.org/barnowl> I did find myself thinking yesterday that it felt like friday already
[17:04:57] <wiesand> Yesterday I spent a couple of hours more than planned on the motorway (and detours).
[17:06:33] <wiesand> I'm still trying to plow my way through the plethora of changes pulled up for 1.8.x
[17:06:48] <wiesand> which just grew today
[17:07:42] <wiesand> the problem is, I'd need a time slot, many hours contiguous, to get a grip on it
[17:07:59] <wiesand> which has been difficult to find recently
[17:08:31] <wiesand> and every time stuff gets pulled up (most of the time w/o paying attention to path conflicts) it gets worse
[17:08:56] <kaduk@jabber.openafs.org/barnowl> I can sympathize with it being hard to find a solid chunk of time
[17:09:00] <wiesand> maybe I'm just no longer up to it and should resign from the "release manager role"
[17:09:28] <wiesand> did I mention that I probably shouldn't communicate at all today?
[17:09:37] <mvita> please don't, I'm sure there are other less drastic things we can do to help
[17:09:44] <meffie> well, we can help with the merge confict stuff for example.
[17:09:48] <kaduk@jabber.openafs.org/barnowl> You did mention that, and I don't think you need to resign
[17:10:05] <meffie> and the news updates.
[17:10:33] <wiesand> the news updates are the least of a problem
[17:10:37] <mvita> I could send you a pony ;-)
[17:11:10] <mvita> those always cheer me up!
[17:11:19] <mvita> (meffie knows)
[17:11:46] <wiesand> well it would probably help keep the lawn in shape, but I have a sensation of it causing other troubles ;-)
[17:12:23] <mvita> not a real pony, the postage is prohibitive
[17:13:24] <wiesand> So, under the above disclaimer: Mike, it was valiant to rebase each and every change in the 1.6.x backlog to make it possible to merge stuff with ease.
[17:14:03] <meffie> yes, i can remove ones you dont think should be in 1.6.x
[17:14:21] <wiesand> Alas, a few of those I do not consider suitable for the "old stable release" at this point in time, at least not any longer.
[17:14:44] <meffie> e.g. just mark them with -2, and i'll remove them from the stack. i think we agree we dont need most of them.
[17:15:56] <wiesand> Some of those were marked "-2" before. I'll have to do it again, and I can, but it adds to the heap of stuff I have to do which has grown so large that I really no longer no where to start.
[17:16:15] <kaduk@jabber.openafs.org/barnowl> Huh, I thought -2 was persistent across new patchsets
[17:16:42] <wiesand> Doesn't prevent anyone from rebasing other changes on top of those…
[17:16:59] <kaduk@jabber.openafs.org/barnowl> True
[17:17:03] <mvita> ah
[17:18:10] <wiesand> er, *know* where to start
[17:18:51] <wiesand> but confusing "know" for "no" was quite possibly a freudian failure…
[17:18:52] <meffie> ok, i can just go though them and pick out the important ones, based on the gerrit history and comments, hows that sound?
[17:19:27] <meffie> and make a new stack that has just a few ones we think we want based on gerrit comments.
[17:19:47] <wiesand> actually, I've been pondering thoughts this afternoon…
[17:20:11] <meffie> e.g. andrew pointed out some that are real bugs, unlike others that are "cleanup"
[17:20:34] <meffie> ok
[17:21:14] <wiesand> I think that 2 weeks ago I mentioned that it would be one of the more attractive options for me to "restart" 1.6.24 from scratch.
[17:21:51] <meffie> i guess i'm saying that ;)
[17:22:27] <meffie> but without making new gerrits, just rebasing the ones we have.
[17:22:53] <meffie> cherry picking that is.
[17:23:53] <meffie> would it be better for you to have new gerrit numbers instead?
[17:24:25] <wiesand> Frankly: While I really really appreciate the lot of work you sank into that rebase campaign: I could just merge all that stuff and create a release - but I wouldn''t consider deploying that release on the systems I'm responsible for.
[17:25:34] <wiesand> Maybe we should aim for smaller steps.
[17:25:42] <meffie> yeah, that makes sense.
[17:26:42] <meffie> how about i make a new stack that has just the serious bug with small fixes?
[17:27:05] <meffie> to start fresh?
[17:28:06] <wiesand> That's one of the thoughts that came to mind. Go back to the last stable releases (which were security ones), and reconsider what we absolutely *need* in the next ones, to get those done and pushed out already.
[17:28:34] <meffie> that sounds like a good plan to me.
[17:28:40] <kaduk@jabber.openafs.org/barnowl> If you use new gerrit numbers for the fresh start, then we can keep
the old stack around as a reference for how things are ordered with
respect to each other, if we want to start changing what's in the
smaller stack
[17:29:16] <meffie> that sound like a good plan. gerrit numbers are cheap right, we can just make more?
[17:29:35] <wiesand> It would make the gerrit view more (actually completely) unwieldy though.
[17:30:23] <kaduk@jabber.openafs.org/barnowl> gerrit numbers are cheap, but view space is not.  Hmm, I wonder if
topics are robust enough to help
[17:31:34] <meffie> what if we abandon the current "open" gerrits, and just restore the ones we want?
[17:32:08] <meffie> then just view status:open ?
[17:32:21] <wiesand> Frankly, I believe this calls for either creating branches or abandoning everything already pulled up (and reverting anything merged since the last release) and restarting the cherry-picking process.
[17:32:37] <wiesand> Mike: yes, something akin' to that.
[17:32:49] <wiesand> Give me a week to think this through?
[17:33:31] <mvita> sure, when you're feeling better
[17:33:33] <wiesand> Of course anything like that would add greatly to the code skew which is already bad.
[17:34:30] <meffie> ok. since if we are not going to take them, gerrit has a way to deal that that by setting the status to "abandon", and i have see that is reversible if we change our mind later. and the abandon ones do not show up on the list of open gerrits, which makes your view better.
[17:34:39] <wiesand> So another thought: maybe focus resources on 1.9/2.0 and keep movement in stable releases to the absolute minimum?
[17:35:55] <mvita> just security releases and such
[17:36:11] <meffie> yes, sounds good.
[17:36:45] <mvita> I'm not averse to that; I think we've already indicated (either explicitly or implied) that 1.6.x is going to be winding down
[17:36:59] <wiesand> Do not abandon stuff right away though, please. I'd like to have a record handy of what we currently have queued, and I'm not sure that gerrit provides a convenient way to get to that.
[17:37:17] <meffie> ok
[17:37:49] <kaduk@jabber.openafs.org/barnowl> The gerrit query cli/ssh interface probably has something that would
be a convenient way to get a list
[17:38:04] <wiesand> Actually, 1.8.x has become (almost) as frightening/overwhelming as 1.6.x …
[17:38:14] <mvita> status:abandoned
[17:38:22] <kaduk@jabber.openafs.org/barnowl> e.g., I have in my shell history
ssh -p 29418 gerrit.openafs.org gerrit query --patch-sets --comments
project:openafs branch:master status:open limit:400
[17:38:59] <wiesand> I don't envy you ;-)
[17:40:34] <kaduk@jabber.openafs.org/barnowl> It's useful for when I'm preparing to work on a long plane flight
without wifi, since I have all the changes fetched locally into git
but need to know what to look at.
[17:42:49] <wiesand> If I fly at at all (and I try to avoid that if at all possible), it's economy - where I am simply unable to work at all because there's not enough space between myself and the back rest before me to even open a notebook ;-(
[17:43:31] <wiesand> at least not to an angle allowing to read the display
[17:43:53] <kaduk@jabber.openafs.org/barnowl> Yeah, most of my laptops do not work.  The XPS13 is just small enough
that it's workable, if I have a place to put my elbows
[17:44:22] <wiesand> place to put your elbows, in economy?!
[17:44:37] <wiesand> god one
[17:44:38] <kaduk@jabber.openafs.org/barnowl> a bit dicey, yes
[17:46:08] <wiesand> back to daily business, I noticed the Catalina changes were pulled up to 1.8.x
[17:46:33] <wiesand> that's good, but I yest have to change for new path conflicts ;-)
[17:46:46] <wiesand> and chances are there are some
[17:47:42] <wiesand> apart from that, I think nothing too serious has happened during the last week?
[17:49:13] <meffie> no other bug reports i know of. i still need to find out more about the "full partition" report we talked about last week.
[17:50:08] <meffie> the openafs foundation announced the cfp deadline has been moved up
[17:50:33] <meffie> the cfp for the workshop in june in columbus ohio
[17:53:20] <wiesand> [please read disclaimer above] this usually means that indicated participation so far is a bit less than desired?
[17:53:35] <kaduk@jabber.openafs.org/barnowl> Not too much to report from here.  I think I merged a few things that
had needed rebasing from last week, and have a few things open in
browser tabs to look at later today.
[17:53:49] <wiesand> ok
[17:54:02] <meffie> no, i heard they wanted to publish the schedule sooner so people could get approval to travel.
[17:54:41] <kaduk@jabber.openafs.org/barnowl> The deadline is closer to now than it used to be, is my understanding.
[17:54:41] <wiesand> that's fine (last year's was too tight IMO)
[17:54:49] <mvita> so what's the (new) deadline?
[17:55:20] <wiesand> from memory, Feb 15… hang on please…
[17:55:25] <meffie> 3 or 4 weeks from now if i recall correctly?
[17:55:46] <mvita> oh, okay - it was originally Mar 15, I didn't see a change
[17:56:08] <wiesand> [OpenAFS] Call For Talks - Feb 15 Deadline *Note Change* - OpenAFS Technologies Workshop 2020
[17:56:27] <meffie> oh, also andrew is working on some changes to improve the situation with pag throttling on linux.
[17:56:29] <wiesand> So that we can announce talks with time for attendees to make travel
plans, we have moved the Call for Talks deadline to *February 15* .
Talks will be announced the following week by February 22.
[17:56:49] <mvita> Oh.  I just found it - in my linux-afs folder
[17:56:56] <mvita> rats
[17:58:48] <wiesand> Mike: good thing probably, but yet another topic sounding fairly intrusive which should probably deferred to 2.0+ ?
[17:59:51] <meffie> not sure, i would think it would be appropriate for 1.9.x at the least.
[18:00:18] <wiesand> ah, moving the deadline from march to february would indicate that there is no anticipated shortage of contributions - good!
[18:00:26] <meffie> correct.
[18:02:44] <wiesand> hmm, I thought 1.9.x was to consolidate the the current master branch (actually, as of a couple of months ago), mostly to get rxgk-phase1 out to the masses?
[18:03:46] <kaduk@jabber.openafs.org/barnowl> Yes
[18:04:03] <wiesand> is "pag throttling on linux" an issue I'm likely experiencing in my cell today?
[18:04:33] mbarbosa leaves the room
[18:04:40] <mvita> you would know it if it was
[18:05:01] <meffie> yes, it would be painful if you hit it.
[18:05:18] <meffie> it is meant to slow down the system :)
[18:05:26] <mvita> severe (but short-lived) performance degradation, usually right after a restart of the client
[18:06:42] <wiesand> Thanks for the info. I'm afraid I still think tackling that is a 2.0+ thing ;-(
[18:06:54] <meffie> ok
[18:07:32] <mvita> it only happens in a high parallel load (HPC or equivalent) environment
[18:08:24] <mvita> lots of users all wanting PAGs at once
[18:08:47] <wiesand> per-client or per-cell?
[18:08:55] <mvita> per client
[18:11:00] <meffie> the throttling only makes sense when using gid based pags. modern clients use keyrings, so it is no longer needed and can kill performance.
[18:11:36] <wiesand> hmm, sounds like this either requires a fairly gigantic client or a scheduler capable of starting split-second jobs at a MHz rate
[18:11:38] <meffie> we'd have to see the patches to make a determination on how risky the change is
[18:11:46] <wiesand> of course
[18:12:28] <meffie> just to let everyone know.
[18:12:29] <mvita> >   wiesand> hmm, sounds like this either requires a fairly gigantic client or a scheduler capable of starting split-second jobs at a MHz rate     yes, that's it
[18:12:50] <wiesand> the latter?
[18:13:01] <mvita> either one, or both can do this
[18:13:09] <mvita> more likely right after a restart
[18:13:55] <mvita> since the throttling is based on a rate threshold - the less time the client has been up, the higher the rate for a given load
[18:14:15] <wiesand> Ah.
[18:14:36] <mvita> so clearly a sub-optimal algorithm
[18:14:45] <wiesand> Ok, if such environments actually exist in production, I understand and feel the pain.
[18:14:46] <mvita> heuristic, I mean
[18:16:19] <wiesand> Our most performant HPC nodes may have 64 job slots, and the scheduler will be able to start ~1job/s at best. I guess that wouldn't cause a bottleneck?
[18:16:30] <mvita> not a problem
[18:17:02] <mvita> this is on the order of thousands in a few seconds
[18:17:11] <wiesand> So yes, I feel the pain, but it sounds "a bit uncommon".
[18:17:24] <mvita> you're not wrong, as the kids say!
[18:20:08] <wiesand> But again, "low risk, little churn" changes can of course make it into stable releases rather easily. It's just that "pag throtteling" didn't immediately sound like that.
[18:20:41] <meffie> ok, sorry for the scary sounding topic. we can move on :)
[18:23:06] <wiesand> Bringing such topics up at least 7 weeks before any decision on them has to be taken will make it much (orders of magnitude) more likely for them to be accepted or at least discussed rationally ;-)
[18:23:43] <wiesand> It's getting late… anything else to discuss today
[18:23:45] <wiesand> ?
[18:23:51] <kaduk@jabber.openafs.org/barnowl> not frrom here
[18:23:56] <mvita> not from me
[18:24:03] <meffie> not here. have a good weekend!
[18:24:09] <mvita> thanks everyone!
[18:24:23] <wiesand> Thanks a lot for listening to my whining today!
[18:24:45] <mvita> heh
[18:24:48] <meffie> not a problem, thanks for making the time.
[18:24:50] <mvita> was there whining?
[18:24:57] <wiesand> oh yes
[18:25:12] <wiesand> Thanks again. CU next week.
[18:25:19] wiesand leaves the room
[18:25:31] <kaduk@jabber.openafs.org/barnowl> Thanks everyone!
[18:28:06] mbarbosa joins the room
[18:43:05] meffie leaves the room
[20:45:24] mbarbosa leaves the room
[21:26:12] mbarbosa joins the room
[22:48:14] mbarbosa leaves the room