- Wednesday, May 31, 2017

release-team@conference.openafs.org

Wednesday, May 31, 2017< ^ >

Room Configuration

Room Occupants

GMT+0
[13:48:58] meffie joins the room
[14:42:15] wiesand joins the room
[14:58:36] <mvita> hi
[14:59:02] <meffie> greetings
[15:01:25] <wiesand> Goood Morning USAAA!
[15:01:25] kadukoafs@gmail.com/barnowlAD8979B7 leaves the room
[15:01:25] kadukoafs@gmail.com/barnowlAD8979B7 leaves the room
[15:01:25] kadukoafs@gmail.com/barnowlAD8979B7 leaves the room
[15:02:32] <wiesand> Looks like the new time slot is working better?
[15:02:40] <meffie> yes, much.
[15:02:47] kadukoafs@gmail.com/barnowl2B662C26 joins the room
[15:02:56] <mvita> yes, thank you so much
[15:02:56] <kadukoafs@gmail.com/barnowl2B662C26> greetings
[15:04:05] <wiesand> re 1.6.21: it's about time to issue a pre
[15:04:40] <wiesand> changes looking mor or less ready to me are https://gerrit.openafs.org/#/q/status:open+project:openafs+branch:openafs-stable-1_6_x+topic:21
[15:04:58] <mvita> looking
[15:05:32] <wiesand> I have a test build with those included running on EL5/6/7 (fs, client, no dbs)
[15:05:48] <mvita> oh, hooray for linux builds working again!
[15:06:24] <meffie> yay
[15:06:29] <wiesand> I think I didn't receive a mail from your daily builders today though...
[15:06:54] <kadukoafs@gmail.com/barnowl2B662C26> The buildmaster is commenting in gerrit, thanks meffie!
[15:06:54] <meffie> it will run at 12:00 est
[15:08:01] <meffie> dont thank me yet, it's still the old buildbot, without the freebsd100 builder
[15:08:31] <wiesand> Buildbot just +1'ed 12628 ;) Thanks Mike!
[15:08:53] <wiesand> scnr
[15:09:22] <kadukoafs@gmail.com/barnowl2B662C26> Still nicer than manually forcing a build after eyeballing the grid
display
[15:09:49] <meffie> ok. sorry the buildbot 9x is not working yet. i'll continue to debug it.
[15:10:02] <wiesand> Back to 1.6.21: what's not in gerrit for 1.6.x is the ubik stuff...
[15:10:33] <kadukoafs@gmail.com/barnowl2B662C26> :(
[15:10:48] <kadukoafs@gmail.com/barnowl2B662C26> I am not sure how I failed to have a chunk of time to work on ubik
this long weekend.
[15:10:59] <wiesand> I tried to cherry-pick 12592 into my test build, but it looks hopeless
[15:11:24] <wiesand> Just beacon.c lacks a dozen significant changes since 2011
[15:11:55] <meffie> marcio has a 1.6.x version of 12592
[15:12:18] <meffie> we could put it in github for you to try.
[15:12:29] <wiesand> Frankly, I doubt that the changes under review for master/1.8 should go into the stable 1.6 release, given the amount of code skew.
[15:13:25] <meffie> that could be true for the logging changes, but i'm not sure if you want to omit the main bug fix
[15:13:28] <kadukoafs@gmail.com/barnowl2B662C26> It is somewhat worrisome, but IIRC the issues attempting to be
addressed would also be present in 1.6
[15:14:10] <wiesand> Got that, but the code is *so* different...
[15:14:33] <meffie> 12592 was a fix for sites running 1.6.x
[15:14:57] <wiesand> Mike: better put it in gerrit (even though it won;t have a chance of being merged).
[15:15:06] <meffie> ok, sounds good.
[15:15:33] <wiesand> But chances are I won't be able to see why it resembles 12592
[15:15:41] <meffie> i'm not sure you dont want to merge though. without this patch, a site can lose quorum forever
[15:16:10] <wiesand> Merge what?
[15:16:28] <meffie> the backport of 12592
[15:17:19] <wiesand> Are there alternatives? Reverting some previous changes?
[15:17:38] <meffie> the previous change fixed a data corruption bug
[15:17:47] <meffie> so that can not be reverted
[15:17:57] <wiesand> This still makes me nervous.
[15:18:18] <meffie> and alternative would be even a large change, and could require new rpcs
[15:18:37] <wiesand> How come my cell is still working?
[15:18:54] <wiesand> Sheer luck?
[15:18:57] <meffie> yes
[15:19:12] <wiesand> Nice to know...
[15:19:39] <meffie> the bug hits only are doing back-to-back vlwrite transactions, like say, moving a large number of volumes.
[15:20:11] <mvita> and a new sync site has just been elected
[15:20:15] <meffie> and the vlserver quorum is lost, on recovery, the write transactions will not allow the quorum to be reestablised.
[15:20:32] <wiesand> Seriously though, we may have a 1.6.21 with what;s currently queued in gerrit for it, in time for Linux 4.12, and a 1.6.22pre1 with the ubik stuff immediately after.
[15:21:09] <meffie> the work around is to stop the vlwrite transactions for at least 75 seconds.
[15:21:14] <mvita> that sounds reasonable
[15:21:22] <wiesand> Since I think that will require serious review and time for folks to test
[15:21:39] <meffie> yes, sounds good. we do have a work-around.
[15:21:52] <wiesand> Mike: can you spell out that "75 seconds and we're safe"?
[15:22:02] <meffie> yes.
[15:22:37] <meffie> if a vlwrite transaction hits in that window, the election process will start over.
[15:23:09] <meffie> so if you get vlwrites coming in, the elections will be restarted over and over!
[15:23:23] <meffie> that's what the bug fixes.
[15:23:35] <meffie> er the patch.
[15:23:56] <wiesand> Some sites may well prefer that over the risk of losing quorum forever.
[15:24:01] <kadukoafs@gmail.com/barnowl2B662C26> You say bug, I say patch
You say tomayto, I say tomahto
[15:24:04] <wiesand> I think mine;s one of those.
[15:24:04] <meffie> it simply avoids restarting the election, and fails the vlwrite in that window
[15:25:29] <meffie> anyway, yes, i would like to see jhutz chime in if possible. andrew has already throughly reviewed the patch.
[15:25:47] <wiesand> This issue aside, I think we'e close to 1.6.21pre1
[15:26:19] <wiesand> Will Andrew thoroughly review the 1.6 pullup too?
[15:26:24] <kadukoafs@gmail.com/barnowl2B662C26> Is there a short list of (two?) things we really need jhutz review
for, assuming that logging and memset are less interesting?
[15:26:39] <meffie> very good. yes, this bug has been around since 1.6.18, the fix can wait until 1.6.22 imho.
[15:27:43] <meffie> > Will Andrew thoroughly review the 1.6 pullup too?
yes
[15:28:53] <wiesand> Ben: Mike nominated 12592 and 12609 as the most urgent ones in last week's meeting
[15:29:57] <kadukoafs@gmail.com/barnowl2B662C26> Probably why I was remembering two as the right number :)
[15:31:10] <meffie> > Is there a short list of (two?) things we really need jhutz review for, assuming that logging and memset are less interesting?
yes, in order of importance (and merge order): 12592, 12609, 12613
[15:31:35] <wiesand> I'm a bit curious about that window size change by Jeff (pulled up as 12627). What gives?
[15:32:17] <meffie> i assume it was found during some sort of performance and/or interop testing.
[15:32:50] <kadukoafs@gmail.com/barnowl2B662C26> Yeah, if the peer sends an ack that increases the receive window, a
streaming sender could stall for 100 ms
[15:33:12] <meffie> ouch
[15:33:56] <kadukoafs@gmail.com/barnowl2B662C26> When receiving an ack, we check if the window is updated by comparing
call->next (the next seq number we would write) against call->tfirst +
call->twind (first unack'd plus window). Previously, we were doing
this check after tfirst was updated, but before twind was updated; now
we do the check after both updates.
[15:35:20] <wiesand> I understand that 100ms is an eternity here. I don't understand why that change is being contributed to OpenAFS (no matter how much it's appreciated ).
[15:35:39] <kadukoafs@gmail.com/barnowl2B662C26> I don't, either, but I reviewed it thoroughly and am happy to accept
it.
[15:36:27] <wiesand> Try to find a translation for "Einem geschenkten Gaul schaut man nicht ins Maul" ;[)
[15:36:54] <mvita> > I'm a bit curious about that window size change by Jeff (pulled up as 12627). What gives?
[15:37:06] <mvita> it may have come from dhowells?
[15:37:09] <mvita> just speculating
[15:37:27] <wiesand> Sounds plausible
[15:38:26] <wiesand> Summarizing 1.6 for today:
[15:38:43] <wiesand> Linux 4.12 is needed
[15:38:57] <wiesand> twind 100ms is wanted
[15:39:17] <wiesand> ubik fixes could wait for 1.6.22 and shouldn't delay 1.6.21pre
[15:39:42] <wiesand> Makes sense?
[15:39:48] <mvita> my German is quite poor but I guessed that one correctly…
[15:39:49] <meffie> well said
[15:40:27] <wiesand> On to 1.8?
[15:40:33] <mvita> yes, I concur on 1.6x plan
[15:40:48] <kadukoafs@gmail.com/barnowl2B662C26> I have a guess, but my cookie/javascript permissions seem to make
google translate sad, so I will have to wait for another day
[15:41:46] <meffie> the interwebs say it means "never look a gift horse in the mouth."
[15:41:51] <kadukoafs@gmail.com/barnowl2B662C26> I got my test VM set up with AFS homedir and
system:authuser-but-no-system:anyuser permissions, but can still log
in fine with a 1.8 client (from debian experimental). Need to try
again with multiple volumes in the path, though.
[15:41:56] <kadukoafs@gmail.com/barnowl2B662C26> That was my guess, yeah.
[15:42:05] <wiesand> It's basically: "If your'e given a horse as a present, don't check its teeth"
[15:42:22] <meffie> yeah, we the same saying.
[15:42:32] <kadukoafs@gmail.com/barnowl2B662C26> And need to make time to review ubik stuff.
[15:43:46] <kadukoafs@gmail.com/barnowl2B662C26> Anyone else have 1.8 topics?
[15:43:58] <kadukoafs@gmail.com/barnowl2B662C26> And I don't know if Mike wants to say more about buildbot or not.
[15:45:06] <meffie> i cleaned up the buildbot dir, to make buildbot8 and buildbot9 separate. all changes for the old buildbot go to the buildbot-8x branch in github now.
[15:45:53] <meffie> buildbot 9 ui is really different, and somethings are not working yet. i'll track it down, and ask on # buildbot if i get blocked.
[15:46:31] <meffie> seems we are stuck with 8 until after the 1.6.21 release, sorry.
[15:47:53] <wiesand> Don't be, thanks for your effort. Even in the present state it's very useful.
[15:48:15] <wiesand> I guess we're done for today?
[15:48:21] <kadukoafs@gmail.com/barnowl2B662C26> Yes, the current state is much improved from the previous one.
[15:48:25] <kadukoafs@gmail.com/barnowl2B662C26> I guess we're done, yeah.
[15:48:34] <wiesand> Please review review review https://gerrit.openafs.org/#/q/status:open+project:openafs+branch:openafs-stable-1_6_x+topic:21
[15:48:46] <meffie> yes sir!
[15:49:15] <mvita> aye aye
[15:49:16] <wiesand> It's what it takes to issue 1.6.21pre (plus a NEWS change)
[15:49:43] <wiesand> Adjourn?
[15:49:52] <meffie> task add review review review review topic 12 due:wednesday
Created task 22.
[15:49:55] <kadukoafs@gmail.com/barnowl2B662C26> Thanks everybody
[15:50:08] <mvita> so long
[15:50:20] <wiesand> Thanks a lot folks! CU same time next week.
[15:50:32] wiesand leaves the room
[16:49:09] meffie leaves the room