[00:41:56] --- dev-zero@jabber.org has become available [00:41:59] --- dev-zero@jabber.org has left: offline [05:10:52] What's the current wisdom on the number of callbacks to have configured on a 1.4.x fileserver? [05:15:15] more is better [05:16:12] How many is too much? [05:16:35] how much memory do you have? [05:17:14] Just memory limited, then? No performance degradation due to long hash chains or anything like that? [05:17:36] churn causes more performance issues than i have seen from the chain depth [05:17:58] (GetSomeSpace looping) [05:18:47] We've got 3Gb on our fileservers. [05:19:09] more memory was being used by the former rx leak [05:19:37] how many hosts do you have per server? [05:21:13] Maximum we have ~1200 clients at present. In theory, home directories are evenly distributed across 4 servers, but worst case we could have all of the clients using homedirectories on the same server. [05:21:46] my goal would be to permit every client to enumerate at least one directory with 30K entires within five minutes without churn. [05:26:55] --- Jeffrey Altman has left: Replaced by new connection [05:27:23] --- Jeffrey Altman has become available [05:45:31] And while I'm here. # of processes? I know there was a historical limit here - did that go away in 1.4.x? [05:46:28] you mean number of threads? [05:46:49] Yes. [05:47:26] (documentation still calls it # of processes - that's my excuse, anyway) [05:47:36] the max is somewhere between 111 and 127. I forget how many background threads we now have [05:47:44] --- matt has become available [05:47:52] So '23' is probably too small then :) [05:47:55] I believe -L will now do the "right thing" and set it to the max [05:48:52] Ah. If that's the case, I suspect some documentation updates are in order. The fileserver manpage claims that -L gives you 12. [05:50:57] if (SawL) { if (!Sawrxpck) rxpackets = 200; if (!Sawsmall) nSmallVns = 600; if (!Sawlarge) large = 600; if (!Sawcbs) numberofcbs = 64000; if (!Sawlwps) lwps = 128; if (!Sawbufs) buffs = 120; if (!SawVC) volcache = 600; } [05:51:19] Just found that. Shall I write a documentation patch? [05:51:31] if you have time. please do [05:51:42] Not really, but it won't take long. [05:54:35] besides the fact that the IBM navigation graphics are so 1996, is there anything else that needs to be done to http://docs.openafs.org/ before it goes live? [05:56:15] make it a wiki? [05:56:21] use a sans font? [05:56:36] Definitely don't make it a wiki. [05:57:17] matt, are you kidding? [05:57:47] I don't care hugely about font issues, but serifed fonts are apparently more readable for long sections of prose (something about the serifs providing a visual cue to make you read along the line of text) [05:58:29] I suspect that we are not specifying any fonts and the browser is using the default [05:58:51] Partly. Having a modern wiki up is something I would like. [05:58:59] I think when we have a modern OpenAFS web style, we should probably apply it to these documents too, with a different printable skin. [05:59:13] what does that have to do with docs.openafs.org? [05:59:24] (that was for matt) [05:59:30] I didn't know about docs. until today, so ok, not much. [05:59:36] But for now, it's immeasurably better than what we had before, so I'd put it live now, and tinker later/ [06:00:45] matt, all docs.openafs.org is the openafs cvs doc/xml tree processed with xsltproc to produce html output. The goal is to retire the IBM docs. [06:01:02] I see now, sorry to create confusion. [06:01:23] At which point, I'll do a little dance, because it means that my first contribution to OpenAFS has finally made it into the world. [06:01:46] simon, I am truly sorry for how long it has taken [06:02:04] It's nobody's fault. I just find it amusing :) [06:13:17] in the last year there have been just 10,000 search queries issues from the www.openafs.org home page [06:13:37] I _so_ want gerrit. Something is very wrong when time to submit the patch is about 10 times the amount of time taken to write it. [06:13:49] jaltman: Against how many hits on the OpenAFS homepage? [06:41:59] > my name's not shadow. And it's certainly not shadow@gmail.com/owlXXXXXXXX [06:43:19] > i don't really understand why you'd > ever want to address someone by userid regardless. I don't think I'd go that far. [06:50:05] simon: I don't have a home page hit count. the search count is given to me by google. [06:51:03] actually. I do have numbers. hold on [06:52:17] ~90,000 in the last year [07:02:42] --- cclausen has left [07:56:25] since Jan 1, for hits on the home page. ~38% direct, ~35% from google search, ~4.5% from wikipedia, ~1.5% from sochop, ~1% from freshmeat. There are also about 1000 hits which are referrals from an ibm.com site. [07:58:35] only 36% of our hits are from returning visitors [07:59:51] sounds like a good percentage of new/potential users [08:00:12] or lots of web crawlers [08:00:48] Any real potential user is going to show up as a returning vistor though. [08:01:01] "it depends" [08:01:25] a real real user may never come back to the web site. why bother? download updates when they're available. install them. works. don't care. [08:03:49] 53% of visitors are on Windows. 28% on Linux, 16% on Mac, less than 1% in order SunOS, unknown, FreeBSD, iPhone, OpenBSD, NetBSD, iPod Touch [08:05:40] top 10 network providers: road runner, comcast, deutsche telekom, MIT, verizon, proxad, ucdavis, ibm, cmu, brasil [08:07:55] unfortunately, none of the release pages, documentation or the mailing list archives are wired for analytics. so I can't really get a feel for who is reading or downloading [08:11:06] since jan 1 the home page was read 46,100 times. The doc index page 17,500, the latest.html page 16, 600, the windows page 16,100, macos 13,100, [08:11:27] I think that gives some rough idea [08:12:41] the 1.4.8 page 10,400, 1.5.57 page 5000, 1.4.10 page 4,450, gsoc 2,250 [08:13:33] Are reads page downloads, or unique readers? [08:14:41] these are page view counts. [08:15:12] a page view is either a reload or switching to another page/tab and returning. [08:15:32] only javascript enabled browsers are counted [08:15:52] Our conversion rate for GSoC wasn't great this year ... [08:16:17] we can wire the release pages easily [08:26:03] I do not believe that all of the gsoc.html viewers were students looking to participate [08:26:36] the vast majority of the hits are either pre or post application period [08:27:52] only 770 hits over that period from sochop and those are split among the home page and the gsoc page [09:36:03] "I would have provided support for a run time loadable table of expiration times that can be specified by the server admin." [09:37:19] I think if I had provided that, people would have felt it was too elaborate, but I'm interested in feedback. I'd lean towards just viced options which set the longer behaviors, and have a safe schedule. [09:38:26] how do you know what is safe for my environment? [09:38:55] How do I know that the default set of timeouts AFS has provided for the last 20 years is safe for anyone's environment? [09:39:01] I'd just really like to see this behaviour be configurable by runtime. [09:39:06] at runtime, even. [09:39:39] There doesn't seem to be any point in it being a #define, given that you get the function signature changes whether you have it enabled or not. [09:39:53] Yeah, that's true, your point is taken. [09:40:00] I'm not saying they are safe. I'm saying that if they are going to be made configurable, making them configurable based upon the desires of one large user is not appropriate. [09:40:14] I'd say that it's a step forwards. [09:40:41] If they're configurable between 2 defaults, then that's better than only having one. And someone else can add the additional knobs when they have the need for them. [09:40:47] how hard is it to read a configuration file if it exists? [09:41:38] --- abo has left [09:41:44] --- stevenjenkins has left [09:42:05] It's not. But if we're going to have the fileserver use a configuration file, then we should have a proper conversation about what should be in it, and what format it should take. [09:42:25] --- abo has become available [09:42:44] That's my instinct too. [09:43:34] --- reuteras has left [09:44:24] --- reuteras has become available [09:44:25] I know that there is one site that currently mods the expiration tables for their closed environment. The proposed changes would not solve their issue. [09:44:26] and it's going to be icky and complicated and we're going to be cyrus [09:45:00] Let's not be Cyrus. [09:45:07] --- stevenjenkins has become available [09:45:19] and I really do not think we want to end up with N versions of the table in the tree [09:45:21] Or at the very least, let's have a set of defaults that work well for the majority of users. [09:46:03] self-tuning would be better [09:46:17] Indeed. Self tuning for _everything_ would be better :) [09:46:26] the expirations should not be strictly table driven. they should be based upon the likelihood that the data is going to change. [09:46:35] Anyway, gone to drink... [09:48:33] I would hope that we can a) actually make an improvement b) not expand scope into self-tuning heuristics [09:49:58] do you have data to show that these values you are proposing are optimal under a broad range of use cases? [09:50:20] No. They happen to be based on Derrick's intuitions. [09:51:57] "i guessed" [09:53:13] But if there actually is a need to have multiple alternate tables, I guess that's plausible. Is a file in the server directory that -doesn't- try to be the configuration file for viced in general, an acceptable option? [09:55:09] jeff's answer will be different from mine i bet [09:55:33] I would not be opposed although I would prefer an attempt at proposing a configuration file format with a firm timeout period on that effort. [09:55:50] no more than two weeks [09:56:11] three weeks so that we can discuss it at the workshop [09:56:19] You mean, you would like to review a file format specification at the workshop. [09:56:49] or better yet have it implemented by the end of the workshop [09:57:06] Well it's certainly not a hard problem, and I guess it's motivated. [09:57:07] the config file parser we use for windows is open source, right? same format as mit krb5? [09:57:32] we could use the mit profile library if you wanted to [09:58:02] windows doesn't use config files anymore [09:58:14] if we are doing to reinvent the wheel "why" is the first question to ask [09:58:16] and they were microsoft INI files [09:58:50] I do not want to re-invent the wheel. I think the question is "which existing file format do we want to use?" [09:59:14] Does krb5 profile meet the need? It seems like it would. [09:59:39] Windows INI files are fine. mit profile files are fine. an XML based configuration would be a lot of code to pull in [10:00:03] I would also really not like to edit XML to configure AFS servers. [10:00:07] xml: ick [10:00:37] mit profile format is very flexible. the library permits combining files so that it is possible to manage configuration as a combination of general and more specific profiles. [10:00:56] --- reuteras has left [10:00:56] --- abo has left [10:01:05] I would prefer that to most things I can think of. [10:01:08] --- stevenjenkins has left [10:01:11] --- reuteras has become available [10:01:21] but the library that mit ships is hideous and the api has some fatal issues. [10:01:33] --- abo has become available [10:01:35] none of the "writing to profile" functions are safe [10:01:47] Does one need to write to the profile? [10:01:52] and there is no notion of "ordering" in the profile [10:02:04] Hm. [10:02:19] I want to be able to update configuration remotely via bos [10:02:45] that is not a requirement for day one but I want it to be possible. [10:03:33] if order is important, then that information needs to be encoded in some manner. [10:04:38] --- stevenjenkins has become available [10:08:16] for example: num_hosts = 3 server_1 = host1 server_2 = host2 server_3 = host3 and then read each of the individual items instead of server = host1 server = host2 server = host3 and then simply reading "server" and getting back three items in the list [10:10:04] Here's a library that even has a name! http://augeas.net/docs/index.html [10:11:33] That seems to have the purpose of editing random config files in other formats. I guess that doesn't stop defining one. [10:19:53] Looks like libconfig is another overkill solution. It's LGPL. [10:24:06] Of course, there could just be a runtime option that let you name your desired vector of timeouts. Crazy? [10:24:42] --- dev-zero@jabber.org has become available [10:31:30] a runtime switch with a comma-separated list is probably simplest. unsure if i like it, but simple, yes [10:31:37] Configuration file aside (project for workshop or whatever). If I created "-cbtimeouts" option which parsed a comma-separated sequence of the expirations, allowing abbreviations for units, e.g. "24h,12h,4h,1h,45m,30m,15m" would that not be acceptable in its own right? [10:32:42] The usage would remind what the slots represent. [10:32:53] --- dev-zero@jabber.org has left: offline [10:39:09] I would love to see 'simple vector of human-readable timeouts' get some props. [10:51:05] --- Russ has become available [10:55:36] command line is fine but to be honest I think the command line configuration of our services has gotten out of hand. [10:56:12] Actually, your idea to do the config file right during the workshop is a good one. I just want to decouple this from that. [10:56:34] why? what deadline do you have that requires it be done today? [10:57:43] I'm sure your client is already running with a patched build. [10:58:09] I have three other much larger features from this group to work on. I'm in the midst of factoring out the fetchvstatus stuff, and trying to think out how I might make -that- upstreamable, and it's a bigger problem. [10:58:26] you have already submitted this one. move on. [11:08:57] yeah, really, if this is *just* cb timeout stuff, suspend it for now, and come back to it [11:09:16] tho we should have the "what config file format/library" discussion soon/now on openafs-devel [11:09:51] Yeah, ok. I have the bigger diffs coming. The last thing I want to care about is how to do their configuration... [11:10:38] I've been trying to think how to do the SyncCounter piece of fetchvstatus, which is the diff I'm cleaning up now. -That- is actually a conundrum. [11:11:02] well, yes. as it was when i thought i could use that [11:11:46] Am I correct that no one uses SyncCounter now, but there are the two claimants? [11:12:08] yes [11:12:09] --- stevenjenkins has left [11:13:03] two organizations with deployed code [11:13:34] --- abo has left [11:14:26] --- abo has become available [11:15:52] --- stevenjenkins has become available [11:15:54] at least [11:15:56] In the can-do spirit. [11:16:09] well, istr osd uses it as a flag [11:16:10] I think I would propose, upstream could in principal accept -both- (or all, as the case may be) changes. But only one interpretation of SyncCounter could be selected at configure time. [11:16:23] i'd propose no [11:16:44] Does http://bugs.debian.org/528785 look like a familiar crash in 1.4.10, maybe even something that already has a delta? [11:16:52] which is why my gut reaction is to accept neither in its current form and force the standardization of a new RPC for FetchStatus that will include additional fields that we know are required. 64-bit time values 64-bit volume version 64-bit osd version 32-bit extended attributes 32-bit stream count and whatever else we have pending [11:17:45] maybe a file checksum [11:18:40] It's obviously time, and people are working on such proposals. [11:18:42] russ, yes [11:19:27] linux26-defer-cred-changing-20090511 [11:19:35] Awesome, thanks. [11:20:09] and yes, given that no widely deployed clients can use SyncCounter for either, stealing them versus just adding a new RPC... add the new RPC [11:22:05] other things which could be useful, if we also assumed they'd be in other rev'd RPCs: a uuid for the fileserver. a 32 bit transaction id, to allow a context to be maintained (e.g. one request from a client would use the same transaction id across all RPCs it generated; each client could manage its transactions however it wanted) [11:22:18] i guess it's not clear the reply would need it, only the request [11:23:43] I think that suggestion there shows, things could go in all sorts of directions. [11:24:24] the sooner we start talking about it the sooner we have consensus and can move on it [11:26:49] Well, Tom has been writing a draft, I haven't seen recent versions. [11:29:16] I think the real issue is, we're -not- actually going to get consensus on the future of the entire interface this way, it has to be possible to make progress piecewise. [11:29:40] yes. hence my suggestion of "get consensus on a new fetchstatus RPC, only" [11:29:43] *first* [11:30:07] we have to be able to get consensus on RPCs or we can never have progress ever [11:33:45] Even that will be hard. I'm not ready to argue about transaction semantics, but I'm convinced (as is Tom) that somehow this protocol needs to incorporate flexible extensions that permit experimentation in different directions. How do we propose something soon that doesn't foreclose options? [11:34:21] i don't expect to argue about transaction semantics. i expect to reserve 32 bits for it. [11:34:34] and worry about how it gets used later. and add a capability indicating such [11:36:13] notably, the only argument it's interesting to have is whether 32 bits is enough [11:37:05] Not at all, I would say. [11:37:34] a 32 bit txn id is not large enough? [11:40:01] What I'd propose is that we define an interface that is highly abstract and extensible, I would assume using unions to reserve space for future developments, but not claiming the space. [11:54:28] --- abo has left [12:38:35] --- reuteras has left [13:29:03] --- tkeiser@sinenomine.net/owl has left [14:03:31] --- tkeiser@sinenomine.net/owl has become available [14:15:28] --- nelhage@mit.edu/lunatique has become available [14:25:42] --- andersk@mit.edu/vinegar-pot has become available [14:51:09] --- geoffreyerffoeg@gmail.com/owlE4C5279C has become available [15:50:24] --- Russ has left: Disconnected [17:14:18] The danger with a highly abstract and extensible interface is that you end up being NFSv4, and no one vendor implements all of you, ever - and two builds in different organisations end up using a ridiculously low common denominator just to communicate. [17:15:07] Achieving consensus is hard, sure. But I think it leads to actually making engineering decisions. Designing something that's flexible enough to be everything to everyone just defers making those decisions. [17:22:55] I take the point, flexibility or abstraction taken to an extreme is bad. But I'd like some flexibility, probably along the lines that is in the xcb interface--specifically to ease future enhancements and exploration. [17:24:07] I would not be so worried about it--if I didn't have constant reminders that AFS protocol interfaces historically, once created, are terribly difficult to update or change. [17:24:43] I also have the counter point, that the reason why nothing ever gets done round here is that we have a tendency to postpone small, incremental, improvements in favour of "one big rewrite". Needless to say, there are never any resources available for the rewrite, and so all of the small things never get done either. [17:25:16] You can't think I'm arguing against that point, right? [17:25:27] Nope. That was me arguing against myself. [17:25:56] I strongly agree--if we -can- make an incremental improvement here, and improve it later in a year, well...yahoo. [17:26:40] I strongly believe that we need to find a way, and soon, to facilitate incremental change. There are far too many changes that get deferred because they're not the 'right' way to do things. But nobody ever has the effort to do the 'right' way. [17:27:27] I'd amplify on that, but, absolutely. Yes. [17:28:04] Plus, there frequently isn't one true way, and people argue extremes. [17:28:25] I don't think the solution is easy, because we don't have a commitment, in the stable tree at least, to make things easy for our users. And constant feature shift and reimplementation doesn't make things easy for them at all. [17:28:39] Sorry - s/we don't/we/ [17:28:53] --- stevenjenkins has left [17:29:40] But, Open Source is generally based around incremental improvement. And we need to be able to do that, otherwise any prospective of attracting non-funded developers is pretty much dead in the water. [17:30:55] Well, not only that, an AFS that cannot modernize is...not going to make it. [17:31:04] Indeed. [17:32:34] --- stevenjenkins has become available [17:32:47] But, I'm really interested in which our codebase can grow, modernize, and evolve, whilst remaining an Open Source project. I think you can do big-bang rewrites if you have the money to pay for that developer time. I just don't believe that they work in a space where you have unfunded developers, or developers that are project funded for small development tasks. [17:34:24] That's certainly one aspect, yes. I think there are a lot of barriers to change, but that's a big one. [17:36:03] For instance, in the current debate, I really like your proposal for a human readable vector of values. (Well, I feel it does provide yet another set of daunting knobs, but providing the defaults are solid, and the knobs are documented, I can live with them) [17:36:57] I'd also really like to see a configuration file format to reduce how out-of-hand our command line switches are getting. But I'm not sure that I want the creation of this file format to block what is a pretty simple feature from the tree. [17:37:46] And I can see "by the workshop" turning in to "write a proposal at the workshop", and then the code to actually implement the thing never being written, and another simple feature bites the dust. [17:46:23] Well, I agree. I think there is an obvious way to have the command line improvement now, and a config file that satisfies everyone when it's ready. [17:46:54] Agreed [17:55:18] --- kaduk@mit.edu/owl has become available [17:55:58] On that note I've mostly finished the xcb split out work I've been doing. The base level of that is a reduced version of Tom's libosi library. The current level still needs a bit more reducing, and there are a few ancillary things to do. However, I think it's nearing readiness to have some more discussion about. [17:56:27] I'd like to see the reduced libosi. In particular, if it has support for atomics. [17:57:16] One of the things I've been staring at recently is the dcache tlock, and thinking how it could go away (on Linux) with a careful application of atomic_inc, atomic_dec, and atomic_read [17:58:24] Well, that's one of the areas. The library has an interface that is basically that of solaris. I think Tom and I are both moving in the direction of using MCAS for msot of that--that's used by xcb, it's a set of mid-level atomic ops and lock-free algorithms using them. [17:58:53] Where the OS provides native mechanisms, I think I'd like to see us use them. [17:59:09] So, for example, use Linux's atomic() operators, rather than growing our own. [17:59:40] Well, thing is, what if you already have them all--and you'd like to say, use the same operators on all x86/x86_64 Unix variants we support? [18:00:07] But that's possible too--currently for solaris, I layered MCAS' interface on solaris atomic ops. [18:01:30] I'm not sure I see the value in using the same operators on Linux and BSD (say) when that stops us from using the same operators as every other Linux kernel module. But I'm happy to have that discussion. [18:02:44] Anyway, must sleep now ... [18:02:50] Night [18:10:54] For posterity, I think worrying over which macros to use for atomics would just mean, we can't merge xcb or other code unless it's proven mechanisms are replaced with some other ones. (And I'm using MCAS in viced and volser, atm, though the atomic ops per se don't care, its data structures tend to use a gc mechanism which I think could go in kernel, but presently isn't.) Hopefully, we can be flexible, and also remember than when I started, I had gatekeeper support to go in this direction. Anyway... [18:11:16] --- matt has left [18:41:31] --- Russ has become available [19:51:17] Which page is this, and where are you getting these statistics. [19:54:51] (stark) xml: wrong Seriously. XML is fine for data files intended to be read _and written_ by machine. For things intended to be created and edited by humans, it is inappropriate. [20:30:08] what about both? [20:45:06] If it's intended for humans to be able to edit it, then it must actually be editable by humans. If it's also going to be read and changed by software, then it is up to the software to adapt to the humans' needs, not the other way around. [20:45:11] good night [21:05:30] --- broder@mit.edu/vinegar-pot has become available [22:12:39] one of the conditions for enabling incremental improvements is providing the configuration capabilities to permit experimental features to be added to binary distributions such that they are disabled by default and can be easily enabled. creating a configuration file will make this much easier to do. I would be fine with any of: Kerberos profile library; Windows INI file format; or OpenSSL files. Support for any of these could be pulled into the source tree without license compaibility issues. [22:13:32] a free MIT licensed Win INI parser is available at http://ndevilla.free.fr/iniparser/html/index.html [22:28:36] openssl files? ick [22:35:35] > we have a tendency to postpone small, incremental, improvements in favour of "one big rewrite" [22:36:02] while true, i think we can do incremental improvements by doing e.g. "make new RPCs which can accomodate everything we plan; turn on a bit at a time" [22:37:00] (assuming, of course, it's extras and not basics you are turning on later)