[00:53:18] --- Russ has left: Disconnected [01:18:21] --- summatusmentis has left [07:08:36] --- dmontuori has become available [07:11:41] --- matt has become available [07:42:03] --- RedBear has left [07:53:43] > i wouldn't have tried that anyway Maybe not, but given what I was trying to accomplish and that sqlite already has pluggable storage backends, I probably would have. [07:54:25] i actually think backing ubik with sqlite (not vice-versa) would be better. i think adamson did similar code for berkeley db [07:59:00] "backing ubik with sqlite" is silly. Storing our flat files in sql tables? What would be the point? [08:03:19] What I'm a good portion through the design of is adding interfaces to ubik which are a shim over part of the sqlite API, such that once you open a ubik transaction, you can mix and match access to the ubik database and calls to sqlite (through our shim), and ubik will provide distributed-ness for the sqlite database and make sure that both sets of changes are treated atomically. [08:11:55] Was that a paste from zephyr? [08:12:13] --- dwbotsch has become available [08:12:23] which? [08:12:31] the sql-lite remarks? [08:12:44] i... don't think so? [08:13:12] it seems to start in the middle of something [08:13:59] i think it came from the hackathon channel [08:14:04] oops sorry [08:15:14] No, it came from here. You were just starking [08:16:16] ok. [08:16:26] well, i guess matt's client wasn't online long enough [08:18:14] i guess not. nothing above that in jabber.openafs.org, either, it seems [08:19:20] [11:58:57] on 10-12 is not before now, in the jabber.openafs.org log? you lie [08:19:51] oh, maybe I did [08:19:58] you fail. [08:20:08] Derrick, you really should stop accusing people of lying when all they're guilty of is being wrong. [08:20:20] people lie all the time [08:20:25] i'm lying right now [08:20:48] > people lie all the time No, people never lie. :-) [08:21:37] _I_ never lie, mostly [08:21:43] liar. [08:38:33] --- reuteras has left [09:09:30] --- haba has become available [09:34:27] --- haba has left [10:10:03] --- summatusmentis has become available [10:25:01] --- dwbotsch has left [10:47:09] --- summatusmentis has left [11:40:30] --- Russ has become available [12:01:05] --- dragos.tatulea has become available [12:08:24] hi [12:10:18] hello [12:20:15] What would you think if openafs had scons instead of autoconf/make? Would it be better? Would it help? [12:21:31] I think we would rather depend on sh and make, which everyone has, than on python and scons, which everyone doesn't have [12:22:13] thought so [12:22:16] If you're asking me, I don't know, I am relatively satisfied with the current system, although I'm sure many people have their issues with it [12:27:52] the one benefit to scons that I can think of is that it would permit the Windows and UNIX build systems to shared to a large extent. Currently Windows has an independent build system and it is often the case that modifications that are made to core libraries due to changes on UNIX do not get reflected in the Windows builds. [12:29:03] if I remember correctly there is also a nice Unit Testing framework built on top of scons [12:29:20] They say that 2/3 scons code is testing code. [12:30:31] I think they mean that 2/3 of scons code is configuration testing to determine which options are available in the environment for building the target [12:31:36] jaltman: True, but it would significantly raise the bar for people building their own binaries on non-Linux, which I don't think we can afford, given that most serious OpenAFS users build their own UNIX binaries, and I suspect that may become more necessary over time. [12:34:46] since the Windows build system is nmake, the system is effectively similar to the Unix one, without the configure probing [12:35:30] similar but incompatible. [12:35:44] scons replaces [n]make [12:36:05] It doesn't have to be incompatible; it just is. [12:36:22] the nmake makefile language is incompatible [12:36:50] yes, but not very, I recognize most rules in use [12:37:12] ie, as cognate with their unix counterparts [12:37:51] to produce a single makefile that works with both requires a makefile preprocessing step. [12:38:16] How is nmake's input language incompatible with that of other makes? [12:38:58] I'm sure someone has a nice web page that describes the differences [12:39:23] (obviously it is incompatible with the language of Makefile.in, but we could certainly move from a model where we have a separate NTMakefile to one in which windows Makefiles are generated by a script of some sort that does the substitutions on Makefile.in [12:39:46] we could adopt the scripts from MIT Kerberos [12:41:09] not that I think doing so should be a priority for anyone [12:42:01] I was just curious on how much would openafs benefit from using scons, because I'm learning it now. [12:42:02] for the IFS the build system is utterly incompatible as it doesn't use nmake at all but the Windows DDK 'build' framework [12:42:56] Okay, I'm tired of waiting for ls -l on /afs/andrew.cmu.edu/usr/shadow. [12:44:10] How long have you been waiting? [12:45:09] Well, yesterday, somewhere around 30 minutes and got bored and stopped it. Today I think 20 minutes, but it managed to get the stats just now. [12:45:29] It's true that I'm using a nat'ed vm, but still... [12:50:20] Well, clearly it shouldn't take that long. It certainly doesn't take that long for me. Is this the result of the bug you are trying to track down, or is it just something that's getting in your way? [12:52:06] Well, yesterday I was using the disconnected code (compiled in, but was in connected state). Today removed the connected code and did it again. [12:52:30] Maybe it cached some data yesterday and today took the rest. Is that possible? [12:53:26] And yesterday, there was traffic trough in the pipe. [12:53:56] you might want to use wireshark to capture and analyze the rx communications so that you can see what is taking so long. [12:56:13] Let me try it again now. I should have the stats in the cache now. [13:00:17] Yeah, somewhere around 5 minutes now. [13:00:40] It shouldn't take 5 minutes to ls that directory, unless your network is _really_ slow. It should take less than a second. [13:01:43] ls -l [13:01:51] ls is way faster [13:01:55] it takes close to 7 seconds from my apartment when nothing is cached [13:02:24] okay, let me fire up wireshark and have a look [13:02:30] and empty the cache first [13:02:51] is fs flush enough? Or should I delete the cache files? [13:03:23] Even that seems insanely long, Jeff. Granted, my RTT to the server is less than a millisecond, but still it takes me practically no time even with the thing flushed. It shouldn't take you 100x as long. [13:03:40] windows stats all objects [13:04:20] fs flushv will be enough to force a new FetchStatus on everything. To actually force the directory data to be refetched, you need to delete the cache files (not while AFS is running, of course) [13:04:24] Jeff, so does ls -l [13:05:03] --- summatusmentis has become available [13:05:47] the directory contains ~1300 objects which in turn requires 26 InlineBulkStatus calls plus the FetchData calls for the directory data [13:06:24] 45 seconds with fs flush * then ls -l [13:06:31] let me delete the cache now [13:07:19] It contains 1523 objects, at the moment. And after flushing the volume, ls -l takes 0.18 seconds. [13:07:43] That's wall time, of course. [13:07:54] Now, I expect it to be more for you, because your RTT is larger. [13:08:50] if you want to perform a fair comparison, install oafw 1.5.54 and test it [13:09:18] Oh, hm. There are also mount points in there for a dozen or so volumes, several of which are out-of-cell and none of which I flushed. [13:11:22] But, the differences between your time and mine can be attributed to minor performance and behavior differences and/or network latency. 45 seconds is too long 5 minutes is way too long 20 minutes is insane [13:12:57] Well, think about it: nat'ed vm over wireless connection [13:13:07] I've seen those sorts of times as a result of packets being dropped [13:14:09] They don't seem to be retransmitted...But then again it's udp, how would I know? [13:14:15] s/would/could [13:14:21] Neither of those should break it that much. Unless your wireless is real slow. Yes, packet loss could account for it. What sort of packet loss do you have to vice7.fs.andrew.cmu.edu ? [13:14:55] rx does retransmission. You'd know because the rx sequence numbers were the same (but not the serial numbers, which IIRC are incremented on retransmits) [13:16:56] --- dmontuori has left [13:17:02] --- dmontuori has become available [13:17:11] Okay, it stopped. 285 seconds. It's better though... [13:17:27] So it's still 5 minutes [13:27:56] Yeah well, doing disconnected ls -l on that dir, does not deadlock. [13:30:06] that is good [13:30:42] It is. But it doesn't help me pinpoint the problem. [13:31:52] Anyway, I don't see how it can block on a dir stat, considering the fact that I'm not touching the code that does that. [13:41:11] I'm now doing ls -l on /afs/andrew.cmu.edu/usr/ [13:42:50] good luck with that. :-) [13:44:58] 15 minutes... [13:45:07] and still going [13:46:55] That one's going to be fun. /afs/andrew/usr is a directory of symlinks to mount points. So for each one you get to stat the symlink, the mount point, and the volume root (at least, if ls is following the link). You also get to stat and read all of /afs/andrew/usr*. It took me about 42 seconds. [13:52:03] I tried leaving it for an hour yesterday, and it wasn't over yet ... [13:56:08] --- dmontuori has left [13:56:08] --- dmontuori has become available [13:58:43] I give up today. Good night. [13:58:47] sleep well [13:59:05] thanks :) [13:59:27] btw, its been 11 minutes and my directory listing of andrew/usr is not complete [14:00:25] Okay. So maybe andrew/usr is not the best place to stat. I learned my lesson. :) [14:00:28] good night [14:01:07] --- dragos.tatulea has left [14:01:10] --- matt has left [14:31:48] ~26 minutes [15:21:45] --- dmontuori has left [15:50:17] --- summatusmentis has left [16:12:47] --- Simon Wilkinson has become available [16:32:10] --- sxw has become available [16:34:38] --- sxw has left [16:52:50] --- Simon Wilkinson has left [17:47:40] --- summatusmentis has become available [17:49:44] /afs/andrew/usr is a bad place to stat [18:02:05] --- summatusmentis has left [19:10:32] --- summatusmentis has become available [20:37:31] --- summatusmentis has left [20:50:14] --- summatusmentis has become available [22:57:21] --- reuteras has become available [23:27:38] --- Russ has left: Disconnected