[00:03:09] --- dragos.tatulea has become available [00:06:34] --- Russ has left: Disconnected [00:09:39] --- SecureEndpoints has left: Disconnected [00:16:21] hi [00:26:08] --- dev-zero@jabber.org has become available [01:11:32] --- manfred furuholmen has become available [01:12:22] --- manfred furuholmen has left [01:15:08] --- manfred furuholmen has become available [01:15:30] --- Simon Wilkinson has left: Lost connection [01:34:32] --- manfred furuholmen has left [01:36:44] --- manfred furuholmen has become available [01:37:50] --- manfred furuholmen has left [02:55:37] --- dragos.tatulea has left [02:57:28] --- dragos.tatulea has become available [03:08:18] --- dev-zero@jabber.org has left [04:14:30] --- dev-zero@jabber.org has become available [05:11:39] --- shadow@gmail.com/owl72084742 has left [05:36:11] --- sxw has become available [05:36:43] --- sxw has left [06:32:15] --- shadow@gmail.com/owl2BB11B64 has become available [06:52:39] --- matt has become available [07:28:38] --- Simon Wilkinson has become available [07:35:41] --- SecureEndpoints has become available [08:22:03] --- sxw has become available [08:25:02] hi [08:25:07] hi [08:27:57] For file pinning, there will be pinning for both dcaches an vcaches. Is that a correct assumption? [08:28:25] without pinning the vcaches you won't know what you have [08:29:10] hi [08:29:46] I was thinking more about the dcaches (remembered they have to be in sync as well :) ). [08:29:51] Yes. The user will pick to pin the vcache, but the dcaches should also be protected from explusion. [08:30:14] well, if you don't protect the dcaches you have no contents, so i assumed that was obvious [08:30:57] There are choices about how you pin the dcaches. You can do something new, or you could leverage the existing buckets code, and have a bucket that can't be expunged. [08:31:16] But the existing buckets code is designed around users picking percentages, which probably isn't the way you'd want the pinning to work. [08:31:35] Who was using buckets? [08:32:08] 1.5 has support for splitting up the dcache into multiple buckets - I think there's one for RO data, and another one for RW data at present. [08:32:21] So you can say X% of my cache should be reserved for RO data, and Y% for RW. [08:34:16] i wrote the buckets. you could just change the policy engine to not use percentages and use the buckets [08:34:23] the policy engine is one function [08:34:43] the idea was someday we'd have a plugin interface or a policy language [08:34:52] It seems there's a choice between a bucket, and having a flag. Given that buckets is there, it would probably make sense to use them. [08:39:35] There...where? Please excuse my ignorance. You mentioned it a while ago but I'll have to dig the logs. [08:41:51] Search for bucket in afs_dcache.c [08:42:44] thanks [08:56:32] Grrr. Someone at work has migrated my homedirectory to a fileserver that doesn't have any firewall holes. Mutter, mutter. [09:00:20] see, when i had powers i just fixed that sort of thing, then raked whoever over the coals. which was like the wrong way to do it, since no one ever learned. [09:02:12] I am fixing it, but what's a pain is that I just spent 20 minutes trying to figure out what was wrong before I released my volume was on a different server. And now I'll have to wait for our configuration management system to propagate the change to our firewalls. [09:24:57] And, after all that, a trivial patch to abstract out the vcache reset code from PFlush so it can be used elsewhere ... /afs/inf.ed.ac.uk/user/s/sxw/Public/openafs-reset-vcache.patch [09:33:56] --- Derrick Brashear has left [09:34:43] What happened to dementia.org? [09:37:59] dementia.org is available [09:39:10] "Though the site seems valid, the browser was unable to establish a connection." [09:40:16] the web server is non-responsive. the cell works [09:40:53] I was trying to reach the wiki. [09:45:07] sxw: I can use buckets for dcaches, but I will still need to insert a pinned state for vcaches. [09:46:03] Which is fine by me. The expired vcaches get into a queue that gets synced on a different thread. [09:46:15] What if the user wants to see the list of pinned files? [09:47:22] They can't. [09:47:45] You can't go easily from a vcache to the path it comes from. So, for now, I'd say "they can't". [09:48:33] Unless we dump them in a file or something... [09:48:49] Yes. But I'd say that's a future feature improvement, rather than something we want from the start. [09:50:30] We could look into each vcache and get it's path if has the pinned state. Costly operation...but more useful than none. [09:51:11] You can't get it's path, because you don't know where it's been mounted. A given vcache can occur in multiple locations in the FS. [09:51:36] At best, you could give a cellname, a volume name, and the path within the volume. [09:52:04] That would still be informative. [09:52:18] yep [09:52:49] Yes, but not essential from the start. [09:53:11] (I think we need to be able to do that exact lookup for conflict warning messages, so there's code that could be share there) [09:53:19] But I don't see it as being on the critical path. [09:53:51] --- dragos.tatulea has left [09:54:32] --- abo has left [09:54:58] BTW: I've put bug https://rt.central.org/rt/Ticket/Display.html?id=124148 into RT as a tracking bug for disconnected operations issues. [09:55:12] I think we might be able to use a similar mechanism for tracking 1.6 blockers. [09:55:21] Good idea [09:57:10] --- stevenjenkins has left [10:00:35] --- stevenjenkins has become available [10:03:20] --- dragos.tatulea has become available [10:03:29] hey dragos [10:03:32] For a user it would be important. [10:04:47] dragos: at some point the cache manager should maintain a list of the paths the user requested be pinned. [10:04:52] Indeed. No argument with that. [10:05:04] (with Dragos's point) [10:05:28] Less sure about paths. It's hard, because you're not actually giving them the behaviour they expect. [10:05:37] what do you mean? [10:06:08] Well, unless we're really clever, moving points around may mean that we're no longer pinning the _paths_ that the user originally asked us to. [10:08:28] the user is going to ask for the following to be pinned: files, directories, or volumes located at the specified paths. the CM should record those paths and the fids (or fid patterns) that are pinned as a result. When reporting disconnected status to the user, we should refer to the configured paths and not to the fids [10:09:37] the status report could include the volume name, the dirs and files (and their fids) for information purposes. we could even give an indication if the path still points at the items that were pinned [10:10:04] I'd rather punt on the whole issues of paths at the moment. It's not easy, given the way that the Unix CM is currently structured. [10:11:30] from the user perspective s/he is concerned about the path. think of a site that makes extensive use of symlinks and mount points to direct the user to the latest version. the user will want that latest version to be pinned when the symlinks / mount points are updated [10:11:53] we can't do that if we are not recording and evaluating paths when returning to the connected state. [10:12:21] I agree that at the moment the CM is not prepared to handle that. That is why I said "at some point" [10:12:47] Cool. I completely see your point, it's just that I'd like to approach the problem in manageable chunks. [10:13:06] absolutely. "at some point" [10:13:35] --- stevenjenkins has left [10:15:30] I suspect that internally, pinning a directory recursively, and pinning the root of a volume will look much the same. [10:16:03] agreed. [10:16:09] the difference is where you stop [10:16:37] Indeed. If the user's asked for a volume, you wouldn't want to cross mount points. [10:17:05] for example, pinning a symlink should pin all of the objects that must be evaluated to get to the target object the user wants to use [10:17:35] --- stevenjenkins has become available [10:17:36] Yes - Dragos and I have discussed that - that when you pin a file, you have to (non recursively) pin every directory under it. [10:18:12] We also discussed about the difference between 'viral' and 'non-viral' pins, where a viral pin, also pins any new items created below it. [10:18:32] we had that discussion during the Summer on the irc channel [10:18:35] yes we did [10:18:53] and during september [10:18:58] going off on a tangent [10:19:04] s/during/in [10:19:06] Disconnected doesn't actually support symlinks and hardlinks at the moment, so pinning them isn't quite so important right now :) [10:20:05] the andrew cell implements a behavior that is driving me nuts. they have a volume root directory filled with relative symlinks that of the form "..//" [10:20:39] and of course there is no well defined parent of a volume root directory [10:21:01] it is entirely defined by the path that is currently being used to access the symlink [10:21:53] Yes. That's nasty. In fact, it will probably break on Linux in interesting ways. Linux doesn't like multiple parents very much. [10:23:13] I wish we could just declare it unsupportable and make sure that none of our clients support it but since previous clients do, we are stuck with it. [10:40:33] --- Russ has become available [12:19:27] --- SecureEndpoints has left [12:41:05] --- edgester has become available [12:50:03] has anyone else noticed that the similarities between openafs and the whole "cloud" storage fad? [13:25:04] --- dragos.tatulea has left [13:27:47] --- dragos.tatulea has become available [13:59:25] --- dev-zero@jabber.org has left [14:15:11] --- Derrick Brashear has become available [14:29:10] there [14:29:28] is a big difference between thinking about breaking a callback, and actually doing it, I find [14:37:49] context? [14:39:03] I put a very misleading vicelog message in xcb implementation--confused the poop out of myself [14:39:16] after a couple of months of working on other things [14:39:43] I'm having not much fun at all with getting disconnected to survive when things go wrong. At the moment, stuff's getting locked at the kernel vfs level, and I can't work out why. [14:40:16] that's not nice--some of the things you've been mentioning worry me for xcb [14:40:40] Which things? [14:40:44] races [14:41:53] The joy is that the Linux VFS uses many many tricks to avoid ever having to hold a lock - so you have to be extra careful with the order you do things in. I think I'm gradually tracking down the interesting issues, though. [14:43:22] should be interesting to read the result [14:43:26] --- dragos.tatulea has left [14:53:36] --- manfred furuholmen has become available [14:54:19] --- manfred furuholmen has left [15:07:34] This is fun. Create some files, go disconnected, delete them, reconnect. Get an error because the files haven't made it to the server yet. [15:08:28] oops [15:13:08] Very confused though. Create obtains the DISCON_LOCK when it starts, releases it once everything is finished. I can't see how we can disconnect whilst the create is in progress. Grrr. [15:14:40] take the glock? * runs away* [15:15:09] This is the Unix cache manager, you don't get to lock anything unless you hold the GLOCK :) [15:31:40] Actually. My bad, bug isn't that. It just has those symptoms. [15:52:26] --- matt has left [15:57:35] --- edgester has left: Lost connection [16:03:49] Well, afs_buffer.c, it does what it does says on the tin. Ooops. [16:06:02] heh [16:09:57] So we happily make a shadow copy of the dcache, but there's still data waiting to be written from the buffers. So we get a shadow copy of the state of the world up to 60 seconds ago. Too tired to fix now. Adding a bug to RT. [16:17:00] --- edgester has become available [16:47:19] --- dev-zero@jabber.org has become available [19:37:59] --- edgester has left [23:38:08] --- Russ has left: Disconnected