[00:24:12] --- dev-zero@jabber.org has become available [00:29:49] --- Manfred has left [00:32:02] --- Manfred has become available [02:10:20] --- Simon Wilkinson has become available [02:10:29] --- Simon Wilkinson has left [02:34:39] --- Manfred has left [02:42:08] --- Manfred has become available [03:14:11] --- dev-zero@jabber.org has left [03:52:28] --- dev-zero@jabber.org has become available [05:19:00] --- stevenjenkins has left [05:23:36] --- matt has become available [06:26:11] --- Adam Goode has left [06:52:17] --- stevenjenkins has become available [07:57:59] --- Russ has become available [07:59:34] --- Simon Wilkinson has become available [08:03:28] stevenjenkins - there is enough material that w. hands on doing stuff, you can go through a whole week -- the training class I did was a whole week and we still didn't get through everything [08:04:21] --- Derrick Brashear has left [08:07:17] --- Simon Wilkinson has left [08:19:29] --- Derrick Brashear has become available [08:22:00] --- mmeffie has left [08:34:44] --- Simon Wilkinson has become available [08:36:26] kula: The cache manager isn't that prickly when you get to know it... [08:37:28] well, the code org of it isn't. [08:37:40] the locking to deal with different vfses is. [08:38:55] --- Simon Wilkinson has left [08:41:30] --- reuteras has left [09:04:51] --- jhutz@jis.mit.edu/owl has become available [09:21:48] --- Derrick Brashear has left [09:31:06] any easy way to pull out the deltas between two release? [09:31:11] err, releases. [09:47:11] --- Simon Wilkinson has become available [09:49:05] stevenjenkins: I can't think of any easy way, no. We discussed this at the hackathon too. [09:53:27] --- Simon Wilkinson has left [09:53:57] --- Manfred has left [10:02:29] No, there's no easy way. In theory, the deltas file records deltas in the order in which they were committed, so if you identify the last delta before each release was tagged, you have a good start. [10:07:38] We could do it if we always did a commit right before the tag with a standard format of the delta name, but that's a going-forward thing. [10:07:58] --- Derrick Brashear has become available [10:08:42] That'll never work. People have this nasty tendency to say "oops", commit something, and re-tag. [10:09:23] sometimes it's not even an oops [10:09:38] the tag doesn't happen always right after the version number commit [10:09:57] Well, I meant, "oops, I wanted to get this in, too" [10:10:52] Yeah, we know that. Russ was proposing a special commit with a standard delta name which would _always_ be done just before the tag, specifically to record the tag location for the delta list. [10:11:23] yeah. too late now [10:11:29] This is all fixed in Git. :) [10:11:40] sure it is [10:15:56] --- SecureEndpoints has become available [10:27:38] speaking of git, does anyone have a gerrit instance up and running for the git process? [10:27:49] * stevenjenkins peers around but doesn't see Simon [10:28:26] "for the git process"? [10:28:59] simon has gerrit up and running but i don't know what you mean by "for the git process" [10:47:12] --- Simon Wilkinson has become available [10:47:29] well, there's simon, but you still never answered what "for the git process" is [10:48:12] Gerrit was up and running. It probably still is, although I haven't fed the backend recently. [10:48:34] yeah, but i think he means something else, and vanished before he could explain [10:48:47] like, maybe, "are the conversion tools in gerrit" [10:48:47] We need to have a discussion about gerrit - whether we're happy to have this dependency on Google. [10:49:14] given that we don't have a git repo of tools, that's sort of nonsensical [10:49:21] is GAE packaged for debian? [10:49:25] if so, this is easy [10:49:31] No. [10:49:33] --- dev-zero@jabber.org has left [10:49:35] damn [10:49:49] You're not really supposed to use the development environment for 'production' use. [10:50:16] My major worry is about losing the history. [10:50:34] Maybe we just need gerrit to feed RT with history when a patch gets pulled up into the tree. [10:51:26] Oh, and while I'm here... [10:51:27] that would be ideal [10:51:52] Should the new rx_MutexIncrement implementations take signed, or unsigned objects as the thing to increment. [10:51:56] ? [10:52:11] At they moment they're not consistent, and spew warnings. [10:52:27] we need jeff's input, this is his work, i just filled in some bits [10:53:30] let me check [10:54:00] if you have your choice of primitives i should see if darwin and solaris also do [10:54:26] rx seems to expect (most) reference counts to be afs_uint32. [10:54:43] signed [10:55:00] at least on Windows, the interlock functions require signed values [10:55:15] Okay - so should we change the base types in rx, or fix the type before we call the OS functions? [10:55:32] we should change the base types in rx to be consistent [10:55:36] i'd prefer fix types before calling [10:55:45] to be consistent with what, though? [10:55:48] (heh) [10:56:02] either way is fine. [10:56:09] do what Derrick says [10:56:29] You should not change things just to change them. We have enough instability resulting from things intended to be real features and bug fixes without introducing gratuitous instability. [10:57:23] It would probably be nicer if rx_MutexIncrement and friends became static inline functions, so we can cast with some kind of type safety. [10:57:46] Is type safety worth the possible performance hit from that? [10:58:26] compile it both ways and look at the resulting code? [10:58:48] I would think that for a good optimizing compiler the results should be very similar [10:59:11] jhutz of course never wants to change anything. i don't want to change everything but i'd like to be able to have warnings mean "an actual problem" and not "just ignore because there are 20000 of them" [10:59:22] ==shadow. [10:59:30] Which is why my tree now has ~3000 warnings. [11:00:16] SecureEndpoints: With a good optimizing compiler (well, gcc, anyway) the results are pretty much identical. The issue is with compilers where we can't do static inline, or builds which disable optimisation for some reason. [11:00:30] jhutz: we are replacing the existing locking model with one built around Interlocked operations. I do not believe it is unreasonable to change the types of the reference count objects to be consistent with the tools that are being using to maintain the counts. [11:00:41] I'm inclined to say that clear type safety is more important than those edge cases... [11:01:15] I would argue that on platforms where we can using static inline we should. For those that cannot, they can fallback to the macros [11:01:16] . [11:01:27] jhutz will now tell you changing how we are locking is bad. [11:02:08] that model makes it sort of hard to ever improve anything. [11:02:15] but hey, what do i know [11:03:03] since the locks are the bottlenecks I don't know how else to improve the performance unless we change the way we manage locks or remove the need for the locks [11:03:08] I think we have to be free to change things majorly in HEAD, and then filter those changes down into the branches according to their risk and reward. Otherwise, we'll just be applying sticking plaster after stacking plaster. [11:03:54] I guess it depends on whether we believe that rx has a stable API, and whether the mutex operations form part of that API. [11:04:27] i'd argue no to the latter and on the fence for the former [11:05:03] I think dewarnification is a worthy goal. But unlike Jeff and Derrick, I prefer that things not get broken. [11:05:25] i don't see how you can tell me what i prefer. [11:05:33] I prefer that things not get broken for my production builds. I'm happy with some level of breakage for things that are clearly works in progress, or experimental. [11:06:05] (the fact that Windows builds from the 'development' branch, is of course a problem with that statement) [11:06:28] that will be fixed, but is aside from the point of what we do on HEAD [11:06:33] sorry about the gerrit hit-and-run. had a phone call. [11:06:36] of course it is. and if the stable branch was one that could compile on 64-bit windows platforms it would not be true [11:06:47] Mutex operations do not seem to be in the interface. Whether a mutex is required to increment a counter or whatever is subordinate to the assertion that incrementing the counter or whatever is an atomic operation (obviously) [11:06:54] > i'd argue no to the latter and on the fence for the former me too. rx _should_ have a stable API, but I'm not sure we're there, and if we are, I don't think it's entirely clear what is in the API. But internal mutex operations certainly are [11:07:09] > i don't see how you can tell me what i prefer. I call them as I see them. [11:07:29] so you see lies. excellent. [11:07:45] if you believe you know what i prefer, say that. if you know what i prefer, say that. [11:07:49] if you wish to lie, say that [11:07:50] > But internal mutex operations certainly are was that the end of the statement? [11:08:05] So, if the mutex operations are internal, and the counters they operate on are also internal, then we should be able to do a type change. [11:08:15] and i hope "But internal mutex operations certainly are" didn't mean "are in the API" right after you say "internal" [11:08:29] I dropped a word or something. I agree with Derrick that the mutexes are not part of the rx API [11:08:40] > then we should be able to do a type change [11:08:40] Cool. So, we're all arguing about nothing, then? [11:09:02] the question is, then, if one should be done. i don't belive we know the answer to that yet [11:09:23] like, basically, should the counter types or the macro types change [11:09:31] Hard to say. My complaints about stability wrt AFS have not been about API changes [11:09:34] well, what does changing the types impact elsewhere in the code [11:10:11] also, we know now what windows native operations do. that's one of like 9 platforms [11:10:37] We're going from unsigned to signed, so it's only overflow we care about - tests failing due to underflow shouldn't be an issue. [11:10:44] They're about people breaking things. For example, if you change something from signed to unsigned or back, that has an effect on the promotion rules for expressions involving that thing and another value of a different type, and the result may be subtle and not what you intended. [11:12:03] for things like refcount, interesting tests are 1, 0, and (not one of those). i bet we could vet such a change against "values of other types" pretty easily. [11:13:22] Hmm. Does Solaris 8 really have OSAtomicXor32Barrier ? [11:14:03] windows native operations do> it's really about the ISAs, only a couple of platforms provide an API for doing this sort of thing, and if they do, we're not really required to use them if its type-inconvenient [11:14:11] But that said... - eliminating warnings is good, as long as the warning is eliminated by fixing the problem and not by introducing a bug to shut up the compiler - I've talked to Matt and Tom about the whole "improve performance by moving to lock-free data structures" thing, and I believe it's a good idea, provided it gets the proper level of testing and doesn't infect stable releases too quickly. [11:15:04] So, a quick platform survey is: [11:15:48] Generic: don't care, Windows: signed, Darwin: signed, Linux: special type, Solaris: unsigned [11:15:53] nothing ever gets enough testing. (not just according to you, either) we can do at best due diligence with the resources we have, or simply never move forward. [11:16:17] i386 i686 x86_64, several variations (none signed, up to quadword on some x86_64) [11:16:31] ppc [11:16:37] s390 390x [11:16:49] > Does Solaris 8 really have OSAtomicXor32Barrier I assume that's a macro; what's its implementation on newer Solaris? [11:16:52] parisc (if we still support that) [11:17:04] looks like ppc [11:17:14] jhutz: I think it's a Solaris macro that's been cut and pasted to the wrong place ... [11:17:21] s/Solaris/Darwin/ even. [11:17:22] "darwin" macro [11:17:24] ? [11:17:27] yes, my fault [11:17:42] the solaris macros were done by me after i did the darwin ones. i cut, pasted, and edited [11:17:51] mcas has macros for ultrasparc, but I switched mcas to using the solaris macros [11:17:52] I can find nothing like that in /usr/include on my sun4x_510 box [11:17:54] problem is i think there's no atomic_xor_32 [11:18:53] jeff, it'd be atomic_ mumble. the OSAtomic stuff is not solaris. [11:18:54] sys/atomic.h does not have xor operations on any Solaris I looked at. If sparc has such an instruction, I don't know what it is. [11:19:01] There's an implementation of rx_MutexXor immediately above that line, which just uses a mutex. [11:19:09] yeah. like i said, it will have to be a generic xor on solaris [11:19:14] i just never went back and glued that in [11:19:35] ah. i did, and forgot to remove one more line [11:19:36] fine [11:20:48] I don't have an 8, but solaris 7 has only atomic_add_{16,32,long,64}{,_nv} cas{32,long,64,ptr} and the membar_* operations [11:20:56] Looks like the Linux kernel will only give us inc, dec, add and sub [11:21:21] You really should look at mcas and libatomicops [11:21:38] And actually, you don't necessarily need a mutex; you can build "atomic" xor out of a normal read and cas, if you're willing to retry. [11:21:49] jhutz: yeah...that's the ticket [11:21:56] but also, ==matt we shouldn't be doing this twice, when it's already been done for us [11:22:16] I wonder if all of this means we should really have a refCount type, that all of these operations take. That way we can just typedef that to whatever the underlying implementation requires. [11:22:30] probably [11:22:47] well, a refcount shouldn't need xor anyway. [11:23:55] Well, nothing in HEAD seems to use Xor. [11:24:07] probably was there for completeness [11:24:23] unless he wanted to do something with flags [11:25:36] We do use rx_MutexOr with connection flags. [11:29:44] mcas/atomicops could presumably be shimmed under macros which are done correctly [11:30:10] seems like a good approach [11:30:20] Yes. I wonder though if the current API is too rich. [11:30:30] since istr you only support a handful of platforms so far [11:30:38] current *which* API [11:30:41] Can you safely mix atomic operations with mutex protected operations in all of our atomic libraries? [11:30:52] The current API as implemented in rx_internal.h [11:31:25] i think from review, only variables whose accesses could be completely converted were [11:31:33] that was the intent. [11:35:19] > I wonder if all of this means we should really have a refCount type, > that all of these operations take. That way we can just typedef that > to whatever the underlying implementation requires. already done as part of libosi. it backs with atomics wherever possible; uses mutexes elsewhere [11:35:35] yeah, can it be split from libosi? [11:36:01] <- whiny bitch [11:36:04] not without pulling in the base parts of libosi [11:36:30] *cries* [11:40:55] but libosi api is...more complex than we necessarily need to manipulate machine words? [11:41:06] (because so is solaris) [11:41:13] matt, I'd like to know why you think that [11:41:21] Hmmm. We seem to play with conn->flags both by holding mutexes, and through the atomic operations... [11:41:29] do i need to call osi_PkgInit to do it? [11:41:40] if so, too complex. [11:41:49] > do i need to call osi_PkgInit to do it? shouldn't need to [11:42:11] I'm not complaining about libosi, I use it... [11:43:01] ok. so you're just saying it's a huge blob to pull in for just this particular use case. that's fair [11:45:01] yes, well, yes [11:45:05] Could it be unpicked? [11:45:18] it would be nice to star. well. we talked about this at the hackathon. start pulling in pieces [11:45:32] but, which platform targets does osi_atomic now support with its existing api? [11:45:52] thread, synchronization, time [11:46:10] thread is blocking on the windows integration issues [11:46:26] osi_atomic only implements solaris, and I have a patch to do aix, so not terribly useful [11:47:08] right, that's what my comment meant--to rapidly do all our platforms, you'd do something more like mcas--a selection of ops and barriers, on native integral and pointer types [11:48:14] maybe more like libatomicops, perhaps, if you want more granularity [11:48:17] that is the comment I was responding to earlier. I don't understand your position. _every_ osi_atomic interface is conditional. you have to test for its existence before using it. the documentation in the header file lists everything you have the _option_ of implementing in a backend [11:49:44] --- dev-zero@jabber.org has become available [11:50:24] is it that my list was long enough that you think developers will ignore it? [11:53:21] this is way ot, and sorry, but, some of the operations, on the original sparc platform, can be little more than type casts around the base operation...why? [11:53:33] not yours, now, solaris atomics [11:53:58] Does the architecture give you atomicity? [11:54:11] indeed, that's where it lives [11:54:26] but, solaris atomics is an api [11:55:39] /afs/inf.ed.ac.uk/user/s/sxw/Public/openafs-rx-atomic-typing.patch is a possible fix to the current problem ... [11:56:33] well, thanks to rich sudlow for loaning me a machine, i know now why the large partition patch doesn't fix ZFS on solaris. i will deal later [12:03:40] Presumably because on old enough sparc, you can only have one CPU in a system and so don't need membars, and the base increment and add operations are already atomic with respect to interrupts [12:05:15] > can be little more than type casts around the base operation...why? pointer to the sources? I don't recall seeing anything like that, but it's been a while since I read through that part of ON [12:06:54] oh. ancient single-cpu sparcs. that would make sense. [12:07:42] --- dmontuori has become available [12:09:44] --- Derrick Brashear has left [12:10:01] Interesting .. They're discussing gerrit 2 on the repo list, and it won't have a GAE dependency. [12:10:59] I wonder if they will have finished writing that before we have git in production :) [12:11:36] --- Derrick Brashear has become available [12:16:57] --- dmontuori has left [12:17:52] --- dmontuori has become available [12:18:53] gerrit2: wow [12:20:00] The major problem with the current gerrit is that it doesn't handle large patch sets very well - primarily because of the incredibly small CPU and memory quotas that you get from Google App Engine. [12:20:44] --- mmeffie has become available [12:21:18] hi all [12:21:24] hi [12:21:31] hi jake. how goes? [12:22:11] it goes, this Linear Algebra tutor is much more helpful than the prof [12:22:13] :) [12:22:23] might acually be able to pass [12:35:26] --- dev-zero@jabber.org has left: Replaced by new connection [12:35:27] --- dev-zero@jabber.org has become available [12:37:49] jake, sorry I'm late [12:38:06] I think we are supposed to be meeting now if I have by timezones straight [12:38:17] seems likely [12:38:33] jeff, no worries :) [12:39:00] Just updating the survey options wiki page now [12:39:04] thanks for setting up the wiki pages [12:39:07] ok [12:39:40] sure thing. I'm leaning towards limeservice, if only because it seems much more granular in it's options [12:40:02] One thing I wanted to chime in, and then I'll leave you alone. [12:40:26] you don't have to leave us alone [12:40:40] I think it's a dangerous path to assume that the current site serves developers needs. I think there is a (seperate) need for a better developer focussed web presence too. [12:40:53] he may have to leave us alone for other reasons [12:41:20] what, another show? [12:41:31] simon, I tend to agree, although the current site serves the needs of developers over the needs of others [12:41:39] simon: second that (vanishing too, sorry to butt in) [12:42:21] summatusmentis: I'm not even sure that it does that. I think it serves the needs of people who know how to use it. [12:42:21] simon, its not that the existing site is sufficient for developers. its just a different project [12:42:52] --- dmontuori has left [12:42:58] Sure. It's just that there's been a lot of 'the current site is targetted at developers' getting said. I just wanted to make sure that people weren't interpretting that as 'the current site is what developers need'. [12:42:59] --- dmontuori has become available [12:43:26] absolutely not [12:44:12] it was developed by developers and the things that have gotten implemented are things that developers wanted. not necessary what they dream of [12:44:22] targeted to doesn't mean meets the needs of [12:44:52] I don't think "targeted to" is right, though. Most of the current site is _not_ targeted to developers. [12:45:03] It was built by developers, which is not the same. [12:46:31] not exclusively the same, but much more likely that the needs of the developers were met attempted to be met [12:47:04] jake: tell us about limeservice? [12:47:56] this may only be because they allow for full featured-ness without paying, so my tests may have been skewed [12:47:57] or more importantly http://www.limesurvey.org/ [12:48:32] are there Debian packages? [12:48:36] it seems as though while the interface takes a little bit of getting used to, it allows for substantially more control over the types of questions asked, or the make-up of the questions [12:49:28] the statistics/analysis tools seemed similar to those offered by others, and I assume the pay-for versions of the others offer the same level of complexity that lime{suervice,survey} does [12:50:31] the interface is all icon driven, which is kind of odd, but it felt like I had more control over content, vs. the provider deciding what we I did or didn't want [12:50:40] -we [12:53:05] the interfaces of surveymonkey/polldaddy were both clean, easy to use, but didn't seemingly allow as much control [12:53:22] if you are comfortable with it perhaps you should try installing limesurvey and see whether configuring it is a pain [12:53:37] I didn't check the availability of packages, but I can do that [12:53:37] if it isn't and there are Debian packages we can ask Stanford to host it. [12:53:51] if not, I suspect we can find somewhere to put it in the next week or two [12:54:04] or if we can make dpkgs [12:54:17] it's distributed in tarballs directly from the site [12:54:25] as dpkgs? [12:54:31] no [12:54:35] ... [12:54:50] the goal is "if it's packaged for debian, stanford will set it up and run it for us" [12:55:24] http://sourceforge.net/project/showfiles.php?group_id=74605&package_id=77691 [12:55:40] pulling up a debian box now to see if it's in the repo [12:56:47] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=472802 [12:57:15] not in tree but it may be possible to at least get it installed from these [12:57:34] I see [12:57:59] --- dmontuori has left [12:58:05] --- dmontuori has become available [12:59:53] it's DB driven, obviously, so it can't be replicated, if that matters [13:00:23] for surveys i don';t think we care overly [13:00:35] as long as we can generate results from it, it's good [13:01:29] agreed [13:05:18] I don't think surveys are content that we will want to replicate [13:07:03] I didn't think so, I was just contemplating hosting stuff. [13:07:43] wiki has been updated to reflect Simon's concern also [13:08:21] thanks [13:08:48] I'm going to have to leave shortly. Any questions bouncing around your head? [13:09:12] (I doubt it because if there were I'm sure you would have asked them by now.) [13:09:25] I don't think so, just next steps? [13:09:47] still kinda waiting on feedback on the list of questions put up on the wiki [13:09:59] I added some feedback on the wiki [13:10:01] :) [13:10:10] you won't get it beyond from us i bet. [13:10:19] no? [13:10:29] cynically speaking [13:10:35] regarding the list of questions. I think the questions need to be more probing [13:10:36] --- Manfred has become available [13:11:04] think about the answers that you need to be able to obtain [13:11:13] probing in what way? specific? or just more breadth? [13:11:24] alright, that's reasonable [13:11:25] leave lots of room for free text responses [13:11:35] oh, also, were you going to upload a sitemap? [13:11:57] if you give people a list of ten features you will most likely get ten features. [13:12:37] --- dmontuori has left [13:12:43] --- dmontuori has become available [13:12:52] I haven't gotten to the map. Been a bit pre-occupied with the windows kernel [13:13:16] that's reasonable. just thought i'd check [13:13:30] the questions should be asked in a way that will get folks to think and respond not just validate our best guess [13:13:42] yes, I absolutely agree [13:14:02] maybe another way to think about it is, ask the questions that will ensure that we can be proven wrong by the responses [13:14:28] alright, I'll spend some more time on questions either tonight or tomorrow [13:15:36] is there a plan in place, or should there be, to get docs/content pulled together? [13:16:37] the preliminary feeling I'm getting is that most content is *somewhere*, either in 'experts' heads, or documented on other sites. How do we know of/access it? [13:16:56] put everyone in one place and poll them one at a time [13:17:22] Work out the content you want [13:17:25] Find the appropriate people to write it [13:17:29] Bribe them suitably. [13:18:02] GSoC suggests chocolate [13:18:05] > the questions should be asked in a way that will get folks to think and > respond Uh, no, don't expect people to think. Write questions that don't require that. Maybe even find someone who understands how to write and interpret surveys [13:18:32] i could try to bribe leah ericson into it [13:19:25] jhutz, I would like to alow people to think if they're willing [13:20:20] although I do agree that people shouldn't necessarily be _required_ to spend 30+ minutes providing fully detailed answers [13:21:14] --- dmontuori has left [13:21:20] --- dmontuori has become available [13:22:30] --- dmontuori has left [13:22:46] --- dmontuori has become available [13:23:52] jake: thanks for the update. I have to go. I will touch base with you later. [13:24:32] sure thing, I'll look at those questions when I can. Good luck in your ventures [13:25:07] anyone have questions/comments, etc.? [13:25:25] I've got class in 15, so I'm gonna head out otherwise [13:32:25] --- Manfred has left [13:32:43] alright, seeing none, I'm gonna run, I'll be around if people want discussions later [13:32:48] ok [13:42:49] --- Manfred has become available [14:19:29] --- Derrick Brashear has left [14:28:34] --- Manfred has left [14:36:55] --- Derrick Brashear has become available [15:15:42] --- matt has left [15:43:12] --- Derrick Brashear has left [15:53:29] --- Derrick Brashear has become available [16:06:31] --- Derrick Brashear has left [17:50:11] --- mmeffie has left [18:09:36] --- dmontuori has left [20:34:29] --- summatusmentis12958 has become available [20:34:30] --- summatusmentis12958 has left [20:34:30] --- summatusmentis81963 has become available [20:34:30] --- summatusmentis81963 has left [21:56:42] --- Manfred has become available [22:06:31] --- Manfred has left [23:15:32] --- reuteras has become available [23:39:25] --- Russ has left: Disconnected [23:45:28] --- Manfred has become available [23:59:16] --- dev-zero@jabber.org has left