[00:33:59] TimStarling: are we still with ubuntu 8.04? [00:34:26] no, mostly 10.04, with a few servers on 12.04 [00:35:33] I'm building a package on hashars request, what would be the dest server? [00:35:40] how can I know? [00:36:17] TimStarling: this is the related bug : [00:36:18] I believe you. If you can file a bug, that'll be very helpful in getting the issue resolved. [00:36:18] IRC is very hit-or-miss. [00:36:18] i'll try to get some timechart [00:36:18] !log killing nagios-wm process which isn't supposed to b [00:36:24] arrrf [00:36:33] TimStarling: https://bugzilla.wikimedia.org/show_bug.cgi?id=38946 [00:39:18] it would be best to make packages for both lucid and precise [00:39:47] TimStarling: ok. will do both. thanks! [00:40:03] or if you make a source package that will work in either distro, I can build the binary packages [00:41:32] TimStarling: I have a tar.gz [00:42:39] we used to have a wikimedia-fonts package, but I think I saw some discussion about it being removed [00:42:51] maybe it should be resurrected [00:43:30] I might build it if needed [00:45:27] yeah, it was hashar who requested that it be removed [00:45:59] I'll speak to him [02:18:12] any op here? I need to send a large file [02:28:13] !log LocalisationUpdate completed (1.20wmf9) at Tue Aug 14 02:28:13 UTC 2012 [02:28:25] Logged the message, Master [02:38:34] matanya: Most of the ops hide in -operaations. :-) [02:38:41] #wikimedia-operations [02:38:43] Typing is hard. [02:38:57] But you can file a Bugzilla ticket and put the file somewhere public, of course. [02:39:33] yes, I know. thought to do it the fast way [02:39:39] thanks anyway [02:40:16] IRC is very hit-or-miss. (o; [02:45:30] !log LocalisationUpdate completed (1.20wmf8) at Tue Aug 14 02:45:30 UTC 2012 [02:45:41] Logged the message, Master [03:26:07] !log on mw8: removed broken package php5-memcached [03:26:16] Logged the message, Master [03:34:00] !log restarting all apaches to get LuaSandbox extension [03:34:09] Logged the message, Master [04:14:10] [19:38] (NEW) Pages don't load under ipv6+chromium / bits.wikimedia.org doesn't answer http requests - https://bugzilla.wikimedia.org/39322 normal; Wikimedia: General/Unknown; (wikimedia.20.orzelf) [04:14:16] Not sure that got linked in here. [04:50:25] orzel: can you get a tcpdump trace of your attempted IPv6 connection? [04:51:07] TimStarling: with -s 0 ? or does it matter ? [04:51:33] if that is the same as -s1500, then yes [04:51:57] orzel: -s 0 ;) [04:52:12] * jeremyb reruns away [05:45:00] !log olivneh synchronized php-1.20wmf9/extensions/E3Experiments/ 'Update E3Experiments to master' [05:45:10] Logged the message, Master [06:47:59] !log running scap to push out CodeEditor and Scribunto sources, not enabled yet [06:48:10] Logged the message, Master [07:07:01] !log tstarling Started syncing Wikimedia installation... : [07:07:10] Logged the message, Master [07:45:59] !log tstarling Finished syncing Wikimedia installation... : [07:46:08] Logged the message, Master [08:18:12] bugzilla throwing internal errors [08:43:40] Nikerabbit: on email ? [08:43:44] i had the same yesterday... [08:44:15] thedj[work]: yes [08:57:48] Nikerabbit: https://bugzilla.wikimedia.org/show_bug.cgi?id=38584 [08:58:39] thedj[work]: you can see my comments there [09:00:00] ah, lol [09:17:39] !log nikerabbit synchronized php-1.20wmf9/extensions/Translate/ 'I18ndeploy: Translate bugs' [09:17:48] Logged the message, Master [11:10:29] I want to create a new bug and mark it as private, but I don't know how to do that... [11:14:36] Trijnstel: use the "Security" component [11:15:06] sorry, "Security" product [11:16:53] ahhh, thnx! [17:32:05] !log catrope synchronized php-1.20wmf9/extensions/UploadWizard 'Deploy 9d9189458ad8d966d952d295918847f00e304cb5' [17:32:16] Logged the message, Master [17:34:34] !log catrope synchronized php-1.20wmf9/.gitmodules [17:34:43] Logged the message, Master [19:00:06] !log catrope synchronized wmf-config/InitialiseSettings.php 'Deploying https://gerrit.wikimedia.org/r/19399' [19:00:16] Logged the message, Master [19:43:58] It looks like we're going to overshoot our deployment window, but no one else is scheduled until 3. Anyone have a problem with that? [20:05:42] 21:03] Fluffernutter anyone else having loading issues? [20:05:44] [21:05] Guerillero yes [20:06:27] (kaldari?) [20:06:42] "loading"? [20:06:51] Watchlists not loading [20:06:55] I believe [20:07:05] (not affecting me, just quoting from #wikipedia-en) [20:07:09] works for me [20:07:20] my watchlist loaded halfway (but without bits), but I can't get other pages to load at all [20:07:48] are there multiple reports? [20:08:20] need to decide whether to hold off on finishing deployment [20:08:46] Well there's two, I'll see if Guerillero's still having issues [20:08:55] (watchlist now won't load at all) [20:11:41] Guerillo says "no", kaldari [20:11:46] So perhaps just a temporary effect [20:12:15] You get a nice spike on https://gdash.wikimedia.org/dashboards/article/ for example [20:13:35] * Fluffernutter can't load that either. I'm cursed. [20:14:05] haha, ignore Fluffernutter then ;) [20:15:19] heh [20:15:29] I would say my network connection is the problem, except I'm still here talking to you guys... [20:15:50] we're just figments of the matrix [20:16:54] maybe your DNS died after you were already connected to IRC [20:18:27] Fluffernutter: You could try "ping en.wikipedia.org" from the commandline [20:19:21] between 80 and 200ms. Which I think is not bad? [20:19:50] It resolves to an IP okay? [20:20:28] yep, seems to (Sorry, I don't mean to put you guys to work on my tech support! Just jonesin' for a watchlist fix) [20:20:53] deployment to test.wikipedia.org compete [20:20:58] Fluffernutter sometimes it isnt you but the servers [20:21:02] Fluffernutter: how about loading noc.wikimedia.org? (gdash is an alias for that) [20:21:03] Speaking of which, loading 91.198.174.225 gives you some interesting output [20:21:04] deploying to en.wiki in a bit here... [20:21:12] kaldari what is being deployed? [20:21:16] new wikimedia? [20:21:31] no [20:21:42] updates to ArticleFeedbackv5 and PageTriage [20:22:52] Also, gdash gives me intermittent 500 errors [20:23:00] Just thought I'd mention that [20:23:09] Fluffernutter: do you get a user/password window at http://208.80.152.165 ? [20:24:38] mutante, noc loads, exceptionally slowly and without any apparent css ("waiting for upload.wikimedia.org"...) [20:24:46] so far nothing loading for 208.80 [20:24:53] yeah, that doesn't load [20:25:05] i think this has to be just my connection [20:25:12] uhm, works for me and then it is not DNS [20:25:20] so ruled that one out [20:25:35] * Fluffernutter considers percussive maintenance on her laptop with a baseball bat... [20:25:50] Fluffernutter: sounds like routing issues somewhere in between then.. hmmm [20:26:17] Fluffernutter: traceroute noc.wikimedia.org [20:26:20] * Damianz hands Fluffernutter the cricket bat [20:27:45] Fluffernutter: try the 'IT crowd fix' ;) [20:28:15] * Damianz turns mutante off and forgets to turn him back on [20:29:29] sends wake-on-lan packet to self [20:31:31] hah [20:31:33] I imagine mutante starting up as HAL [20:35:42] !HAL is status: completely operational, and all my circuits are functioning perfectly. [20:35:42] Key was added [20:35:57] * Damianz makes WOPR noise [20:40:44] time for scap [20:45:34] !log kaldari Started syncing Wikimedia installation... : [20:45:43] Logged the message, Master [20:46:26] the internets are back, thanks for your help guys :) i shall leave you to your deployment [20:50:12] Fluffernutter: Glad the internets are back :) [21:05:02] !log kaldari Finished syncing Wikimedia installation... : [21:05:12] Logged the message, Master [21:25:47] !log kaldari synchronized php-1.20wmf9/extensions/PageTriage/tools/cleanupPageTriage.php 'adding PageTriage maintainence script' [21:25:57] Logged the message, Master [22:32:35] gn8 folks [23:53:56] !log catrope synchronized wmf-config/CommonSettings.php 'Move VE Parsoid from cadmium to wtp1' [23:54:05] Logged the message, Master