[00:08:40] PROBLEM - Puppet freshness on amslvs2 is CRITICAL: Puppet has not run in the last 10 hours [00:08:40] PROBLEM - Puppet freshness on owa3 is CRITICAL: Puppet has not run in the last 10 hours [00:10:19] PROBLEM - Disk space on srv221 is CRITICAL: DISK CRITICAL - free space: / 279 MB (3% inode=61%): /var/lib/ureadahead/debugfs 279 MB (3% inode=61%): [00:19:37] PROBLEM - Puppet freshness on owa1 is CRITICAL: Puppet has not run in the last 10 hours [00:19:37] PROBLEM - Puppet freshness on owa2 is CRITICAL: Puppet has not run in the last 10 hours [00:20:49] RECOVERY - Disk space on srv221 is OK: DISK OK [00:29:13] PROBLEM - MySQL Slave Delay on db1035 is CRITICAL: CRIT replication delay 182 seconds [00:31:19] RECOVERY - MySQL Slave Delay on db1035 is OK: OK replication delay 0 seconds [01:57:58] PROBLEM - Disk space on stafford is CRITICAL: DISK CRITICAL - free space: /var/lib/puppet 760 MB (3% inode=92%): [02:18:03] !log LocalisationUpdate completed (1.19) at Mon Mar 26 02:18:03 UTC 2012 [02:18:09] Logged the message, Master [02:42:49] PROBLEM - MySQL Replication Heartbeat on db1018 is CRITICAL: CRIT replication delay 231 seconds [02:43:07] PROBLEM - MySQL Slave Delay on db1018 is CRITICAL: CRIT replication delay 239 seconds [02:46:34] PROBLEM - Puppet freshness on brewster is CRITICAL: Puppet has not run in the last 10 hours [02:51:13] RECOVERY - MySQL Replication Heartbeat on db1018 is OK: OK replication delay 0 seconds [02:51:31] RECOVERY - MySQL Slave Delay on db1018 is OK: OK replication delay 0 seconds [03:24:31] PROBLEM - Puppet freshness on db59 is CRITICAL: Puppet has not run in the last 10 hours [03:24:49] PROBLEM - Disk space on srv223 is CRITICAL: DISK CRITICAL - free space: / 192 MB (2% inode=61%): /var/lib/ureadahead/debugfs 192 MB (2% inode=61%): [03:35:28] RECOVERY - Disk space on srv223 is OK: DISK OK [04:03:20] PROBLEM - Puppet freshness on amslvs4 is CRITICAL: Puppet has not run in the last 10 hours [04:28:50] RECOVERY - Disk space on stafford is OK: DISK OK [04:53:19] PROBLEM - Disk space on search1022 is CRITICAL: DISK CRITICAL - free space: /a 4135 MB (3% inode=99%): [04:55:25] PROBLEM - Disk space on search1021 is CRITICAL: DISK CRITICAL - free space: /a 3747 MB (3% inode=99%): [05:08:01] PROBLEM - Disk space on search1022 is CRITICAL: DISK CRITICAL - free space: /a 4445 MB (3% inode=99%): [05:16:07] PROBLEM - Disk space on db1004 is CRITICAL: DISK CRITICAL - free space: / 284 MB (3% inode=89%): /var/lib/ureadahead/debugfs 284 MB (3% inode=89%): [05:16:43] PROBLEM - MySQL disk space on db1004 is CRITICAL: DISK CRITICAL - free space: / 284 MB (3% inode=89%): /var/lib/ureadahead/debugfs 284 MB (3% inode=89%): [06:00:53] PROBLEM - Disk space on search1021 is CRITICAL: DISK CRITICAL - free space: /a 4430 MB (3% inode=99%): [06:00:53] PROBLEM - Disk space on search1022 is CRITICAL: DISK CRITICAL - free space: /a 4430 MB (3% inode=99%): [06:22:02] PROBLEM - Disk space on search1021 is CRITICAL: DISK CRITICAL - free space: /a 3733 MB (3% inode=99%): [06:53:15] https://en.wikipedia.org/w/api.php?format=jsonfm&action=query&list=users&usprop=blockinfo|groups|editcount|registration&ususers=Mistress%20Selina%20Kyle&meta=globaluserinfo&guiprop=groups|unattached&guiuser=Mistress%20Selina%20Kyle - anyone know why a block is showing up despite the user not being blocked? [06:57:49] Timotheus_Canens: It looks like there is a block set on the user. At least, it's showing up in the logs. [06:58:07] matthewrbowker: in 2006...there's no current block [06:58:50] and there has been a good number of blocks and unblocks since that particular block, yet that block keeps showing up [06:59:09] Timotheus_Canens: That is strange, because that's the only block in the log. And it hasn't expired. [06:59:33] matthewrbowker: hmm, what log are you looking at? [06:59:42] Timotheus_Canens: http://en.wikipedia.org/w/index.php?title=Special:BlockList&action=view&ip=Mistress_Selina_Kyle [07:00:22] matthewrbowker: but you can't unblock it... [07:00:31] Error: Block ID Mistress Selina Kyle not found. It may have been unblocked already. [07:00:42] File a bug in bugzilla please [07:00:48] and it is obviously not in effect since she can edit [07:01:18] That is very peculiar. I'm still seeing it. [07:02:31] also you don't see it if you use space instead of underscores [07:03:52] Hmmm... could there be two users with the same name, except one using underscores and one spaces? [07:07:59] ah, https://bugzilla.wikimedia.org/show_bug.cgi?id=34014 [07:08:06] PROBLEM - Disk space on search1022 is CRITICAL: DISK CRITICAL - free space: /a 3705 MB (3% inode=99%): [07:08:06] PROBLEM - Disk space on search1021 is CRITICAL: DISK CRITICAL - free space: /a 3609 MB (3% inode=99%): [07:09:28] Timotheus_Canens: That would make sense. :P [07:15:07] Actually you can just unblock #175820 since that's the block ID helpfully provided in bugzilla :D [08:22:03] RECOVERY - Puppet freshness on mw19 is OK: puppet ran at Mon Mar 26 08:21:37 UTC 2012 [08:26:33] RECOVERY - Puppet freshness on mw1073 is OK: puppet ran at Mon Mar 26 08:26:12 UTC 2012 [08:28:03] RECOVERY - Puppet freshness on mw27 is OK: puppet ran at Mon Mar 26 08:27:58 UTC 2012 [08:29:06] RECOVERY - Puppet freshness on mw30 is OK: puppet ran at Mon Mar 26 08:28:50 UTC 2012 [08:31:39] RECOVERY - Puppet freshness on mw33 is OK: puppet ran at Mon Mar 26 08:31:23 UTC 2012 [08:32:33] RECOVERY - Puppet freshness on mw45 is OK: puppet ran at Mon Mar 26 08:32:08 UTC 2012 [08:33:09] RECOVERY - Puppet freshness on mw59 is OK: puppet ran at Mon Mar 26 08:32:47 UTC 2012 [08:34:24] RECOVERY - Puppet freshness on mw72 is OK: puppet ran at Mon Mar 26 08:34:13 UTC 2012 [09:32:18] RECOVERY - Puppet freshness on brewster is OK: puppet ran at Mon Mar 26 09:31:54 UTC 2012 [09:32:18] RECOVERY - Squid on brewster is OK: TCP OK - 0.006 second response time on port 8080 [09:32:54] TimStarling: https://gerrit.wikimedia.org/r/2675 (remove bots from #wikimedia-tech) add a rebase issue because Ryan submitted the same stuff later one [09:33:08] TimStarling: I have resent the patch to just remove nagios-wm from here :-] [09:36:57] PROBLEM - Puppet freshness on ssl2 is CRITICAL: Puppet has not run in the last 10 hours [09:41:04] PROBLEM - Puppet freshness on ms-be3 is CRITICAL: Puppet has not run in the last 10 hours [09:41:40] RECOVERY - Puppet freshness on ssl2 is OK: puppet ran at Mon Mar 26 09:41:16 UTC 2012 [09:45:16] PROBLEM - Puppet freshness on search1016 is CRITICAL: Puppet has not run in the last 10 hours [09:58:10] RECOVERY - Puppet freshness on ms-be3 is OK: puppet ran at Mon Mar 26 09:57:41 UTC 2012 [09:58:28] PROBLEM - Puppet freshness on search1006 is CRITICAL: Puppet has not run in the last 10 hours [09:59:13] RECOVERY - Puppet freshness on db59 is OK: puppet ran at Mon Mar 26 09:59:05 UTC 2012 [10:10:19] PROBLEM - Puppet freshness on owa3 is CRITICAL: Puppet has not run in the last 10 hours [10:10:19] PROBLEM - Puppet freshness on amslvs2 is CRITICAL: Puppet has not run in the last 10 hours [10:10:19] PROBLEM - RAID on searchidx2 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [10:12:34] RECOVERY - RAID on searchidx2 is OK: OK: State is Optimal, checked 4 logical device(s) [10:21:16] PROBLEM - Puppet freshness on owa2 is CRITICAL: Puppet has not run in the last 10 hours [10:21:16] PROBLEM - Puppet freshness on owa1 is CRITICAL: Puppet has not run in the last 10 hours [10:24:47] Time to live exceeded? [10:25:03] oh, working again :) [13:24:05] RECOVERY - MySQL disk space on db42 is OK: DISK OK [13:25:44] RECOVERY - Disk space on db42 is OK: DISK OK [13:45:14] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 3.740 seconds [13:48:23] RECOVERY - DPKG on db42 is OK: All packages OK [13:54:41] PROBLEM - DPKG on db42 is CRITICAL: DPKG CRITICAL dpkg reports broken packages [13:56:47] RECOVERY - DPKG on db42 is OK: All packages OK [14:04:53] PROBLEM - Puppet freshness on amslvs4 is CRITICAL: Puppet has not run in the last 10 hours [14:18:59] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:25:08] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 4.144 seconds [14:58:51] PROBLEM - Host db59 is DOWN: PING CRITICAL - Packet loss = 100% [15:00:39] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:04:51] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 9.102 seconds [15:11:18] RECOVERY - Host db59 is UP: PING OK - Packet loss = 0%, RTA = 0.37 ms [15:15:46] is or should bots.wmflabs.org/ have https:// ??? [15:15:48] PROBLEM - DPKG on db59 is CRITICAL: Connection refused by host [15:16:15] PROBLEM - MySQL disk space on db59 is CRITICAL: Connection refused by host [15:16:24] PROBLEM - Disk space on db59 is CRITICAL: Connection refused by host [15:17:27] PROBLEM - RAID on db59 is CRITICAL: Connection refused by host [15:17:36] PROBLEM - SSH on db59 is CRITICAL: Connection refused [15:20:36] PROBLEM - Disk space on srv220 is CRITICAL: DISK CRITICAL - free space: / 0 MB (0% inode=61%): /var/lib/ureadahead/debugfs 0 MB (0% inode=61%): [15:35:27] RECOVERY - Disk space on srv220 is OK: DISK OK [15:40:33] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:44:45] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 9.898 seconds [15:58:58] see ya [16:19:42] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:25:51] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.950 seconds [16:43:44] [[Tech]]; Edgars2007; /* Global things */ new section; https://meta.wikimedia.org/w/index.php?diff=3595436&oldid=3571844&rcid=3201132 [17:01:24] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:05:36] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 7.401 seconds [17:41:06] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:43:48] !log nikerabbit synchronized php-1.19/extensions/WebFonts/ 'i18ndeploy r114492' [17:47:24] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.036 seconds [18:06:45] PROBLEM - Packetloss_Average on locke is CRITICAL: CRITICAL: packet_loss_average is 8.32450678571 (gt 8.0) [18:13:03] RECOVERY - Packetloss_Average on locke is OK: OK: packet_loss_average is 1.14684903509 [18:21:09] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:27:18] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.028 seconds [19:00:59] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:07:08] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.025 seconds [19:07:14] !log rebooting ms1001 (new kernel) [19:08:02] PROBLEM - Packetloss_Average on emery is CRITICAL: CRITICAL: packet_loss_average is 8.36590393162 (gt 8.0) [19:20:38] RECOVERY - Host cp1019 is UP: PING OK - Packet loss = 0%, RTA = 26.58 ms [19:22:44] RECOVERY - Packetloss_Average on emery is OK: OK: packet_loss_average is 3.55783869565 [19:25:53] PROBLEM - Frontend Squid HTTP on cp1019 is CRITICAL: Connection refused [19:26:47] PROBLEM - Backend Squid HTTP on cp1019 is CRITICAL: Connection refused [19:33:32] PROBLEM - Puppet freshness on brewster is CRITICAL: Puppet has not run in the last 10 hours [19:41:02] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:43:19] I am also curious if I can control the width of a page. I want to prevent http://en.wikipedia.org/wiki/Template:Ranks_and_insignia_of_NATO/Generic/Army overflow with an h-bar [19:46:26] PROBLEM - Puppet freshness on search1016 is CRITICAL: Puppet has not run in the last 10 hours [19:46:54] needs a puppet master [19:47:02] RECOVERY - Frontend Squid HTTP on cp1019 is OK: HTTP OK HTTP/1.0 200 OK - 27535 bytes in 0.162 seconds [19:47:11] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.021 seconds [19:47:47] RECOVERY - Backend Squid HTTP on cp1019 is OK: HTTP OK HTTP/1.0 200 OK - 27399 bytes in 0.174 seconds [19:49:09] ToAruShiroiNeko: I am not sure I understand :D [19:49:30] ToAruShiroiNeko: each column has the width of the largest picture [19:49:30] hashar ok [19:49:42] do you see how the table exeeds the article space? [19:49:47] ToAruShiroiNeko: so the table width is at least the width of all pictures [19:49:53] right [19:50:12] mediawiki knows how wide my screen is [19:50:16] so if your window is smaller, the browser show an horizontal scrolling bar [19:50:22] thats fine [19:50:26] the problem is the overflow [19:50:43] it goes beyond the log out line [19:50:51] ohh [19:51:03] I want to keep it within that border [19:51:14] but there is too many countries so it will exeed no matter what I do [19:51:22] so I want to limit the width [19:51:35] ideally I want the grades to always show [19:52:02] I see [19:52:07] not sure how you can do that using html [19:52:19] simple solution is frames [19:52:26] which is hardly simple [19:52:45] or java maybe [19:52:52] I think this would be an interesting feature [19:53:38] CSS 3 has the overflow parameter http://www.brunildo.org/test/Overflowxy2.html [19:53:44] you might be using that [19:54:19] but it would apply to the whole table and you will probably not be able to keep the left column to "stick" [19:54:41] we have the same issue with veryyy long tables [19:54:56] it would be great to have the first row to always be shown [19:55:02] and move as we scroll down [19:55:27] hmm [19:55:30] interesting [19:55:33] which one would I want? [19:55:49] It could be two tables [19:55:56] or that http://acatalept.com/common/test/fixed-table.html /D [19:56:01] though I dislike the idea the second I typed it [19:56:14] I want the hbar only [19:56:19] not the vbar [19:56:20] that is the same issue [19:56:34] that example use javascript though :) [19:56:48] well how would I use overflow just to gain an hbar? :/ [19:57:00] this is templated so it is trivial to apply changes to every cell [19:57:47] going to try something [19:57:51] ok [19:58:09] mind that changes will not immidiately apply [19:58:31] you must clear chache [19:58:53] *purge [20:00:32] PROBLEM - Puppet freshness on search1006 is CRITICAL: Puppet has not run in the last 10 hours [20:00:32] PROBLEM - Puppet freshness on db59 is CRITICAL: Puppet has not run in the last 10 hours [20:03:11] ToAruShiroiNeko: a pure CSS version : http://www.imaputz.com/cssStuff/bigFourVersion.html [20:03:19] ToAruShiroiNeko: but that is with the first row only [20:03:36] and the header have the wrong size [20:03:59] ToAruShiroiNeko: I am not sure your problem could be solved by only using CSS :/ [20:08:07] PROBLEM - Packetloss_Average on emery is CRITICAL: CRITICAL: packet_loss_average is 9.12571974359 (gt 8.0) [20:09:16] hashar well give it your best shot [20:09:31] :P [20:09:37] that surely can be done using javascript though [20:10:40] another thing I would like to do is [20:10:43] but for notes [20:11:43] PROBLEM - Puppet freshness on amslvs2 is CRITICAL: Puppet has not run in the last 10 hours [20:11:43] PROBLEM - Puppet freshness on owa3 is CRITICAL: Puppet has not run in the last 10 hours [20:12:19] RECOVERY - Packetloss_Average on emery is OK: OK: packet_loss_average is 1.44555983051 [20:22:04] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:22:49] PROBLEM - Puppet freshness on owa1 is CRITICAL: Puppet has not run in the last 10 hours [20:22:49] PROBLEM - Puppet freshness on owa2 is CRITICAL: Puppet has not run in the last 10 hours [20:28:13] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.045 seconds [20:31:02] hashar one final thing is if you could explain me why this displays correctly: http://en.wikipedia.org/wiki/Template:Ranks_and_insignia_of_NATO/Generic/Army/OR-03 [20:31:08] but the one on the previous page does not [20:31:38] ToAruShiroiNeko: both look the same to me (using Safari) [20:32:06] the or-3 line looks broken [20:32:14] http://en.wikipedia.org/wiki/Template:Ranks_and_insignia_of_NATO/Generic/Army [20:32:20] see how UK and us ranks dont properly display? [20:32:39] it only reads Lance Bombardier [20:32:52] when it should read Lance Corporal/Lance Bombardier [20:32:54] both linked [20:33:10] US rank is Private First Class as well [20:33:13] I got an insignia just above "Lance Bombardier" [20:36:22] right [20:36:25] the image shows fine [20:36:30] the problem is with the text [20:36:43] the insignia is correct [20:44:06] hashar do you see it now? [20:45:03] what? [20:45:14] http://en.wikipedia.org/wiki/Template:Ranks_and_insignia_of_NATO/Generic/Army ? [20:46:05] ToAruShiroiNeko: that is what I got : http://imgur.com/V5L52 [20:50:33] yes [20:50:37] you see the problem [20:50:48] no text under both US and UK insignia [20:51:00] compare it to how they appear under http://en.wikipedia.org/wiki/Template:Ranks_and_insignia_of_NATO/Generic/Army/OR-03 [20:51:37] see the difference? [21:01:49] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:04:42] * ToAruShiroiNeko stabs hashar [21:05:02] ToAruShiroiNeko: :) [21:05:32] any thoughts? [21:05:46] ToAruShiroiNeko: maybe it is a caching issue [21:05:52] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.690 seconds [21:05:55] you might have an old version of that template (the OR-3) [21:09:31] not really [21:09:41] OR-4 and OR-3 are using the same template [21:09:44] it is a copy paste [21:11:25] it seems like a chache issue I cannot shake off [21:11:25] http://en.wikipedia.org/wiki/User:%E3%81%A8%E3%81%82%E3%82%8B%E7%99%BD%E3%81%84%E7%8C%AB/Sandbox#Insignia [21:11:32] that is a direct copy paste of code [21:11:39] and the problem does not show itself [21:12:32] it does show when I post all templates tho [21:12:39] I really am puzzled [21:41:03] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:47:30] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 4.157 seconds [22:29:24] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:32:22] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.021 seconds [22:35:34] PROBLEM - Host ps1-a3-sdtpa is DOWN: PING CRITICAL - Packet loss = 100% [22:35:52] PROBLEM - Host ps1-a4-sdtpa is DOWN: PING CRITICAL - Packet loss = 0%, RTA = 2111.02 ms [22:37:04] RECOVERY - Host ps1-a3-sdtpa is UP: PING OK - Packet loss = 0%, RTA = 1.94 ms [22:37:31] PROBLEM - Host ps1-b2-sdtpa is DOWN: PING CRITICAL - Packet loss = 44%, RTA = 2162.35 ms [22:39:19] RECOVERY - Host ps1-b2-sdtpa is UP: PING OK - Packet loss = 0%, RTA = 2.77 ms [22:39:55] RECOVERY - Host ps1-a4-sdtpa is UP: PING OK - Packet loss = 0%, RTA = 2.25 ms [22:53:16] gn8 folks [23:03:46] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:09:55] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 1.251 seconds [23:18:17] !log awjrichards synchronizing Wikimedia installation... : Syncing MobileFrontend to r114504 changes per http://www.mediawiki.org/wiki/Extension:MobileFrontend/Deployments#26_March.2C_2012 [23:31:23] sync done. [23:35:10] PROBLEM - LVS HTTP on m.wikimedia.org is CRITICAL: HTTP CRITICAL - pattern not found [23:45:40] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:51:49] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK HTTP/1.1 400 Bad Request - 335 bytes in 0.036 seconds