[00:46:57] PROBLEM - Long running screen/tmux on labstore1007 is CRITICAL: CRIT: Long running SCREEN process. (PID: 1876, 1738068s 1728000s). [00:49:32] (03PS2) 10ArielGlenn: archiveloader full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280102 [00:52:00] what's going on with dbstore1002? [00:52:20] I just now see there's a failed delete there [00:52:30] is this known? [00:57:47] https://phabricator.wikimedia.org/T161294 this is the last reference I see to something going on with that and related hosts [00:57:53] mutante: any ideas? [00:58:32] I kinda need to pass this to someone in a real tz, it's 3 am and I need to get some sleep [00:59:49] apergos: whats' up [00:59:51] (03PS3) 10Dzahn: icinga: script to send custom SMS to Icinga contacts [puppet] - 10https://gerrit.wikimedia.org/r/400615 (https://phabricator.wikimedia.org/T82937) [01:00:10] i didnt get a text because the phone is down maybe [01:00:15] (03CR) 10jerkins-bot: [V: 04-1] icinga: script to send custom SMS to Icinga contacts [puppet] - 10https://gerrit.wikimedia.org/r/400615 (https://phabricator.wikimedia.org/T82937) (owner: 10Dzahn) [01:00:28] I don't know that it pages for that host [01:01:36] Cant find record in imagelinks, Error_code: 1032: handler error HA_ERR_KEY_NOT_FOUND not loving that at all [01:07:05] anyways, can you have an eye on it or pass it on as needed? [01:07:48] eh, yea, well, it doesnt seeem like we need to call a DBA just yet, but that's really afaict [01:07:51] yea [01:08:30] ok, thanks [01:08:54] gotta get some shuteye, hopefully it can stay that way til tomorrow when a dba is awake [01:09:32] good night! yes, i think so [01:10:16] i think based on "it's just slave" we can assume that [01:10:30] from motd "dbstore1002 is a Delayed Slave " [01:13:46] (03PS4) 10Dzahn: icinga: script to send custom SMS to Icinga contacts [puppet] - 10https://gerrit.wikimedia.org/r/400615 (https://phabricator.wikimedia.org/T82937) [01:14:07] (03CR) 10jerkins-bot: [V: 04-1] icinga: script to send custom SMS to Icinga contacts [puppet] - 10https://gerrit.wikimedia.org/r/400615 (https://phabricator.wikimedia.org/T82937) (owner: 10Dzahn) [01:31:53] (03PS5) 10Dzahn: icinga: script to send custom SMS to Icinga contacts [puppet] - 10https://gerrit.wikimedia.org/r/400615 (https://phabricator.wikimedia.org/T82937) [03:27:37] PROBLEM - MariaDB Slave Lag: s1 on dbstore1002 is CRITICAL: CRITICAL slave_sql_lag Replication lag: 859.34 seconds [03:49:46] RECOVERY - MariaDB Slave Lag: s1 on dbstore1002 is OK: OK slave_sql_lag Replication lag: 206.15 seconds [04:24:26] PROBLEM - puppet last run on lvs4007 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [04:54:26] RECOVERY - puppet last run on lvs4007 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [07:41:38] mutante, apergos yeah, no need to call us if dbstore1002 crashes, we need to rebuild that hosts anyways, its consistency sucks [07:41:41] I will fix it now [07:41:45] that table I mean [07:43:27] RECOVERY - MariaDB Slave SQL: s5 on dbstore1002 is OK: OK slave_sql_state Slave_SQL_Running: Yes [07:43:53] !log Fixing dbstore1002 dewiki.pagelinks [07:44:05] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [07:58:18] (03PS1) 10Marostegui: db-eqiad.php: Depool db1110 [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400630 [08:00:31] (03CR) 10Marostegui: [C: 032] db-eqiad.php: Depool db1110 [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400630 (owner: 10Marostegui) [08:01:57] (03Merged) 10jenkins-bot: db-eqiad.php: Depool db1110 [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400630 (owner: 10Marostegui) [08:02:12] (03CR) 10jenkins-bot: db-eqiad.php: Depool db1110 [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400630 (owner: 10Marostegui) [08:03:33] !log marostegui@tin Synchronized wmf-config/db-eqiad.php: Depool db1110 to reimport dewiki.imagelinks on dbstore1002 (duration: 00m 52s) [08:03:43] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [08:19:21] marostegui: ok gtk [08:19:44] !log Stop replication in sync on dbstore1002.s5 and db1110 [08:19:54] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [08:20:17] what is the underlying issue that causes these consistency problems anyways? is that known? [08:20:35] apergos: yeah, lots of past crashes+tokudb [08:20:53] ouch [08:21:11] apergos: the fact that we are running row based replication (by mistake) on s5, makes these consistency problems arise, as row based replication will break if there is a missing row [08:21:26] RECOVERY - MariaDB Slave Lag: s5 on dbstore1002 is OK: OK slave_sql_lag Replication lag: 231.07 seconds [08:22:08] oh joy [08:22:10] we have fixed lots of incosistencies on dbstore1002 already, but it will not be 100% till we rebuild it anyways [08:22:22] I had seen that on the past ticket, that there were a lot of things fixed [08:22:33] didn't realize a rebuild was still needed though [08:23:03] so wait, it's still running on toku? I thought all those hosts had been converted [08:23:21] nope, only codfw hosts [08:23:34] dbstore1002 (analytics) and dbstore1001 (backups) still run toku :( [08:24:23] we will go back to statement replication on s5 the 9th of Jan, when we do the split for s5/s8 as a read_only period is required, we will restart the master to go back to STATEMENT [08:26:04] gotcha [08:28:25] (03PS1) 10Marostegui: Revert "db-eqiad.php: Depool db1110" [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400631 [08:30:15] (03CR) 10Marostegui: [C: 032] Revert "db-eqiad.php: Depool db1110" [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400631 (owner: 10Marostegui) [08:32:43] (03Merged) 10jenkins-bot: Revert "db-eqiad.php: Depool db1110" [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400631 (owner: 10Marostegui) [08:32:58] (03CR) 10jenkins-bot: Revert "db-eqiad.php: Depool db1110" [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400631 (owner: 10Marostegui) [08:34:18] !log marostegui@tin Synchronized wmf-config/db-eqiad.php: Repool db1110 (duration: 00m 52s) [08:34:27] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [08:40:20] I have to go run an errand, back in about 90 minutes, maybe a little less [10:05:29] (03CR) 10TerraCodes: [C: 031] "> > Wouldn't it break things like global userpages having "hi, you" [mediawiki-config] - 10https://gerrit.wikimedia.org/r/397768 (https://phabricator.wikimedia.org/T182541) (owner: 10EddieGP) [10:11:25] (03Draft2) 10Jayprakash12345: Turn on mapframe for Arabic Wikipedia [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400682 [10:12:07] (03PS3) 10Jayprakash12345: Turn on mapframe for Arabic Wikipedia [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400682 (https://phabricator.wikimedia.org/T183764) [10:22:16] (03CR) 10TerraCodes: [C: 031] Turn on mapframe for Arabic Wikipedia [mediawiki-config] - 10https://gerrit.wikimedia.org/r/400682 (https://phabricator.wikimedia.org/T183764) (owner: 10Jayprakash12345) [12:26:48] (03PS1) 10ArielGlenn: make all misc dump jobs plus xml dumps config use hiera setting for php [puppet] - 10https://gerrit.wikimedia.org/r/400692 [12:38:05] (03PS2) 10ArielGlenn: make all misc dump jobs plus xml dumps config use hiera setting for php [puppet] - 10https://gerrit.wikimedia.org/r/400692 [12:55:49] (03PS3) 10ArielGlenn: make all misc dump jobs plus xml dumps config use hiera setting for php [puppet] - 10https://gerrit.wikimedia.org/r/400692 [13:03:10] (03CR) 10ArielGlenn: [C: 032] make all misc dump jobs plus xml dumps config use hiera setting for php [puppet] - 10https://gerrit.wikimedia.org/r/400692 (owner: 10ArielGlenn) [14:29:47] PROBLEM - puppet last run on tin is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:06] PROBLEM - puppet last run on nitrogen is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:06] PROBLEM - puppet last run on rhodium is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:06] PROBLEM - puppet last run on terbium is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:16] PROBLEM - puppet last run on cp4025 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:18] PROBLEM - puppet last run on mw1188 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:46] PROBLEM - puppet last run on labsdb1004 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:56] PROBLEM - puppet last run on mw1282 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:56] PROBLEM - puppet last run on conf1002 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:56] PROBLEM - puppet last run on dysprosium is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:30:57] PROBLEM - puppet last run on mw1312 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:31:06] PROBLEM - puppet last run on contint2001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:31:07] PROBLEM - puppet last run on elastic1043 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:31:26] PROBLEM - puppet last run on conf1003 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:31:26] PROBLEM - puppet last run on mw1187 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:31:56] PROBLEM - puppet last run on db1072 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:31:56] PROBLEM - puppet last run on analytics1065 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:32:56] PROBLEM - puppet last run on einsteinium is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:33:36] PROBLEM - puppet last run on analytics1051 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:33:37] PROBLEM - puppet last run on db1067 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:33:56] PROBLEM - puppet last run on mw1275 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:34:36] PROBLEM - puppet last run on ms-be1018 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:41:56] uh/ [14:42:15] apergos i think it may be puppetdb. [14:43:25] nittrogen puppetdb grrrr [14:45:12] I've restarted it over there, let's see what happens [14:46:58] !log restarted puppetdb on nitrogen, puppetdb refusing to hand out facts again [14:47:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [14:47:31] reran puppet on a couple of those hosts, it's fine now [14:47:56] PROBLEM - puppet last run on analytics1069 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:48:13] * apergos eyes analytics1069 [14:48:16] PROBLEM - puppet last run on restbase1007 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:48:27] PROBLEM - puppet last run on labvirt1011 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:48:56] PROBLEM - puppet last run on db1031 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:49:10] ah it is reporting a run from before the restart [14:49:26] PROBLEM - puppet last run on cp1046 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:49:26] PROBLEM - puppet last run on db1098 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:49:46] PROBLEM - puppet last run on mw1328 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:49:47] RECOVERY - puppet last run on tin is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [14:49:56] PROBLEM - puppet last run on etcd1001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:50:06] RECOVERY - puppet last run on nitrogen is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [14:50:06] PROBLEM - puppet last run on mw1250 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:50:46] PROBLEM - puppet last run on cp1072 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:50:47] PROBLEM - puppet last run on sca1004 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:50:57] PROBLEM - puppet last run on analytics1031 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:51:16] PROBLEM - puppet last run on mw1227 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:55:06] RECOVERY - puppet last run on mw1250 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [14:58:38] RECOVERY - puppet last run on db1067 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:58:56] RECOVERY - puppet last run on mw1275 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [14:59:36] RECOVERY - puppet last run on ms-be1018 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:00:06] RECOVERY - puppet last run on rhodium is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:00:06] RECOVERY - puppet last run on terbium is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:00:07] so that was an oom of the java process btw, in case anyone does the backread [15:00:16] RECOVERY - puppet last run on cp4025 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:00:16] RECOVERY - puppet last run on mw1188 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:00:33] Dec 29 14:27:18 nitrogen kernel: [15805774.341714] Out of memory: Kill process 19649 (java) score 358 or sacrifice child [15:00:43] Dec 29 14:27:18 nitrogen kernel: [15805774.342477] Killed process 19649 (java) total-vm:12285380kB, anon-rss:6243284kB, file-rss:0kB, shmem-rss:0kB [15:00:46] RECOVERY - puppet last run on labsdb1004 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:00:56] RECOVERY - puppet last run on mw1282 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:00:56] RECOVERY - puppet last run on conf1002 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:00:56] RECOVERY - puppet last run on dysprosium is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:00:57] RECOVERY - puppet last run on mw1312 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:01:06] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [15:01:07] RECOVERY - puppet last run on elastic1043 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:01:26] RECOVERY - puppet last run on conf1003 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [15:01:27] RECOVERY - puppet last run on mw1187 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [15:01:56] RECOVERY - puppet last run on db1072 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [15:01:56] RECOVERY - puppet last run on analytics1065 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [15:02:56] RECOVERY - puppet last run on einsteinium is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [15:03:36] RECOVERY - puppet last run on analytics1051 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [15:15:47] RECOVERY - puppet last run on cp1072 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [15:15:47] RECOVERY - puppet last run on sca1004 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures [15:15:57] RECOVERY - puppet last run on analytics1031 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [15:16:16] RECOVERY - puppet last run on mw1227 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [15:17:56] RECOVERY - puppet last run on analytics1069 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:18:16] RECOVERY - puppet last run on restbase1007 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:18:27] RECOVERY - puppet last run on labvirt1011 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [15:18:56] RECOVERY - puppet last run on db1031 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [15:19:26] RECOVERY - puppet last run on cp1046 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [15:19:26] RECOVERY - puppet last run on db1098 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [15:19:46] RECOVERY - puppet last run on mw1328 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [15:19:56] RECOVERY - puppet last run on etcd1001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [15:30:48] (03PS2) 10ArielGlenn: enable pagelogs to be dumped by several processes in parallel [puppet] - 10https://gerrit.wikimedia.org/r/399589 (https://phabricator.wikimedia.org/T181935) [15:31:58] (03CR) 10ArielGlenn: [C: 032] enable pagelogs to be dumped by several processes in parallel [puppet] - 10https://gerrit.wikimedia.org/r/399589 (https://phabricator.wikimedia.org/T181935) (owner: 10ArielGlenn) [15:35:45] (03PS11) 10ArielGlenn: ability to do xmlpageslogging several pieces at a time in parallel [dumps] - 10https://gerrit.wikimedia.org/r/394857 (https://phabricator.wikimedia.org/T181935) [15:36:27] (03CR) 10ArielGlenn: [C: 032] ability to do xmlpageslogging several pieces at a time in parallel [dumps] - 10https://gerrit.wikimedia.org/r/394857 (https://phabricator.wikimedia.org/T181935) (owner: 10ArielGlenn) [15:38:33] !log ariel@tin Started deploy [dumps/dumps@51d4fd6]: enable pageslogging dump in parallel jobs [15:38:36] !log ariel@tin Finished deploy [dumps/dumps@51d4fd6]: enable pageslogging dump in parallel jobs (duration: 00m 02s) [15:38:43] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:38:52] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [16:39:06] Hi, is there anybody around who would be willing to help me with a login related query? [16:40:02] probably. feel free to just ask [16:51:46] (03PS3) 10ArielGlenn: archiveloader full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280102 [17:42:27] (03PS4) 10ArielGlenn: archiveloader full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280102 [18:23:14] (03PS5) 10ArielGlenn: archiveloader full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280102 [18:31:51] !log gerrit: restart service to apply change from Dec 20th, avoid logspam due to unapplied config change [18:32:01] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:32:31] done. it's back [18:37:26] PROBLEM - puppet last run on labsdb1009 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_operations/mediawiki-config] [18:37:47] PROBLEM - puppet last run on labsdb1010 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_operations/mediawiki-config] [18:39:07] PROBLEM - puppet last run on kafka2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 6 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_mediawiki/event-schemas] [18:41:13] ... runs puppet on kafka2002 [18:41:23] that better be temp due to the gerrit restart and not a real problem [18:41:24] (03PS6) 10ArielGlenn: archiveloader full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280102 [18:44:06] RECOVERY - puppet last run on kafka2002 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:47:26] RECOVERY - puppet last run on labsdb1009 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [18:47:47] RECOVERY - puppet last run on labsdb1010 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [18:49:21] yes, i ran it on all 3.. it's nothing [18:52:02] thought so [19:05:09] (03Abandoned) 10Paladox: contint: Doin't install the npm package on stretch [puppet] - 10https://gerrit.wikimedia.org/r/397720 (owner: 10Paladox) [21:01:43] (03CR) 10ArielGlenn: [C: 032] archiveloader full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280102 (owner: 10ArielGlenn) [21:21:20] (03PS2) 10ArielGlenn: fixup-interwikis: full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280103 [21:22:06] (03CR) 10ArielGlenn: [C: 032] fixup-interwikis: full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280103 (owner: 10ArielGlenn) [21:26:13] (03PS2) 10ArielGlenn: thumbDateAnalysis full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280104 [21:28:12] (03CR) 10ArielGlenn: [C: 032] thumbDateAnalysis full pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280104 (owner: 10ArielGlenn) [21:41:06] (03PS2) 10ArielGlenn: pylint and pep8 for scripts related to media tarball creation [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280108 [21:56:45] (03PS2) 10ArielGlenn: toy offline reader: pylint and pep8 [dumps] (ariel) - 10https://gerrit.wikimedia.org/r/280109 [23:34:56] (03PS5) 10Phantom42: Fix linewrap issue on wikimedia error page [puppet] - 10https://gerrit.wikimedia.org/r/395552 (https://phabricator.wikimedia.org/T180656) [23:36:53] (03CR) 10Phantom42: "Sorry for some delays! I added needed css for that. Now long words are handled correctly and hyphens are placed if it is possible (is brow" [puppet] - 10https://gerrit.wikimedia.org/r/395552 (https://phabricator.wikimedia.org/T180656) (owner: 10Phantom42)