[00:05:30] 10Toolforge, 10Outreachy (Round-15): Improvements for the Toolforge 'webservice' command - https://phabricator.wikimedia.org/T175768#3602769 (10bd808) [00:15:12] !log tools forced to clean out exim queue as the filesystem used up all inodes [00:15:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [00:34:53] 10Tool-Article-request, 10Toolforge, 10User-Matthewrbowker: Uptimerobot monitoring for the Articlerequest tool flaps - https://phabricator.wikimedia.org/T175623#3602835 (10Matthewrbowker) I've switched the monitor to use the http://tools.wmflabs.org/articlerequest/ link, though I have the same monitor runnin... [00:37:59] Just logged in to Toolsadmin for the first time (been a while since I had to do adminy stuff with tools) and... holy shit we're in the future [00:39:06] and, holy damn you can create diffusion repos through it? [00:39:32] (I still take it we can't delete tool accounts yet though?) [00:43:12] !log tools Disabled puppet on tools-mail [00:43:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [00:43:43] 10Toolforge, 10Tracking: Tools that should get deleted (tracking) - https://phabricator.wikimedia.org/T133777#3602850 (10Liuxinyu970226) [00:43:46] !log tools Stopped exim on tools-mail [00:43:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [00:44:27] harej: no tool account deletion yet, no. It turns out to be a tricky problem to automate. [00:44:56] harej: but diffusion repo creation has been a thing for a year :) [00:45:09] through toolsadmin? [00:46:08] yeah. that was the first feature I built into it [00:46:54] creating tools and toolinfo.json there is pretty new [00:47:04] hmm. it *has* been a while, i suppose [00:47:47] !log tools Archived and then purged /var/spool/exim4/msglog on tools-mail [00:47:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [00:50:25] also, is there a reason why september dumps are not available in /public/dumps even if they are available on dumps.wikimedia.org? [00:52:51] harej: because ... not sure. We don't get realtime updates from the dumps generation server. Not sure what the schedule is [00:52:58] Fascinating. [00:53:21] In the meantime, should I just download the dump to my tool account? [00:53:38] please don't [00:53:59] you will melt the NFS server and people will be sad [00:54:07] see, that's what I want to avoid doing [00:54:52] I guess for now I will use the slightly older thing. But thank you for updating me [00:55:17] (I was worried at first... then I was like... it's only a month old, come on.) [00:55:22] !log tools Archived and then purged /var/spool/exim4/input on tools-mail [00:55:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [00:56:04] harej: feel free to file a bug and when I'm not fighting a bigger fire I'll look into the dumps thing [00:56:29] What would be nice would be if the dumps server was somehow hooked into the toolforge file system [00:56:46] like, going to /mnt/dumpsland or whatever would take you to the dumps [00:57:45] harej: we're working on it. new servers and new process coming [00:57:50] :] [00:57:59] Data Services™! [01:00:29] (how big is the toolforge's hard drive, anyway? not that i am planning a stress test or anything) [01:01:44] harej: type `df -h` on a bastion [01:02:01] there are many volumes of many sizes [01:02:09] hmm, less than i thought [01:02:16] wait [01:02:24] * harej missed the bit that said "28T" [01:02:28] :) [01:02:31] but that's the dumps [01:02:48] other than that, surprisingly little space all things considered [01:02:53] the /home is 8T [01:03:02] you may be pleased to hear i am planning on dropping a 25 GB database [01:03:04] and so is the tool's home [01:03:44] (how soon i do so depends on how much i want to piss off my downstream users ;) [01:05:36] 10Tools, 10I18n: Rename Tool labs to Toolforge in Persönliche Bekanntschaften tool - https://phabricator.wikimedia.org/T175774#3602915 (10Liuxinyu970226) [01:06:12] 10Tools, 10I18n: Rename Tool labs to Toolforge in Persönliche Bekanntschaften tool - https://phabricator.wikimedia.org/T175774#3602927 (10Liuxinyu970226) [01:09:39] !log tools Removed user WiktCAPT from project [01:09:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [01:27:49] PROBLEM - Puppet errors on tools-exec-1402 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [01:30:12] I tried creating a virtualenv just now and it did not succeed because "ensurepip is not available" [01:32:46] I've definitely done this before yet I can't find the documentation on Wikitech [01:40:35] (I ended up following the webserver instructions, which seem to work) [01:43:21] harej: there is a bug in the python3 venv lib on the Trusty bastions [01:43:43] there's an open task about it somewhere in Phab [01:43:53] not too likely to be fixed directly by us [01:44:05] it's an upstream problem? [01:44:26] yeah. a bug in the Ubuntu python3 package(s) [01:44:31] ah [01:45:22] the recommended fix upstream the last I looked was to use pip to install a non-broken package :/ [01:51:40] !log tools Nuked all messages in the exim spool on tools-mail [01:51:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [02:02:50] RECOVERY - Puppet errors on tools-exec-1402 is OK: OK: Less than 1.00% above the threshold [0.0] [02:10:12] !log tools Really disabled puppet on tools-mail [02:10:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [02:23:51] PROBLEM - Puppet errors on tools-exec-1402 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [02:33:25] PROBLEM - Puppet errors on tools-paws-worker-1016 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [03:08:22] RECOVERY - Puppet errors on tools-paws-worker-1016 is OK: OK: Less than 1.00% above the threshold [0.0] [03:28:50] RECOVERY - Puppet errors on tools-exec-1402 is OK: OK: Less than 1.00% above the threshold [0.0] [03:39:37] PROBLEM - Puppet errors on tools-exec-1411 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [04:34:38] RECOVERY - Puppet errors on tools-exec-1411 is OK: OK: Less than 1.00% above the threshold [0.0] [06:34:13] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1412 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:00:12] PROBLEM - Puppet errors on tools-exec-1430 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:09:14] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1412 is OK: OK: Less than 1.00% above the threshold [0.0] [07:34:51] RECOVERY - Puppet errors on tools-exec-1430 is OK: OK: Less than 1.00% above the threshold [0.0] [08:46:50] 10Tools: Need easier tool for working on redundancy than "Inhalte übernommen" Template Tool (german WP) - https://phabricator.wikimedia.org/T175698#3603502 (10Nanimo5) [09:40:34] (03PS1) 10Jean-Frédéric: Fix wlm-latest tool [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377724 [09:40:36] (03PS1) 10Jean-Frédéric: Update Commons database config in api code [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377725 [09:59:57] How do I see a vandal's ip on a wiki if I can access logstash kibana but have no shell access? [10:19:59] bd808: hi, can you check if ReleaseTaggerBot (forrestbot) is working on Toolforge? [10:20:01] I suspect maybe the grid job is stuck or cancelled? [10:21:13] valhallasw`cloud: / legoktm as well -- ReleaseTaggerBot isn't working for some days [10:22:03] for wmf.17 and past branches it does, but not for wmf.18 et seq [10:22:20] cf. T175626 [10:22:20] T175626: @ReleaseTaggerBot not working for wmf/1.30.0-wmf.18 et seq. - https://phabricator.wikimedia.org/T175626 [11:14:10] 10Toolforge, 10Tools: Response time from https://tools.wmflabs.org/scholia/ varies - https://phabricator.wikimedia.org/T175703#3603954 (10Fnielsen) Added as a Scholia issue: https://github.com/fnielsen/scholia/issues/192 [11:15:14] 10Toolforge, 10Tools: Response time from https://tools.wmflabs.org/scholia/ varies - https://phabricator.wikimedia.org/T175703#3603958 (10Fnielsen) 05Open>03Resolved a:03Fnielsen [11:25:53] (03CR) 10Lokal Profil: "Why did you drop the timestamp from the echo?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377483 (https://phabricator.wikimedia.org/T174871) (owner: 10Jean-Frédéric) [11:26:06] PROBLEM - Puppet errors on tools-worker-1023 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [11:34:00] (03CR) 10Lokal Profil: [C: 031] "one question mark. but if that is on purpose or fixed then go ahead and +2" (031 comment) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377488 (https://phabricator.wikimedia.org/T174261) (owner: 10Jean-Frédéric) [11:38:36] (03CR) 10Lokal Profil: [C: 031] "Same question mark as for Egypt. it should also be possible to make use of "governorate" for adm1 but honestly I don't know if we ever use" (031 comment) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377508 (https://phabricator.wikimedia.org/T174340) (owner: 10Jean-Frédéric) [11:40:47] (03PS7) 10Jean-Frédéric: Add Egypt in Arabic eg_ar [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377488 (https://phabricator.wikimedia.org/T174261) [11:41:39] (03CR) 10Lokal Profil: "would you mind removing the trailing whitespace while you are updating the file?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377725 (owner: 10Jean-Frédéric) [11:41:41] (03PS2) 10Jean-Frédéric: Add Irak in Arabic (iq_ar) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377508 (https://phabricator.wikimedia.org/T174340) [11:42:01] (03CR) 10Jean-Frédéric: [C: 032] Add Egypt in Arabic eg_ar (031 comment) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377488 (https://phabricator.wikimedia.org/T174261) (owner: 10Jean-Frédéric) [11:42:18] (03CR) 10Jean-Frédéric: "Good catch with the trailing space :)" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377508 (https://phabricator.wikimedia.org/T174340) (owner: 10Jean-Frédéric) [11:42:50] (03CR) 10Jean-Frédéric: "> it should also be possible to make use of "governorate" for adm1 but honestly I don't know if we ever use those fields." [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377508 (https://phabricator.wikimedia.org/T174340) (owner: 10Jean-Frédéric) [11:42:57] (03Merged) 10jenkins-bot: Add Egypt in Arabic eg_ar [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377488 (https://phabricator.wikimedia.org/T174261) (owner: 10Jean-Frédéric) [11:43:42] (03CR) 10jenkins-bot: Add Egypt in Arabic eg_ar [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377488 (https://phabricator.wikimedia.org/T174261) (owner: 10Jean-Frédéric) [11:44:55] (03CR) 10Lokal Profil: [C: 032] "+2 ing but we should make sure we get the report pages in there." [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377508 (https://phabricator.wikimedia.org/T174340) (owner: 10Jean-Frédéric) [11:45:26] (03CR) 10Jean-Frédéric: "> would you mind removing the trailing whitespace while you are updating the file?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377725 (owner: 10Jean-Frédéric) [11:45:37] (03PS2) 10Jean-Frédéric: Update Commons database config in api code [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377725 [11:45:52] (03Merged) 10jenkins-bot: Add Irak in Arabic (iq_ar) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377508 (https://phabricator.wikimedia.org/T174340) (owner: 10Jean-Frédéric) [11:46:39] (03CR) 10jenkins-bot: Add Irak in Arabic (iq_ar) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377508 (https://phabricator.wikimedia.org/T174340) (owner: 10Jean-Frédéric) [11:52:34] (03CR) 10Lokal Profil: [C: 04-1] "in addition to inline comments: some trailing whitespace still left in the main PHP file." (032 comments) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377724 (owner: 10Jean-Frédéric) [11:55:30] (03CR) 10Lokal Profil: [C: 032] Update Commons database config in api code [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377725 (owner: 10Jean-Frédéric) [12:01:05] RECOVERY - Puppet errors on tools-worker-1023 is OK: OK: Less than 1.00% above the threshold [0.0] [12:07:16] (03CR) 10Jean-Frédéric: "> Why did you drop the timestamp from the echo?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377483 (https://phabricator.wikimedia.org/T174871) (owner: 10Jean-Frédéric) [12:13:27] (03PS1) 10Jean-Frédéric: Allow to choose the year in wlm-latest tool [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377743 [12:15:02] (03PS2) 10Jean-Frédéric: Fix wlm-latest tool [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377724 [12:15:03] (03PS3) 10Jean-Frédéric: Update Commons database config in api code [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377725 [12:15:06] (03PS2) 10Jean-Frédéric: Allow to choose the year in wlm-latest tool [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377743 [12:15:19] (03CR) 10Jean-Frédéric: "> in addition to inline comments: some trailing whitespace still left" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377724 (owner: 10Jean-Frédéric) [12:16:05] (03CR) 10Jean-Frédéric: "I added the per-year config into a separate changeset, 377743" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377724 (owner: 10Jean-Frédéric) [12:23:24] (03CR) 10Lokal Profil: "> > Why did you drop the timestamp from the echo?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377483 (https://phabricator.wikimedia.org/T174871) (owner: 10Jean-Frédéric) [12:25:20] 10Tools, 10Wiki-Loves-Monuments: Redirect toolserver.org/~erfgoed/stream/ - https://phabricator.wikimedia.org/T175671#3604125 (10JeanFred) See also T175822 [12:28:05] (03CR) 10Lokal Profil: "> I added the per-year config into a separate changeset, 377743" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377724 (owner: 10Jean-Frédéric) [12:34:41] (03CR) 10Lokal Profil: Allow to choose the year in wlm-latest tool (032 comments) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377743 (owner: 10Jean-Frédéric) [12:35:42] PROBLEM - Puppet errors on tools-exec-1408 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [13:05:43] RECOVERY - Puppet errors on tools-exec-1408 is OK: OK: Less than 1.00% above the threshold [0.0] [13:07:39] How do you see the list of ips that have accessed a beta labs wiki if you have access to logstash online? [13:07:49] Is it somewhere in apache? [13:14:27] I got it [13:24:58] https://tools-static.wmflabs.org is down [13:27:00] Steinsplitter: yup... seems so [13:27:29] 10Cloud-Services: Restart tools-static - https://phabricator.wikimedia.org/T175825#3604298 (10Steinsplitter) [13:27:58] 10Tools: Restart tools-static - https://phabricator.wikimedia.org/T175825#3604313 (10Steinsplitter) [13:28:56] Anoop-Rao? [13:30:46] PROBLEM - Puppet errors on tools-static-11 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [13:30:53] "2017/09/13 13:11:10 [warn] 16654#16654: invalid parameter "spdy": ngx_http_spdy_module was superseded by ngx_http_v2_module in /etc/nginx/sites-enabled/static-server:21" [13:31:28] !log tools static down due to apparent nginx package upgrade/config change [13:31:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [13:34:22] PROBLEM - Puppet errors on tools-static-10 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [13:39:34] PROBLEM - Puppet errors on tools-docker-registry-01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [13:53:44] Technical Advice IRC meeting starting at 3 pm UTC/5 pm CEST in channel #wikimedia-tech, hosts: @addshore & @Tobi_WMDE_SW - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [13:54:22] !log wikispeech Deploy latest from Git master: 346d7a2 (T175124) [13:54:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikispeech/SAL [13:54:26] T175124: Buffer animation is hidden too early - https://phabricator.wikimedia.org/T175124 [13:57:43] !log tools apt-get install nginx-common on tools-static-1[01] [13:57:47] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [13:58:50] PROBLEM - Puppet staleness on tools-mail is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [43200.0] [14:10:45] RECOVERY - Puppet errors on tools-static-11 is OK: OK: Less than 1.00% above the threshold [0.0] [14:19:21] 10Tool-Pageviews: Weird behaviour in Basque Wikipedia most viewed: not a logic value - https://phabricator.wikimedia.org/T175795#3604480 (10MusikAnimal) [14:20:32] Steinsplitter, thedj: https://tools-static.wmflabs.org/static/ is ok again. thanks for shoulting [14:20:40] *shouting [14:24:20] RECOVERY - Puppet errors on tools-static-10 is OK: OK: Less than 1.00% above the threshold [0.0] [14:28:38] 10Cloud-Services, 10Beta-Cluster-Infrastructure, 10Puppet, 10Release-Engineering-Team (Next), 10User-Joe: Re-think puppet management for deployment-prep - https://phabricator.wikimedia.org/T161675#3604507 (10chasemp) p:05Triage>03Normal [14:33:49] RECOVERY - Puppet staleness on tools-mail is OK: OK: Less than 1.00% above the threshold [3600.0] [14:50:55] Technical Advice IRC meeting starting in 10 minutes in channel #wikimedia-tech, hosts: @addshore & @Tobi_WMDE_SW - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [14:59:05] (03PS1) 10Rush: herron wmcs wide root [labs/private] - 10https://gerrit.wikimedia.org/r/377778 [15:01:11] (03CR) 10Rush: [V: 032 C: 032] herron wmcs wide root [labs/private] - 10https://gerrit.wikimedia.org/r/377778 (owner: 10Rush) [15:04:15] 10Tool-Pageviews: Weird behaviour in Basque Wikipedia most viewed: not a logic value - https://phabricator.wikimedia.org/T175795#3604636 (10MusikAnimal) 05Open>03Resolved a:03MusikAnimal @Theklan Indeed, these are [[ https://tools.wmflabs.org/topviews/faq/#false_positive | false positives ]]. When you see... [15:05:55] 10Tools: Restart tools-static - https://phabricator.wikimedia.org/T175825#3604642 (10TheDJ) 05Open>03Resolved a:03TheDJ [15:06:21] bd808: thx ! [15:11:18] 10Tool-Pageviews: Weird behaviour in Basque Wikipedia most viewed: not a logic value - https://phabricator.wikimedia.org/T175795#3604660 (10Theklan) Thanks! Yes, results are quite different there and in the app. If you see yesterday, "Hardware" and "software" are over "computer" (ordenagailua). I imagine that ye... [15:15:57] !log tools Running outbound message purge via clush to @tools-exec [15:16:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [15:18:43] 10Wikibugs, 10Differential, 10Phabricator, 10Gerrit-Migration, and 2 others: Create conduit method to query the feed and return records with relevant details populated instead of just a bunch of phids - https://phabricator.wikimedia.org/T123417#3604696 (10Aklapper) > there aren't any differential methods t... [15:30:15] (03PS2) 10Jean-Frédéric: Split categorization out of daily update job [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377483 (https://phabricator.wikimedia.org/T174871) [15:30:23] (03CR) 10Jean-Frédéric: "Done :)" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377483 (https://phabricator.wikimedia.org/T174871) (owner: 10Jean-Frédéric) [15:34:16] !log tools Running inbound message purge via clush to @tools-exec [15:34:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [15:34:28] (03CR) 10Jean-Frédéric: "> I realised the files were created with space indentation but your" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377724 (owner: 10Jean-Frédéric) [15:37:35] PROBLEM - Puppet errors on tools-prometheus-02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [15:41:07] (03PS3) 10Jean-Frédéric: Allow to choose the year in wlm-latest tool [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377743 [15:41:52] (03CR) 10Jean-Frédéric: Allow to choose the year in wlm-latest tool (032 comments) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377743 (owner: 10Jean-Frédéric) [15:44:32] RECOVERY - Puppet errors on tools-docker-registry-01 is OK: OK: Less than 1.00% above the threshold [0.0] [15:52:34] RECOVERY - Puppet errors on tools-prometheus-02 is OK: OK: Less than 1.00% above the threshold [0.0] [15:55:57] PROBLEM - Puppet errors on tools-prometheus-01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [16:10:58] RECOVERY - Puppet errors on tools-prometheus-01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:19:04] (03PS1) 10Volans: keyholder: add Cumin openstack master key [labs/private] - 10https://gerrit.wikimedia.org/r/377796 (https://phabricator.wikimedia.org/T175712) [16:26:00] (03CR) 10Volans: [V: 032 C: 032] keyholder: add Cumin openstack master key [labs/private] - 10https://gerrit.wikimedia.org/r/377796 (https://phabricator.wikimedia.org/T175712) (owner: 10Volans) [16:53:16] !log tools.heritage Deploy latest from Git master: c5b8ffb, 837707f, 5799d26 (T174261), d330733 (T174340) [16:53:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.heritage/SAL [16:53:21] T174261: Add Egypt in Arabic to monuments database - https://phabricator.wikimedia.org/T174261 [16:53:22] T174340: Add Iraq in Arabic to monuments database - https://phabricator.wikimedia.org/T174340 [17:04:16] PROBLEM - Puppet errors on tools-bastion-05 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [17:20:46] hey bd808 is there a *tool* that lets people create books from articles (different than ocg/collections) ? [17:21:09] dr0ptp4kt: yup. [17:21:54] bd808: link? [17:22:22] 10Wikibugs, 10Differential, 10Phabricator, 10Gerrit-Migration, and 2 others: Create conduit method to query the feed and return records with relevant details populated instead of just a bunch of phids - https://phabricator.wikimedia.org/T123417#3605099 (10ksmith) @JAufrecht : Perhaps eventually phlogiston... [17:22:33] dr0ptp4kt: https://tools.wmflabs.org/wsexport/tool/book.php [17:23:27] dr0ptp4kt: there is a "Download/print" section in the en.wikisource.org sidebar that links to it. Possibly other places [17:23:44] I think it makes more epubs than anything else [17:24:07] bd808: thx. do you know if it can also do other projects, namely wikipedia? [17:24:18] RECOVERY - Puppet errors on tools-bastion-05 is OK: OK: Less than 1.00% above the threshold [0.0] [17:24:21] i can try to read the fine source [17:24:23] I think its wikisource specific [17:24:56] the code there is actually pretty nice if you ignore the horrible abuse of jsub for a "processing queue" [17:25:26] bd808: thx [17:40:58] 10Cloud-VPS (Project-requests), 10User-Zppix: Request creation of Zppix-Wiki-AI VPS project - https://phabricator.wikimedia.org/T175846#3605167 (10Zppix) [17:47:25] 10Wikibugs, 10Differential, 10Phabricator, 10Gerrit-Migration, and 2 others: Create conduit method to query the feed and return records with relevant details populated instead of just a bunch of phids - https://phabricator.wikimedia.org/T123417#3605210 (10JAufrecht) In the long run, yes, either pulling dat... [17:51:31] 10Wikibugs, 10Differential, 10Phabricator, 10Gerrit-Migration, and 2 others: Create conduit method to query the feed and return records with relevant details populated instead of just a bunch of phids - https://phabricator.wikimedia.org/T123417#1929298 (10greg) "Conduit" is just the Phab app name for the c... [17:55:16] bd808: do you have more information regarding the mail issue? [17:55:56] I'm afraid it could have been related to an issue we had with our server [17:57:17] * Platonides sees the description that it was not [18:28:04] 10Cloud-VPS (Project-requests), 10User-Zppix: Request creation of Zppix-Wiki-AI VPS project - https://phabricator.wikimedia.org/T175846#3605350 (10chasemp) p:05Triage>03Normal We will discuss in our weekly meeting next tuesday :) [18:28:25] 10Tools, 10I18n: Rename Tool labs to Toolforge in Persönliche Bekanntschaften tool - https://phabricator.wikimedia.org/T175774#3605352 (10chasemp) p:05Triage>03Normal [18:28:42] 10Toolforge, 10Outreachy (Round-15): Improvements for the Toolforge 'webservice' command - https://phabricator.wikimedia.org/T175768#3605355 (10chasemp) p:05Triage>03Normal [18:28:53] 10Tools: Need easier tool for working on redundancy than "Inhalte übernommen" Template Tool (german WP) - https://phabricator.wikimedia.org/T175698#3605357 (10chasemp) p:05Triage>03Normal [18:33:39] PROBLEM - Puppet errors on tools-exec-1406 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [18:36:43] (03PS1) 10Merlijn van Deen: Make sure get_master_branches returns a list [labs/tools/forrestbot] - 10https://gerrit.wikimedia.org/r/377816 (https://phabricator.wikimedia.org/T175626) [18:48:23] bd808: where is the task for the email issue ? [18:48:45] framawiki: it is not public at this time [18:50:04] oh, ok. [18:51:25] the summary I can give you is that a Toolforge user abused the shared mail services. [18:52:54] on a happier note, you can see the replag status for all of the Wiki Replica backend servers now -- https://tools.wmflabs.org/replag/ [18:55:11] !log tools.wikibugs restarting wb2-grrrrit to pull in I1dce797340788ce12fda27102abdbb7f97849107 / T174772 [18:55:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL [18:55:15] T174772: Ignore libraryupgrader in wikibugs - https://phabricator.wikimedia.org/T174772 [18:56:15] hm. [19:03:34] (03PS1) 10Merlijn van Deen: Bugfix: relevant user may be uploader or author [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/377821 [19:03:47] hurray. [19:04:06] (03CR) 10Merlijn van Deen: [C: 032] "Already deployed" [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/377821 (owner: 10Merlijn van Deen) [19:04:21] 10VPS-project-libraryupgrader, 10Wikibugs, 10Patch-For-Review: Ignore libraryupgrader in wikibugs - https://phabricator.wikimedia.org/T174772#3605580 (10valhallasw) 05Open>03Resolved a:03valhallasw [19:04:43] (03CR) 10Merlijn van Deen: [C: 032] Make sure get_master_branches returns a list [labs/tools/forrestbot] - 10https://gerrit.wikimedia.org/r/377816 (https://phabricator.wikimedia.org/T175626) (owner: 10Merlijn van Deen) [19:06:15] (03CR) 10jerkins-bot: [V: 04-1] Bugfix: relevant user may be uploader or author [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/377821 (owner: 10Merlijn van Deen) [19:06:18] (03CR) 10jerkins-bot: [V: 04-1] Bugfix: relevant user may be uploader or author [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/377821 (owner: 10Merlijn van Deen) [19:06:28] love you too, jenkins [19:06:39] not sure why you need to tell me twice, though :| [19:08:39] RECOVERY - Puppet errors on tools-exec-1406 is OK: OK: Less than 1.00% above the threshold [0.0] [19:34:45] bd808: I think people could miss the replag-by-databasename [19:35:01] (me included) [19:35:29] not everyone knows what shard a database is on [19:35:44] zhuyifei1999_: trying to figure out a nicer way to show that. [19:35:47] but noted [19:35:51] k [19:36:48] also when is the email issue going to be public? /me is curious about what happened :P [20:07:43] hi i am wondering could wikibugs be restarted please? -operations has no gerrit but, ie wikibugs. [20:07:55] need restarting wb2-grrrrit [20:07:59] https://www.mediawiki.org/wiki/Wikibugs [20:08:11] legoktm or cloud team ^^ :) [20:11:14] 10Wikibugs: Restart wikibugs - https://phabricator.wikimedia.org/T175861#3605757 (10Paladox) [20:12:50] 10Wikibugs: Restart wikibugs - https://phabricator.wikimedia.org/T175861#3605773 (10Paladox) p:05Triage>03High [20:13:05] 10Wikibugs: Restart wikibugs - https://phabricator.wikimedia.org/T175861#3605757 (10Dzahn) https://tools.wmflabs.org/?tool=wikibugs [20:41:11] foo [20:41:32] Bryan loves unicorns [20:57:16] lol [20:58:33] madhuvishy: about? [20:58:37] 10Wikibugs: Restart wikibugs - https://phabricator.wikimedia.org/T175861#3605958 (10valhallasw) 05Open>03Resolved a:03valhallasw Somehow `wb2-grrrrit` has crashed (OOM?). Restarted with `fab start_jobs`; seems OK again [20:58:44] I have a change I staged on shinken and I can't tell why IRC isn't notified [20:58:53] need some shinken-wm wisdom [20:58:53] 10Wikibugs: Restart wikibugs - https://phabricator.wikimedia.org/T175861#3605961 (10Paladox) thanks :). [20:59:11] chasemp: yes [20:59:29] madhuvishy: I disabled puppet to test something on shinken-01 [20:59:36] take al ook at /etc/shinken/customconfig/basic-instance-checks.cfg [20:59:41] service_description exim queue length [20:59:43] that emails [21:00:07] and alerts afaict a group that has the correct folks includeing irc-labs [21:00:08] contactgroups.cfg: members guest,chasemp,madhuvishy,andrewbogott,bd808,scfc_de,valhallasw,irc-labs [21:00:12] but nothing ever gets to irc [21:00:27] aah [21:02:24] looking [21:08:52] is it still trying to go to -labs? [21:10:28] I don't think so [21:10:32] madhuvishy: have to hop on a call, thanks for looking [21:41:30] madhuvishy: any luck? [21:42:25] chasemp: no I had to step away too for some package delivery stuff, I'm looking now [21:51:35] chasemp: were you running something to test? [21:52:04] (on a quick call) but I was setting customconfig/basic-instance-checks.cfg [21:52:09] 10Wikibugs, 10Differential, 10Phabricator, 10Gerrit-Migration, and 2 others: Create conduit method to query the feed and return records with relevant details populated instead of just a bunch of phids - https://phabricator.wikimedia.org/T123417#3606090 (10mmodell) a:03mmodell [21:52:11] thresholds for teh queue check to 1 [21:52:16] to force it to fail to alert [21:52:19] 10Wikibugs, 10Goal: Get rid of screen scraping in Wikibugs - https://phabricator.wikimedia.org/T1175#3606109 (10mmodell) [21:52:22] 10Wikibugs, 10Differential, 10Phabricator, 10Gerrit-Migration, and 2 others: Create conduit method to query the feed and return records with relevant details populated instead of just a bunch of phids - https://phabricator.wikimedia.org/T123417#3606107 (10mmodell) 05stalled>03Open [21:52:55] 10Wikibugs, 10Differential, 10Gerrit-Migration: Broadcast Differential activity to IRC - https://phabricator.wikimedia.org/T116330#3606110 (10mmodell) 05stalled>03Open [21:53:04] 10Wikibugs, 10Differential, 10Gerrit-Migration, 10Release-Engineering-Team (Kanban): Broadcast Differential activity to IRC - https://phabricator.wikimedia.org/T116330#3606112 (10mmodell) a:03mmodell [22:14:10] 10Cloud-VPS (Project-requests), 10User-Zppix: Request creation of Zppix-Wiki-AI VPS project - https://phabricator.wikimedia.org/T175846#3605167 (10Zppix) Blocking: T175726 [22:16:08] 10Wikibugs, 10Differential, 10Phabricator, 10Gerrit-Migration, and 2 others: Create conduit method to query the feed and return records with relevant details populated instead of just a bunch of phids - https://phabricator.wikimedia.org/T123417#3606199 (10JAufrecht) Added {T175872} for Phlogiston to switch... [22:49:05] zhuyifei1999_: per-wiki service names are back on https://tools.wmflabs.org/replag/ [22:53:08] nice [23:26:50] (03PS1) 10Lokal Profil: Include tools directory in php linting [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377921 [23:28:03] PROBLEM - Puppet errors on tools-exec-1439 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [23:34:43] quiddity: I saw your mail. I got time after around 10PM CDT; is explaining it around that time okay for you? [23:35:41] zhuyifei1999_, no rush at all. (But yes I can be around at that time). Email was so you can ignore until whenever. Also feel free to tell me to ask someone else. :) [23:37:36] well I think bd808 and yuvi knows everything about it anyways :) [23:38:04] * bd808 denies all knowledge [23:38:22] (Context for the pinged, I was asking him where to copyedit the UI in our cdnjs) [23:38:34] eh... yeah [23:39:04] its a hack that may or may not be in version control anywhere... [23:40:09] apparently https://gerrit.wikimedia.org/r/labs/tools/cdnjs-index [23:40:11] I just want to fix the wording in: "Clicking 'Use' will take show you list of files distributed with the library" but was hoping it was simple, and didn't require a phab task or discussion [23:40:35] I get "not found" at that URL [23:40:42] https://gerrit.wikimedia.org/r/#/admin/projects/labs/tools/cdnjs-index [23:40:52] I think that url only works in git [23:41:01] its the origin in the clone on toolforge [23:41:35] quiddity: so yeah give a patch to that repo and add me as reviewer [23:41:49] woo, that works, ty! [23:42:00] cc me as well please? [23:44:30] (03Draft2) 10Quiddity: Fix the wording in the cdnjs UI. [labs/tools/cdnjs-index] - 10https://gerrit.wikimedia.org/r/377923 [23:47:42] ^ lgtm (at least for viewing the patch on mobile) [23:50:46] PROBLEM - Puppet errors on tools-exec-1437 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [23:51:24] quiddity: if I merge this I almost feel obligated to make you figure out the content hash display issue too. :) [23:52:00] (03CR) 10BryanDavis: [V: 032 C: 032] Fix the wording in the cdnjs UI. [labs/tools/cdnjs-index] - 10https://gerrit.wikimedia.org/r/377923 (owner: 10Quiddity) [23:58:53] (03PS1) 10Lokal Profil: Ensure all added categories are pywikibot.Category objects [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377927 (https://phabricator.wikimedia.org/T175839)