[00:37:10] PROBLEM - Puppet errors on tools-exec-1436 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [01:42:08] RECOVERY - Puppet errors on tools-exec-1436 is OK: OK: Less than 1.00% above the threshold [0.0] [02:33:07] PROBLEM - Puppet errors on tools-exec-1436 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [03:13:08] RECOVERY - Puppet errors on tools-exec-1436 is OK: OK: Less than 1.00% above the threshold [0.0] [03:41:54] (03PS1) 10Legoktm: Remove unused variable [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375109 [03:41:56] (03PS1) 10Legoktm: Move CONCURRENT into docker.py [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375110 [03:41:58] (03PS1) 10Legoktm: Support skins [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375111 [03:42:25] (03CR) 10Legoktm: [C: 032] Remove unused variable [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375109 (owner: 10Legoktm) [03:42:29] (03CR) 10Legoktm: [C: 032] Move CONCURRENT into docker.py [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375110 (owner: 10Legoktm) [03:42:32] (03CR) 10Legoktm: [C: 032] Support skins [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375111 (owner: 10Legoktm) [03:42:44] (03Merged) 10jenkins-bot: Remove unused variable [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375109 (owner: 10Legoktm) [03:42:46] (03Merged) 10jenkins-bot: Move CONCURRENT into docker.py [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375110 (owner: 10Legoktm) [03:42:49] (03Merged) 10jenkins-bot: Support skins [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375111 (owner: 10Legoktm) [03:51:12] PROBLEM - Puppet errors on tools-exec-1405 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [04:16:12] RECOVERY - Puppet errors on tools-exec-1405 is OK: OK: Less than 1.00% above the threshold [0.0] [04:29:14] (03PS1) 10Legoktm: Make run.py test mode allow any repo [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375115 [04:29:16] (03PS1) 10Legoktm: Add script to upgrade libraries [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375116 [04:31:46] (03CR) 10Legoktm: [C: 032] Make run.py test mode allow any repo [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375115 (owner: 10Legoktm) [04:31:55] (03CR) 10Legoktm: [C: 032] "I sure hope this works." [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375116 (owner: 10Legoktm) [04:32:03] (03Merged) 10jenkins-bot: Make run.py test mode allow any repo [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375115 (owner: 10Legoktm) [04:32:13] (03Merged) 10jenkins-bot: Add script to upgrade libraries [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375116 (owner: 10Legoktm) [04:38:14] (03PS1) 10Legoktm: upgrade: Temporarily lower concurrency for safety [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375117 [04:38:25] (03CR) 10Legoktm: [C: 032] upgrade: Temporarily lower concurrency for safety [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375117 (owner: 10Legoktm) [04:38:48] (03Merged) 10jenkins-bot: upgrade: Temporarily lower concurrency for safety [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375117 (owner: 10Legoktm) [04:41:35] (03PS1) 10Legoktm: upgrade: Just need the repo name from get_extension_list [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375118 [04:41:42] (03CR) 10Legoktm: [C: 032] upgrade: Just need the repo name from get_extension_list [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375118 (owner: 10Legoktm) [04:41:59] (03Merged) 10jenkins-bot: upgrade: Just need the repo name from get_extension_list [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375118 (owner: 10Legoktm) [04:50:03] PROBLEM - Puppet errors on tools-bastion-03 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [05:23:03] 10Data-Services, 10DBA: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#1677380 (10Marostegui) They are not being replicated because they are marked as private table: https://github.com/wikimedia/puppet/blob/production/manifests/realm.pp#L201 Why? That I don't kn... [05:27:38] tools-bastion-03 is really slow right now. Connecting took a while and so did `become`ing my tool. tools-bastion-02 seems fine though. [05:38:56] PROBLEM - Puppet errors on tools-worker-1020 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [05:53:57] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3572337 (10Legoktm) [06:15:08] (03PS1) 10Legoktm: Add support for upgrading just canaries [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375342 (https://phabricator.wikimedia.org/T174646) [06:17:36] (03CR) 10Legoktm: [C: 032] Add support for upgrading just canaries [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375342 (https://phabricator.wikimedia.org/T174646) (owner: 10Legoktm) [06:29:38] PROBLEM - Puppet errors on tools-exec-1406 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:09:38] RECOVERY - Puppet errors on tools-exec-1406 is OK: OK: Less than 1.00% above the threshold [0.0] [07:13:59] RECOVERY - Puppet errors on tools-worker-1020 is OK: OK: Less than 1.00% above the threshold [0.0] [07:27:17] extremely slow: matanya@tools-bastion-03:~$ uptime [07:27:17] 07:26:25 up 29 days, 6:36, 31 users, load average: 12.69, 13.07, 12.20 [07:33:48] 10VPS-project-libraryupgrader: libraryupgrader should support npm / package.json - https://phabricator.wikimedia.org/T174767#3572475 (10Legoktm) [07:36:03] matanya: I hear bastion-02 is fine [07:36:15] indeed [07:54:06] 10Toolforge: Make it less cumbersome to bootstrap and update python webservices - https://phabricator.wikimedia.org/T174769#3572512 (10Legoktm) [08:00:40] PROBLEM - Puppet errors on tools-exec-1440 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [08:03:07] (03Merged) 10jenkins-bot: Add support for upgrading just canaries [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375342 (https://phabricator.wikimedia.org/T174646) (owner: 10Legoktm) [08:05:10] 10VPS-project-libraryupgrader, 10Patch-For-Review: Create list of canary repositories for libraryupgrader - https://phabricator.wikimedia.org/T174646#3572549 (10Legoktm) 05Open>03Resolved a:03Legoktm [08:10:11] (03PS1) 10Jean-Frédéric: Use CommonsCat item in API output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375348 [08:32:02] (03PS1) 10Legoktm: Add a bunch of type hints [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375350 [08:32:12] (03CR) 10Legoktm: [C: 032] Add a bunch of type hints [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375350 (owner: 10Legoktm) [08:32:26] 10Data-Services, 10DBA, 10Patch-For-Review: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#3572591 (10jcrespo) Security seems to agree on it being possibly public: https://phabricator.wikimedia.org/T103011#3536648 I will ask him to +1 that patch. [08:33:03] 10VPS-project-libraryupgrader, 10Wikibugs: Ignore libraryupgrader in wikibugs - https://phabricator.wikimedia.org/T174772#3572592 (10Legoktm) [08:38:53] 10Data-Services, 10DBA, 10Patch-For-Review: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#3572605 (10jcrespo) Once that is merged, it may take some time to deploy, as we have to create the tables copying it from production- apparently it is only used on wikis... [08:40:40] RECOVERY - Puppet errors on tools-exec-1440 is OK: OK: Less than 1.00% above the threshold [0.0] [08:42:36] Toolforge so slow [08:57:20] 10Cloud-VPS (Quota-requests), 10Discovery, 10Wikidata, 10Wikidata-Query-Service: WDQS testing setup platform sizing - https://phabricator.wikimedia.org/T169133#3572646 (10Gehel) @Smalyshev I'm wondering what the performance bottleneck is. I suppose it is IO, but with the current state of wdqs-beta, it is h... [08:57:28] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Update wikiloves tool with WLM 2017 data - https://phabricator.wikimedia.org/T174775#3572647 (10JeanFred) [09:03:21] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Update wikiloves tool with WLM 2017 data - https://phabricator.wikimedia.org/T174775#3572663 (10JeanFred) 05Open>03Resolved Documenting the steps I followed: * Get start/date data from https://commons.wikimedia.org/wiki/Template:WLM-is-running * Transform th... [09:35:32] (03Merged) 10jenkins-bot: Add a bunch of type hints [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/375350 (owner: 10Legoktm) [09:41:58] 10Data-Services, 10DBA, 10Patch-For-Review: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#3572734 (10Marostegui) Yeah - I thought about creating it on enwikisource to unblock this ticket first and then slowly create it on the big wikis and so forth [09:46:44] 10Data-Services, 10DBA, 10Patch-For-Review: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#3572744 (10jcrespo) > I thought about creating it on enwikisource to unblock this ticket first Sadly, it is all or none, because replication will break otherwise (break... [10:25:04] RECOVERY - Puppet errors on tools-bastion-03 is OK: OK: Less than 1.00% above the threshold [0.0] [10:28:36] 10Tool-wikiloves: In Wikiloves tool, for some countries the 'show images' doesn't work - https://phabricator.wikimedia.org/T174778#3572793 (10Effeietsanders) [10:55:20] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Increase update frequency of wikiloves tool - https://phabricator.wikimedia.org/T174780#3572831 (10JeanFred) [11:09:54] 10Tool-wikiloves: In Wikiloves tool, for some countries the 'show images' doesn't work - https://phabricator.wikimedia.org/T174778#3572849 (10JeanFred) 05Open>03Resolved Fixed. [11:48:01] 10Data-Services, 10DBA, 10Patch-For-Review: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#3572881 (10Reedy) Wow... It's on enwiki https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/sql/ProofreadIndex.sql I'd propose dropping it from... [11:50:09] 10Data-Services, 10DBA, 10Patch-For-Review: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#3572883 (10Marostegui) >>! In T113842#3572881, @Reedy wrote: > Wow... It's on enwiki > > https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/sql... [11:51:03] PROBLEM - Puppet errors on tools-bastion-03 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [11:51:07] 10Data-Services, 10DBA, 10Patch-For-Review: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#3572887 (10jcrespo) > I'd propose dropping it from anywhere that isn't in this list... I would support such an action, if it is an optional extension, it should not crea... [11:54:45] 10Data-Services, 10DBA, 10Patch-For-Review: `pr_index`to be replicated to Labs public databases - https://phabricator.wikimedia.org/T113842#3572890 (10Reedy) I'm guessing someone messed up onetime running it over all wikis, rather than a subset. Or at some point, maybe we didn't have a way of making it easie... [12:26:03] RECOVERY - Puppet errors on tools-bastion-03 is OK: OK: Less than 1.00% above the threshold [0.0] [12:29:33] (03CR) 10Jean-Frédéric: [C: 032] Merge Wikidata branch into master [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/374899 (https://phabricator.wikimedia.org/T174146) (owner: 10Lokal Profil) [12:30:39] (03Merged) 10jenkins-bot: Merge Wikidata branch into master [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/374899 (https://phabricator.wikimedia.org/T174146) (owner: 10Lokal Profil) [12:51:20] (03CR) 10jenkins-bot: Merge Wikidata branch into master [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/374899 (https://phabricator.wikimedia.org/T174146) (owner: 10Lokal Profil) [13:10:49] shinken-wm: matanya any better now? [13:20:43] 10Cloud-VPS (Quota-requests), 10Discovery, 10Wikidata, 10Wikidata-Query-Service: WDQS testing setup platform sizing - https://phabricator.wikimedia.org/T169133#3573058 (10chasemp) @Gehel We could bump up the quota temporarily to allow rebuild of those instances with a flavor that has a larger disk. We h... [13:27:03] 10Cloud-VPS (Quota-requests), 10Discovery, 10Wikidata, 10Wikidata-Query-Service: WDQS testing setup platform sizing - https://phabricator.wikimedia.org/T169133#3573088 (10Gehel) That would be great! If you grant us access to that flavor with 300GB disk, we don't event need a quota extension. We have one in... [13:31:36] (03PS2) 10Jean-Frédéric: Use CommonsCat item in API output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375348 [13:35:27] (03PS3) 10Jean-Frédéric: Skip Wikidata harvesting during daily harvesting [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375065 [13:52:06] PROBLEM - Puppet errors on tools-bastion-03 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [13:58:21] PROBLEM - Puppet errors on tools-exec-gift-trusty-01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [14:27:07] RECOVERY - Puppet errors on tools-bastion-03 is OK: OK: Less than 1.00% above the threshold [0.0] [14:48:26] !log tool.admin Deployed 4cab928 (Link to Striker for tool creation) T149458 [14:48:27] bd808: Unknown project "tool.admin" [14:48:27] T149458: Manage shared tool accounts via Striker - https://phabricator.wikimedia.org/T149458 [14:48:34] !log tools.admin Deployed 4cab928 (Link to Striker for tool creation) T149458 [14:48:37] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.admin/SAL [14:49:04] 10cloud-services-team (FY2017-18), 10Goal: Program 10 Outcome 1: PaaS is easy to use - https://phabricator.wikimedia.org/T166403#3573291 (10bd808) [14:49:07] 10Cloud-Services, 10MediaWiki-extensions-OpenStackManager, 10Toolforge: The future of service groups and service users on Labs - https://phabricator.wikimedia.org/T162945#3573292 (10bd808) [14:49:09] 10Cloud-Services, 10Toolforge, 10Community-Tech-Tool-Labs, 10Epic: Tools web interface for tool authors (Brainstorming ticket) - https://phabricator.wikimedia.org/T128158#3573293 (10bd808) [14:49:12] 10Striker, 10cloud-services-team (Kanban), 10Epic, 10Patch-For-Review, 10User-bd808: Manage shared tool accounts via Striker - https://phabricator.wikimedia.org/T149458#3573290 (10bd808) 05Open>03Resolved [14:53:19] RECOVERY - Puppet errors on tools-exec-gift-trusty-01 is OK: OK: Less than 1.00% above the threshold [0.0] [15:21:12] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Update wikiloves tool with WLM 2017 data - https://phabricator.wikimedia.org/T174775#3572647 (10LilyOfTheWest) Even when this task is closed, I think it deserves love. :D [15:24:27] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Update wikiloves tool with WLM 2017 data - https://phabricator.wikimedia.org/T174775#3572647 (10Effeietsanders) Especially when closed - what would otherwise happen to the motivation to close them XD [16:56:59] 10Cloud-Services, 10Operations, 10ops-eqiad, 10Patch-For-Review: rack/setup/install labstore100[67].wikimedia.org - https://phabricator.wikimedia.org/T167984#3573619 (10RobH) I can see the issue, both labstore1006 and labstore1007 have it. When checking the boot order in bios, it lists the following: ``... [17:14:16] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573697 (10demon) Sadly, we cannot restrict (easily, would require some plugin/submit rule work) on the Gerrit side. If we're going to b... [17:25:07] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573713 (10Legoktm) >>! In T174760#3573697, @demon wrote: > Sadly, we cannot restrict (easily, would require some plugin/submit rule wor... [17:44:08] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573774 (10demon) >>! In T174760#3573713, @Legoktm wrote: >>>! In T174760#3573697, @demon wrote: >> Sadly, we cannot restrict (easily, w... [17:45:56] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573778 (10demon) But also....if we're using the API we're having to auth using the actual LDAP password for the user...which is even mo... [17:48:38] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573784 (10Legoktm) >>! In T174760#3573774, @demon wrote: > Not really sure. Basically just kind of thinking the whole "don't depend on... [17:52:47] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573792 (10Paladox) @demon we could do want upstream gerrit did for polygerrit. (without a plugin). it's done in the All-Project. [17:55:05] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573803 (10demon) Do what? You can't do path-based matching in Gerrit ACLs. Just branch (maybe topic?) based. [18:01:19] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573852 (10Paladox) You can do paths. Upstream did it. [18:04:36] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573858 (10Paladox) See https://gerrit-review.googlesource.com/#/c/gerrit/+/118072/3/rules.pl [18:06:19] 10VPS-project-libraryupgrader, 10Gerrit, 10Repository-Ownership-Requests: Grant libraryupgrader +2 rights for some library bumps - https://phabricator.wikimedia.org/T174760#3573863 (10demon) Ugh, Prolog. Craziest idea they ever had. No. [18:20:42] (03CR) 10Lokal Profil: [C: 032] "why did the not get submitted?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375065 (owner: 10Jean-Frédéric) [18:21:38] (03Merged) 10jenkins-bot: Skip Wikidata harvesting during daily harvesting [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375065 (owner: 10Jean-Frédéric) [18:22:22] (03CR) 10jenkins-bot: Skip Wikidata harvesting during daily harvesting [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375065 (owner: 10Jean-Frédéric) [18:26:38] (03CR) 10Jean-Frédéric: "> why did the not get submitted?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375065 (owner: 10Jean-Frédéric) [18:32:48] (03CR) 10Lokal Profil: "> > why did the not get submitted?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375065 (owner: 10Jean-Frédéric) [18:35:50] (03CR) 10Jean-Frédéric: "> > > why did the not get submitted?" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375065 (owner: 10Jean-Frédéric) [18:46:56] (03PS3) 10Lokal Profil: Make handling of wikidata urls more compatible with other handlers [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/374901 [18:50:05] (03CR) 10Lokal Profil: [C: 032] "Just a rebase. No change in code" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/374901 (owner: 10Lokal Profil) [18:51:11] (03Merged) 10jenkins-bot: Make handling of wikidata urls more compatible with other handlers [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/374901 (owner: 10Lokal Profil) [18:52:08] (03CR) 10Lokal Profil: [C: 032] Use CommonsCat item in API output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375348 (owner: 10Jean-Frédéric) [18:53:10] (03Merged) 10jenkins-bot: Use CommonsCat item in API output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375348 (owner: 10Jean-Frédéric) [18:55:03] (03CR) 10jenkins-bot: Make handling of wikidata urls more compatible with other handlers [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/374901 (owner: 10Lokal Profil) [18:56:18] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Increase update frequency of wikiloves tool - https://phabricator.wikimedia.org/T174780#3573980 (10JeanFred) I changed the crontab from `56 23 * * *` (once a day) to `0 * * * *` (once an hour). Might be a bit extreme but hey, YOLO! [18:56:24] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Increase update frequency of wikiloves tool - https://phabricator.wikimedia.org/T174780#3573981 (10JeanFred) 05Open>03Resolved [18:57:43] (03CR) 10jenkins-bot: Use CommonsCat item in API output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375348 (owner: 10Jean-Frédéric) [19:00:35] !log tools.heritage Deploy latest from Git master: 2be9e28 (T166528), d3aa65a, 61086ce, 637a1c0, 6bbcb0a (T174146), 566ab17, eae2643, 9b8e2f4 [19:00:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.heritage/SAL [19:00:41] T174146: Merge Wikidata branch into master - https://phabricator.wikimedia.org/T174146 [19:00:41] T166528: Rename -lang argument -langcode - https://phabricator.wikimedia.org/T166528 [19:09:23] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Increase update frequency of wikiloves tool - https://phabricator.wikimedia.org/T174780#3572831 (10Legoktm) You may want to set your crontab to not fire exactly at the hour - a lot of crons firing off every hour is going to slow everything down. Running it at th... [19:14:53] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Increase update frequency of wikiloves tool - https://phabricator.wikimedia.org/T174780#3574026 (10JeanFred) >>! In T174780#3574016, @Legoktm wrote: > You may want to set your crontab to not fire exactly at the hour - a lot of crons firing off every hour is goin... [19:43:41] 10Striker, 10cloud-services-team (FY2017-18), 10Epic, 10Patch-For-Review, 10User-bd808: Manage shared tool accounts via Striker - https://phabricator.wikimedia.org/T149458#3574076 (10bd808) [19:45:06] 10Cloud-Services, 10cloud-services-team (Kanban), 10Patch-For-Review: Investigate and implement alternative for showmount based check at instance boot time - https://phabricator.wikimedia.org/T171508#3574077 (10bd808) [19:45:26] 10Cloud-Services, 10cloud-services-team (Kanban): Revert patch that adds a temporary exception to the block-for-export check for the testlabs project - https://phabricator.wikimedia.org/T174590#3574078 (10bd808) [19:48:21] 10Toolforge, 10cloud-services-team (Kanban): No module named 'pymysql' -- python3-mysql installed on toolforge bastions but not on exec nodes - https://phabricator.wikimedia.org/T174439#3562040 (10bd808) The preferred solution for Python 2/3 packages is for the tool to use a virtual environment, but if this is... [19:48:36] 10Cloud-Services, 10Toolforge, 10Tracking: Packages to be added to toollabs puppet - https://phabricator.wikimedia.org/T55704#3574086 (10bd808) [19:48:38] 10Toolforge, 10cloud-services-team (Kanban): No module named 'pymysql' -- python3-mysql installed on toolforge bastions but not on exec nodes - https://phabricator.wikimedia.org/T174439#3574085 (10bd808) [20:07:33] 10Toolforge, 10cloud-services-team (Kanban): No module named 'pymysql' -- python3-mysql installed on toolforge bastions but not on exec nodes - https://phabricator.wikimedia.org/T174439#3574121 (10bd808) ``` $ clush -b -w @tools-bastion -- 'dpkg -l|grep python3-mysql.connector' --------------- tools-bastion-[0... [20:31:26] 10Tools: Tool "replacer" loads bootstrap from bootstrapcdn - https://phabricator.wikimedia.org/T172827#3574230 (10Benjavalero) 05Open>03Resolved [20:31:28] 10Tools, 10Toolforge-standards-committee, 10Privacy: Hunt for Toolforge tools that load resources from third party sites - https://phabricator.wikimedia.org/T172065#3574231 (10Benjavalero) [21:32:34] 10Toolforge, 10cloud-services-team (Kanban): No module named 'pymysql' -- python3-mysql installed on toolforge bastions but not on exec nodes - https://phabricator.wikimedia.org/T174439#3574341 (10Jar) ``` Traceback (most recent call last): File "core/pwb.py", line 262, in if not main(): File... [22:01:06] 10Toolforge, 10cloud-services-team (Kanban): No module named 'pymysql' -- python3-mysql installed on toolforge bastions but not on exec nodes - https://phabricator.wikimedia.org/T174439#3574430 (10bd808) Ok. I checked for the wrong Python mysql library the first time. Here are the results for the `python3-pyms... [23:18:45] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1416 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [23:22:28] 10Toolforge, 10cloud-services-team (Kanban): No module named 'pymysql' -- python3-mysql installed on toolforge bastions but not on exec nodes - https://phabricator.wikimedia.org/T174439#3574550 (10Jar) @bd808 I now using mysql.connector, Thank you.