[00:05:48] (03CR) 10BryanDavis: [C: 032] Break in-process cache of model choice fields [labs/striker] - 10https://gerrit.wikimedia.org/r/373708 (https://phabricator.wikimedia.org/T173963) (owner: 10BryanDavis) [00:07:32] (03Merged) 10jenkins-bot: Break in-process cache of model choice fields [labs/striker] - 10https://gerrit.wikimedia.org/r/373708 (https://phabricator.wikimedia.org/T173963) (owner: 10BryanDavis) [00:11:20] 10Tool-stewardbots, 10WorkType-NewFunctionality: Let stewardbot display temporary rightchanges too - https://phabricator.wikimedia.org/T164204#3551536 (10Melos) a:03Melos [00:13:03] (03CR) 10BryanDavis: [C: 032] Guard against LDAP modify with no changes for maintainers [labs/striker] - 10https://gerrit.wikimedia.org/r/373709 (owner: 10BryanDavis) [00:14:08] (03Merged) 10jenkins-bot: Guard against LDAP modify with no changes for maintainers [labs/striker] - 10https://gerrit.wikimedia.org/r/373709 (owner: 10BryanDavis) [00:14:26] 10Tools: Add code snippets with integrity hashes to CDNJS interface - https://phabricator.wikimedia.org/T173996#3547691 (10Legoktm) Is there a reason we have a full clone/mirror of cdnjs? Did we consider using a proxy like we do for the fonts? [00:37:14] PROBLEM - Puppet errors on tools-exec-1441 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [00:57:55] PROBLEM - Puppet errors on tools-exec-1410 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [01:07:55] 10Data-Services, 10DC-Ops, 10Operations, 10ops-codfw: Split up labstore external shelf storage available in codfw between labstore2001 and 2 - https://phabricator.wikimedia.org/T171623#3551607 (10ops-monitoring-bot) Completed auto-reimage of hosts: ``` ['labstore2002.codfw.wmnet'] ``` Of which those **FAI... [01:12:15] RECOVERY - Puppet errors on tools-exec-1441 is OK: OK: Less than 1.00% above the threshold [0.0] [01:20:22] PROBLEM - Puppet errors on tools-exec-1419 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [01:21:43] (03PS1) 10BryanDavis: Handle invalid ssh keys in LDAP [labs/striker] - 10https://gerrit.wikimedia.org/r/373720 (https://phabricator.wikimedia.org/T174112) [01:22:10] 10Striker, 10cloud-services-team (Kanban), 10Patch-For-Review, 10User-Zppix, 10User-bd808: Fatal error when SSH key data is invalid - https://phabricator.wikimedia.org/T174112#3551624 (10bd808) a:03bd808 [01:25:37] (03CR) 10BryanDavis: [C: 032] Handle invalid ssh keys in LDAP [labs/striker] - 10https://gerrit.wikimedia.org/r/373720 (https://phabricator.wikimedia.org/T174112) (owner: 10BryanDavis) [01:26:45] (03Merged) 10jenkins-bot: Handle invalid ssh keys in LDAP [labs/striker] - 10https://gerrit.wikimedia.org/r/373720 (https://phabricator.wikimedia.org/T174112) (owner: 10BryanDavis) [01:55:22] RECOVERY - Puppet errors on tools-exec-1419 is OK: OK: Less than 1.00% above the threshold [0.0] [02:29:55] 10Tools: Add code snippets with integrity hashes to CDNJS interface - https://phabricator.wikimedia.org/T173996#3551657 (10zhuyifei1999) Dunno why, but the relevant ticket is T96799 [02:31:07] 10Tools: Add code snippets with integrity hashes to CDNJS interface - https://phabricator.wikimedia.org/T173996#3551659 (10zhuyifei1999) I think (might be wrong) yuvi said something about serving from local disk being super-fast compared to /static serving from NFS [02:34:18] PROBLEM - Puppet errors on tools-redis-1001 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [02:52:58] RECOVERY - Puppet errors on tools-exec-1410 is OK: OK: Less than 1.00% above the threshold [0.0] [03:09:18] RECOVERY - Puppet errors on tools-redis-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [03:22:05] 10Tools: Add code snippets with integrity hashes to CDNJS interface - https://phabricator.wikimedia.org/T173996#3551670 (10bd808) >>! In T173996#3551544, @Legoktm wrote: > Is there a reason we have a full clone/mirror of cdnjs? Did we consider using a proxy like we do for the fonts? Hosting it ourselves seems n... [03:38:05] PROBLEM - Puppet errors on tools-exec-1413 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [03:43:57] PROBLEM - Puppet errors on tools-exec-1410 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [04:03:02] RECOVERY - Puppet errors on tools-exec-1413 is OK: OK: Less than 1.00% above the threshold [0.0] [04:53:56] RECOVERY - Puppet errors on tools-exec-1410 is OK: OK: Less than 1.00% above the threshold [0.0] [04:55:53] 10Data-Services, 10Structured-Data-Commons, 10Wikidata: Provide backwards compatibility views for toolforge replica [MCR] - https://phabricator.wikimedia.org/T174047#3551720 (10bd808) [05:25:54] 10Data-Services, 10DC-Ops, 10Operations, 10ops-codfw: Split up labstore external shelf storage available in codfw between labstore2001 and 2 - https://phabricator.wikimedia.org/T171623#3551757 (10madhuvishy) @Papaul, thanks for splitting up the shelves! I've reimaged the servers, and that part looks right... [05:35:51] 10Cloud-Services, 10cloud-services-team (Kanban), 10Operations: labstore systemd state Icinga alarms - https://phabricator.wikimedia.org/T151322#3551776 (10madhuvishy) 05Open>03Resolved 2001 is done too. ``` root@labstore2001:~# systemctl --failed --no-legend root@labstore2001:~# ``` Closing this now. [07:38:10] https://github.com/legoktm/wmflabs-lib/blob/master/wmflabs/__init__.py#L99 [07:38:24] app.before_request(wmflabs.redirect_to_https) [07:38:29] super easy :) [07:38:37] I suppose I probably need to rename that module now... [09:12:00] 10Tools: s51203__baglama2_p generating lag on tools-db - https://phabricator.wikimedia.org/T174140#3552053 (10jcrespo) [09:13:39] 10Tools: s51203__baglama2_p generating lag on tools-db - https://phabricator.wikimedia.org/T174140#3552069 (10jcrespo) [09:28:26] 10Tools: s51203__baglama2_p generating lag on tools-db - https://phabricator.wikimedia.org/T174140#3552075 (10Magnus) I just started this manually, I do this once a month for the GLAM community. It should subside quickly now. [09:40:33] 10Tools: s51203__baglama2_p generating lag on tools-db - https://phabricator.wikimedia.org/T174140#3552101 (10jcrespo) 05Open>03Resolved a:03jcrespo [09:45:03] PROBLEM - Puppet errors on tools-exec-1432 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [09:59:27] !log wikispeech Deploy latest from Git master: cdedb07, 827fbb2 (T172094), 827fbb2, 2791dad, a7e5eaf, 03ca986, d84db16 (T173516), 7cfd1ca (T164251, T146454) [09:59:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikispeech/SAL [09:59:35] T146454: WikispeechNamespaces can't contain namspaces that aren't defined as constants - https://phabricator.wikimedia.org/T146454 [09:59:35] T172094: npm 3 requires peer dependencies to be explicitly defined in package.json - https://phabricator.wikimedia.org/T172094 [09:59:35] T164251: Use API action to preprocess text - https://phabricator.wikimedia.org/T164251 [09:59:35] T173516: Use json file extension for the stylelint config file - https://phabricator.wikimedia.org/T173516 [10:00:05] 10Striker: toolsadmin not recognizing that user is associated with tool accounts - https://phabricator.wikimedia.org/T165009#3552178 (10Aklapper) [10:19:51] (03PS1) 10Lokal Profil: Correct api parameter used in serach page [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/373857 (https://phabricator.wikimedia.org/T174125) [10:20:03] RECOVERY - Puppet errors on tools-exec-1432 is OK: OK: Less than 1.00% above the threshold [0.0] [10:22:10] (03PS2) 10Lokal Profil: Correct api parameter used on search page [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/373857 (https://phabricator.wikimedia.org/T174125) [10:27:32] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Ensure wikiloves tool detects first uploader, not latest - https://phabricator.wikimedia.org/T173003#3552291 (10Lokal_Profil) >>! In T173003#3516319, @JeanFred wrote: > I’m not actually sure it worked :-( There are still eg SteinsplitterBot in http://tools.wmfla... [10:29:10] PROBLEM - Puppet errors on tools-exec-1407 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [10:52:41] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Ensure wikiloves tool detects first uploader, not latest - https://phabricator.wikimedia.org/T173003#3552401 (10Effeietsanders) If we adapt the Polish competition (or WikiDaHeim) to be included in WLM as they should anyway, we could check those as test sets :) [10:55:28] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Ensure wikiloves tool detects first uploader, not latest - https://phabricator.wikimedia.org/T173003#3552402 (10JeanFred) >>! In T173003#3552291, @Lokal_Profil wrote: >>>! In T173003#3516319, @JeanFred wrote: >> I’m not actually sure it worked :-( There are stil... [10:56:33] 10Tool-wikiloves: Bug on a page of http://tools.wmflabs.org/wikiloves - https://phabricator.wikimedia.org/T173583#3552404 (10JeanFred) 05Open>03Invalid Closing this as Invalid. My bet is on general issues with the harvesting. That being said, the fonrt-end should know better than to throw an error because so... [11:00:04] (03CR) 10Jean-Frédéric: [C: 032] Correct api parameter used on search page [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/373857 (https://phabricator.wikimedia.org/T174125) (owner: 10Lokal Profil) [11:01:06] (03Merged) 10jenkins-bot: Correct api parameter used on search page [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/373857 (https://phabricator.wikimedia.org/T174125) (owner: 10Lokal Profil) [11:01:49] (03CR) 10jenkins-bot: Correct api parameter used on search page [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/373857 (https://phabricator.wikimedia.org/T174125) (owner: 10Lokal Profil) [11:04:08] RECOVERY - Puppet errors on tools-exec-1407 is OK: OK: Less than 1.00% above the threshold [0.0] [11:07:09] 10Tool-wikiloves, 10Wiki-Loves-Monuments (2017): Ensure wikiloves tool detects first uploader, not latest - https://phabricator.wikimedia.org/T173003#3552412 (10JeanFred) 05Open>03Resolved [11:29:34] !log tools.heritage Deploy latest from Git master: 598e5f9 (T174125) [11:29:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.heritage/SAL [11:29:38] T174125: Extract the WLM-IL list of monuments with no photo - https://phabricator.wikimedia.org/T174125 [12:21:01] PROBLEM - Puppet errors on tools-exec-1433 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [12:43:50] (03PS7) 10Addshore: Add phabricator-upstream to #wikimedia-dev and #wikimedia-devtools [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/333553 (owner: 10Paladox) [12:43:56] (03CR) 10Addshore: [C: 032] Add phabricator-upstream to #wikimedia-dev and #wikimedia-devtools [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/333553 (owner: 10Paladox) [12:44:18] (03Merged) 10jenkins-bot: Add phabricator-upstream to #wikimedia-dev and #wikimedia-devtools [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/333553 (owner: 10Paladox) [12:44:28] (03CR) 10jenkins-bot: Add phabricator-upstream to #wikimedia-dev and #wikimedia-devtools [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/333553 (owner: 10Paladox) [12:47:15] !log tools.wikibugs Updated channels.yaml to: 772229bbac5c6239fd8d858fcbba2f123951394e Add phabricator-upstream to #wikimedia-dev and #wikimedia-devtools [12:47:41] (03PS1) 10Jean-Frédéric: Rename -lang argument -langcode [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/373878 (https://phabricator.wikimedia.org/T166528) [12:48:44] (03CR) 10Jean-Frédéric: "I cherry-picked this from the Wikidata branch, as I needed it to get the ball rolling for Malaysia harvesting." [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/373878 (https://phabricator.wikimedia.org/T166528) (owner: 10Jean-Frédéric) [12:56:38] 10VPS-project-Wikipedia-Requests: Request creation of deep-learning-services VPS project - https://phabricator.wikimedia.org/T172421#3552799 (10Ebraminio) Any news around? This really delayed my work as some of the models I am going to use for the upcoming works (can be tracked on the given link) needs more syst... [12:58:44] (03PS1) 10Jean-Frédéric: Do not crash when unkwnown_fields report cannot be created [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/373880 [13:00:51] Hey guys, not sure who to CC to this https://phabricator.wikimedia.org/T172421 can you have a look and CC related people? No comment for 23days and it delayed my works really [13:00:59] RECOVERY - Puppet errors on tools-exec-1433 is OK: OK: Less than 1.00% above the threshold [0.0] [13:06:07] 10Cloud-VPS (Project-requests): Request creation of deep-learning-services VPS project - https://phabricator.wikimedia.org/T172421#3552841 (10chasemp) [13:06:28] 10Cloud-VPS (Project-requests): Request creation of deep-learning-services VPS project - https://phabricator.wikimedia.org/T172421#3498172 (10chasemp) p:05Triage>03Normal [13:59:59] 10Cloud-VPS (Project-requests): Request creation of deep-learning-services VPS project - https://phabricator.wikimedia.org/T172421#3498172 (10Andrew) @Ebraminio sorry about the delay in responding -- this task wasn't tagged with 'Project-requests' and so was largely invisible until just now. We'll review it dur... [14:56:51] 10Cloud-VPS (Project-requests): Request creation of deep-learning-services VPS project - https://phabricator.wikimedia.org/T172421#3498172 (10bd808) >>! In T172421#3498886, @Ebraminio wrote: > Also not something that I would use immediately (as I am not going to do such training this soon, and even on that case,... [15:04:26] 10Cloud-Services, 10Tools-Kubernetes, 10Kubernetes: newer npm for nodejs Kubernetes instances - https://phabricator.wikimedia.org/T169451#3398652 (10akosiaris) FYI, npm hasn't made it to stretch, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=857986 . The package is orphaned unfortunately. [15:13:12] 10Cloud-Services, 10cloud-services-team (Kanban), 10Patch-For-Review: nova-fullstack is losing instances on creation - https://phabricator.wikimedia.org/T165555#3553227 (10Andrew) There were two more failures during the last few days. Both instances were up and working fine when I checked. The fullstack te... [15:28:52] 10Cloud-VPS (Project-requests), 10artificial-intelligence: Provide large disk space to WikiBrain for memory-mapped file - https://phabricator.wikimedia.org/T161554#3553266 (10Andrew) Hello! I believe that this request has now been fulfilled; can you confirm that you're ready to roll? [16:40:00] 10Data-Services, 10cloud-services-team (Kanban), 10Analytics, 10DBA, 10Research: Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#3553473 (10bd808) [16:52:20] 10wikitech.wikimedia.org: Install two Responsive Vector gadgets on Wikitech - https://phabricator.wikimedia.org/T174178#3553488 (10Quiddity) [17:27:10] 10Data-Services, 10cloud-services-team (Kanban), 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3553602 (10Lucas_Werkmeister_WMDE) 05Open>03Resolved Seems to be done – thanks @Andrew! [17:32:31] 10Toolforge, 10cloud-services-team, 10DBA: set up hiwikiversity on labsdb1010 - https://phabricator.wikimedia.org/T174182#3553611 (10Andrew) [17:32:55] 10Data-Services, 10cloud-services-team (Kanban), 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447813 (10Andrew) 05Resolved>03Open Blocked by https://phabricator.wikimedia.org/T174182 [17:33:10] 10Toolforge, 10cloud-services-team, 10DBA: set up hiwikiversity on labsdb1010 - https://phabricator.wikimedia.org/T174182#3553628 (10Andrew) [17:33:13] 10Data-Services, 10cloud-services-team (Kanban), 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3553627 (10Andrew) [17:37:27] 10Data-Services, 10cloud-services-team (Kanban), 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3447813 (10Sjoerddebruin) >>! In T170927#3553623, @Andrew wrote: > Blocked by https://phabricator.wikimedia.org/T174182... [18:25:25] There no tables for hiwikiversity_p yet(?) [18:26:42] T171829 [18:26:42] T171829: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829 [18:26:47] (no) [18:29:18] 10Data-Services, 10DBA: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3477438 (10bd808) blocker: {T174182} [18:29:31] 10Data-Services, 10DBA: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3553865 (10bd808) [18:29:53] 10Data-Services, 10cloud-services-team, 10DBA: set up hiwikiversity on labsdb1010 - https://phabricator.wikimedia.org/T174182#3553611 (10bd808) [18:46:31] 10Data-Services, 10DBA: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3553943 (10Marostegui) [18:46:35] 10Data-Services, 10cloud-services-team, 10DBA: set up hiwikiversity on labsdb1010 - https://phabricator.wikimedia.org/T174182#3553940 (10Marostegui) 05Open>03Resolved a:03Marostegui Our beloved bug... It is now fixed. [18:46:37] 10Data-Services, 10cloud-services-team (Kanban), 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3553944 (10Marostegui) [18:47:38] 10Data-Services, 10DBA: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3477438 (10Marostegui) The blocker is fixed and so is this one too: ``` mysql:root@localhost [hiwikiversity_p]> show tables; +---------------------------+ | Tables_in_hiwikiversity_p | +------... [18:47:46] 10Data-Services, 10DBA: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3553952 (10Marostegui) 05Open>03Resolved a:03Marostegui [18:54:11] 10Data-Services, 10cloud-services-team (Kanban), 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3553979 (10Marostegui) >>! In T170927#3553623, @Andrew wrote: > Blocked by https://phabricator.wikimedia.org/T174182 Tha... [19:00:44] 10Toolforge, 10Tools, 10Developer-Relations, 10Toolforge-standards-committee: Make sure abandoned useful tools are properly advertised so potentially interested new maintainers could find them - https://phabricator.wikimedia.org/T159595#3553992 (10Quiddity) +1, yes please, Andre! [19:18:46] 10Data-Services, 10cloud-services-team (Kanban), 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3554087 (10Andrew) 05Open>03Resolved >>! In T170927#3553979, @Marostegui wrote: >>>! In T170927#3553623, @Andrew wrot... [19:21:15] 10Data-Services, 10cloud-services-team (Kanban), 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3554091 (10bd808) a:05madhuvishy>03Andrew [19:26:29] 10Cloud-Services, 10Toolforge, 10cloud-services-team (Kanban): Build new tools puppetmaster - https://phabricator.wikimedia.org/T169350#3554100 (10Andrew) 05Open>03Resolved [19:26:40] 10Cloud-Services, 10Cloud-VPS, 10cloud-services-team (Kanban), 10Operations: Puppet CA: virt1000.wikimedia.org' will expire on 2017-08-15 - https://phabricator.wikimedia.org/T168110#3554101 (10Andrew) 05stalled>03Resolved We're on new puppetmasters with new certs now. [19:26:46] 10Cloud-Services, 10Cloud-VPS, 10cloud-services-team (Kanban), 10Operations: Puppet CA: virt1000.wikimedia.org' will expire on 2017-08-15 - https://phabricator.wikimedia.org/T168110#3554104 (10Andrew) [19:26:48] 10Cloud-VPS, 10cloud-services-team (Kanban), 10Operations, 10Patch-For-Review: Switch to new labs puppetmasters - https://phabricator.wikimedia.org/T171786#3554103 (10Andrew) 05Open>03Resolved [20:09:11] 10Toolforge, 10Technical-Tool-Request, 10Toolforge-standards-committee (Maintainer needed): New Maintainer needed for LanguageTool WikiCheck on Toolforge - https://phabricator.wikimedia.org/T152049#3554173 (10bd808) [20:11:01] 10Toolforge, 10Tools, 10Developer-Relations, 10Toolforge-standards-committee: Make sure abandoned useful tools are properly advertised so potentially interested new maintainers could find them - https://phabricator.wikimedia.org/T159595#3554175 (10bd808) Created #help-wanted [22:13:28] 10cloud-services-team (Kanban), 10wikitech.wikimedia.org, 10User-bd808: Install two Responsive Vector gadgets on Wikitech - https://phabricator.wikimedia.org/T174178#3554668 (10bd808) 05Open>03Resolved a:03bd808 {F9175031} [22:19:27] !log rcm Neon: Running update [22:19:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL [22:20:59] !log rcm Xenon: Running update [22:21:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL [22:21:17] !log rcm Oxygen: Running update [22:21:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL [22:22:47] PROBLEM - Puppet errors on tools-exec-1416 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [22:24:31] 10Data-Services, 10cloud-services-team (Kanban), 10User-bd808: Promote beta test of new Wiki Replica servers - https://phabricator.wikimedia.org/T172704#3554696 (10bd808) a:03bd808 [22:26:17] 10Toolforge, 10Kubernetes: Make tools-webservice use the official kubernetes python client rather than pykube - https://phabricator.wikimedia.org/T159892#3554698 (10bd808) [22:27:37] 10Toolforge, 10cloud-services-team (Kanban), 10Documentation, 10User-bd808: Update [[Help:Toolforge/Developing]] docs - https://phabricator.wikimedia.org/T174084#3554702 (10bd808) [22:28:12] 10Toolforge, 10cloud-services-team (Kanban), 10Documentation, 10User-bd808: Update code and/or docs for "How can I detect if I'm running in Labs?" - https://phabricator.wikimedia.org/T174082#3554705 (10bd808) [22:30:02] 10Toolforge, 10Documentation, 10User-bd808: Record hangout of basic Tool Labs access and use - https://phabricator.wikimedia.org/T162654#3170364 (10bd808) [22:31:35] 10Toolforge, 10Kubernetes, 10Tracking: Packages to be installed in Toolforge Kubernetes Images (Tracking) - https://phabricator.wikimedia.org/T140110#3554716 (10bd808) [22:31:37] 10Toolforge, 10Tools-Kubernetes, 10cloud-services-team (Kanban), 10Kubernetes: Install DjVuLibre and XPDF packages for Kubernetes containers on Tool Labs - https://phabricator.wikimedia.org/T166985#3554715 (10bd808) 05Open>03stalled [22:42:42] (03PS1) 10BryanDavis: Bump striker submodule [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/373978 [22:47:37] (03CR) 10BryanDavis: [C: 032] Bump striker submodule [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/373978 (owner: 10BryanDavis) [22:47:43] (03Merged) 10jenkins-bot: Bump striker submodule [labs/striker/deploy] - 10https://gerrit.wikimedia.org/r/373978 (owner: 10BryanDavis) [22:57:46] RECOVERY - Puppet errors on tools-exec-1416 is OK: OK: Less than 1.00% above the threshold [0.0]