[07:32:49] kostajh: we don't allow exec for deploy users, sorry. What do you want to run exactly? [07:59:20] jayme: the context for this is T279411. The command I'd like to run is `FLASK_PROFILE=1 flask query --title=Barack_Obama --project=wikipedia --wiki-domain=cs --max-recommendations=15` [08:12:17] kostajh: I changed it to "--page-title=Barack_Obama" and ran it in staging https://phabricator.wikimedia.org/P15208 [08:14:06] jayme: thanks, there should have been some profiler output though... one sec [08:44:38] jayme: I've pushed https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/677493 as an alternative means to getting this info [08:49:02] kostajh: +1ed [08:59:22] 10serviceops, 10SRE, 10Traffic: Feedback for new service IP flowchart - https://phabricator.wikimedia.org/T279296 (10akosiaris) Hi, thanks for this! This is interesting, some feedback from my side: * What is the intended audience? If it's an SRE or at least someone who knows what LVS is, it's pretty fine,... [10:17:10] 10serviceops, 10SRE, 10Traffic: Feedback for new service IP flowchart - https://phabricator.wikimedia.org/T279296 (10ayounsi) Audience is indeed SREs, to be used to help them know what the (high level and preferred) options are when deploying a new service. `Service-deployment-requests` is nice, I'd imagine... [11:37:46] 10serviceops, 10SRE, 10Traffic: Feedback for new service IP flowchart - https://phabricator.wikimedia.org/T279296 (10akosiaris) >>! In T279296#6979590, @ayounsi wrote: > Audience is indeed SREs, to be used to help them know what the (high level and preferred) options are when deploying a new service. OK, go... [14:45:35] 10serviceops, 10SRE, 10ops-codfw, 10Patch-For-Review: decom 8 codfw appservers purchased on 2016-06-02 - https://phabricator.wikimedia.org/T277780 (10Papaul) [14:45:36] 10serviceops, 10Patch-For-Review: decom 28 codfw appservers purchased on 2016-05-17 (mw2215 through mw2242) - https://phabricator.wikimedia.org/T277119 (10Papaul) [16:01:52] 10serviceops, 10SRE, 10WMF-JobQueue, 10Sustainability (Incident Followup): Have some dedicated jobrunners that aren't active videoscalers - https://phabricator.wikimedia.org/T279100 (10Legoktm) + 1 to those from me too. Note that we'll also need to make this distinction in codfw too. >>! In T279100#697307... [18:37:27] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: (Need By: TBD) rack/setup/install conf200[456].codfw.wmnet - https://phabricator.wikimedia.org/T275637 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by pt1979 on cumin2001.codfw.wmnet for hosts: ` conf2004.codfw.wmnet ` The log... [19:01:52] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: (Need By: TBD) rack/setup/install conf200[456].codfw.wmnet - https://phabricator.wikimedia.org/T275637 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['conf2004.codfw.wmnet'] ` and were **ALL** successful. [19:04:31] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: (Need By: TBD) rack/setup/install conf200[456].codfw.wmnet - https://phabricator.wikimedia.org/T275637 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by pt1979 on cumin2001.codfw.wmnet for hosts: ` conf2005.codfw.wmnet ` The log... [19:10:24] 10serviceops, 10Add-Link, 10Data-Persistence (Consultation), 10Growth-Team, 10Patch-For-Review: Determine why querying is slow and what we can do about it - https://phabricator.wikimedia.org/T279411 (10kostajh) [19:30:22] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: (Need By: TBD) rack/setup/install conf200[456].codfw.wmnet - https://phabricator.wikimedia.org/T275637 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['conf2005.codfw.wmnet'] ` and were **ALL** successful. [19:30:41] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: (Need By: TBD) rack/setup/install conf200[456].codfw.wmnet - https://phabricator.wikimedia.org/T275637 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by pt1979 on cumin2001.codfw.wmnet for hosts: ` conf2006.codfw.wmnet ` The log can be found in `/var... [19:31:21] 10serviceops, 10SRE, 10WMF-JobQueue, 10Sustainability (Incident Followup): Have some dedicated jobrunners that aren't active videoscalers - https://phabricator.wikimedia.org/T279100 (10Dzahn) ACK, thanks Legoktm. That does make sense. [19:54:23] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: (Need By: TBD) rack/setup/install conf200[456].codfw.wmnet - https://phabricator.wikimedia.org/T275637 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['conf2006.codfw.wmnet'] ` and were **ALL** successful. [19:54:44] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: (Need By: TBD) rack/setup/install conf200[456].codfw.wmnet - https://phabricator.wikimedia.org/T275637 (10Papaul) [19:55:33] 10serviceops, 10DC-Ops, 10SRE, 10ops-codfw: (Need By: TBD) rack/setup/install conf200[456].codfw.wmnet - https://phabricator.wikimedia.org/T275637 (10Papaul) 05Open→03Resolved @Joe this is complete. [21:07:48] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad, 10User-jijiki: (Need By: TBD) rack/setup/install mc10[37-54].eqiad.wmnet - https://phabricator.wikimedia.org/T274925 (10Jclark-ctr) @jijiki Racking these host i only have 2 available spots in D4 will any of the ones in this rack be decommissioned soon? [21:25:34] 10serviceops, 10SRE: bring 26 new mediawiki appserver in codfw into production, rack A3 (mw2377 - mw2402) - https://phabricator.wikimedia.org/T278396 (10Dzahn) [21:25:49] 10serviceops, 10SRE: bring 26 new mediawiki appserver in codfw into production, rack A3 (mw2377 - mw2402) - https://phabricator.wikimedia.org/T278396 (10Dzahn) 05Open→03Resolved rack A3 completed [21:26:03] 10serviceops, 10SRE: bring 26 new mediawiki appserver in codfw into production, rack A3 (mw2377 - mw2402) - https://phabricator.wikimedia.org/T278396 (10Dzahn) mw2397 - mw2402 set to Active in Netbox [21:46:27] 10serviceops: bring 9 new mediawiki appserver in codfw into production, new rack A5 (mw2403 - mw2411) - https://phabricator.wikimedia.org/T279599 (10Dzahn) [22:12:56] 10serviceops: bring 10 new mediawiki appserver in codfw into production, new rack A5 (mw2402 - mw2411) - https://phabricator.wikimedia.org/T279599 (10Dzahn) [22:13:55] 10serviceops, 10SRE: bring 25 new mediawiki appserver in codfw into production, rack A3 (mw2377 - mw2401) - https://phabricator.wikimedia.org/T278396 (10Dzahn) [22:14:03] 10serviceops: bring 10 new mediawiki appserver in codfw into production, new rack A5 (mw2402 - mw2411) - https://phabricator.wikimedia.org/T279599 (10Dzahn) [22:16:10] 10serviceops: bring 10 new mediawiki appserver in codfw into production, new rack A5 (mw2402 - mw2411) - https://phabricator.wikimedia.org/T279599 (10Dzahn) [22:16:54] 10serviceops: bring 10 new mediawiki appserver in codfw into production, new rack A5 (mw2402 - mw2411) - https://phabricator.wikimedia.org/T279599 (10Dzahn) plan: 4 more API, 4 more app, 2 more jobrunners, should bring us to totals of 65 / 65 / 20 [22:20:31] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad, 10User-jijiki: (Need By: TBD) rack/setup/install mc10[37-54].eqiad.wmnet - https://phabricator.wikimedia.org/T274925 (10Jclark-ctr) a:05Jclark-ctr→03Cmjohnson @Cmjohnson I have finished up these remaining ones i can do once i get space in A7 and D4. nam... [22:21:17] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad, 10User-jijiki: (Need By: TBD) rack/setup/install mc10[37-54].eqiad.wmnet - https://phabricator.wikimedia.org/T274925 (10Jclark-ctr)