[03:57:35] 10serviceops, 10Gerrit, 10Operations: Gerrit loads very slowly - https://phabricator.wikimedia.org/T215855 (10Paladox) Thanks @hashar [10:59:51] 10serviceops, 10Gerrit, 10Operations: Gerrit loads very slowly - https://phabricator.wikimedia.org/T215855 (10hashar) I have pasted P8073 content to [[ https://fastthread.io/ | fastthread.io ]]. It is an analyzer for Java thread dumps. https://fastthread.io/ft-thread-report.jsp?dumpId=1&oTxnId_value=c865888... [11:18:09] 10serviceops, 10Gerrit, 10Operations: Gerrit loads very slowly - https://phabricator.wikimedia.org/T215855 (10hashar) ` zcat /var/log/apache2/gerrit.wikimedia.org.https.access.log.9.gz|cut -b-13|sort|uniq -c 17821 2019-02-11T06 55594 2019-02-11T07 52925 2019-02-11T08 54292 2019-02-11T09 74124 2019-... [14:11:35] 10serviceops, 10Analytics, 10Operations, 10Research, and 4 others: Transferring data from Hadoop to production MySQL database - https://phabricator.wikimedia.org/T213566 (10akosiaris) 05Open→03Stalled Per comment above. [15:12:21] fsero: ottomata: Found out why helm upgrading would not in some cases cause a pod refresh. https://gerrit.wikimedia.org/r/#/c/operations/deployment-charts/+/491760/ [15:12:48] with the above the template is correctly evaluated and checksummed [15:18:43] akosiaris: huh. [15:18:45] config.app? [15:19:19] it's the name of the template in config.yaml [15:19:25] ahhh right [15:19:26] I wanted to avoid naming it config.yaml [15:19:28] interesting, ok! [15:19:32] feel free to rename though [15:19:39] so that's a better way to grab the rendered template [15:19:50] via the name rather than the file path [15:19:57] got it thanks [15:20:35] how are tests btw? everything eventgaty looking okish? [15:20:47] should we move on with deployment to the production clusters? [15:20:51] ya! i can produce. [15:21:17] we need to get some schema stuff settled first...we could do it now but since we are baking them in i'd rather get that settled than do multiple tag/buiolds [15:21:30] akosiaris: q, we recently deploy the ability to use multiple endpoints from mediawiki [15:21:38] can we configure a staging endpoint there? [15:21:43] or should we not do it that way? [15:21:47] there == ? [15:21:51] in mw prod [15:21:57] no, don't [15:21:58] or...is there a mw staging? [15:22:03] k [15:22:08] we don't need to, we can and will test in beta [15:22:21] actually, i was able to get the docker image running in beta using only horizon interface!!! [15:22:24] pretty amazing! [15:22:38] had to do a couple of patches to _joe_ 's stuff but it works! [15:22:42] cool [15:22:44] that's nice [15:22:55] <_joe_> ottomata: oh do tell? [15:23:05] <_joe_> I wrote it in the void basically [15:23:08] i think i added you as reviewer on them last week... [15:23:10] but [15:23:20] <_joe_> yeah I get too many review requests, sorry [15:23:22] it was adding the class and then setting hiera properly [15:23:28] they were very simple fixes let me find [15:23:59] actually just 2 [15:24:04] typo fix [15:24:04] https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/490400/ [15:24:05] and [15:24:22] use of @version in the script [15:24:22] https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/490409/ [15:24:31] in the systemd unit* [15:30:55] <_joe_> ah sigh [15:30:57] <_joe_> great [15:31:01] <_joe_> I'm happy it works [15:31:09] <_joe_> one less headache right now [16:24:51] Is there a WMF-preferred docker base image? [16:25:55] for production, I believe docker-registry.wikimedia.org/wikimedia-stretch [16:26:55] awesome, thanks! [17:04:42] 10serviceops, 10Operations, 10Traffic, 10Wikidata, and 2 others: [Task] move wikiba.se webhosting to wikimedia cluster - https://phabricator.wikimedia.org/T99531 (10Addshore) Okay, so the question that I have now been asked is "why we can't simply do a DNS re-route without changing the owner". So, why can'... [17:04:53] 10serviceops, 10Operations, 10Traffic, 10Wikidata, and 3 others: [Task] move wikiba.se webhosting to wikimedia cluster - https://phabricator.wikimedia.org/T99531 (10Addshore) [17:26:22] 10serviceops, 10Operations, 10Traffic, 10Wikidata, and 3 others: [Task] move wikiba.se webhosting to wikimedia cluster - https://phabricator.wikimedia.org/T99531 (10BBlack) There are different layers of "handing off" DNS management which are being conflated, but to run through them in order: 1) ** "Point...