[07:14:11] (03CR) 10Lokal Profil: "> Patch Set 1:" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459543 (owner: 10Lokal Profil) [07:41:56] (03CR) 10Jean-Frédéric: "> Patch Set 2:" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459543 (owner: 10Lokal Profil) [09:54:26] (03CR) 10Jean-Frédéric: [C: 032] Mock get_template_link [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459634 (owner: 10Lokal Profil) [09:56:41] (03Merged) 10jenkins-bot: Mock get_template_link [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459634 (owner: 10Lokal Profil) [09:56:56] (03CR) 10Jean-Frédéric: [C: 032] Make report page tests work offline [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/456635 (https://phabricator.wikimedia.org/T203251) (owner: 10Lokal Profil) [09:58:02] (03CR) 10jenkins-bot: Mock get_template_link [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459634 (owner: 10Lokal Profil) [09:58:35] (03Merged) 10jenkins-bot: Make report page tests work offline [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/456635 (https://phabricator.wikimedia.org/T203251) (owner: 10Lokal Profil) [10:00:07] (03CR) 10jenkins-bot: Make report page tests work offline [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/456635 (https://phabricator.wikimedia.org/T203251) (owner: 10Lokal Profil) [10:22:28] !log wikitextexpadded myself as project admin while investigating where 10.68.16.2 is assigned and why (T202636) [10:22:29] arturo: Unknown project "wikitextexpadded" [10:22:29] T202636: Allow routing between eqiad and eqiad1 regions - https://phabricator.wikimedia.org/T202636 [10:22:40] !log wikitextexp added myself as project admin while investigating where 10.68.16.2 is assigned and why (T202636) [10:22:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikitextexp/SAL [10:33:43] !log admin T202636 manually reserve 10.68.23.253 (in nova-network) [10:33:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:33:45] T202636: Allow routing between eqiad and eqiad1 regions - https://phabricator.wikimedia.org/T202636 [10:36:54] !log admin T202636 creating br-compat bridge in eqiad1 for the compat network [10:36:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:52:31] (03CR) 10Jean-Frédéric: "recheck" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459562 (owner: 10Lokal Profil) [10:52:41] (03CR) 10jerkins-bot: [V: 04-1] Use StatisticsTable [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459562 (owner: 10Lokal Profil) [10:53:28] !log admin T202636 creating all the compat-network configuration in neutron [10:53:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:53:31] T202636: Allow routing between eqiad and eqiad1 regions - https://phabricator.wikimedia.org/T202636 [11:30:34] (03PS1) 10Jean-Frédéric: Add Bash script to run arbitrary Erfgoed Python script [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459769 [11:45:02] (03PS1) 10GTirloni: Add gtirloni to cloud-wide root [labs/private] - 10https://gerrit.wikimedia.org/r/459773 (https://phabricator.wikimedia.org/T203489) [11:53:11] (03PS2) 10Jean-Frédéric: Add Bash script to run arbitrary Erfgoed Python script [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459769 [12:40:12] (03PS1) 10Jean-Frédéric: Include empty sources in Monuments database statistics [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459783 (https://phabricator.wikimedia.org/T203349) [12:40:15] !help is it possible for cloud project to get a custom domain with https? [12:40:16] This key already exists - remove it, if you want to change it [12:40:32] oh I thought that was for calling help :/ [12:40:39] Wm-bot is not being helpful... [12:40:51] Not sure what's up with that [12:41:03] !help https [12:41:03] Nikerabbit: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [12:41:12] Oh interesting [12:41:25] I need to poke that bot at one point. [12:41:26] "is" is probably a special key to define something [12:41:31] lol [12:41:54] I don't know your answer for sure, unfortunately. [12:42:00] arturo: do you know? [12:42:33] It's a bit early for the rest of the team, so if we don't know, you can go with creating the task, and I can bring it up. [12:43:12] mmm [12:43:51] Nikerabbit: yes, that should be possible. Custom domain are possible AFAIK and HTTPS too, so I don't see any problem with mixing both [12:44:00] cc bstorm_ [12:44:22] Makes sense. I mean, we do our https with let's encrypt anyway [12:44:24] arturo: it may need a public ip- as https for a custom domain [12:44:40] jynus: we can do that as well [12:44:43] Yeah, a floating IP in the project would probably do that [12:44:44] is different that reusing a common wmflabs cert [12:44:45] then ok [12:45:21] Nikerabbit: I don't have the docs at hand right now, though [12:45:22] Like I said, I don't know the specifics, and Nikerabbit, you should probably create a task either way. We can huddle about it :) [12:45:32] I looked for some...no luck so far. [12:45:34] sure, I will create a task [12:45:40] Thanks [12:49:05] Nikerabbit: custom domain is possible as long as it's under .wmflabs.org. Other top levels are a whole different can of worms. [12:49:19] For https you'd need to manager your own cert and all that. [12:50:22] ^that is what I meant, but andrew expressed it better [12:51:22] andrewbogott: we are specifically looking for a custom tld with https [12:52:23] hmm, not tld, but not wmflabs.org either [12:52:49] ok — if you make a phab task we can look at it but I'm not sure we've done that before [12:54:18] sure (I think wikiapiary.com has such, but it might be a special case) [12:54:36] If it's directly under wmflabs.org and you are able to live behind a web proxy, use the existing proxy. [12:54:42] It has a *.wmflabs.org cert [12:54:48] Not LE or anything, it's an old one [12:55:20] GlobalSign, current one is good until November [12:56:24] oh, not wmflabs.org but running in labs? You'll need a floating IP (regardless of HTTP or HTTPS), and to provide your own cert. I would use LE rather than uploading a paid cert into labs if it were me [12:57:42] Krenair: yes exactly that case, and LE is okay [12:57:51] are you moving translatewiki to WMCS? :) [12:59:01] paravoid: this is for another project, but we can discuss that too separately [12:59:10] heh [12:59:12] nah I was just curious :) [13:00:21] ok, but future hosting of twn is something that needs be thought about [13:01:58] sure :) [13:02:33] (03PS2) 10Jean-Frédéric: Include empty sources in Monuments database statistics [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459783 (https://phabricator.wikimedia.org/T203349) [13:38:45] (03CR) 10Andrew Bogott: [C: 031] Add gtirloni to cloud-wide root [labs/private] - 10https://gerrit.wikimedia.org/r/459773 (https://phabricator.wikimedia.org/T203489) (owner: 10GTirloni) [14:21:43] (03PS3) 10Jean-Frédéric: Include empty sources in Monuments database statistics [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459783 (https://phabricator.wikimedia.org/T203349) [16:08:17] !log admin restarted nova-network after killing all dnsmasq procs in labnet1001 for T202636 [16:08:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:08:20] T202636: Allow routing between eqiad and eqiad1 regions - https://phabricator.wikimedia.org/T202636 [16:13:25] !log bastion force-reboot bastion-restricted-01 bc it lost the IP address, and to force a new DHCP query [16:13:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Bastion/SAL [16:52:14] !log admin again, restarted nova-network after killing all dnsmasq procs in labnet1001 for T202636 [16:52:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:52:17] T202636: Allow routing between eqiad and eqiad1 regions - https://phabricator.wikimedia.org/T202636 [16:58:10] !log bastion add myself as project admin [16:58:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Bastion/SAL [17:14:22] !log quarry disabling puppet on quarry-main-01, quarry-runner-0{1,2} T202588 [17:14:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [17:14:25] T202588: Do the big Quarry migration - https://phabricator.wikimedia.org/T202588 [17:22:48] !log quarry doing another backup of main db: `sudo mysqldump quarry | sudo tee /data/project/dump-$(date '+%Y-%m-%d').sql > /dev/null` T202588 [17:22:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [17:22:51] T202588: Do the big Quarry migration - https://phabricator.wikimedia.org/T202588 [17:37:48] (03CR) 10GTirloni: [C: 032] Add gtirloni to cloud-wide root [labs/private] - 10https://gerrit.wikimedia.org/r/459773 (https://phabricator.wikimedia.org/T203489) (owner: 10GTirloni) [17:38:02] (03CR) 10GTirloni: [V: 032 C: 032] Add gtirloni to cloud-wide root [labs/private] - 10https://gerrit.wikimedia.org/r/459773 (https://phabricator.wikimedia.org/T203489) (owner: 10GTirloni) [17:38:17] (03CR) 10GTirloni: [C: 032] Add gtirloni to cloud-wide root [labs/private] - 10https://gerrit.wikimedia.org/r/459773 (https://phabricator.wikimedia.org/T203489) (owner: 10GTirloni) [17:38:42] (03CR) 10GTirloni: [V: 032 C: 032] Add gtirloni to cloud-wide root [labs/private] - 10https://gerrit.wikimedia.org/r/459773 (https://phabricator.wikimedia.org/T203489) (owner: 10GTirloni) [17:42:23] !log tools.versions Update to b71fd65 (Revert "Add a cache-busting parameter when fetching from noc.wm.o") [17:42:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.versions/SAL [18:55:47] jem: did you ever open a ticket about your imagemagick issue? Or did you decide you can live without pdf? [19:28:26] andrewbogott: bd808: VE problems editing wikitech [19:28:38] can you reproduce or is it EU only ? [19:28:56] * andrewbogott tries [19:29:16] akosiaris: Error loading data from server: apierror-visualeditor-docserver-http: HTTP 500 ? [19:29:24] andrewbogott: yes that [19:29:39] ok so this is not related to us just depooling esams [19:29:44] ok we proceed then. thanks! [19:30:59] I think I remember VE edit issues in the last DC switchover [19:31:06] for wikitech [19:31:27] it started having issues before we even depooled eqiad btw [19:31:35] when we routed esams to codfw [19:33:02] anyway I 'll stick around and help debug later on, but everything else seems to be fine so we move forward [19:46:23] Yeah, looks like RB/Parsoid aren't able to talk to the database for wikitechwiki? https://wikitech.wikimedia.org/w/api.php?action=visualeditor&format=json&paction=parse&page=Deployments&uselang=en is the 500ing error. [20:46:07] T163438 [20:46:08] T163438: VisualEditor broken on wikitech when codfw is primary: "Error loading data from server: apierror-visualeditor-docserver-http: HTTP 500." - https://phabricator.wikimedia.org/T163438 [21:06:33] bd808: that ^ [21:06:42] gtirloni: thanks. And welcome btw [21:07:57] akosiaris: thanks! [21:08:30] found parsoid errors on wtp2013.codfw.wmnet [21:11:41] "levelPath": "fatal/request", [21:11:42] "msg": "Config Request failure for \"https://wikitech.wikimedia.org/w/api.php\": 404", [21:11:45] I am guessing [21:11:50] that's a start... [21:12:03] I am sensing a rabbithole, but what the heck