[01:58:40] !help Hello, I've created the discordwiki.srwiki-dev.eqiad1.wikimedia.cloud instance on Cloud VPS, but I'm having problems with connecting. [01:58:40] If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-kanban [01:58:47] channel 0: open failed: connect failed: No route to host [01:58:47] stdio forwarding failed [01:58:47] ssh_exchange_identification: Connection closed by remote host [01:59:01] It got created before 12 minutes, and in the log I see that system is installed. [01:59:14] My config file is correct, I [01:59:27] *I've removed the known_hosts for each case. [02:07:39] Connecting from srwiki-dev.srwiki-dev.eqiad1.wikimedia.cloud works.. Umm... [03:06:20] hello friends! Where can I find info on the (average? current?) replag for enwiki_p? [03:07:04] (historical?) [03:20:48] https://grafana.wikimedia.org/d/000000303/mysql-replication-lag?viewPanel=1&orgId=1 [03:21:06] audiodude, that mixes prod and labs tho, filter for what you need [03:22:39] sweet thanks! [03:23:33] AntiComposite: are the units seconds? [03:24:13] I believe they're the number of records that the database is behind [03:25:24] but it might be seconds [03:28:50] (we really need to put more units on graphs in grafana) [03:40:08] thanks again! [09:56:59] !log admin moved cloudvirts 1030, 1029, 1028, 1027, 1026, 1025 away from the 'standard' host aggregate to 'maintenance' (T269172) [09:57:02] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:57:02] T269172: Unable to create new instance of flavour "g2.cores8.ram16.disk1120" - https://phabricator.wikimedia.org/T269172 [11:12:54] !log admin Reset the properties for the flavor g2.cores8.ram16.disk1120 to correct quotes (T269172) [11:12:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:12:58] T269172: Unable to create new instance of flavour "g2.cores8.ram16.disk1120" - https://phabricator.wikimedia.org/T269172 [12:41:48] !log admin disable puppet in all cloudnet servers to merge conntrackd change T268335 [12:41:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [12:41:52] T268335: cloud: neutron l3 agent: improve failover handling - https://phabricator.wikimedia.org/T268335 [15:08:42] !log admin removing all osds on cloudcephosd1012 for rebuild, T268746 [15:08:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [15:08:46] T268746: [ceph] cloudcephosd1004-1015 think that their hard drives are HDD when they are SSD - https://phabricator.wikimedia.org/T268746 [15:33:00] !log admin [codfw1dev] conntrackd is now up and running in cloudnet200x-dev nodes (T268335) [15:33:02] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [15:33:04] T268335: cloud: neutron l3 agent: improve failover handling - https://phabricator.wikimedia.org/T268335 [15:36:25] !log admin conntrackd is now up and running in cloudnet1003/1004 nodes (T268335) [15:36:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [17:25:27] !log admin [15:51] failovering neutron virtual router in eqiad1 (T268335) [17:25:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [17:25:30] T268335: cloud: neutron l3 agent: improve failover handling - https://phabricator.wikimedia.org/T268335 [20:03:56] !log admin removing all osds on cloudcephosd1010 for rebuild, T268746 [20:04:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [20:04:01] T268746: [ceph] cloudcephosd1004-1015 think that their hard drives are HDD when they are SSD - https://phabricator.wikimedia.org/T268746 [21:06:38] !log tools.wdqs-wmil-tutorial Replaced webservice with a custom ingress redirecting to https://wdqs-tutorial.toolforge.org/ [21:06:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wdqs-wmil-tutorial/SAL [21:35:17] !log tools.lexeme-forms deployed e5291d5cda (more Esperanto translations) [21:35:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [22:28:29] bd808: https://phabricator.wikimedia.org/T169368#6665132 (did I resolve your ticket from 2017) [22:28:46] finally merged that (because it touched httpd globally) [22:29:10] but that makes it possible now for any role using httpd module to toggle if it wants manual files to be purged or not [22:29:52] mutante: it sounds like simplelamp2 is the fix, yes :) [22:30:27] bd808: the fix is that httpd now has "purge_manual_config" and not always hardcoded to "true" [22:30:34] ok, cool :) [22:31:56] I did not merge that for a long time because the "what's the worst that could happen" was "you could purge all files from all apaches globally at once" :o [22:34:31] that's a pretty strong failure scenario :) [22:35:38] haha, yea :)