[00:03:07] PROBLEM - Puppet errors on tools-worker-1014 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [00:27:08] PROBLEM - Puppet errors on tools-exec-1407 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [00:38:06] RECOVERY - Puppet errors on tools-worker-1014 is OK: OK: Less than 1.00% above the threshold [0.0] [01:02:11] RECOVERY - Puppet errors on tools-exec-1407 is OK: OK: Less than 1.00% above the threshold [0.0] [01:28:18] PROBLEM - Puppet errors on tools-exec-1420 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [01:37:57] PROBLEM - Puppet errors on tools-worker-1020 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [02:43:51] RECOVERY - Puppet errors on tools-exec-1420 is OK: OK: Less than 1.00% above the threshold [0.0] [03:18:30] (03CR) 10Addshore: [C: 031] Ignore libraryupgrader in grrrit-wm [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/375870 (https://phabricator.wikimedia.org/T174772) (owner: 10Merlijn van Deen) [03:19:11] (03CR) 10Addshore: [C: 031] Logging to irc no longer supported -- fall back to stdout [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/375858 (owner: 10Merlijn van Deen) [03:43:00] RECOVERY - Puppet errors on tools-worker-1020 is OK: OK: Less than 1.00% above the threshold [0.0] [03:49:55] PROBLEM - Puppet errors on tools-exec-1414 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [04:14:55] RECOVERY - Puppet errors on tools-exec-1414 is OK: OK: Less than 1.00% above the threshold [0.0] [04:48:50] 10Cloud-VPS, 10MediaWiki-Vagrant: Different provider base images have different /etc/apt/sources.list settings - https://phabricator.wikimedia.org/T175055#3595296 (10bd808) [05:22:58] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/377189 (owner: 10L10n-bot) [06:34:51] PROBLEM - Puppet errors on tools-exec-1428 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:35:39] PROBLEM - Puppet errors on tools-exec-1417 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:08:57] PROBLEM - Puppet errors on tools-worker-1020 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:09:51] RECOVERY - Puppet errors on tools-exec-1428 is OK: OK: Less than 1.00% above the threshold [0.0] [07:10:40] RECOVERY - Puppet errors on tools-exec-1417 is OK: OK: Less than 1.00% above the threshold [0.0] [07:29:09] PROBLEM - Puppet errors on tools-worker-1014 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:43:59] RECOVERY - Puppet errors on tools-worker-1020 is OK: OK: Less than 1.00% above the threshold [0.0] [07:55:12] 10Quarry: Provide a way to hyperlink Quarry results/output - https://phabricator.wikimedia.org/T74874#771075 (10Tgr) Ideally this would be defined in the query code, e.g. ```lang=sql SELECT page_namespace, page_title, -- @quarry:format page_title el_to, -- @quarry:format url gt_lat, gt_lon, -- @quarry:f... [07:59:55] 10Quarry: Make query URLs have a sluggified version of the title in them - https://phabricator.wikimedia.org/T75885#784350 (10Tgr) It doesn't sound too expensive to just generate the slug from the title on the fly, without storing any new data. [08:04:36] 10Quarry: Allow comments on queries - https://phabricator.wikimedia.org/T71543#720495 (10Tgr) Just export query description and SQL to a wiki page (with a Flow talk page), subscribe query owner to it? Not too much effort and would also solve {T90509}. [08:06:59] 10Quarry: Have an easy way to ban users from Quarry - https://phabricator.wikimedia.org/T104322#1413399 (10Tgr) If the concern is abusive users (as oppose to well-intentioned but clueless ones), just block them on meta and rely on the OAuth block data? [08:09:06] RECOVERY - Puppet errors on tools-worker-1014 is OK: OK: Less than 1.00% above the threshold [0.0] [08:34:58] PROBLEM - Puppet errors on tools-worker-1020 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [08:36:14] 10Cloud-Services, 10Operations: rack/setup/install labstore100[67].wikimedia.org - https://phabricator.wikimedia.org/T167984#3595460 (10fgiunchedi) @madhuvishy no problem! I've added a section about this to https://wikitech.wikimedia.org/wiki/Platform-specific_documentation/HP_DL3N0_Gen9 too [09:00:41] !help Bug report (message: Script error: No such module "Shortcut".) @ https://en.wikipedia.beta.wmflabs.org/wiki/Wikipedia:Mass_message_senders top of page [09:00:42] sau226: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [09:13:49] 10Tool-Attribution-Generator, 10TCB-Team: Make it possible to attribute text from Wikipedia - https://phabricator.wikimedia.org/T175524#3595621 (10Katja_Ullrich_WMDE) [09:15:00] RECOVERY - Puppet errors on tools-worker-1020 is OK: OK: Less than 1.00% above the threshold [0.0] [09:25:37] 10Quarry: Provide a way to hyperlink Quarry results/output - https://phabricator.wikimedia.org/T74874#3595648 (10Tgr) As a very simple first step, fields that match an URL regexp could be linkified. [10:26:05] 10Data-Services, 10DBA: Significant replication lag for the s1, s2, and s4 wikis on labsdb100[13] - https://phabricator.wikimedia.org/T175487#3594707 (10jcrespo) From the labsdb1001 log: ``` InnoDB: We intentionally crash the server, because it appears to be hung. 2017-09-09 18:21:02 7fe8cdbfe700 InnoDB: Asse... [10:33:31] 10Data-Services, 10DBA: Significant replication lag for the s1, s2, and s4 wikis on labsdb100[13] - https://phabricator.wikimedia.org/T175487#3595894 (10jcrespo) The actual replication problem comes from labsdb1001 running out of space over the weeked- I will search what is to be blamed. I doubt it is the repl... [10:47:49] tool labs is slow... any hint what might be happening? [10:58:15] Server load? tabbycat [11:03:43] no idea [11:04:10] Probably server load or CDN error [11:07:54] (03Abandoned) 10Hashar: Jenkins job validation (DO NOT SUBMIT) [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/375806 (owner: 10Hashar) [11:07:56] (03Abandoned) 10Hashar: Jenkins job validation (DO NOT SUBMIT) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/375808 (owner: 10Hashar) [12:35:28] 10Cloud-Services, 10Tools: cewbot using '* * * * *' cron that could be replaced with .bigbrotherrc - https://phabricator.wikimedia.org/T163572#3596300 (10Kanashimi) Thank you. Done. [12:37:00] 10Cloud-Services, 10Tools: cewbot using '* * * * *' cron that could be replaced with .bigbrotherrc - https://phabricator.wikimedia.org/T163572#3596302 (10Kanashimi) 05Open>03Resolved a:03Kanashimi [12:37:02] 10Cloud-Services: IO issues for Tools instances flapping with iowait and puppet failure - https://phabricator.wikimedia.org/T161898#3596304 (10Kanashimi) [12:41:34] 10cloud-services-team (FY2017-18), 10Developer-Relations, 10Goal: Program 4 Outcome 1: improve documentation - https://phabricator.wikimedia.org/T166401#3596331 (10Qgil) [13:07:02] 10Quarry: The star icon will not show/load in Quarry's star button - https://phabricator.wikimedia.org/T175466#3596392 (10zhuyifei1999) IIRC, you can't star your own query... [13:15:13] 10Data-Services, 10cloud-services-team (Kanban), 10User-bd808: Define naming scheme for connecting to new wiki replica cluster - https://phabricator.wikimedia.org/T174860#3575183 (10Andrew) I've created the db.svc.eqiad.wmflabs. domain. Managing this domain will be a bit of a pain, since it's in noauth-proj... [13:28:14] 10cloud-services-team (Kanban), 10DC-Ops, 10Operations, 10ops-eqiad: labvirt1015 crashes - https://phabricator.wikimedia.org/T171473#3466006 (10Cmjohnson) You have successfully submitted request SR953656459. [13:40:01] 10Cloud-VPS (Project-requests), 10cloud-services-team: Create a project for Wikimedia Armenia - https://phabricator.wikimedia.org/T175567#3596615 (10Ladsgroup) [14:43:34] sau226: The #wikimedia-releng channel is probably a better place to look for people who want to help fix up the Beta Cluster wikis. Beta Cluster is just 1 of 200+ projects in Cloud VPS and the WMCS team does not do anything specific for it beyond keeping OpenStack running. [14:44:31] tabbycat: any more specific report than something unspecified in Toolforge feeling slow? [14:45:02] I know the Wiki Replicas were (are?) bogged down [15:00:23] Our blog post is live -- https://blog.wikimedia.org/2017/09/11/introducing-wikimedia-cloud-services/ [15:02:15] \o/ [15:06:01] bd808: the TechOps href points to enwiki/OpenStack [15:06:03] also, nice work :) [15:07:19] paravoid: doh. I"ll see if I can get that link fixed. Looks like it got messed up in the gdoc at some point [15:07:22] also, under "What is being rebranded and where", you could probably conver to
    , probably copy/paste error :) [15:18:55] 10Toolforge, 10Tools: wsexport tool leaking files in /tmp - https://phabricator.wikimedia.org/T166337#3597173 (10Tpt) An other approach to avoid this problem is maybe to stop doing the ePub to PDF conversion using the grid engine and migrate to a single Kubernetes container doing both the conversion and the we... [15:20:47] 10cloud-services-team (FY2017-18), 10Wikimedia-Blog-Content, 10User-bd808: Publish a blog post covering the Cloud Services rebranding process - https://phabricator.wikimedia.org/T170288#3597181 (10bd808) 05Open>03Resolved a:03bd808 Blog post is live: https://blog.wikimedia.org/2017/09/11/introducing-wi... [15:20:49] 10cloud-services-team (FY2017-18), 10Goal, 10User-bd808: Perform initial Cloud Services rebranding - https://phabricator.wikimedia.org/T168480#3597185 (10bd808) [15:25:43] paravoid: I sent an email to Ed with those things to fix up. Both were bugs in the gdoc we wrote for him to copy to the blog. [15:27:42] yeah, figured :) [15:28:07] cool stuff! [15:29:20] Working with Mel on this post was a much nicer experience than I remember having 2.5 years ago when I last wrote for the main blog. :) [15:44:55] !log git revoking puppet for gerrit-test3. [15:44:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL [15:47:19] !log git recreating gerrit-test3. [15:47:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL [15:47:27] 10Cloud-VPS (Project-requests), 10cloud-services-team (Kanban), 10User-bd808: Request creation of project-smtp VPS project - https://phabricator.wikimedia.org/T174618#3597291 (10herron) Thanks! I'm logged into horizon and see the option to launch instance. Should I be able to manage volumes for project-smt... [15:49:39] 10Cloud-VPS (Project-requests), 10cloud-services-team (Kanban), 10User-bd808: Request creation of project-smtp VPS project - https://phabricator.wikimedia.org/T174618#3597317 (10bd808) We do not have support for external volumes in Cloud VPS today. The closest thing we have is per-project NFS, but that is pr... [15:53:11] (03PS1) 10Rush: designate private to match for changeset 376848 [labs/private] - 10https://gerrit.wikimedia.org/r/377289 [15:54:49] (03CR) 10Rush: [V: 032 C: 032] designate private to match for changeset 376848 [labs/private] - 10https://gerrit.wikimedia.org/r/377289 (owner: 10Rush) [15:56:24] 10Toolforge: Increases the memory available to the corenlp tool container - https://phabricator.wikimedia.org/T175593#3597355 (10Tpt) [16:00:41] 10Data-Services, 10DBA: Significant replication lag for the s1, s2, and s4 wikis on labsdb100[13] - https://phabricator.wikimedia.org/T175487#3597389 (10jcrespo) So the replicas are catching up without problem https://tools.wmflabs.org/replag/ (right now the lag on s2 and s4 are almost down to 5 hours from 40h... [16:14:53] I’m trying to figure out why I can’t ssh into suggestbot-prod.eqiad.wmflabs, when I can ssh into the primary bastion just fine. All I get is “Permission denied (publickey)”, and the access help page doesn’t really help with that, as I’m using “ProxyCommand” to ssh in. [16:15:29] It’s a newly created instance, btw, I set it up on Saturday. [16:17:07] Nettrom: are you able to ssh into other non-bastion instances? [16:17:27] andrewbogott: good question, is there a specific one I could use for testing? [16:17:46] Nettrom: I guess another way of phrasing my question is: "have you ever done this before?" [16:18:14] andrewbogott: not on labs, I use it to ssh into the WMF stats machines all the time, but with a different SSH key [16:18:22] I can't necessarily know what VMs you would have access to [16:18:41] so, that instance is definitely working fine (I can access it with my normal user key) [16:18:49] what project is it in? [16:19:00] suggestbot [16:19:25] and AFAIK this is the only instance I have access to [16:21:25] can you show me what you have in your .ssh/config for your proxycommand? [16:21:46] “ProxyCommand ssh -a -W %h:%p nettrom@primary.bastion.wmflabs.org” [16:22:41] I grabbed the configuration settings from https://wikitech.wikimedia.org/wiki/Help:Access#Accessing_instances_with_ProxyCommand_ssh_option_.28recommended.29 [16:22:43] the whole section though? [16:22:47] `id nettrom` there does show the expected group membership [16:23:23] bd808: I don't see 'nettrom' in auth.log [16:23:32] so I don't think it's getting as far as checking a key on that host [16:23:34] here’s the whole section: https://gist.github.com/nettrom/b654e80f0432e0ec3f3075e7504e23dd [16:23:35] me neither, but there is "error: AuthorizedKeysCommand /usr/sbin/ssh-key-ldap-lookup returned status 1" [16:24:38] `/usr/sbin/ssh-key-ldap-lookup nettrom` works though [16:25:10] Nettrom: try sshing with -v and see what it says? [16:25:22] the lookup errors are andrewbogott and I logging in so red herring [16:26:36] andrewbogott: updated https://gist.github.com/nettrom/b654e80f0432e0ec3f3075e7504e23dd with the output [16:27:10] yeah, so the bastion itself is rejecting you [16:28:04] Does the file /Users/morten/.ssh/wmcloud actually exist? [16:28:21] gimme a sec [16:29:57] andrewbogott: it’s always worked, I just wanted to update the gist so it’s documented :) [16:31:58] "it's always worked" <- what has always worked? [16:32:15] ssh’ing into the bastion server itself, which is why I’ve been puzzled about this [16:32:55] I can’t say I understand why I can ssh into it, but get rejected when I want to proxy through it [16:33:09] can you try adding the IdentityFile %d/.ssh/wmcloud line to your Host *.eqiad.wmflabs section? [16:33:17] sure thing! [16:33:21] Or else explicitly add -i ~/.ssh/ to your commandline [16:33:22] Nettrom: by bastion you mean primary.bastion.wmflabs.org? [16:35:02] andrewbogott: adding the IdentityFile line didn’t work, explicitly adding the key with `-i` didn’t work either [16:35:16] and the file actually exists? [16:35:47] yes [16:36:05] andrewbogott: I don't think ssh -i applies to the ssh within proxycommand, or does it? [16:36:22] it does [16:36:38] o.O [16:37:16] 10Tools, 10cloud-services-team (Kanban), 10User-bd808: Update replag tool to show per-host lag - https://phabricator.wikimedia.org/T175600#3597554 (10bd808) [16:38:09] zhuyifei1999_: oh, I misunderstood the question I think [16:38:20] the -i passes the key to the final host and not to the proxy, I think [16:38:21] Nettrom: by 'ssh’ing into the bastion server itself' do you mean ssh nettrom@primary.bastion.wmflabs.org [16:38:28] zhuyifei1999_: yes [16:39:18] that would be weird, why would proxycommand ssh fail and not non-proxycommand ssh [16:39:45] same host, same username... or do you have an alias set up? [16:39:58] not that I’m aware of, no [16:40:10] i.e. alias ssh=ssh -i [16:40:13] o.O [16:40:39] Nettrom: I assume you get the same results if you use suggestbot-prod.suggestbot.eqiad.wmflabs? (which, generally you should be using those complete names anyway) [16:42:08] Yep, same results! And thanks for letting me know there’s a full name to it, I’ll make sure to use that from now on :) [16:42:49] Nettrom: could you -v the proxycommand ssh? [16:43:07] eg. ProxyCommand ssh -v -a -W %h:%p .... [16:43:17] zhuyifei1999_: sure thing, give me a second [16:43:52] ah, this is odd [16:44:03] I’ll update the gist [16:44:11] it’s using the wrong key [16:45:09] (03PS1) 10Rush: openstack values from designate refactor [labs/private] - 10https://gerrit.wikimedia.org/r/377303 [16:45:15] https://gist.github.com/nettrom/b654e80f0432e0ec3f3075e7504e23dd updated [16:46:21] (03PS2) 10Rush: openstack values from designate refactor [labs/private] - 10https://gerrit.wikimedia.org/r/377303 [16:46:52] (03CR) 10Rush: [V: 032 C: 032] openstack values from designate refactor [labs/private] - 10https://gerrit.wikimedia.org/r/377303 (owner: 10Rush) [16:47:05] there we go [16:47:46] adding a line to the config for “Host primary.bastion.wmflabs.org” so it reads the identify file worked [16:48:06] oh, of course, it's wmflabs.org so doesn't match the other lines [16:48:14] 'of course' in retrospect :) [16:48:20] exactly! [16:48:58] but I thought I tried to add the key to the ProxyCommand, which didn’t work, but anyways… [16:49:12] what I don't understand is, why would `ssh nettrom@primary.bastion.wmflabs.org` find the right key? [16:49:24] because I specified it on the command line [16:49:30] ... [16:49:42] duh [16:50:17] I think this ends up in the “I have enough ssh keys for it to confuse me” folder [16:53:00] thanks so much for helping me out with this one everyone, really happy to get a suggestbot instance up so I can start moving the bot over! [16:53:30] np [17:00:40] 10Data-Services, 10DBA: Significant replication lag for the s1, s2, and s4 wikis on labsdb100[13] - https://phabricator.wikimedia.org/T175487#3597682 (10jcrespo) I had to kill a few queries that were blocking the replication- this should only happen one time; but applications should be prepared to fail and ret... [17:15:24] 10Cloud-VPS (Project-requests), 10cloud-services-team: Create a project for Wikimedia Armenia - https://phabricator.wikimedia.org/T175567#3597717 (10bd808) [17:16:03] 10Cloud-VPS (Project-requests), 10cloud-services-team: Create a project for Wikimedia Armenia - https://phabricator.wikimedia.org/T175567#3596615 (10bd808) @Ladsgroup please fill out the project request template components that I have added for you. [17:27:46] 10Cloud-VPS (Project-requests), 10cloud-services-team: Create a project for Wikimedia Armenia - https://phabricator.wikimedia.org/T175567#3597746 (10Ladsgroup) [17:27:58] 10Cloud-VPS (Project-requests), 10cloud-services-team: Create a project for Wikimedia Armenia - https://phabricator.wikimedia.org/T175567#3596615 (10Ladsgroup) Done :) [17:37:02] my tool was having some kind of problem with the webservice... requests (even trivial ones to the index.php that doesn't hit any databases) would just hang forever, since a few hours ago. I did `webservice restart` and it's fine now. [17:37:19] (just in case it's relevant for anyone else) [18:29:52] 10Toolforge, 10User-Matthewrbowker: Articlerequest tool goes up and down often - https://phabricator.wikimedia.org/T175623#3598142 (10Matthewrbowker) [18:34:21] 10PAWS: Pandas to_clipboard function does not work in remote environments like PAWS - https://phabricator.wikimedia.org/T155763#3598185 (10Neil_P._Quinn_WMF) >>! In T155763#3491400, @yuvipanda wrote: > Closing, since the modle of pasting won't really ever work with remote kernels like PAWS :) I'm thrilled that... [18:53:49] bd808, hi i am wondering how do i set the forwarded port in vagrant again please? [18:54:23] paladox: vagrant forward-port --help [18:54:28] thanks [18:55:06] hmm vagrant reload stalls for me [18:57:32] 10Cloud-Services, 10Operations, 10ops-eqiad, 10Patch-For-Review: rack/setup/install labvirt10(19|20).eqiad.wmnet - https://phabricator.wikimedia.org/T172538#3598262 (10Cmjohnson) [18:58:14] paladox: I've seen that before as well in Cloud VPS. The LXC containers do not always restart nicely. I usually reboot the host VM [18:58:23] yep [18:58:24] (when they are stuck) [18:58:33] thanks [18:59:08] 10Cloud-Services, 10Operations, 10ops-eqiad, 10Patch-For-Review: rack/setup/install labvirt10(19|20).eqiad.wmnet - https://phabricator.wikimedia.org/T172538#3501521 (10Cmjohnson) All the on-site work has been completed for labvirts1019-20. @robh lmk if you want to take it from here [18:59:46] 10Cloud-Services, 10Operations, 10ops-eqiad, 10Patch-For-Review: rack/setup/install labvirt10(19|20).eqiad.wmnet - https://phabricator.wikimedia.org/T172538#3598266 (10RobH) a:05Cmjohnson>03RobH [19:04:53] bd808: with regards to your previous question, it was a fact rather than a feeling, but it happened like 12 hours ago and I've not tested the tool again [19:08:38] tabbycat: you did not give any really actionable points (still) [19:08:40] Toolforge is a lot of stuff [19:10:55] bd808: http://tools.wmflabs.org/meta/stalktoy took a life to load and give results; and the results ain't accurate either, seems that there's replication lag? [19:13:06] s1 is still behind -- https://tools.wmflabs.org/replag/ [19:15:43] 10Cloud-VPS, 10cloud-services-team (Kanban), 10Continuous-Integration-Infrastructure, 10Nodepool, and 2 others: figure out if nodepool is overwhelming rabbitmq and/or nova - https://phabricator.wikimedia.org/T170492#3598327 (10hashar) From this week-end logs, nova.network.manager had the same chain of logs... [19:16:33] hmm, then the tool has a problem as it outputs outdated data for a db in s3 [19:20:43] hola, I am trying to create a dns domain on top of this instance that i "own": dashiki-01.dashiki.eqiad.wmflabs [19:20:52] but it does not display in horizon... [19:21:05] nuria_: did you select the dashiki project? [19:21:18] milimetric: in horizon? [19:21:20] yea [19:22:06] milimetric: ah i see., there are two projects, thank you let me try again [19:23:54] milimetric: ok, got it and created it, will wait a bit for it to be created [19:24:37] cool, I think the proxy stuff takes effect right away [19:24:46] nuria_: ^ [19:26:45] it should be pretty much instant [19:27:17] unless you had hit the new hostname already in your browser and it locally cached the NXDOMAIN response [19:33:38] bd808 i think the ldapauth mediawiki vagrant class does not correctly register the file though it exists, it seems it is never picked up [19:33:50] so i have to manually add it to localsettings.php [19:34:00] bd808: mmm no browser, just curl: curl http://page-creation.wmflabs.org but still not there... [19:34:12] bd808: maybe i am missing a step? i just added it to horizon [19:34:55] paladox: the main wiki does not get LDAPAuth. http://ldapauth.wiki.local.wmftest.net:8080/wiki/Special:Version does -- note that name may be different if you are not running locally. [19:34:55] i get [19:34:56] Forbidden [19:34:56] You don't have permission to access / on this server. [19:35:21] oh [19:35:42] nuria_: I get an error too. That looks like is coming from your target VM [19:35:58] can you get the landing page you want with curl locally there? [19:36:36] bd808: sorry, i do not get a http error, do you? [19:36:43] bd808: just a dns one [19:37:14] nuria_: yes, I get a 403 response [19:37:35] bd808: ah not me, just dns, let me look [19:37:40] local.wmftest.net sounds like vagrant? [19:38:21] zhuyifei1999_: yes. paladox was asking about MediaWiki-Vagrant things [19:39:50] nuria_: this is what I see from my laptop -- https://phabricator.wikimedia.org/P5984 [19:40:38] * zhuyifei1999_ gtg [19:41:35] bd808: ok, looks like a permission issue on server [19:46:26] bd808 i forgot how i can make my self an admin on the ldap wiki. I am wondering what is the command again please? [20:20:26] paladox: mwscript createAndPromote.php --wiki=ldapauthwiki --help [20:20:37] thanks [20:54:38] (03CR) 10Merlijn van Deen: [C: 032] Ignore libraryupgrader in grrrit-wm [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/375870 (https://phabricator.wikimedia.org/T174772) (owner: 10Merlijn van Deen) [20:54:41] (03CR) 10Merlijn van Deen: [C: 032] Logging to irc no longer supported -- fall back to stdout [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/375858 (owner: 10Merlijn van Deen) [20:55:14] (03Merged) 10jenkins-bot: Ignore libraryupgrader in grrrit-wm [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/375870 (https://phabricator.wikimedia.org/T174772) (owner: 10Merlijn van Deen) [20:55:23] (03CR) 10jenkins-bot: Ignore libraryupgrader in grrrit-wm [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/375870 (https://phabricator.wikimedia.org/T174772) (owner: 10Merlijn van Deen) [20:55:55] (03Merged) 10jenkins-bot: Logging to irc no longer supported -- fall back to stdout [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/375858 (owner: 10Merlijn van Deen) [20:56:03] (03CR) 10jenkins-bot: Logging to irc no longer supported -- fall back to stdout [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/375858 (owner: 10Merlijn van Deen) [21:50:56] 10cloud-services-team (FY2017-18), 10Datasets-General-or-Unknown, 10Goal: Begin migrating customer-facing Dumps endpoints to Cloud Services - https://phabricator.wikimedia.org/T168486#3598640 (10bd808) [21:56:05] 10Data-Services: Puppetize and setup initial lvms and directory structures for labstore1006|7 - https://phabricator.wikimedia.org/T171539#3598662 (10bd808) [21:56:39] 10Data-Services, 10Datasets-General-or-Unknown: Setup periodic rsync jobs from dataset1001/dumpsdata1001|2 to labstore1006|7 - https://phabricator.wikimedia.org/T171541#3598663 (10bd808) [21:57:17] 10Data-Services: Figure out how NFS failovers will work for the dumps servers - labstore1006|7 - https://phabricator.wikimedia.org/T171540#3598664 (10bd808) [21:57:44] 10Data-Services, 10cloud-services-team (FY2017-18), 10Goal: Program 7 Outcome 3: data services - https://phabricator.wikimedia.org/T166402#3598665 (10bd808) [22:01:28] 10Data-Services, 10cloud-services-team (FY2017-18), 10DBA, 10Goal: Decommission labsdb1001 and labsdb1003 - https://phabricator.wikimedia.org/T142807#3598701 (10bd808) [22:02:34] 10Cloud-VPS, 10cloud-services-team (Kanban), 10Patch-For-Review: Refactor OpenStack Puppet to account for Neutron - https://phabricator.wikimedia.org/T171494#3598703 (10bd808) [22:32:53] 10Cloud-VPS: Grant Bmansurov access to "Recommendation-api" Cloud VPS Project - https://phabricator.wikimedia.org/T175643#3598818 (10bmansurov) [22:33:11] (03Draft1) 10Paladox: Check gerrit2 instead of gerrit for http domain for now [labs/icinga2] - 10https://gerrit.wikimedia.org/r/377360 [22:33:14] (03PS2) 10Paladox: Check gerrit2 instead of gerrit for http domain for now [labs/icinga2] - 10https://gerrit.wikimedia.org/r/377360 [22:33:18] (03CR) 10Paladox: [V: 032 C: 032] Check gerrit2 instead of gerrit for http domain for now [labs/icinga2] - 10https://gerrit.wikimedia.org/r/377360 (owner: 10Paladox) [22:38:10] (03Draft1) 10Paladox: Fix syntax error [labs/icinga2] - 10https://gerrit.wikimedia.org/r/377362 [22:38:12] (03PS2) 10Paladox: Fix syntax error [labs/icinga2] - 10https://gerrit.wikimedia.org/r/377362 [22:38:15] (03CR) 10Paladox: [V: 032 C: 032] Fix syntax error [labs/icinga2] - 10https://gerrit.wikimedia.org/r/377362 (owner: 10Paladox) [22:49:58] (03Draft2) 10Zppix: Update Wikimedia-AI's irc config [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/377364 [23:03:07] 10Cloud-VPS: Grant Bmansurov access to "Recommendation-api" Cloud VPS Project - https://phabricator.wikimedia.org/T175643#3598889 (10DarTar) Baha kindly agreed to help with this, the request is supported on my end if you need an approval from me. [23:05:06] 10Cloud-VPS: Grant Bmansurov access to "Recommendation-api" Cloud VPS Project - https://phabricator.wikimedia.org/T175643#3598893 (10Krenair) adding current admins to process request [23:27:10] 10Cloud-VPS: Grant Bmansurov access to "Recommendation-api" Cloud VPS Project - https://phabricator.wikimedia.org/T175643#3598818 (10bd808) @DarTar you can ask @schana or @ellery add @bmansurov to the project using the form at https://wikitech.wikimedia.org/wiki/Special:NovaProject. They just need to put "recom... [23:27:53] 10Cloud-VPS, 10VPS-Projects, 10Recommendation-API: Grant Bmansurov access to "Recommendation-api" Cloud VPS Project - https://phabricator.wikimedia.org/T175643#3598925 (10bd808)