[00:43:04] <wikibugs>	 (03PS1) 10Arlolra: Notify #mediawiki-parsoid for Parsing-Team project bugs [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425730
[00:50:52] <wikibugs>	 (03CR) 10Arlolra: Notify #mediawiki-parsoid for Parsing-Team project bugs (031 comment) [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425730 (owner: 10Arlolra)
[00:52:20] <wikibugs>	 (03PS2) 10Arlolra: Notify #mediawiki-parsoid for more project bugs [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425730
[01:41:25] <wikibugs>	 (03PS1) 10Legoktm: Send Parsing-Team bugs to the #mediawiki-parsoid channel [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425736
[01:42:00] <wikibugs>	 (03CR) 10Legoktm: [C: 032] Send Parsing-Team bugs to the #mediawiki-parsoid channel [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425736 (owner: 10Legoktm)
[01:42:20] <wikibugs>	 (03Merged) 10jenkins-bot: Send Parsing-Team bugs to the #mediawiki-parsoid channel [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425736 (owner: 10Legoktm)
[01:42:28] <wikibugs>	 (03CR) 10jenkins-bot: Send Parsing-Team bugs to the #mediawiki-parsoid channel [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425736 (owner: 10Legoktm)
[01:45:26] <wikibugs>	 (03PS3) 10Legoktm: Notify #mediawiki-parsoid for more project bugs [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425730 (owner: 10Arlolra)
[01:46:21] <wikibugs>	 (03CR) 10Legoktm: [C: 032] Notify #mediawiki-parsoid for more project bugs [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425730 (owner: 10Arlolra)
[01:46:42] <wikibugs>	 (03Merged) 10jenkins-bot: Notify #mediawiki-parsoid for more project bugs [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425730 (owner: 10Arlolra)
[01:46:49] <wikibugs>	 (03CR) 10jenkins-bot: Notify #mediawiki-parsoid for more project bugs [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/425730 (owner: 10Arlolra)
[01:50:38] <wikibugs>	 !log tools.wikibugs Updated channels.yaml to: 3b1b66271bd97799a723e4e1367c4586f3f9d3e5 Notify #mediawiki-parsoid for more project bugs
[01:50:39] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL
[02:22:36] <paladox>	 !help there seems to be a dns issue on https://gerrit-test.wmflabs.org/gerrit/ and https://gerrit-new.wmflabs.org/r/  (runs on gerrit-test) 
[02:22:36] <wm-bot>	 paladox: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team
[02:22:39] <paladox>	 i am getting
[02:22:40] <paladox>	 Failed to load resource: The network connection was lost.
[02:22:40] <paladox>	 http://gerrit-test.wmflabs.org/gerrit/elements/gr-app.js
[02:24:04] <chicocvenancio>	 paladox: why do think it is a dns issue?
[02:24:15] <paladox>	 chicocvenancio it's happened before
[02:24:23] <paladox>	 keeps happening at random times too.
[02:24:29] <andrewbogott>	 paladox: I think they're working for me
[02:24:36] <paladox>	 andrewbogott oh
[02:24:37] <chicocvenancio>	 I'm getting `net::ERR_SPDY_PROTOCOL_ERROR`
[02:24:43] <andrewbogott>	 at least, https://gerrit-new.wmflabs.org/r/ and https://gerrit-test.wmflabs.org/gerrit/q/status:open seem ok
[02:25:18] <paladox>	 Failed to load resource: The operation couldn’t be completed. (kCFErrorDomainCFNetwork error 303.)
[02:25:47] <paladox>	 i see https://phabricator.wikimedia.org/F16925909
[02:25:51] <paladox>	 andrewbogott chicocvenancio ^^
[02:26:38] <andrewbogott>	 paladox: you can test dns with 'dig' — they should both resolve to 208.80.155.156
[02:26:43] <andrewbogott>	 If they do, then it's not a dns problem :)
[02:27:34] <paladox>	 andrewbogott i see https://phabricator.wikimedia.org/P6984
[02:27:45] <paladox>	 hmm
[02:27:48] <andrewbogott>	 yep, looks right
[02:28:09] * chicocvenancio got the same ip
[02:28:12] <paladox>	 i doint think this is a gerrit problem, otherwise gerrit in prod would be affected.
[02:29:12] <chicocvenancio>	 a few weeks ago I had a weird similar error with PAWS
[02:29:28] <chicocvenancio>	 clearing cookies solved it there, does no good here
[02:29:51] <paladox>	 https://stackoverflow.com/questions/25077284/what-is-kcferrordomaincfnetwork-code-303
[02:31:05] <paladox>	 http://www.edugeek.net/forums/home-access-plus/77702-kcferrordomaincfnetwork-error-303-a.html
[02:31:44] <chicocvenancio>	 paladox: what browser are you using?
[02:31:56] <paladox>	 chicocvenancio safari + chrome
[02:32:55] <chicocvenancio>	 things worked for me in firefox and edge
[02:33:02] <chicocvenancio>	 did not work on Chrome
[02:33:19] <paladox>	 oh
[02:54:16] <paladox>	 works now
[07:20:48] <wikibugs>	 (03CR) 10Merlijn van Deen: [C: 04-1] "I don't understand how the proposed change would accomplish that." [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/424965 (owner: 10Paladox)
[08:25:38] <doctaxon>	 Hi! Has anything been changed with API:Logevents yesterday?
[08:26:17] <doctaxon>	 /w/api.php?action=query&format=json&list=logevents&titles=&lelimit=3
[08:26:46] <doctaxon>	 seems to be new or changed -> 
[08:26:47] <doctaxon>	 "params": {
[08:26:47] <doctaxon>	                     "0": 176404259,
[08:26:47] <doctaxon>	                     "1": 171268042,
[08:26:47] <doctaxon>	                     "2": "20180412082451"
[08:26:49] <doctaxon>	                 },
[08:27:14] <doctaxon>	 andrewbogott , chasemp: ^ do you know something about it
[08:30:47] <doctaxon>	 annika: Do you know anything about it?
[08:39:38] <jynus>	 doctaxon: cloud may not be right channel to ask, for production API I would ask on #wikimedia-tech
[08:39:58] <jynus>	 however, I may have heard something about logging changes
[08:40:32] <eddiegp>	 doctaxon: On which wiki did you observe the change?
[08:40:37] <jynus>	 See if this could have something to do with it https://phabricator.wikimedia.org/T184485
[08:40:56] <jynus>	 (no idea on the effect on api, though)
[08:42:30] <doctaxon>	 in "params" has been "curid" and "oldid" before, but not "0" and "1" and "2"
[08:55:47] <jynus>	 "Store the fact that it was autopatrolled in the rc_patrolled field instead (currently boolean tinyint, could add a third state for autopatrolled)" that would fit with patrolled/unpatrolled/autopatrolled
[09:16:48] <eddiegp>	 Amir1: yt?
[09:17:10] <eddiegp>	 Was talking about the above issue in -tech right now.
[09:17:16] <Amir1>	 eddiegp: what's up, in a meeting for the next six hours
[09:17:40] <eddiegp>	 Amir1: Have you tested "Stop logging autopatrol actions everywhere" on FlaggedRevs wikis?
[09:18:12] <eddiegp>	 What doctaxon reported above seems related as far as I can tell.
[09:18:36] <Amir1>	 eddiegp: I have tested on Persian Wikipedia
[09:18:48] <Amir1>	 it's partially flaggedRev
[09:19:06] <Amir1>	 I will take a look ASAP
[09:19:39] <eddiegp>	 Problem exists on enwiki and dewiki (both flaggedrevs) but not frwiki (no flaggedrevs). And it seems to only appear for the 'review' log type (which comes from flaggedrevs).
[09:19:46] <eddiegp>	 Thanks!
[09:21:14] <Amir1>	 eddiegp: is there a phabricator card?
[09:21:24] <eddiegp>	 Not yet, should I create one?
[09:23:17] <Amir1>	 please
[09:23:25] <eddiegp>	 Okay, will do.
[09:28:40] <doctaxon>	 eddiegp: please link the phab card here
[09:34:49] <eddiegp>	 Amir1: doctaxon: https://phabricator.wikimedia.org/T192063
[09:45:57] <Amir1>	 eddiegp: It's very unlikely that they are related, I investigate it regardless but the config change only stops adding new log entries 
[09:50:49] <eddiegp>	 I can't find anything backported to enwiki/dewiki yesterday though, and wmf-config also doesn't have anything for these wikis other than your change yesterday. The train hasn't reached these wikis yet, so that's also not it.
[09:53:06] <eddiegp>	 That is, if it's not your change, I don't know what else it'd be.
[10:22:54] <doctaxon>	 eddiegp: it is since 23:09 UTC
[10:31:48] <eddiegp>	 From SAL: "23:09 der.eckson@tin: Synchronized wmf-config/InitialiseSettings.php: Stop logging autopatrol actions everywhere (T184485) (duration: 01m 18s)"
[10:31:49] <stashbot>	 T184485: Stop logging autopatrol actions - https://phabricator.wikimedia.org/T184485
[10:32:16] <eddiegp>	 doctaxon: How did you check that it started exactly at that time?
[10:32:56] <doctaxon>	 I'm storing each logevents in a list
[10:33:11] <doctaxon>	 23:09 was the last incoming
[10:33:56] <doctaxon>	 the last incoming with param "curid"
[10:34:43] <eddiegp>	 Can you paste that one (into a pastebin and link here)?
[10:36:28] <doctaxon>	 pageid 9479005 ns 0 title {Bahnstrecke Opole–Wrocław} revid 176394554 parentid 176394478 mino
[10:36:31] <doctaxon>	 r {} user Altsprachenfreund timestamp 2018-04-11T23:09:47Z comment {/* Verlauf und Zustand */
[10:36:34] <doctaxon>	 }
[10:41:34] <doctaxon>	 eddiegp: I'm checking it on: Maybe there is something wrong with the lecontinue. I will come back with results.
[10:44:13] <eddiegp>	 doctaxon: https://de.wikipedia.org/wiki/Spezial:ApiSandbox#action=query&format=json&list=logevents&lestart=2018-04-11T23%3A09%3A00.000Z&leuser=Altsprachenfreund&lelimit=2
[10:44:33] <eddiegp>	 These are the events for the last edit you got.
[10:45:20] <eddiegp>	 There's the patrol event, which has your 'curid', 'previd' and 'auto' fields, and then there's the 'review' field, which has '0', '1', '2'.
[10:46:56] <eddiegp>	 So Zaches comment on the task seems accurate, so what happened might be: You did use the patrol event, until that one vanished. The review event didn't change at all, but your tool never looked at it.
[10:48:02] <doctaxon>	 and there's the last log from user Altsprachenfreund today: no "curid" any more, do you see the difference?
[10:48:11] <doctaxon>	 https://de.wikipedia.org/wiki/Spezial:ApiSandbox#action=query&format=json&list=logevents&leuser=Altsprachenfreund&lelimit=2
[10:50:36] <doctaxon>	 eddiegp:  ^
[10:51:20] <eddiegp>	 What I linked were two log entries, for a single edit, the one log entry of type patrol, the other of type review. What you linked are two log entries for two edits, both log entries of type review
[10:51:59] <doctaxon>	 but Altsprachenfreund has edited this both 
[10:52:09] <eddiegp>	 What I'm saying is: The review log entry did not change from curid/oldid to 0/1. The review log entry already was 0/1 before.
[10:53:25] <eddiegp>	 The thing that you saw before, having the curid/oldid, was the log entry of type patrol.
[10:53:56] <doctaxon>	 if Altsprachenfreund edits there have to be patrols, too, but there is not one patrol
[10:54:00] <doctaxon>	 it
[10:54:06] <doctaxon>	 is autopatrol
[10:54:27] <eddiegp>	 Right. That was the change that was introduced yesterday: There no longer is a logentry for autopatrol.
[10:54:28] <doctaxon>	 the last edits are not logged as patrolled
[10:55:12] <doctaxon>	 okay
[10:55:55] <doctaxon>	 eddiegp: where can I get the log of the patrols now?
[10:56:59] <eddiegp>	 Either from the RC feed (I'd have to read the task for details where exactly in there you'd get it), or by using the 'review' event instead.
[11:02:25] <eddiegp>	 doctaxon: Is your code public by any chance?
[15:02:09] <sebastian-wmse>	 !log wikispeech Migrated wiki to an instance running a newer Linux version (T191758).
[15:02:11] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikispeech/SAL
[15:02:11] <stashbot>	 T191758: Migrate to new Wikispeech wiki server - https://phabricator.wikimedia.org/T191758
[21:06:42] <mutante>	 how do you get a list of "All of the tools-bastion-*, tools-exec-*, and tools-webgrid-* instances should"
[21:06:54] <mutante>	 like the actual numbers where this says -*
[21:10:36] <bd808>	 mutante: https://tools.wmflabs.org/openstack-browser/project/tools
[21:11:06] <mutante>	 bd808: ok! cool, i'll merge that later today
[21:11:24] <bd808>	 mutante: I think that if the role change works on tools-bastion-02.tools.eqiad.wmflabs and tools-exec-1401.tools.eqiad.wmflabs it should work everywhere
[22:24:07] <mutante>	 Notice: /Stage[main]/Toollabs::Exec_environ/Package[language-pack-mr]/ensure: created
[22:25:01] <mutante>	 --- /var/lib/locales/supported.d/local	2015-04-22 18:32:38.847090958 +0000
[22:25:11] <mutante>	 Scheduling refresh of Exec[locale-gen]
[22:25:25] <paladox>	 hmm
[22:25:44] <mutante>	 that's normal. after adding a new language package
[22:25:49] <mutante>	 you have to regenerate the locales
[22:25:51] <mutante>	 and puppet does it
[22:25:55] <paladox>	 oh
[22:25:58] <mutante>	 it's still running
[22:26:32] <mutante>	 +mr_IN.UTF-8 UTF-8
[22:27:33] <mutante>	 bd808: ^ done. tools-exec-1401 has it applied and no issues
[22:27:58] <mutante>	 it ran "locale-gen" and that should be all
[22:28:06] <bd808>	 mutante: nice!
[22:28:29] <mutante>	 it just takes a little while to generate all the locales.. but finished fine
[22:28:52] <bd808>	 yeah, locale-gen can take a while for sure
[22:29:17] <mutante>	 also done on tolls-bastion-02
[22:29:20] <mutante>	 tools
[22:29:30] <mutante>	 i'll let the rest just happen 
[22:31:10] <bd808>	 I'll keep my eye on the puppet failure feed for a while just in case
[22:31:41] <mutante>	 cool
[23:01:05] <awight>	 !log deployment-prep Try gerrit-based submodules for ORES, T180627
[23:01:08] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL
[23:01:08] <stashbot>	 T180627: [Blocked] Support git-lfs - https://phabricator.wikimedia.org/T180627