[03:08:13] Nemo_bis: I think that was maybe related to some recent Anomie work? [08:42:30] Ivy: yes, I was thinking of https://lists.wikimedia.org/pipermail/wikitech-l/2017-November/089190.html too. But I had not understood it would have ruined our pretty domain names as well. :) [10:27:04] Hi, can I ask here which is the correct Wikipedia SSL certificate - by GlobalSign, or by DigiCert? [10:48:45] vmarsik: why do you need to know that? However, it seems, both, a GlobalSign signed certificate, as well as a certificate signed by DigiCert are valid and may be used. [10:49:34] FlorianSW: got lots of CertPartrol warnings, could not find out if both are correct or if there is a MITM SSL attack somewhere [10:49:52] thanks [10:50:56] vmarsik: it seems to depend on which cluster your browser connects to [10:51:51] I see, just could not find anywhere if they are correct or not [15:09:17] Nemo_bis: Huh, looks like we may have support for longer edit summaries by the end of this month. [15:09:38] https://phabricator.wikimedia.org/T174569 is moving along. [15:22:00] Finally people will be able to use twice as many snarky words in their ostensibly constructive messages. [15:35:54] /c/c [17:24:07] Niharika: ping [17:24:26] Hey Hauskatze. What's up? [17:24:51] Niharika: we've got a stuck global rename, jobs need to be re-enqueued; up to run a script to do that? [17:24:58] there's documentation avalaible [17:25:42] T184263 in case you say 'yes'; otherwise I can understand [17:25:46] T184263: Global rename failure on account Mega809 - https://phabricator.wikimedia.org/T184263 [17:25:46] Hauskatze: Can you point me to the docs? It's a Friday and normally we don't do any deploys/scripts on Fridays so, how urgent is this? [17:26:25] well, I'm not sure I can assess the urgency; the account is locked out until the rename is finished though [17:26:45] didn't know scripts were not run on friday either [17:27:55] I haven't ever seen anyone run one so not sure. [17:28:03] Let me look at the script in question. [17:28:21] I see MarcoAurelio commented on the ticket. [17:28:47] yep [17:29:26] Niharika: I am that person fyi :) [17:32:27] Ah! I see. The script seems harmless but I see Marco asked to check logs first. I'd honestly let someone more familiar with the script do it because I sure won't be able to make much sense of what happened. I've not messed with renames before. [17:32:40] Maybe legoktm can help. [17:33:16] Niharika: legoktm , hoo or tgr|away come to my mind, yep [17:33:35] I'm more familiar with renames as I perform them on-wiki [17:33:56] but I don't have access to db logs so I'm not sure what made that rename fail [17:37:52] Niharika: the docs are at https://wikitech.wikimedia.org/wiki/Stuck_global_renames [17:39:56] I linked that on the ticket [17:41:36] tgr: Thanks. Looks straightforward. I can run that script now, Hauskatze. Gimme a moment to look up the logs. [17:41:37] the script just creates a LocalRenameUserJob the same way the special page would, so it's fairly safe [17:42:07] except maybe the ignorestatus option which could cause a mess if the rename job is still running [17:42:13] tgr: well, I'd run showJobs.php first [17:42:26] that works, yeah [17:42:31] no, no --ignorestatus here please [17:43:05] but normally you shouldn't need it anyway, --ignorestatus was added back when the status wasn't really reliable, that should not be an issue today [17:43:26] I wish I had a little window to see the db logs to understand this kind of things better [17:45:11] Hauskatze: you can sign a volunteer NDA to get access to logstash [17:45:30] tgr: I think it's more complicated than that [17:45:40] https://wikitech.wikimedia.org/wiki/Volunteer_NDA [17:45:42] There's nothing in logstash. [17:45:57] Do logs get scrubbed periodically? [17:46:30] they don't live forever, but long enough, something like 30 or 90 days [17:46:47] tgr: comment in support from employee, its manager, sign off by Victoria, Cobblestone signature and L2 as well [17:47:01] too boring :P [17:47:01] The docs mention two channels but neither of them have anything regarding this particular rename. [17:47:34] Hauskatze: It's fun. :P [17:48:12] Maybe I'm just bad at searching logstash. [17:48:52] I mean, I have access to huge ammounts of private data IRL because of my job and I am under various NDAs myself and I can certainly understand, and support strict rules in that regard [17:52:12] Hauskatze: well, file a task, explain that you are a volunteer developer who deals with production bugs and could do it better with LDAP nda access, and drop a link to some channel with WMF devs on it [17:52:43] I can support it but I'm sure a number of other people would too [17:53:30] anyway, don't you already have an NDA due to being a steward? [17:54:19] I have a question about showJobs.php. That script only shows the number of jobs running but is there a definite way to find out is the rename job for this username is still running? [17:54:59] "Before running the script, verify if the jobs are running. This is important. That can be done using showJobs.php. If jobs aren't running, you can proceed with:" [17:55:20] Niharika: mwscript showJobs.php --wiki=eswiki --list [17:55:32] I think it was --list [17:55:44] Jobs are clearly running as that script output keeps changing. The docs say if jobs are not running, I'm to follow the command below. What is jobs *are* running? [17:56:03] I guess LocalRenameJobs [17:56:23] tgr: yes, of course; and as OTRS former agent and OTRS admin [17:56:43] there are two job classes used in rename [17:57:46] https://doc.wikimedia.org/mediawiki-core/master/php/showJobs_8php_source.html <-- list of -- [17:58:15] tgr: I'm confused over the language. When it says "If jobs aren't running,..." does it mean the specific rename job isn't running or any job isn't? [17:58:31] This specific rename job is in abandoned status. [17:59:04] Niharika: yeah, some kind of job is always running (a few dozen, more likely) [17:59:13] you can 1) check the rename status [17:59:36] https://es.wikipedia.org/wiki/Especial:GlobalRenameProgress/Mega809 tells you that [18:00:06] it shows that the job has failed (ie. GlobalRename thinks it's not running anymore) [18:00:19] Right. [18:00:50] 2) check the actual job queue with showJobs --type [18:01:23] mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=eswiki --logwiki=metawiki "Mega849" "Mega809" would be the command if everything is fine to go [18:01:48] and scap log that? [18:01:57] or just !_log it manually [18:02:23] Yeah, looks like everything is good to go. [18:02:32] yeah, just say !log on the ops channel [18:02:59] And it failed. [18:03:05] https://www.irccloud.com/pastebin/EAunugkM/ [18:03:47] weird [18:04:13] tgr: Does ^ mean the job actually succeeded but it thinks it failed? [18:05:04] global rename status still displays "failure" so I don't think it's running [18:07:34] Niharika: so Special:GlobalRenameUser schedules one LocalRenameUserJob per wiki, and that job schedules a bunch of RenameUserJob jobs [18:07:49] one per DB table where rows must be renamed, more or less [18:08:06] Okay. [18:08:08] I'll add that to the docs [18:08:23] tgr: we've got a conflict; the user was created at eswiki but not renamed [18:08:36] I can locally usurp that so you can try again [18:09:23] registration data of https://es.wikipedia.org/wiki/Especial:Registro/Mega809 == Mega849 [18:09:32] so the error means either the job did not actually fail but global rename ended up with a fail status somehow, or it failed halfway with some DB rows already renamed [18:09:45] so it kind of started doing something but got halted [18:09:54] in that case I think you have to fix things by hand [18:10:12] I can move that account locally [18:10:12] I'll look at it after I wake up properly :) [18:10:20] and then we can try again [18:10:39] Thanks tgr. [18:11:35] that could misplace some edits if they were moved already (although that would be a first for such an error I think), let me look at the B first [18:13:39] Niharika: btw I usually just search for the username in logstash, unless they have some common word as the name that works pretty well [18:13:49] cannot see suppressed or deleted edits, nor visible edits for Mega409@eswiki; but sure [18:13:58] tgr: I did but nothing turned up on it. [18:16:59] there are log entries from Jan 2/3, maybe you didn't set the time window large enough [18:17:24] they are the same error though (user already exists) so not really helpful [18:19:55] Oh. [18:21:12] 99% of the time what happens is that a DB transaction takes long due to high load or whatever, the DB load balancer kills it, and since that's a completely generic error there is no way to link it to the job which had its transaction killed and possibly ended up in an inconsistent state [18:25:54] tgr: is local usurpation of the conflicting user an option here? [18:26:01] I can do that real quick [18:26:10] no edits whatsoever visible to me [18:26:28] not sure, is it urgent? [18:26:48] no, I don't think so [19:22:50] in comparison to the previous 2-3 hours... [19:22:55] oh, wrong chat [19:48:14] I made a few changes on translatewiki which does not seem to propagate to nowiki. [19:48:18] https://translatewiki.net/wiki/MediaWiki:Whatlinkshere/nb [19:48:39] Something that isn't restarted after the holiday? [19:50:11] https://github.com/wikimedia/mediawiki/commits/master/languages/i18n/nb.json [19:50:20] iMy fault, it has propagated… [19:50:40] * Reedy slaps jeblad [19:50:51] * jeblad kicks his own but [22:24:09] tgr: just usurped that account as suggested, can we try the script again? [22:27:18] Hauskatze: seems to have worked [22:28:51] tgr: yep, seems it's running :D [23:05:29] what the &action=parsermigration-edit is suposed to do? [23:05:47] 'cause every time I use it... it apparently does nothing [23:06:59] https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy [23:07:30] I think that might be "a link called "Edit with migration tool" is added to the toolbox of all articles, which can show the current (Tidy) and expected (RemexHTML) output side-by-side, and can preview article text changes in the same side-by-side view."