[00:09:03] <grrrit-wm>	 (03CR) 10Gergő Tisza: "Some comments from Pau:" (032 comments) [extensions/MultimediaViewer] - 10https://gerrit.wikimedia.org/r/185670 (https://phabricator.wikimedia.org/T77726) (owner: 10Unicodesnowman)
[02:20:34] <grrrit-wm>	 (03PS1) 10Gergő Tisza: Fix up HookHandler documentation [extensions/CommonsMetadata] - 10https://gerrit.wikimedia.org/r/188501 
[02:20:36] <grrrit-wm>	 (03PS1) 10Gergő Tisza: Use long cache TTL for shared DB files [extensions/CommonsMetadata] - 10https://gerrit.wikimedia.org/r/188502 (https://phabricator.wikimedia.org/T86955) 
[02:21:00] <grrrit-wm>	 (03CR) 10Gergő Tisza: [C: 04-1] "Not tested yet." [extensions/CommonsMetadata] - 10https://gerrit.wikimedia.org/r/188502 (https://phabricator.wikimedia.org/T86955) (owner: 10Gergő Tisza)
[04:55:45] <wikibugs>	 3Multimedia, MediaWiki-File-management: "thumb.php" should not throw fatal MWException when missing query parameters - https://phabricator.wikimedia.org/T88508#1013635 (10Krinkle) 3NEW
[07:24:18] <wikibugs>	 3Multimedia, MediaWiki-File-management: bitdepth of UploadStashFile is wrong - https://phabricator.wikimedia.org/T36952#1013820 (10polybuildr) @Tgr, uploaded a new patch set with an updated commit message that mentions the need for some larger restructuring. Please do review it and let me know if it's okay as a...
[10:26:12] <wikibugs>	 3Multimedia, MediaWiki-File-management: [InstantCommons] Some hotlinks of Commons files are not protocol-relative - https://phabricator.wikimedia.org/T50133#1014050 (10jayvdb) I cant reproduce this on http(s)://www.omegawiki.org/DefinedMeaning:pink_%286772%29  However it might only occur if the image doesnt need...
[14:14:21] <rillke>	 Upload Wizard seems to be broken.
[14:14:27] <rillke>	 https://commons.wikimedia.org/wiki/Commons:Upload_help#Upload_wizard_-_the_NEXT_button_is_missing
[14:17:09] <rillke>	 But only in non-debug mode.
[14:17:36] <rillke>	 "TypeError: wizard.steps.file.maybeStartProgressBar is not a function"
[14:17:54] <rillke>	 With ?debug=true, everything works just fine
[15:22:48] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: UploadWizard: File name is not allowed - https://phabricator.wikimedia.org/T87062#1014787 (10Herzi.Pinki) Upload through UW worked for me this morning on FF, but now (after 14 UTC) no upload happens again. The behavior is different: I get the ''share from Flickr''...
[15:26:34] <marktraceur>	 (for the record we're handling this in -commons and -operations)
[15:59:40] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: UploadWizard: File name is not allowed - https://phabricator.wikimedia.org/T87062#1015012 (10Tgr) > I get the ''share from Flickr'' button.   Can you clarify what you mean by that?  Do you think it is still the same bug, or something unrelated?
[16:00:17] <guillom>	 FYI: https://commons.wikimedia.org/wiki/Commons:Village_pump#Upload_wizard_not_working
[16:00:30] <guillom>	 Ah never mind :)
[16:23:06] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: UploadWizard: File name is not allowed - https://phabricator.wikimedia.org/T87062#1015104 (10Herzi.Pinki)  > Do you think it is still the same bug, or something unrelated?  Could be unrelated, but I don't know. Do you need more diagnostic data?
[16:23:56] <marktraceur>	 Thanks guillo
[16:23:59] <marktraceur>	 m.
[17:11:54] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: UploadWizard broken - https://phabricator.wikimedia.org/T88576#1015180 (10Steinsplitter)
[17:14:33] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: UploadWizard broken - https://phabricator.wikimedia.org/T88576#1015191 (10Smuconlaw) Here's what my Firefox web console reports:  "Storage error" load.php:150 "[Exception... "Out of Memory"  nsresult: "0x8007000e (NS_ERROR_OUT_OF_MEMORY)"  location: "JS frame :: ht...
[17:50:02] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: UploadWizard broken - https://phabricator.wikimedia.org/T88576#1015327 (10RobLa-WMF) a:3MarkTraceur Mark made us aware of this in our status meeting this morning.  He's on it; he'll likely need some help debugging this, since ResourceLoader may be involved.
[18:12:14] <tgr>	 marktraceur: the UW issue sounds like a problem with localStorage-based caching in ResourceLoader
[18:13:16] <marktraceur>	 I wouldn't be surprised
[18:13:20] <marktraceur>	 But I cleared my localStorage
[18:13:23] <marktraceur>	 I guess I'll do it again
[18:13:55] <tgr>	 wmf15 was just reverted due to performance regressions
[18:14:13] <tgr>	 maybe that messed up the RL cache somehow?
[18:14:13] <marktraceur>	 Yup
[18:14:16] <marktraceur>	 Well
[18:14:16] <marktraceur>	 Maybe.
[18:14:46] <marktraceur>	 But the real question is, how?
[18:15:33] <tgr>	 do we know when the error started?
[18:15:56] <tgr>	 we can probably look at the funnel logs
[18:16:08] <marktraceur>	 About 14:14 UTC today, but it may have been earlier
[18:16:12] <marktraceur>	 That was just the first report I saw
[18:16:20] <marktraceur>	 Oh.
[18:16:29] <marktraceur>	 tgr: Actually the report on-wiki is from 13:18
[18:16:38] <marktraceur>	 https://commons.wikimedia.org/wiki/Commons:Upload_help#Upload_wizard_-_the_NEXT_button_is_missing
[18:18:34] <tgr>	 the rollback was around 10 AM UTC
[18:22:03] <robla>	 I was just about to make the same speculation (about wmf15->14 being the UW problem cause)
[18:22:23] <robla>	 seems pretty likely
[18:22:45] <marktraceur>	 Well, we didn't see any issues with wmf14 when it was out on commons...right?
[18:22:54] <robla>	 I'm going to bet that some cache somewhere got corrupted
[18:22:54] <tgr>	 it definitely is, there is a clear drop in the number of people reaching the second step of UW, starting the exact same hour
[18:23:19] <marktraceur>	 That seems pretty self-evident at this point, but usually touching and syncing is enough to fix the RL cache
[18:23:52] <robla>	 doesn't RL rely on manually rolled version numbers?
[18:23:59] <Keegan>	 13:20 is the first report on Commons
[18:24:52] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: UploadWizard: File name is not allowed - https://phabricator.wikimedia.org/T87062#1015431 (10Tgr) 5Open>3Resolved Lack of next button is T88576 which is unrelated. Closing this as fixed.
[18:25:20] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: Next button is missing in UploadWizard after revert to wmf15 - https://phabricator.wikimedia.org/T88576#1015433 (10Tgr)
[18:26:04] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: Next button is missing in UploadWizard after revert to 1.25wmf14 - https://phabricator.wikimedia.org/T88576#1015434 (10RobLa-WMF)
[18:27:22] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: Next button is missing in UploadWizard after revert to 1.25wmf14 - https://phabricator.wikimedia.org/T88576#1015437 (10Tgr) The number of people getting through the file select step starts dropping at 10 AM UTC: ``` mysql:research@analytics-store.eqiad.wmnet [log]>...
[18:27:52] <marktraceur>	 robla, Keegan, tgr, they say it's working now
[18:28:01] <marktraceur>	 Maybe my huge touch-everything worked
[18:28:05] <robla>	 huzzah!
[18:28:16] <Keegan>	 Percussive maintenance ;)
[18:28:19] <robla>	 "jiggle the handle"
[18:28:24] <Keegan>	 good show
[18:29:05] <marktraceur>	 Gonna go resolve some bugs and definitely not be snarky about people blaming us for it
[18:30:42] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: Next button is missing in UploadWizard after revert to 1.25wmf14 - https://phabricator.wikimedia.org/T88576#1015445 (10MarkTraceur) 5Open>3Resolved Seems to be resolved on Commons now according to Dragonfly67, but keep an eye out for further troubles.
[18:30:55] <tgr>	 there is a fair chance of all other JS stuff being broken as well, then
[18:31:09] <tgr>	 marktraceur: can you document on Phabricator how you fixed it?
[18:31:19] <marktraceur>	 uh, well, if it was me, then sure
[18:31:58] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: Next button is missing in UploadWizard after revert to 1.25wmf14 - https://phabricator.wikimedia.org/T88576#1015447 (10MarkTraceur) For the record, I touched every JS file in UploadWizard, and the UploadWizardHooks.php file, and then I synced the whole UW directory...
[18:32:14] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: Next button is missing in UploadWizard after revert to 1.25wmf15 - https://phabricator.wikimedia.org/T88576#1015448 (10Tgr)
[18:34:56] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: Next button is missing in UploadWizard after revert to 1.25wmf14 - https://phabricator.wikimedia.org/T88576#1015461 (10Tgr)
[18:35:22] <marktraceur>	 18:34, 4 February 2015‎ MarkTraceur (talk | contribs)‎ . . (154,556 bytes) (+413)‎ . . (→‎Upload wizard not working: yes it is)
[18:37:24] <marktraceur>	 So...around 5 hours of outage? Sad times.
[18:45:20] <wikibugs>	 3Multimedia, MediaWiki-extensions-UploadWizard: Next button is missing in UploadWizard after revert to 1.25wmf14 - https://phabricator.wikimedia.org/T88576#1015478 (10Smuconlaw) After a few false starts (file selection window did not show thumbnails; selecting a file had no effect at all; Firefox crashed) and pu...
[19:20:50] <wikibugs>	 3Multimedia, MediaWiki-File-management: "thumb.php" should not throw fatal MWException when missing query parameters - https://phabricator.wikimedia.org/T88508#1015641 (10brion) Presumably this should throw an HTTP 400 error with some sensible text instead?
[19:27:35] <wikibugs>	 3Multimedia, MediaWiki-File-management: "thumb.php" should not throw fatal MWException when missing query parameters - https://phabricator.wikimedia.org/T88508#1015658 (10MZMcBride)
[20:30:50] <wikibugs>	 3Multimedia, Analytics: Track image context and pass information onto X-Analytics - https://phabricator.wikimedia.org/T85922#1015847 (10Tgr) So what exactly should be done here? # add a query parameter (say `&x-analytics=fc=mv`?) to all MediaViewer file requests - done in T77882 # add a query parameter (`&x-anal...
[23:10:44] <wikibugs>	 3Multimedia, operations: Errors when generating thumbnails should result in HTTP 400, not HTTP 500 - https://phabricator.wikimedia.org/T88412#1016305 (10Tgr)
[23:13:41] <wikibugs>	 3Multimedia, operations: Errors when generating thumbnails should result in HTTP 400, not HTTP 500 - https://phabricator.wikimedia.org/T88412#1016321 (10Tgr) See also T75935 and T74328.
[23:40:37] <wikibugs>	 3Multimedia, operations: Errors when generating thumbnails should result in HTTP 400, not HTTP 500 - https://phabricator.wikimedia.org/T88412#1016386 (10Tgr) > I'm sure this is known already but it'd seem more logical for such errors to return a 400 to the client since it isn't strictly the server's fault, what...