[03:48:17] [[Tech]]; SecretName101; /* Tools for compiling list of Wikimedia Commons users by contributions/uploads */ new section; https://meta.wikimedia.org/w/index.php?diff=20415450&oldid=20414795&rcid=16196086 [05:21:10] any idea why I'd be getting 504 connection timed out on a file upload POST request to api.php? [20:52:15] I'm trying to debug an issue where I'm submitting a POST request to api.php and getting back 504 Connection Timed Out. it seems to happen for a certain file and not others. any idea what might be going on and how to figure it out? [20:52:46] this is being generated from ia-upload, which is on toolforge, but I can reproduce it locally [20:57:35] which file is it? [20:58:54] is it only right now? [20:59:07] there seem to be a few routing issues at this moment [20:59:27] it was reported by an ia-upload user a couple days ago [20:59:30] although I don't think those would affect the wiki [20:59:38] hmm [20:59:47] which query is it? [20:59:53] the file is https://archive.org/download/southerninida00fepe/southerninida00fepe.pdf [20:59:58] I'm trying on beta commons at present [21:00:06] one sec [21:01:33] ok, so it's this request https://github.com/addwiki/mediawiki-api/blob/master/src/Service/FileUploader.php#L118 [21:01:46] which is called from https://github.com/wikisource/ia-upload/blob/master/src/ApiClient/CommonsClient.php#L101 [21:02:00] I'm working on running it outside this code [21:02:22] which filename were you using? [21:02:31] was it at commons? [21:02:53] filename doesn't really matter, it's the source file that causes the issue [21:03:08] they uploaded it to commons already manually (not using ia-upload) [21:10:16] I've been trying nonsense filenames on beta commons. they don't exist since the uploads haven't succeeded [21:10:52] if it also fails on betacommons that's good [21:10:58] easier to pinpoint [21:11:00] right [21:11:45] in fact, I think I had access to beta servers [21:20:16] I do [21:20:28] but I'm not sure which is the right machine [21:21:11] ok, well I can try it now and tell you the filename [21:21:51] will that help? [21:22:21] it could [21:22:45] although for a later step :/ [21:24:10] ok :) [21:27:17] it was served by a machine deployment‐mediawiki‐07 [21:27:22] which is not in the project list... [21:30:55] let me know if/when you want me to try the upload again [21:33:36] Platonides, deployment-mediawiki-07.deployment-prep.eqiad.wmflabs is on the list of instances for deployment-prep... [21:33:55] krenair@deployment-mediawiki-07:~$ [21:38:38] I got there [21:38:55] but I don't see that on https://openstack-browser.toolforge.org/project/deployment-prep [21:39:07] wait [21:39:10] I know see it [21:39:36] and why didn't the browser search find it? [21:40:07] anyway what's the problem in beta exactly? maybe I can help [21:40:42] it's not in beta [21:40:52] ningu has problems uploading a certain file [21:40:59] it fails in pro and also in beta [21:41:11] I hoped that I might be able to find something in the logs [21:41:39] yeah, it's easier to test on beta and not mess anything up on main commons [21:43:48] and the php error log seems to be sent to deployment-fluorine07 [21:45:42] which them seems to forward to deployment-logstash0 [21:45:44] 3.deployment-prep.eqiad.wmflabs [21:46:26] so http://logstash-beta.wmflabs.org/ [21:47:44] Elasticsearch is not working? :/ [21:49:42] well, can you at least try uploading at beta, ningu ? [21:49:48] will do [21:50:40] just uploaded Southerninida00fepe2.pdf and got the 504 response [21:51:08] it does chunked upload and the error occurs in sending the first (in this case only) chunk [21:52:55] I see the acce log entry, but no error [21:52:58] 172.16.1.181proxy:fcgi://127.0.0.1:8000/20099POSThttp://commons.wikimedia.beta.wmflabs.org/w/api.php-application/json [21:53:09] 37096441172.16.1.181proxy:fcgi://127.0.0.1:8000/200169POSThttp://commons.wikimedia.beta.wmflabs.org/w/api.php-application/json [21:57:38] hrm, ok [22:05:25] any way we can debug this further?