[11:11:12] does anyone know if got.wikipedia.org is compatible with utf8mb4? i'm having certain mysql errors. [11:11:32] _mysql_exceptions.OperationalError: (1267, "Illegal mix of collations (utf8mb4_general_ci,EXPLICIT) and (utf8_general_ci,COERCIBLE) for operation 'like'") [11:11:58] for the query = 'SELECT page_id, page_title FROM page WHERE page_namespace=0 AND page_is_redirect=0 AND CONVERT(page_title USING utf8mb4) COLLATE utf8mb4_general_ci LIKE '+'"%'+keyword+'%"'+'ORDER BY page_id'; [11:12:26] in the query i want to compare strings with case insensitive, this is why i do those convert-collate. [11:12:41] but this got text kills it [11:16:19] mmecor: from the wiki replicas? [11:16:24] yes [11:17:08] I'm unsure what the actual data there is encoded as [11:17:28] i thought utf-8 could handle anything [11:18:26] Well, it can, but you still must know and declare encodings [11:19:19] Can you create the full query in quarry? [11:19:29] I'm on mobile at the moment, it's harder to understand the query [11:19:50] ok [11:20:20] i tell u as soon as i have it [11:23:46] chicocvenancio: https://quarry.wmflabs.org/query/25381 [11:25:09] don't understand much where the sql problem is, because it works perfectly with the rest of langs [11:26:54] argh, it gets cut [11:27:07] i'm not sure quarry is compatible with got alphabet [11:27:21] Huh, it runs in quarry [11:27:28] https://quarry.wmflabs.org/query/25382 [11:28:40] https://quarry.wmflabs.org/query/25381 [11:28:50] i had to change of browser [11:29:31] it didn't send the latter part of the query (you'll see that it was cut in LIKE) [11:30:01] Order by page_id? [11:30:15] SELECT page_id, page_title FROM page WHERE page_namespace=0 AND page_is_redirect=0 AND CONVERT(page_title USING utf8mb4) COLLATE utf8mb4_general_ci LIKE "%𐌲𐌿𐍄𐍂𐌰𐌶𐌳𐌰%"ORDER BY page_id [11:30:33] here is the query. i can't submit it [11:30:49] SELECT page_id, page_title FROM page WHERE page_namespace=0 AND page_is_redirect=0 AND CONVERT(page_title USING utf8mb4) COLLATE utf8mb4_general_ci LIKE "%𐌲𐌿𐍄𐍂𐌰𐌶𐌳𐌰%" ORDER BY page_id [11:35:30] What is that keyword? [11:37:06] 𐌲𐌿𐍄𐍂𐌰𐌶𐌳𐌰 [11:37:36] which means gothic language [11:38:34] Sure, but they are valid unicode? [11:39:14] i have no idea [11:39:44] As in https://en.wikipedia.org/wiki/Gothic_(Unicode_block) [11:41:09] i retrieved it from wikidata to create a file with native language names, so it must be [11:41:50] yes, it makes sense with this article bout gothic unicode block [11:42:55] Yeah, it seems to be [11:42:56] https://www.irccloud.com/pastebin/is6Jskrk [11:44:53] yes [11:47:14] this works in quarry https://www.irccloud.com/pastebin/bKbLIcqI [11:47:28] But it fails when I add the keyword [11:51:32] yes, same for me [11:53:43] the same happens when i take out the collade-convert part [11:53:44] SELECT page_id, page_title FROM page WHERE page_namespace=0 AND page_is_redirect=0 AND page_title = '𐌲𐌿𐍄𐍂𐌰𐌶𐌳𐌰' [11:53:53] https://quarry.wmflabs.org/query/25381 [11:58:34] Ahhh [12:03:52] Your cursor, it's using the utf8 set see https://dev.mysql.com/doc/refman/5.7/en/charset-connection.html [12:04:13] You need to set the client encoding [12:05:35] what encoding should I use for the mysqlconnection? [12:05:46] `set names utf8mb4;` [12:06:23] mysql_con_read = mdb.connect(host=languagecode+"wiki.analytics.db.svc.eqiad.wmflabs",db=languagecode + 'wiki_p',read_default_file=os.path.expanduser("~/replica.my.cnf"),charset='utf8') [12:06:29] perhaps here, in charset [12:07:43] Might work [12:08:56] But, do you need utf8mb4? [12:12:11] now i'll see if it works [12:12:39] i'm not sure if i need it. i'm a little at lost with these encodings [12:22:31] it works [12:22:37] hooray [12:22:41] thanks chicocvenancio [12:27:51] No problem, you're welcome [16:08:26] andrewbogott: (pinging you directly as your work king on wikitech) but is it expected for https://wikitech.wikimedia.org/ to redirect to newwikitech? [16:08:49] paladox: no, and it doesn't for me... [16:08:54] that's surprising [16:09:30] andrewbogott: though https://wikitech.wikimedia.org/wiki/Main_Page works doing / dosent [16:09:38] Ie if it has a slash [16:09:39] paladox: I'll try again [16:09:41] !log tools.stashbot Restarted bot to clear stale session cookies following wikitech host migration [16:09:42] bd808: Failed to log message to wiki. Somebody should check the error logs. [16:09:46] grrrr [16:10:12] "OAuthAuthorizationError: The authorization headers in your request are not valid: Invalid signature" that's new :/ [16:10:33] paladox: ok, I just saw it happen for me too [16:10:43] Ok thanks [16:12:13] paladox: found a local hack that was probably doing that… better now? [16:12:39] andrewbogott: yep [16:12:43] Works, thanks [16:12:47] cool. Thanks for noticing! [16:13:24] Your welcome :) [16:13:29] bd808: I just fixed one smallish issue that might have confounded your tests, might be worth one more restart [16:15:16] !log tools.stashbot Second restart after newhorizon redir bug fixed [16:15:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stashbot/SAL [16:15:24] that's better :) [16:15:45] * bd808 pets stashbot [17:08:08] !log wikilabels u_wikilabels=> update campaign set active = 'f' where wiki = 'fawiki' and form = 'damaging_and_goodfaith'; [17:08:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [17:12:35] andrewbogott oh yay, the switched kept me logged in :) (no log out) :) [17:13:11] paladox: had you logged in to newwikitech? [17:13:21] chicocvenancio possibly [17:13:28] though carn't remeber [17:14:43] probably you had a session on newwikitech already [17:15:18] ok [18:48:39] !help Could you grant "Shell users" permission to account "Ivan A. Krestinin"? [18:48:40] ivan-a-krestinin: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [18:49:16] ivan-a-krestinin: hi. what are you trying to do that you think shell user will fix? [18:49:45] it should be granted automatically these days when you become a member of a project [18:49:51] I want to move KrBot to Labs. [18:50:17] ivan-a-krestinin: great. What you probably want then is to become a Toolforge member [18:50:49] * bd808 checks to see if that's already done [18:51:41] It must be shown on https://wikitech.wikimedia.org/wiki/Special:Preferences#mw-prefsection-personal ? I do not see it. [18:52:22] ivan-a-krestinin: you have your developer account created. The next steps are at https://wikitech.wikimedia.org/wiki/Help:Getting_Started#Toolforge_users [18:52:28] specifically -- https://toolsadmin.wikimedia.org/tools/membership/apply [18:56:03] Thank you, I created request on https://toolsadmin.wikimedia.org/tools/membership/apply [19:46:41] Hello, tomorrow we have a warm-up session so local participants for upcoming participants can get ready for upcoming Wikimedia Hackathon 2018 https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2018/Warm-up_event_Orlandai Among other things I was planning to check with participants to request LDAP accounts, toolforge accounts and so. [19:47:13] My fault I didn't go through that process myself early this week for toolforge request. Would it be possible someone to review my request (Toniher) before tomorrow (Brussels time)? Sorry about this. [20:04:00] Hello toniher1 let me check that [20:07:25] chicocvenancio: thanks! [20:34:28] toniher1: done [20:35:28] chicocvenancio: thanks a lot! Sorry for not having applied more in advance [20:36:29] Ping me if you need help getting started in Toolforge or use the `! hel.p`(no dot) keyword here [21:52:17] How do people typically configure passwords for labs services (e.g. Cassandra in my case) in labs? [21:52:23] In hiera [21:52:29] Adding it to a Hiera: page in the clear seems like a bad idea [21:53:28] RoanKattouw i think the only way is to do it in either hiera: or through horizion [21:54:34] Gotcha [21:58:44] Doesn't labs have a "private" puppet repo? [21:58:58] Reedy oh wait it does [21:59:11] RoanKattouw ^^ [21:59:15] RoanKattouw: Should be able to use the private repo... Then the code should be mostly be agnostic [21:59:42] (agnostic between prod and beta/labs/whatever you want to call it) [22:01:03] Reedy actually, that may not work [22:01:31] as the private repo uses roles, so needs to be pulled in by one of the roles in the prod branch [22:02:00] Reedy though there's /etc/puppet/private [22:02:07] which is seperate to the private repo [22:04:25] also the "labs/private" repo IS NOT PRIVATE [22:05:23] Secrets ATM are usually done by having a project puppet master and having local labs/private commits [22:05:29] It's not perfect [22:08:39] Hmm OK so I can look on the deployment-prep puppet master then [22:10:35] Yup I see local commits with secrets there [22:10:45] Thanks chicocvenancio [22:10:49] * chasemp [22:12:34] legoktm it is if you do it on the local puppet master [22:12:38] and git commit it locally [22:13:26] Well, not when lots of people have access to it [22:13:30] It's not very private [22:13:57] ah yeh [22:40:08] Hmm OK so now I'm stuck on a puppet/hiera thing [22:40:29] Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Illegal title type at index 0. Expected String, got Integer at /etc/puppet/modules/profile/manifests/redis/master.pp:25:33 on node deployment-maps01.deployment-prep.eqiad.wmflabs [22:41:07] But, https://wikitech.wikimedia.org/wiki/Hiera:Deployment-prep/host/deployment-maps01 defines the relevant hiera entry ("profile::redis::master::instances") as an array with one element which is a string [22:42:28] ( gehel ---^^ ) [22:43:25] So I'm very confused that it says "expected string, got integer" [22:44:06] I even tried changing "6379" to "6379xyz" but strangely that didn't change the error [22:44:51] RoanKattouw: already in bed, but will have a look tomorrow... Where did you run it? [22:45:17] deployment-maps01.eqiad.wmflabs [22:46:10] Thanks, I'll let you know what I understand there... [23:14:26] mmm..maybe someone can help here... [23:14:34] wikitech is erroing on this page: https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Pageview_hourly/Sanitization [23:14:55] https://www.irccloud.com/pastebin/VJ8WeUrj/ [23:17:19] andrewbogott ^^ [23:17:32] nuria_: looking [23:18:28] nuria_: is that page some elaborate live-query data stream thing, or just a normal wiki page? [23:21:25] bd808: I have the feeling I just need to run update.php on the new wikitech host as a first step for ^ but would like reassurance that that's not a reckless/destructive thing to try [23:22:01] Or, um, Reedy, same question [23:22:29] andrewbogott: its a really, really bad thing for a main project wiki. I'm not 100% for Wikitech. [23:22:30] You shouldn't need to run it, no [23:22:45] What's the actual errror message? [23:22:46] glad I asked then [23:22:58] Reedy: nuria_'s paste is all I know so far [23:23:13] wikitech logs don't make it to mwlog1001 do they? [23:23:15] why would that one page trip over an rdbms error? /me looks for clues [23:23:19] in theory they do [23:23:22] Reedy: they should [23:23:27] and they should be in logstash [23:23:30] andrewbogott: i'd say "normal wiki page" [23:23:43] well, that's weird [23:23:46] andrewbogott: as normal as they make it [23:23:53] andrewbogott: it is analytics docs [23:24:02] 2018-03-09 23:22:23 [cf2e5469c24dfb1b8f52f202] labweb1001 labswiki 1.31.0-wmf.24 exception ERROR: [cf2e5469c24dfb1b8f52f202] /wiki/Analytics/Data_Lake/Traffic/Pageview_hourly/Sanitization Wikimedia\Rdbms\DBQueryError from line 1233 of /srv/mediawiki/php-1.31.0-wmf.24/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? [23:24:02] Query: SELECT math_inputhash,math_mathml,math_tex,math_svg,math_input FROM `mathoid` WHERE math_inputhash = '$��\'��S渳' LIMIT 1 [23:24:02] Function: MathRenderer::readFromDatabase [23:24:02] Error: 1146 Table 'labswiki.mathoid' doesn't exist (10.64.16.79) [23:24:11] math! [23:24:14] Sounds like an extension has been enabled on wikitech (purposely or otherwise) [23:24:18] its the worst :) [23:24:34] ah, math [23:24:42] https://wikitech-static.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Pageview_hourly/Sanitization fwiw [23:24:52] andrewbogott: so, you need to run... [23:25:00] math was definitely present on the old wikitech... [23:25:05] or at least tried to be [23:25:06] mathoid != math [23:25:19] mwscript sql.php --wiki=labswiki extensions/Math/db/mathoid.mysql.sql [23:26:12] Reedy: I trust you, but, what does that do? [23:26:20] creates the mathoid table [23:26:32] hm, ok [23:26:35] * andrewbogott does [23:26:46] I'm guessing that wikitech page added some maths logic... [23:27:16] this is the piecewise equivalent of running update.php which is safer to do [23:27:51] Warning: No such file or directory in /srv/mediawiki/php-1.31.0-wmf.24/maintenance/sql.php on line 94 [23:27:55] looking to see what that's about [23:28:31] $file = fopen( $this->getArg( 0 ), 'r' ); [23:28:38] yeah, not helpful [23:28:38] Is it wanting a full path? [23:28:48] mwscript sql.php --wiki=labswiki /srv/mediawiki/php-1.31.0-wmf.24/extensions/Math/db/mathoid.mysql.sql [23:28:56] ls -al /srv/mediawiki/php-1.31.0-wmf.24/extensions/Math/db/mathoid.mysql.sql [23:29:03] just to make sure the file is there and I've not typoed [23:29:21] ok, here we go [23:29:23] Query OK, 0 row(s) affected [23:29:39] And https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Pageview_hourly/Sanitization works again [23:29:41] nuria_: it's back [23:29:57] I wonder if the default math mode was changed recently or something [23:30:19] Reedy, you should know that feeding me instant and effective solutions like that just encourages me to never learn anything myself [23:30:23] nice! [23:30:41] andrewbogott: It's that or being sarcastic and patronising [23:30:48] And bd808 will probably just slap me [23:30:48] andrewbogott: jajaja, i SO know what you mean [23:31:04] Reedy: patronizing to myself [23:31:15] Do you want me to explain a bit more? [23:31:26] Reedy: I love when you are sarcastic and patronizing, just not in this channel :) [23:31:33] No, I understand now! I just never would've gotten there on my own. [23:32:27] If I may rephrase my earlier statement: Thank you for knowing everything so I don't have to :) It's very nice to have you back. [23:32:58] :P [23:33:06] There's a couple of other exceptions for labswiki in there [23:33:09] Nothing too worrying... [23:33:09] MWException from line 332 of /srv/mediawiki/php-1.31.0-wmf.24/includes/jobqueue/JobQueue.php: Got delayed 'cdnPurge' job; delays are not supported. [23:33:20] Cannot access the database: Can't connect to MySQL server on '10.64.48.11' (110) (10.64.48.11:3314) {"exception_id":"5dda08163ac58387dc759762","exception_url":"/wiki/Special:Uploads/Legoktm","caught_by":"mwe_handler"} [23:34:18] is it supposed to talk to db1097.eqiad.wmnet? [23:34:25] that's commons iirc? [23:34:46] bd808: in theory it talks to commons via public API rather than via db1097 [23:34:53] But for a while (a few days ago) it was trying to [23:34:58] so if it's an old enough log message, it's that [23:35:26] *nod* [23:35:30] Only a few hours ago [23:35:31] 5 ish [23:35:36] hm [23:35:42] that's unexpected [23:35:59] https://wikitech.wikimedia.org/wiki/Special:Uploads/Legoktm [23:36:02] ^ replicable [23:36:37] wtf [23:37:06] https://phabricator.wikimedia.org/T189343 [23:37:07] curious, why is it looking on commons for that? That should only show local wiki uploads right? [23:37:33] I'm confused why MobileFrontend is getting involved too [23:37:43] andrewbogott: apparently not [23:37:44] https://en.wikipedia.org/wiki/Special:Uploads/Reedy [23:37:50] Those are my uploads on commons [23:38:22] https://wikitech.wikimedia.org/wiki/Special:Uploads says "(Cannot access the database: Cannot access the database: Can't connect to MySQL server on '10.64.0.164' (110) (10.64.0.164:3314))" too, so not just with le.goktm prefix [23:38:43] suffix [23:38:47] oh, that makes me thing we shouldn't even have that page then.. [23:38:53] local uploads are https://wikitech.wikimedia.org/wiki/Special:ListFiles/Andrew_Bogott apparently? [23:39:09] Oh, is Special:Uploads a MF page? [23:39:14] I guess that'd explain the simplicity of it [23:39:16] yeah [23:39:25] its a mf hack page [23:39:58] andrewbogott: I probably wouldn't worry about it too much [23:40:10] 2018-03-09 19:00:30 [937c1ccbbdfe8fcc2da0e426] labweb1001 labswiki 1.31.0-wmf.24 exception ERROR: [937c1ccbbdfe8fcc2da0e426] /w/index.php?title=User:Bstorm&action=submit MWException from line 332 of /srv/mediawiki/php-1.31.0-wmf.24/includes/jobqueue/JobQueue.php: Got delayed 'cdnPurge' job; delays are not supported. {"exception_id":"937c1ccbbdfe8fcc2da0e426","exception_url":"/w/index.php?title=User:Bstorm&action=submit","caught_by":" [23:40:10] other"} [23:40:16] If it's really just that page I'm pretty happy to just say "don't do that then" [23:40:47] The jobqueue might be a real thing. I'm trying to have a local jobqueue rather than using the central one but I didn't really think about it that hard [23:40:57] Would've expected it to carry over from the old wikitech setup [23:43:30] Reedy: I don't think I know what "Got delayed 'cdnPurge' job" means. That doesn't mean that /all/ jobs are failing does it? [23:44:00] Nope, it doesn't [23:44:09] Just the purges to the... varnish layer? I guess [23:44:32] oh, huh. bd808 and I verified that purges were working, or at least thought we did [23:47:09] Reedy: I turn out to have to run but I made a note about that here: https://phabricator.wikimedia.org/T189347 [23:47:24] I encourage you to think about this issue, or not think about it, as you prefer. [23:47:43] at least some purges are working [23:48:20] it may be that purges which result from template change propagation are not or something like that