[00:14:53] Aftеr tһe acquіsitⅰoᥒ bỿ Ꮲriⅴаte Internᥱt Ꭺϲϲess, Freеᥒoⅾе іѕ ᥒoᴡ beiᥒg uѕed tο pᥙsh ІⅭⲞ ѕcaⅿs һttpѕ:∕/ᴡwᴡ.cοindеѕk．com/һaᥒԁsһɑke-rеveаled-∨cs－back-ⲣⅼɑn-to╴givе-awɑy-100-ⅿilliⲟᥒ-iᥒ－cryрto/ [00:36:29] Aftеr the acquisitiⲟn by Priᴠаtе Iᥒtеrᥒеt Acⅽeѕs， ᖴreeᥒode is ᥒⲟw beiᥒɡ ᥙseⅾ tഠ рuѕһ ICO ѕcɑms httpѕ:᜵∕wwᴡ․cഠinⅾesk.сom/hɑndshake-reveɑleⅾ-vcs-back-рⅼɑᥒ－to-givе-awaỿ-100−miⅼⅼіoᥒ-іᥒ-crypto／ [02:34:20] Ꭺfter tһᥱ ɑcquisіtіon bу Priⅴatᥱ Ⅰnternet Aϲcеss, Frᥱeᥒoⅾe іѕ now beiᥒɡ ∪sed to push ΙCO scamѕ һttpѕ:／/ᴡᴡw.ⅽഠindesk．com/handѕһakᥱ-rеvealеd-vcѕ-bɑck-ⲣlan﹣to-givе╴away-100-ⅿillіοn-ⅰn－ϲrypto⧸ [06:38:57] Αfter the ɑϲquіsitіon by Ρriᴠаtе Ιᥒternеt Аccess, ᖴrеᥱᥒodᥱ is nοw beiᥒɡ uѕed to рusһ IᏟΟ sⅽams һttⲣs:᜵/wᴡw.сoⅰnԁеsk.com᜵һaᥒdshakе﹣reveаlеⅾ-∨ϲѕ-bаϲk-plaᥒ－to-gіve╴аway－100╴miⅼⅼion-іᥒ⎼crуpto/ [07:38:03] "Ꭺⅼⅼ tഠⅼd, ᕼaᥒԁѕhаkᥱ aims to giⅴe $250 wοrth ഠf its tⲟkens to *eаch⋆ ∪ser οf tһe webѕites thе compɑnỿ hɑs рartnersһiⲣѕ ᴡіtһ – GitHub, the P2P Fo∪nԁаtⅰoᥒ aᥒd *ᖴREENOᗪᎬ*, ɑ chat ⅽhannеⅼ for рeer-tⲟ﹣pеer prοϳectѕ． Aѕ sᥙcһ, ... [08:04:30] (03PS1) 10Urbanecm: Make bug report form work [labs/tools/commons-mass-description] - 10https://gerrit.wikimedia.org/r/462644 [08:04:46] (03CR) 10Urbanecm: [C: 032] Make bug report form work [labs/tools/commons-mass-description] - 10https://gerrit.wikimedia.org/r/462644 (owner: 10Urbanecm) [08:05:14] (03Merged) 10jenkins-bot: Make bug report form work [labs/tools/commons-mass-description] - 10https://gerrit.wikimedia.org/r/462644 (owner: 10Urbanecm) [10:36:12] "All tⲟⅼd, Ⲏɑndshɑke aiⅿs to giᴠe $ᒿ50 ᴡorth οf іts tokеᥒѕ to *ᥱaϲh* usᥱr of tһе websiteѕ tһe cοmⲣɑᥒy һas partᥒᥱrѕhiрs wіtһ – GⅰtᎻubˏ tһe P2Ρ Fഠuᥒdatiοᥒ aᥒd ＊ᖴREENⲞᎠE*, a ϲhat cһаᥒnel fഠr pᥱᥱr-to-peеr projеϲts. Αѕ suϲһ, ... [10:39:01] "All tⲟⅼdˏ Haᥒԁѕhakе aіms tഠ ɡіve $ᒿ50 worth οf ⅰts tokеnѕ to *each⋆ uѕer of the ᴡebsіtеs the сomрaᥒу haѕ partnerѕhiрѕ ᴡіth – ᏀⅰtНubᛧ the P2P Fouᥒԁation аnd ⋆FᖇᎬENODE*, a chɑt сhanᥒel for pееr˗to-peеr ⲣrοjеϲtѕ. As ѕucһ, ... [11:10:51] "Aⅼⅼ tⲟⅼⅾ, Handѕhɑkе аimѕ to ɡіⅴe $250 wоrth of ⅰts tokenѕ to ﹡еaϲh* ᥙѕer ⲟf tһe wеbsiteѕ thᥱ coⅿрɑᥒу һɑѕ partᥒеrshіps ᴡith – GitⲎubᛧ thе ⲢᒿP ᖴⲟᥙnⅾation anԁ ﹡FRΕENОDΕ*, a ϲhɑt channel fοr pеᥱr﹣to-рeᥱr projects. Ꭺs ѕucһ, ... [11:55:52] zhuyifei1999_: you around? [11:56:45] "Αll tolԁˏ Hanԁshake ɑimѕ to ɡⅰve $250 worth оf its tഠkeᥒs to *eаch﹡ uѕer of tһe websⅰtᥱs the coⅿрɑny һaѕ раrtnershіpѕ with – ᏀⅰtH∪bˏ thе P2Ⲣ Founԁatiοᥒ and ＊FREENODE＊ˏ ɑ chat ⅽһɑnᥒel fⲟr peᥱr-to-peer prοјеctѕ． As suϲһˏ dеvᥱloⲣеrs wһο һɑᴠe existⅰnɡ acϲⲟᥙᥒtѕ ഠᥒ eаϲh ... [12:09:38] !log quarry make myself projectadmin [12:09:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [12:28:47] ＂Αll tοⅼⅾ, Hаᥒdsһаkе ɑiⅿѕ tഠ givе $ᒿ50 ᴡоrtһ оf its tokens to *eacһ* user of tһᥱ ᴡᥱbsіtes the ϲoⅿрaᥒy hɑs partnerships with – GіtНub, ... [12:36:17] "Αlⅼ told, Ηaᥒԁѕһake aiⅿѕ to ɡive ＄ᒿ50 ᴡⲟrth of its tokenѕ to *eɑⅽh＊ user of tһe wеbsіtеs the cоmpaᥒy has рartnerѕһіps wіth – GitΗ∪b, ... [12:36:21] thе P2Ρ Fouᥒⅾation аᥒd *ᖴᎡEEΝOᗪΕ*, а cһat cһɑᥒnel fοr рeer╴to－pᥱer prοϳᥱcts. As sucһ, ⅾeveloⲣers ᴡho hаᴠe ᥱxⅰstiᥒg ɑccⲟᥙᥒts oᥒ each coᥙⅼԁ receive up tο $750 ᴡⲟrth of Ꮋаᥒdshake tokens.＂ [12:51:56] ＂Aⅼⅼ tοldˏ Hɑnⅾѕһake ɑiⅿs tο ɡі∨е ＄250 ᴡⲟrth οf itѕ tοkens tഠ *ᥱach* ᥙѕеr оf thе ᴡebѕiteѕ thе ⅽⲟⅿpany һas pɑrtnerѕhіⲣѕ ᴡіtһ – ԌіtΗub, the P2Ⲣ Fo∪ndаtiⲟn anԁ ＊FREENODE*ˏ a ϲһat chɑᥒᥒеl fⲟr реer-to╴pеer proϳеcts. ... [13:56:19] bd808: is it possible to read from a database on toolforge from a maintenance script on enwiki? I assume not but wanted to check [14:16:57] that would not be a good idea kostajh -- assuming you mean from within production [14:17:14] chasemp: that's what I thought, just wanted to verify. thanks [14:17:32] no worries, thanks for asking first :) [14:40:09] "All toldˏ Ηɑndshakᥱ аiⅿs to ɡіve ﹩ᒿ50 ᴡοrth оf ⅰts tഠkenѕ to *еaⅽһ＊ usеr of tһe webѕitеѕ tһе ϲοmраny һɑs рartnersһipѕ with – GitHub, ... [15:35:28] (03PS1) 10D3r1ck01: Add Pywikibot-core phab board to #wikimedia-dev-africa [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/462740 [15:44:53] arturo: yeah [15:45:00] (sorry, late) [16:25:20] zhuyifei1999_: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/quarry <--- I'm writting that [16:25:55] ok [16:26:14] oh I never used /var/log/syslog, been using journalctl for too long o.O [16:26:21] any comments/corrections/suggestions are welcome :-) [16:27:08] zhuyifei1999_: specially common maintenance operations, if any [16:27:24] for example, how do we kill a long query? [16:27:41] it gets killed automatically after 30 mins [16:27:51] there's a killer.py iirc [16:28:08] it's executed on the -web-01 instance via cron [16:28:08] did we have any case in which that didn't work? [16:28:51] well, when it doesn't work, it's either: 1. the script broke entirely 2. it actually worked [16:29:12] ok :-) [16:29:28] do you remember what were the last issues with quarry? [16:30:09] by it actually worked I mean the query finished executing, but the worker could not store the results due to random issues (duplicate column names, OOM, etc), and the query status is falsely 'running' [16:30:48] the last issue? probably https://www.mediawiki.org/wiki/Topic:Ule7vtcxk9egnhmw ? /me is gonna reply there in a few mins [16:38:14] arturo: mind debugging puppet a bit for me? [16:38:32] sure [16:38:47] profile::quarry::database should use mariadb::config with default max_allowed_packet = 16M [16:39:11] that class applies the config to /etc/my.cnf [16:39:22] but there max_allowed_packet is empty [16:39:47] ok, give me a second and will focus on this [16:39:53] and I'm pretty sure that cause mariadb to recognize it as 'minimum value possible' (1024) [16:39:57] ok thanks [16:54:38] zhuyifei1999_: found the issue [16:54:43] (I believe) [16:54:57] do we have a bug number? [16:59:43] not yet [16:59:57] do you plan to open one? [17:00:08] the report is https://www.mediawiki.org/wiki/Topic:Ule7vtcxk9egnhmw [17:00:48] well, if it's minor enough, /me is kinda lazy [17:01:09] np [17:01:15] just for sanity in the commit message :-P [17:01:21] k [17:02:04] aaand the fix is: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/462769 [17:05:07] ah [17:09:01] !log wmflabsdotorg giving novaadmin the 'admin' role on this project. This is needed to create domains until we have a version of designateclient that sudo-project-id [17:09:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wmflabsdotorg/SAL [17:23:42] arturo: how should I mention the query results are stored in sqlite dbs on NFS? (it's bad I know, but T178520) [17:23:43] T178520: Find somewhere else (not NFS) to store Quarry's resultsets - https://phabricator.wikimedia.org/T178520 [17:28:19] zhuyifei1999_: oh, you can create an architecture subsection for that I guess [17:29:13] umm, I so bad at writing docs :( [17:29:35] ok, I can do it then [17:29:39] :-) [17:30:04] it's /data/project/quarry/results/ [17:31:25] I would like to document as well any other admin procedure we know for quarry, cleanups, restarts, whatever [17:31:32] code updates... [17:32:13] git fetch && git checkout FETCH_HEAD [17:32:25] and then restart the relevant services via systemctl [17:33:19] do we tag releases or something? or simply deploy as we go? [17:33:28] deploy as we go :) [17:33:45] well, ideally we should have some sort of tag, bug meh [17:37:32] zhuyifei1999_: as which user do you fetch the code? [17:37:43] right now, root [17:37:56] (not ideal) [17:38:10] and then chown? [17:38:42] isn't `git fetch && git checkout FETCH_HEAD` the same as `git pull`? [17:38:48] (/me never remembers to chown) [17:39:05] well, git pull stays on a branch [17:39:36] we could use git pull I guess [17:39:59] also, you need to do this by hand in all workers, right? [17:40:03] (and the web node) [17:40:10] * zhuyifei1999_ just slightly hate if ever there's a local commit, it gets an ugly 'merge' [17:40:14] yeah [17:40:29] (but we rarely ever have local commits) [17:41:14] we could probably figure out a cumin command to do the code update in all nodes at once [17:41:31] sure, that's be great [17:42:21] I wonder, should we chown at all? I mean, I wouldn't want it to be writable to uwsgi in case it's compromised [17:42:51] well, it's currently quarry:www-data [17:43:11] yes, it's from puppet [17:43:34] when the repo is initially cloned, puppet cloned it as quarry:www-data [17:44:39] (so right now the repo is a mixture of root ownership and quarry ownership) [17:44:49] I see [17:50:02] zhuyifei1999_: I don't find the NFS stuff persisted in puppet [17:50:15] ? [17:50:57] the mounts [17:51:02] well, the mount [17:51:35] it’s the legacy code [17:51:50] ? [17:51:58] mounts used to be default for labs projects [17:52:16] oh, if a project has a NFS mount, then it mounts it by default? [17:52:34] when it turned to opt-in, quarry opted-in [17:53:47] the config is in some yaml in the puppet repo, don’t remember exactly where [17:54:29] ok [17:55:29] so quarry uses mostly the same mechanism to mount NFS as toolforge [17:56:03] ok [18:01:44] found it. it's nfs-mounts.yaml in module labstore [18:01:48] I think [18:02:22] zhuyifei1999_: so, nothing to document from the administration POV? [18:02:34] I think so [18:02:37] (apart of code updateS) [18:02:54] then that's very little for an admin to do :-P [18:02:55] it shoudl automount on boot [18:03:21] *auto-mount, (not autofs :P) [18:04:41] sometimes when some query is stuck in 'running' but not actually running I check the celery logs [18:05:04] zhuyifei1999_: when you started managing this stuff, do you remember anything that you may want to know but nobody knew? [18:06:47] ummm, there's a certain js file in the repo (compiled.js iirc) that's clearly generated, but I didn't know generated by what. now it's documented in README... [18:06:59] (generated by nunjucks fwiw) [18:07:40] BTW the logs shows A LOT of python backtraces everywhere, that's probably something worth reviewing in the codebase [18:07:47] I don't remember anything else that took me a long time to figure out [18:07:53] which host? [18:09:09] https://www.irccloud.com/pastebin/WLNP6x7W/ [18:09:27] the ones I know of that cause backtraces are T205222 T172086 and the mediawikiwiki thread earlier [18:09:28] T205222: The absence of resultset case is not handled - https://phabricator.wikimedia.org/T205222 [18:09:29] T172086: Gigantic query results cause a SIGKILL and the query status do not update - https://phabricator.wikimedia.org/T172086 [18:10:17] arturo: sorry to interrupt, but did you approve my toolforge request? Or is this a coincidental name thing? [18:10:22] ^ malformed uri pointing to non-existent result set. probably T205222 but didn't look in detail [18:10:38] Operator873: checking [18:11:01] Operator873: https://toolsadmin.wikimedia.org/tools/membership/status/390 <--- is that you? [18:11:17] arturo: yes. [18:11:27] then it's me :-) [18:11:39] The reason I ask is it is still asking me to request access????? [18:12:16] Operator873: where? There could be an error somewhere [18:12:17] https://usercontent.irccloud-cdn.com/file/3PyctbA1/image.png [18:12:19] Operator873: re-login [18:12:26] zhuyifei1999_: d'uh [18:12:28] I hadn't tried that [18:12:29] stand by [18:12:48] yup [18:12:50] that worked [18:12:53] :) [18:12:59] sorry to bother arturo. And many thanks zhuyifei1999_ [18:13:16] you are welcome Operator873 :-) and thanks zhuyifei1999_ [18:13:48] some of our codebase just can't realize some permissions changed (and making them aware can be damn complicated) [18:13:53] (np) [18:14:23] I believe [kick codebase] could work [18:14:32] :) [18:15:46] arturo: it's google docs accessing this malformed uri. see nginx access.log [18:15:57] https://phabricator.wikimedia.org/T197256 [18:16:02] like wtf [18:17:48] lol [18:19:12] thanks for your time zhuyifei1999_, really appreciated :-P [18:19:16] np [18:19:17] I'm off for the day [18:19:20] ok [18:21:31] Operator873: oh if you have any issues running bots on toolforge, fell free to ask [18:22:04] zhuyifei1999_: I appreciate it. I'm probably going to take advantage of that [18:22:11] :) [18:48:41] zhuyifei1999_: Well I didn't think I would be asking this soon, but I'm having troubles with WinSCP. It's probably PEBKAC issue, but I just can't wrap my head around it. I've followed the guide and keep bouncing off with "no supported authentication methods available" [18:48:49] Do you have any common suspects I can double check? [18:49:16] did you upload your public key and pointed winscp at your private key? [18:50:10] I do have a public key listed on my account on toolsadmin [18:50:31] which host are you connected to? login.tools.wmflabs.org? [18:50:47] correct [18:52:03] you also specified the username as operator873 right? [18:52:25] yes [18:52:48] I don't see the username in /var/log/auth.log [18:53:00] https://usercontent.irccloud-cdn.com/file/SF7UcRYc/image.png [18:53:22] what is the timestamp of your login request? [18:54:27] that's an interesting error message [18:54:58] Does WinSCP have your private key Operator873 ? [18:56:08] zhuyifei1999_: 2018-09-25 13:55:32.097 [18:56:29] Krenair: I have linked to the local file in the Authentication setup [18:57:54] Sep 25 18:55:14 tools-bastion-03 sshd[5908]: Invalid user docker from [REDACTED IP] [18:57:54] Sep 25 18:55:14 tools-bastion-03 sshd[5908]: input_userauth_request: invalid user docker [preauth] [18:57:55] Sep 25 18:55:14 tools-bastion-03 sshd[5908]: Connection closed by [REDACTED IP] [preauth] [18:57:55] I assume this is not relevant ^ [18:58:25] won't be Operator873 [18:59:47] Sep 25 18:56:09 tools-bastion-03 sshd[6506]: Connection from 10.68.17.221 port 58496 on [10.68.23.58 = internal tools-bastion-03 IP] port 22 [18:59:47] Sep 25 18:56:09 tools-bastion-03 sshd[6506]: Did not receive identification string from [10.68.23.58 = internal tools-bastion-03 IP] [18:59:47] shouldn't be relevant either [19:00:08] not me [19:00:53] no that won't be it [19:00:58] oh those 10.* are prometheus and shinken [19:01:10] zhuyifei1999_: I sent you a notice [19:01:12] yeah you should be seeing external IPs zhuyifei1999_ [19:01:42] for Operator873 [19:01:53] Operator873: ok. I don't see that IP in the logs at all [19:02:54] lovely [19:03:01] alright let me recheck [19:03:32] just to make sure, you are connecting to login.tools.wmflabs.org right? I'm checking the logs on that host [19:04:09] (disclosure: I'm tired and that may be a factor) [19:04:29] https://usercontent.irccloud-cdn.com/file/qhN0qzZU/image.png [19:04:53] Operator873? [19:05:04] ? [19:05:14] Your username is operator873 [19:05:37] Krenair: I've tried operator873 with no luck either [19:06:49] what does login.tools.wmflabs.org resolves to on your side? it should be 208.80.155.163 [19:08:05] it does [19:08:21] and 'telnet login.tools.wmflabs.org 22' shows 'SSH-2.0-OpenSSH_6.9p1 Ubuntu-2~trusty1' ? [19:08:57] correct [19:09:11] alright you're almost certainly talking to the real login.tools.wmflabs.org [19:09:30] have you tried connecting with a different client? [19:09:58] no, not as yet. I use WinSCP for my shell (irc bots) and was hoping to keep to one lol [19:10:52] I am curious about why you don't show up in the logs [19:11:17] zhuyifei1999_, did you grep for username (case insensitive) as well as IP? [19:11:23] Operator873: are you sure you are not on the IP with the last digit as 8? [19:11:46] Krenair: yes, searched for the given IP and 'r873' [19:11:57] neither got results [19:11:59] I have seen a network before that sends out HTTP traffic and SSH traffic from different external IPs [19:12:15] so be careful if you're using some 'find my IP' service [19:12:31] hmm... interesting... standby let me verify [19:12:40] I'm gonna search for this IP I'm suspecting to be Operator873's [19:13:44] https://www.irccloud.com/pastebin/o3VZM6uM/ [19:14:54] I doubt it's that as Operator873 showed a screenshot with username Operator873 [19:15:07] that one is the mention of that IP before the-connect-without-identification-string at Sep 25 19:09:00 [19:16:58] 2018-09-25 14:05:48.892 [19:17:04] was my last attempt [19:17:24] UTC-5? [19:17:55] yes [19:19:12] from Sep 25 19:04:16 to Sep 25 19:05:18 only internal IPs are connecting [19:19:38] at 19:05:41 something is interesting: [19:20:08] Sep 25 19:05:41 tools-bastion-03 sshd[12933]: Connection from [IP REDACTED] port 42322 on 10.68.23.58 port 22 [19:20:08] Sep 25 19:05:47 tools-bastion-03 sshd[12933]: Received disconnect from [IP REDACTED]: 11: Normal Shutdown, Thank you for playing [preauth] [19:20:08] Sep 25 19:05:47 tools-bastion-03 sshd[12933]: Disconnected from [IP REDACTED] [preauth] [19:20:16] that IP ends with a 0 [19:20:51] give me a moment... may have uncovered an issue [19:21:34] then only internal IPs are connection from Sep 25 19:06:01 till Sep 25 19:08:19 [19:25:10] I believe the issue is on my end here and is a result of my backup internet access. [19:26:28] ok [19:36:53] Operator873: any luck? [19:38:36] gtirloni: I'm still trying to determine what's going on. I managed to auto ban myself off my shell so I'm waiting for it to reset [19:40:58] ok [20:48:33] (03PS1) 10Lokal Profil: Apply same filters to database statistics as to update monuments [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462809 (https://phabricator.wikimedia.org/T203349) [21:15:21] (03PS1) 10Lokal Profil: Rename test file [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462814 [21:21:46] (03PS1) 10Lokal Profil: Replace pywikibot.output calls with more appropriate output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462815 [22:02:28] (03PS1) 10Lokal Profil: Move config filtering logic to monuments_config [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462821 [22:12:48] (03PS1) 10Lokal Profil: Make use og monuments_config.filtered_countries [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462823 [22:17:25] (03CR) 10Jean-Frédéric: [C: 032] Apply same filters to database statistics as to update monuments [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462809 (https://phabricator.wikimedia.org/T203349) (owner: 10Lokal Profil) [22:17:59] (03CR) 10Jean-Frédéric: [C: 032] Rename test file [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462814 (owner: 10Lokal Profil) [22:18:28] (03CR) 10Jean-Frédéric: [C: 032] Replace pywikibot.output calls with more appropriate output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462815 (owner: 10Lokal Profil) [22:19:49] (03CR) 10Lokal Profil: "making database atatistics skip wlpa will be a follow-up patch" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462823 (owner: 10Lokal Profil) [22:19:53] (03Merged) 10jenkins-bot: Apply same filters to database statistics as to update monuments [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462809 (https://phabricator.wikimedia.org/T203349) (owner: 10Lokal Profil) [22:20:03] (03Merged) 10jenkins-bot: Rename test file [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462814 (owner: 10Lokal Profil) [22:20:55] (03Merged) 10jenkins-bot: Replace pywikibot.output calls with more appropriate output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462815 (owner: 10Lokal Profil) [22:22:29] (03CR) 10jenkins-bot: Apply same filters to database statistics as to update monuments [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462809 (https://phabricator.wikimedia.org/T203349) (owner: 10Lokal Profil) [22:24:10] (03CR) 10jenkins-bot: Rename test file [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462814 (owner: 10Lokal Profil) [22:25:45] (03CR) 10jenkins-bot: Replace pywikibot.output calls with more appropriate output formats [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462815 (owner: 10Lokal Profil) [22:29:10] (03CR) 10Lokal Profil: Make use og monuments_config.filtered_countries (032 comments) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462823 (owner: 10Lokal Profil) [22:30:27] (03CR) 10Lokal Profil: "recheck" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462821 (owner: 10Lokal Profil) [22:31:20] (03CR) 10jerkins-bot: [V: 04-1] Move config filtering logic to monuments_config [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/462821 (owner: 10Lokal Profil) [22:55:59] �H�]S��~x)H[��W��׵h����ڳ�g^'�ePG��?& [22:56:02] ��v��Z_��5��x�8Y h��<�ht��dDu�I�` [22:56:04] ��k�xk��k��]��O=��e&8,��-_ĥ/� �_!��~fA� [22:56:07] ��8m��{�չ��%��d@�'�� +*�AG@��TK[�R�� [22:56:10] ��v��IS)c�l&Z��3�z�K�U5�� [22:57:08] hmm shinken-wm disconnected. [22:57:34] �g�z�H��I��kw|@^�F�ǹ9&*��_�ո��>zZ��'��.`� [22:57:37] �tqva(V�4��s��&�c+�ޫ�O��>]��י""9�v��Z� [22:57:39] �2j:1��R6�t�4�/~4��ST [22:57:42] � ��U��|�k�d�1�[�v�*��ۆ5�_{W�>��U��h [22:57:51] Krenair.. [22:58:25] ok [22:59:35] !log shinken restart ircecho service that had stopped for some reason [22:59:37] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Shinken/SAL [22:59:51] thanks Krenair [23:01:18] I wonder if those characters made it fail [23:01:31] is it written in python by chance ? [23:05:39] Platonides, it is [23:06:20] * Platonides suspects a UnicodeException there [23:07:25] * bd808 is not sure why shinken-wm would be reading the channel feed [23:07:35] though it won't be doing any parsing of messages sent to it/the channel [23:07:58] and it's using a framework which you'd hope could cope with this [23:08:48] well, it seems suspicious that it disconnected precisely when a bot started sending garbage here [23:09:00] and python isn't exactly lenient with encodings [23:09:01] it probably needs this magic -- https://github.com/bd808/python-ib3/blob/master/ib3/__init__.py#L35-L40 [23:47:23] �1�N0\(��yH��=��8#k��;��7D��R��Vw [23:50:32] �vx��<�8�+� [23:50:35] ��:��`V\�-�EEv�9