[03:32:04] !log tools removed /data/project/.elasticsearch.ini, owned by root and mode 644, leaks the creds of /data/project/strephit/.elasticsearch.ini Might need to cycle it as well... [03:32:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [04:20:55] !log telnet Restarted service using instructions from https://meta.wikimedia.org/wiki/Telnet_gateway [04:20:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Telnet/SAL [09:02:47] Is tools-bastion-03 overloaded? [09:03:24] load 3, high CPU wait, nfs busy doing something, commands take ages [09:15:47] Let me check [09:18:34] !log tools killed io intensive tool job in bastion [09:18:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:19:08] Nemo_bis: hopefully it is fixed now [10:51:47] !log tools package upgrades in tools-checker-01 tools-clushmaster-01 and tools-docker-builder-05 [10:51:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [11:22:02] !log tools package upgrades in tools-mail, tools-grid-master, tool-logs-02 [11:22:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [11:35:22] !log tools package upgrades in tools-package-builder-01 tools-prometheus-01 tools-static-10 and tools-redis-1001 [11:35:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [11:43:23] !log tools package upgrades in tools-webgrid-lightttpd-1401 [11:43:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [13:51:53] What happened to -labs? [13:54:08] ? [13:54:12] It redirects here [13:55:13] Labs vanished in a puff of logic and hence became a cloud. [13:56:04] andre__: screenshotted :) [13:57:24] I didn't get redirected Reedy [13:58:12] WFM [13:58:53] Just opened IRSSI and joined #wikimedia-labs ended up here [13:58:57] or, here actually [14:00:04] I just get `Cannot join 22#wikimedia-labs (20Channel is invite only)` [14:00:17] /shrug [14:00:32] Hello everyone. Do you know wether there is a DB replica for gerrit.wikimedia.org? [14:02:25] africanhope: you mean like wikireplicas but for gerrit? [14:03:09] My mode is +Zi so maybe I'm not redirected because I'm invisible? [14:03:37] jynus: Yes, I would like to host a script on ToolForge. The script would run some queries on a replica of gerrit [14:04:34] there is not, setting public replicas requires an intensive work of sanitization, and we currently do none for not-first-party software [14:04:48] africanhope: do you want to gather statistics? [14:05:26] there is a REST api https://dev.vaadin.com/review/Documentation/rest-api.html [14:05:31] you could work with [14:05:31] jynus: yes, I started played around with the REST API which by the way works quite fine. [14:06:00] jynus: Thanks, I guess I am on the right path [14:06:14] or if they are interesting in general you could submit a patch to production- although that will be a more time consuming effort [14:08:40] jynus: actually, the API is a good fit. The replica idea was to avoid having to use a user database for storing the stats gathered [14:10:16] looking at the documentation, user DB seems not to be a good choice for keeping stats online for a long time, I am wrong? (https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#ToolsDB_Backups_and_Replication) [14:10:51] you can call the api and generate a user database with the stats [14:11:16] but if it is for long term, and not dynamic, maybe CSV files would be better? [14:11:56] I think so, I was considering storing data into files, either json or csv [14:12:00] it depends very much on what you want to do, it is difficult to give general advice [14:12:52] jynus: the script gathers patches submitted by a set of usernames, say a team [14:12:53] africanhope: why not user dbs? Lack of backups? [14:13:50] chicocvenancio: the documentation warns about the fact that user DB gets erased, if I understood it well [14:14:05] chicocvenancio: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#ToolsDB_Backups_and_Replication [14:14:38] databases do not get erased, those were only for 3 very heavy ones [14:15:20] what we say is we are not responsible if those are accidentaly droped- in other words, the users are responsible to keep copies if needed [14:15:29] jynus: thanks for the explanation [14:15:59] for example, if you run "DROP" accidentally, we cannot recover them [14:16:08] Yes, I thought there was some misunderstanding there [14:16:24] but we don't acctively drop them [14:16:51] Backups should be done to locations outside WMCS [14:17:16] chicocvenancio: that makes sense indeed [14:18:27] I think the confusion arised because we "removed" user databases from wikireplicas [14:18:48] but we didn't drop them, just they are on separate servers from the replicas [14:24:12] jynus: thanks again for the clarifications [15:15:03] jynus your looking at quite the old rest api :) [15:15:10] They used a really old version [15:37:06] Technical Advice IRC meeting starting in 25 minutes in channel #wikimedia-tech, hosts: @addshore & @chiborg - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [15:42:01] hi! I'd like some review from wmcs folks on https://gerrit.wikimedia.org/r/c/412860/ [16:20:42] godog: I have 0 minutes as I'm due fo ra meeting now :) but I thought we already did this? I'll try to come back today to it today [16:26:18] !log tools Rebooting tools-docker-registry-01, NFS mounts are in a bad state [16:26:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [16:26:46] chasemp: I thought so too, and then realized lab hosts are in .wikimedia.org not .wmnet :| [16:26:55] thanks for taking a look tho! [16:57:01] !log tools deploying https://gerrit.wikimedia.org/r/#/c/410177/ [16:57:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [16:59:55] !log tools puppet is broken across the cluster due to last change [16:59:59] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [17:02:54] !log tools revert last change https://gerrit.wikimedia.org/r/#/c/413198/ [17:02:59] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [17:02:59] !log paws deleted query-killer k8s deployment T187818 [17:03:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [17:03:01] T187818: Analyse PAWS query killer - https://phabricator.wikimedia.org/T187818 [17:24:09] !log tools another try: merged https://gerrit.wikimedia.org/r/#/c/413202/ [17:24:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [18:12:57] could someone look at african hope's toolforge access request, I trust them personally so I think it should be quite an easy accept imo. [18:15:07] !log tools puppet should be fine across the fleet [18:15:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [18:15:14] Zppix: done [18:15:20] ty [19:02:31] !log tools disabled puppet on tools-static-* pending change 413197 [19:02:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [19:54:02] !log wikilabels sudo -u www-data /srv/wikilabels/venv/bin/wikilabels new_campaign cawiki "Qualitat de l'edició (mostra 40k)" damaging_and_goodfaith DiffToPrevious 1 50 (T187771) [19:54:05] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [19:54:05] T187771: Edit quality campaign for Catalan Wikipedia - https://phabricator.wikimedia.org/T187771 [19:54:26] !log wikilabels less ~/cawiki.revisions_for_review.5k_2017.json | sudo -u www-data ../venv/bin/wikilabels task_inserts 68 (T187771) [19:54:29] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [21:37:50] (03PS1) 10Bstorm: cdnjs: Remove the need for a packages.json altogether [labs/tools/cdnjs-index] - 10https://gerrit.wikimedia.org/r/413257 [21:39:56] (03CR) 10Bstorm: [V: 032] "Proof that I'm not going to let that cron become tech debt. ;-)" [labs/tools/cdnjs-index] - 10https://gerrit.wikimedia.org/r/413257 (owner: 10Bstorm) [21:44:08] (03CR) 10Zhuyifei1999: [C: 031] cdnjs: Remove the need for a packages.json altogether [labs/tools/cdnjs-index] - 10https://gerrit.wikimedia.org/r/413257 (owner: 10Bstorm) [21:45:33] (03CR) 10Bstorm: [V: 032 C: 032] cdnjs: Remove the need for a packages.json altogether [labs/tools/cdnjs-index] - 10https://gerrit.wikimedia.org/r/413257 (owner: 10Bstorm) [22:43:47] for whatever reason, Firefox 58 segfaults when I naively tried it on tools-dev [23:17:22] legoktm: did it have a display set?