[21:13:04] paladox are you doing stuff with prod bot [21:47:41] Nope [21:51:14] Zppix https://github.com/martynsmith/node-irc/issues/485 [21:51:26] haven't looked at the logs tonight to confirm it is the same problem [21:51:30] but i belive it is [21:52:16] paladox im already on shell want me to look to see what logs are reporting? [21:52:38] Yes [21:52:45] but i think they may be deleted [21:53:08] paladox no, i believe grrrit-wm logs it oustide of kubectl in its own files [21:53:10] let me look [21:53:16] Ok [21:56:32] this [21:56:33] tools.lolrrit-wm@tools-bastion-03:~$ kubectl logs grrrit-wm-230500525-onw7j --previous [21:56:33] Error from server: previous terminated container "bot" in pod "grrrit-wm-230500525-onw7j" not found [21:56:38] should have worked but failed [21:57:07] https://phabricator.wikimedia.org/P4483 [21:57:16] is what i got [21:59:34] Zppix where did you get that log from? [21:59:48] lolrrit-wm.err [22:00:04] Thats an old log [22:00:24] if you do ls -la where that log is located [22:00:25] theres 200 thousand error log files i cant tell [22:00:27] you will see the date [22:01:11] https://phabricator.wikimedia.org/P4483 [22:01:13] Woops [22:01:17] -rw-r--r-- 1 tools.lolrrit-wm tools.lolrrit-wm 82440 Sep 15 2015 lolrrit-wm.err [22:01:19] Zppix ^^ [22:01:27] it;s from sept 15 2015 [22:01:30] oh [22:02:12] Zppix apparently it has been running for 3 days [22:02:17] and no restarts [22:02:25] i think it may have been flood protection [22:02:34] paladox it said remote host closed connection [22:02:36] not excess flod [22:02:38] flood* [22:02:46] yes. [22:02:48] Strange [22:02:51] so it wasnt a flood [22:02:58] but NAME READY STATUS RESTARTS AGE [22:02:58] grrrit-wm-230500525-onw7j 1/1 Running 0 3d [22:03:01] ^^ it shows that [22:03:14] its online now [22:03:32] yep [22:04:03] paladox maybe labs/coqiad lost internet connection for brief time and cause the bot to lose connection and then node just restarted the client not the job [22:04:51] Zppix node would not have restarted [22:04:59] it would have said RESTARTS 1 [22:05:02] instead of 0 [22:05:21] whats the correct log file name [22:06:17] It would be [22:06:26] https://wikitech.wikimedia.org/wiki/Grrrit-wm#Log_files [22:06:28] Zppix ^^ [22:07:56] error: prefix=leguin.freenode.net, server=leguin.freenode.net, command=err_nosu [22:07:56] chnick, rawCommand=401, commandType=error, args=[grrrit-wm, travis-ci, No such n [22:07:56] ick/channel] [22:07:56] error: prefix=leguin.freenode.net, server=leguin.freenode.net, command=err_nosu [22:07:56] chnick, rawCommand=401, commandType=error, args=[grrrit-wm, travis-ci, No such n [22:07:56] ick/channel] [22:07:56] error: prefix=leguin.freenode.net, server=leguin.freenode.net, command=err_nosu [22:07:57] chnick, rawCommand=401, commandType=error, args=[grrrit-wm, travis-ci, No such n [22:07:57] ick/channel] [22:08:09] Yes but that wont cause it to fail [22:08:41] there a crapton of those errors tho [22:08:43] because it has done it mutiple times so it would have shown the bot disconnecting and reconnecting [22:10:18] why is it erroring anyway [22:10:40] Not sure [22:10:59] thats not a command.. [22:11:41] i know [22:11:45] was only joking [22:12:02] any changes need merging [22:12:04] ? [22:12:25] yea [22:12:55] https://gerrit.wikimedia.org/r/#/c/320294/ [22:16:08] !log merging https://gerrit.wikimedia.org/r/#/c/320294/ [22:16:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lolrrit-wm/SAL [22:16:21] Thanks [22:16:52] tell me when its deployed [22:16:58] cause i wanna watch the logs [22:20:22] !log deploying https://gerrit.wikimedia.org/r/#/c/320294/ [22:20:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lolrrit-wm/SAL [22:20:24] Zppix ^^ [22:20:37] ty [22:20:38] i will restart [22:20:59] Wait [22:21:11] I need to do npm install [22:21:19] since it is installing a new package [22:21:20] woops [22:21:40] oh well a double restart wont hurt it any [22:21:45] @ paladox [22:21:59] ok [22:22:01] yep [22:22:06] ready now? [22:22:09] Nope [22:22:14] ok tell me when [22:22:21] ok [22:22:24] i can restart it [22:22:29] ok [22:22:29] when it is finished npm install [22:22:30] :) [22:22:38] ill check logs right after just let me know when uve done that [22:23:09] Zppix restarted now [22:23:10] (i think its something to do when it starts up im just curious and i also wanna find anything that can be optimised to speed it up [22:23:31] It failed [22:23:34] yep [22:23:35] let me see why it failed [22:23:36] majorily [22:24:53] paladox idk if its swig or node [22:25:02] its swig [22:26:38] Zppix fixed it [22:26:42] i will be submiting a patch [22:26:54] i found why it failed, i put a function as colors instead of colour [22:26:57] color i mean [22:31:33] k [22:31:55] !log deployed https://gerrit.wikimedia.org/r/#/c/322487/ [22:31:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lolrrit-wm/SAL [22:36:14] shoudlnt tool-grrrit on phab and tool-lolrrit on phab be merged together considering they are the same [22:36:17] @ pala [22:36:18] @ paladox [22:36:32] Not sure [22:36:42] depending on how long ago we last merged a patch [22:36:54] they are both the same thing [22:40:40] Yep, but they are two repo's [22:48:04] so [22:48:07] they are the same bot [22:48:08] same repo [22:48:25] Yes, but it is depending on how long we last merged a patch [22:48:32] oh wait, which repos are you talking about [22:48:40] could i have the links please? [22:50:22] https://phabricator.wikimedia.org/diffusion/1982/ [22:50:28] https://phabricator.wikimedia.org/diffusion/TGRT/ [22:51:57] they both looked like the have the same update [22:52:02] (they updated) [22:52:41] do we just keep both or just merge them together and have 1 only [22:52:45] i dont see the need for 2 [22:53:20] Not sure [22:53:32] we carn't merge them, we can only deactivate them, [22:53:33] them [22:54:03] ik i was going to have someone do that [22:54:06] ill ask [22:54:53] Ok