[07:27:42] bd808: so when I look at https://tools.wmflabs.org/?list#!/search/apt-browser I don't see any description, but https://toolsadmin.wikimedia.org/tools/id/apt-browser has a toolinfo with a description and tag...did I not set something up properly? [14:10:46] legoktm: I think they draw from different sources? [14:15:14] legoktm: for what it's worth, the too doesn't show up in hay's directory at all, which is consistent with it not showing up in the admin tool either. [14:19:42] hi! [14:20:19] I'm using redis in a toolslab project for message-passing between celery and django [14:20:59] but it's very unreliable, more than half the messages don't get delivered, and the keys it uses get deleted [14:21:33] so I guess the celery instance is not meant to be used for that? Just for some non-critical caching maybe? [14:22:01] is there a better alternative for message-passing between a web server and a worker (run in a kubernetes pod)? [14:31:31] pintoch: redis on toolforge is mostly meant to be used as a cache. i'm not sure how useful it is as a queue seeing as it's meant to be shared by many tenants and with priority being given to more frequently used data [14:31:57] makes a lot of sense [14:32:26] do you know if there is any alternative such as RabbitMQ or other sorts of brokers? [14:32:29] pintoch: https://wikitech.wikimedia.org/wiki/Help:Toolforge#Redis notes that the instance only has 12 GB of RAM. If you're interested in a Redis instance that you can control I would recommend applying for a Cloud VPS project [14:33:45] right… I might just bypass the issue with some ugly polling on some SQL database instead :-/ [14:34:27] (because my needs are quite basic) [14:36:34] I would be interested in learning more about your use case. We cover the basic shared hosting / k8s pod stuff pretty well, and the more advanced run-your-own-VM case well, but are missing a lot of stuff in the middle. [15:10:29] (03PS1) 10Andrew Bogott: Added new dummy files for labtestpuppetmaster puppet certs [labs/private] - 10https://gerrit.wikimedia.org/r/429430 [15:10:42] (03CR) 10Andrew Bogott: [V: 032 C: 032] Added new dummy files for labtestpuppetmaster puppet certs [labs/private] - 10https://gerrit.wikimedia.org/r/429430 (owner: 10Andrew Bogott) [15:29:23] legoktm: I checked the "webservice" box -- https://toolsadmin.wikimedia.org/tools/id/apt-browser/info/id/274/history?version_id2=391&version_id1=370 -- hopefully that fixes it. [16:26:33] pintoch: just to make sure, did you prefix your keys in redis? tools-redis has 4.6G 'unused' (wasted) memory and 4.5G memory used for kernel file buffer & cache (so can be released at any time), so redis server can use up to 9.1G more out of 15G memory [16:27:34] keys randomly get deleted never occurred to me, and yes I use it for some queues, caches, and expiring data [16:31:08] afaik redis doesn't randomly delete things. it runs out of memory and complains. [16:31:33] (unlike say memcached that does prune to make it fit) [17:40:34] zhuyifei1999_: good point - I still don't know if it's possible to prefix the keys used by celery and the underlying libraries [17:55:23] bd808: ah perfect, ty :) [18:05:07] pintoch: yeah sigh. celery also uses redis pubsub and you might run into other's pubsubs, so I would really recommend using a dedicated redis instance [18:17:04] !log tools SIGTERM tools-bastion-03 PID 6562 tools.zoomproof celery worker [18:17:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [18:17:36] ebernhardson: you can configure redis to push out old data, as far as i know, but i understand it's not the default [18:19:30] (and i think even if you do configure it in that way it's only when space runs out, not just for the hell of it) [18:21:42] https://redis.io/topics/lru-cache [18:26:28] !log tools `$ write` doesn't seem to be able to write to their tmux tty, so echoed into their pts directly: `# echo -e '\n\n[...]\n' > /dev/pts/81` [18:26:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [18:47:07] HareJ: that was a good read, thanks! [20:44:19] I just got accepted into toolforge. I can ssh in but I cant create a tool or do anything in the admin console, it says I need to still request membership. On the list I'm shown as approved. My page was edited to say that I was approved. What am I missing? [20:45:47] DovAlp: have you tried logging out of toolsadmin and then back in again? There is a known issue with that being required. [20:46:09] Hold on let me try [20:46:43] Well would you look at that. Thanks! [20:46:50] nice. [20:47:06] I've had a bug open to find a fix for that for months now :/ [20:47:53] In the mean time someone should add a note to the toolforge wiki help page [20:48:00] The issue is that the permissions system on the toolsadmin side caches data when you login and then never looks again [20:48:31] Oops [20:49:39] fixing that requires doing some deep work with the Django permissions system [20:53:11] DovAlp: I tried to add a note about this issue -- https://wikitech.wikimedia.org/w/index.php?title=Help:Getting_Started&diff=1789673&oldid=1788092 [20:53:57] Perfect! [20:54:02] That should do it [20:54:06] bd808: what if we invalidated the cache upon that kind of change taking place? [20:54:45] HareJ: yes, that's what needed to happen but its a bit more complex than just dropping a memcache key [20:54:50] HareJ: Where is the toolsadmin source located? [20:55:04] https://phabricator.wikimedia.org/source/labs-striker/ [20:55:15] ("striker" is the code name for the software) [20:55:38] Ahh ok [20:56:02] HareJ, DovAlp: there are some notes at T144943 [20:56:03] T144943: Groups and tools only refreshed at login - https://phabricator.wikimedia.org/T144943 [20:58:06] the mechanism in use is related to the django-auth-ldap plugin in use and how I configured it [20:59:56] Is it something that can be solved in striker/settings.py or is a bigger problem? [21:00:20] Sorry, not a django guy I'm flying a little blind here [21:05:11] there's a setting that can turn off the initial cache I believe. But then it needs to be replaced with code that can periodically sync with the state of the LDAP directory. [21:06:00] the easy solution is to simply add a "Your new privileges will be available on next login" to the welcome message :P [21:06:12] That would be a nice easy fix [21:06:17] the tricky bit here is that the auth plugin does not make Django ask LDAP directly when the code asks a question like "is the current user in group X". That still happens in the local mysql db [21:06:54] Platonides: you are honestly a genius. Let's do that now! [21:07:39] Where would we do that? [21:07:57] in that case I must be a lazy genius xD [21:08:06] its Template:ToolsGranted on wikitech. I'm editing now [21:09:52] Programmers are said to be very lazy. They expend all this effort on automating things so that they don't have to do it by hand anymore! [21:10:16] yep [21:10:29] then we spend more time in automation than we would doing it manually each time [21:10:44] https://xkcd.com/1319/ [21:14:48] +bd808: post the commit when your finished. Im curious :) [21:15:47] https://wikitech.wikimedia.org/w/index.php?title=Template:ToolsGranted&diff=1789674&oldid=1771387 [21:16:44] Platonides: along the lines of the XKCD you posted, I try to remember the lessons from https://xkcd.com/1205/ [21:17:03] I actually have links to both of those from my wikitech user page ;) [21:17:34] That looks great. [21:18:45] both are applicable :) [21:20:42] I remembered "there's a good xkcd strip on this theme" → search → find one → return [21:21:39] I'd swear anomie has all XKCDs memorized, but he claims to just be fast with search as well [22:18:36] any good way to "archive" results from a PAWS notebook to phabricator? Basically i feel like the data would live longer if the results were stored in phab [22:19:27] i guess i could upload a pdf [22:21:05] or phab paste? [22:21:25] paste isnt great for a bunch of tables and graphs [22:21:34] ah, true [22:28:00] ebernhardson: https://phabricator.wikimedia.org/P7047 [22:28:10] created a paste.. then attached a PDF to the paste by drag and drop [22:28:24] just like you can drag files into task comments. but inside a paste [22:29:04] ebernhardson: write an output mode that produces Phab style markdown and you win. ;) [22:29:09] you could attach all the files to a single paste that has a little description text [22:31:50] huh, interesting. trying