[11:49:32] !log tools introduce placeholder DNS records for toolforge.org domain. No services are provided under this domain yet for end users, this is just us testing (SSL, proxy stuff etc). This may be reverted anytime. [11:49:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [11:49:56] Hello. I need to create "profiles" for users of my tool. Now it is ~200 users, but in furure can be more. How do you think, I can do it via tables in db of tool in toolsdb? (1 user = 1 table) Or is it too many tables? Are there any technical and "ethical" limitations on Toolforge? [11:50:17] Iluvatar_: yes [11:50:27] let me find some docs [11:51:42] first, make sure what you are planning fits into our terms of use: https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use#What_can_and_can%E2%80%99t_be_done_with_user_information? [11:53:36] second, make sure you take into account that toolsdb is a shared database, with limited resources for many users [11:54:30] you have restricted number of connections and you should plan your app to handle being throttled from the DB side [12:12:11] I collect only usernames for public statistics, log of actions in-wiki via tool, log of internal chat (2 week), and personal settings of tool. My tool like “online Huggle” (for reverts vandalism). How do you think, its normal if in privacy agreement explained it? I should remove all users settings and count of actions after 30 days? [12:12:21] arturo: [12:14:00] it sounds good honestly. But also I'm no expert and I would recommend to open a phab task to discuss with the WMCS team if you still have doubts and double check that everything will be OK before doing the changes [12:14:04] Iluvatar_: ^^^ [12:15:47] Ok, thank you. [12:55:07] !log admin [codfw1dev] created a new subnet neutron object to hold the new CIDR for floating IPs (cloud-codfw1dev-floating - 185.15.57.0/29) T239347 [12:55:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [12:55:10] T239347: create a 'normal' network for codf1dev neutron w/public IPs - https://phabricator.wikimedia.org/T239347 [13:46:11] <[1997kB]> What is the maximum count of tables in toolsdb on Labs? [14:00:17] <[1997kB]> nevermind, I think one of my colleague dev already asked that. [15:01:06] Technical Advice IRC meeting starting in 60 minutes in channel #wikimedia-tech, hosts: @CFisch_WMDE - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [15:28:30] Iluvatar_: a table per user does not sound like something that will scale well at all. A table per "thing" that you are storing with a relation to a users table for each row is probably a better idea. [15:39:01] We plan to make it possible for users to create their own settings presets. Therefore, it way also does not fit :(. Hmm... [15:50:54] Technical Advice IRC meeting starting in 10 minutes in channel #wikimedia-tech, hosts: @addshore - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [16:15:49] Ok, I really hate IRC; I guess I left this message on the wrong channel, sorry ... [16:16:08] @zhuyifei1999_ - thanks, I'll have to look into strace. The access log doesn't look like it's being "flooded" with only a handful of requests per minute, but maybe that's enough to overwhelm it. The error.log file does show the server occasionally saying "all handlers ... are down" or "backend is overloaded; we'll disable it for 1 seconds and send [16:16:08] the request to another backend instead: reconnects: 1 load: 262 " which sounds ominous. [16:16:43] And - how does one use 'strace' on a tool? It doesn't seem to be installed on the kubernetes pod??? [16:27:34] !log wikiloop add quota for 2 floating IPs T240414 [16:27:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikiloop/SAL [16:27:37] T240414: Request for Floating/Public IP address for WikiLoop - https://phabricator.wikimedia.org/T240414 [17:04:00] My web service seems to have recovered today - at least so far no crashes! [17:19:22] So I guess I'm realizing I don't have a good grasp of monitoring tools available for cloud applications. Is there anything visible in grafana that's useful? What do people use generally? [17:19:58] The wikitech documentation mentions "Ganglia" which seems to have been discontinued... [17:24:39] apsmith: right now we really do not have any visible monitoring tools for things running on the Kubernetes cluster in toolforge. [17:25:32] Ganglia was used in the past for physical hardware. That has migrated to prometheus and grafana. [17:26:50] the monitoring we do have in the Cloud VPS/Toolforge space today is at https://grafana-labs.wikimedia.org/ and https://shinken.wmflabs.org/ [17:33:36] !log tools updated package in aptly for toollabs-webservice to 0.53 [17:33:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [17:43:27] @+bd808 - Ah, I didn't know there was a special grafana for labs. Shinken seems to require a login? [17:43:49] it is guest/guest :) [17:46:23] Ok... [18:50:19] I will let you know when I see apsmith and I will deliver that message to them [18:50:19] @notify apsmith grafana and shinken monitors the toolforge hosts the kubernetes are running on. I don't think they are what you are looking for. making strace available for the general users is unfortunately still an open bug [19:13:34] !log deployment-prep deploying ores 80b1e62 [19:13:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [19:47:16] !log wikidata-dev hard reboot of wikidata-misc to test recovering from crashes T235069 [19:47:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikidata-dev/SAL [19:47:20] T235069: Termbox in labs should be able to recover crashes - https://phabricator.wikimedia.org/T235069