[02:41:01] (03CR) 10Scardenasmolinar: "Whenever I check this patch out with `git fetch ssh://suecarmol@gerrit.wikimedia.org:29418/mediawiki/extensions/ORES refs/changes/83/11967" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1196783 (https://phabricator.wikimedia.org/T407087) (owner: 10Tim Starling) [03:09:20] (03CR) 10Scardenasmolinar: "Just saw the error of my ways. Disregard this." [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1196783 (https://phabricator.wikimedia.org/T407087) (owner: 10Tim Starling) [06:01:28] FIRING: [3x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing [06:52:08] good morning! [07:06:28] FIRING: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing [07:14:13] good morning from here as well! [07:16:45] Good morning [08:31:28] Hola dpogorzelski ! [08:31:38] Hello! [08:36:57] o/ [08:39:04] Morning, Dawid ;) [08:52:34] 06Machine-Learning-Team, 10Wikimedia-GitHub: Add Dawid Pogorzelski to WMF GitHub organization - https://phabricator.wikimedia.org/T407839 (10isarantopoulos) 03NEW [09:19:38] 10Lift-Wing: LiftWing fiwiki-damaging model returning 500 - https://phabricator.wikimedia.org/T407784#11292903 (10BWojtowicz-WMF) I've looked through our [[ https://logstash.wikimedia.org/goto/ace4e035e8f6f55f75f774a93afaabb2 | Logstash ]] hunting for 500 errors for `fiwiki-damaging` in the last month. Indeed in... [09:26:26] 06Machine-Learning-Team, 07Essential-Work: Introduce re-try mechanisms for MW API requests in LiftWing models - https://phabricator.wikimedia.org/T407843 (10BWojtowicz-WMF) 03NEW [10:08:40] * klausman lunch [10:15:58] hi [10:16:23] is there some public chatgpt or some other instance for beta testing of tool for wiki [10:16:38] so self hosted by wmf instead of the public one [10:47:14] 06Machine-Learning-Team, 10Add-Link-Structured-Task, 06Growth-Team: Introduce case sensitivity to machine learning model for Add a Link - https://phabricator.wikimedia.org/T405185#11293213 (10OKarakaya-WMF) hello @KStoller-WMF , and @Trizek-WMF , I think it should not be complicated to get offline scores fo... [11:06:43] FIRING: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing [11:08:48] Hii team, I'm looking for 1 more reviewer besides Aiko to look into the patch introducing Caching for articletopic: https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1176448. When found, I'd schedule a ~30min meeting for the 3 of us so we can go through the code together and discuss live :) [11:22:29] 06Machine-Learning-Team, 10Wikimedia-GitHub: Add Dawid Pogorzelski to WMF GitHub organization - https://phabricator.wikimedia.org/T407839#11293302 (10Aklapper) (To avoid misunderstandings, note that there is no WMF GitHub organization but only a Wikimedia GitHub organization.) [11:25:49] o/ took a first pass at the airflow ML Pipelines docs: https://wikitech.wikimedia.org/wiki/Machine_Learning/Airflow_ML_Pipelines [11:25:49] feel free to review and edit as needed. [11:27:13] o/ gry: Unfortunately, we do not yet serve any LLM instance on LiftWing that could be accessed by the public [11:32:31] I've applied the pending admin-ng changes, so the alerts should clear soon [11:33:47] klausman: thank you <3 [11:42:43] kevinbazira: thank you for the docs, I’ll be a lot of help when developing the backfilling pipeline for the Cache :D I found a couple of very small nits when going through it, will update the page soon [11:43:53] bartosz: glad you find it helpful :) [11:43:54] nits and other updates are welcome [12:00:06] hello, [12:00:06] thank you @kevinbazira : I can add add-a-link to the list [12:00:06] thank you @bartosz : I can join the code review if there are no volunteers yet. [12:00:07] @gry : I'm running ollama (gptoss and more) on mllab2. It's only available within the instance. But I guess it's not what you're looking for :) [12:01:28] FIRING: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing [12:01:33] ozge_: np! feel free to add add-a-link to the list of current ML pipelines in prod [12:06:28] RESOLVED: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing [12:55:52] ozge_: sweet, thank you! [13:27:50] 06Machine-Learning-Team, 10Wikimedia-GitHub: Add Dawid Pogorzelski to WMF GitHub organization - https://phabricator.wikimedia.org/T407839#11293635 (10Reedy) I've sent an invitation. Please make sure you have 2FA enabled (we have no way of checking this before anyone is actually added to the GH org) [15:05:33] 06Machine-Learning-Team, 10Add-Link-Structured-Task, 10Community Feedback (Growth), 06Growth-Team: Introduce case sensitivity to machine learning model for Add a Link - https://phabricator.wikimedia.org/T405185#11294169 (10KStoller-WMF)