[00:56:33] (03PS2) 10Eileen: Fix conflict handling for initials that are lower case [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160255 [00:56:33] (03PS1) 10Eileen: Fix dedupe to retain state [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160366 (https://phabricator.wikimedia.org/T379684) [01:18:27] (03PS1) 10Eileen: Update geocoder to latest remote [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160374 (https://phabricator.wikimedia.org/T379684) [02:16:07] (03CR) 10Ejegg: [C:03+2] Align optional UI for gravy as we do for dlocal [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1159560 (https://phabricator.wikimedia.org/T397095) (owner: 10Wfan) [02:16:13] (03CR) 10Ejegg: [C:03+2] "recheck" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1159560 (https://phabricator.wikimedia.org/T397095) (owner: 10Wfan) [02:27:25] (03CR) 10Ejegg: [C:03+2] Update CiviCRM image to 1.1.2 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1160294 (owner: 10Jgleeson) [02:35:23] (03Merged) 10jenkins-bot: Align optional UI for gravy as we do for dlocal [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1159560 (https://phabricator.wikimedia.org/T397095) (owner: 10Wfan) [02:37:09] (03CR) 10Ejegg: [V:03+2 C:03+2] Update CiviCRM image to 1.1.2 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1160294 (owner: 10Jgleeson) [02:37:55] eileen: if you're still on, could you give this a glance? https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1160233 [02:40:37] (03PS3) 10Eileen: Fix conflict handling for initials that are lower case [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160255 [02:40:39] (03CR) 10Ejegg: [C:03+2] Fix conflict handling for initials that are lower case [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160255 (owner: 10Eileen) [02:41:40] ejegg: is that part of some moving parts? [02:42:12] it obviously doesn't do much by itself - or is the waitFor already implemented? [02:42:38] eileen: it's implemented in the base class in SmashPig [02:42:51] we're using it for the PendingQueueConsumer, which is outside of CIvi [02:43:20] we pass in a -w flag to trigger it on the SmashPig queue consumer script [02:43:24] ah yeah - just need to bounce around a bit more! [02:43:33] (03CR) 10Eileen: [C:03+2] Support waitForMessages flag for queue consumers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160233 (owner: 10Ejegg) [02:43:42] well I guess if it's already a thing! [02:45:44] thanks! [02:45:58] It'll help get people their login links more quickly for the donor portal [02:56:16] (03Merged) 10jenkins-bot: Fix conflict handling for initials that are lower case [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160255 (owner: 10Eileen) [03:00:20] (03Merged) 10jenkins-bot: Support waitForMessages flag for queue consumers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160233 (owner: 10Ejegg) [05:12:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Refund shows as contribution line without original contribution line - https://phabricator.wikimedia.org/T393036#10926477 (10RKumar_WMF) I am seeing refund as a separate line and also the amount is little different. CID: 67937095. [08:14:16] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Settled Adyen Donation - Settled Gravy Donation - Not Appearing in Civi - https://phabricator.wikimedia.org/T395340#10926876 (10EBrill-WMF) Hi Team, I have another example of this issue from Spain: ZD 1688231 --> donation for EUR 15... [08:22:14] 06Fundraising-Backlog, 07fr-acoustic: Acoustic fields wishlist - discussion - https://phabricator.wikimedia.org/T394089#10926887 (10MSuijkerbuijk_WMF) @AKanji-WMF @Eileenmcnaughton Hi Anil and Eileen, do we have a timeline on this? Thanks! [08:23:00] 06Fundraising-Tech-Roadmap: Pre-renewal notice to Active Annual recurring donors - https://phabricator.wikimedia.org/T393432#10926888 (10MSuijkerbuijk_WMF) Hi team, I want to kick off work with Content team for this piece. Do you have a timeline for this? Thanks [10:41:59] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Donations Set Up in May Failing Second Instalment (Japan/Gravy) - https://phabricator.wikimedia.org/T397002#10927454 (10jgleeson) It looks like the recurring bug we found yesterday has a wider reach than just Japan. We're now seeing it in other countries... [12:00:03] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Donations Set Up in May Failing Second Instalment (Japan/Gravy) - https://phabricator.wikimedia.org/T397002#10927818 (10jgleeson) After digging into this, the issue is becoming clear. Using Brazil as an example. For standard card payments routed to dLoca... [12:03:43] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Disable Fiscal Number checks on recurring Google Pay & Apple Pay charges - https://phabricator.wikimedia.org/T397322 (10jgleeson) 03NEW [12:04:41] (03PS1) 10Jgleeson: Test to confirm failing app payments in Brazil [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) [12:40:04] (03PS21) 10Damilare Adedoyin: WIP: Create donor portal page for recurring donation pause process. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1154315 [12:40:10] (03PS22) 10Damilare Adedoyin: Create donor portal page for recurring donation pause process. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1154315 [13:22:56] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1160769 [13:25:25] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1160769 (owner: 10Ejegg) [13:27:04] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1160769 (owner: 10Ejegg) [13:29:06] (03CR) 10Reedy: [C:03+2] build: Updating brace-expansion to 1.1.12 [extensions/DonationInterface] (REL1_39) - 10https://gerrit.wikimedia.org/r/1157126 (owner: 10Libraryupgrader) [13:29:40] (03CR) 10Reedy: [C:03+2] build: Updating brace-expansion to 1.1.12 [extensions/FundraisingEmailUnsubscribe] (REL1_39) - 10https://gerrit.wikimedia.org/r/1157353 (owner: 10Libraryupgrader) [13:29:44] (03CR) 10Reedy: [C:03+2] build: Updating brace-expansion to 1.1.12 [extensions/FundraisingTranslateWorkflow] (REL1_39) - 10https://gerrit.wikimedia.org/r/1157357 (owner: 10Libraryupgrader) [13:45:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Unable to link donors via relationships in CiviCRM - https://phabricator.wikimedia.org/T394702#10928244 (10Ejegg) That's a pretty old custom field in our database. Despite being marked as required it only has 311 values... [13:48:29] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Unable to link donors via relationships in CiviCRM - https://phabricator.wikimedia.org/T394702#10928287 (10Ejegg) @JSimmons Sorry I tagged your sound-alike in the last question [13:54:54] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Unable to link donors via relationships in CiviCRM - https://phabricator.wikimedia.org/T394702#10928341 (10JSimmons) Hi @Ejegg - I'm not certain what the 'solicitor level' is. This was a new error Rosie and I encountered... [13:57:14] 06Fundraising-Backlog: Direct Mail NCOA Updates - https://phabricator.wikimedia.org/T392252#10928365 (10MDemosWMF) @Eileenmcnaughton if you have some guidance on the above that would be great. I can also come to civi fortnightly next week so we can create the import template together if needed. [13:59:10] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Unable to link donors via relationships in CiviCRM - https://phabricator.wikimedia.org/T394702#10928370 (10Ejegg) Ah, I see, so you were just adding it because it wouldn't let you add the relationship otherwise. It looks... [14:02:10] (03PS1) 10Damilare Adedoyin: Add maintenance script to get transaction details [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160788 [14:02:37] (03CR) 10CI reject: [V:04-1] Add maintenance script to get transaction details [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160788 (owner: 10Damilare Adedoyin) [14:02:59] (03PS2) 10Damilare Adedoyin: Add maintenance script to get transaction details [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160788 [14:17:09] (03PS1) 10Jforrester: build: Update MediaWiki requirement to 1.45 [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1160798 [14:38:04] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Unable to link donors via relationships in CiviCRM - https://phabricator.wikimedia.org/T394702#10928534 (10RLewis) @Ejegg thanks for the ping on this, I still can't add relationships via this way due to the solicitor fi... [14:45:57] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Unable to link donors via relationships in CiviCRM - https://phabricator.wikimedia.org/T394702#10928554 (10Ejegg) @RLewis I set it to 'not required' about 45 minutes ago. Have you tried since then? If not, want to try ag... [14:53:12] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16014 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [14:55:56] (03CR) 10Jgleeson: [C:03+2] "This will be really handy. Thanks!" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160788 (owner: 10Damilare Adedoyin) [14:56:08] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16188 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [14:56:14] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16192 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [14:57:46] thanks jgleeson! [14:58:14] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16314 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [14:59:38] !log civicrm upgraded from 63302c18 to 10eac2f8 [14:59:41] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:00:18] 06Fundraising-Backlog, 06FR-donorrelations: Request to Revert field back to what it was - https://phabricator.wikimedia.org/T397342 (10SHust) 03NEW [15:00:55] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07Epic: EPIC: placeholder for MG Prospecting - https://phabricator.wikimedia.org/T230804#10928624 (10AKanji-WMF) 05Open→03Declined We can start a fresh phab for this as it may relate to FY2526 work and beyond. [15:01:08] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16488 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [15:01:14] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16492 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [15:01:26] (03Merged) 10jenkins-bot: Add maintenance script to get transaction details [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160788 (owner: 10Damilare Adedoyin) [15:03:13] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16614 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [15:03:17] 06Fundraising-Backlog: graceful handling of donors when payments wiki does not offer a payment option - https://phabricator.wikimedia.org/T254259#10928632 (10AKanji-WMF) 05Open→03Declined Payment orchestration will cover this. [15:06:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16792 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [15:06:21] weird, I thought I had that right, but "Unknown api parameter: setWaitForMessages" [15:06:45] when I try wmf-cv api4 -vv WMFQueue.Consume timeLimit=595 queueConsumer=NewChecksumLink queueName=new-checksum-link waitForMessages=1 [15:07:23] hi fr-tech, Donor Relations is asking about monitoring recurring subscriptions. I know we have a dashboard at https://frmon.wikimedia.org/d/e0a72c1b-cd9e-4a25-b781-c981d51f58db/jack-big-english-dash?orgId=1&from=now-7d&to=now&timezone=utc&refresh=1m&viewPanel=panel-35 in Grafana, but it's likely not suitable for teams outside of fr-tech. [15:07:23] This made me think, could we develop a user-friendly version of our fr-tech dashboards? We could then share this with other teams to help them monitor key touchpoints in the system that we already track. There's a lot of good data available, it's just a matter of presenting it in a useful format. As a follow-on step, we could add alerting, ideally for us first, and then for other interested parties. [15:08:07] jgleeson: if we could get some dashboards in Civi pulling from the data lake that would be great [15:08:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16914 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [15:08:18] I know eileen has at least expressed that wish before [15:08:33] (03CR) 10Reedy: [C:03+2] build: Update MediaWiki requirement to 1.45 [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1160798 (owner: 10Jforrester) [15:09:46] ejegg: I've not seen any of that yet so can't comment. I was thinking along the lines of opening up our system stats that we record and Prometheus scrapes. Do we have similar data in the date lake? [15:11:03] ah right, probably not. I think that's all derived from the db so far [15:11:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17092 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [15:13:13] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17214 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [15:16:09] ejegg: do you know if we can add alerting yet in grafana? I feel like Jeff_Green and dallas talked about that maybe being available recently [15:16:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17393 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [15:16:26] I haven't heard any more about that jgleeson [15:16:39] jgleeson: we haven't gotten to it yet [15:16:54] oh weird, NOW the api call works [15:17:08] hmm, maybe it was running in wmff? [15:17:15] btw all these check_mysql alerts are because I changed the way we're doing database dumps, I adjusted monitoring but it's still rolling out [15:17:16] I just updated that codebase [15:17:23] thanks Jeff_Green [15:18:15] RECOVERY - check_mysql on frdb1004 is OK: Uptime: 1159187 Threads: 4 Questions: 65429519 Slow queries: 646 Opens: 3745 Open tables: 1294 Queries per second avg: 56.444 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 13736 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [15:18:51] ejegg: the failure streaks in the middle of the week on this graph stand out. I wonder why the failures tail in on the end of each run? https://frmon.wikimedia.org/d/e0a72c1b-cd9e-4a25-b781-c981d51f58db/jack-big-english-dash?orgId=1&from=now-7d&to=now&timezone=utc&refresh=1m&viewPanel=panel-35 [15:19:13] do we roll over previous failures which then inflates the next run (if they fail again) [15:19:20] I guess we do [15:19:49] and then it likely processed them new-to-old [15:19:57] based on that graph [15:21:29] !log enabled queue consumer to send donor portal login links [15:21:32] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:29:03] 06Fundraising Tech Chaos Holding Pen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Email edit ui: allow users to send draft MC emails to themselves - https://phabricator.wikimedia.org/T302444#10928727 (10AKanji-WMF) @NNgu-WMF is this something we still hope to implement? [15:33:33] (03Merged) 10jenkins-bot: build: Update MediaWiki requirement to 1.45 [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1160798 (owner: 10Jforrester) [15:34:01] (03PS1) 10Jforrester: build: Update MediaWiki requirement to 1.45 [extensions/FundraisingTranslateWorkflow] - 10https://gerrit.wikimedia.org/r/1160849 [15:35:10] Jeff_Green or dwisehaupt: could one of you please apply this puppet patch in my homedir on frpm1002? ~/puppet/0001-Add-SmashPig-config-for-login-link-queue.patch [15:35:18] thanks! [15:35:54] sure. [15:38:01] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Unable to link donors via relationships in CiviCRM - https://phabricator.wikimedia.org/T394702#10928768 (10RLewis) @Ejegg - this works now, thank you so much, [15:38:20] ok. merged in. it'll make it out on the next puppet runs. [15:39:13] thanks dwisehaupt [15:55:54] hmm, still getting 403 forbitten on getDonorSummary [15:56:21] using the exact same checksum / params / api_key as works on getCommunicationsPreferences [15:57:10] (03PS23) 10Damilare Adedoyin: WIP: Create donor portal page for recurring donation pause/cancel process. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1154315 [15:59:46] (03PS1) 10Damilare Adedoyin: Create donor portal page for recurring donation cancel process. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160869 (https://phabricator.wikimedia.org/T388749) [16:00:19] (03PS24) 10Damilare Adedoyin: Create donor portal page for recurring donation pause process. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1154315 (https://phabricator.wikimedia.org/T388749) [16:00:28] (03PS2) 10Damilare Adedoyin: Create donor portal page for recurring donation cancel process. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160869 (https://phabricator.wikimedia.org/T388749) [16:00:39] (03PS3) 10Damilare Adedoyin: WIP: Create donor portal page for recurring donation cancel process. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160869 (https://phabricator.wikimedia.org/T388749) [16:01:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20088 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [16:01:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20092 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [16:03:49] oops, meeting [16:04:57] ACKNOWLEDGEMENT - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20088 Dwisehaupt known - backup recovery https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [16:04:57] ACKNOWLEDGEMENT - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 20092 Dwisehaupt known - backup recovery https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [16:05:54] (03CR) 10CI reject: [V:04-1] WIP: Create donor portal page for recurring donation cancel process. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160869 (https://phabricator.wikimedia.org/T388749) (owner: 10Damilare Adedoyin) [16:23:58] (03PS1) 10Jforrester: build: Update MediaWiki requirement to 1.45 [extensions/LandingCheck] - 10https://gerrit.wikimedia.org/r/1160886 [16:43:14] 06Fundraising-Backlog: URENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350 (10SHust) 03NEW [16:51:09] RECOVERY - check_mysql on frdb2004 is OK: Uptime: 1885538 Threads: 4 Questions: 191578452 Slow queries: 877 Opens: 5114 Open tables: 1340 Queries per second avg: 101.604 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 9600 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [16:54:45] jgleeson|food: can I close this one? https://phabricator.wikimedia.org/T387289 if not I'll ask about updates in the processor call [17:01:13] RECOVERY - check_mysql on frdb2005 is OK: Uptime: 1727669 Threads: 4 Questions: 166166089 Slow queries: 811 Opens: 4807 Open tables: 1322 Queries per second avg: 96.179 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8438 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [17:03:46] 06Fundraising-Backlog, 06FR-donorrelations: URENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350#10929077 (10SHust) [17:04:48] ejegg: I had no memory of that one. was trying to see where it got to [17:12:03] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: URENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350#10929103 (10AKanji-WMF) [17:12:42] 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: Request to Revert field back to what it was - https://phabricator.wikimedia.org/T397342#10929106 (10AKanji-WMF) [17:13:11] 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: Snooze field needs fixing - https://phabricator.wikimedia.org/T397251#10929107 (10AKanji-WMF) [17:13:28] 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: Deduper interface needs resizing - https://phabricator.wikimedia.org/T397250#10929108 (10AKanji-WMF) [17:16:35] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: URGENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350#10929120 (10AKanji-WMF) p:05Triage→03High [17:17:08] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: URGENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350#10929130 (10AKanji-WMF) Confirming I'm seeing the same error. [17:25:46] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: URGENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350#10929177 (10Dwisehaupt) ` 10:20 < dwisehaupt> not sure if these are related, but in a log file i am seeing:... [17:36:30] (03CR) 10Damilare Adedoyin: [C:03+2] "Thanks for the quick fix wfan. Looks good to me." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160308 (https://phabricator.wikimedia.org/T397199) (owner: 10Wfan) [17:41:36] 06Fundraising-Backlog, 10fundraising-tech-ops, 10Wikimedia-Fundraising-CiviCRM, 10Fr-drupal-upgrade-2025: Update sync-to-staging to also sync the angular files - https://phabricator.wikimedia.org/T396693#10929235 (10Dwisehaupt) These files/dirs are also sync'd to the off colo host, so we will retain them i... [17:45:34] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Cleanup old custom fields - https://phabricator.wikimedia.org/T226461#10929256 (10Ejegg) Let's see if we can clean up the 'Solicitor Level' custom field on relationships as well - see {T394702} [17:46:12] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Unable to link donors via relationships in CiviCRM - https://phabricator.wikimedia.org/T394702#10929259 (10Ejegg) 05Open→03Resolved a:03Ejegg Noted the field for cleanup on an existing ticket to clean up old fi... [17:53:20] 06Fundraising Tech Chaos Holding Pen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Email edit ui: allow users to send draft MC emails to themselves - https://phabricator.wikimedia.org/T302444#10929290 (10NNgu-WMF) I think it would be great! I'm not in charge of TY email coding any more tho - @ppen... [17:54:54] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: URGENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350#10929293 (10SHust) **Update:** Half of our colleagues have encountered the same issue and aren’t able to lo... [18:01:09] 06Fundraising Tech Chaos Holding Pen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Email edit ui: allow users to send draft MC emails to themselves - https://phabricator.wikimedia.org/T302444#10929313 (10ppenloglou) I'd love to have that option! [18:17:21] (03Merged) 10jenkins-bot: Add redirect text for gravy to align dlocal UI [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160308 (https://phabricator.wikimedia.org/T397199) (owner: 10Wfan) [18:17:46] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: URGENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350#10929339 (10jgleeson) @SHust we've rolled back a deployment that went out last night, and the issue appears... [18:20:30] !log civicrm rolled back from 10eac2f8 to 63302c18 [18:20:33] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:33:18] (03PS1) 10Damilare Adedoyin: Synchronize standalone using wmf [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1160979 [18:38:01] jgleeson: I repurposed the standalone clean sync for this, I can create a new script if that's not what you had in mind for it [18:40:18] (03CR) 10Wfan: [C:03+2] Synchronize standalone using wmf [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1160979 (owner: 10Damilare Adedoyin) [18:40:46] (03CR) 10Jgleeson: [C:04-1] Synchronize standalone using wmf [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1160979 (owner: 10Damilare Adedoyin) [18:40:59] wfan: damilare|brb let's add a new script and leave the original [18:41:04] just copy it sideways [18:41:26] we can then remove the vanilla standalone stuff as a combined unit if needed at a later point [18:42:23] something like scripts/mac/sync-push-civicrm-wmf.sh [18:42:29] good call [18:47:14] (03PS1) 10Wfan: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1160988 [18:47:31] (03CR) 10Wfan: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1160988 (owner: 10Wfan) [19:00:31] !log payments-wiki upgraded from aa102260 to f56db8e6 [19:00:34] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:01:14] brb relocating for call [19:02:20] (03PS1) 10Wfan: Revert "Allow email login" This reverts commit 9f18357a95288a69988886478dfe298f01bf6608. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160998 [19:02:32] thanks wfan ! [19:04:11] (03CR) 10Ejegg: [C:03+2] Revert "Allow email login" This reverts commit 9f18357a95288a69988886478dfe298f01bf6608. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160998 (owner: 10Wfan) [19:04:17] (03CR) 10Ejegg: [V:03+2 C:03+2] Revert "Allow email login" This reverts commit 9f18357a95288a69988886478dfe298f01bf6608. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1160998 (owner: 10Wfan) [19:05:08] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1161000 [19:05:27] (03CR) 10Ejegg: [V:03+2 C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1161000 (owner: 10Ejegg) [19:07:54] !log civicrm upgraded from 63302c18 to 670b3f6b [19:07:57] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:11:37] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 10Fr-drupal-upgrade-2025: URGENT - Donor relations agent unable to access Civi - https://phabricator.wikimedia.org/T397350#10929473 (10SHust) We're back in business, team, thank you! [19:32:23] 10fundraising-tech-ops, 10BDC-Implementation: Service monitoring for FR analytics platform. - https://phabricator.wikimedia.org/T386259#10929524 (10Jgreen) [19:33:14] (03PS1) 10Ejegg: Allow email login [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161015 [19:33:14] (03PS1) 10Ejegg: Temporary hack to allow login without cache clear [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161016 [19:39:01] (03PS2) 10Jgleeson: Disable fiscal_number validation checks for google/apple pay. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) [19:53:52] (03CR) 10Eileen: [C:03+2] Temporary hack to allow login without cache clear [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161016 (owner: 10Ejegg) [19:54:52] (03CR) 10Eileen: [C:03+2] Allow email login [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161015 (owner: 10Ejegg) [20:01:34] if anyone has a chance Sandra would be excited to get this patch https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1160366 [20:12:48] (03Merged) 10jenkins-bot: Allow email login [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161015 (owner: 10Ejegg) [20:14:13] (03Merged) 10jenkins-bot: Temporary hack to allow login without cache clear [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161016 (owner: 10Ejegg) [20:21:39] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: PIX and Boleto after completing the donation do not show TY page - https://phabricator.wikimedia.org/T394979#10929693 (10MBeat33) [20:28:30] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Import missing Gravy ApplePay & GooglePay donations - https://phabricator.wikimedia.org/T395691#10929715 (10MBeat33) belated yet heartfelt thank you for this, @jgleeson I'll send the TY emails for the 6 that still need them [20:35:46] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Unable to load 'change log' - https://phabricator.wikimedia.org/T396040#10929753 (10Dwisehaupt) I'm not sure if it will make a huge difference but I did bump up those two values. Give it another shot and see if there is a change in... [20:36:40] (03PS1) 10Ejegg: Explicit permissions for getDonorSummary [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161039 [20:37:05] eileen: ^^^ thanks for that suggestion! [20:37:46] (03CR) 10Eileen: [C:03+2] Explicit permissions for getDonorSummary [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161039 (owner: 10Ejegg) [20:38:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Import missing Gravy ApplePay & GooglePay donations - https://phabricator.wikimedia.org/T395691#10929767 (10jgleeson) Thank you @MBeat33! Please let me know if you'd like us to extract any data regarding the recurring charges that we were unable to im... [20:39:13] thanks! [20:40:42] (03CR) 10Krinkle: "check experimental" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160191 (https://phabricator.wikimedia.org/T397230) (owner: 10Reedy) [20:40:44] ok, it's finally sending identifier: on staging [20:42:13] (03CR) 10Krinkle: "@Reedy This is using vendor@fundraising/REL1_43 rather than vendor@master. I haven't looked at the failure closely yet, but it might be th" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160191 (https://phabricator.wikimedia.org/T397230) (owner: 10Reedy) [20:42:20] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Gravy PayPal recurring charge failures (INVALID_PARAMETER_SYNTAX) - https://phabricator.wikimedia.org/T387289#10929782 (10Ejegg) 05Open→03Resolved a:03Ejegg I don't see any of these errors in the last couple months. I think Gravy must have fixed... [20:45:47] (03CR) 10Ejegg: "Looks like at least one failure is based on the version of PHP running on CI:" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1160191 (https://phabricator.wikimedia.org/T397230) (owner: 10Reedy) [20:46:12] (03PS3) 10Jgleeson: Disable fiscal_number validation checks for google/apple pay. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) [20:46:38] (03CR) 10CI reject: [V:04-1] Disable fiscal_number validation checks for google/apple pay. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) (owner: 10Jgleeson) [20:46:56] ok fr-tech, that one disabled the fiscal number checks by overriding addCountrySpecificRequiredFields() as discussed on tech-talk [20:47:16] thanks jgleeson - looks like google validator still overrides both methods? [20:47:17] ah CI then disabled my patch [20:47:25] oh rly [20:47:26] lemme see [20:48:06] it does. I'll fix that [20:48:31] (03PS4) 10Jgleeson: Disable fiscal_number validation checks for google/apple pay. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) [20:53:15] (03Merged) 10jenkins-bot: Explicit permissions for getDonorSummary [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161039 (owner: 10Ejegg) [20:54:12] (03PS5) 10Jgleeson: Disable fiscal_number validation checks for google/apple pay. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) [20:56:39] 10fundraising-tech-ops: Migrate backup civi host from civi2001 to civi2002 - https://phabricator.wikimedia.org/T375038#10929848 (10Dwisehaupt) [21:01:59] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Import missing Gravy ApplePay & GooglePay donations - https://phabricator.wikimedia.org/T395691#10929872 (10MBeat33) 05Resolved→03Open Yes, thank you, and if I can help with crafting the .csv please let me know [21:52:05] (03CR) 10Umherirrender: [C:03+2] build: Update MediaWiki requirement to 1.45 [extensions/LandingCheck] - 10https://gerrit.wikimedia.org/r/1160886 (owner: 10Jforrester) [21:52:24] 10fundraising-tech-ops, 10decommission-hardware: Decommission frack hosts: civi2001 - https://phabricator.wikimedia.org/T397380 (10Dwisehaupt) 03NEW [21:52:41] 10fundraising-tech-ops, 10decommission-hardware: Decommission frack hosts: civi2001 - https://phabricator.wikimedia.org/T397380#10930066 (10Dwisehaupt) a:03Dwisehaupt [21:52:53] (03PS1) 10Eileen: Update gitignore [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161055 (https://phabricator.wikimedia.org/T396174) [21:52:54] (03PS1) 10Eileen: Remove drupal support in test bootstrap [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161056 (https://phabricator.wikimedia.org/T396174) [21:52:56] (03PS1) 10Eileen: Update composer.json for new paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161057 (https://phabricator.wikimedia.org/T396174) [21:53:12] 10fundraising-tech-ops, 10decommission-hardware: Decommission frack hosts: civi2001 - https://phabricator.wikimedia.org/T397380#10930075 (10Dwisehaupt) [22:08:12] (03PS1) 10Eileen: Fix path in comments [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161062 (https://phabricator.wikimedia.org/T396174) [22:08:13] (03PS1) 10Eileen: Path fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161063 (https://phabricator.wikimedia.org/T396174) [22:08:15] (03PS1) 10Eileen: Update rpow paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161064 (https://phabricator.wikimedia.org/T396174) [22:13:24] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Queue consume failed with code 137 - (ERROR Killed) on the donations queue consumer and recurring queue consumer - https://phabricator.wikimedia.org/T383937#10930111 (10AnnWF) [22:14:01] (03CR) 10CI reject: [V:04-1] Update composer.json for new paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161057 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [22:17:10] (03PS2) 10Eileen: Fix path in comments [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161062 (https://phabricator.wikimedia.org/T396174) [22:17:19] (03PS2) 10Eileen: Path fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161063 (https://phabricator.wikimedia.org/T396174) [22:17:29] (03PS2) 10Eileen: Update rpow paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161064 (https://phabricator.wikimedia.org/T396174) [22:23:19] 10fundraising-tech-ops, 10decommission-hardware: Decommission frack hosts: civi2001 - https://phabricator.wikimedia.org/T397380#10930117 (10Dwisehaupt) [22:24:54] (03Merged) 10jenkins-bot: build: Update MediaWiki requirement to 1.45 [extensions/LandingCheck] - 10https://gerrit.wikimedia.org/r/1160886 (owner: 10Jforrester) [22:26:47] (03CR) 10CI reject: [V:04-1] Update rpow paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161064 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [22:27:33] (03CR) 10CI reject: [V:04-1] Path fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161063 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [22:30:44] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Queue consume failed with code 137 - (ERROR Killed) on the donations queue consumer and recurring queue consumer - https://phabricator.wikimedia.org/T383937#10930138 (10AnnWF) [22:40:24] (03PS2) 10Eileen: Update composer.json for new paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161057 (https://phabricator.wikimedia.org/T396174) [22:59:39] 06Fundraising-Backlog, 10fundraising-tech-ops, 10Wikimedia-Fundraising-CiviCRM, 10Fr-drupal-upgrade-2025: Update sync-to-staging to also sync the angular files - https://phabricator.wikimedia.org/T396693#10930187 (10Dwisehaupt) [23:14:45] (03CR) 10Wfan: [C:03+2] Path fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161063 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:31:48] (03CR) 10Ejegg: [C:03+2] Update composer.json for new paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161057 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:32:00] (03PS6) 10Jgleeson: Disable fiscal_number validation checks for google/apple pay. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) [23:32:04] (03CR) 10Ejegg: [C:03+2] Disable fiscal_number validation checks for google/apple pay. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) (owner: 10Jgleeson) [23:33:59] (03PS3) 10Eileen: Fix path in comments [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161062 (https://phabricator.wikimedia.org/T396174) [23:34:03] (03CR) 10Ejegg: [C:03+2] Fix path in comments [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161062 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:34:11] (03Merged) 10jenkins-bot: Path fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161063 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:34:40] (03PS2) 10Eileen: Update gitignore [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161055 (https://phabricator.wikimedia.org/T396174) [23:34:46] (03CR) 10CI reject: [V:04-1] Remove drupal support in test bootstrap [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161056 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:34:46] (03CR) 10CI reject: [V:04-1] Update composer.json for new paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161057 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:34:46] (03CR) 10CI reject: [V:04-1] Fix path in comments [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161062 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:34:55] (03PS2) 10Eileen: Remove drupal support in test bootstrap [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161056 (https://phabricator.wikimedia.org/T396174) [23:35:05] (03PS3) 10Eileen: Update composer.json for new paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161057 (https://phabricator.wikimedia.org/T396174) [23:35:15] (03Merged) 10jenkins-bot: Disable fiscal_number validation checks for google/apple pay. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1160746 (https://phabricator.wikimedia.org/T397322) (owner: 10Jgleeson) [23:35:29] (03PS4) 10Eileen: Fix path in comments [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161062 (https://phabricator.wikimedia.org/T396174) [23:35:43] (03PS3) 10Eileen: Update rpow paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161064 (https://phabricator.wikimedia.org/T396174) [23:36:27] (03CR) 10Ejegg: [C:03+2] Update rpow paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161064 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:38:02] (03PS1) 10Ejegg: Update SmashPig to 0.9.6.8 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161073 [23:38:04] (03CR) 10Ejegg: [C:03+2] Update SmashPig to 0.9.6.8 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161073 (owner: 10Ejegg) [23:39:43] (03CR) 10Eileen: "ejegg - making sense of the script is scope creep :-)" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1159638 (https://phabricator.wikimedia.org/T276297) (owner: 10Eileen) [23:45:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Unable to load 'change log' - https://phabricator.wikimedia.org/T396040#10930244 (10Ejegg) Sadly, still timing out. @Eileenmcnaughton if we wanted to change that temp table to be durable rather than in memory, do you know the best w... [23:53:21] (03Merged) 10jenkins-bot: Update gitignore [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161055 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:53:44] (03Merged) 10jenkins-bot: Remove drupal support in test bootstrap [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161056 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen) [23:53:45] (03Merged) 10jenkins-bot: Update composer.json for new paths [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1161057 (https://phabricator.wikimedia.org/T396174) (owner: 10Eileen)