[00:21:09] Anyone knowledgable with en-wiki's systems here? [00:23:04] MasterofPuppets: What's your question? I can't promise I know the answer but I might, and if not someone else might [00:23:29] En-wiki's localhost loopback is suddenly spawning dozens of edits for no apparent reason. [00:23:51] We're not supposed to block it, but most of them have been bad-faith vandalism. [00:24:03] I'm at a loss as to what I should do: https://en.wikipedia.org/wiki/Special:Contributions/127.0.0.1 [00:24:25] hrm, sounds like a bug [00:25:12] Eek [00:25:13] Usually it's XFF issues when cache/proxies start editing with 10.*.*.* [00:25:32] Yeah that sounds like XFF problems [00:25:53] I'd re-block it but I'm not sure what the collateral would be. [00:26:02] LeslieCarr: Could it be the Oceania switch maybe? [00:26:04] Since it's not like a normal rangeblock or anything. [00:26:24] If the Varnishes in ulsfo aren't in the XFF list... [00:26:25] it's possibly ? [00:26:34] Let's correlate times [00:26:39] Let me find out when Mark did that switch [00:26:54] 14:52 mark: Sending text (all wikis) traffic from OC to ulsfo [00:27:07] RoanKattouw: IP started spewing out edits at 15:34 [00:27:12] That's pretty much when this started taking off [00:27:20] There are a few from before that too though [00:27:21] 109.0.128.10.in-addr.arpa name = cp4009.ulsfo.wmnet. [00:27:28] yeah we had some before then [00:27:29] ah yep [00:27:30] But it got much more frequent starting around 14:51 [00:27:31] Yeah, the XFFs have ulsfo ips [00:27:46] that's when we moved all oceana text to ulsfo [00:27:52] 13:51 logmsgbot: mark synchronized wmf-config/squid.php 'Update cache list with ulsfo text caches' [00:28:00] 108, 109, 117, 118, 110 [00:28:01] So that must not have worked well then [00:28:13] Reedy: How'd you find that data? [00:28:38] CheckUser [00:28:54] hah, right [00:29:02] Any ideas on how to minimize this change's impact on our side? Or is blocking it short-term our best option? [00:29:27] It lists an external ip, 127.0.0.1 and then the ulsfo proxy [00:29:33] I expect we'll either fix this quickly or roll the change back [00:29:44] If it didn't kill the chrome tab [00:30:18] IP: 127.0.0.1 XFF: 118.67.X.X, 127.0.0.1, 10.128.0.116 [00:30:33] OK so Mark's squid.php change is incomplete [00:30:44] It only has cp4011, 4012, 4019, 4020 [00:30:57] What? [00:30:57] # ulsfo [00:30:57] '10.128.0.108', # cp4008 [00:30:57] '10.128.0.109', # cp4009 [00:30:57] '10.128.0.110', # cp4010 [00:30:59] '10.128.0.116', # cp4016 [00:31:01] '10.128.0.117', # cp4017 [00:31:03] '10.128.0.118', # cp4018 [00:31:10] Oh, hah [00:31:16] Sorry I found those other ones at the bottom of the file [00:31:24] There's 3 # ulsfo ;) [00:31:33] But, wait, WTF, so the offending IPs are in the squids list [00:31:39] MasterofPuppets: I wouldn't block. Just give a few minutes. [00:31:39] So maybe there's an Apache out of sync or whatever? [00:31:51] Alright, standing by. [00:31:53] Reedy or Roan may be able to fix the issue. And it's nice to know when something isn't quite right, which we'd lose from a user block. [00:32:00] Or IP block, as it were. [00:32:21] MasterofPuppets: We'll try to fix it for a bit, and if we can't figure it out soon I'll probably ask LeslieCarr to roll back the Oceania change [00:32:27] Mark did the obvious/usual changes [00:32:40] * RoanKattouw dshes [00:32:42] Is there a script to check that the Apaches are all synced? [00:32:56] lol [00:33:00] Elsie: the IP's already been blocked once, I unblocked it due to fear of affecting innocent users. [00:33:04] Reedy: :-) [00:33:22] (diff) (hist) . . List of people educated at Haileybury, Melbourne . . 00:07 . . 127.0.0.1 (Talk | block) (Previously blocked) (‎Politics) [00:33:57] OK WTF [00:34:07] All Apaches are up-to-date, at least the ones in mediawiki-installation [00:34:13] Maybe there are live Apaches that aren't in the node list? [00:34:18] Seemingly started 24th october [00:34:33] https://wikitech.wikimedia.org/wiki/Server_Admin_Log [00:34:50] but only yesterday do the XFFs start showing [00:35:17] Where'd you get 24 October? [00:36:03] https://en.wikipedia.org/w/index.php?title=Special:Contributions&limit=500&target=127.0.0.1 [00:36:18] https://en.wikipedia.org/wiki/Special:CheckUser for the last 30 days [00:36:31] (diff) (hist) . . Western Eagles . . 02:18 . . Widzewiak16 (Talk | contribs | block) is the last edit it shows [00:36:40] If you're seeing edits as 127.0.0.1, that's probably bugs in other software? [00:36:46] all breaks chrome [00:37:07] I'm not sure what you're looking at, but I think various tools don't log properly. [00:37:30] https://bugzilla.wikimedia.org/show_bug.cgi?id=34838 [00:37:33] I found a cite. [00:37:41] loads of AFT v5 archiving [00:38:05] Offhand MW seems to have been configured correctly [00:38:07] > echo wfIsTrustedProxy('10.128.0.117') [00:38:09] 1 [00:38:17] On which host? [00:38:24] That was tin [00:38:32] I'll run that cluster-wide now [00:38:35] If it hadn't it would be showing as the squid ips as editing [00:38:56] I think CheckUser data isn't completely reliable. [00:39:05] I'd point at a change from 31 October. [00:40:11] https://meta.wikimedia.org/wiki/Special:Contributions/127.0.0.1 :-) [00:40:13] (Unrelated.) [00:41:06] Hmm [00:41:33] The CheckUser going back to the 24th would match up with a deploy to enwiki [00:41:33] I'm trying to see if localhost is editing on other wikis. [00:41:40] http://tools.wmflabs.org/guc/?user=127.0.0.1 finally loaded. [00:41:51] But most of the edits is/was before the usual deploy time [00:42:14] Deploy time of MediaWiki, you mean? [00:42:29] yeah [00:42:47] en.wikibooks.org also has some recent edits. [00:43:13] And simple. [00:43:29] potentially anywhere the AUS/NZ might edit [00:43:30] Scary [00:43:37] Whodathought? [00:44:15] You think it's the ULSFO change? [00:44:30] Is ArticleFeedback still actively polluting CU data? [00:44:36] I don't see a relevant open bug. [00:44:38] no, it stopped [00:44:45] K. [00:45:19] https://en.wikibooks.org/wiki/Special:Contributions/127.0.0.1 [00:45:35] https://simple.wikipedia.org/wiki/Special:Contributions/127.0.0.1 [00:45:53] en.wikiquote and en.wikiversity [00:46:00] And zh.wikipedia.org. [00:46:03] Sigh. [00:48:50] RoanKattouw: Thoughts? [00:48:57] Hmm [00:49:13] Reedy: CheckUser doesn't tell us what Apache saved it, right? [00:49:40] It stores UA and XFF and IP. [00:49:51] All Apaches in the deploy group report that the ulsfo IPs are trusted [00:49:52] It's all client info, not server info, basically. [00:50:03] Yeah [00:50:32] The frequency is high enough that someone could set up a debugging log for edits. [00:50:36] Maybe. [00:51:21] One focused at en.wiki would work. [00:51:50] Two edits at 00:50. [00:53:49] Yeah I might do that next [00:53:54] RoanKattouw: Do you or Reedy have time to diagnose now? [00:53:59] Yes [00:54:30] I think if you can tail en.wiki edits for a few minutes, you can catch the issue. [00:54:30] Well, it's getting onto 2am here... I need to drive to Utrecht via Gronau later today (post sleep) [00:54:45] I'll take it [00:54:49] Or just add in a quick if IP == check to debug it. [00:54:56] Yeah I'm about to do that [00:54:58] According to one of the other admins the loopback on -en is causing a bunch of autoblocks for legitimate users, as well? [00:55:09] Looking through Pybal manifests to see if there's a missing server somewhere [00:55:22] MasterofPuppets: Example legit user being hit? [00:55:30] Autoblocks get logged when triggered, don't they? [00:56:28] ProcseeBot is flooding the block log... [00:56:37] Elsie: not sure, see https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Edits_from_127.0.0.1 [00:56:43] Jesus, does it does this every day? [00:56:47] do this * [00:58:25] MasterofPuppets: Thanks, commented there. [00:59:43] OK, I found an existing log, xff.log, that logs XFF things [00:59:57] Elsie: thanks. [00:59:58] It confirms a whole bunch of 127.0.0.1 stuff, but no server names [01:00:18] https://en.wikipedia.org/wiki/Special:Contributions/127.0.0.1 - Any idea what's going on here? [01:00:27] Why's a loopback address making edits? [01:00:32] Jeske_Couriano: Roan is looking into it. [01:00:37] It's a bug. [01:01:02] Okay, danke. [01:01:15] no ::1 I guess [01:04:13] MasterofPuppets: Do you know of an example user specifically who's been hit? [01:04:15] I'm deploying some logging code to see where this is coming from [01:04:35] Elsie: I was going off of what Elockid said, but I can dig through RFU or ask him. [01:04:39] oh [01:04:44] probably text varnishes [01:04:46] I looked at RFU. [01:05:01] And I looked at Elockid's contributions. [01:05:03] Hah [01:05:05] I didn't see anything off-hand. [01:05:05] Look at that [01:05:26] mw1046 [CENSORED IP], 127.0.0.1, 10.128.0.116, 10.64.32.107 [01:06:09] So that's cp1055 forwarding for cp4016 forwarding for 127.0.0.1 forwarding for CENSORED IP [01:06:29] Elsie: I'll ask. [01:06:37] I think this is probably misconfiguration of the ulsfo Varnishes [01:06:43] LeslieCarr: Are you still around? I need your help now [01:07:02] RoanKattouw: hey, I called it :P [01:07:26] ori-l: I knew it was ulsfo-related, it's just that the theory was that MW wasn't trusting their XFF :) [01:07:33] Turns out they're generating garbage XFFs [01:08:08] ulsfo has nginx on the same node as varnish [01:08:29] nginx as in IPV6/SSL termination? [01:08:33] yep [01:08:36] Iiiintersting [01:08:41] Let me log XFP too then [01:09:45] XFP? [01:09:57] x-forwarded-proto [01:10:06] RoanKattouw: hey [01:10:08] http header [01:10:12] whatcha need [01:10:19] edits from 127.0.0.1 [01:10:25] possibly related to ulsfo text varnishes [01:10:45] https://en.wikipedia.org/wiki/Special:Contributions/127.0.0.1 [01:10:48] about ten days ago this was an issue with geoip lookups [01:10:56] these were ryan's fixes, in case they help: [01:10:57] https://gerrit.wikimedia.org/r/#/c/90732/ [01:11:00] https://gerrit.wikimedia.org/r/#/c/90738/ [01:11:03] ori-l: It was also an issue with geoip when HTTPS termination was first setu p [01:11:10] in pmtpa [01:11:10] and https://gerrit.wikimedia.org/r/#/c/91091/ [01:11:19] Ironically the pmtpa IP range geolocates to San Francisco, CA [01:11:24] heh [01:12:02] Hmm [01:12:06] So there are some HTTP hits in there [01:12:35] Although one of them is IPv6 [01:12:42] I think that lends reasonable credence to your theory [01:12:50] LeslieCarr: I need you to roll back Mark's ulsfo change [01:13:03] "Reasonable credence" is like a +.5. [01:13:09] LeslieCarr: HTTPS and IPv6 termination in ulsfo breaks XFF, produces bogus XFF headers with 127.0.0.1 in them [01:13:44] LeslieCarr: Until this issue is fixed we cannot have any traffic go through ulsfo, so we have to revert more than Mark's latest change, we have to completely shun ulsfo for text (upload and bits are OK) [01:14:06] s/any traffic/any text traffic/ [01:14:43] RoanKattouw: okay [01:14:49] can you send an email saying that while I do it ? [01:15:06] This isn't a problem with Mark sending enwiki traffic through ulsfo, it's just that that exposed the issue to us. [01:15:09] LeslieCarr: Will do [01:18:36] RoanKattouw: so all text is in generic-map except for mobile-lb [01:18:43] does mobile-lb need to be reverted as well ? [01:19:26] I think that gateway allows edits, right? [01:19:35] Any lb that edits might flow through needs to be reverted [01:19:43] I *think* mobile-lb allows edits, so let's revert it [01:19:58] hrm [01:19:59] okay [01:20:03] MobileFrontend doesn't allow anon edits. [01:20:06] At the app level. [01:20:07] oh yay [01:20:09] cool [01:20:12] so no need to revert [01:20:16] well then that was easy [01:20:21] Even though it really fucking should allow anon edits. :-) [01:20:28] Elsie: file a bug :P [01:21:27] YuviPanda: https://bugzilla.wikimedia.org/show_bug.cgi?id=53069 [01:23:00] it'll take a little bit for everything to drain away, because dns ttl + some resolvers are bad and ignore ttl [01:23:51] OK, good [01:24:06] That should take care of it [01:24:19] Elsie: got it [01:24:26] https://en.wikipedia.org/wiki/User_talk:JG66#autoblock.28s.29 [01:24:35] User that was autoblocked due to this glitch. [01:24:39] RoanKattouw: ^ [01:24:40] MasterofPuppets: Should be fixed now, per LeslieCarr it might take a while for everything to update [01:24:58] RoanKattouw: Ah, what do I tell the masses? [01:24:59] Are we sure there aren't any lingering autoblocks? [01:25:04] just curious, what was the problem? [01:25:18] RoanKattouw: If you could post to WP:AN or WP:VPT... :-) [01:25:19] Elsie: no idea how to check. [01:25:30] !log reverted the ulsfo traffic check, due to editors appearing to come from 127.0.0.1 [01:25:36] and now i am logging offline [01:25:38] I can't remember how autoblocks work in MediaWiki. [01:25:40] i joined near the tail end, something about load balancers? [01:25:40] Elsie: I'll respond to that AN thread that MasterofPuppets started [01:25:42] roan has my phone if needed [01:25:42] Do they hit the main account or just the underlying IP? [01:25:45] Logged the message, Mistress of the network gear. [01:25:48] LeslieCarr: Go for it [01:25:53] Thanks, LeslieCarr. [01:25:57] yw [01:26:01] LeslieCarr: And thanks for sticking around [01:27:18] Elsie: ah, comprehensive list of related bugs there. I don't expect anything to change though [01:27:29] Elsie: although IIRC the error rate / abusefilter problem was fixed a month or so ago [01:27:41] Email sent to ops list [01:27:44] Now responding to VPT, then AN [01:27:45] YuviPanda: The fact that there are no dependencies is revealing. [01:28:42] Elsie: bug dependencies? *shrug*, Mobile Team uses Mingle for pretty much everything - the AbuseFilter issue was tracked there and resolved. [01:28:57] I don't know if the other ones are being worked on (I'm not part of Mobile Web) [01:29:04] YuviPanda: The mobile team seems pretty worryingly detached. [01:29:06] From Wikimedia. [01:29:26] I don't understand disallowing anon edits. [01:29:30] Or using Mingle. [01:29:32] But whatever. [01:29:39] Elsie: well, almost every team uses Mingle now, except for VE and platform and us (mobile app) [01:30:36] Elsie: It's basically impossible to keep track of what we're all working on without Mingle [01:30:40] Elsie: did you follow what happened when Mobile Web uploading was enabled? [01:30:55] Elsie: I guess that's the primary driver behind not allowing anon user edits [01:31:05] YuviPanda: People might edit? [01:31:08] Elsie: I would literally have no idea what is going on in my team, or what I need to work on, without having Mingle or something. [01:31:25] werdna: I was looking at http://www.thoughtworks.com/products/mingle-agile-project-management/ [01:33:17] Maybe one day the desktop software will disallow anon editing. [01:33:47] werdna: I think you mean "without a dashboard." [01:34:05] that's what I meant by "Mingle or something" [01:34:12] Fair enough. [01:34:23] I don't have an issue with people tracking what they're working on. [01:34:40] I do take issue with using non-free software or violating core foundation priniciples. [01:34:46] Principles, even. [01:34:55] now, if you have particular gripes with Mingle specifically, then you're welcome to add them to my list (and all of our lists – there are plenty of things that aren't nice about it). [01:35:06] That particular issue is among the many we all have. [01:35:34] It's probably too internal for me to get involved. They're all terrible. [01:35:37] We use LeanKit where I work. [01:35:52] However, what would be more helpful than whinging about the fact that we use one piece of nonfree software to organise our work on the free software that we do use is if you found something that fit our needs better than Mingle. [01:36:08] Roankattouw: Thanks for getting this resolved so quickly! [01:36:09] we used LeanKit for a while. I don't remember specifically what was wrong with it. [01:36:25] werdna: I think there are more systemic issues with mobile development. [01:36:30] Where it seems to do whatever it wants. [01:36:37] MasterofPuppets: You're welcome! I spun my wheels a bit because I believed the issue was somewhere else, sorry about that. Now writing to VPT, AN next [01:37:11] RoanKattouw: hey, still fixed it within the hour. :) ghoulishly-good support, we have. [01:37:18] werdna: No Architecture Summit for you? [01:38:09] Elsie: I got caught up arranging it with Terry, and the email got lost, and yesterday I said "Oh shit, the deadline was last week" [01:38:39] Elsie: So I emailed RobLa, figured it was worth ~~asking~~ begging for forgiveness. [01:38:55] It took you and Terry weeks to edit the wiki page? :-) [01:39:12] well, I mailed Terry asking about that and January travel in general. [01:39:16] and then the email got lost [01:39:21] and I finally got hold of him yesterday [01:39:25] Ahha. [01:40:17] I played around with the Flow prototype recently. [01:40:23] It made me sad. [01:40:34] I haven't yet figured out how to best channel that sadness. [01:40:58] what about it? [01:41:14] It seems very similar to LiquidThreads. [01:41:24] Elsie: and on "It seems to do whatever it wants", that sounds like a pretty good self-contained, self-motivated team to me. [01:41:37] Elsie: what do you dislike about it that you disliked about LiquidThreads? [01:41:44] Or it sounds as though they've gone rogue. ;-) [01:41:46] the name [01:41:49] ;) [01:41:57] heh [01:41:59] werdna: Well, there's no preview button. [01:42:02] well, that's not really my department, is it? [01:42:03] But there is a "Thanks" button. [01:42:09] Elsie: hey, LiquidThreads had a preview button :p [01:42:10] ... LQT doesn't have a preview button? [01:42:14] It seems to still waste a ton of screen space for a simple reply. [01:42:24] werdna: My point re: LQT was that this was a total rewrite. [01:42:31] Which are often terrible ideas in software development. [01:42:37] And so it's now many man-hours later, [01:42:44] and we seem to have probably regressed from LQT. [01:42:50] Not that LQT was a real charm to use. [01:43:04] I'm supposed to be cautiously optimistic that the backend is better. [01:43:10] And that the frontend can be reworked. [01:43:12] But I dunno. [01:43:31] I think spam and vandalism will be problematic. [01:43:35] well, on missing features, the important thing to note is that this is still software in development. [01:43:49] Is that what prototype means? :-) [01:43:53] Yes, I realize it's in development. [01:44:12] +Elsie -mingle alternatives are redmine or trac http://alternativeto.net/software/mingle/ /me hides again [01:44:12] But if the train is headed west and it should be headed north, I don't really know what to do about that. [01:44:26] Heh. [01:45:16] Elsie: well, better than it heading South :p [01:45:24] Agreed! [01:45:26] Elsie: well, have you engaged on the discussion pages yet? [01:45:33] There are like nine of them. [01:45:45] And it's mostly Brandon responding with "it won't so bad! it's a prototype!" [01:45:51] be so * [01:45:54] My feeling is that they're relatively productive, but that's just my vague impression [01:46:03] They exist, which is nice. [01:46:07] There's feedback being given. [01:46:14] Whether any of it is being acted upon... no idea. [01:46:23] Elsie: well, have a chat with ironholds or quiddity or maryana if you do want to participate constructively [01:46:46] I don't think Oliver or Maryana would like to chat, heh. [01:47:06] Nick probably does, but... again, the whole ship is headed in the wrong direction. [01:47:15] (My understanding is that) our approach is that if you do believe that talk pages need to be more inclusive, and the best way to do this is with a piece of software, then we'll engage with you. [01:47:16] Talking to the people in the press room doesn't really fix that. [01:47:26] How generous. [01:47:42] In 2013, I pulled back both article feedback and VisualEditor. [01:48:15] I think we're trying to learn from past mistakes. Those being both types 1 and 2 – in other words, engaging too much with people who will never agree with us, and not engaging through the whole purpose with people who will work with us. [01:48:19] Because what the Wikimedia Foundation has in mind and what the editing community have in mind don't seem to be in sync in many ways. [01:48:32] werdna: Is there any document anywhere describing what went wrong with LQT? [01:48:37] I can appreciate learning from mistakes. [01:48:42] I don't think anyone's done a postmortem on LQT [01:48:52] Before starting a rewrite of it? [01:48:54] mostly because our examples are really Echo, ArticleFeedback, VisualEditor [01:48:58] Do you see why I'm filled with sadness? [01:49:15] well, from my perspective LQT went wrong in a rather obvious way. [01:49:39] Then a postmortem would be easy to write, presumably. [01:50:00] An underresourced organisation put a singular inexperienced developer on a project, and expected him to be project manager, part-time designer, community liaison and sole software engineer. [01:50:59] despite having no experience whatsoever in design or project management, or any strong conception of what the product was to become. [01:51:29] * Elsie nods. [01:51:33] But then I think about VE. [01:51:39] Or ArticleFeedback. [01:51:55] And I'm not sure whether adding a bunch of people to the mix is helping or just making it a larger waste of resources. [01:52:03] right, that's why I think it's a better idea to focus on those. [01:52:05] Though VE isn't a waste, that isn't my point. [01:53:02] Flow is probably on the same timeline trajectory that fucked VE. [01:53:07] For lack of a better phrasing. [01:53:21] In what way? [01:53:32] If James (or Erik or whomever) hadn't pushed so hard for VE to be deployed January 1, people wouldn't hate it so much. [01:53:37] Err, July 1. [01:53:40] LQT was certainly underresourced, I don't think that's in doubt. I tried to pick up a bunch of LQT bugs a few months ago but found that once users noticed they were creating bugs faster than I could fix, and there was still a backlog of over a hundred [01:53:43] Months are hard. [01:53:51] how it got anywhere in the first place seems like a miracle to me [01:54:03] Flow is slated for... end of this year or beginning of next, isn't it? [01:54:14] * Elsie looks. [01:54:27] Krenair: It's like seven years old or something. [01:54:54] > Limited, opt-in release on select WikiProject discussion spaces [01:54:58] By December 2013, projected. [01:55:39] Of course, "let your voice be heard" links to an LQT page. ;-) [01:56:53] Krenair: yeah. Sigh. LQT. [01:56:56] Definitely a lesson [01:57:09] and LQT is actually 9 years old [01:57:10] for the record [01:57:16] I had 2006 in my head. [01:57:19] I started work on it four and a half years ago [01:57:38] I thought it was worked on as part of GSoC 2006. [01:57:41] 2004? really? I guess that explains a lot [01:58:13] oh no, 2007 actually [01:58:14] you're right [01:58:33] https://www.mediawiki.org/w/index.php?title=Extension:LiquidThreads&dir=prev&action=history [01:58:34] andrew-macbook:LiquidThreads andrew$ git log | tail -n5 | grep Date [01:58:35] I'm going to toss an interesting tangent in here -- Isarra got me thinking about how many wikipedians are not on irc; and how it's a mini cabal -- I'm wondering how silly of an idea it would be to write an IRC backed talk page [01:58:35] Date: Sat Oct 13 00:22:32 2007 +0000 [01:58:52] presumably with a substantially different format [01:58:58] I think we should have a wikipedia-chat [01:59:04] maybe a souped up IRC client [01:59:11] how it's a mini cabal? [01:59:13] For real-time chat? [01:59:13] haha [01:59:15] seriously? [01:59:31] *nods* I was thinking of integrating https://kiwiirc.com/ [01:59:44] I think there are already MediaWiki IRC extensions. [01:59:51] I looked briefly [01:59:51] And there's been lots of talk about real-time collaboration. [02:00:01] Combining EtherPad, etc. with per-article chat. [02:00:02] But... [02:00:09] right; marktraceur has good stuff with the interactive editing thing [02:00:10] it's not an easy problem. :-) [02:00:22] IRC isn't really a cabal. [02:00:22] nope [02:00:29] If anyone can download a client and join. [02:00:36] Or use a web-based client. [02:00:40]