[06:01:19] !log tools Restarted maintain-dbusers process on labstore1004. (T233530) [06:01:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [06:01:22] T233530: When user create tool via toolsadmin, it doesn't create replica.my.cnf - https://phabricator.wikimedia.org/T233530 [11:32:24] Hello, Is the limit of max. 10 connections to the db (replicas) at the same time, new? [11:38:55] not as far as I’m aware [11:58:18] can the limit be raised? [12:02:16] you can try creating a Phabricator task [12:02:23] though I’m not aware of a request that was grated [12:02:25] *granted [12:02:33] https://phabricator.wikimedia.org/T227462 was declined [12:02:44] https://phabricator.wikimedia.org/T232210 was resolved by fixing a connection leak in the tool [12:03:00] I would recommend looking for connection leaks in your code first :) [12:05:44] i run around 30 scripts, that may explain. maybe i should use two different accounts. [12:05:52] thanks^^ [12:06:58] well.. changing the cron may work as well. [12:08:35] ah, I see ^^ [12:08:52] yeah, spreading them over different times sounds good too [13:28:23] Lucas_WMDE: thanks a lot for your tips about SQL dumps, it helped :) [13:40:33] yay :) [16:28:13] !log tools.admin Add "-quiet" to crontab for updatetools.py to reduce email spam [16:28:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.admin/SAL [16:58:25] !log tools deployed tools-manifest 0.20 and restarted webservicemonitor [16:58:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [18:45:07] bstorm_: fyi, your comment is desired on https://phabricator.wikimedia.org/T208612#5517067 [18:45:34] thx, I'll take a look [18:49:08] :) [19:07:59] !log openstack reconfigure keystone in eqiad1 with apache and mod_wsgi T223907 [19:08:02] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Openstack/SAL [19:08:02] T223907: Set up HA endpoints for keystone, glance, nova, designate apis - https://phabricator.wikimedia.org/T223907 [19:34:07] Hi cloud team - I'm experiencing a funny thing within the analytics project [19:34:31] I have created an instance, then dropped it to recreate it with larger disk space - But the space had not changed in the new instance :( [19:34:42] Could it be because of name being reused? [19:39:24] Hi joal, I wouldn't expect that to happen. What's the VM name and what flavor sizes did you use for the deleted one and new one? [19:40:36] Hi jeh - Thanks for answering - project is analytics, and the instance name is janus1-1 - I create first a medium (I think), then a bigdisk2, finally a xlarge --> available disk space is 20Gb on the three of them [19:40:39] joal and jeh, some image sizes require LVM to add the disk. Sometimes it's just a part of physical disk but not added to the os [19:41:05] Ahhhhhhh! Thanks bstorm_ - Makes sense [19:41:29] bstorm_: Would we have some docs around that (I'm no LVM master) [19:44:28] thanks bstorm_, that is the case here [19:44:46] found this doc https://wikitech.wikimedia.org/wiki/Help:Resize_root_partition [19:45:22] jeh: I'm actually very ok mounting a new partition :) [19:45:35] you can see the additional free space on vda3 using lsblk [19:45:40] https://www.irccloud.com/pastebin/WtoHxSWF/ [19:47:03] Awesome jeh - Will (hopefully) manage my way with that [19:47:20] Thanks again bstorm_ and jeh :) [19:50:20] if you don't need to expand the root device, you should be able to create a logical volume on the existing volume group [19:52:40] quick instructions https://www.irccloud.com/pastebin/9KZnVxJm/ [19:52:52] don't forget to add that to /etc/fstab if you go that route [19:53:25] Awesome jeh :) [20:25:07] if you're around jeh - Looks like I've messed up :( [20:25:25] the janus1-1 instance is stuck in emerg after reboot [20:25:47] I'm thinking it's becasue of my fstab line added [20:26:04] which path did you take? resize root or create a logical volume on the free space? [20:26:24] I created a logical volume [20:26:45] Followed exactly the procedure you pasted, except mounting in /mnt/data [20:27:01] Then I blkid, and used that in fstab [20:27:15] did you check the console log for that instance in horizon? [20:27:29] I did - Indeed it tells me: Dependency failed for File System C?u-oBhs-ZWgj-KAwu-hsa1-71a4-p8j1aY. [20:28:09] * jeh looking [20:31:09] If it's OK with you, I'm going to shut that instance down and see if I can correct the fstab entry [20:31:28] please jeh - It's unoperational as it is :0 [20:31:45] Many thanks for this jeh - I'm sorry for the mess :( [20:40:03] jeh - I can swipe / recreate the instance if you want - let me know [20:40:21] it's OK, the instance is coming back online now [20:40:38] you rock jeh - Mistake in fs-check number, was it? [20:41:40] that and I updated the dump pass parameters for a non / disk [20:41:44] `/dev/vd/vol1 /mnt/data ext4 defaults 0 0` [20:41:56] yeah ok - Sorry for that :( [20:42:03] no problem at all :) [20:42:19] * joal hides for not having played with partitions for long [20:42:24] :) [20:43:02] Connected \o/ [20:43:46] great! [20:45:34] I owe you one jeh - Beer one of those days :) [20:45:48] (or any other beverage you may like) [20:45:57] happy to help :) [20:54:00] Hey does anyone know if meta's globalblocks table is viewable from the replicas (metawiki_p) [20:57:54] I don't think that table lives in metawiki? [20:58:06] Krenair: any idea where i'd find it [20:58:13] already looking, one mo [20:58:35] Zppix: centralauth_p [20:58:59] Krenair: that will give me global ip blocks too? [20:59:56] Zppix, the centralauth_p database contains the table for globalblocks [21:00:10] Krenair: oh :P brain fart [21:00:21] Krenair: i misread centralauth and thought of it as a table for some reason [21:00:21] well, technically the view. [21:00:26] Just like how the centralauth database in production contains the table for globalblocks. [21:00:55] It's actually not a CentralAuth thing but we reuse the database for some reason. [21:01:37] probably not well documented anywhere, but yeah centralauth_p is really the "global" tablespace for most things that go cross-wiki. Basically the database there started for centralauth data and has grown over time to include other things that are cross wiki. [21:01:39] Krenair: thanks anyways been looking for it and finally just decided to ask [21:02:02] Probably meant less grants etc. [21:02:46] IIRC the prod grants work by allowing any table in a DB matching wik%, then a special one for centralauth [21:03:17] That's fine for wiki creations but introducing a new global feature... easiest thing is to stick it in the CA DB