[11:08:18] Hi guys, why I can not create hard links from my tool folder to /shared folder? (unfortunately I need hard links instead of symlinks). Both folders are on the same filesystem and /etc/sysctl.conf seems to have the same copy protection for symlinks and hard links. So I don't understand why cp -s works and cp -l just doesn't (throws permission error) [11:10:28] the same for ln vs. ln -s [11:10:55] you can’t create hard links to directories [11:10:57] ever [11:11:00] on any Unix system [11:11:18] I'm trying to create hard links to files [11:11:33] (I should mention that before) [11:11:43] oh, okay, it sounded like you wanted to link /shared itself :) [11:13:44] what’s the full command you’re trying to run? I’m still not clear in which direction you want the link to go [11:17:20] https://pastebin.com/065fKcUw [11:20:10] (full command - not working: https://pastebin.com/TvBR8fL9 vs. the same command for symlinks - working: https://pastebin.com/vBsRJsr5) [11:22:24] hm, strace doesn’t give more info either (linkat returns EPERM) [11:22:31] then I’m out of ideas :/ [11:38:51] I've played with it a little and found this: I can ln inside my tool, but not to files in other tools. /shared/pywikibot/ contains symlinks to /data/project/pywikibot/ so this could be it. How to fix permissions of files in /shared/pywikibot or in /data/project/pywikibot (I should the maintainer of both of them as well) in order to allow hard links [11:38:51] ? [11:39:35] *allow hard links from other tools, like /data/project/xyzbot [11:50:28] !log toolsbeta add sssd hiera config for `toolsbeta-test-k8s-master` prefix [11:50:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [11:55:04] And why is there ln to other tools not allowed, but ln -s allowed? [11:56:54] it probably is related to fs.protected_hardlinks [11:57:22] unfortunately /proc/sys/fs/protected_* are all root-readable only [11:58:12] creating symlinks would still be allowed in that case because protected_symlinks applies when following symlinks, not when creating them [11:59:43] This hardlink permission might be intentional, perhaps bd808 knows, what's the reason behind this and how to overcome it when I'm the maintainer of both source and destination? [12:00:36] we won't have that logic in the short term Dvorapa. The logic to detect if a maintainer is the same in both tools and the allow symlinks [12:00:54] why you don't just copy the files? or better, have a shared (external) git repo for both tools? [12:02:20] I thought this could be easily adjustable somehow in pywikibot tool as all bot tools might want to hard link to pywikibot and they can't [12:03:18] This is another option, but Toolforge help recommends not to clone/download local copy when there is /shared directory of the tool [12:04:30] So I wanted to use /shared folder preferably before cloning pywikibot [12:13:12] I’m not sure what benefits you gain by hardlinking pywikibot from /shared [12:13:20] I guess you save storage space [12:13:25] but you don’t get automatic updates that way [12:13:56] you should get automatic updates [12:14:13] only with a symlink, I think [12:14:24] well, depends on how the update is implemented, I suppose [12:14:33] maybe [12:14:38] if it writes to the same file, and no other files are created, you get the update [12:14:43] I wouldn’t rely on that [12:15:23] That's why I have the script to recreate symlinks to /shared every day, but when I tried to change it to hard links, it failed [12:15:50] but why do you need to recreate symlinks? [12:15:58] with symlinks it should be good enough to link the top-level directory once [12:15:59] The only solution for me is then to git clone pywikibot and never bother with /shared anymore [12:16:41] I need to add some of my configuration files into the folder [12:17:14] ah ok [12:18:17] so symlinks worked for me, until I needed to modify one of the pywikibot files directly [12:18:35] then I needed hard links, which doesn't work [14:14:56] hi, we've been suffering issues with our puppetmaster on the traffic labs projects since Jun 25 06:35:55, at that point existing puppet clients began to fail with the following error: "Error: Failed to find traffic-upload-stretch.traffic.eqiad.wmflabs via exec: Execution of '/usr/local/bin/puppet-enc traffic-upload-stretch.traffic.eqiad.wmflabs'" [14:15:19] any change on your side that could explain this behavior? [14:17:14] vgutierrez: Hi. Sounds like the Jessie python3.4 `3.4.2-1+deb8u3` package error we ran into. [14:17:54] hmm /usr/local/bin/puppet-enc isn't there in our systems apparently [14:18:00] root@traffic-upload-stretch:/var/log# ls /usr/local/bin/puppet-enc [14:18:00] ls: cannot access '/usr/local/bin/puppet-enc': No such file or directory [14:18:14] jeh: could you elaborate please? [14:18:28] are you using a stand alone puppet master? [14:20:56] I checked traffic-upload-stretch. It's using the `traffic-puppetmaster.traffic.eqiad.wmflabs` puppet master. [14:21:13] indeed [14:21:20] and it used to work :) [14:22:10] The master runs puppet-enc, the client log message is a report from the master. [14:22:25] if you check /var/log/puppet.log.7.gz in that box you'll see that it was working till Jun 25 06:35, last successful run was on Jun 25 06:05 [14:22:28] hmmm interesting [14:22:57] on the puppet master if you run `apt update` and `apt-get install --only-upgrade python3.4 python3.4-minimal` it'll fix the error you're running into [14:25:20] yep, that fixed the issue [14:26:10] if you're curious, there's more info at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=931044 [15:51:03] !log tools.wikibase-databridge-storybook added hack to update storybook about every hour by running `webservice --backend kubernetes node10 start` and executing the update in the package.json and then sleep for the rest of the hour [15:51:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibase-databridge-storybook/SAL [15:52:40] heh [16:58:11] !log toolsbeta add puppet prefix `toolsbeta-test-k8s-lb` for T215531 [16:58:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [16:58:13] T215531: Deploy upgraded Kubernetes to toolsbeta - https://phabricator.wikimedia.org/T215531 [17:02:07] !log toolsbeta re-creating instance `toolsbeta-test-k8s-lb-01` with more CPU for T215531 [17:02:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [17:03:36] !log toolsbeta updated FQDN `toolsbeta-k8s-master.toolsbeta.wmflabs.org` with 172.16.6.9 (the new LB VM) for T215531 [17:03:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [17:03:39] T215531: Deploy upgraded Kubernetes to toolsbeta - https://phabricator.wikimedia.org/T215531 [17:13:47] !log toolsbeta re-creating instance `toolsbeta-test-k8s-master-1` with more CPU for T215531 [17:13:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [17:13:52] T215531: Deploy upgraded Kubernetes to toolsbeta - https://phabricator.wikimedia.org/T215531