Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new backuppc package w/ dependency perl-file-rsyncp #5886

Merged
merged 2 commits into from
Aug 30, 2019

Conversation

ghost
Copy link

@ghost ghost commented Apr 8, 2018

Maintainer: me
Compile tested: Marvell Armada, WRT1200AC, master
Run tested: Marvell Armada, WRT1200AC, LEDE 17.01, ran successfully in production for a while

Description:
This branch adds two new packages: backuppc and perl-file-rsyncp, which is needed for the rsync xfer method in backuppc. The backuppc packaging uses ideas from debian's packaging to make things work.

BackupPC is a high-performance, enterprise-grade system for backing up Linux and WinXX PCs and laptops to a server's disk: http://backuppc.sourceforge.net/info.html

The packaging uses a separate user "backuppc" to run the daemon and the web interface. It supports uhttpd and configures HTTP basic auth for the web interface automatically, when uhttpd is used.

@kofec
Copy link
Contributor

kofec commented Feb 11, 2019

Hi, Compilation was successful for raspberry PI 1 but during installation on target platform I have:

.....
It is also recommended to follow the steps in
https://wiki.openwrt.org/doc/uci/uhttpd#securing_uhttpd
to secure access to uhttpd.
Can't load '/usr/lib/perl5/5.28/auto/Cwd/Cwd.so' for module Cwd: Error relocating /usr/lib/perl5/5.28/auto/Cwd/Cwd.so: Perl_my_cxt_init: symbol not found at /usr/lib/perl5/5.28/XSLoader.pm line 87.
 at /usr/lib/perl5/5.28/Cwd.pm line 71.
Compilation failed in require at /usr/lib/perl5/5.28/File/Path.pm line 6.
BEGIN failed--compilation aborted at /usr/lib/perl5/5.28/File/Path.pm line 6.
Compilation failed in require at /usr/share/backuppc/lib/BackupPC/Storage/Text.pm line 44.
BEGIN failed--compilation aborted at /usr/share/backuppc/lib/BackupPC/Storage/Text.pm line 44.
Compilation failed in require at /usr/share/backuppc/lib/BackupPC/Storage.pm line 41.
BEGIN failed--compilation aborted at /usr/share/backuppc/lib/BackupPC/Storage.pm line 41.
Compilation failed in require at /usr/share/backuppc/lib/BackupPC/Lib.pm line 43.
BEGIN failed--compilation aborted at /usr/share/backuppc/lib/BackupPC/Lib.pm line 43.
Compilation failed in require at /usr/share/backuppc/bin/BackupPC line 60.
BEGIN failed--compilation aborted at /usr/share/backuppc/bin/BackupPC line 60.

@ghost
Copy link
Author

ghost commented Feb 11, 2019

Hi,

Can't load '/usr/lib/perl5/5.28/auto/Cwd/Cwd.so' for module Cwd: Error relocating /usr/lib/perl5/5.28/auto/Cwd/Cwd.so: Perl_my_cxt_init: symbol not found at /usr/lib/perl5/5.28/XSLoader.pm line 87.
at /usr/lib/perl5/5.28/Cwd.pm line 71.

@kofec: There is something wrong with your perl installation. Cwd.so is from perlbase-cwd and should be loadable by the perl it has been built against. Are you sure your perl packages are consistently installed from one source?

@kofec
Copy link
Contributor

kofec commented Feb 12, 2019

@nonicknamewasleftforme
Perl was installed from repository.
It is strange but:

root@RpiTestowe:/tmp# perl -MCPAN -e shell
Can't load '/usr/lib/perl5/5.28/auto/Cwd/Cwd.so' for module Cwd: Error relocating /usr/lib/perl5/5.28/au           to/Cwd/Cwd.so: Perl_my_cxt_init: symbol not found at /usr/lib/perl5/5.28/XSLoader.pm line 87.
 at /usr/lib/perl5/5.28/Cwd.pm line 71.
Compilation failed in require at /usr/lib/perl5/5.28/File/Spec/Unix.pm line 4.
BEGIN failed--compilation aborted at /usr/lib/perl5/5.28/File/Spec/Unix.pm line 4.
Compilation failed in require at /usr/lib/perl5/5.28/File/Spec.pm line 21.
Compilation failed in require at /usr/lib/perl5/5.28/CPAN.pm line 6.
BEGIN failed--compilation aborted at /usr/lib/perl5/5.28/CPAN.pm line 6.
Compilation failed in require.
BEGIN failed--compilation aborted.
root@RpiTestowe:/tmp# vi /usr/lib/perl5/5.28/XSLoader.pm
root@RpiTestowe:/tmp# ls -al /usr/lib/perl5/5.28/auto/Cwd/
drwxr-xr-x    2 root     root          4096 Feb 12 08:18 .
drwxr-xr-x   24 root     root          4096 Feb 12 08:21 ..
-rwxr-xr-x    1 root     root         12296 Feb  6 06:39 Cwd.so

@kofec
Copy link
Contributor

kofec commented Feb 12, 2019

@kofec
Copy link
Contributor

kofec commented Feb 12, 2019

It looks like I have resolved issue with perl - I just include the package backuppc to image and the perl is also included as dependency. I have new issue:
"Error: Wrong user: my userid is 0, instead of 864(backuppc)".
I understand that I can change the user in config but I didn't touch configuration (default one) - only image upgrade.

@kofec
Copy link
Contributor

kofec commented Feb 12, 2019

After reinstall:

root@RpiTestowe:~# opkg --force-reinstall install backuppc_3.3.2-1_arm_arm1176jzf-s_vfp.ipk 
Removing package backuppc from root...
stopped process in pidfile '/data/backuppc/log/BackupPC.pid' (pid 1239)
Installing backuppc (3.3.2-1) to root...
Configuring backuppc.

To protect access to the backuppc web interface, HTTP basic authentication in
uhttpd for http://RpiTestowe/cgi-bin/BackupPC_Admin has been configured:
user: backuppc
pass: ........

It is also recommended to follow the steps in
https://wiki.openwrt.org/doc/uci/uhttpd#securing_uhttpd
to secure access to uhttpd.
couldn't do /data/backuppc/log/status.pl: Permission deniedCan't create LOG file /data/backuppc/log/LOG at /usr/share/backuppc/bin/BackupPC line 1862.
root@RpiTestowe:~# vi /usr/share/backuppc/bin/BackupPC

@ghost
Copy link
Author

ghost commented Feb 12, 2019

I have new issue:
"Error: Wrong user: my userid is 0, instead of 864(backuppc)".

My bet would be you tried to run /usr/share/backuppc/bin/BackupPC directly from a root shell? You're supposed to use the init script, which runs it as user backuppc.

couldn't do /data/backuppc/log/status.pl: Permission deniedCan't create LOG file /data/backuppc/log/LOG at /usr/share/backuppc/bin/BackupPC line 1862.

And now you have files lying around that are owned by root and cannot be overwritten when the init script is called through the package reinstall. Please try:
chown -R backuppc:backuppc /data/backuppc && /etc/init.d/backuppc restart

@kofec
Copy link
Contributor

kofec commented Feb 13, 2019

Hi,
Regarding the owner. I think that the answer is here:
https://forum.openwrt.org/t/is-openwrt-a-single-user-os/31283/2

The ROM is built with root:root ownership of everything, but that can be changed by run-once scripts. I recall recently reading a consideration of buildroot or fakeroot or the like to allow generation of ROM images with varied ownership.

That is why after upgrade the whole image with Backuppc integrated I have issue with rights. But the question is why postinstall was not run ?

@ghost
Copy link
Author

ghost commented Feb 14, 2019

Hi,
Regarding the owner. I think that the answer is here:
https://forum.openwrt.org/t/is-openwrt-a-single-user-os/31283/2

The ROM is built with root:root ownership of everything, but that can be changed by run-once scripts. I recall recently reading a consideration of buildroot or fakeroot or the like to allow generation of ROM images with varied ownership.

That is why after upgrade the whole image with Backuppc integrated I have issue with rights. But the question is why postinstall was not run ?

Oh, OK! You're right, the buildroot calls mksquashfs with -root-owned. So changing file/directory ownership in postinstall of opkg has no effect in that case. Bummer. Thanks for catching this.
So, if I cannot fix ownerships at build or install time, I can only do it at runtime. I changed the init script accordingly in commit d2aaade. Would you like to try?

@kofec
Copy link
Contributor

kofec commented Feb 21, 2019

Hi, After try I see 2 issues:

Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc:
Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc: To protect access to the backuppc web interface, HTTP basic authentication in
Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc: uhttpd for http://RpiTestowe/cgi-bin/BackupPC_Admin has been configured:
Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc: user: backuppc
Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc: pass: .............
Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc:
Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc: It is also recommended to follow the steps in
Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc: https://wiki.openwrt.org/doc/uci/uhttpd#securing_uhttpd
Tue Feb 19 09:05:53 2019 daemon.notice procd: /etc/rc.d/S95backuppc: to secure access to uhttpd.
Tue Feb 19 09:05:56 2019 daemon.notice procd: /etc/rc.d/S95backuppc: Can't locate Getopt/Std.pm in @INC (you may need to install the Getopt::Std module) (@INC contains: /usr/share/backuppc/lib /usr/lib/perl5/5.28) at /usr/share/backuppc/bin/BackupPC line 66.
Tue Feb 19 09:05:56 2019 daemon.notice procd: /etc/rc.d/S95backuppc: BEGIN failed--compilation aborted at /usr/share/backuppc/bin/BackupPC line 66.
Tue Feb 19 09:05:57 2019 user.info watchcat: starting all tasks

1 - Password was change
2 - some library is missing - I have to check way

@kofec
Copy link
Contributor

kofec commented Feb 21, 2019

I have add missing perl package so only issue with password changing left:

Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc:
Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc: To protect access to the backuppc web interface, HTTP basic authentication in
Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc: uhttpd for http://RpiTestowe/cgi-bin/BackupPC_Admin has been configured:
Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc: user: backuppc
Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc: pass: ..............
Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc:
Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc: It is also recommended to follow the steps in
Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc: https://wiki.openwrt.org/doc/uci/uhttpd#securing_uhttpd
Wed Feb 20 23:06:36 2019 daemon.notice procd: /etc/rc.d/S95backuppc: to secure access to uhttpd.

@ghost ghost force-pushed the feature/backuppc branch from e96fef3 to 34ea53b Compare February 21, 2019 15:51
@ghost
Copy link
Author

ghost commented Feb 21, 2019

2 - some library is missing - I have to check way

I've added the dependency on perlbase-getopt. Hopefully, this was the last missing one.

1 - Password was change

I don't know, why. It doesn't happen on my device. The init script checks for an existing user "backuppc" in /etc/httpd.conf. Does that exist on yours?

@kofec
Copy link
Contributor

kofec commented Feb 24, 2019

root@RpiTestowe:~# cat /etc/httpd.conf
/cgi-bin/BackupPC_Admin:backuppc:...........................................................

@kofec
Copy link
Contributor

kofec commented Feb 24, 2019

Ok I found why. The file is not backup during upgrade:

root@RpiTestowe:~# sysupgrade -l | grep -i http
/etc/config/uhttpd
/etc/domoticz/scripts/templates/HTTPRequest.dzVents

@ghost
Copy link
Author

ghost commented Feb 26, 2019

Ok I found why. The file is not backup during upgrade:

root@RpiTestowe:~# sysupgrade -l | grep -i http
/etc/config/uhttpd
/etc/domoticz/scripts/templates/HTTPRequest.dzVents

Thanks again! I've added a file /lib/upgrade/keep.d/backuppc to the package to improve on that.

@kofec
Copy link
Contributor

kofec commented Mar 2, 2019

When I want to use rsync I have issue:

root@RpiTestowe:~# perl -MFile::RsyncP -e '1'
Can't locate AutoLoader.pm in @INC (you may need to install the AutoLoader module) (@INC contains: /usr/lib/perl5/5.28) at /usr/lib/perl5/5.28/File/RsyncP/Digest.pm line 47.
Compilation failed in require at /usr/lib/perl5/5.28/File/RsyncP.pm line 45.
BEGIN failed--compilation aborted at /usr/lib/perl5/5.28/File/RsyncP.pm line 45.
Compilation failed in require.
BEGIN failed--compilation aborted

And I have to install:

perlbase-autoloader - 5.28.1-1 - AutoLoader perl module
root@RpiTestowe:~# opkg install perlbase-autoloader
Installing perlbase-autoloader (5.28.1-1) to root...
Downloading http://downloads.openwrt.org/snapshots/packages/arm_arm1176jzf-s_vfp/packages/perlbase-autoloader_5.28.1-1_arm_arm1176jzf-s_vfp.ipk
Configuring perlbase-autoloader.

@ghost
Copy link
Author

ghost commented Mar 2, 2019

root@RpiTestowe:~# perl -MFile::RsyncP -e '1'
Can't locate AutoLoader.pm in @inc (you may need to install the AutoLoader module) (@inc contains: /usr/lib/perl5/5.28) at /usr/lib/perl5/5.28/File/RsyncP/Digest.pm line 47.

yes, I have now also systematically scanned RsyncP for dependencies and included them in the Makefile. I think I have missed so many dependencies, because my build contains "shorewall", which satisfies a lot of them. Thank you for bearing with me.

@kofec
Copy link
Contributor

kofec commented Mar 2, 2019

After upgrade I lost whole configuration - all define hosts are lost

@ghost
Copy link
Author

ghost commented Mar 2, 2019

After upgrade I lost whole configuration - all define hosts are lost

Are the host configuration files (/data/backuppc/conf/$hostname.pl) really gone? Also, what does $(sysupgrade -l | grep backuppc) tell you? Maybe just the file /data/backuppc/conf/hosts was overwritten with the one from the package? This doesn't happen when upgrading the package with opkg. Maybe the behavior is different again when using a sysupgrade image.

@neheb
Copy link
Contributor

neheb commented Jul 2, 2019

What is the status of this?

@ghost
Copy link
Author

ghost commented Jul 3, 2019

What is the status of this?

Well, kofec's last issue I cannot reproduce (the sysupgrade config seems to work for me (tm)) and everything else he reported was fixed. So I think these packages are at least "good enough".

@neheb
Copy link
Contributor

neheb commented Jul 5, 2019

I suggest squashing commits as it makes it easier to review. One commit per package.

@ghost
Copy link
Author

ghost commented Jul 7, 2019

I suggest squashing commits as it makes it easier to review. One commit per package.

Thank you for the review! I applied all your suggestions. When I do rebase/squash should I add your "Reviewed-by"?

@neheb
Copy link
Contributor

neheb commented Jul 8, 2019

Sure

@neheb
Copy link
Contributor

neheb commented Jul 8, 2019

define Build/Prepare
is not good.

Change PKG_BUILD_DIR instead of shuffling it around

@ghost ghost force-pushed the feature/backuppc branch from d7edb4d to cf5ad5e Compare July 8, 2019 05:50
@ghost
Copy link
Author

ghost commented Jul 8, 2019

define Build/Prepare

is not good.
Change PKG_BUILD_DIR instead of shuffling it around

Weird. I seem to remember that this didn't work because of the uppercase letters in the Path. Something must have changed for me, because now it works. I've included the change in the squashed commits I just force-pushed.

Signed-off-by: Carsten Wolff <[email protected]>
Reviewed-by: Rosen Penev <[email protected]>
@ghost ghost force-pushed the feature/backuppc branch from cf5ad5e to fc3884f Compare July 8, 2019 06:28
# ownerships other than root are lost.
preconfigure() {
# create backuppc group and user if needed
source /lib/functions.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be placed before START. See https://github.com/openwrt/packages/blob/master/utils/lxc/files/lxc-auto.init for an example. source throws a warning in shellcheck: SC2039: In POSIX sh, 'source' in place of '.' is undefined.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

# inform user
echo
echo "To protect access to the backuppc web interface, HTTP basic authentication in"
echo "uhttpd for http://${HOSTNAME}/cgi-bin/BackupPC_Admin has been configured:"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This throws a warning in shellcheck: SC2039: In POSIX sh, HOSTNAME is undefined.

I don't know if this applies to busybox sh. If it does, there's probably some function in /lib/functions.sh or network.sh

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know about this one. $HOSTNAME exists in both ash and bash and I've seen it being used in other packages. I've found nothing in /lib and the "hostname" command doesn't exist in openwrt. The only alternative would be to read it from /proc directly, but this doesn't look less ugly to me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading from proc is what cifsd-tools does. It's nice to shut up shellcheck.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I've changed it to use uci to fetch the configured hostname.

@ghost ghost force-pushed the feature/backuppc branch from fc3884f to e805fe4 Compare August 4, 2019 19:34
Signed-off-by: Carsten Wolff <[email protected]>
Reviewed-by: Rosen Penev <[email protected]>
@ghost ghost force-pushed the feature/backuppc branch from e805fe4 to 72b8707 Compare August 7, 2019 22:32
@neheb
Copy link
Contributor

neheb commented Aug 30, 2019

This is overdue. Merging.

@neheb neheb merged commit d990519 into openwrt:master Aug 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants