Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support lz4 and gzip #458

Open
aklyuk opened this issue Nov 21, 2023 · 6 comments
Open

Support lz4 and gzip #458

aklyuk opened this issue Nov 21, 2023 · 6 comments

Comments

@aklyuk
Copy link

aklyuk commented Nov 21, 2023

Hello. I build docker image from last version carbon-c-relay 3.7.4
I get error:

[2023-11-21 16:21:58] starting carbon-c-relay v3.7.4 (2022-02-13), pid=1
[2023-11-21 16:21:58] /etc/carbon-c-relay/carbon-c-relay.conf:2:26: feature gzip not compiled in
[2023-11-21 16:21:58]   type linemode transport gzip
[2023-11-21 16:21:58]                           ^^^^
[2023-11-21 16:21:58] failed to read configuration '/etc/carbon-c-relay/carbon-c-relay.conf'

On this line i add --with-gzip:
./configure --disable-maintainer-mode --with-gzip && make
but get error

checking for gzopen in -lz_missing_header... no
configure: error: in `/opt/carbon-c-relay-build':
configure: error: --with-gzip was given, but test for gzip failed

How to compile gzip or lz4 support?

@grobian
Copy link
Owner

grobian commented Nov 21, 2023

you need zlib-devel or something similar, such that the configure script can find it

@aklyuk
Copy link
Author

aklyuk commented Nov 22, 2023

Thank.
Finally RUN incruction looks like:

RUN \
  apk add --no-cache build-base zlib-dev lz4-dev && \
  touch conffile.tab.c conffile.tab.h conffile.yy.c \
  configure.ac Makefile.am aclocal.m4 configure \
  Makefile.in config.h.in && \
  ./configure --disable-maintainer-mode --with-gzip --with-lz4 && make

But i get error:
/etc/carbon-c-relay/carbon-c-relay.conf:13:0: cannot use UDP transport for compressed/encrypted stream

It is true, carbon-c-relay don't work with inbound compressed udp?

my config is:

listen
  type linemode transport gzip
    127.0.0.1:2003 proto udp
  ;

@grobian
Copy link
Owner

grobian commented Nov 22, 2023

compression is done in streams, if you lose one packet, you'd not be able to do anything, same for encryption, hence the requirement for TCP

@aklyuk
Copy link
Author

aklyuk commented Nov 23, 2023

compressed tcp is working.
But, i get this message:
buffer overflow during read of gzip stream

carbon-c-relay is running in docker.
I have increased sysctl's parameters - net.ipv4.tcp_rmem, net.core.rmem_max to 1Gb, but no changes.
I have build docker-image with it ENTRYPOINT instruction -
ENTRYPOINT ["/usr/bin/carbon-c-relay", "-f", "/etc/carbon-c-relay/carbon-c-relay.conf", "-U", "268435456"] - no changes.

What can i do?

@grobian
Copy link
Owner

grobian commented Nov 25, 2023

hmmm, who is generating/sending the gzipped stream?

@aklyuk
Copy link
Author

aklyuk commented Nov 27, 2023

sending is carbon-c-relay v3.3
receiving is carbon-c-relay v.3.7.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants