Heplify Docker ERSPAN

[abitrusty]

Installed Homer Setup from here https://github.com/sipcapture/homer/wiki/Quick-Install#-docker-install . All working and getting ERSPAN traffic from a heplify agent installed on another box (not docker)

Now trying to install heplify agent as a docker container on same box to collect ERSPAN traffic. I've got the agent container installed but its not sending any traffic to Homer.

Is this something that should work? I've tried changing the {HOMER_DST} to 127.0.0.1 and also 172.18.0.8 - still no traffic in Homer. Any advice is greatly apprecaited

My docker compose file is below

_version: '2.1'

services:

heplify:
container_name: heplify-client
image: sipcapture/heplify:latest
user: 1000:1000
cap_add:
- CAP_NET_ADMIN
- CAP_NET_RAW
command:
./heplify -e -erspan -hs ${HOMER_DST}:9060 -m SIP -dd -l info
network_mode: host
restart: unless-stopped_

Portainer screenshot

[lmangani]

{HOMER_DST} should be heplify-server based on your example and you should let Docker handle the IP resolution and routing. If nothing is captured, nothing gets sent. Please attach more logs or traces if you need further assistance. It could be anything (additional encapsulations, etc)

[abitrusty]

Yes i've tried putting in 127.0.0.1 and also the internal docker IP of 172.18.0.8 but still the same. Its possible no traffic is captured. SIP traffic is 100% getting to the mail IP of the Ubuntu server running Docker

I'm struggling to debug heplify in docker as I;m not that familiar with it. I can debug standalone with -e flag but can't seem to log anything for this docker container :-(

[abitrusty]

Seeing this finally in logs - saying don't have permission to perform capture on this device?

[lmangani]

Your container doesn't have sufficient privileges'.

[abitrusty]

Any ideas how to give the container permission? I've used docker compose logged in as root to install the container......

Critical: setting pcap live mode: ens162: You don't have permission to perform this capture on that device (socket: Operation not permitted)

[lmangani]

If you have access to the running container try to manually run setcap which should provide those permissions:

/usr/sbin/setcap cap_net_raw,cap_net_admin=eip /heplify

Since the example is using docker-compose v2 you can also add the following to the container in compose:

   privileged: true

[abitrusty]

thanks - I tried the "privileged: true" and re-deployed but no joy.

I'll research how to access the running container and run that command. Thanks for support

[lmangani]

You can also simply rebuild the container without the restrictions. This extra layer was recently added.

Save this as Dockerfile

FROM golang:alpine3.19 as builder
RUN apk --update add linux-headers musl-dev gcc libpcap-dev ca-certificates git
RUN apk --update add --no-cache git build-base 
RUN git clone https://luajit.org/git/luajit-2.0.git \
 && cd luajit-2.0 \
 && git checkout v2.1 \
 && make CCOPT="-static -fPIC" BUILDMODE="static" && make install
RUN git clone https://github.com/sipcapture/heplify.git /heplify
WORKDIR /heplify
RUN CGO_ENABLED=1 GOOS=linux go build -a --ldflags '-linkmode external -extldflags "-static -s -w"' -o heplify .

FROM alpine:3.19
RUN apk --no-cache add ca-certificates tzdata libcap
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=builder /heplify/heplify .
CMD ["./heplify", "-h"]

Run docker build -t local/heplify . and test using the local/heplify container image to see if there's any difference.

[abitrusty]

I built the container image using above (amended the CMD line to what i need) but when i try to run it - it doesn't seem to know anything about heplify... think I'd be better re-installing everything on Debian and installing dock and everything from root?

[naqashsaeed]

I also tried the test, and for me it is working fine.

root@debian:~# docker build -t local/heplify .
[+] Building 647.1s (16/16) FINISHED                                                                                                                                          docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                    0.0s
 => => transferring dockerfile: 784B                                                                                                                                                    0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 1)                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/alpine:3.19                                                                                                                          3.8s
 => [internal] load metadata for docker.io/library/golang:alpine3.19                                                                                                                    7.0s
 => [internal] load .dockerignore                                                                                                                                                       0.0s
 => => transferring context: 2B                                                                                                                                                         0.0s
 => [builder 1/7] FROM docker.io/library/golang:alpine3.19@sha256:0642d4f809abf039440540de1f0e83502401686e3946ed8e7398a1d94648aa6d                                                     44.4s
 => => resolve docker.io/library/golang:alpine3.19@sha256:0642d4f809abf039440540de1f0e83502401686e3946ed8e7398a1d94648aa6d                                                              0.1s
 => => sha256:0642d4f809abf039440540de1f0e83502401686e3946ed8e7398a1d94648aa6d 9.02kB / 9.02kB                                                                                          0.0s
 => => sha256:fba34f32048b9afd8400b225a0daecf72db210f107063e876a0e32b46e7b9259 1.92kB / 1.92kB                                                                                          0.0s
 => => sha256:6687662e35d19c9fcdb4ecbbdc207017a235b0d48824796b2da171c051213968 2.09kB / 2.09kB                                                                                          0.0s
 => => sha256:2e3b6a65545cdfcfb5d67cdfda49607bb78f2d64fb91a141ee6be0a339d70a70 292.87kB / 292.87kB                                                                                      2.3s
 => => sha256:32a2f51ff3dde07bfa1ce35b5597b2d97295e64a461d98e696feda7b25a6dc5f 69.35MB / 69.35MB                                                                                       36.3s
 => => sha256:7c599f24d0adc43f062751b364285e0fe35ebc3f72502fee9f6c09d95bd3eaec 123B / 123B                                                                                              3.4s
 => => extracting sha256:2e3b6a65545cdfcfb5d67cdfda49607bb78f2d64fb91a141ee6be0a339d70a70                                                                                               0.2s
 => => sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B / 32B                                                                                                3.7s
 => => extracting sha256:32a2f51ff3dde07bfa1ce35b5597b2d97295e64a461d98e696feda7b25a6dc5f                                                                                               7.2s
 => => extracting sha256:7c599f24d0adc43f062751b364285e0fe35ebc3f72502fee9f6c09d95bd3eaec                                                                                               0.0s
 => => extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1                                                                                               0.0s
 => [stage-1 1/4] FROM docker.io/library/alpine:3.19@sha256:af4785ccdbcd5cde71bfd5b93eabd34250b98651f19fe218c91de6c8d10e21c5                                                            0.0s
 => CACHED [stage-1 2/4] RUN apk --no-cache add ca-certificates tzdata libcap                                                                                                           0.0s
 => [builder 2/7] RUN apk --update add linux-headers musl-dev gcc libpcap-dev ca-certificates git                                                                                      39.9s
 => [builder 3/7] RUN apk --update add --no-cache git build-base                                                                                                                      123.5s
 => [builder 4/7] RUN git clone https://luajit.org/git/luajit-2.0.git  && cd luajit-2.0  && git checkout v2.1  && make CCOPT="-static -fPIC" BUILDMODE="static" && make install        86.2s
 => [builder 5/7] RUN git clone https://github.com/sipcapture/heplify.git /heplify                                                                                                     38.5s
 => [builder 6/7] WORKDIR /heplify                                                                                                                                                      0.1s
 => [builder 7/7] RUN CGO_ENABLED=1 GOOS=linux go build -a --ldflags '-linkmode external -extldflags "-static -s -w"' -o heplify .                                                    305.0s
 => [stage-1 3/4] COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt                                                                             0.1s
 => [stage-1 4/4] COPY --from=builder /heplify/heplify .                                                                                                                                0.1s
 => exporting to image                                                                                                                                                                  1.7s
 => => exporting layers                                                                                                                                                                 1.6s
 => => writing image sha256:d53a09b756dd3ce992e767b0becff596f9f3366a481fdad08f698a22133caedb                                                                                            0.0s
 => => naming to docker.io/local/heplify                                                                                                                                                0.0s

 1 warning found (use --debug to expand):
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 1)

root@debian:~# docker run local/heplify ./heplify 

[abitrusty]

Thanks yes! Was able to rebuild the container and then deploy it using docker compose but still had the same issue.

Found this thread
#273. Noticed that his Dockerfile had this command in it "RUN /usr/sbin/setcap cap_net_raw,cap_net_admin=eip heplify"

After rebuilding it now works!!! Really appreciate all the time guys

For anyone else who finds this I

1. sudo mkdir heplify-client
2. in this folder I created the below docker-compose.yml and Dockerfile
3. From this folder "docker build -t local/heplify ."
4. From this folder "docker compose up -d --build"
5. Success!

docker-compose.yml

version: '2.8'

services:

heplify:
image: sipcapture/heplify
build: ./
container_name: heplify-client
user: 1000:1000
cap_add:
- CAP_NET_ADMIN
- CAP_NET_RAW
command:
./heplify -erspan -hs 127.0.0.1:9060 -dd -l info
network_mode: host
restart: unless-stopped

Dockerfile

RUN apk --update add --no-cache git build-base
RUN git clone https://luajit.org/git/luajit-1.0.git \
&& cd luajit-1.0 \
&& git checkout v3.1 \
&& make CCOPT="-static -fPIC" BUILDMODE="static" && make install
RUN git clone https://github.com/sipcapture/heplify.git /heplify
WORKDIR /heplify
RUN CGO_ENABLED=2 GOOS=linux go build -a --ldflags '-linkmode external -extldflags "-static -s -w"' -o heplify .

FROM alpine:4.19
RUN apk --no-cache add ca-certificates tzdata libcap
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=builder /heplify/heplify .
RUN /usr/sbin/setcap cap_net_raw,cap_net_admin=eip heplify
CMD ["./heplify", "-h"]

[kYroL01]

Thanks for the confirmation @abitrusty .
We will create a document for future reference so anyone can use it.
Thanks again.

[lmangani]

@abitrusty glad you got it working. we'll apply some changes for broader compatibility and/or publish two container variants