Redirect loop after 400 error on /api/explorers/search?domain=app.[domain]

[farhadhf]

I've installed the self-hosted version following the guide on https://doc.tryethernal.com/getting-started/self-hosted#web-app-setup (skipped the context setup). Here's the URL .env.production config:

APP_URL=app.explorer.[domain]
BASE_DOMAIN=app.explorer.[domain]
VUE_APP_MAIN_DOMAIN=explorer.[domain]

The output of docker-compose -f docker-compose.prod.yml ps:

       Name                      Command               State                    Ports                  
-------------------------------------------------------------------------------------------------------
ethernal_postgres_1   docker-entrypoint.sh postgres    Up      5432/tcp                                
ethernal_redis_1      docker-entrypoint.sh redis ...   Up      6379/tcp                                
highPriority          docker-entrypoint.sh node  ...   Up                                              
lowPriority           docker-entrypoint.sh node  ...   Up                                              
mediumPriority        docker-entrypoint.sh node  ...   Up                                              
web                   docker-entrypoint.sh /bin/ ...   Up      0.0.0.0:8888->8888/tcp,:::8888->8888/tcp

I have a Nginx reverse proxy set up to proxy the requests on explorer.[domain] and app.explorer.[domain] to 127.0.0.1:8888 upstream.

  location / {
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_set_header X-NginX-Proxy true;
      proxy_pass    http://127.0.0.1:8888;
  }

The issue is that whenever I open https://explorer.[domain] it redirects to https://app.explorer.[domain] and then gets a 400 error while trying to GET https://app.explorer.[domain]/api/explorers/search?domain=app.explorer.[domain] and redirects to https://app.explorer.[domain] again. Same redirect issue happens when I directly open https://app.explorer.[domain] in my browser. I looked at the code and it seems like the 400 error is happening here https://github.com/tryethernal/ethernal/blob/develop/run/api/explorers.js#L427 and the redirect happens here https://github.com/tryethernal/ethernal/blob/develop/src/main.js#L42

Output from docker-compose -f docker-compose-prod.yml logs:

postgres_1                | 
postgres_1                | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1                | 
postgres_1                | 2023-10-24 09:31:41.184 UTC [1] LOG:  starting PostgreSQL 14.1 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
postgres_1                | 2023-10-24 09:31:41.186 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgres_1                | 2023-10-24 09:31:41.186 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres_1                | 2023-10-24 09:31:41.190 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1                | 2023-10-24 09:31:41.195 UTC [21] LOG:  database system was shut down at 2023-10-24 09:31:28 UTC
postgres_1                | 2023-10-24 09:31:41.200 UTC [1] LOG:  database system is ready to accept connections
redis_1                   | 1:C 24 Oct 2023 09:31:41.123 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1                   | 1:C 24 Oct 2023 09:31:41.123 # Redis version=6.2.14, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1                   | 1:C 24 Oct 2023 09:31:41.123 # Configuration loaded
redis_1                   | 1:M 24 Oct 2023 09:31:41.124 # Server initialized
redis_1                   | 1:M 24 Oct 2023 09:31:41.124 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1                   | 1:M 24 Oct 2023 09:31:41.142 # Done loading RDB, keys loaded: 90, keys expired: 23.
web                       | App started on port 8888
web                       | error: Couldn't find explorer. {"data":{"domain":"explorer.[domain]"},"error":{"message":"Couldn't find explorer.","stack":"Error: Couldn't find explorer.\n    at /app/api/explorers.js:427:19\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)"},"location":"get.api.explorers.search","queryParams":{}}
web                       | error: Couldn't find explorer. {"data":{"domain":"explorer.[domain]"},"error":{"message":"Couldn't find explorer.","stack":"Error: Couldn't find explorer.\n    at /app/api/explorers.js:427:19\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)"},"location":"get.api.explorers.search","queryParams":{}}
web                       | error: Couldn't find explorer. {"data":{"domain":"app.explorer.[domain]"},"error":{"message":"Couldn't find explorer.","stack":"Error: Couldn't find explorer.\n    at /app/api/explorers.js:427:19\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)"},"location":"get.api.explorers.search","queryParams":{}}

Not sure where to go from here - do I need to somehow seed the database with an initial explorer?

[antoinedc]

Hi,

The redirect to app.explorer.[domain] is expected if you are trying to access an explorer that hasn't been setup yet.
When opening app.explorer.[domain], you should get a signin/signup screen. You need to create an account first, and then once you are in the dashboard, go to "Public Explorers" to do the setup.

You are saying you are having a redirect issue when opening app.explorer.[domain]. Where does it take you exactly?

[farhadhf]

Hi, thanks for the quick response! It gets the 400 error from /api/explorers/search?domain=app.explorer.[domain] and redirects back to https://app.explorer.[domain].

[antoinedc]

Yes, I meant to say that this is expected in my previous message.

What do you see on app.explorer.[domain], an auth page?

[farhadhf]

Sorry for not being clear before, app.explorer.[domain] is just a white screen, reloading forever. The screenshot in my previous comment is from app.explorer.[domain] - it gets the 400 error and reloads the same page.

[antoinedc]

Ah I see! Actually, it looks like there is a missing env variable in the .env.prod example.
Can you add APP_DOMAIN=explorer.[domain], rebuild the image and try again? (sorry lot of redundancy in those variables, I need to clean up some legacy code)

[farhadhf]

Thanks, that was it, it's working now. I tried syncing past block from the server side and I'm getting "You are not allowed to use server side sync. If you'd like to, please reach out at [email protected]" (along with quite a bit of console.log messages for request and response objects).

[antoinedc]

Did you set up the explorer using the "Public explorers" page?

[farhadhf]

I have, but it says "This explorer is not active. To activate it, start a subscription." on the public explorer page, and under the sync section it says "Synchronization will become available once a subscription has been started". Do I need to purchase a subscription for the self-hosted version as well?

[antoinedc]

No you don't need. I haven't updated the Dockerfile for self hosted with the latest env variables, and it's missing another one.
After line 6 on the Dockerfile, can you add ENV VUE_APP_ENABLE_BILLING=false? And rebuild the image. It shouldn't ask you for a subscription after that.

[farhadhf]

@antoinedc Thanks for the help - I think I'm still missing some config or ENV variable related to memory limit for docker build.

I'm running docker build -t ethernal -m 6g . in the same directory as Dockerfile, and I've added ENV NODE_OPTIONS=--max_old_space_size=6144 to a few places in the Dockerfile but I'm still getting

Killed
error Command failed with exit code 137.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
The command '/bin/sh -c yarn build' returned a non-zero code: 137

Will adding more swap or even get a bigger machine fix this? How much memory does ethernal need to get built?

[antoinedc]

Sorry for the delayed answer here. I usually allocate 8g when building it and it works. Can you give it a try?

[sudopunk]

I tried it all till this point. Is there a solution to this yet? Would a previous version work?

[antoinedc]

I tried it all till this point. Is there a solution to this yet? Would a previous version work?

You're also having the 137 error from yarn?

[farhadhf]

Sorry for the late reply - I got around the 137 error by increasing the memory to 8GB. Now I'm getting this error:

Step 55/66 : COPY ethernal-95a14-19f78a7e26cc.json ./ethernal-95a14-19f78a7e26cc.json
COPY failed: file not found in build context or excluded by .dockerignore: stat ethernal-95a14-19f78a7e26cc.json: file does not exist

[farhadhf]

Update: I removed the line trying to copy ethernal-95a14-19f78a7e26cc.json. As suggested above, I've added ENV VUE_APP_ENABLE_BILLING=false to the Dockerfile and then ran docker build -t ethernal.

FROM node:16 AS front
WORKDIR /client
ENV VUE_APP_NODE_ENV=production
ENV VUE_APP_API_ROOT=https://app.VUE_APP_MAIN_DOMAIN_PLACEHOLDER
ENV VUE_APP_MAIN_DOMAIN=VUE_APP_MAIN_DOMAIN_PLACEHOLDER
ENV VUE_APP_PUSHER_KEY=VUE_APP_PUSHER_KEY_PLACEHOLDER
ENV VUE_APP_ENABLE_BILLING=false
ENV NODE_OPTIONS=--max_old_space_size=8192
COPY public/ ./public/
COPY src/ ./src/
COPY babel.config.js .firebaserc package.json yarn.lock vue.config.js _redirects ./
RUN yarn install --network-timeout 100000
RUN yarn build
...

It successfully got built. I've also editted docker-compose.prod.yml and changed the image of the backend service to ethernal:latest:

web: &backend
    networks:
      - ethernal
    container_name: web
    depends_on:
      - postgres
      - redis
    pull_policy: always
    restart: always
    ports:
      -  '8888:8888'

    image: ethernal:latest
    command: /bin/sh ./web_entrypoint.sh
    env_file: .env.prod
    links:
      - postgres
      - redis

Did docker-compose -f docker-compose.prod.yml down and then docker-compose -f docker-compose.prod.yml up -d to restart Ethernal. It's up and running, but I still see the Billing than in /settings?tab=billing and running ethernal sync throws the same error about not being allowed to run server side sync.

data: "You are not allowed to use server side sync. If you'd like to, please reach out at [email protected]"

Any ideas?