Self-Hosting Deployment Questions

aware-ravi · October 26, 2020, 10:11pm

Hello Lamp Consortium,

I’m currently using Beiwe to run a study on ~200 participants (and growing) at treatment clinics across the United States. However, I recently discovered mindLAMP and it seems much more in line with my eventual goal of attempting interventions for the patients in my study. I attempted to follow the instructions in the self hosting guide but have run into a number of issues that I was hoping to get help with. I’m planning on self hosting the lamp-server, lamp-dashboard and self-publishing the Android and iOS apps to their respective app stores.

I’ve currently followed all of the instructions up to the testing section on https://docs.lamp.digital/Deploying-the-LAMP-Platform-e281278b00c1475caee5ccf3ffaafba0#5b06a18009154c3cbe12c6419c44278a

—Unfortunately, there’s no mention of the default admin username and password - how do I go about creating an admin account for myself? Or is this the same as the couchDB username/password?—

(Found the answer to this in this topic: Question about Migration to CouchDB from MSSQL)

However, I’m running into this issue in my lamp-server logs:

  (node:1) UnhandledPromiseRejectionWarning: Error: getaddrinfo ENOTFOUND database
      at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26)
  (Use `node --trace-warnings ...` to show where the warning was created)
  (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
  (node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
  (node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379

I set up an SSL certificate through AWS Certificate Manager and DNS entries through my GoDaddy account but when I attempt to log in through dashboard, I get the error NET::ERR_CERT_AUTHORITY_IN which makes sense since I haven’t actually installed my SSL certificate anywhere. Would it be possible to clarify how to do this using Traefik?
How should I deploy an entirely local stack? What configuration changes should I make? I’m assuming traefik won’t work locally since it requires TLS?
For these values in the docker stack:
traefik.http.routers.lamp_database.rule: ‘Host(db.example.com)’
traefik.http.routers.traefik_dashboard.rule: ‘Host(router.example.com)’

Should I set up CNAME entries in my DNS configuration to point from these endpoints to my EC2 IP address?

If I wanted to set up the lamp dashboard as well, would it be best to set it up in the same stack? Or should I just use a separate stack entirely? Is there anything in particular I need to configure?

Thank you for taking the time to read through this and apologies for the laundry list of questions. mindLAMP looks like an amazing platform and I’m super excited to use it.

Thanks,
Ravi

rbilden · October 27, 2020, 1:09pm

Hi Ravi,

Here are my answers below. Would you mind filling out the following surveys so we can get a better idea of how you are using LAMP?

The admin password is now auto generated and you can find it in the console logs. Change the declaration to “http://admin:PASSWORD@database:5984/“
Traefik will automatically use letsencrypt to generate an SSL cert but this can be turned off by configuring the docker-compose file.
The traefik will work locally still, but you will need to turn off https, as correctly surmised.
Yes - they should be entries from your chosen domain name, then replace these strings in the config.
The same stack should work fine! There is nothing major to change about it.

Hope this helps!

Rebecca

aware-ravi · October 27, 2020, 4:05pm

Thank you for the quick reply Rebecca!

Will definitely fill out those forms.

Had some follow up questions:

Re: #1:

I was able to get everything mostly working with some slight changes!

However, I still don’t see the administrator password in my server logs. My couchdb logs mention a missing ‘_users’ database, but not sure if that’s the root cause or not. Some more guidance here would be great - I can paste my logs for both at the bottom of this along with my modified stack.

My coworker was able to get one generated, but it doesn’t mention the email address, just the password. Are the logs supposed to mention the auto generated email address as well?

Stack:

version: '3.7'
services:
  lamp-server:
    image: bidmcdigitalpsychiatry/lamp-server:latest
    container_name: lamp-server
    environment:
      HTTPS: 'off'
      SCHEDULER: 'off'
      ROOT_KEY: 'b6630d8e63476a937d96b7769ce78dce15fd1fd45ece44ca4816ef30accf46d9'
      CDB: 'http://admin:ddaf49c54689f099@lamp-database:5984/'
      PUSH_API_KEY: 'YOUR_PUSH_KEY_HERE'
      PUSH_API_GATEWAY: 'YOUR_PUSH_GATEWAY_HERE'
    networks:
      - public
    depends_on:
      - lamp-database
    ports:
    - "8081:3000"
    deploy:
      mode: replicated
      # labels:
      #   traefik.enable: 'true'
      #   traefik.http.routers.lamp_server.entryPoints: 'websecure'
      #   traefik.http.routers.lamp_server.rule: 'Host(`api.example.com`)'
      #   traefik.http.routers.lamp_server.tls.certresolver: 'default'
      #   traefik.http.services.lamp_server.loadbalancer.server.port: 3000
      placement:
        constraints:
          - node.role == manager
  lamp-database:
    image: apache/couchdb:latest
    container_name: lamp-database
    volumes:
      - /tmp/couchdb:/opt/couchdb/data
      # - /data/couchdb:/opt/couchdb/data
    networks:
      - public
    environment:
      COUCHDB_USER: 'admin'
      COUCHDB_PASSWORD: 'ddaf49c54689f099'
    deploy:
      mode: replicated
      # labels:
      #   traefik.enable: 'true'
      #   traefik.http.routers.lamp_database.entryPoints: 'websecure'
      #   traefik.http.routers.lamp_database.rule: 'Host(`db.example.com`)'
      #   traefik.http.routers.lamp_database.tls.certresolver: 'default'
      #   traefik.http.services.lamp_database.loadbalancer.server.port: 5984
      placement:
        constraints:
          - node.role == manager
    ports:
    - "4369:4369/tcp"
    - "5984:5984/tcp"
    - "9100:9100/tcp"
  lamp-dashboard:
    image: bidmcdigitalpsychiatry/lamp-dashboard:latest
    container_name: lamp-dashboard
    ports:
    - "8080:80"
networks:
  public:
    external: true

Server Logs

  (node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
      at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
  (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 8)
  Server routing initialized.
  Initializing database connection...
    ActivitySpec database online.
    SensorSpec database online.
    Researcher database online.
    Study database online.
    Participant database online.
    Activity database online.
    Sensor database online.
    ActivityEvent database online.
    SensorEvent database online.
    Credential database online.
    Tag database online.
  Database verification complete.
  Running with schedulers disabled.
Initialization complete.

Database Logs:

[info] 2020-10-27T16:02:15.474268Z nonode@nohost <0.11.0> -------- Application chttpd started on node nonode@nohost
[info] 2020-10-27T16:02:15.543635Z nonode@nohost <0.11.0> -------- Application couch_index started on node nonode@nohost
[info] 2020-10-27T16:02:15.543762Z nonode@nohost <0.11.0> -------- Application couch_mrview started on node nonode@nohost
[info] 2020-10-27T16:02:15.544322Z nonode@nohost <0.11.0> -------- Application couch_plugins started on node nonode@nohost
[error] 2020-10-27T16:02:15.557565Z nonode@nohost emulator -------- Error in process <0.317.0> with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,399}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,374}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,96}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,198}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,145}]}]}
[notice] 2020-10-27T16:02:15.559193Z nonode@nohost <0.316.0> -------- chttpd_auth_cache changes listener died because the _users database does not exist. Create the database to silence this notice.
[info] 2020-10-27T16:02:15.732990Z nonode@nohost <0.11.0> -------- Application couch_replicator started on node nonode@nohost
[info] 2020-10-27T16:02:15.758707Z nonode@nohost <0.11.0> -------- Application couch_peruser started on node nonode@nohost
[info] 2020-10-27T16:02:15.803112Z nonode@nohost <0.11.0> -------- Application ddoc_cache started on node nonode@nohost
[info] 2020-10-27T16:02:15.885260Z nonode@nohost <0.11.0> -------- Application dreyfus started on node nonode@nohost
[info] 2020-10-27T16:02:15.994737Z nonode@nohost <0.11.0> -------- Application global_changes started on node nonode@nohost
[info] 2020-10-27T16:02:15.995345Z nonode@nohost <0.11.0> -------- Application jiffy started on node nonode@nohost
[info] 2020-10-27T16:02:16.088235Z nonode@nohost <0.11.0> -------- Application jwtf started on node nonode@nohost
[info] 2020-10-27T16:02:16.150629Z nonode@nohost <0.11.0> -------- Application ken started on node nonode@nohost
[info] 2020-10-27T16:02:16.203474Z nonode@nohost <0.11.0> -------- Application mango started on node nonode@nohost
[info] 2020-10-27T16:02:16.293520Z nonode@nohost <0.11.0> -------- Application setup started on node nonode@nohost
[info] 2020-10-27T16:02:16.448573Z nonode@nohost <0.11.0> -------- Application smoosh started on node nonode@nohost
[info] 2020-10-27T16:02:16.448640Z nonode@nohost <0.11.0> -------- Application snappy started on node nonode@nohost
[info] 2020-10-27T16:02:16.448986Z nonode@nohost <0.11.0> -------- Application recon started on node nonode@nohost
[notice] 2020-10-27T16:02:20.459310Z nonode@nohost <0.439.0> 53e53748b0 lamp-database:5984 10.0.9.4 admin GET /_all_dbs 200 ok 283
[notice] 2020-10-27T16:02:20.489184Z nonode@nohost <0.440.0> 6d2e111a1e lamp-database:5984 10.0.9.4 admin GET /_all_dbs 200 ok 4
[notice] 2020-10-27T16:02:20.530238Z nonode@nohost <0.316.0> -------- chttpd_auth_cache changes listener died because the _users database does not exist. Create the database to silence this notice.
[error] 2020-10-27T16:02:20.531331Z nonode@nohost emulator -------- Error in process <0.469.0> with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,399}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,374}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,96}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,198}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,145}]}]}

rbilden · October 27, 2020, 5:09pm

Hi Ravi,

Right now the admin username always defaults to “admin”–you might have to try logging in first with a fake pass for it to generate at all.

CouchDB always does the “_users” error. If you create that db manually it will go away.

The “connect ECONNREFUSED 127.0.0.1:6379” is unrelated and can be ignored.

Best,

Rebecca

aware-ravi · October 28, 2020, 6:29pm

Thank you so much for the help! I was able to test that the server was able to talk to my database locally, but connecting through the lamp dashboard failed since it tried to use https. I ended up switching to using the EC2 Instance Setup and got pretty much everything working after a lot of trial and error

(note to others doing the traefik setup: I used GoDaddy and had to specify the following to get it working:

  - "--certificatesresolvers.default.acme.dnschallenge=true"
  - "--certificatesresolvers.default.acme.dnschallenge.provider=godaddy"
  - "--certificatesresolvers.default.acme.dnschallenge.delaybeforecheck=30"

I also had to remove these two lines:
# - “–providers.file.directory=/traefik-config”
# - “–providers.file.watch=true”)

Would you mind if I proposed some changes to the Self Deployment guide?

rbilden · October 28, 2020, 7:01pm

Hi Ravi,

That would be very welcome!

Looking forward to it.

Also, would you mind filling out this form? https://docs.google.com/forms/d/e/1FAIpQLSeNyY469zPcwVoZ8gKTwW8dLuzRDXpObLcR8JtHdUlRhAQcyQ/viewform?gxids=7757

Best,

Rebecca

aware-ravi · October 30, 2020, 2:51am

Done!

The main things I would change would be:

1. Providing a separate tutorial specifically for local development.

For this tutorial, I’d remove the following from stack definition (since it doesn’t matter for local development:

All traefik config
Mention of the PUSH_API_KEY/PUSH_API_GATEWAY
Removal of the public network creation (docker will auto-create a network for the stack already, and since we’re not deploying the traefik config, there’s no need for this)

I’d also add:

A short walkthrough of setting up portainer.

$ docker volume create portainer_data 
$ docker run -d -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

Updating the CDB environment variable to have port 5984 suffixed to it.
IIRC, I believe I also had to update the ports section of the stack to include:

    ports:
    - "5984:5984/tcp"

The testing section can ensure the server and database are both up by using basic curl commands (since a GET / request on both the server and database should return JSON.
Adding instructions about getting the admin credentials (and a note about the correct email for the admin account).

At the end of the day, I ended up running this command to get it to print the admin password:

docker exec -it lamp-server "/bin/sh" -c "cd build && node -e 'require(\"./repository\").CredentialRepository._adminCredential(process.env.ROOT_KEY)'"

For bonus points, it would be nice to have a tutorial for creating a lamp set-up using multiple docker-machines as a docker swarm. (definitely more of a nice-to-have though)

2. Fleshing out the EC2 Instance setup (or even having a tutorial using ‘docker compose up’ to deploy it to ECS or similar)

The instructions for the EC2 instance set-up are slightly off (ex: there’s no need to expose all of those ports publicly, I was able to get away with only exposing ports 80 and 443 publicly. (I used 22 for short-term ssh’ing and initial stack deployment).

3. General changes:

Update the lamp database set-up to create both /data and /data/couchdb folders (otherwise you’ll get an error when deploying the stack).

4. Minor Tweaks I’d suggest for the docker stacks:

Remove

- “–providers.file.directory=/traefik-config”
- “–providers.file.watch=true”)

Since the suggested domain configuration uses wildcard notation, it would be good to add a note about DNS challenges for Lets Encrypt SSL certification generation (since DNS challenges are the only way to generate an SSL cert for a wildcard domain). See here
Add a ‘depends_on lamp database’ for the server so that the lamp database is prioritized over the server

aware-ravi · October 30, 2020, 4:28am

Now that I was able to get everything set up for traefik, the lamp server and the database, I was running into a couple more issues. Sorry for the deluge of questions again, but it seems like I’m the first one to run into these problems Hopefully this will help future users that run into the same issues.

For the prevent tab, when I click the sensors, there’s none listed and when I attempt to use the API to fetch available sensors and sensor specs, it returns an empty array. Do I need to create these manually? Is there any way to create a study with all currently available sensors and sensor specs?

When I fetched the sensor events for my test patient (using my Android phone), it only showed lamp.analytics, despite granting all permissions.

    {
      "timestamp": 1604029277131,
      "sensor": "lamp.analytics",
      "data": {
        "device_type": "Dashboard",
        "user_agent": "LAMP-dashboard/a5368c5 Mozilla/5.0 (Linux; Android 11; Pixel 3a Build/RP1A.201005.004; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/86.0.4240.110 Mobile Safari/537.36"
      }
    }

One thing to note here for debugging is that I didn’t see a persistent notification in my notification bar (which is how I believe the Android Aware framework keeps itself alive in the background). That leads me to believe that no sensor data is being uploaded at all, which is very concerning.

When I hit the ‘manage’ button as a patient, it goes to a greyed out screen with a loading icon:

Any ideas why this might be happening? I don’t see anything happening on the network tab which leads me to believe there might be a bug here. Do I have to create these activities through the API?

I’ve created a researcher and a participant, but I’m having trouble accessing the api using lamp-py. I did:

pip install lamp-core

followed by:

import LAMP
LAMP.connect('endpoint', 'admin', 'admin_password')
LAMP.Participant.all()

but I get the error

UnicodeError: encoding with 'idna' codec failed (UnicodeError: label too long)

However, if I do a plain curl with an Authorization header, then it works fine, so it appears to be a problem with the python client specifically. Based on stack overflow, it seems the URL passed is too long for some reason.

How would I go about populating the ‘Learn’ tab with resources? When logging in as a research or an admin, I can’t seem to find any place to add resources.
Could you expand more on how automations work and how to use them? The documentation goes pretty into depth about technical architecture, but doesn’t seem super helpful as a user guide. The docs also mention templates for automations but I couldn’t find those. I’m planning on building a just-in-time intervention delivery system, and automations seem like a pretty good tool to have for that.

avaidyam · October 30, 2020, 3:40pm

Hi @aware-ravi,

Thanks so much for the in-depth responses! We really appreciate it! I’ll be making some changes to the deployment documentation to reflect your suggestions. I do have some comments and responses to your questions, however:

The scope of the documentation doesn’t cover local deployment or testing, but we’ll work with our engineering team to add more info on that.
Unfortunately, setting up Portainer is outside of the scope of our documentation, and we have removed the related documentation to avoid confusion.
Thanks for catching the CDB variable issue! You shouldn’t actually have to add the ports section for Docker Swarm – are you using Docker Compose instead?
We’ll add a quick cURL command for testing each.
Not sure what you mean by “correct email for admin account” but we’ll clarify how to locate your admin credentials.
You should instead just check the logs for the LAMP_server service as it reports the admin credential on initialization.
Unfortunately Docker Machine and multi-node Swarm setups are out of the scope of our documentation. We expect that the individual deploying the platform has experience with Docker and Docker Swarm.
This is a good catch, our EC2 instance instructions are out of date with respect to what ports need to be opened, and we’ll update that.
We recently switched from using a Docker volume to bind-mounting in the tutorial to follow best-practices, so this is a good catch and we’ll add the prerequisite step!
We’ll remove the traefik-config monitoring from the tutorial, but the domain configuration is separate from Traefik requesting ACME certificates, and as long as the labels in the docker-compose.yml are not using a wildcard domain, any kind of challenge will work.
depends_on is not supported by Docker Swarm, so we won’t be including this change.
Currently the Sensors API is not linked to the native apps and all sensor data is being collected at maximum frequencies allowed. This will change soon, however, and require that Sensors be created to enable data collection from devices.
The Sensors section of the Prevent tab does not currently show raw data; we’ll add that as a disclaimer.
Please submit a bug report from our documentation home page if you think you’ve encountered a bug with sensor data.
The Manage page issue will be resolved in our next release early next week.
This seems like an issue specific to your deployment that is linked to this recent Python bug.
From the management interface, use the Activities tab to create new Tips, Surveys, etc. which will then show up for individuals in your study.
We are not able to share more details on using the Automations framework at this time, and most of the tooling around it is internal to our team and our studies right now. Once we reach a stable state, we’ll be sure to notify you through a forum newsletter with more information. (And yes, the Automations framework is exactly what you would need to use to deliver just-in-time adaptive interventions.)

Hope that helps!

aware-ravi · October 30, 2020, 5:20pm

Hi Aditya,

Thank you for replying! Your team has been super helpful in getting this thing going - I’ll shoot you guys an email since I’m planning to make a decision regarding migrating our study from Beiwe to MindLAMP over the next few days. There’s a lot of really really cool features here (chat, cognitive tests, medication tracking, and automations), but I do have a few concerns.

Totally understand your pushback about too much hand-holding and the scope of documentation, but I think there’s definitely a balance between too little and too much here. In my experience prioritizing local development has been pretty helpful since it makes it easier for people to build and test code changes - but at the end of the day, totally your call.

Re:

You should instead just check the logs for the LAMP_server service as it reports the admin credential on initialization.

I tried this, but it didn’t seem to work for me unfortunately. Not a huge deal since I found a workaround.

Currently the Sensors API is not linked to the native apps and all sensor data is being collected at maximum frequencies allowed. This will change soon, however, and require that Sensors be created to enable data collection from devices.

Please submit a bug report from our documentation home page if you think you’ve encountered a bug with sensor data.

Gotcha, looks like I’m probably facing a bug then. I’ll submit a bug report asap. I ran into this on both my own build of the Android App and using the MindLAMP app from the Play Store. I recall that the documentation mentioned using the MindLAMP app from the Play Store wouldn’t work with a custom backend, could that be why I’m facing these issues?

The Manage page issue will be resolved in our next release early next week.

Awesome, please keep me posted! Is there a tracking Github issue for that?

This seems like an issue specific to your deployment that is linked to this recent Python bug.

Got this working turned out to be a silly issue on my end.

We are not able to share more details on using the Automations framework at this time

Do you know a general time frame when this would be complete?

avaidyam · October 30, 2020, 5:44pm

Hi @aware-ravi,

Thank you for replying!

Not a problem!

In my experience prioritizing local development has been pretty helpful since it makes it easier for people to build and test code changes - but at the end of the day, totally your call.

Our documentation is primarily focused on enterprise-grade deployments, and our development vendor manages all local development so unfortunately we can’t provide much support for that.

I recall that the documentation mentioned using the MindLAMP app from the Play Store wouldn’t work with a custom backend, could that be why I’m facing these issues?

As long as you’ve specified your server address and you’re using TLS the native apps should work with your custom backend.

Is there a tracking Github issue for that?

We’re working on an improved public-facing issue tracker, but we expect the fix for this to be live by Monday.

Do you know a general time frame when this would be complete?

Unfortunately we do not have a fixed time frame, but likely some time early 2021.

Hope this helps!

aware-ravi · October 30, 2020, 6:06pm

That makes total sense, thanks for clarifying. Your development vendor is ZCO right? I’m assuming you guys state the requirements and ZCO builds it?

As long as you’ve specified your server address and you’re using TLS the native apps should work with your custom backend.

Oh great, thank you for clarifying.

We’re working on an improved public-facing issue tracker, but we expect the fix for this to be live by Monday.

Awesome, looking forward to it!

Cheers,
Ravi

avaidyam · October 30, 2020, 6:22pm

Your development vendor is ZCO right? I’m assuming you guys state the requirements and ZCO builds it?

Yes, we work with Zco to develop the LAMP Platform.

Glad we could help!

ninaes · November 4, 2020, 11:00am

Hi everyone,
Trying to deploy the new server and having some issues. I have the database up and running, but the server logs shows some scary messages.
Here’s how I try to run the server:

docker run --name lamp2-server-with-env --env-file ./env.list --expose=5984 -p 5984:5984 -p 80:3000 -d docker.pkg.github.com/bidmcdigitalpsychiatry/lamp-server/lamp-server:participant-fix-redis

the contents of the env.list:

HTTPS=off
SCHEDULER=off
ROOT_KEY=XXXX
CDB=https://admin:XXXX@IP/
PUSH_API_KEY=YOUR_PUSH_KEY_HERE
PUSH_API_GATEWAY=YOUR_PUSH_GATEWAY_HERE

and here’s the server log:

Initializing LAMP API server…
(node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(Use node --trace-warnings ... to show where the warning was created)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 1)
(node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 2)
(node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 3)
(node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 4)
(node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 5)
(node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 6)
(node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 7)
(node:1) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 8)
(node:1) UnhandledPromiseRejectionWarning: Error: connect ETIMEDOUT 13.238.201.44:443
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 9)

Any ideas how to get passed this stage? I tried both:

docker.pkg.github.com/bidmcdigitalpsychiatry/lamp-server/lamp-server:participant-fix-redis

and

bidmcdigitalpsychiatry/lamp-server:latest

Thanks

avaidyam · November 4, 2020, 12:31pm

Hi @ninaes!

You can actually ignore those despite how scary they look - they’re for the Redis adapter that is under construction still. Once that’s completed, adding a REDIS_HOST URL to the env variables will silence these errors. It is a bug, however, that this variable being missing spews a bunch of error messages, and that should be fixed soon.

In addition, please change your CDB variable to match this template: http://admin:PASSWORD@database:5984/, as the correct protocol is http and the correct port is 5984. In a previous version, the communication between the database and the API server was TLS encrypted erroneously, and that has been removed. (Docker’s overlay network encrypts all communication anyways so double-encryption was an error.)

Hope that helps!

ninaes · November 9, 2020, 11:58am

thanks @avaidyam. So I got the database initialization message
Now I’m not sure how to find the admin credentials. I tried what @aware-ravi suggested, and I got the password once, but now somehow it does not work anymore (says port 3000 is already in use, shouldn’t it? ):

$docker exec -it lamp2-server-with-env-2 “/bin/sh” -c “cd build && node -e ‘require("./repository").CredentialRepository._adminCredential(process.env.ROOT_KEY)’”
Initializing LAMP API server…
(node:47) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(Use node --trace-warnings ... to show where the warning was created)
(node:47) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 1)
(node:47) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:47) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:47) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 2)
(node:47) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:47) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 3)
(node:47) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:47) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 4)
(node:47) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:47) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 5)
(node:47) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:47) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 6)
(node:47) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:47) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 7)
(node:47) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
(node:47) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v21.2.0 Documentation). (rejection id: 8)
Server routing initialized.
Initializing database connection…
ActivitySpec database online.
SensorSpec database online.
Researcher database online.
Study database online.
Participant database online.
Activity database online.
Sensor database online.
ActivityEvent database online.
SensorEvent database online.
Credential database online.
Tag database online.
Database verification complete.
Running with schedulers disabled.
Initialization complete.
events.js:292
throw er; // Unhandled ‘error’ event
^
Error: listen EADDRINUSE: address already in use :::3000
at Server.setupListenHandle [as _listen2] (net.js:1318:16)
at listenInCluster (net.js:1366:12)
at Server.listen (net.js:1452:7)
at Function.listen (/usr/src/app/node_modules/express/lib/application.js:618:24)
at /usr/src/app/build/app.js:130:29
at step (/usr/src/app/build/app.js:44:23)
at Object.next (/usr/src/app/build/app.js:25:53)
at fulfilled (/usr/src/app/build/app.js:16:58)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
Emitted ‘error’ event on Server instance at:
at emitErrorNT (net.js:1345:8)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
code: ‘EADDRINUSE’,
errno: -98,
syscall: ‘listen’,
address: ‘::’,
port: 3000
}

PS, I’m not using swarm, and the container logs does not show any credentials.

avaidyam · November 9, 2020, 12:13pm

Hi @ninaes,

Looks like there’s another process taking port 3000 on your instance?
The administrator credential is only logged once, on database initialization and first authorization request.
1. To reset the credential, delete any/all documents stored in the credential database whose origin is null.
  1. Do not delete any design documents (whose _id fields are prefixed with _design/).
2. Then, when you try to authorize as admin, you’ll receive a new key.
3. Please save this key as it will not change on reboot/reload.

Hope this helps!

ninaes · November 10, 2020, 10:27pm

Thanks, @avaidyam!
Resetting the credentials did the trick.
Server and database are both up and running now

nicholaide · November 20, 2020, 3:20pm

Hi @avaidyam! How many public IPs do you recommend for a standard deployment? Standard meaning no changes to mindLAMP, just using existing app and backend code from Github.

Thank you,
Nicholai

avaidyam · November 20, 2020, 7:29pm

@nicholaide, just one should be sufficient. If you’re in need of a load balancer, our team uses Traefik (as described in the docs) and it works well.

Topic		Replies	Views
Error on Connection Data & Analysis	5	525	November 4, 2021
Deploy Server Deployment & Development	15	652	February 11, 2021
General availability of the new LAMP Platform Latest News	9	544	January 5, 2022
Error Messages upon set-up Deployment & Development	5	335	December 8, 2021
Unable to start server Deployment & Development	42	671	November 13, 2020

Self-Hosting Deployment Questions

Stack:

Server Logs

Database Logs:

1. Providing a separate tutorial specifically for local development.

2. Fleshing out the EC2 Instance setup (or even having a tutorial using ‘docker compose up’ to deploy it to ECS or similar)

3. General changes:

4. Minor Tweaks I’d suggest for the docker stacks:

Related topics