SensorSpec is not showing up

Hi @lido and @khackett,

I am sorry that you are continuing to encounter issues! We have put together a cURL command that should work directly with the API - I have tested this myself so I am fairly confident this should work.

curl -L -X POST 'https://API_ADDRESS/sensor_spec'  / 
-H 'Content-Type: application/json' /
-H 'Accept: application/json' /
-H "authorization:ADMIN:PASSWORD"/
--data-raw '{
  "name": "lamp.accelerometer"
}'

You need to replace 3 things:

  • API_ADDRESS with your api address (e.g. api.lamp.digital)
  • ADMIN:PASSWORD with you admin credentials
  • lamp.accelerometer for each new appropriate spec name

Unfortunately, you will have to run this once each for each sensor spec you want to initialize, so it is clunkier than the other way. We will also work on a script to automate this process, but if there are specific sensors you need asap you should be able to use this command.

I hope that this is helpful - I will get that automated script to you as soon as possible but I know with the holiday approaching you may need this intermediate script now.

1 Like

Hi @LukeS,

Thanks for this. Unfortunately I receive the same “503.unimplemented” error using the provided curl command as I did when running the python script you provided earlier:

{
  "error": "503.unimplemented"
}

Here’s the curl command I ran:

#!/usr/bin/env bash

ADDR="api.mindlamp.temple.edu"
USER="admin"
PASS="..."

curl -L -X POST "https://$ADDR/sensor_spec" \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-H "authorization:$USER:$PASS" \
--data-raw '{
  "name": "lamp.accelerometer"
}'

How can I verify the mindlamp api code I’m running matches what you’re running?

Hopefully, this is helpful info…

The digest matches what’s on docker hub for this image:

$ docker images --digests bidmcdigitalpsychiatry/lamp-server
REPOSITORY                           TAG       DIGEST                                                                    IMAGE ID       CREATED        SIZE
bidmcdigitalpsychiatry/lamp-server   2021      sha256:d29f8a21b4ce8579b5d1538e5010755b7fb671f4e7f674de0e1c78d2ef56e334   e21ac6c5057e   7 months ago   281MB

Pulled the image again:

$ docker image pull bidmcdigitalpsychiatry/lamp-server:2021
2021: Pulling from bidmcdigitalpsychiatry/lamp-server
Digest: sha256:d29f8a21b4ce8579b5d1538e5010755b7fb671f4e7f674de0e1c78d2ef56e334
Status: Image is up to date for bidmcdigitalpsychiatry/lamp-server:2021
docker.io/bidmcdigitalpsychiatry/lamp-server:2021
$ docker image ls
REPOSITORY                           TAG                IMAGE ID       CREATED         SIZE
traefik                              <none>             daa4d30f224b   2 months ago    96.8MB
bidmcdigitalpsychiatry/lamp-server   2021               e21ac6c5057e   7 months ago    281MB
apache/couchdb                       3.1.1              cabe93489203   8 months ago    189MB
nats                                 2.1.9-alpine3.12   7b1589771fc3   9 months ago    16.2MB
redis                                6.0.8-alpine       8835e6aeca99   13 months ago   31.1MB

Updated the service with --force

$ docker service ls
ID             NAME                 MODE         REPLICAS   IMAGE                                     PORTS
8dcjztei74bn   lamp_cache           replicated   1/1        redis:6.0.8-alpine
ria1usfe8a3s   lamp_database        replicated   1/1        apache/couchdb:3.1.1
r3ywbi9mdkzh   lamp_message_queue   replicated   1/1        nats:2.1.9-alpine3.12
t4bibc1ko5x5   lamp_server          replicated   1/1        bidmcdigitalpsychiatry/lamp-server:2021
z6b0dkjuifv5   router_traefik       replicated   1/1        traefik:latest                            *:80->80/tcp, *:443->443/tcp
$ docker service update lamp_server --force
lamp_server
overall progress: 1 out of 1 tasks
1/1: running
verify: Service converged

API still fails to load the accelerometer sensor:

$ ./sensors.sh
{
  "error": "503.unimplemented"
}

Hi @lido

That is very helpful - I think you have found the main issue here! The image from docker hub is actually a bit out of date and has not been updated recently - the change happened just before I arrived but for a few reasons we switched to hosting the image on github directly, at ghcr.io/bidmcdigitalpsychiatry/lamp-server:2021 (ghcr.io/bidmcdigitalpsychiatry/lamp-server:2021)

That explains both of the issues - why we weren’t able to use this new feature and why attempting to update the image didn’t lead anywhere. If you replace the image you are using for your lamp_server service with this new one, I think both of the scripts (cURL or Python) will start working.

Thank you, @LukeS !

Those 8 extra characters (ghcr.io/) made all the difference. :wink:

Happy to report that after switching to the image hosted on GitHub’s container registry, the python script you provided worked as expected.

$./sensors.py
[LAMP Platform] SensorSpec Initialization Tool
You must enter your username, password, and server address to continue installation.
Username: admin [CANNOT BE CHANGED]
Password: ...
Server Address: api.mindlamp.temple.edu
Creating SensorSpec 'lamp.accelerometer'...
Creating SensorSpec 'lamp.accelerometer.motion'...
Creating SensorSpec 'lamp.analytics'...
Creating SensorSpec 'lamp.blood_pressure'...
Creating SensorSpec 'lamp.bluetooth'...
Creating SensorSpec 'lamp.calls'...
Creating SensorSpec 'lamp.device_state'...
Creating SensorSpec 'lamp.distance'...
Creating SensorSpec 'lamp.flights'...
Creating SensorSpec 'lamp.gps'...
Creating SensorSpec 'lamp.gps.contextual'...
Creating SensorSpec 'lamp.gyroscope'...
Creating SensorSpec 'lamp.heart_rate'...
Creating SensorSpec 'lamp.height'...
Creating SensorSpec 'lamp.magnetometer'...
Creating SensorSpec 'lamp.respiratory_rate'...
Creating SensorSpec 'lamp.segment'...
Creating SensorSpec 'lamp.sleep'...
Creating SensorSpec 'lamp.sms'...
Creating SensorSpec 'lamp.steps'...
Creating SensorSpec 'lamp.weight'...
Creating SensorSpec 'lamp.wifi'...

@khackett , I hope this helps get us past the sensor issues you encountered. Please test when you can and let me know if things are working as expected now.

Have a safe and happy holiday everyone.

Thank you Luke and Lido for troubleshooting this!

-John

Hi @lido - it appears this worked! I am now seeing the ability to add all the individual sensors on the researcher/dashboard end. I will go ahead and add them all to our study, do some testing and see what data I can pull on the backend afterwards. Thanks to everyone on this thread for helping out – have a wonderful Thanksgiving!

Hi again @lido @LukeS @spatel - after about a week of pilot testing on a few phones, it appears that the following sensors are not being collected on either iPhone 12 Pro (software version 15.1.1) or Android (google pixel) devices: accelerometer.motion; bluetooth; calls; device_state; distance; flights; gps.contextual; gyroscope; magnetometer; segment; sleep; sms; wifi [these were all sensors that @lido configured per the screenshot above]. Interestingly, lamp.steps and lamp.telephony were collected on the iPhone but not the Android device. Is this expected/is there anything you would recommend to ensure we have at a minimum the calls/sms data collected across both types of devices? I’m also puzzled why the lamp.telephony sensor is collecting data for the iPhone even though it wasn’t technically configured/loaded.

Just as an FYI the following sensors are being successfully collected on the iPhone: accelerometer; analytics; gps; screen_state; steps; telephony

And on the Android: accelerometer; analytics; gps; screen_state

Thank you again for your help with this!

Hi @khackett,

Thanks for letting us know about this. Some of the sensors that you mentioned that are not collecting are deprecated or require further configuration and not used actively, so it makes sense they may not have data. These include accelerometer.motion, device_state, flights, gps.contextual, distance and segment. I will look into why the remainder of your sensors are not being collected with @LukeS.

lamp.telephony should have data on calls/sms, but let me know if that is not the case. It is a bit unique and can be collected from the phone despite being enabled.

Lastly, there are some minor differences in sensor collection/quality between Android and iPhone devices since sometimes the Android build is behind the iPhone, but these should be fixed soon to account for having the same sensor collection. Let me know if this helps and feel free to reach out with further questions!

Thanks for the quick reply @spatel! Please keep us posted on why the remainder of the sensors are not being collected and on the timeline for Android having the same sensors (we will likely start recruiting just iPhone users to get around this for now).

To answer your question, it looks like lamp.telephony is just collecting phone calls (not sms/texts) per the attached screenshot.

Hi @spatel @LukeS - just following up on the missing text/sms data for iphones. Right now this appears to be the one missing piece for us before we start recruiting iPhone users. Neither lamp.telephony nor lamp.sms are generating text data even though the study is configured to include these sensors. Are there any other sensors we should be trying, or anything in the phone settings that need to be manually configured (other than allowing all permissions after downloading mindLAMP as directed in the docs)? Thanks again for your help on this!

Hi all, we are having the same problem.

Hi @khackett and @laurelmorris,

We have checked on this and have found that iOS is not setup to support SMS logs and this issue may be occurring because though Android does support SMS logs, this may not be happening lately due to the OS version and the phone type.

Best,
Suraj