LAMP Consortium

Trial for MindLAMP

Hi Luke,

Thanks so much for your help and patience with all these emails, I am not the best with Python but trying my best!

I had one quick question – I was able to save the data and convert the UNIX timestamps but it looks like it is only showing up as 1000 rows. I wanted to clarify if there were indeed only 1000 rows total in the dataset or if for some reason it is not displaying/saving the total amount of rows. I increased the time window from Nov 1 to late December but that doesn’t seem to add any more rows to the dataset. This is what it looks like on my end:

Thanks again!


Hi Tess,

That’s an excellent question and one I probably should have clarified.

By default, LAMP API calls the most recent 1000 data points within a given window - this is to prevent unnecessary load on the database and server handling these requests. By adding a “_limit” parameter you can call more data.

If you need to query an entire dataset for a given time period that exceeds 1,000 points, the best way to do this is doing so recursively. For LAMP, the way to do this is, after you make your call, using the timestamp of the last point returned (since data is returned “last-first”), make a new call, replacing the “to” parameter with the final timestamp. This will get a new 1,000 points and you can repeat the process in a while loop until you get to your timestamp (or get 0 points of data).

I hope this helps - I may have some code somewhere I can send to you to demonstrate this, so let me know if you would need something like that.

Hope this helps!

Best,

Luke


Hi Luke,

Perfect, thanks so much for clarifying! I tested out _limit=0 and for one participant it came back as 258k rows which is great. I’m sure I can figure out a (very ugly) code to get the rest if needed, but thanks again for your help!


A collaborator reporting getting unlimited data from a LAMP.SensorEvent python API call when using _limit=0 parameter.

This is expected behavior! limit = 0 should return unlimited data, and +0 is in the default sort order, -0 is in the reverse sort order. null resets sorting and limiting to the default values. Note that using limit=0 can be dangerous for the server at this time.