Data recording for scientific experiments

[momsi]

Hi,

First of all I'd like to say that I am fairly new to the world of microcontrollers and hardware programming. I do know programming, and know python quite well, I'm an engineer though, and not a professional computer scientist. So forgive me if I may ask stupid questions

I heard about MicroPython and I thought it might be a good solution for the following problem.

I just began working on a new scientific experiment and the technical infrastructure is quite old.... Like DOS old. So it's quite a pain to use.

I have experience in analysis of data and the automation of this with python but in in this project, the problem is also in acquiring the data.

Its actually just analouge data, voltage and current.

I think it's possible to measure data with a MicroPython board via the analog IO pins. Right?

What board would you recommend for this?

What frequency could you record data with. How many data points per second and on how many channels?

Would there be a significant latency between the various channels on one datapoint, meaning: if I record 3 different values at the same time, how much latency is between the the 3 values?

Tl:dr it comes down to this: measure various different voltages and output the data as file for later analysis. Possible with MicroPython?

Hope you can help me. Thanks in advance.

[stijn]

What frequency could you record data with. How many data points per second and on how many channels?

You should really be asking this the other way around: you're the one who has to specify how many channels etc. I mean, we use 16 channels at 40kHz because we have that many analog signals from sensors etc and because we're interested in freqency content up to 20kHz, but there's no reason that applies for you as well.. (this is just for illustration; data is not recorded using uPy, and I wouldnt know what board achieves such rates).

If I record 3 different values at the same time, how much latency is between the the 3 values

That depends on the implementation of the ADC. Often it's 1 sample period. But might as well be 0. Apart from that you'll have to figure out what performance you need with respect to noise and drift, whether absolute values matter (i.e. do you need a calibrated ADC with certificate?) and take that with you in search of the hardware. You don't mention how critical the recording of data is for this particular experiment, but normally in science you don't just settle for something without researching if it fullfills all specs

[momsi]

Well as I mentioned I just started working for this experiment so honestly I don't know all details yet.

I thought there were only a few boards that support MicroPython anyway and that someone could tell me, with the most suitable board (whatever that might be) if this would generally possible and what the limitations would be.

[stijn]

Generally speaking it's possible, yes. Boards are listed here btw: https://github.com/micropython/micropyt ... ds-Summary. But without details it's really not possible to just tell you 'this is the most suitable board'..

Some practical limitations: a lot of boards use SD cards and those might not exactly be the best kind of storage for such data with regards to corruption (and write speed, depending on data bandwith) and requires possibly non-trivial action to get the data from the storage to the PC which analyzes it. But again, totally depends on the application. Also better keep a very good logbook, and/or make sure you can enter filenames or accurate date/time into your program: from experience I can tell you something as simple as mixing up data sets (i.e. which file belongs to what recording session?) is a major cause of problems in science.

[ThomasChr]

I would vote for the Pyboard, it‘s official, stable and has a far better ADC then one of the cheap ESP Boards. It has no WiFi but the new Pyboard-D will have it. Sending the data via WiFi straight to a Server eliminates local storage which is indeed not suited for heavy write cycles.

And if the ADC of the Pyboards STM-ARM does not fit, don‘t forget that you can sinply attach a dedicated ADC IC.

[nekomatic]

Is there a reason that you'd want to use an embedded board for this task, rather than a desktop PC or laptop? I don't want to discourage anyone from getting into MicroPython and the pyboard but to be honest, if I were faced with this sort of requirement my first call would be a USB-connected data acquisition interface and datalogging software running on a PC. You can do a lot with something like a pyboard if you're prepared to spend time developing and testing it, but the performance of an off-the-shelf unit will already be tested and specified.

Unless you need high precision, high speed or high channel count this isn't expensive, and will get you up and running and doing science much quicker than tinkering with an embedded board. You could start by looking at something like a National Instruments USB-6000 for example, which comes with free DAQExpress software, but there are loads of alternatives. If that's not suitable, please tell us more about your requirements e.g. channel count, speed, precision and accuracy, any size or accessibility constraints, and the sort of budget you're working to?

You mention simultaneous sampling for example: how close in time do your samples actually need to be taken? Genuine simultaneous-sampling data acquisition hardware can be had if that's what you need, but it isn't cheap.

[momsi]

First of all, thank for all your replies.

Well, like I said, I just started working on the project and don't have very much info until now.

It's just that I heard of Micro Python and i thought to myself: you possibly could do that with an Arduino, and with Micro Python this would be even nicer, since I know Python for data analysis very well. And the hardware would be very affordable.

Maybe I've been a little over motivated and hastily on this

I do know that the old hardware works with a dedicated data acquisition card that was very expensive .... but it was expensive 30 years ago (and in not exaggerating on this one)... so I don't think it would be that unlikely, that the hardware requirements could be met by a modern microcontroller. Wich would be super cheap and, for me, a better solution than some dedicated and unmodifiable hardware.

However, as I learn more about the experiment, I will reassess the situation and probably come back here.

Thanks again

[ThomasChr]

I think it‘s important to just start. If it works, good. If not, you‘ve spent a few bucks and learned a lot.

Surely we‘ll help you along the way!

Thomas