Using FlashAir, what's the timestamp on the data files? - Printable Version +- Apnea Board Forum - CPAP | Sleep Apnea (https://www.apneaboard.com/forums) +-- Forum: Public Area (https://www.apneaboard.com/forums/Forum-Public-Area) +--- Forum: Software Support Forum (https://www.apneaboard.com/forums/Forum-Software-Support-Forum) +--- Thread: Using FlashAir, what's the timestamp on the data files? (/Thread-Using-FlashAir-what-s-the-timestamp-on-the-data-files) Pages:
1
2
|
RE: Using FlashAir, what's the timestamp on the data files? - biorn - 04-21-2021 (04-21-2021, 12:16 AM)cathyf Wrote:(04-20-2021, 09:04 AM)Dog Slobber Wrote:(04-19-2021, 03:06 PM)cathyf Wrote: Ah, well. it was a thought... Yes my first answer was in this way, there is a lot of solution to update the time from the modem but but resmed did not do it. And the new way to get the time on the 5g network is more more efficiant thant gps. (but it's not the subject) So on the flashair you could do a simple trick. That need some investigation but on the main lines for the signal when a new file is created the flashair will change the reference time in the file to the actual time know by the flashair. Maybe there is better solutions but it's the first I got. RE: Using FlashAir, what's the timestamp on the data files? - cathyf - 04-21-2021 (04-21-2021, 04:41 AM)biorn Wrote: So on the flashair you could do a simple trick. That need some investigation but on the main lines for the signal when a new file is created the flashair will change the reference time in the file to the actual time know by the flashair. Maybe there is better solutions but it's the first I got. I'm thinking that it's better that the files that the CPAP is writing should only be read by any other applications. If I understand how FlashPap works, it is copying the files from the flash drive over the network to the computer. That would be the logical point to correct the file metadata by setting the correct UTC timestamps on the computer's copies. Since OSCAR already has some clock drift code in it, probably a logical place to go from there is for OSCAR to compare the time stamps on the files with the time strings in the files and the time strings embedded in the file names. For example, here's my DATALOG/20210420 files: Code: bash-3.2$ ls -lUT Of course all of the time stamps on those files are according to what the time the CPAP thinks it is. If there were a way to make copies with the correct times in the file timestamps while times according to the CPAP clock are embedded in the filename and in the data that the CPAP wrote into the file, then OSCAR is probably 95% of the way to looking at the difference and being able to correct all of the times in all of the data, right? (I have some experience with how this works because I have always managed my data in a particular way that is similar in spirit to what flashpap does. I created a .dmg image on my computer and copied the flash drive to it. Each time I load data, I put the card in my computer, open my .dmg image, copy the files from the card to the computer, eject the flash drive and go put it back into my cpap, and then come back to my computer and load the data from the .dmg into OSCAR (and before that Sleepyhead.) In 6-1/2 years I've only done a handful of nights with the card in the computer rather than in the CPAP machine! Since all my experience is with reading copies of my data, I know how it works when the files on the card in the machine all have time stamps that correspond to what time the CPAP thought it was when they were created and written, but the files that OSCAR (/Sleepyhead) works with are copies and so their timestamps could be manipulated/corrected if there was some way to know what the correction is. And then since the times are in two places -- the metadata on the file and the name of the file, if the copies had the different timestamps then the difference between the CPAP clock and the real time would be a known constant for each file set. At one point in March I went two nights with the machine clock set to 12 hours in the future. I was able to see that when I copied the files to my computer and looked at them less than 12 hours before I had been asleep, my computer was perfectly happy to tell me that the creation times on the files was in the future according to the computer's clock! I actually went in and edited all of the filenames to move them back 12 hours, and all of the recognizable date/times that I could find inside the files I also edited the times back from PM to AM. It more or less worked -- I couldn't find the date/time in the settings so OSCAR thinks that it doesn't know the settings for the first day in the run, but that's minor. Here is the directory listing, and those are the creation timestamps that the cpap machine put on the files, while the filenames were all manually corrected by me.) Code: bash-3.2$ cd ../20210315 RE: Using FlashAir, what's the timestamp on the data files? - biorn - 04-21-2021 I hope I understood all you said, cause my english is worst. First thing is, timestamp creation of a file should never be used. (to much possiblities of corruption) so don't care about it. I don't remember this part of acquisition but Oscar don't should be car about the file name (the time include in the name) because each file include the start time of the data and that in this way edf format was created. Maybe I'm wrong and oscar use it as baseline. In all case it's better to have the name corrected and the crc too. (I mean oscar check it) this part of the file: [attachment=31663] And the last thinks as I can see in your screenshot, the time isn't instant so the script should be care about hours difference and not the minutes or secondes I think. Sorry if I missunderstoud something RE: Using FlashAir, what's the timestamp on the data files? - cathyf - 04-22-2021 (04-21-2021, 03:33 PM)biorn Wrote: I hope I understood all you said, cause my english is worst. Ok, I see what you mean but I'm going someplace different. I'm trying to figure out if there is a simple way to get ResMed data so that it can synchronize properly with other data sources like pulse-oximeters, etc. If everyone is synched to the real time (i.e. the atomic clock in Colorado) then it's trivial. The internal clock in the cpap keeps time well enough that it's going to be off by the same amount for an entire night, at least to the resolution that we need. Every file has metadata, written by the flashcard device driver, and that metadata records the creation date/time according to the ResMed's system clock -- so what time the cpap machine thought that it was when it wrote the metadata. I think -- from what you guys are telling me -- that the FlashAir subsystem actually can know how far off the ResMed system clock is, because the FlashAir has a network connection to your router which knows the atomic-clock-in-Colorado time and it simultaneously has read access to the flash card files and their file metadata which is being written by a device driver that is looking at the cpap system clock. So, for example, on my plain old ordinary flash card that was in my cpap on April 5th, the first file that got written in the DATALOG/20210405 directory is named 20210405_223835_EVE.edf has a creation time according to the device's clock of Apr 5 22:38:36 2021 has data inside the file that includes the string 05.04.2122.38.37768 In other words, the time that the file was created has been recorded in three different places by two different programs that were both relying on the same clock. The ResMed code that runs the CPAP machine constructed the name of the file at 22:38:35 cpap clock time. The ResMed code issued the file create command which the disk driver obeyed at 22:38:36 according to the cpap clock. And then the ResMed code constructed the string that it wrote inside the file at 22:38:37.768. What I think is true (if I understand correctly) is that if this had been a FlashAir card there is another tiny computer that is in the FlashAir device. That computer is talking to the router, and the router is connected to the internet, so that computer can know what time it REALLY is. If the FlashAir computer is sitting there reading the card, waiting for files to appear, then the FlashAir will know when the file appears. If the FlashAir can be programmed so that when a new DATALOG/*/*.edf file appears the FlashAir immediately retrieves the real date & time from the router, then the FlashAir can compare the real date and time that the file appeared with what the cpap clock thinks the date and time is that the file was created. And *boom* the FlashAir card knows exactly how much the cpap clock is wrong. If the FlashAir card can know how far the CPAP's clock has drifted, then somebody who understands the FlashPap scripts and somebody who understands OSCAR could figure out how to get FlashPap to tell OSCAR how to fix up all of the times in the ResMed data so that they are correct according to the real time. Once the ResMed data has the correct atomic-clock-in-Colorado times, then every other data stream which also has the correct atomic-clock-in-Colorado times can be easily synchronized with the ResMed data. If someone else has already tried to figure out how to make this work and that person discovered something that makes it impossible, I'll move on. But if you guys who know more about this than I do think it might work, then I would be game to buy myself a FlashAir card and take the deep dive into the code and see if I can figure out how to get the FlashAir to tell me how to correct all of the time data that my cpap has recorded with the wrong times. What I was hoping when I first asked the question was that the FlashAir computer was already correcting the metadata on the copies of the files that it is responsible for copying over to the computer. But I didn't actually expect that to be happening, since that sort of metadata manipulation is not the usual practice when copying files over networks. (And since I was somewhat aware of the arguments that were made back in the early days of the Internet about times and clocks, I even understand why it's not generally a good idea.) RE: Using FlashAir, what's the timestamp on the data files? - biorn - 04-22-2021 ha got it. You just want the flashair write somewhere the drift clock information. But in fact oscar will drift all the data. So you always got the same difference with other data like oxymeter. The airsense will be ok but the oxymeter will be drifted The best way rest what I was purpose. Or just adjust the clock of the airsense RE: Using FlashAir, what's the timestamp on the data files? - Crimson Nape - 04-22-2021 I don't mean to rain on your parade, but I haven't read any ideas to addressing the following. The FlashAir card is nothing more than a storage device with wireless connectivity. It doesn't possess the logic to know when the data was actually created. Creation time has to be provided by the device creating the data. If a file was created a 3AM but wasn't written to the FlashAir until 7AM, are the times inaccurate at the time of creation and/or at the time the data is being transferred? How does the FlashAir determine when data being written to it is in real-time as opposed to being historic data? Then we have the CPAP itself. For a Resmed, all reporting parameters are not totally functional up to one minute after the displayed starting time. When all parameters are reporting, when was the observation made as opposed to when it is being reported? Another consideration are time variables provided by the oximeter's sensor location and the patient's pulse rate. Even if you were able to synchronize the CPAP and the oximeter to the exact second, there is a lag time from when an event occurs and the time as to when the sensor reads this change. If a patient was wearing the sensor on a toe and is experiencing bradycardia, there is going to be a larger time delta as opposed to having tachycardia and wearing the sensor on a finger. Both the CPAP and oximeter have a ramp, or normalization period, after pressing the start button. On the CPAP, there is a blower pressure ramp-up after staring that is separate from the Ramp option. When does the actual start time occur? As soon as you press the start button, or when the blower is at full pressure? For the oximeter, it takes several seconds for it to acquire and report a pulse and SpO2 value. How long is the normalization time for any given recording session? Viatom/Wellue data files provide a starting time and date, a time duration, and file size in their header. Based on file size and time duration determines the sampling rate of the data. Based on the math proving this, the reported start time is that of the first recorded sample. Since there isn't any value, that I'm aware of, for the time between power-up and sensor normalization, this is the unknown. Based on all the variables I listed above, I don't believe that a time synchronization is feasible or possible. I am interested on your ideas as how to address these unknowns. It may be something simple that I'm not seeing. Please take into consideration that I have been known to march to a different drummer though. - Red RE: Using FlashAir, what's the timestamp on the data files? - biorn - 04-24-2021 (04-22-2021, 09:11 AM)Crimson Nape Wrote: I don't mean to rain on your parade, but I haven't read any ideas to addressing the following. I was thinking the discussion was on timezone problem. not on micro second synch. Sorry if I missunderstood something But flashair is a connecteded object, it's work like a computer in fact. Nucleus RTOS I presume. I don't know when the file is created I just read the screen provided in the discussion and deduct the file is created at start of recording. But yeah you could also use the file crate timestamp drift to drift the starting time in the data of the file if it's an historique file. maybe there is a spi solution, I don't now this card. |