+- Apnea Board Forum - CPAP | Sleep Apnea (https://www.apneaboard.com/forums)
+-- Forum: Public Area (https://www.apneaboard.com/forums/Forum-Public-Area)
+--- Forum: Software Support Forum (https://www.apneaboard.com/forums/Forum-Software-Support-Forum)
+--- Thread: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! (/Thread-Time-Zones-and-Clock-Skews-and-Poorly-Designed-Devices-Oh-My)
Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - 0xdec0ded - 05-08-2022
Hi everyone, I'm new to Sleep Apnea and OSCAR (diag. MOSA 1/14/2022, finally received my first APAP from my DME after a very lengthy delay on 5/6/2022). I'm a software engineer by trade as well as a Linux enthusiast (zealot? are we still using that one?).
In addition to my ResMed 11 there in the sidebar I bought a Wellue/Viatom Checkme O2 Max for oximetry.
I'm just getting started and after trying to import both of these sets of data into OSCAR I ran into some issues, did some research, and made some observations I wanted to share.
After finding that all my CPAP data was time shifted wrt my oximetry data I searched around here and found some interesting threads (sorry I can't make them links yet, newbie):
.../forums/Thread-Airsense-11-Time-Zone-Change
.../forums/Thread-Time-zone
.../forums/Thread-Feature-request-Air11-time-zone-have-OSCAR-do-the-right-thing
Which led to discovering the clinical menu and that my already beloved DME did not set the time zone correctly (came set to GMT-5). This also led to the related discoveries that I can't change the time zone without erasing all the data, I can't set the time at all, and that the unit won't adjust the time zone for DST.
So without syncing (which I have no intention of doing, so drift will also become a problem) it's not even clear which time zone I should set it to, if I change it at all. At best I'm only kicking the problem down the road. Due to the noon-noon issue, I may be better served setting a time zone which keeps my anticipated wake time further away from noon during standard time, though that isn't a huge concern.
As you can see, yet a babe barely opening my eyes to this world and I'm already swimming in tough issues!
Despite my naivety, and not having taken one look at the code for OSCAR yet, I'm now going to lay out my list of assertions based on the research I've done, and in a true show of hubris, try to come to some conclusions about what could be done about this mess.
I fully expect that some of my assertions may not be correct or imprecise or simply too narrow in scope as my experience here is very limited. Further, what I suggest may be wholly impractical due to various implementation details. Still, I believe there is some value in sharing and discussing my fresh perspective, especially in light of the equipment that I have which is only going to become more widespread over time.
Thanks for making it this far, and without further ado...
-----
whereas ResMed AirSense 11 AutoSet does not allow the user to set the time
whereas some users are without cellular reception
now, therefore some device clocks will be incorrect due to drift with the device unable to sync time via callback/NTP
whereas ResMed devices do not adjust the timezone for DST (only the time IF they are syncing)
whereas changing the timezone requires erasing all device data
now, therefore changing the device timezone should be avoided
now, therefore some device clocks will be incorrect half the year due to DST changes if there is no cellular reception
now, therefore some device clocks will be set incorrectly for multiple reasons (drift, DST) with no other recourse
whereas ResMed devices split data at noon based on the device timezone setting and clock
whereas shift workers regularly need to sleep across noon localtime
now, therefore it's advantageous for shift workers to set an artificial timezone on ResMed devices different from their localtime
now, therefore some device timezones will be set incorrectly for multiple reasons (shift workers, DST) with no other recourse
whereas users may wish to sync their data across multiple devices with different incorrect timezones and clocks
whereas some users travel with their devices to other timezones
whereas some users wish to view/analyze their data while traveling in the localtime of their destination
whereas some users wish to view/analyze their data when they return home in the localtime of their home
whereas some users permanently relocate to a different timezone
now, therefore potentially incorrect device timezones and time data should be saved as-is and not interpreted by the local timezone of the import
now, therefore the display of time information should be configurable to a preference timezone when viewing (default=local timezone)
whereas clock drift can (and usually does) increase over time
whereas users may have limited to no control for syncing times across devices
now, therefore different data imports (both for different devices, and for the same device at different times) may have different clock skews
whereas some devices do not provide a clock at all (only time offset series data)
now, therefore each data item (not for an entire profile) should store a clock skew/start time preference (determined at import, configurable later, option to disable for existing behavior, default=0 [no skew])
whereas some devices do not provide a timezone at all (only time offset series data)
whereas some data items may have been created with one device timezone setting, in a different recording timezone (e.g. travel) and then viewed in a third different local timezone (e.g. home)
now, therefore each data item should store a timezone preference for how the data is interpreted when displayed, separate from the device timezone setting (determined at import, configurable later, option to disable for existing behavior, default=local timezone)
Let the flaming begin!
RE: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - pholynyk - 05-08-2022
TL;DR
The code is in GitLab, It's written in C++, and uses the QT5 platform.
RE: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - 0xdec0ded - 05-08-2022
Hi Phil,
I'm afraid we're getting off on the wrong foot here.
Was your terse response a passive-aggressive way of letting me know that you don't like what I've said, or a genuine expression of exasperation at the length of my message? The latter seems somewhat unlikely given your response within 5 minutes on a Sunday, and the fact you have contributed 420 public commits to the project since 2017. If you really didn't read my message I can try to summarize it for you, though this is a complicated issue so I was trying to come at it from the top-down so you might follow my reasoning (or easily point out where I've gone wrong).
While I would love to contribute to OSCAR, I'm curious if my contributions would be welcome if you aren't even willing to read what I have to say or engage in a design discussion? I admit my message was long, but would you prefer I just blindly drop a massive PR on you instead? Before I could meaningfully contribute I would want to know what's been tried before or whether there are obvious holes in my logic, invalid assertions, etc. I'm guessing that is information you would have off the top of your head based on your experience with the project and may even be aware of planned features to address some or all of this.
Thanks for your time and I'd be happy to try and summarize my issue and proposal if that would help.
RE: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - pholynyk - 05-08-2022
It was all the Whereas and now therefores listing problems, some of which only apply to the AS11... and not much evidence that you have looked at the available data files, and your solution is to redesign the database.
I would suggest you PM Gideon with your qualifications and ask to be admitted to the Apnea Software Talk forum. That is the place to discuss feature enhancements and database re-design.
RE: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - 0xdec0ded - 05-08-2022
I did try to specify things that are exclusive to the AS11 (like the inability to set the time being the key one) but my knowledge of the older units is even more limited so I could have overstated things in my other assumptions; I was definitely hoping to get feedback anywhere that is the case.
In proposing the addition of a couple fields during an import I envisioned that they would be provided by the user (not from the data files). They would have reasonable defaults which could also be used to convert old data and to keep things simple for novice users (e.g. like an import wizard). I figure the user is the only one who can tell you what you don't know (e.g. how multiple incomplete or inconsistent/conflicting sets of data should line up). If that would require re-designing the database that would be rather unfortunate, but in any case it was not my intention to suggest a database redesign! Perhaps once I familiarize myself with the schema it will become clear why this might be so difficult. I'm a long way from implementation space, really just checking assumptions and floating high-level ideas for discussion.
I'll PM Gideon as you suggest and if admitted, start a discussion about this there.
Thanks again for taking the time to respond.
RE: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - Crimson Nape - 05-08-2022
Not to be one, but all the "whereases" made me think it was a legal pleading.
RE: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - 0xdec0ded - 05-08-2022
If you've ever tried to make a feature request on an OSS support forum before, maybe it was!
It was just a way for me to organize my thoughts as I was trying to assimilate all the various information I was finding while reading forum threads related to this (as I was searching for a solution of course, which morphed into proposing one once I was left with more questions than answers).
RE: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - sawinglogz - 05-09-2022
If you're interested in contributing, there's also a Mantis database of long-term goals, one of which is to handle clock skew. You might find the notes there already address what you've raised, or you might be able to refine our thinking.
Disclaimer: we inherited a codebase with some...interesting design decisions (or lack thereof), so it's a bit of a repairing-the-airplane-while-in-flight situation. As a result, we prioritize supporting new CPAPs and improving import accuracy. A database redesign is in the cards, but fixing it right won't be easy.
RE: Time Zones, and Clock Skews, and Poorly Designed Devices, Oh My! - sawinglogz - 05-09-2022
p.s. Thank you for your CheckMe O2 Max upload. Yours is the first sample file with an odd number of samples. I've already merged a fix for the upcoming beta.