Best way to keep full backups without redundancy
Can and would someone please advise me? The situation: my hard drive is filling up due to my sloppy method of backing up ALL data from an Autoset A10. Regular antivirus scanning overworks the drive, takes far too long and slows down my W10 PC.
I have routinely copied from the SD card to my hard drive each night's set of two folders (DATALOG, SETTINGS) and three files (.crc, .tgt and .edf) generated by, or as updated by, the Autoset A 10 "last night". Oximetry data is embedded in those files as well. Each night of data is put into a new folder, i.e., last night's data is in the folder "170225" in my
CPAP folder. (I have noticed that the Autoset creates subfolders in the DATALOG folder that are named similarly, but it is no problem.)
Thinking I'd copy all to a jumpdrive and clear some storage space I found, duh, that there is, for a year and a half accumulation, at least a couple hundred gigabytes of files. My questions come down to two:
1. Isn't each daily (nightly) set of files indexed and self-referencing within itself so one cannot simply move a desired set of dated daily subfolders of the DATALOG folder, for a given sequence of nights into another date's DATALOG folder. I doubt one could see all such moved data upon downloading all to a blank SD card; it seems likely that necessary referencing and indexing would be absent.
2. Is the only way to preserve coverage, in view of a 30 day detailed data storage limit for SH or RS (as I recall), simply to retain, say, every 20th-25th daily folder, starting from the first one recorded and then deleting all 19-24 folders in between?--a scheme for preserving a means of reaching all days recorded...recognizing that one has to create a new patient name for each dataset (span of time) of interest.
Some past experience shuffling data:
I have found it necessary a time or two, for reasons forgotten, to erase the data card and download from storage older stored info, for its review, onto that emptied card. (I think the trouble I addressed was that in Sleepyhead or ResScan there were limits on how far back I could revisit Daily sleep records: summaries and detail.)
I have no particular qualifications or expertise with respect to the apnea/cpap/sleep related content of my posts beyond my own user experiences and what I've learned from others on this site. Each of us bears the burden of evaluating the validity and applicability of what we read here before acting on it.
Of my 3 once-needed, helpful, and adjunctive devices I have listed, only the accelerometer remains operative (but now idle). My second CMS50I died, too, of old age and the so-so Dreem 2 needs head-positioning band repair--if, indeed, Dreem even supports use of it now.
02-27-2017, 09:34 AM
(This post was last modified: 02-27-2017, 09:35 AM by Sleeprider.)
RE: Best way to keep full backups without redundancy
Sleepyhead automatically creates a backup and can rebuild data at anytime from those backups. I can't imagine what use these backups might be. Most of us use data to view our current therapy and observe any trends. Keeping full SD card backups serves no purpose...delete them, and keep one backup at 6-month intervals if you ever want to go back and look for some reason. There is a point where healthy curiosity and self-help devolves into obsession, and you may have found it.
If you're going to do backups like this, just use a back-up program and do incremental backups. It will identify the changed file and add it.
RE: Best way to keep full backups without redundancy
Here are some suggestions that may solve your problem. I have listed two different scenarios to address the type of SD card that you may be using.
Standard non-WiFi card:
Assumptions:
- You are using a Windows version 7 or newer.
- You have a Directory named: C:\A10Data
- The SD card displays as: Drive H:\
Create or save the following as a “.bat” file:
ROBOCOPY "H:\" "C:\A10Data" /MIR
After inserting your SD card in your PC, double-click this bat file name or create a shortcut on your desktop. This will add any file that is on the SD card to you backup directory and remove any file from the directory that is no longer on the SD card.
To automate this process using a WiFi SD card:
Assumptions:
- You are using a Windows version 7 or newer.
- You have a Directory named: C:\A10Data
- The SD card has the following parameters set in the SD_WLAN\CONFIG file:
- APPNAME=A10
- WEBDAV=1
- DHCP_Enabled=NO
- IP_Address=192.168.1.39
- Subnet_Mask=255.255.255.0
Create or save the following as a “.bat” file:
ROBOCOPY "\\A10" "C:\A10Data" /MIR
or (not both)
ROBOCOPY "\\192.168.1.39" "C:\A10Data" /MIR
Go to your Task Scheduler and create a task to run at a time when you know you won’t be using your CPAP. You can set it to run daily, weekly, or whenever you wish.
This will preform the same procedure as the first example only you've automated the process.
I hope this may be what you are looking for or provide a spark to your imagination.
RE: Best way to keep full backups without redundancy
Thank you for your constructive, DSM-IV-free responsiveness, Crimson.
You stated, "This [procedure] will add any file that is on the SD card to you backup directory and remove any file from the directory that is no longer on the SD card." (My emphasis added.)
Do I misunderstand that the removal means that the older information is continually being removed as subsequent nights of collected data are recorded (backed up)?
Though it may be hard to "imagine" I still have unexplained, extended--but rarer--periods of sleep with a similar, anomalous (pathological?) breathing waveform. (No need to complain here about my sleep MD's common, blinkered interest in and focus only on compliance and meeting the AHI (or is it RDI) <5 treatment standard. I expect, one day soon, to consult a pulmonologist with summary examples, trend, etc., in hand.
Lagniappes: (1) With a math and physics background, mostly forgotten now, recurring, unexplained patterns/waveforms still catch my interest, especially those bearing on one's vital functions. (2) Between my first days owning a computer in 1982 (Kaypro II, 64k RAM, only two 180k floppy disk drives, using CP/M) and now I've not only been a DOS system hotshot, but have also stranded enough data files in proprietary backup programs to become comfortably lazy and out of date in using such primitive copying methods as I described earlier.
I see your suggestion as automation of a simple copying technique and will see what I can do with it if it maintains a comprehensive, accessible archive. Again, thank you.
I have no particular qualifications or expertise with respect to the apnea/cpap/sleep related content of my posts beyond my own user experiences and what I've learned from others on this site. Each of us bears the burden of evaluating the validity and applicability of what we read here before acting on it.
Of my 3 once-needed, helpful, and adjunctive devices I have listed, only the accelerometer remains operative (but now idle). My second CMS50I died, too, of old age and the so-so Dreem 2 needs head-positioning band repair--if, indeed, Dreem even supports use of it now.
03-01-2017, 02:20 PM
(This post was last modified: 03-01-2017, 02:21 PM by TBMx.)
RE: Best way to keep full backups without redundancy
/MIR is not really what you want.
drop the /MIR and instead use /E (copies all subfolders^^)
robocopy on windows has a ton more options which you can see with the usual "robocopy /?"
(or you could just simply copy the card with windows always in the same folder - since windows 10 the explorer comes with a "nag-free" folder integration mode^^ - simply hit the "yes [, please do whatever you think is right]" button once and you'll be done)
might also be a good idea to simply buy an external hard-drive / NAS / or whatever is capable of long-time storage (USB-Pen flashdrives do NOT qualify for that!)
03-02-2017, 02:20 AM
(This post was last modified: 03-02-2017, 02:23 AM by srlevine1.
Edit Reason: typos
)
RE: Best way to keep full backups without redundancy
As the former chief technologist (retired) of a computer company, here is how I do it ...
I have found that keeping the original data files from the SD card to be worthless as it is simply unprocessed raw data that is of zero value unless it is read and re-formatted by analysis software such as ResScan or #SleepyHead.
I do a daily backup of my entire system (which includes both the ResScan and SleepyHead data directories.) This is done to one of three external hard drives.
Semi-annually, I burn the accumulated data in the directory to a DVD which holds 4.7 gigabytes of data. I then delete any data older than 1.5 years.
I have replaced two laptops and experienced one catastrophic disk failure since 2008.
I rarely look back at data older than one year and I have never had a doctor ask for raw data or data on a disk.
I do print a year's summary report to pdf at the end of the year for historical records and file it with my 1040 tax file.
Best practice is something that you will actually do on a consistent and periodic basis using the following guideline: "What would happen if you didn't have that historical data."
It is my understanding that my ResMed AutoSense AutoSet SD card carries 365 days of usage data, 30 days of detail data and 30 days of high resolution data. In addition the system may transmit usage data to MyAir or ResMed's AirView cloud system. ( http://www.resmed.com/epn/en/healthcare-...cards.html) ResScan stores both the usage and detail data if set correctly. Ditto SleepyHead.
Best of luck with whatever backup system you decide to use.
"The object in life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane." -- Marcus Aurelius
|