When setting up scheduled backups in FileMaker Server I always select the optional verify backups option. I guess a backup that is not verified will run faster, But I don't want to have unverified backups if something goes wrong. As this is an option that is by default off I am interested to know when it is good to run backups with verify turned off? I am just paranoid? Is verifying backups not a good idea?
Different things I would say here.
I think the verify option can be always on. If it is clear to you the verification process is taking resources (longer backup, users feel a performance drain when the backup is triggered), than you most likely will want to turn it off and keep it on a single daily backup, off peak usage.
Note that there is also a newer option in FileMaker server to verify all your databases at once. That verification is done on the live files (not the backup) and all files are closed in the process. I presume a file that fails the verification will not be opened again by the server, leaving you in a safer state than if you miss the warning from the backup and the live file stayed up and running in that bad state.
It is not a topic where I can say I have a ton of experience, so it could be that I what I mention above would benefit from someone else who has more "time behind the wheel" correcting anything I may say that is not quite right.
due to the fact that a standard schedule will create individual folders (repeations, timestamp) that will fullfill every external backup-device sooner or later - we often have 'single-day-backup(schedules)', means for example per weekday one separate schedule with no repeations but with unique names ('monday lunch'). Those backups will be copied to an external device (NAS, whatever) by an utility (here mostly chronosync).
With this setup, You can have (for example) the 'sunday-schedule' with the verify-option 'on', the rest of the backups during weekdays 'off' for speed, with bigger files faster...
(We do have hourly-schedule if needed and the setup from above will get more complicated when having many folders with separate schedules, etc)
You can also define a schedule just to verify (imho, I'm off-office)
I agree. I am curious as to why it is off be default!
yes, speed. I think it is important to have the option. Speed or safety seems to be the choice. When it comes to backups I would default to safety over speed. However the Claris developers set the default to speed over safety.
I would prefer speed and safety of course. If the default is not to verify, then I wonder how important the verify option is?
Personally I do as others seem to do: work-day backups are not verified, so they run as fast as possible, out-of-hours backups have the verify flag set. Which seems to be good practice.
However I hope I never have to restore from corrupted, unverified backup. I guess the hope is that things will be fine, or if not I can go back to last night's verified backup. Still the idea of "fingers crossed" all will be fine, makes me uneasy!
All of us would, but those stand at opposite ends of the same continuum. It applies to cars we drive on the road, it applies to CPUs (I believe Meltdown to be an example of that) and pretty much everything you can imagine, including backups.
If we go back to the triangle "cheap, fast, good" where you can only pick 2 (borrowed here), speed is clearly something you chose OVER other options, at the price of degrading something else.
Finding your way out of this will get you many friends and many people who will be suspicious and take a closer look at what is done to "achieve" this.
@Bobino interesting thoughts. However I don't see where cost comes in to this. I also disagree that there is a continuum, that would imply a sliding scale between max. speed and max. verification, with 50% speed and 50% verification between each extreme! This is not a triangle, this is two sides of a coin: heads you gain speed and lose verification tails the reverse.
I treat backup and verification as separate processes. Backup often, verify regularly.
In earlier versions of FMS large files would take a long time to verify. And while FMS was verifying one file it was too busy to pay very much attention to other things. If there was any issue in a file, such as a malformed image in a container field ( very common ), FMS would claim the file was corrupt. In the days before external container storage file bloat from contained resources was a serious issue. I remember clients force-quitting the server to stop it because verification was taking so long. That doesn't do a lot for file integrity BTW.
sounds like good advice
Here is a good balance between safety/monitoring and speed...
- Verify the daily backup that runs off hours. Make sure (email) notification are on.
- Do NOT Verify the hourly backups that run while people are working the most.
In several deployments with mission-critical data we have hourly (10), daily, and weekly backups scheduled, plus progressive backups enabled. Hourly are not verified, but all others are. This makes for about 23 copies of the databases which (some) are copied to a network storage device (RAID) plus cloud storage less frequently. This setup has been a comfortable balance of redundancy and performance.
We have had to recover from hourly copies on a few occasions and had no issues, but I agree with others that verification provides some peace of mind.
In a nutshell the consensus seems to be: back up often and verify regularly.
During working hours do not verify to improve speed, so as not to disrupt users. Run a verified backup "out of hours", preferably daily, and make sure to monitor for errors.