Bacula Virtual/Synthetic Full and Forever Incremental Backups

The Bacula’s Virtual Full backup level is often called Synthetic Backup or Consolidation in other backup systems. It permits you to consolidate the previous Full backup plus the most recent Differential backup and any subsequent Incremental backups into a new Full backup. This new Full backup will then be considered as the most recent Full for any future Incremental or Differential backups.

The Virtual Full backup is accomplished without contacting the client by reading the previous backup data. It reads the volume used to the last Full and Incrmental/Differential backups and write on a volume in the same or different pool, informed by the NextPool Directive. In some respects the VirtualFull resembles a Migration Job, but it has many more specific features.

Configuring the VirtualFull to be written into a different Pool from where your prior backups are saved always guarantees that you will not get a deadlock situation attempting to read and write to the same volume in the Storage daemon. But in general, you can set your Next Pool to point to the same Original backup Pool. In any case, once a VirtualFull has been created, and a restore is done involving the most current Full, it will read the Volume or Volumes by the VirtualFull regardless in which Pool the Volume is found.

The VirtualFull execution can be done manually, changing the Job level in the run command, or systematically, in the Job Schedule.

A typical Job resource definition might look like the following:

Job {
Name = "Backup_Job"
Type = Backup
...
Schedule = virtual_schedule
Pool = Syntetic_Fulls
NextPool= Synthetic_Fulls
}

Alternatively, the NextPool can be set in the Pool resource. The same thing for the Storages, if they are different (from original and destination Pools), such as follows:

# Orignal Pool
Pool {
  Name = Daily_Inc
  Pool Type = Backup
  Recycle = yes            # Automatically recycle Volumes
  AutoPrune = yes          # Prune expired volumes
  Volume Retention = 7d  # one year
  NextPool = Weekly_VFull
  Storage = File1
}

# Destination Pool
Pool {
  Name = Weekly_VFull
  Pool Type = Backup
  Recycle = yes            # Automatically recycle Volumes
  AutoPrune = yes          # Prune expired volumes
  Volume Retention = 30 days  
  Storage = File2
}

To run a manual VirtualFull backup, use such following command in the bconsole:

run job=Backup_Job level=VirtualFull

In order to program regular VirtualFull, define a Bacula Schedule resource such as follows:

Schedule {
  Name = virtual_schedule
  Run=Differential  Pool=Daily    Mon-Thu         at 19:00
  Run=VirtualFull   Pool=Weekly   2nd-5th Friday  at 19:00
  Run=VirtualFull   Pool=Monthly  1st Friday      at 19:00
}

If the Virtual Full is run, and there are no prior Jobs, the Virtual Full will fail with an error.

The Progressive Virtual Full

In Bacula version 9.0.0, there was added a new Directive named Backups To Keep that permits you to implement Progressive Virtual Fulls within Bacula. Sometimes this feature is known as Incremental Forever with Consolidation.

The Incremental Forever backup is the most economic from the storage point of view, since it will only occupy the storage space required by one VirtualFull and subsequent Incrementals, but the administrator might find it harder to deploy it with more complex corporate backup policies, such as based on the grandfather-father-son rotation scheme. Also, is very important all pieces of the backups can be restore, since one missing data can impact the unique version of the backup data if only one consolidation is kept at a time.

To implement the Progressive Virtual Full feature, simply add the BackupsToKeep directive to your Virtual Full backup Job resource. The value specified on the directive indicates the number of backup jobs that should not be merged into the Virtual Full (i.e. the number of backup jobs that should remain after the Virtual Full has completed). The default is zero, which reverts to a standard Virtual Full than consolidates all the backup jobs that it finds.

The new BackupsToKeep directive is specified in the Job Resource and has the form:

Job {
  Name = "ForeverIncremental"
  Type = Backup
  Level = VirtualFull
  ...
  Accurate = Yes
  Backups To Keep = 30
  DeleteConsolidatedJobs = yes
}

The 30 value in the prior example is the number of backups to retain. When this directive is present during a Virtual Full (it is ignored for other Job types), it will look for the most recent Full backup that has more subsequent backups than the value specified. In the above example the Job will simply terminate without action, unless there is a Full back followed by at least 31 backups of either level Differential or Incremental.

Assuming that the last Full backup is followed by 32 Incremental backups, a Virtual Full will be run that consolidates the Full with the first two Incrementals that were run after the Full. The result is that you will end up with a Full followed by 30 Incremental backups.

The new directive DeleteConsolidatedJobs (Job Resource) expects a yes or no value that if set to yes will cause any old Job that is consolidated during a Virtual Full to be deleted. In the example above we saw that a Full plus one other job (either an Incremental or Differential) were consolidated into a new Full backup. The original Full plus the other Job consolidated will be deleted. The default value is no.