Enterprise Bacula Cloud Storage Plugin in BackBlaze

Enterprise Bacula Cloud plugin use S3 compatible API to store backups, which currently are not supported by BackBlaze. However, Minio distributed object storage server, written in Go and open sourced under Apache License Version 2.0, that can be used to translate all the incoming S3 API calls into equivalent B2 storage API calls, which means that all Minio buckets and objects are stored as native B2 buckets and objects. S3 object layer is transparent to the applications which use S3 API. This way you can simultaneously use both Amazon S3 and B2 APIs without compromising any features.

Configuration

As shown in Figure 1, click in “Show Account ID and Application Key”. Copy and paste the account and generated key information into a notepad. Also, create a bucket with your favorite name.

Enterprise Bacula Cloud Storage Plugin in BackBlaze 1

Figure 1 – B2 buckets dashboard

Download, install and start local Mini gateways with the following commands, replacing the ACCESS_KEY and SECRET_KEY that were fetched earlier.

wget https://dl.minio.io/server/minio/release/linux-amd64/minio -P /usr/bin
chmod +x /usr/bin/minio
export MINIO_ACCESS_KEY=eae4xxxxxfa
export MINIO_SECRET_KEY=00134a639d6xxxxxx6914a046b13b04635a
minio gateway b2 &

Add similar configurations to your bacula-sd.conf. This can also be done using Enterprise Bacula Bweb GUI.

Cloud {
  Name = BackBlaze
  Driver = "S3"
  HostName = "localhost:9000"
  BucketName = "heitor-bacula"
  AccessKey = "eae4xxxxxfa"
  SecretKey = "00134a639d6xxxxxx6914a046b13b04635a"
  Protocol = HTTP
  Upload = EachPart
  UriStyle = Path 
}

Device {
  Name = BB
  Device Type = Cloud
  Cloud = BackBlaze
  Archive Device = /opt/bacula/backups
  Maximum Part Size = 10000000
  Media Type = CloudType
  LabelMedia = yes
  Random Access = Yes;
  AutomaticMount = yes
  RemovableMedia = no
  AlwaysOpen = no
}

Attach the new device and associated pool to Bacula Director.

Autochanger {
  Name = BB
# Do not use "localhost" here
  Address = hfaria-asus-i5                # N.B. Use a fully qualified name here
  SDPort = 9103
  Password = "O0yFMuPy2jZM8L7eMg9TQccW4SvXdVl-n"
  Device = BB
  Media Type = CloudType
  Maximum Concurrent Jobs = 10        # run up to 10 jobs a the same time
  Autochanger = BB                 # point to ourself
}

Pool {
  Name = BBPool
  Pool Type = Backup
  Recycle = yes                       # Bacula can automatically recycle Volumes
  AutoPrune = yes                     # Prune expired volumes
  Volume Retention = 365 days         # one year
  Maximum Volume Bytes = 50G          # Limit Volume size to something reasonable
  Maximum Volumes = 100               # Limit number of Volumes in Pool
  Storage = BB
}

Restart both bacula-sd and bacula-dir daemons to apply changes.

Run a test job (e.g) using bconsole. Backup log (messages) should display information about upload success and transfer rate:

run job=BackupClient1 pool=BBPool yes
messages
+----------------------------------------------------------------------------------------------------+
| logtext                                                                                              |
+----------------------------------------------------------------------------------------------------+
| hfaria-asus-i5-dir JobId 31: Start Backup JobId 31, Job=BackupClient1.2018-03-20_18.43.26_03         |
| hfaria-asus-i5-dir JobId 31: Using Device "BB" to write.                                             |
| hfaria-asus-i5-sd JobId 31: Volume "teste3" previously written, moving to end of data.               |
| hfaria-asus-i5-sd JobId 31: s3_driver.c:583 Using S3 cloud driver Host=localhost:9000 Bucket=heitor-bacula |
| hfaria-asus-i5-sd JobId 31: Elapsed time=00:00:01, Transfer rate=8.581 M Bytes/second                |
| hfaria-asus-i5-sd JobId 31: Cloud Upload transfers:                                                  |
| hfaria-asus-i5-sd JobId 31: teste3/part.18    state=done    size=8.588 MB duration=14s               |
| hfaria-asus-i5-sd JobId 31: Sending spooled attrs to the Director. Despooling 4,698 bytes ...        |
| hfaria-asus-i5-dir JobId 31: Bacula Enterprise hfaria-asus-i5-dir 8.10.1 (13Feb18):
  Build OS:               x86_64-pc-linux-gnu-bacula-enterprise ubuntu 16.04
  JobId:                  31
  Job:                    BackupClient1.2018-03-20_18.43.26_03
  Backup Level:           Full
  Client:                 "hfaria-asus-i5-fd" 8.10.1 (13Feb18) x86_64-pc-linux-gnu-bacula-enterprise,ubuntu,16.04
  FileSet:                "Full Set" 2018-03-18 09:42:04
  Pool:                   "BBPool" (From Command input)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "BB" (From Pool resource)
  Scheduled time:         20-mar-2018 18:43:26
  Start time:             20-mar-2018 18:43:28
  End time:               20-mar-2018 18:43:43
  Elapsed time:           15 secs
  Priority:               10
  FD Files Written:       22
  SD Files Written:       22
  FD Bytes Written:       8,578,863 (8.578 MB)
  SD Bytes Written:       8,581,196 (8.581 MB)
  Rate:                   571.9 KB/s
  Software Compression:   None
  Comm Line Compression:  40.0% 1.7:1
  Snapshot/VSS:           no
  Encryption:             no
  Accurate:               no
  Volume name(s):         teste3
  Volume Session Id:      2
  Volume Session Time:    1521582178
  Last Volume Bytes:      662 (662 B)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK |
| hfaria-asus-i5-dir JobId 31: Begin pruning Jobs older than 6 months .                                |
| hfaria-asus-i5-dir JobId 31: No Jobs found to prune.                                                 |
| hfaria-asus-i5-dir JobId 31: Begin pruning Files.                                                    |
| hfaria-asus-i5-dir JobId 31: No Files found to prune.                                                |
| hfaria-asus-i5-dir JobId 31: End auto prune.                                                         |
+----------------------------------------------------------------------------------------------------+

 

Disponível em: pt-brPortuguês (Portuguese (Brazil))enEnglishesEspañol (Spanish)

Leave a Reply