Environment Variable | Description |
---|---|
AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY | AWS access key with permissions to write to the destination bucket. |
AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY | AWS access key with permissions to write to the destination bucket. |
AWS_SESSION_TOKEN | AWS session token (if required). |
Environment Variable | Description |
---|---|
MINIO_ACCESS_KEY | MinIO access key with permissions to write to the destination bucket. |
MINIO_SECRET_KEY | MinIO secret key with permissions to write to the destination bucket. |
/admin
to a Dgraph Alpha HTTP
address and port (default, โlocalhost:8080โ). Like with all /admin
endpoints,
this is only accessible on the same machine as the Alpha server unless
whitelisted for admin operations.
You can look at BackupInput
given below for all the possible options.
MINIO_ACCESS_KEY
and MINIO_SECRET_KEY
to correspond to Azure
Storage Account AccountName
and AccountKey
.
Once you have the AccountName
and AccountKey
, you can access Azure Blob
Storage locally using one of these methods:
credentials.json
, you can access GCS locally using one of
these methods:
secure=false
with the destination endpoint
in the destination
field:
accessKey
, secretKey
, and sessionToken
parameters can be used to
override the default credentials. Please note that unless HTTPS is used, the
credentials is transmitted in plain text so use these parameters with
discretion. The environment variables should be used by default but these
options are there to allow for greater flexibility.
The anonymous
parameter can be set to โtrueโ to allow backing up to S3 or
MinIO bucket that requires no credentials (i.e a public bucket).
forceFull
field to
true
in the mutation. Each series of backups can be identified by a unique ID
and each backup in the series is assigned a monotonically increasing number. The
following section contains more details on how to restore a backup series.
listBackups
endpoint that lists the
backups in the given location along with the information included in the
manifest.json
file. An example of a request to list the backups in the
/data/backup
location is included below:
location
field is required.
[Manifest]
type. The fields inside the Manifest
type
corresponds to the fields in the manifest.json
file.
export_backup
tool lets you convert a binary backup into an exported
folder.
If you need to upgrade between two major Dgraph versions that have incompatible
changes, you can use the export_backup
tool to apply changes (either to the
exported .rdf
file or to the schema file), and then import back the dataset
into the new Dgraph version.
uid
are changed to [uid]
. Then use the updated schema
when loading data into Dgraph latest.
For example, for the following schema:
export_backup
tool to convert your binary backup into an
export folder.
g01.schema.gz
file.
export_backup
tool you get the speed benefit from the binary backups,
which are faster than regular exports. So if you have a big dataset, you donโt
need to wait a long time until an export is completed. Instead, just take a
binary backup and convert it to an export only when needed.
dgraph.r9.u0108.1621
). The tree of the directory should look like this:
v20.03.1
and v1.2.3
and allow you to encrypt your backups and restore them. This
documentation describes how to implement encryption into your binary backups.
Starting with v20.07.0
, we also added support for Encrypted Backups using
encryption keys sitting on HashiCorp Vault.
Encrypted
flag in manifest.jsonEncrypted
flag is added to the manifest.json
. This flag indicates if
the corresponding binary backup is encrypted or not. To be backward compatible,
if this flag is absent, itโs presumed that the corresponding backup is not
encrypted.
For a series of full and incremental backups, per the current design, we donโt
allow the mixing of encrypted and unencrypted backups. As a result, all full and
incremental backups in a series must either be encrypted fully or not at all.
This flag helps with checking this restriction.
--encryption key-file=value
. Starting with v20.07.0, the Dgraph Alpha
server can alternatively be configured to interface with a
HashiCorp Vault server to obtain keys.
encryption key-file=value
flag or vault
superflag was used for
encryption-at-rest and will now also be used for encrypted backups./admin
endpoint with the following format:
UID
and timestamp
leases are updated accordingly. The latest backup to be restored should contain
the same number of groups in its manifest.json
file as the cluster to which it
is being restored.
RestoreInput
that
can be passed into the mutation.
isPartial
to
true
in your normal restore.
To incrementally restore from a backup to a live cluster, execute a mutation on
the /admin
endpoint with the following format:
/admin
endpoint with the following format:
--encryption key-file=value
, is now part of the restore utility, so you can
use it to decrypt the backup. The file specified using this flag must contain
the same key that was used for encryption during backup. Alternatively, starting
with v20.07.0
, the vault
superflag can be used to restore a backup.
You can use the dgraph restore
command to restore the postings directory from
a previously-created backup to a directory in the local filesystem. This command
restores a backup to a new Dgraph cluster, so itโs not designed to restore a
backup to a Dgraph cluster that is currently live. During a restore operation, a
new Dgraph Zero server might run to fully restore the backup state.
You can use the --location
(-l
) flag to specify a source URI with Dgraph
backup objects. This URI supports all the schemes used for backup.
You can use the --postings
(-p
) flag to set the directory where restored
posting directories are saved. This directory contains a posting directory for
each group in the restored backup.
You can use the --zero
(-z
) flag to specify a Dgraph Zero server address to
update the start timestamp and UID lease using the restored version. If no
Dgraph Zero server address is passed, the command will complain unless you set
the value of the --force_zero
flag to false. If do not pass a zero value to
this command, you need to manually update the timestamp and UID lease using the
Dgraph Zero serverโs HTTP โassignโ endpoint. The updated values should be those
that are printed near the end of the commandโs output.
You use the --backup_id
optional flag to specify the ID of the backup series
to restore. A backup series consists of a full backup and all of the incremental
backups built on top of it. Each time a new full backup is created, a new backup
series with a different ID is started. The backup series ID is stored in each
manifest.json
file stored in each backup folder.
You use the --encryption key-file=value
flag in cases where you took the
backup in an encrypted cluster. The string for this flag must point to the
location of the same key file used to run the cluster.
You use the --vault
superflag to specify the
HashiCorp Vault server address (addr
), role id
(role-id-file
), secret id (secret-id-file
) and the field that contains the
encryption key (enc-field
) that was used to encrypt the backup.
The restore feature creates a cluster with as many groups as the original
cluster had at the time of the last backup. For each group, dgraph restore
creates a posting directory (p<N>
) that corresponds to the backup group ID.
For example, a backup for Dgraph Alpha group 2 would have the name
.../r32-g2.backup
and would be loaded to posting directory p2
.
After running the restore command, the directories inside the postings
directory need to be manually copied over to the machines/containers running the
Dgraph Alpha servers before running the dgraph alpha
command. For example, in
a database cluster with two Dgraph Alpha groups and one replica each, p1
needs
to be moved to the location of the first Dgraph Alpha node and p2
needs to be
moved to the location of the second Dgraph Alpha node.
By default, Dgraph will look for a posting directory with the name p
, so make
sure to rename the directories after moving them. You can also use the -p
option of the dgraph alpha
command to specify a different path from the
default.
--zero
/-z
to update the timestamp.