Douklar and DataSync and S3 Batch Operations
DataSync and S3 Batch Operations move very large amounts of storage; Douklar joins S3 with images, repositories and items in one run.
Which to use
- DataSync and S3 Batch Operations
- Use DataSync for storage-only moves at the highest throughput.
- Douklar
- Use Douklar when the bucket is one part of an app’s move, or to run S3 Batch Operations with a prepared manifest.
What it does
AWS DataSync is a managed transfer service for S3, EFS, FSx, on-premises storage and other clouds. S3 Batch Operations runs one operation over a list of objects at AWS scale.
Where it is the better tool
- Storage breadth and throughput: they are the right tools for very large single-service storage moves.
- S3 Batch Operations works at AWS scale with almost no setup.
Where Douklar is different
- DataSync does not move ECR images, Git repositories or DynamoDB items, and its cross-account setup takes work.
- Douklar uses S3 Batch Operations itself for large buckets: it checks readiness, builds the manifest of missing or changed objects, shows a cost estimate per job and holds the job until you start it.
- Douklar joins the S3 part of a move with the images, repositories and items that belong to the same app.