Opened 23 months ago
Last modified 23 months ago
#90 new task
Cost of backup is estimated
| Reported by: | matthew | Owned by: | chris |
|---|---|---|---|
| Priority: | major | Milestone: | Maintenance |
| Component: | backups | Version: | |
| Keywords: | Cc: | ||
| Estimated Number of Hours: | 0.5 | Add Hours to Ticket: | 0 |
| Billable?: | yes | Total Hours: | 1.1 |
Description
It would be useful to estimate the expected cost of back up, by estimated the size and correlating this with the advertised price page.
Attachments (1)
Change History (11)
comment:1 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.1
- Total Hours set to 0.1
comment:2 Changed 23 months ago by matthew
thank you
comment:3 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.1
- Total Hours changed from 0.1 to 0.2
comment:4 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.1
- Total Hours changed from 0.2 to 0.3
No joy in mounting the Crin2 s3ql backups yet, this is where it is getting stuck:
Backing up old metadata... Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 3)... Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 4)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 5)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 6)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 7)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 8)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 9)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 10)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 11)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 12)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 13)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 14)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 15)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 16)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 17)...
comment:5 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.1
- Total Hours changed from 0.3 to 0.4
I'm afraid that the attempts to mount the Crin2 s3ql backups are on-going...
WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 66)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 67)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 68)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 69)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 70)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 71)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 72)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 73)...
Changed 23 months ago by chris
comment:6 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.15
- Total Hours changed from 0.4 to 0.55
I'm afraid the Crin2 s3ql backup filesystem still hasn't mounted:
WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 57)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 58)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 59)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 60)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 61)...
The Advania storage prices are here:
But I don't know which rate is used.
comment:7 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.15
- Total Hours changed from 0.55 to 0.7
It finally got there but then failed for another reason (Backend reports that fs is still mounted elsewhere?!):
WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 59)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 60)... WARNING: Encountered ConnectionTimedOut (send/recv timeout exceeded), retrying Backend.copy (attempt 61)... Cleaning up local metadata... Completed fsck of s3c://s.qstack.advania.com:443/crin2/ Filesystem check of s3c://s.qstack.advania.com:443/crin2 success Autodetected 65501 file descriptors available for cache entries Ignoring locally cached metadata (outdated). ERROR: Backend reports that fs is still mounted elsewhere, aborting. Mounting s3c://s.qstack.advania.com:443/crin2 on /media/s3ql/crin2 failed
So trying again:
mnt-s3ql crin2 No problem with disconnected endpoints Starting fsck of s3c://s.qstack.advania.com:443/crin2/ Ignoring locally cached metadata (outdated). File system is marked as clean. Use --force to force checking. Filesystem check of s3c://s.qstack.advania.com:443/crin2 success Autodetected 65501 file descriptors available for cache entries Ignoring locally cached metadata (outdated). Downloading and decompressing metadata ...
Mounting Crin4:
mnt-s3ql crin4 No problem with disconnected endpoints Starting fsck of s3c://s.qstack.advania.com:443/crin4/ Using cached metadata. File system is marked as clean. Use --force to force checking. Filesystem check of s3c://s.qstack.advania.com:443/crin4 success Autodetected 65501 file descriptors available for cache entries Using cached metadata. Setting cache size to 4839 MB Mounting s3c://s.qstack.advania.com:443/crin4/ at /media/s3ql/crin4... Mounting s3c://s.qstack.advania.com:443/crin4 on /media/s3ql/crin4 success
And the space used:
df -h Filesystem Size Used Avail Use% Mounted on s3c://s.qstack.advania.com:443/crin4/ 1.0T 59G 966G 6% /media/s3ql/crin4
comment:8 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.15
- Total Hours changed from 0.7 to 0.85
And success!
Ignoring locally cached metadata (outdated). Downloading and decompressing metadata... Reading metadata... ..objects.. ..blocks.. ..inodes.. ..inode_blocks.. ..symlink_targets.. ..names.. ..contents.. ..ext_attributes.. Setting cache size to 3001 MB Mounting s3c://s.qstack.advania.com:443/crin2/ at /media/s3ql/crin2... Mounting s3c://s.qstack.advania.com:443/crin2 on /media/s3ql/crin2 success
Space wise:
df -h Filesystem Size Used Avail Use% Mounted on s3c://s.qstack.advania.com:443/crin2/ 1.0T 127G 898G 13% /media/s3ql/crin2
So the total:
307+127+59 493
But remember that this 493G is how big the filesystem appears without deduplication and without compression -- it will hopefully be using a lot less space on the server.
comment:9 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.1
- Total Hours changed from 0.85 to 0.95
And with all the filsystems mounted on Crin3 this is how it looks:
df -h Filesystem Size Used Avail Use% Mounted on udev 489M 0 489M 0% /dev tmpfs 100M 11M 90M 11% /run /dev/mapper/CRIN3--vg-root 15G 9.6G 4.5G 69% / tmpfs 499M 0 499M 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 499M 0 499M 0% /sys/fs/cgroup /dev/sda1 236M 60M 165M 27% /boot tmpfs 100M 0 100M 0% /run/user/1000 crin2:/ 121G 25G 91G 22% /media/sshfs/crin2 s3c://s.qstack.advania.com:443/crin4/ 1.0T 59G 966G 6% /media/s3ql/crin4 s3c://s.qstack.advania.com:443/crin2/ 1.0T 127G 898G 13% /media/s3ql/crin2 s3c://s.qstack.advania.com:443/crin1/ 1.0T 309G 716G 31% /media/s3ql/crin1 crin1:/ 121G 27G 88G 24% /media/sshfs/crin1 crin4:/ 15G 6.3G 7.8G 45% /media/sshfs/crin4
I suggest we should do some follow up on ticket:66.
comment:10 Changed 23 months ago by chris
- Add Hours to Ticket changed from 0 to 0.15
- Total Hours changed from 0.95 to 1.1
Based on the prices on ticket:90#comment:6 the cost for the lead tier in US dollars per month:
493 / 40 * 2.25 = 27.73125
And the the platinum tier in US dollars per month:
493 / 40 * 8.98 = 110.6785
However we don't know how much disk space we are actually using and we don't know which tier we are on... so the only thing I think we could say with confidence is that they are probably invoicing for something in the region of £20 and £100 per month?
Note: See
TracTickets for help on using
tickets.
I have started to mount the s3ql filesystems on Crin3 so we can get an idea of the space usage, but I don't think this will be accurate as the data will be compressed and de-duplicated on the server, what about logging into Advania and checking if they display space used?
There is also this issue, ticket:66 -- in the past it has been really time consuming to delete old backups so this hasn't been done manually as often as it should have and also it hasn't been automated, sorting this issue out should make a big difference.