Shrinking VM Disks


#1

Hi, https://www.bytemark.co.uk/docs/servers/cloud/discs/resize/ says

At present, you cannot decrease the size of a disc. We have purposefully disabled this functionality as it can lead to filesystem corruption and data loss.

The UI for editing a disk says

Resize Root Disc “disc-1”: (Discs cannot be made smaller)

But when sizing a new disk there’s no mention it can’t be shrunk in future. Nor does https://panel.bytemark.co.uk/servers/add-server when creating a new VM warn that they can’t be shrunk. Can you please consider updating all the bits of disk-size UI to have the warning so the initial size is considered more carefully. Otherwise it’s easy to think I’ll go for extra now and shrink later when I see what’s needed.

Is there a plan to allow shrinkage in future? I realise you’ll have users that would give back disk still being used by the filesystem, but perhaps the panel can look at the partition table to see it’s unallocated? Presumably, gparted on the existing rescue image supports shrinking the filesystem today, so it’s disk shrinkage that’s the issue? And you could always only enable it after it’s requested from support@.

Thanks, Ralph.


#2

Not necessarily of help, but on BigV I have each BigV disk as a PV for my LVM Volume and then divide it up as I need. To shrink, you do the LVM ‘magic’ and then remove the PV from the Volume and delete it.


#3

Thanks, @TonyRogers, that is useful. But it won’t work for the VM’s initial root disk as that can’t be removed and substituted by another. And it still requires forewarning in the UI that disks can’t be shrunk.


#4

You’re right about the first disk. Is it possible to slurp an image off somewhere else, then delete and reinstate the primary disk, and unslurp the image? I don’t like those high-risk shuffles myself though. It’s relatively cheap to temporarily spin up another BigV for rescue purposes. Maybe support@ can suggest a safer way forwards.


#5

It is possible to boot to a rescue system, so you could in theory add another smaller disk, boot to the rescue system, then copy (dd?) the data across to the new disk. Make the new disk the boot disk, check everything is ok, and then delete the old one. Though I don’t know how easy/reliable that would be to do in practice.


#6

I’d be happy shovelling bits around to juggle disks of different size, but as I said above, the web UI forbids removing the root disk: “[Root disc cannot be deleted]”.

https://www.bytemark.co.uk/docs/api/cloud/discs/ has a DELETE HTTP verb, but it doesn’t mention if some disks can’t be deleted, so maybe that’s a possibility, or just incomplete documentation. And then https://www.bytemark.co.uk/docs/servers/cloud/discs/ tells me I can increase, doesn’t warn I can’t decrease, and says drives can be removed from a shut-down VM, but doesn’t except the root disk.

It’s only https://www.bytemark.co.uk/docs/servers/cloud/discs/resize/ that says “At present, you cannot decrease the size of a disc” as I mentioned at the top, and that doesn’t say the root disk can’t be removed either.


#7

I suppose I could use the rescue image to shrink the filesystem on VM1’s root disk, making the used area small enough for the desired smaller disk size. Then create VM2 with that smaller size, and use the rescue image on that to copy the used bits of disk across the network from VM1. VM1 at this point would still be running the rescue image so its disk isn’t being used by running processes. Then boot VM2 and if all well, delete VM1, root disk and all. But I’d have different hostname, IP addresses, and perhaps other things, plus I’d unnecessarily shuffle disk contents over the network. It’s not equivalent, assuming it would work.


#8

I found out you can pretty much do anything if you are willing to get down and be dirty on the command line. I actually managed to completely screw up everything, so I had to rebuild my vm from scratch from backups, but in this process I manage to resize the root partition, add swap space to a new partition, and make a btrfs volume on the original main partition (/dev/vda2) but mount root as a subvolume on that. I also have a second ‘archive’ disk on the machine, and I was able to dd /dev/zero to the beginning of it and turn the whole disk into a partitionless btrfs filesystem.

The way I did most of the really complicated stuff was from the control panel, insert a cd, and choose the ‘debian current live’ iso. I picked the just command line version. I then open a vnc terminal from the control panel and watch it boot to a login prompt from this cd on the vnc terminal window…

It took me a while to find it, but eventually the internet gave out that you logged in via user user and password live followed by sudo -i to get to be root. I needed the btrfs-tools package to play around with btrfs but a apt-get install btrfs-tools added that to this rescue system for me. From there I was able to shrink the root filesystem, use fdisk to delete and recreate the /dev/vda2 partition to be smaller, and then grow the root file system back up to fill the partition again. And in a chroot was able to run update-grub and grub-install to reinstall grub into the boot sector

Once I was complete, I was able to remove the CD and then restart the vm so I could then login via ssh.


#9

An interesting tale of heroic endeavour, but I’m happy pushing bits about. :‑)
The problem is that the disk cannot be made smaller, thus the bill not reduced, and the UI doesn’t warn of this at appropriate times.

I’ll email Bytemark since they haven’t acknowledged the problem here.


#10

The control panel does warn that disks cannot be made smaller. Click on your disk and it gives you the message. It tells you this when creating a disk too.

Personally, I do not see this as a problem as it is so quick to create a new vm and migrate data.


#11

@phill104, I think you haven’t read this thread carefully, including my opening post where I said

The UI for editing a disk says

Resize Root Disc “disc-1”: (Discs cannot be made smaller)

You’re half agreeing with me, and the other half is wrong. :‑)

Here’s defining the disks on a new VM; no warning.

Adding a disk to an existing VM; no warning.

Here’s the warning: when editing an existing disk, which is too late.

Personally, I do not see this as a problem as it is so quick to create a new vm and migrate data.

It isn’t quick to duplicate the OS, e.g. boot into rescue, cope with the different IP address, coordinate DNS updates, etc. Copying an arbitrary directory of data, fine, yes, but that’s not the issue.


#12

I am not really agreeing with you. I think there is already enough warning such as here - https://www.bytemark.co.uk/docs/servers/cloud/discs/resize/. I would expect people to read the docs and understand this when undertaking a task such as provisioning a server. Then again, as a bloke I hardly ever read a manual at home.

Shrinking a root volume is not only risky, but also can hit the disk array quite hard as it is moving a lot of data around. With us using shared arrays this is probably not a good idea. I am sure this was all taken into consideration and is why the feature is cot available. Like you, I can only speculate.


#13

I think there is already enough warning such as here - https://www.bytemark.co.uk/docs/servers/cloud/discs/resize/.

That’s the link I gave in the first line of my first post for that very reason.

Shrinking the disk is not unavailable to protect the root-volume disk array since I can create and delete data all day long on the same disk array, and I can also currently shrink the filesystem at detailed above. It’s shrinking the disk allocation that can’t be done, so the bill can’t be reduced. It can’t be done to protect idiot users who don’t know what they’re doing and lose data, but I come here to run Unix and that has a long tradition of power, letting me shoot both my feet with a single bullet if I so wish.


#14

Hi all!

It’s sort of possible to resize disks down by shuffling stuff around, or even set up btrfs on vda2 with the correct arcane incantations (props to @akc42 on that!), @ralph is right that it’s not fully clear everywhere that it’s not possible to make a disk volume smaller, especially in creating a new VM where it’s most important.

So, while it’s technically possible to shrink disks, we’d have to make sure that the end of the partition space wasn’t used at all (which we can’t be totally certain about in all cases) and even then there’s still the fairly significant possibility of data loss if something went wrong.

With it being a fairly uncommon requirement, It’s not something available in either the panel or the client, and there’s not really any plan for adding it in the near future as it would require a huge amount of testing before we’d be confident enough to enable it (it would honestly need to be totally bulletproof).

I’ll look into updating the docs, and speak to the relevant people here to see if we can get the panel updated with the notices in all the relevant places.


#15

Would it partially solve the problem if with Bytemark assistance, the current first disk could be replaced with a newer (non-enlarged) one and the data copied across. I imagine the with some behind the scenes assistance this ought to be possible.


#16

Hi @pcammish, Thanks for getting the UI and documentation looked at.

On the risk of losing data, I can already delete a disk as long as I jump the numerous “Are you really sure?” and “Beware of the leopard” hurdles. So this could be another case for “You asked for it” UI confirmation. What if the disk space to be lost had to be filled with a value, e.g. 0xff if that’s good for SSDs, to show I’d already nuked the data and so had nothing to lose?

Interestingly, deleting a VM just asks once if I’m sure then, poof!, it’s gone, disks and all.

Alternatively, as the system stands today, can I add a second disk, make it be bootable, e.g. grub in the MBR, etc? If I then make the “root” disk unbootable, will BigV pootle along and use the next disk? If so, I could clone root to a smaller disk and start booting from that instead. This just leaves being able to delete the current root disk, e.g. assign root-ness to another disk and then delete the non-root disk as two UI steps.


Reformatting root disk
#17

No problem!

I suspect it’s one of those things where it comes up so infrequently that it’s not been a significant issue.[quote=“ralph, post:16, topic:2562”]I could clone root to a smaller disk and start booting from that instead.[/quote]
As is, the ‘disks’ in the Cloud Server platform are effectively disk images which are being mounted by the virtual machine process, so in theory (and I haven’t tested this so don’t try it in production!) it should be possible to clone the existing disk onto a smaller disk, and then remove the original root disk via the bytemark command-line client, which would then move things up one so ‘disc-2’ becomes ‘disc-1’ and so on, but this isn’t exposed in the panel for a few technical and usability issues (again, mainly a fairly large chance of breaking things).