I think you’ve put more thought into how to get started than many others would! You have a pretty good plan from what it seems. My thoughts from each section below.
Hardware: I’m partial to Crucial and Kingston for storage that is affordable and dependable
OS: I’d probably spin up a Debian install if I were in your shoes and run my services using docker-compose files. It’s a quick and easy to get up and running, and despite the ease, there is still the option to do a lot of customization when you want to, and that will make it easy to learn more at your own pace and leisure.
Services: For the CalDav portion, I’m really liking Radicale.
Security: PiVPN is what I’m running on my actual RPi along with PiHole, and it was a super simple setup. I connect via Wireguard from any of my other devices.
Ultimately up to you, but I’d go with no GUI and just use ssh (and sftp if you need to do file transfers).
When I was using Docker, it was headless because the GUI just ate up space and resources I didn’t need. All your interaction will be in the shell anyway, launching your compose.yml files.
But, if dealing with a headless machine sounds like more trouble than you want to try, install the DE if your choice and breathe easy because it’ll still work perfectly fine.