I found out about #debsums. It popped in my head that "I can use this to see which files I changed on my server and back them up!"
sudo debsums -ca | xargs -I {} zpaq a server-debsums-ca-.zpaq {}
The above uses my favorite archiver--#zpaq (please check it out!), to create incremental, highly compressed backups of such files debsum indicates to us.
This isn't flawless, though. It skips my sshd_config and not every Debian package has an md5 checksum list of every file.