Revisiting bsdiff as a tool for digital preservation

by @beet_keeper

I introduced bsdiff in a blog in 2014. bsdiff compares the differences between two files, e.g. broken_file_a and corrected_file_b and creates a patch that can be applied to broken_file_a to generate a byte-for-byte match for corrected_file_b.

On the face of it, in an archive, we probably only care about corrected_file_2 and so why would we care about a technology that patches a broken file?

In all of the use-cases we can imagine the primary reasons are cost savings and removing redundancy in file storage or transmission of digital information. In one very special case we can record the difference between broken_file_a and corrected_file_b and give users a totally objective method of recreating corrected_file_b from broken_file_a providing 100% verifiable proof of the migration pathway taken between the two files.

#ac3 #Archives #audio #audiovisual #Audit #authenticity #av #Bash #bsdiff #checksums #Code4Lib #corruption #corruptionIndex #digipres #DigitalArchiving #DigitalForensics #digitalLiteracy #DigitalPreservation #DigitalStorage #diplomatics #FileFormats #glitch #glitchAudio #GlitchArt #integrity #PreservationAnalysis #PreservationMetadata #provenance #sensitivityIndex #Storage

What is the checksum of a directory? Introducing sumfolder1

by @beet_keeper

In Fractal in detail: What information is in a file-format identification report? I describe the different ways of dissecting the information in a file-format identification report.

A file-format identification report is a data-rich artifact created during the processing of digital collections.

I had the idea of using this type of report to attach a checksum to an archival collection (files, and directories) as a whole. This is done using methods akin to a Merkle Tree, similar to those in source control systems such as Git, and Web3 Blockchain projects like Bitcoin.

This project is called sumfolder1.

#Archives #Bitcoin #checksums #collectionIntegrity #digipres #DigitalPreservation #DROID #MerkleTree #Python #SoftwareDevelopment #web3

The sensitivity index: Corrupting Y2K

by @beet_keeper

In December I asked “What will you bitflip today?” Not long after, Johan’s (@bitsgalore) Digtial Dark Age Crew released its long lost hidden single Y2K — well, I couldn’t resist corrupting it.

Fixity is an interesting property enabled by digital technologies. Checksums allow us to demonstrate mathematically that a file has not been changed. An often cited definition of fixity is:

Fixity, in the preservation sense, means the assurance that a digital file has remained unchanged, i.e. fixed — Bailey (2014)

It’s very much linked to the concept of integrity. A UNESCO definition of which:

The state of being whole, uncorrupted and free of unauthorized and undocumented changes.

Integrity is massively important at this time in history. It gives us the guarantees we need that digital objects we work with aren’t harboring their own sinister secrets in the form of malware and other potentially damaging payloads.

These values are contingent on bit-level preservation, the field of digital preservation largely assumes this; that we will be able to look after our content without losing information. As feasible as this may be these days, what happens if we lose some information? Where does authenticity come into play?

Through corrupting Y2K, I took time to reflect on integrity versus authenticity, as well as create some interesting glitched outputs. I also uncovered what may be the first audio that reveals what the Millennium Bug itself may have sounded like! Keen to hear it? Read on to find out more.

#ac3 #Archives #audio #audiovisual #authenticity #av #Bash #checksums #Code4Lib #corruption #corruptionIndex #digipres #DigitalArchiving #digitalLiteracy #DigitalPreservation #diplomatics #FileFormats #flac #glitch #GlitchArt #glitchaudio #integrity #mp3 #sensitivityIndex #wav

Stressful start to the year?

You have one less worry with #CouchDB 😌

Our latest article covers @couchdb’s #checksums feature: a built-in safeguard that helps protect your data from the impact of ever-dreaded disk corruption.

It’s just one of the features that makes CouchDB your data’s safe place.

Full post on our blog: https://neighbourhood.ie/blog/2025/01/22/how-couchdb-prevents-data-corruption-checksums

@zenbrowser Nice! Would you pls publish a regular page on your site with all the #hashsums / #checksums for the #tarballs? Alternatively provide these in your #github pages? Eg, github.com/zen-browser/desktop… really should include these sums, but afaict does not. This request is for both Stable & Twilight.

#ZenBrowser

Verify Your Downloads Like a PRO!

https://spectra.video/videos/watch/fb438752-a012-4c10-80d8-2387eb2a4a84

@ParadeGrotesque I approve of your The Unix Way mastery, though isn't there also something like this (#openbsd) #checksums

sha512 -h hashes.512 files/*
to make the hashfile

whence
sha512 -C hashes.512 files/*

@Sevoris it's all based on #checksums. Even if you change the location of the file, the #SHA256 checksum for the file will stay the same. It can log changes per operation between renames and edits. But if you edit a file and then rename it in the same commit, Git will think you deleted the old file and created a new one.

@Anoxinon Da sind wir wieder. \o/
Und alles schön safe, mit #borg #backup, #btrfs #snapshot und nun auch #postgres #checksums

Computing #Adler32 #Checksums at 41 GB/s

Nice review
https://wooo.sh/articles/adler32.html

"#Linux #Fu: #Roll With The #Checksums"

https://hackaday.com/2022/06/22/linux-fu-roll-with-the-checksums/

ARCH / MANJARO USERS: Want To Check For File Changes not matching source original?

#Manjaro #Arch #Linux #integrity #checksums #malware #foss #Pinephone

https://www.buymeacoffee.com/politictech/arch-linux-checking-installed-package-integrity-checksums-file-changes

DEBIAN USERS: Ever want to check currently installed packages to look for original file modifications? (ie: malware)

Try out debsums + rdebsums here:

#Debian #Linux #rdebsums #debsums #integrity #checksums #malware #foss #Mobian #Pinephone #Librem5

https://www.buymeacoffee.com/politictech/essentials-checking-file-and-package-checksum-integrity

Linux Beginners Guide: Checking File And Package Checksum Integrity

#Linux #Checksums #Integrity #FOSS #Debian #debsums #SHA1 #SHA256 #SHA512 #Cryptography #Hash #Howto #Tutorial #Shell #Bash

https://politictech.wordpress.com/2022/01/05/linux-essentials-checking-file-and-package-checksum-integrity/

For file integrity, 11.30.2021.img file checksums additionally listed on the 🔐secure, #anonymous 🧅 :tor: #Gitea Tor Hidden Service (#E2EE between tor clients) + hosted outside download server:
#Checksums #Integrity #Linux #Tor #HiddenService #Infosec #Security
http://gg6zxtreajiijztyy5g6bt5o6l3qu32nrg7eulyemlhxwwl6enk6ghad.onion/RightToPrivacy/Pinedio-RAK-Armbian/src/branch/master/checksums

Avoid #backdoor downloads by comparing #checksums for file integrity (avoid #MITM (#ManInTheMiddle) #backdoor file swaps/bad downloads). (PGP signatures for future video)
🔒 :linux:

TIP: Where possible, compare to checksums listed on separate server (such as gitlab project) from download server (helps further verify integrity).

We download & Pop!_OS Linux in this example & install it next! #PopOS #System76 #Infosec #Cybersecurity #linux

On #Odysee:

https://odysee.com/@RTP:9/popos-beginners-install%2C-checking:5

Addressing #CPAN #vulnerabilities related to #checksums

HTTP only Blog post
http://blogs.perl.org/users/neilb/2021/11/addressing-cpan-vulnerabilities-related-to-checksums.html

My digital life: Sequel

#drwing #comic #nicknames #checksums #geek

Tool of the day: #jacksum

jacksum is a #java based tool for computing and verifying #checksums, CRCS and message digests. It supports 58 algorithms like #cksum , #SHA256, #SHA512, #MD5 etc.

#debian #tooloftheday #gnulinux

New #ITtools required for functional #transparency & #democracy ..:

a) #uncensorable and #verifiable #citizen #voting . Secured & enforced by e.g. #blockchain + #distributed #checksums and #P2P #storage .

b) Enforce real #representativeGovernment through citizen-controlled transparency for decision-making in #politics using #multilayer #digitalidentity & #decentralized #socialMedia .

c) #MultiFactor #authentication incl. a mix of #biometrics + manual factors. Controlled by each person, not a #centralAuthority . No need for #passports etc. anymore.

d) Decentralized / #synchronized storage, verifiable (#irrefutable) by anyone, any time, anywhere. (Soon technical & #economical #reality. We're at the brink of a #disruptive storage #technology #breakthrough . Needed for most parts of modern #progress / #society / #business )

#ITstrategy
#perspective
#future
#socialResponsibility

@Laurelai
@aral

https://mastodon.host/media/2xAEp1jLcivmN0OhdoM