Lmst

New blog post about one way to keep your large-file storage size down when using git.

https://dbat.codeberg.page/posts/git-large-file-technique.html

I hope it's not bad advice. Please let me know of bugs etc.

Also, if anyone has other techniques, will be happy to try them and add them to the post.

🦇

#Codeberg #git #git-lfs #GameDev #LargeFiles

I opened an "issue" on Codeberg to try get some discussion about my recent git-lfs experiences and the ideas floating in my head for a better way to handle large binary files while using git.

I'd appreciate any feedback etc. It's here:
https://codeberg.org/Codeberg/Community/issues/1910

I'm a git-nitwit, so factor that in! 🤭
🦇

#git #gamedev #LargeFiles

Rough idea:

touch big.blend
git rm --cached big.blend
git commit
git push
git add big.blend
git commit
git push

...but scripted somehow, so that only happens when big.blend has changed.

3/

#git #gamedev #LargeFiles

LFS does not do it. Git-annex does not seem to do it.

All the `assume-unchanged` and `skip-worktree` do not really do it.

Is there some way you can think of?

2/

#git #gamedev #LargeFiles

It is just *bewildering* to me that git does not have a way to ignore changes to a tracked file.

Game dev use-case:
repo/
codefile001
codefile002
binary_files/
huge001.blend
huge002.png
huge003.wav

I want to add/commit/push as usual, because I *want* the all those files online (backups) and for others to fetch.

I do NOT want those binary files to duplicate. When I push a new version, then *replace* binary files that changed!

😕 🦇
1/

I recently needed to read in a very large file in Python (around 150 million lines), it would read in 50 lines, do some work and then carry on. But it had a habit of crashing out (normally when I forgot and closed my laptop). So I created a stateful reader, that way when I started the execution again it would resume from the last saved checkpoint.

#Python #LargeFiles #Sqlite

https://dazfuller.uk/posts/python-stateful-reader/