SSDs, but specially RAM prices are skyrocketing: https://arstechnica.com/gadgets/2025/11/spiking-memory-prices-mean-that-it-is-once-again-a-horrible-time-to-build-a-pc

These are excellent times for using compression to reduce your storage pressure.

https://www.blosc.org/posts/roofline-analysis-blosc2/

#Blosc2 #Compression #HPC

Interact with your vasts remote datasets right in your phone! 📱

I've built a demo Jupyter notebook that connects to a Cat2Cloud server from an Android phone and slices into an 8 TB dataset, downloading a 1 MB chunk in under 100 milliseconds. ⚡

The 8 TB dataset is from the Gaia DR3 catalogue. As it turns out, there are ~1000 stars in a cube of 100 light-years in our vicinity; the space is mostly empty.🔭 🌌

Try this out by visiting: https://cat2.cloud/demo/roots/@public/large/slice-gaia-3d.ipynb

#Cat2Cloud #Blosc2 #BigData

Our @EuroSciPy 2025 tutorial on modern Blosc2 features is now online! 🚀

For the first time, we present our holistic view on how compression can revolutionize data handling, sharing, and computing.

Learn how Blosc2 boosts performance for large datasets, how to serve data online with Caterva2, and compute directly in the cloud.

Watch now! 👉 https://www.youtube.com/watch?v=BdpTtzX2cuk

#Blosc2 #EuroSciPy #Python #DataScience #BigData #HPC #OpenSource

💊IronPill 2💊
In the second of our series of short videos ("ironPills") showcasing ironArray's work, we see how Blosc2 can be used to power heavy-duty linear algebra (100GB!) workflows
⚡1.5-2x faster than PyTorch + h5py!
🧱 automated chunking optimised for your machine's cache hierarchy
🐍 simple one-line syntax 𝚋𝚕𝚘𝚜𝚌𝟸.𝚖𝚊𝚝𝚖𝚞𝚕(𝙰, 𝙱, 𝚞𝚛𝚕𝚙𝚊𝚝𝚑='𝚘𝚞𝚝.𝚋𝟸𝚗𝚍')

See blog here: https://ironarray.io/blog/la-blosc

#Blosc2
#Data
#SignalProcessing
#LinearAlgebra

💊 IronPill 1💊
In the first of a series of short videos ("ironPills") showcasing ironArray's work, we see how Blosc2 can be used to calculate Fourier approximations:
⚡5x faster than NumPy
👣 fraction of the memory footprint
🐍 pythonic one-line syntax 𝚜𝚞𝚖(𝚊 * 𝚌𝚘𝚜(𝚝) + 𝚋 * 𝚜𝚒𝚗(𝚝), 𝚊𝚡𝚒𝚜=𝟷)

See full notebook here: https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/ironpill1.ipynb

(inspired by this blog post: https://towardsdatascience.com/numexpr-the-faster-than-numpy-library-that-no-ones-heard-of/)

#Blosc2
#Data
#SignalProcessing

🗣️ Announcing Python-Blosc2 3.8.0 🚀

A step closer to compliance with the array-api standard: data-apis.org/array-api!
This is an effort across all array-based libraries so that your code works (e.g. for both blosc2 and NumPy) by simply changing the import statement below!

Highlights:
✅ C-Blosc2 updated to latest 2.21.2
✅ Incorporate isnan, isfinite, isinf
✅ Better indexing coverage
✅ linspace and arange functions more numerically stable
✅ Improved array-api compliance

#arrayapi #Blosc2

Struggling to get performant code from LLMs? 🤔 They can't do the empirical, target-specific optimization needed for modern CPUs.

We can help! 💪 We've spent countless hours profiling and micro-benchmarking Blosc2 for you. Use our C/Python implementations as building blocks for your high-performance apps.

Ready to dive in? 🚀
🔹 EuroSciPy Talk: https://www.blosc.org/docs/2025-EuroSciPy-Blosc2.pdf
🔹 Tutorial: https://github.com/Blosc/EuroSciPy2025-CCC-Tutorial

#HPC #Python #Blosc2 #Optimization

📢 Great to see the community building powerful tools on Blosc2! 🚀

Check out compress-image: a new C++/Python library for working with compressed images directly in memory.

It allows you to keep lots of images in RAM while minimizing I/O and memory footprint.

Kudos to Emil Dohne for this fantastic work!

Project here: https://github.com/EmilDohne/compressed-image

#Blosc2 #FOSS #ImageProcessing #DataCompression #Cpp #Python

🚀 We are thrilled to announce *TreeStore*, a new class in Python-Blosc2! It lets you give your datasets a hierarchical structure while keeping the speed and efficiency of `NDArray` instances. ⚡️

📝 We've blogged all about it here:
https://www.blosc.org/posts/new-treestore-blosc2/

It's in beta, and you can start using it now in the latest Python-Blosc2 v3.7.2. Enjoy!

#Python #Blosc2 #TreeStore #DataScience #OpenSource

🚀 Caterva2 2025.8.7 is out!

This release features a major refactoring for a simpler, more robust system. Best of all, client APIs are unaffected. ✅

🌟 Highlights:

The new cat2agent 🤖: A CLI client to watch a directory and auto-sync it with a Caterva2 server.
New stack and concat commands in the web UI for easier data manipulation.
Full release notes: https://github.com/ironArray/Caterva2/releases

Learn more about Caterva2: https://ironarray.io/caterva2

#Caterva2 #Blosc2 #HDF5 #DataStorage #OpenSource #DataScience

🗣️ Announcing Python-Blosc2 3.6.1

!Unlock new levels of data manipulation with Blosc2! 🚀

We've introduced a major improvement: powerful fancy indexing and orthogonal indexing for Blosc2 arrays.

We've tamed the complexity of fancy indexing to make it intuitive, efficient, and consistent with NumPy's behavior. 💪

Read all about it on our blog! 📝 https://www.blosc.org/posts/blosc2-fancy-indexing/

Compress Better, Compute Bigger!

#Blosc2 #Python #DataScience #BigData #NumPy #Performance #HPC

Thanks to the advanced double partitioning techniques in #Blosc2, our #Caterva2 package can serve small slices of big datasets (3.8 GB) through internet in less than the blink of an eye.

See how you can do that with the help of #JupyterLite in #Cat2Cloud using two different techniques:

1) Plain Python-Blosc2 library for quick and dirty access
2) Caterva2 Python client for a more heavy-duty and flexible operation

Try it out! 👉 https://cat2.cloud/demo/roots/@public/examples/large-dataset-indexing.ipynb?roots=%40public

#BigData #RemoteAccess #Performance

#Blosc2 now runs directly in your browser! Leveraging the power of #WASM, #Pyodide, and #JupyterLite, you can harness efficient, adaptable compression through the web's universal interface. Experience the future of large-scale data processing without leaving your browser window.

Compress Better, Compute Bigger, Share Faster

#WebAssembly #DataCompression #WebDevelopment #DataScience

📢 We are pleased to announce the integration of a new stack feature in #Blosc2 🚀, which allows for stacking large arrays along a new axis.

Performance benchmarks show that while aligned chunks yield the best results, #Blosc2 with unaligned chunks can still outperform #NumPy—a welcome discovery! 🎉

Many thanks to Luke Shaw for his excellent work on this new functionality. 🙏

We've updated our recent blog post:
Check it out! 🔗 https://www.blosc.org/posts/blosc2-new-concatenate/#stacking-arrays

#Python #DataScience #Performance #OpenSource

📢 Blosc2 just launched a super-efficient array concatenation feature! 🎉

Combine massive arrays quickly and with minimal memory. If your array chunks are aligned, it's even faster – no need to decompress first! Perfect for big data tasks.

Check out our blog post: https://www.blosc.org/posts/blosc2-new-concatenate/

Compress Better, Compute Bigger

#DataScience #Blosc2 #DataStorage #Performance #MachineLearning

🚀 C-Blosc2 2.18.0 is out now!

✨ What's new:

* Introducing b2nd_concatenate() - now you can easily join b2nd arrays together!

* Fixed mmap files to flush modified pages only in write mode (thanks Jan Sellner!)

Get the full details: https://github.com/Blosc/c-blosc2/blob/main/RELEASE_NOTES.md

#Blosc2 #DataCompression #OpenSource #CLibrary #Performance

🚀 Excited to share more about Caterva2, your ultimate gateway to Blosc2/HDF5 repositories! 🚀

Caterva2 is designed to redefine how you interact with large datasets.

Want to see it in action? 🤔 We've just released a new introductory video showcasing Caterva2's main functionalities! 🎬

👉 https://ironarray.io/caterva2

#Caterva2 #Blosc2 #HDF5 #BigData #DataManagement #FreeSoftware #Python #DataScience #Tech

#Python-Blosc2 is hitting 1 million weekly downloads on PyPI! 🎉 https://pypacktrends.com/?packages=blosc2&packages=blosc&time_range=2years

Users are rapidly adopting #Blosc2, which now accounts for over 95% of downloads compared to Blosc1. 📈 This success is thanks to our amazing users and community contributors. 🙏 We're dedicated to making Python-Blosc2 even better. 🚀

Our motto: Compress Better, Compute Bigger! 💪

#Milestone #CommunitySupport #DataCompression

Now it's @FrancescAlted to introduce the #Blosc2 #compression algorithm to reduce #HDF5 file size.

💡 Did you know you can supercharge your #HDF5 datasets with #Blosc2? 🚀

Leverage hdf5plugin (https://hdf5plugin.readthedocs.io) to integrate Blosc2 as a filter within HDF5. Create, write, and read data using popular Python wrappers like h5py or PyTables, while achieving excellent performance! 💨

More speed?

* h5py users: b2h5py offers optimized reads for n-dim slices.
* PyTables users: Optimized support is already built-in.

Learn more: https://www.blosc.org/posts/pytables-b2nd-slicing/

Compress Better, Compute Bigger :-)