RE: https://wisskomm.social/@ioer/115899330915763542

I really took a deep dive into #datashader with this map: Locals & Tourists in Germany, as derived from 67 Million Geo-Social Media Posts (2007-2022) in Germany. The data includes public shared posts from Instagram, Flickr, Twitter and iNaturalist.

I always wanted to create such a map, following the footsteps of Eric Fisher's Locals & Tourists dataset from 2011 [1].

I shared the code for producing this map here [2]. The repository is available here [3]. This includes some neat methods for various #geospatial processing tasks in #Python, such as exporting a datashader map to a #GeoTiff [4] with the help of #Xarray and #Rasterio.

Finally, all of this was created in a privacy-preserving way using #HyperLogLog, which allowed me to share the code and abstracted data publicly for full reproducibility and transparency. [6] #FAIR

Below you'll find the link to the (quite succinct) publication in Natur und Landschaft in Karten (#NuL).

[1]: https://www.flickr.com/photos/walkingsf/albums/72157624209158632
[2]: https://code.ad.ioer.info/wip/digital_traces_map/html/03_visualization.html
[3]: https://gitlab.hrz.tu-chemnitz.de/ad/digital_traces_map/
[4]: https://gitlab.hrz.tu-chemnitz.de/s7398234--tu-dresden.de/base_modules/-/blob/main/raster.py?ref_type=heads#L78
[5]: https://www.nul-online.de/article-7301410-1111/landschaft-und-natur-in-karten-.html
[6]: https://doi.org/10.71830/VDMUWW

🚀 PolSARpro update released!

This new version includes:

• a new tutorial showing how to run PolSARpro on BIOMASS data directly in the cloud via the ESA MAAP platform (no local download needed)
• access to an ALOS-1 sample dataset to get started with the tutorials
• bug fixes and stability improvements

📦 Available on conda-forge
🔗 Code: https://github.com/satim-co/PolSARpro
📘 Docs: https://polsarpro.readthedocs.io

#PolSAR #SAR #RemoteSensing #EarthObservation #OpenSource #Python #ESA #MAAP #BIOMASS #Dask #Xarray

I added to "band_data" a '_CRS' attribute as dict with 'url', 'wkt' & 'projjson' infos (EPSG 3857), saved the whole thing as :zarr: file and hoped :gdal: can open it in :qgis:

However :gdal: seems to have trouble finding the coordinate reference and assumes the origin is 0,0 (Extent 0.0, -8192.0 : 4096.0, 0.0).

Now, my question: where do I store the "_CRS" dict correctly? Do I also need to add a GeoTransform infos? If yes, where?

2/2 #xarray #gdal #zarr

Sorry, that I misuse this platform for my silly question:

I have a :xarray: dataset
Dimensions: (time: 1, band: 4, y: 8192, x: 4096)
Coordinates:
* time datetime64[ns] 8B 2023-06-15
* band <U5 80B 'Red' 'Green' 'Blue' 'Alpha'
* y float32 33kB 7.064e+06 7.064e+06
* x float32 16kB 1.194e+06 1.194e+06
Data variables:
band_data (time, band, y, x) uint8 134MB dask.array<chunksize=(1, 4, 1024, 1024), meta=np.ndarray>
spatial_ref int64 8B ...

1/2 #xarray #gdal #zarr

Does anybody have experience with #xarray for #numpy labeled arrays?

My use case is rather simple with a 2d ndarray and I'm mostly after easier development for myself and contributors if it's possible to switch from ndarray[:, 0] and ndarray[:, 1] to ndarray[fieldname]

#python

@jeremy beautiful, thanks. 🎉

#zarr
#xarray

@jeremy useful to add #zarr #xarray emojis? 🙏 😀

🚨 New version of xarray-grass 🚨

I'm glad to announce that I've release version 0.4.0 of xarray-grass! It comes with many improvements: 🚀 Lazy loading of GRASS space-time datasets, 📅 Better management of time dimensions, including support of units when writing relative-time series to GRASS
🗘 Automatic transposition of arrays when writing to GRASS.

Try it today !

https://pypi.org/project/xarray-grass/

@grassgis #xarray

if you wonder about coordinates in arrays and netcdf and Zarr and #xarray check out this awesome video and gallery, to see where xarray is going

https://www.youtube.com/watch?v=I-NHCuLhRjY

https://xarray-indexes.readthedocs.io/

excellent presentation by Deepak Cherian of #earthmover

made some huge af virtual #Zarr today, each var is 12Tb uncompressed, all referenced to public available netcdf servers but not using netcdf lib at all for read

(Reliant on pretty recent #GDAL or #xarray if you want to explore)

https://github.com/mdsumner/virtualized

been plodding towards this a long time, and finally closed the circle

More to come 👌

Anyone would know a workaround about the lack of native support for complex valued arrays in #xarray netcdf writer?

#datascience #python

I am excited to announce xarray-grass, a new free software Python library designed to bridge two open source data science heavy weights: @grassgis and #xarray (https://xarray.dev/).

Although xarray-grass is in its nascent phase, I encourage you to check out the repository on GitHub (https://github.com/lrntct/xarray-grass) and experiment with it. Your insights and contributions will play a significant role in the project's future.

#geo #RemoteSensing #earthobservation people! Has anyone got an example of using the #Copernicus #DataSpace -> https://documentation.dataspace.copernicus.eu/ to go from searching using #pystac to getting a working #xarray dataset for #Sentinel2 reflectances? #python #geopython

🚀 EO-Tools 2025.4.0 is out!

🌐 New in this release:
- Support for Sentinel-1C products!
- H-Alpha dual-pol decomposition from SLCs 🛰️

📘 Dive in: https://eo-tools.readthedocs.io
💻 Code: https://github.com/odhondt/eo_tools

#EO_tools #RemoteSensing #Sentinel1 #SAR #EarthObservation #OpenSource #Python #Dask #Xarray

I am really looking forward to a time when scientific data analysis is less of a constant fuckaround and fight with technical bullshit. I'd *really* like

- #netCDF natively supporting complex numbers
- #Python #xarray and #pandas to natively support physical units (#pint is great on its own but the integrations leave a LOT to be desired)
- #Jupyter notebooks to suck less (crashes, glitches, widget plots not saved statically, an effing BUILTIN formatter, etc.)
- proper data pipeline systems
...

𝗚𝗲𝗼𝘀𝗽𝗮𝘁𝗶𝗮𝗹 𝗣𝘆𝘁𝗵𝗼𝗻 𝗧𝘂𝘁𝗼𝗿𝗶𝗮𝗹𝘀
SpatialThoughts provides tutorials which cover a broad range of geospatial topics and technologies, e.g., #GeoPandas, #XArray, #dask, and more. Each technology is described in a notebook with step-by-step explanation. Check it out.
https://www.geopythontutorials.com

GSPy - A New Toolbox And Data Standard For Geophysical Datasets
--
https://doi.org/10.3389/feart.2022.907614 <-- shared paper
--
https://doi.org/10.5066/P9XNQVGQ | https://code.usgs.gov/g3sc/gspy <-- shared code repository
--
[an older paper, but code is in active and ongoing development/evolution]
#GIS #spatial #mapping #geophysics #geophysical #NetCDF #datatypes #code #opensource #library #dataformats #standardisation #standardization #openstandard #portable #metadata #Python #package #GSPy #methods #workflows #xarray #CRS #opendata #architecture #toolbox

Justus made a great intro on using #DGGS through #xarray #xdggs at the #Pangeo showcase talk. Xdggs is now in a stage where you can use it fairly robustly with #HEALPIX and #H3. Other integrations like for #DGGRID are developed as separate plugins.

https://youtube.com/watch?v=bAMGFKsxsjs&si=hnfpk04VMn-rggEV

... while I find ChatGPT increasingly useful for technical things like telling me how to manipulate #xarray datasets in #python, or how to add a docstring to my python routine.

I am moving all my computing libraries to #xarray, no regrets. It is a natural way to manipulate datasets of rectangular arrays, with named coordinates and dimensions: https://xarray.dev/
There are several possible backends, including #dask which allows lazy data loading.
I had the pleasure of meeting some of the devs last week, who showed me a preview of the upcoming `DataTree` structure which is going to make this library even more versatile!

#Python #numpy #ScientificComputing