#InformationTheory

2025-06-22

This is an example of amortized complexity and Strassen's "asymptotic spectra" (nice monograph by Zuiddam & Wigderson: math.ias.edu/~avi/PUBLICATIONS)

Strassen developed this to understand the #complexity of matrix multiplication and #tensors, but it turns out to also show up in a bunch of places:
- #Entropy
- #Quantum information
- Shannon capacity of graphs
- Communication complexity en.wikipedia.org/wiki/Communic
- Circuit complexity (Robere & Zuiddam eccc.weizmann.ac.il/report/202)

#math #probability #ComputationalComplexity #TCS #InformationTheory

2025-06-22

Real coin flips are ~49-51 not 50-50 scientificamerican.com/article

But you can guarantee equal probability with a simple trick! Flip 2x in a row starting with the same side up.

HT->call it H
TH->call it T
HH,TT->try again

(due to von Neumann en.wikipedia.org/wiki/Randomne)

This leads to randomness extractors: from a given random process, what's the biggest uniform distribution you can get efficiently?

Randomness extractors give another interpretation of #entropy:

avg # bits needed to *describe* the outcome
=
# uniformly random bits you can *extract* from the outcome

#math #probability #ComputationalComplexity #TCS #InformationTheory

2025-06-11

Today, the copies of the second edition of the first volume of the textbook series "Media Technology Knowledge" that I edited have finally been delivered:

degruyterbrill.com/document/do

2025-06-02

#PhysicsJournalClub
"Model-free estimation of the Cramér–Rao bound for deep learning microscopy in complex media"
by I. Starshynov et al.

Nat. Photon. (2025)
doi.org/10.1038/s41566-025-016

As everybody who ever tried to orient themselves while immersed in thick fog knows, scattering scrambles information. The question "how much information is still there?" is not particularly interesting as the answer is "essentially all of it", as elastic scattering can't destroy information. A much more interesting question is "how much information can we retrieve?" In order to even try to give an answer we need to be a bit more specific, so the authors placed a small reflective surface behind a scattering layer and asked how much information about its transverse position could be retrieved. This is a well-posed question, and the answer takes the form of a "Cramér–Rao bound" (en.wikipedia.org/wiki/Cram%C3%).
After estimating this upper bound, the authors investigate how well a trained neural network can do at this task, and show that a specifically built convolutional neural network can almost reach the theoretical bound.

[Conflict of interest: Ilya Starshynov (the first author) did his PhD in my group.]

#Physics #InformationTheory #Optics #MachineLearning

BeyondTheEdge - MSCA NetworkBeyondEdge_MSCA
2025-05-23

Join us next Tuesday, 27 May 2025 at 17:00 CET!

🌀 Beyond The Edge Seminar #5
🎙️ Dr Thomas F. Varley (University of Vermont)
🗣️ “Dark information in complex systems”

Discover the synergistic patterns our usual network models overlook, with examples from neuroscience.

🔗 beyondtheedge.network/event/se

2025-05-18

TOSID Update:
Added formal paper presentation
github.com/ha1tch/TOSID/blob/m

As PDF:
github.com/ha1tch/TOSID/blob/m

The TOSID (Taxonomic Ontological Semantic IDentification System) is a comprehensive taxonomic framework designed to uniquely identify and classify any conceivable entity across the universe. The system uses a structured alphanumeric code that embeds hierarchical information about an entity's fundamental nature, origin, scope, and specific identity.

Latest working (non-paper) version:
github.com/ha1tch/TOSID/blob/m

Repo:
github.com/ha1tch/TOSID

#foss #semantic #taxonomy #compsci #computation #informationtheory #ontology #tosid

2025-05-18

New Project!

The TOSID (Taxonomic Ontological Semantic IDentification System) is a comprehensive taxonomic framework designed to uniquely identify and classify any conceivable entity across the universe. The system uses a structured alphanumeric code that embeds hierarchical information about an entity's fundamental nature, origin, scope, and specific identity.

github.com/ha1tch/TOSID/blob/m

#foss #semantic #taxonomy #compsci #computation #informationtheory #ontology

Fredqoop
2025-05-09

Le Web2 est rapide, mais terriblement "bruyant". 🤔 Pourquoi l'info semble-t-elle si polluée, manipulée ou censurée ?

Les plateformes centralisées introduisent du bruit qui dégrade la qualité du signal.

UPlanet ! En utilisant IPFS (stockage distribué), Nostr (protocole ouvert) et une architecture de clés décentralisée, réduit ce bruit systémique.

💪 copylaradio.com/blog/blog-1/po


UPlanet ! En utilisant IPFS (stockage distribué), Nostr (protocole ouvert) et une architecture de clés décentralisée, réduit ce bruit systémique.
2025-05-04

Next step in our NLP timeline is Claude Elwood Shannon, who already laid the foundations for statistical language modeling by recognising the relevance of n-grams to model properties of language and predicting the likelihood of word sequences.

C.E. Shannon ""A Mathematical Theory of Communication" (1948) web.archive.org/web/1998071501

#ise2025 #nlp #lecture #languagemodel #informationtheory #historyofscience @enorouzi @tabea @sourisnumerique @fiz_karlsruhe @fizise

Slide from the Information Service ENgineering lecture 02, Natural Language Processing 01. Title: NLP Timeline.
A black & white portrait picture of Claude Elwood Shannon (1916-2001) is shown on the left side of a timeline marked with "1948". Shannon is depicted in front of an old 1950s "electronic" computer. The text on the right side of the timeline says: Claude Shannon proposed the idea of using n-grams as a means to analyse the statistical properties of language in "A Mathematical Theory of Communication" (1948). While Shannon's primary focus was on communication and information transmission, he recognised the relevance of n-grams in modeling language and predicting the likelihood of word sequences.

BIbliographical reference: 
Shannon, Claude Elwood (July 1948). A Mathematical Theory of Communication, Bell System Technical Journal. 27 (3): 379–423.
2025-04-29

Wow, awesome paper.
One does not often see the intersection of #biology, #statisticalphysics and #informationtheory .

The emergence of eukaryotes as an evolutionary algorithmic phase transition.

pnas.org/doi/10.1073/pnas.2422

2025-04-29

Have you ever heard of molecular communication? 🔬

🔎 This field of research is applied on the Internet of Bio-Nano Things (#IoBNT) and focuses on #InformationTheory & #CommunicationTechnology.

🔹Currently, our participant Pit Hofmann - PhD student at @tudresden - is working with #Zeiss on developing a prototype, more precisely, a molecular communication platform.

👇 Click here to find out more:
softwarecampus.de/en/projekt/m

#SoftwareCampus
#eHealth #NetworkTechnology

2025-03-28

It's done! I have just given print approval for "Medientechnisches Wissen Vol. 1", 2nd edition. The book, originally published on 2017, grew from 306 to 428 pages - mostly because of an additional chapter on .

degruyter.com/document/isbn/97

Tom Le Goc :debian: :bzh:tom_legoc@toot.community
2025-03-28

Je ne sais pas si ça compte en tant que #VendrediLecture, mais très bon livre en/de cours :

"Information Theory, Inference, and Learning Algorithms" David J.C. MacKay

inference.org.uk/mackay/itila/

#IA #InformationTheory

Couverture du livre
2025-03-21

🤔 Is our universe trapped inside a black hole?
@Spacecom

「 each and every black hole in our universe could be the doorway to another "baby universe." These universes would be unobservable to us because they are also behind an event horizon, a one-way light-trapping point of no return from which light cannot escape, meaning information can never travel from the interior of a black hole to an external observer 」

space.com/space-exploration/ja

#blackhole #space #informationtheory #cosmology

2025-03-08

Weekly Update at the Open Journal of Astrophysics – 08/03/2025

Time for the weekly Saturday morning update of papers published at the Open Journal of Astrophysics. Since the last update we have published four new papers, which brings the number in Volume 8 (2025) up to 25 and the total so far published by OJAp up to 260.

In chronological order of publication, the four papers published this week, with their overlays, are as follows. You can click on the images of the overlays to make them larger should you wish to do so.

The first paper to report is “Partition function approach to non-Gaussian likelihoods: information theory and state variables for Bayesian inference” by Rebecca Maria Kuntz, Heinrich von Campe, Tobias Röspel, Maximilian Philipp Herzog, and Björn Malte Schäfer, all from the University of Heidelberg (Germany). It was published on Wednesday March 5th 2025 in the folder Cosmology and NonGalactic Astrophysics and it discusses the relationship between information theory and thermodynamics with applications to Bayesian inference in the context of cosmological data sets.

 

You can read the officially accepted version of this paper on arXiv here.

The second paper of the week  is “The Cosmological Population of Gamma-Ray Bursts from the Disks of Active Galactic Nuclei” by Hoyoung D. Kang & Rosalba Perna (Stony Brook), Davide Lazzati (Oregon State), and Yi-Han Wang (U. Nevada), all based in the USA. It was published on Thursday 6th March 2025 in the folder High-Energy Astrophysical Phenomena. The authors use models for GRB electromagnetic emission to simulate the cosmological occurrence and observational detectability of both long and short GRBs within AGN disks

You can find the officially accepted version of this paper on arXiv here.

The next two papers were published on Friday 7th March 2025.

The distribution of misalignment angles in multipolar planetary nebulae” by Ido Avitan and Noam Soker (Technion, Haifa, Israel) analyzes the statistics of measured misalignment angles in multipolar planetary nebulae implies a random three-dimensional angle distribution limited to <60 degrees. It is in the folder Solar and Stellar Astrophysics.

Here is the overlay:

 

The official published version can be found on the arXiv here.

The last paper to report this week is “The DESI-Lensing Mock Challenge: large-scale cosmological analysis of 3×2-pt statistics” by Chris Blake (Swinburne, Australia) and 43 others; this is a large international collaboration and I apologize for not being able to list all the authors here!

This one is in the folder marked Cosmology and NonGalactic Astrophysics; it presents an end-to-end simulation study designed to test the analysis pipeline for the Dark Energy Spectroscopic Instrument (DESI) Year 1 galaxy redshift dataset combined with weak gravitational lensing from other surveys.

The overlay is here:

 

You can find the “final” version on arXiv here.

That’s all for this week. It’s good to see such an interesting variety of topics. I’ll do another update next Saturday

#3x2ptAnalysis #ActiveGalacticNuclei #arXiv241113625v2 #arXiv241212548v2 #arXiv241217714v2 #arXiv250104549v2 #BayesianInference #Cosmology #CosmologyAndNonGalacticAstrophysics #DESI #DiamondOpenAccess #DiamondOpenAccessPublishing #entropy #GammaRayBursts #HighEnergyAstrophysicalPhenomena #InformationTheory #numericalSimulations #planetaryNebulae #SolarAndStellarAstrophysics #StatisticalMechanics #WeakLensing

Client Info

Server: https://mastodon.social
Version: 2025.04
Repository: https://github.com/cyevgeniy/lmst