#informationtheory

2025-05-04

Next step in our NLP timeline is Claude Elwood Shannon, who already laid the foundations for statistical language modeling by recognising the relevance of n-grams to model properties of language and predicting the likelihood of word sequences.

C.E. Shannon ""A Mathematical Theory of Communication" (1948) web.archive.org/web/1998071501

#ise2025 #nlp #lecture #languagemodel #informationtheory #historyofscience @enorouzi @tabea @sourisnumerique @fiz_karlsruhe @fizise

Slide from the Information Service ENgineering lecture 02, Natural Language Processing 01. Title: NLP Timeline.
A black & white portrait picture of Claude Elwood Shannon (1916-2001) is shown on the left side of a timeline marked with "1948". Shannon is depicted in front of an old 1950s "electronic" computer. The text on the right side of the timeline says: Claude Shannon proposed the idea of using n-grams as a means to analyse the statistical properties of language in "A Mathematical Theory of Communication" (1948). While Shannon's primary focus was on communication and information transmission, he recognised the relevance of n-grams in modeling language and predicting the likelihood of word sequences.

BIbliographical reference: 
Shannon, Claude Elwood (July 1948). A Mathematical Theory of Communication, Bell System Technical Journal. 27 (3): 379–423.
2025-04-29

Wow, awesome paper.
One does not often see the intersection of #biology, #statisticalphysics and #informationtheory .

The emergence of eukaryotes as an evolutionary algorithmic phase transition.

pnas.org/doi/10.1073/pnas.2422

2025-04-29

Have you ever heard of molecular communication? 🔬

🔎 This field of research is applied on the Internet of Bio-Nano Things (#IoBNT) and focuses on #InformationTheory & #CommunicationTechnology.

🔹Currently, our participant Pit Hofmann - PhD student at @tudresden - is working with #Zeiss on developing a prototype, more precisely, a molecular communication platform.

👇 Click here to find out more:
softwarecampus.de/en/projekt/m

#SoftwareCampus
#eHealth #NetworkTechnology

2025-03-28

It's done! I have just given print approval for "Medientechnisches Wissen Vol. 1", 2nd edition. The book, originally published on 2017, grew from 306 to 428 pages - mostly because of an additional chapter on .

degruyter.com/document/isbn/97

Tom Le Goc :debian: :bzh:tom_legoc@toot.community
2025-03-28

Je ne sais pas si ça compte en tant que #VendrediLecture, mais très bon livre en/de cours :

"Information Theory, Inference, and Learning Algorithms" David J.C. MacKay

inference.org.uk/mackay/itila/

#IA #InformationTheory

Couverture du livre
2025-03-21

🤔 Is our universe trapped inside a black hole?
@Spacecom

「 each and every black hole in our universe could be the doorway to another "baby universe." These universes would be unobservable to us because they are also behind an event horizon, a one-way light-trapping point of no return from which light cannot escape, meaning information can never travel from the interior of a black hole to an external observer 」

space.com/space-exploration/ja

#blackhole #space #informationtheory #cosmology

2025-03-08

Weekly Update at the Open Journal of Astrophysics – 08/03/2025

Time for the weekly Saturday morning update of papers published at the Open Journal of Astrophysics. Since the last update we have published four new papers, which brings the number in Volume 8 (2025) up to 25 and the total so far published by OJAp up to 260.

In chronological order of publication, the four papers published this week, with their overlays, are as follows. You can click on the images of the overlays to make them larger should you wish to do so.

The first paper to report is “Partition function approach to non-Gaussian likelihoods: information theory and state variables for Bayesian inference” by Rebecca Maria Kuntz, Heinrich von Campe, Tobias Röspel, Maximilian Philipp Herzog, and Björn Malte Schäfer, all from the University of Heidelberg (Germany). It was published on Wednesday March 5th 2025 in the folder Cosmology and NonGalactic Astrophysics and it discusses the relationship between information theory and thermodynamics with applications to Bayesian inference in the context of cosmological data sets.

 

You can read the officially accepted version of this paper on arXiv here.

The second paper of the week  is “The Cosmological Population of Gamma-Ray Bursts from the Disks of Active Galactic Nuclei” by Hoyoung D. Kang & Rosalba Perna (Stony Brook), Davide Lazzati (Oregon State), and Yi-Han Wang (U. Nevada), all based in the USA. It was published on Thursday 6th March 2025 in the folder High-Energy Astrophysical Phenomena. The authors use models for GRB electromagnetic emission to simulate the cosmological occurrence and observational detectability of both long and short GRBs within AGN disks

You can find the officially accepted version of this paper on arXiv here.

The next two papers were published on Friday 7th March 2025.

The distribution of misalignment angles in multipolar planetary nebulae” by Ido Avitan and Noam Soker (Technion, Haifa, Israel) analyzes the statistics of measured misalignment angles in multipolar planetary nebulae implies a random three-dimensional angle distribution limited to <60 degrees. It is in the folder Solar and Stellar Astrophysics.

Here is the overlay:

 

The official published version can be found on the arXiv here.

The last paper to report this week is “The DESI-Lensing Mock Challenge: large-scale cosmological analysis of 3×2-pt statistics” by Chris Blake (Swinburne, Australia) and 43 others; this is a large international collaboration and I apologize for not being able to list all the authors here!

This one is in the folder marked Cosmology and NonGalactic Astrophysics; it presents an end-to-end simulation study designed to test the analysis pipeline for the Dark Energy Spectroscopic Instrument (DESI) Year 1 galaxy redshift dataset combined with weak gravitational lensing from other surveys.

The overlay is here:

 

You can find the “final” version on arXiv here.

That’s all for this week. It’s good to see such an interesting variety of topics. I’ll do another update next Saturday

#3x2ptAnalysis #ActiveGalacticNuclei #arXiv241113625v2 #arXiv241212548v2 #arXiv241217714v2 #arXiv250104549v2 #BayesianInference #Cosmology #CosmologyAndNonGalacticAstrophysics #DESI #DiamondOpenAccess #DiamondOpenAccessPublishing #entropy #GammaRayBursts #HighEnergyAstrophysicalPhenomena #InformationTheory #numericalSimulations #planetaryNebulae #SolarAndStellarAstrophysics #StatisticalMechanics #WeakLensing

Rowan Brad QuniQNFO@mstdn.science
2025-03-03

The sets of all math (M), communication (C), and physical matter (P) are subsets of information (I):

M ⊆ I (M is a subset of I)
C ⊆ I (C is a subset of I)
P ⊆ I (P is a subset of I)

Alternatively, we can express this as the union of the sets:
(M ∪ C ∪ P) ⊆ I (The union of M, C, and P is a subset of I)

#InformationalUniverse #IUH #InformationTheory #Epistemology #DataScience
#Mathematics #Proof #Physics #SetTheory #Ontology #Reality #DigitalAge #AI #QuantumInformation #QNFO #StickyNote

This is copied from part of a list of books posted by Rhys Ting of books he read in 2024 that rewired his brain. Looks like a group of very interesting books.

#books #bookrec #TBR #informationTheory

List of book posted by Rhys Ting that rewired his brain in 2024.
2025-01-23

This has given me a reading list.

The 7 Most Influential Papers in Computer Science History terriblesoftware.org/2025/01/2 #InformationTheory, #Papers, #Sql, #Turing

James Kerranethatrobotdev
2025-01-02

(1/2)
Hello, and Happy New Year! I added "Projects" section to www.jameskerrane.com/projects, where I will compile all of my previous software development projects and track new ones that I am working on.

This year, I want to learn how to write an efficient algorithm for how to solve the Factory Light Puzzle, using the fundamentals of . Since I am unfamiliar with information theory and these types of problems, it's time to learn!

Screenshot of a page on www.jameskerrane.com with the title "OneShot Refuge Factory Puzzle Solver", published on January 2, 2025, and tagged "Information Theory". Includes the subheadings "Status", "End Goal", "Preliminary Research", and "Roadmap".
Victoria Stuart 🇨🇦 🏳️‍⚧️persagen
2024-12-13

How little we really know about entropy
quantamagazine.org/what-is-ent
news.ycombinator.com/item?id=4

* 200 years ago French engineer introduced idea quantifying universe’s inexorable slide into decay
* entropy is less a fact about the world than a reflection of our growing ignorance
* embracing that truth is leading to rethinking everything f. rational decision-making to the limits of machines

2024-11-26

To make progress on the reducing emissions and addressing the climate crisis, we must win the information war.

The election of a climate-denying President shows America and progressives are losing this war, but the reasons are complex.

Here is my hypothesis on what’s happened, how we got here, and the path forward:

jjwebster.com/2024/11/22/the-d

#ClimateChange #disinformation #informationtheory #maxwellsdemon #trump #putin #china #greenwashing #reality #uspol #matrix #redpill #blackpill #truth

Client Info

Server: https://mastodon.social
Version: 2025.04
Repository: https://github.com/cyevgeniy/lmst