Interested in contributing to our project? On November 06 at 17:00 (CEST), we're holding an online meetup. Regardless of whether you're already contributing to datarequests.org, whether you're interested in joining, or you just want to get to know us: Our contributor meetup is supposed to be a space to ask questions on participating, for learning on how to contribute to datarequests.org and Tweasel, and where data protection nerds can meet and network on various issues.

https://www.datarequests.org/verein/event/contributor-meetup-11-2024/

Our open request database at https://data.tweasel.org/ had been practically unusable for a while with pages taking way too long to load.

Turns out we were getting hammered with ridiculous amounts of requests by inconsiderate LLM crawlers. :|
This should now be fixed—we are now blocking their user agents in our reverse proxy. Thanks to @ubernauten for this very helpful blog post which pointed us in the right direction: https://blog.uberspace.de/2024/08/bad-robots/

More details in this issue: https://github.com/tweaselORG/data.tweasel.org/issues/2 ^b

@rugk Indeed, just as @rufposten said in https://chaos.social/@tracktor/113117584454373101, we have been in contact about this already.

We've actually just started working on extending our functionality for websites this month. I'll do my best to publish a new devlog as soon as I find the time—a lot has happened since the last one. ^b

Android: Der Beitrag stellt die Vorbereitung des Testgeräts sowie Werkzeuge (Frida, Magisk) zur Analyse des Datensendeverhaltens von Apps vor. Reinschauen! ✌️ 👇

https://www.kuketz-blog.de/in-den-datenstrom-eintauchen-ein-werkzeugkasten-fuer-analysten-von-android-apps/

#share #android #frida #objection #tweasel #pirogue #tls #ssl #CertificatePinning #mitmproxy #proxy #intercepting #analyse #datenschutz #sicherheit #privacy #security #dsgvo

New data in our open request database!

I've just finished another monkey run on 2,358 #Android apps. That's another 70k requests from April 2024 that can be used for understanding and researching #tracking. ^b

#tweasel #privacy

We have also started doing legal research, looking into relevant complaints, court submissions, decisions, rulings, DPA recommendations, and legal commentary regarding tracking. This is to inform our decisions on how we establish tracker IDs as personal data in our complaints and also to prepare for writing our complaint templates.

As always: Have a look at the blog post for the full details.
https://www.datarequests.org/devlog/tweasel-update-4/

We’re back after the summer with our fourth #tweasel devlog: https://www.datarequests.org/devlog/tweasel-update-4/

A few highlights: We’ve been busy improving the documentation of our TrackHAR adapters to provide better reasoning on why we think properties contain certain data types. We’ve also written a script for debugging our adapters, which allows us to run them against all matching requests in our open request database.

We already announced the database in a previous toot: https://chaos.social/@dev_at_datarequestsORG/111006402859017611 ^b #privacy #tracking

@rufposten @taketwo This is meant at mostly for researchers and journalists. We are building a much easier platform for users to run their own analyses, this is just one of the steps on the way there, we are excited to share.

To publish the data, we are using the phenomenal @datasette. This allows you to run arbitrary SQL queries against the data.

Just a few ideas to get you started:

* endpoints that were contacted by the most apps: https://data.tweasel.org/data?sql=select+count(distinct+regex_replace(%27%40.%2B%3F%24%27%2C+coalesce(initiator%2C+%27%3Cno+app+ID%3E%27)%2C+%27%27))+appCount%2C+count(1)+requestCount%2C+endpointUrl+from+requests%0D%0Awhere+endpointUrl+is+not+null%0D%0Agroup+by+endpointUrl++order+by+appCount+desc+limit+101%3B
* requests by a particular app, e.g. Airbnb on Android: https://data.tweasel.org/data?sql=select+link(dataset%2C+id)%2C+initiator%2C+platform%2C+runType%2C+startTime%2C+method%2C+httpVersion%2C+endpointUrl%2C+scheme%2C+host%2C+port%2C+path%2C+content%2C+headers%2C+cookies+%0D%0Afrom+requests%0D%0Awhere+initiator+like+%3AappId+||+%27%40%25%27%0D%0Alimit+101%3B&appId=com.airbnb.android
* longest requests: https://data.tweasel.org/data?sql=select+length(content)+%2B+length(path)+%2B+length(headers)+%2B+length+(cookies)+as+length%2C+link(dataset%2C+id)%2C+initiator%2C+platform%2C+runType%2C+startTime%2C+method%2C+httpVersion%2C+endpointUrl%2C+scheme%2C+host%2C+port%2C+path%2C+content%2C+headers%2C+cookies+from+requests%0D%0Aorder+by+length+desc%0D%0Alimit+10%3B

We'd love to hear what you discover! We're currently using the data to better document our tracker wiki. More on that in our next devlog.

Our open request database is online: https://data.tweasel.org/ \o/

We regularly run #traffic analyses on thousands of #Android and #iOS apps. As we want to enable as many people as possible to look into the inner workings of trackers, we are publishing our datasets for other researchers, activists, journalists, and anyone else who is interested in understanding #tracking. There are already 250k requests from between January 2021 and July 2023, with more to come in the future. ^b
#tweasel #privacy

We made a few UX improvements: You don’t have to specify the IP address on iOS anymore as we use a usbmuxd proxy. In CLI, you don't need to specify an app ID anymore.
We also fixed a bunch of bugs.

Finally, we have a new docs site and gave a workshop at the @digitalcourage #Aktivcongress. We have also collected some new traffic data.

You can find our new docs here: https://docs.tweasel.org/

And as always, there's much more in the blog post: https://www.datarequests.org/devlog/tweasel-update-3/

We have just published our third #tweasel devlog. You can read it here: https://www.datarequests.org/devlog/tweasel-update-3/

One of the major changes we have made is switching to the @httptoolkit #Frida unpinning script for bypassing certificate pinning on Android. We had run an analysis comparing its performance to the script we used before and found that it works better for our use case. As a bonus, this change allowed us to close two related issues. ^b #privacy #Android #iOS

Finally, we have https://trackers.tweasel.org/, a wiki that explains how TrackHAR recognizes and decodes requests, and provides some sample values of real observed transmissions from our #research data. We hope that this will become a valuable resource for everyone who wants to dig deeper into #tracking.

There's much more in the blog post, so check it out if you're interested. And stay tuned for more updates!

https://www.datarequests.org/devlog/tweasel-update-1/

Another tool we released is TrackHAR (https://github.com/tweaselORG/TrackHAR), a library for detecting #tracking data transmissions from #traffic in #HAR format. It uses custom adapters (and soon, indicator matching) to handle different tracking endpoints and extract the transmitted data.

You can also use TrackHAR through our CLI. Just provide a traffic recording as a HAR file and you'll get a list of the detected tracking data transmissions, including the actual values.

We also released cyanoacrylate (https://github.com/tweaselORG/cyanoacrylate), a toolkit for large-scale automated #traffic #analysis of mobile apps on #Android and #iOS. It uses #mitmproxy to capture the HTTP(S) traffic of apps in HAR format and appstraction to instrument physical devices or emulators. Cyanoacrylate handles the management of certificate authorities and #WireGuard mitmproxy setup automatically.

You can use cyanoacrylate without writing any code through our CLI (https://github.com/tweaselORG/cli).

We published our first update blog post for #tweasel. Our plan is to do these biweekly from now on.

https://www.datarequests.org/devlog/tweasel-update-1/

A lot has happened since our last update in January. We have released a set of tools and library for mobile #tracking analysis.

First up: appstraction (https://github.com/tweaselORG/appstraction), an abstraction layer for instrumenting #Android and #iOS. It allows you to install, uninstall, start, stop apps, manage emulator snapshots, clipboard, proxy, and certificates, etc. ^b

If you're instead interested in #Android apps on #GooglePlay, here's my earlier parse-play library: https://github.com/baltpeter/parse-play

Reverse-engineering those APIs was quite the rodeo as well. I've written an in-depth blog post on that: https://benjamin-altpeter.de/android-top-charts-reverse-engineering/

For an idea on why we need reliable automated access to this data, have a look at our analysis of data safety labels: https://www.datarequests.org/blog/android-data-safety-labels-analysis/

We hope that others will use these libraries as well. Please reach out if you need additional features!

As the library is using internal #API endpoints, I've had to resort to doing a lot of #ReverseEngineering. That was quite the rabbit hole. I've tried to document as much as possible of the arcane knowledge about Apple internals I gained along the way in these issues:

https://github.com/tweaselORG/parse-tunes/issues/1
https://github.com/tweaselORG/parse-tunes/issues/2
https://github.com/tweaselORG/parse-tunes/issues/3
https://github.com/tweaselORG/parse-tunes/issues/6

Our first project is to create reusable and well-documented tools and libraries from the code (https://github.com/baltpeter/thesis-mobile-consent-dialogs) I already wrote for my master's thesis (https://benjamin-altpeter.de/doc/thesis-consent-dialogs.pdf).

And we've just made our first release: parse-tunes is a library for fetching select data on #iOS apps from the #Apple App Store via undocumented internal iTunes APIs.

https://github.com/tweaselORG/parse-tunes

Currently, it can fetch charts of the most popular apps, and meta data (including privacy labels) for individual apps.

Introducing #tweasel. @zner0L and I (@baltpeter) will be working on fighting #tracking in mobile #apps thanks to #NLnet funding (https://nlnet.nl/project/TrackingWeasel/). Our goal is to automate complaints against tracking under the #GDPR and #ePrivacy directive.

We don't have a website yet (this is our behind-the-scenes account, after all), but all code will of course be FOSS (https://github.com/tweaselORG) and we'll report here.

First overview in our #FireShonks talk: https://media.ccc.de/v/fire-shonks-2022-49115-tracking-in-apps-ist-das-legal-eine-bersicht-ber-die-mobile-trackinglandschaft (DE with EN dub). ^b