Ok you beautiful nerds, can I pay one of you to build a #scraper of all the report cards of sfusd schools?
They all live in drives like this:
https://drive.google.com/drive/mobile/folders/1ibs4BHu8-CQhCTukHX3TG3IdnMcE6PEn?usp=sharing
Also can I see your code/any documentation when you’re done? This isn’t something I know how to do, but I’d like to learn and just don’t have the bandwidth to tinker it all out.
If interested, send me an idea of how much you’d charge.
Một nhà phát triển đang thăm dò nhu cầu về công cụ thu thập dữ liệu LinkedIn (scraper) để lấy thông tin hồ sơ, việc làm, công ty. Họ muốn tìm hiểu các trường hợp sử dụng từ cộng đồng. Giải pháp tương tự cũng có sẵn cho Facebook, Zillow, Google Maps.
#LinkedInScraper #CôngCụ #ThuThậpDữLiệu #SideProject #PhátTriểnPhầnMềm #Developer #DataExtraction #Scraper
Tự động hóa 40% quá trình tìm kiếm việc làm với bộ công cụ scraper và AI đơn giản #TìmKiếmViệc #Automation #AI #Scraper #JobSearch #TựĐộngHóa #CôngNghệ
https://www.reddit.com/r/SideProject/comments/1ozmc2c/automated_40_percent_of_my_job_search/
staying stealthy and reliable so your agents always get what they need without friction
"Staying stealthy"!? Hey #Mozilla, stealthy against whom? The companies trying to track users across the open web? That's the type of tech your charter calls for right!? No?
It can click, scroll, search, and submit just like a human, navigating complex flows with real-time feedback and adaptive behavior. Get full control of
the web, without the complexity.
You're building tech for the parasites to scrape the open web "without friction"? You're mimicing the very users your foundation claims to put first against extractive systems. Do you have no sense of shame or at least irony!? We're looking to you to be the last bastion that can defend the web at scale and you're out here building a Trojan horse in the cover of darkness.
It's not that I have just lost respect I've lost hope, I am not sorry for this very confrontational and scorched earth post. I'm sad and seething!
MWoffliner 1.17 has been released!
https://www.npmjs.com/package/mwoffliner/v/1.17.0
This is a minor new version of our flagship #MediaWiki #scraper but the changelog is still huge with really a lot of bug fixes!
https://github.com/openzim/mwoffliner/releases/tag/v1.17.0
Bad morning,
I just woke up to find out that openAI scraped my entire website, ignoring robots.txt
Building a scraper on top of pg_net in PostgreSQL: I found myself in need of a way to chain requests together (which is what you do when you scrape). It is a simple library built on top of pg_net.
Since people are dunking on #Meta #scraping again I'll share one tidbit: when @jonah and I was investigating some performance issues, I noticed that Meta-ExternalAgent was scraping /auth/sign_up and one specific invite link with different `accept` parameters (which indicates acceptance of rules), however because Mastodon returns 200 (and shows the rules again) on invalid `accept` parameters the #scraper just keeps going...
#Darkdump - L'outil #OSINT ( #Open #Source #Intelligence ) qui fouille le #dark #web pour vous
En gros,vous tapez un mot-clé,et l’outil va #scraper les sites .onion pour en extraire des emails, des métadonnées,des mots-clés,des images,des liens vers les réseaux sociaux
https://korben.info/darkdump-outil-osint-fouille-dark-web.html
#Darkdump - L'outil #OSINT ( #Open #Source #Intelligence ) qui fouille le #dark #web pour vous
En gros,vous tapez un mot-clé,et l’outil va #scraper les sites .onion pour en extraire des emails, des métadonnées,des mots-clés,des images,des liens vers les réseaux sociaux
korben.info/darkdump-out...
Darkdump - L'outil OSINT qui f...
🔍 / #software / #automation / #API
#ZenRows turns hours of tedious lead hunting into a fast, reliable, and automated data workflow that fills your data pipelines.
🐱🔗 https://laravista.altervista.org/CatLink/links/315
#catlink #SoftwareAutomation #SoftwareAutomationAPI #scraper #DataExtraction
Our small team vs millions of bots
https://www.fsf.org/blogs/sysadmin/our-small-team-vs-millions-of-bots
#fsf #freesoftware #ddos #javascrip #anubis #botnet #llm #scraper #crawler #proprietarysoftware #malware
Ich hasse #ki #scraper so sehr dafür, dass ich meine Werke mit hässlichen #Wasserzeichen und XML veröffentlichen muss… Was ein Aufwand. x_X
»Cloudflare Introduces Default Blocking of A.I. Data Scrapers«
Nett, wird aber kaum funktionieren. Weil: Fortgeschrittene Scraper nutzen Browser-Emulation und rotierende IPs, um sich als echte Nutzer auszugeben und technische Erkennung zu umgehen. Da es sich nur um eine serverseitige Maßnahme ohne rechtliche Bindung handelt, können solche Akteure die Sperren leicht und folgenlos ignorieren.
https://www.nytimes.com/2025/07/01/technology/cloudflare-ai-data.html
/kuk
Mastodon untersagt das Training von KIs mit Inhalten der eigenen Plattform.
Neue #Nutzungsbedingungen gegen #KITraining 🚫 #Mastodon verbietet ab 1. Juli explizit das #Scrapen und die Nutzung von User-Daten zum #Training von #KIModellen auf seinem #Hauptserver.
Klarer #Schutz der #Community 🤝 Die neuen Regeln untersagen automatisierte Tools wie #Bots und #Scraper, um Daten abzugreifen – mit Ausnahme von normalen Suchmaschinen und Browsern. (1/2)
