Lmst

GitHub - ai-robots-txt/ai.robots.txt: A list of AI agents and robots to block. https://github.com/ai-robots-txt/ai.robots.txt #OpenSource #robots.txt #htaccess #haproxy #Apache #GitHub #block #caddy #nginx #AI

Now I've updated my .htaccess with the "8G firewall" setup. Results to be seen in a month; I'd expect a lot of these to disappear.
https://perishablepress.com/8g-firewall/

#firewall #webmaster #htaccess

I have a recent project to stop (LLM training) crawlers from copyright-thefting my website. I find these bots with either a hidden-link tarpit or by looking for single access events (no css), which I then ban if they come from a cloud server. So far I have learned:

• Amazon AWS, Google Cloud, Microsoft Azure, and Chinese telecom companies are pretty easy to block. These were the early heavy hitters.
• Huawei has little cloud server farms all over the world. I seem to still find about one a day.
• Some mysterious entity rents servers all over the world and crawls by sniping one page at a time. The snipes come in clusters, so all these bots are running the same crawler, with some but not complete inter-communication. Popular cloud companies are OVH Cloud, EGI Hosting, Web2Objects, Host Royale, Digital Ocean, Cloud Innovation, ....

#apache #litespeed #htaccess #crawlers #botfarms

https://codeberg.org/skewray/htaccess

Keeping AI companies from copyright-violating a website for LLM training is difficult, but not impossible. I got pretty far using Apache .htaccess. Still a work in progress, but at this point, most unidentified crawlers are from Brazil and Africa. I've put up the .htaccess code snippets up on Codeberg.

#apache #litespeed #htaccess

https://codeberg.org/skewray/htaccess

Semrush ist eines der bekanntesten SEO-Analyse-Tools auf dem Markt. Es durchsucht Websites regelmäßig mit seinem Bot (SemrushBot), um Daten wie Keywords, Backlinks, Rankings und vieles mehr von deiner Website zu erfassen und zu analysieren. Hier sind 5 effektive, schnell umzusetzende Methoden, wie du Semrush von deiner Website aussperren kannst. 👇

https://teufelswerk.net/semrushbot-blockieren-so-schuetzt-du-jede-website-egal-ob-wordpress-joomla-typo3-oder-statisch/

#SEO #semrush #botblocker #bots #website #websecurity #cybersecurity #wordpress #joomla #typo3 #nginx #robotstxt #htaccess

A comprehensive guide for 301 redirect files
Learning how to use 301 redirects htaccess files are extremely useful, which you can understand further with this article.
Website: https://ondigitals.com/301-redirect-htaccess/
#ondigitals #redirect #301redirect #htaccess

#htaccess는 본 지 수십 년이 됐지만 어떻게 쓰는 건지 모르겠다. 다른 사람들이 만들어 놓은 거 가져다 쓴다.

💪 Proteggi il tuo sito dall'hotlinking con una regole nell'htaccess
Risparmia sulla larghezza di banda del tuo host riducendo il fenomeno dell'hotlink...

👉 https://www.selectallfromdual.com/blog/1521

#️⃣ #hotlink #htaccess

✨ How to Redirect URLs with Query Parameters (in-depth/complete guide) https://htaccessbook.com/redirect-query-strings/ #Apache #htaccess

Photo of my book, .htaccess made easy (print version)

I am struggling to make this work. I have this site structure:
https://example.com/index.html
https://example.com/blog.html (which servers as an index for my blog)
https://example.com/blog/0001.html (a post)
https://example.com/blog/0002.html (another post, etc.)

I'd like that if a visitor types "example.com/blog" then blog.html from the upper folder should be loaded, but I can't find the proper rule(s) to add to an .htaccess file, or where to place it.

I would also like to avoid moving blog.html to blog/index.html, but I could deal with it.

Can somebody provide me with proper instructions? I've tried searching with different terms but I don't quite get the answer to my problem, or I am not understanding what they do.

This is a shared hosting, so modifying httpd.conf is not on the table.

#sysadmin #htaccess #fedihelp

To display websites under #YunoHost, a special #My_Webapp application needs to be installed for each configured domain. Okay, I get that. Of course, there are pitfalls, but that's probably inevitable.

My first attempt with re-form.eu has been a failure so far. But with the knowledge I gained, I managed to get clean-web.org up and running successfully. At least.

This blog, as you know, relies on #ssi, or #shtml pages. This has to be specifically enabled in the #nginx web server, which I did. I don't know if it's in the right place or in the right way. I have the information from the original nginx documentation, and the server didn't report any errors after reloading. Furthermore, there's an #htaccess file in the web directory that specifies index.shtml as the first file to be launched. Nevertheless, only a 404 page appears.

I'd be open to suggestions 🤔 #Linux #VPS

Um Webseiten unter #YunoHost anzeigen zu können, braucht es für jede eingerichtet Domain die Installation einer speziellen #My_Webapp Applikation.Na gut, habe ich begriffen. Dass da natürlich Fallstricke lauern, ist wohl unvermeidlich.

Mein erster Versuch mit re-form.eu ist bislang ein Misserfolg. Mit den Kenntnissen dann aber clean-web.org erfolgreich zum Laufen gebracht. Immerhin.

Das Blog hier setzt bekanntlich auf #ssi, also #shtml-Seiten. Dies muss im #nginx Webserver extra aktiviert werden, was ich tat. Ob an der richtigen Stelle, in der richtigen Weise, weiß ich nicht. Ich habe die Angaben aus der originalen nginx-Dokumentation und der Server meldete keinen Fehler, nachdem er neu geladen wurde. Zudem existiert im Web-Verzeichnis eine #htaccess-Datei, die index.shtml als zuerst startende Datei vorgibt. Trotzdem erscheint nur eine 404-Seite.

Ich wäre für Vorschläge empfänglich 🤔 #Linux #VPS

⚡ How to Block Darknet Market Spam https://perishablepress.com/block-darknet-market-spam/ #WordPress #Apache #htaccess #bots #spam #security

Screenshot showing a bunch of darknet market spam as displayed on the WordPress Comments screen

Simple Redirects with .htaccess

プラグイン「Redirection」によるリダイレクトに失敗

.htaccess に301リダイレクトを記述していたら #WordPress のサイトヘルスで怒られた。

https://ishii00141.stars.ne.jp/20250318-1407-1962/

そこで、プラグイン #Redirection をインストールしてリダイレクトを設定したのだけど、「/@ishii」から「/wp/ishii/」へのリダイレクトに失敗して404エラーページを表示した。「/@hitorinezumi」から「/wp/hitorinezumi/」へも同様である。「/wp/@ishii」から「/wp/ishii/」へや「/wp/@hitorinezumi」から「/wp/hitorinezumi/」へは問題なかった。リダイレクト先をドメイン付きで表示しても404エラーになった。

「リダイレクトを確認」したら、次のように表示された。

リダイレクトによる問題ではない可能性が高いとのこと。「どういうことですか？」のリンク先を見てもよく分からない。

Redirect is returning a 404 – Redirection
https://redirection.me/support/problems/redirect-returning-404/

プラグイン #ActivityPub がリダイレクトを禁止しているのだろうか？そうだとしても404エラーは変である。

プラグイン #Redirection によるリダイレクトを無効にしても変わらないので、有効化に戻しておくが、以前に試した時は問題がなかった。プラグイン #ActivityPub のアップデートでエラーが生じるようになったのだろうか？

ちなみに、こちらのサイトは、次のURLにアクセスできる。

https://ishii00141.stars.ne.jp/@ishii

#Htaccess #ActivityPub #WordPress #エラー #バグ #プラグイン

https://ishii00141.stars.ne.jp/20250318-1530-1968/

.htaccessでリダイレクトしてはいけないの？

もう一つの #WordPress サイトのサイトヘルスに「1件の致命的な問題」が指摘されていた。プラグイン #ActivityPub に関するもの。

1件の致命的な問題
致命的な問題とは、サイトのパフォーマンスやセキュリティに多大な影響を与える可能性がある項目です。最優先で、これらの問題を解決してください。
投稿者 URL にアクセスできません
投稿者 URL の https://hitorinezumi.cloudfree.jp/wp/author/ishii/ は別のページに転送されています。これは通常「Yoast SEO」などの SEO プラグインによって行われます。

次のページに、次のコードで転送しているのだけど…。

https://hitorinezumi.cloudfree.jp/wp/ishii/

Redirect 301 /wp/author/ishii /wp/ishii/

リダイレクト先は、プラグイン #ActivityPub の投稿者URLではないけれど、リダイレクトすることでどのような問題が生じるのだろう？プラグインを使う方法なら問題ないのはどうして？

#Htaccess #ActivityPub #WordPress #プラグイン

https://ishii00141.stars.ne.jp/20250318-1407-1962/

🔥 [#OldPost] - C'è un aggiornamento sul blog ⚠️:
XML-RPC è un protocollo che consente la comunicazione tra computer su una rete, permettendo a un programma di eseguire funzioni su un altro computer. Utilizza XML per formattare le richieste e le risposte e HTTP come mezzo di trasporto. In pratica, un client invia una richiesta a un server specificando quale funzione eseguire e con qual...

👉 https://www.selectallfromdual.com/blog/1472

#️⃣ #altervista #htaccess #wordpress #xmlrpc