Page 21 of 23
<<     < >     >>

Blocking bots


The other day I was emailing with Matthew “Starbreaker” Graybosch about his recent post titled “robots.txt: the Nuclear Option”. If you’re a regular reader of this site you know I love this kind of stuff and I especially love nuclear options when it comes to fighting silly tech.

I experimented with blocking everything in the past but this recent exchange made me want to revisit this idea. With perfect timing, Robb Knight posted “Perplexity AI Is Lying about Their User Agent” and that was all the extra motivation I needed to join the fun.

I already had a 403 in place for Mastodon because I don’t want to get a shit ton of traffic coming my way every time someone posts a link of mine but I loved Matthew’s idea of returning a 402.

So I grabbed 180 or so entries from the Dark Visitors’s agents list and set up an NGINX redirect based on those UA. Gonna be interesting to see if this has any effect on the server so I’ll write a follow-up.

I tried to leave out all the RSS fetchers because I love RSS, RSS is great and if you’re using RSS in 2024 you’re an awesome person BUT I might have inadvertently broken some RSS feed out there with this move. If you notice something not working properly let me know and I’ll fix it.

Thank you for keeping RSS alive. You're awesome.

Email me :: Sign my guestbook :: Support for 1$/month :: See my awesome supporters :: Subscribe to People and Blogs

Manu's Feed

16 Jun 2024 at 20:15

The Breakers. Beautiful walk along the cliffs and in the mansion. The Vanderbilts really had a stunning amount of wealth.

Manton Reece

16 Jun 2024 at 18:58

I watched The Matrix, The Devil’s advocate and Fight Club and loved all three. So I asked ChatGPT for five suggestions of movies I might watch next and it came up with some interesting ideas.

Dave's famous linkblog

16 Jun 2024 at 18:26

The New Alt Media

Anil Dash:

You might have noticed, it’s not a super fun time to be in the publishing industry, especially if you’re trying to do journalism. The years-long drumbeat of bad news issuing from nearly every newsroom has left people understandably despairing about what’s going to happen next.

I’m not a Pollyanna; I don’t think everything is just magically going to turn out okay. But for the first time in decades, I’m seeing a movement that gives me hope. Call it the New Alt Media.

Robin Rendle

16 Jun 2024 at 17:53
 Adam Kotsko:

Simply put, the way we were taught the market for consumer goods is supposed to work does not seem to hold anymore. The market was supposed to incentivize businesses to offer attractive products, at competitive prices, in a convenient format, and then customers were supposed to respond to those positive signals by rewarding them with their business. Now businesses increasingly take actively customer-hostile actions — locking up products, replacing paper menus with cumbersome QR codes, and of course chronically understaffing everything, which is the root of all of these issues — and insulate themselves from any feedback.


16 Jun 2024 at 17:39

That which takes wing inside us must come to perch, but that which takes flight in fog and storm grows lost.


Sometime in 1993, as I walked along a street in my hometown, Carndonagh, County Donegal, Ireland, a car pulled up alongside me, triggering sudden dread. The window came down, and I was met by the dark, inquisitory eyes of my father.

“Why aren’t you at Mass?” he asked.

I see myself, fierce and lean in a Slayer T-shirt, bristling with the rage of the nihilist. I longed to escape the claustrophobic small town and the towering shadow of the Catholic Church. For once I was impelled to tell the truth.
“Look,” I said, “I have no faith. I don’t believe in God anymore and can’t go on with the pretense.”

I was met with an imprisoning silence. But what my father said next astonished me. “OK,” he replied. “Just don’t tell your mother.”

But that young atheist soon recognized his error. Where there is human being, there is human spirit. The feeling of aliveness. The staggering complexity of personhood. The fundamental dignity that each person seeks in a cosmos that cannot know them. And where there is human spirit, there is the pursuit of meaning. If you live in a post-faith world, as many of us do, the question of our intrinsic meaning must be confronted. How are we to define our suffering? What might give our lives significance within an unresponsive universe? To begin this conversation, one must truly encounter the self. […]

The essential self is calling always for our attention, but its voice is stifled by the slam and tumult of modern life. Its voice cannot be heard amid the babel, and it is silenced entirely before the infinite scroll of the smartphone. I have been meditating for one-third of my life, and this essential self seems to me an aspect of mind that is somehow higher, wheeling soundlessly in a private sky. You must stop and look up in order to find it, although in times of crisis it has been known to swoop down and hoist you off your feet with its talons. […]

Today, life lived on the hamster wheel of distraction has created an absurdity within the grand absurdity of existence. Many people live with partial minds not even conscious of the problem of meaning. We are no longer alienated from the world, but alienated from ourselves. We should beware a culture that has exchanged meaning for information. When conversation with the essential self grows silent, pathology is invited in. We slouch about at a loss for something we cannot quite explain. A malaise sets in that is despair without the knowledge of despair. Some unseen, unaccountable pain must be assuaged and we grow consumed by anger and cast about for blame. The irrational erupts from within and seeks a target in society. The shadow of the irrational is now everywhere about us. […]

That which takes wing inside us must come to perch, but that which takes flight in fog and storm grows lost. Deep beneath the vast economic and political failings of our age there lies a spiritual crisis, a tectonic shift beginning to quake and tear at the bedrock of our ethical societies in the West. The modern age has created a religious problem that can no longer be answered by religion, nor can it be addressed by the current faith in techno-science. We live in an age that fears silence and does not contemplate the true cost of this fear.

Paul Lynch, from “When We Fear Silence, We Abandon the Self. The constant distractions of modern life have become an excuse to avoid the search for meaning.” (NY Times, June 12, 2024)

Notes: Paul Lynch is the author of five novels. His latest, “Prophet Song,” won the 2023 Booker Prize. See excerpts from the Prophet Song here.

Live & Learn

16 Jun 2024 at 17:27



One of the things I hate, yes, I really hate it, is cigarette smoke. I get angry when I smell the smoke of the neighbors who are smoking directly in front of the entrance door of our apartment build, while we are trying to let fresh air in. But situations like smelling smoke at train stations or bus stops make me feel really uncomfortable as well.

I immediately think about all the toxic chemicals I am forced to breathe. I think about all the damage those might cause. And I think about cancer. And what depresses me the most is that some smokers get so old, while people who never smoke and live quite healthy sometimes die so early.

I don’t understand why there are no stricter rules for smokers, to better protect non-smokers. Like not smoking on bus stations. A ban on smoking in front of buildings, near child playgrounds. Or, generally, a ban on smoking when other non-smokers are nearby.

According to the WHO, 8 million people die due to smoking every year, “including an estimated 1.3 million non-smokers who are exposed to second-hand smoke”.

Kurzgesagt made a good video about smoking. Why it’s so rewarding at first, what it does to the body, and why it’s a terrible idea to smoke.

Thanks for listening to my little rant. I had to let that out.

P.S.: Just before pressing the Publish button, I found this post from 2018. My opinion hasn’t changed a bit. Health is such a luxury good, don’t waste it!

Interactions & Comments

Jan-Lukas Else

16 Jun 2024 at 17:09
<<     < >     >>

Refresh complete

(222) All feeds

Last 24 hours
Download OPML
A Very Good Blog by Keenan
A Working Library
Alastair Johnston
Andy Sylvester's Web
Anna Havron
annie mueller
Annie Mueller
Apple Annie's Weblog
Artcasting test feed
Articles – Dan Q
Austin Kleon
* posts
Bix Dot Blog
Brandon's Journal
Chris Coyier
Chris Lovie-Tyler
Chris McLeod's blog
CJ Chilvers
CJ Eller
Colin Devroe
Colin Walker – Daily Feed
Content on
Dave's famous linkblog
Dino's Journal 📖
E L S U A ~ A blog by Luis Suarez
Flashing Palely in the Margins
Floating Flinders
For You
Frank Meeuwsen
Hello! on Alan Ralph
Human Stuff from Lisa Olivera
Into the Book
Jake LaCaze
James Van Dyne
Jan-Lukas Else
Jim Nielsen's Blog
Jo's Blog
Kev Quirk
lili's musings
Live & Learn
Lucy Bellwood
Maggie Appleton
Manton Reece
Manu's Feed
Meadow 🌱
Minutes to Midnight RSS feed
Nicky's Blog
Notes – Dan Q
On my Om
One Man & His Blog
Own Your Web
Paul's Dev Notes
reverie v. reality
Robin Rendle
Robin Rendle
Sara Joy
Scripting News
Scripting News for email
Sentiers – Blog
Simon Collison | Articles & Stream
the dream machine
The Marginalian
Tracy Durnell
Winnie Lim
yours, tiramisu
Žan Černe's Blog

About Reader

Reader is a public/private RSS & Atom feed reader.

The page is publicly available but all admin and post actions are gated behind login checks. Anyone is welcome to come and have a look at what feeds are listed — the posts visible will be everything within the last week and be unaffected by my read/unread status.

Reader currently updates every six hours.



Colin Walker Colin Walker