Robots.txt

 A few weeks ago, I saw a flurry of conversation about how you can now disallow OpenAI from indexing your personal website using robots.txt:

User-agent: GPTBot
Disallow: /

That felt a bit “ex post facto“ as they say. Or, as Jeremy put it, “Now that the horse has bolted—and ransacked the web—you can shut the barn door.”

But folks seemed to be going ahead and doing it anyway and I thought to myself, “Yeah, I should probably do that too…” (especially given how “fucking rude” AI is in not citing its sources).

But I never got around to it.

Tangentially, Manuel asked: what if you updated your robots.txt and blocked all bots? What would happen? Well, he did it and after a week he followed up. His conclusion?

the vast majority of automated tools out there just don't give a fuck about what you put in your robots.txt

That’s when I realized why I hadn’t yet added any rules to my robots.txt: I have zero faith in it.

Perhaps that faith is not totally based in reality, but this is what I imagine a robots.txt file doing for my website:

Photograph of a “DO NOT ENTER” sign on a rock cliff and people have passed it and are standing out on the edge of the cliff.

Photograph at a beach with a sign that says “POISONOUS SPECIES DO NOT STEP INTO WATER” and people are all standing in the surf.

Photograph of a sign painted on the ground that says “NO DOGS ALLOWED” and there’s an adorable puppy sitting on the “NO” looking at the camera.



Jim Nielsen’s Blog

27 Sep 2023 at 20:00

Software is What We Learned Along the Way

 Trent absolutely nails it:

[the why is] where I provide the most value as a designer. I am not merely the picker of fonts, the dropper of shadows, the executor of deliverables. My greatest value as a designer lies in orchestrating the process and gathering insights — applying the whys to the create the optimal what.

Those insights don’t cease to be valuable after a product ships, but instead become more valuable as historical resources that contextualize design — provenance. This accounting of how we arrived at our solution should be retained and organized, accessible to others when needed to prevent repeat work and inform confident decision making.

This is a great articulation of the value and purpose of “design”. As Baldur said, aesthetics (color, type, etc.) are part of what design does but it is not what you hire a designer to do.

Great designers help you get you to the what, but they start by defining the why and its rationale. Unless well documented, archived, and made easily accessible to teams, these rationales are often lost over time.

New team members ask, “Why does the product look and function the way it does?” The only answer is often, “Because that’s the way it was when I got here.”

This is illustrated perfectly in this line, again from Baldur:

Software is the insights of the development team made manifest.

A software product is the manifestation of a living body of insights from the people who build it.

What’s in prod is only half the equation. Why it’s in prod (in the state it’s in) is often only understood in the collective minds of a team of people.

If you can’t capture that to contextualize historical decisions and then improve current and future decision making, you’re falling prey to the classic blunder of history: those who don’t learn from it are doomed to repeat it.

Anyway, excuse me while I look into Luro a bit more…



Jim Nielsen’s Blog

26 Sep 2023 at 20:00

Build Great Software By Repeatedly Encountering It

 Robin in “Vibe driven development” (which I took notes):

the only way to build a great product is to use it every day, to stare at it, to hold it in your hands to feel its lumps. The data and customers will lie to you but the product never will.

Oof. That lands with me.

As a professional, it’s easy to become mired in the boundaries we draw around specialization, discipline, and craft.

For example, “design” is too broad. We need UI design, UX design, service design, visual design, content design, and more. Each comes with its specialities, best practices, rules, and guidelines that demand a role in making software.

I'm not discounting any of that.

But, to Robin's point, there’s something in the simplicity of: you just gotta use the thing, every day, over and over and over.

My brain thinks of it like a rock you want to make smooth. You can discuss how to best smooth its edges, debate what tools will do it best, and so forth.

Or you can be like a river of water that washes over the rock incessantly day after day after day, smoothing over every last rough edge by encountering it over and over and over again.

You can get pretty damn far by incessantly encountering a piece of software over and over, fixing everything you find along the way that doesn’t feel right.



Jim Nielsen’s Blog

24 Sep 2023 at 20:00



Refresh complete

ReloadX
Home
(236) All feeds

Last 24 hours
Download OPML
A Working Library
Žan Černe's Blog
Alan Ralph
Alastair Johnston
*
Andy Sylvester's Web
Anna Havron
annie mueller
*
Annie Mueller
*
Austin Kleon
bgfay
*
Bix Dot Blog
*
Chris Coyier
Chris Lovie-Tyler
CJ Chilvers
CJ Eller
Colin Devroe
*
Colin Walker – Daily Feed
*
Colin Walker – Live Feed
Content on Kwon.nyc
*
Dave's famous linkblog
*
daverupert.com
*
David Heinemeier Hansson
Dino’s Journal 📖
dispatches
E L S U A ~ A blog by Luis Suarez
*
everything changes
Excursions
*
Flashing Palely in the Margins
*
For You
hailun
inessential.com
Into the Book
*
jabel
*
James Van Dyne
*
Jan-Lukas Else
*
Jason Fried
*
Jim Nielsen’s Blog
*
Kev Quirk
*
lili's musings
*
Live & Learn
Lucy Bellwood
Maggie Appleton
*
Manton Reece
*
Manu's Feed
maya.land
Musings
*
Nicky's Blog
Oblivion With Bells
*
On my Om
*
Pablo Morales
rebeccatoh.co
*
Rhoneisms
ribbonfarm
*
Scripting News
*
Scripting News for email
Sentiers — Blog
Simon Collison | Articles & Stream
*
strandlines
the dream machine
*
The Marginalian
*
thejaymo
tomcritchlow.com
*
Tracy Durnell
writing.as.amit
*
yours, tiramisú

About Reader


Reader is a public/private RSS & Atom feed reader.


The page is publicly available but all admin and post actions are gated behind login checks. Anyone is welcome to come and have a look at what feeds are listed — the posts visible will be everything within the last week and be unaffected by my read/unread status.


Reader currently updates every six hours.


Close
Colin Walker Colin Walker colin@colinwalker.blog