More on comment spam

I’ve been getting more comment spam that’s not automatically nixed by Akismet. Under Settings > Discussion you can add terms to a blacklist so that comments are automatically placed in the bin but I wanted to go one step further.

I found a code snippet described as automatically deleting spam comments but it actually prevents them from being submitted in the first place using wp_die to kill the process on detection of specified terms.

It didn’t seem to work properly throwing “empty needle” errors when the found term was not the first in the list. Not only that but to edit the list of prohibited terms you would have to edit the code itself.

So I reworked it, fixed the empty needle errors by changing the detection method and created a simple plugin which lets you specify banned terms via a settings page in wp-admin.

When a banned term is found the comment is prevented from being submitted and wp_die kicks in displaying a reason and the prohibited term:

I’ll see how it goes and will be adding more items as I come across them, but let me know if you have any problems submitting comments and what term is specified.

You can check out the plugin on GitHub.

More on comment spam

The confusion about the indiewebComments

Alex Kearney wrote about her first two years of indieweb. It makes for a very interesting read.

One thing that really came through was the confusion people feel when looking to "join the #indieweb" - it illustrates a common misconception that sites have to implement every bit of technology going.

Dave Winer said in a blog post that the indieweb community "chose such an inclusive name, but have an exclusive approach" giving RSS as an example, arguing that this established web technology has been ignored.

He obviously has a vested interest because RSS is his baby; I also thought some of his initial comments about JSON Feed were particularly harsh.

Such a comment understandably hasn't gone down too well in some quarters and there is a sense of defensiveness. But there is also a move for the community to look at itself, especially the wiki, in order to identify why these confusions and misconceptions arise.

When new technology is introduced it threatens the status quo whether it intends to or not.

Incumbents can reject 'new' because the status quo is how they've always done things and see no reason to change.

Conversely, the inventors of 'new' will have done so because the status quo doesn't meet their needs or they feel there is a better way.

It's a tricky situation but doesn't need to be either/or.

It's not all about the tech

At its core the indieweb has a set of principles such as owning your data, building tools for yourself and dogfooding them on your own site. The principles even state that user experience design is more important than protocols.

I wrote recently that the indieweb:

"promotes and relies on the open web but recognises that the closed web exists, plays a large part of people’s lives, and tries to integrate with it"

So much of what is discussed and developed as part of the movement relates not to the replacement of social networks and data silos but the integration with them whilst retaining ownership.

That doesn't sound very exclusive to me.

The problem, however, goes back to the perception that the indieweb is effectively insular because of the knowledge required to implement all the various technical elements. Plugins can only take you so far.

The community wrestles with the self-realised existential crisis that it is, currently, a developer community not a user community.

What to implement?

Look back at the principles above.

Okay, they talk about building tools but put the emphasis on the experience before protocols.

For me the indieweb is an idea, a way of doing things rather than the specific technology used to achieve it.

When examining how to establish the number of indieweb properties that existed it was suggested you could:

"consider a page part of the IndieWeb if it has a microformats2 class or advertizes a webmention or micropub endpoint."

This is looking at it from a technical perspective - an obvious indication that the page or site has implemented some form of identifiable indieweb technology. Not all are required, just one.

Yet there is still a problem, and that is the apparent insistence on the implementation of specific technologies as implied by the guides and documentation.

So much for design over protocols.

It is entirely possible for a site to be considered part of the indieweb and conform to its principles without any of these elements being present.

Stepping back

Just as the likes of Winer can seem too close to an established technology so proponents of a new way can be too focused.

Perhaps this is because many of the indieweb developers have been involved for a number of years and, psychologically, moved beyond the initial stages. They can see the destination and are driving full speed to get there.

Perhaps the principles become obscured by the need to get the tools ready for the next generation but they haven't even begun the journey so see completely different scenery.

In the drive to create systems that are simple enough for anyone to use (and we are nowhere near that stage) the how has become more important than the why.

Fortunately, the community already acknowledges the need to step back and view things from a user's perspective rather than that of a developer.

The confusion about the indieweb

Getting plugged in – epilogue

Since raising the version of the 'Likes and Replies' plugin to 0.9.0 and calling it a release candidate I don't think I've come across any problems. It's hard to believe that the last commit logged on GitHub was 17 days ago.

Where does the time go?

There might be some tidying up I could do to the code but, on the whole, I think it's achieved exactly what I was after and reached its natural conclusion.

That's not to say there won't be future updates if I think of new functionality, find problems or want to do things differently. But, for now, I think it's safe to put down a marker and bump it to a 1.0 release.

Although the plugin is reasonably simple I've learnt some good techniques which can be applied again and again while also learning from mistakes that, hopefully, won't be repeated.

It's been fun to go through the process of documenting this publicly, with nowhere to hide, and am glad I could contribute something to those who may use it.

Getting plugged in – epilogue

One thing I like that Dave Winer does on his blog is have posts listed chronologically for any given day - you start the day at the top and read down in time order. We read top to bottom normally so it's natural.

So I've been toying with the idea of a "Today" page to list just the posts from, well, today in chronological order then, maybe, having it as the front page of the blog.

It's easy enough to create a custom WP_Query to do this but I use a plugin to control post excerpt appearance which doesn't seem to want to play with a page based on a custom page template.

It looks as though you can't use pre_get_posts to target a specific page so a custom query appears to be the only way.

I'll keep digging for a solution.

Status

Des asked if I was planning to release my Webmentions Directory as a plugin rather than a page template.

I hadn't considered it but he got me thinking.

I wondered about the best way to do it and came up with creating a shortcode that can be entered on any post or page, and also in a template with the do_shortcode() function.

An initial version is in place on my site and seems to be working fine. It relies on the indieweb Semantic Linkbacks plugin being installed and domain exclusions can only be added by modifying the plugin code - for now.

I will look to add a settings page for the easy addition of exclusions. For example, I have my own domain and micro.blog currently excluded from the list.

As always, you can find it on GitHub.

Status

Getting plugged in – part 6: includes

There are times when I feel like a bit of an idiot. This is one of those times!

As you will no doubt recall, I was trying to separate out the plugin actions into various included files. The relevant code triggered correctly when posting via the REST API (I.e. from Workflow) but the action that should be run when posting natively failed.

I couldn't work out why.

So, I started disabling the safety checks to see if I could find out why it wasn't working. I soon discovered that it was the nonce check that was failing.

And then it dawned on me...

I had copied the code directly from the main plugin file to an include but not altered the condition for the nonce check.

Why did this matter?

Well, the original condition was basename( __FILE__ ) - used when you want to check that the action is being triggered within the same file. Quite obviously it now wasn't, so the check failed and the code never ran.

Changing the condition to a manually entered string meant the nonce verification would now pass and the code be triggered.

(At least we know that nonce verification works.)

I have now separated all functions into includes called from the main plugin file and everything appears to be working nicely.

I have, therefore, bumped the plugin version to 0.9.0 and called it a release candidate. Barring any glitches or obvious problems this will be the final code.

As always, the latest version is available in the GitHub repository.

Getting plugged in – part 6: includes

Getting plugged in – part 5: settings

What started as a quick update to split the plugin into parts (so that it wasn't all one monolithic file) became quite a major one.

My original plan was to move both hooks for updating the post content to separate included files - it hasn't quite gone according to plan.

The hook for wp_insert_post_data (used when posting via Workflow) has worked fine. When the save_post hook and function were moved to a separate file, however, I couldn't get it to trigger.

I have had to move it back into the main plugin file until I find a solution but this setback gave me an excuse to look at adding a settings page and adding the code for that to an include instead.

As when creating the meta box, this was very much a case of doing it in stages, ensuring each worked before moving on.

Settings page

First up we need to create an admin menu entry using the admin_menu hook and add_menu_page() function:

add_action('admin_menu', 'landr_menu');

function landr_menu() {
    add_menu_page('Likes and Replies Settings', 'Likes and Replies', 'administrator', 'landr-settings', 'landr_settings_page', 'dashicons-admin-generic', 3 );
}

The parameters for the function are as follows:

  • the settings page title (in the header title tag)
  • the text for the menu item
  • who can see the menu item (capability)
  • the URL slug for the page
  • the callback function used to populate the page
  • the menu item icon
  • the position in the menu (if not entered it will be at the bottom)

That's all you need to create a settings page, it'll be empty but it's surprising how simple it is to create. Now we just need to add to it.

Form and options

We use the callback function mentioned above to populate the settings page - whatever you include within the function gets display. Essentially you are just building a HTML form but with a few special features.

I wanted to have the option to change the text used by the plugin to precede the like and reply links so that's what the form here will set.

The form fields need to be identified so their values can be saved so we first have to make WordPress aware of the options we are going to be saving to the database. They must be contained in a settings group which we can think of as like a HTML fieldset.

function landr_settings() {
    register_setting( 'landr-settings-group', 'like_text' );
    register_setting( 'landr-settings-group', 'reply_text' );
}

The form itself is simple and pretty standard but should include at least the below:

  • the post method points to WordPress' built in options.php for saving any settings
  • calling the settings group for the form using settings_fields() - it must match the group registered above
  • the fields will be named the same as the registered settings.
  • the standard WordPress submit_button();

Registering the settings we intend to use, calling the settings group and naming the fields correctly means that WordPress takes care of everything for us. Using options.php in the post method does exactly as you would imagine: saves the settings to the options table in the database.

Using the values

We use the saved settings in a couple of ways: firstly, the form itself needs to show the current values when loaded, and the code to update the post needs to pull them in.

Luckily, reading the values is simple because of where they are stored - we just use the get_option() function with the setting name as a parameter. Then, just to be on the safe side, we escape the values to remove any invalid characters:

esc_attr( get_option('reply_text') );

We just echo this wherever the option is needed.

Defaults, activation and deactivation

Whenever options are being written to the database it is good practice to remove them if the plugin is deactivated or uninstalled - we don't want to leave unnecessary data lying around.

We also need to set some defaults in place when the plugin is activated. Yep, you guessed it, we're using hooks again:

register_activation_hook( __FILE__, 'landr_activate' );
register_deactivation_hook(__FILE__, 'landr_deactivate');

function landr_activate() {
    add_option('like_text', 'Liked:');
    add_option('reply_text', 'In reply to:');
}

function landr_deactivate() {
    delete_option('like_text');
    delete_option('reply_text');
}

__FILE__ is used in the hooks because they are included in the main plugin file.

And that's it, we're done.

I will try to get the save_post hook working from an include in a future revision but, for now, the updates have been pushed to the GitHub repository.

Getting plugged in – part 5: settings

Getting plugged in – part 4.5

After getting the meta box working I realised that the code hooked into save_post wasn't being triggered when posting via the Workflow app.

The app is probably using the WordPress REST API to create the post which doesn't behave in the same way as native posting and bypasses the hook.

John Johnston suggested hooking into wp_insert_post_data instead for posts made in this way.

I didn't know anything about this method so had to do a bit of research but discovered that it was actually pretty simple to use.

The main caveat was that, as I needed to access the post ID to work with the meta data, I had to pass the second, optional parameter $postarr to my function or it wouldn't work.

add_filter( 'wp_insert_post_data', 'filter_post_data', '99', 2 );

function filter_post_data( $content, $postarr )

It was then largely a case of repurposing the same code from the save_post hook to build the updated post content and return it from the function:

$content['post_content'] = /* new content value here */;

return $content;

It seemed to be working when viewing the post preview but when going to edit the post draft in wp-admin only the original post content was showing.

And I couldn't work out why.

I broke the site a few (a lot) times trying to figure it out before, eventually, realising that the content displayed by the front end and the edit post page are actually saved in different places.

The post as seen by the reader takes its content from the post_content field but the edit post screen pulls its version from post_content_filtered - I had been updating the former but not the latter, hence the confusion.

With this accounted for everything works as planned either locally within WordPress or when posted from Workflow.

Thanks John.

Getting plugged in – part 4.5

Getting plugged in – part 4: meta boxes

Numerous tutorials exist for adding and using meta boxes; some manage to make it seem like a dark art by rushing through too much in one go without explaining exactly what is going on.

This isn't going to be a guide, more a detailing of the steps I have taken to get a meta box in place for adding the 'liked' and 'reply' custom fields on posts then converting those into #indieweb webmention links.

Display a simple meta box

This is the easy bit, and I actually mean easy.

WordPress provides the add_meta_boxes hook and add_meta_box() function to easily register a custom box. We first need to create a function which calls add_meta_box() then add that function as an action to the add_meta_boxes hook:

function landr_custom_meta() {
    add_meta_box( 'landr_meta', 'Like and Replies', 'landr_meta_callback', 'post' );
}

add_action( 'add_meta_boxes', 'landr_custom_meta' );

The add_meta_box() function needs four parameters although more can be used to control where the meta box sits on the page, these are: ID, the box title, a callback function which actually does the work, and the type of post we want it to be used with.

Just to prove everything is set up properly a callback function can be created:

function landr_meta_callback() {
    echo 'Likes and replies go here.';
}

And, just like that we have a meta box.

It doesn't do anything yet, but it was easy to create. Now comes the fun stuff.

Adding fields and saving values

Getting the required fields in is just a case of adding some standard HTML (so I won't go in to that here) ensuring that the fields are named properly so they can be referenced later.

With the fields added we need to be able to write their values to the database if they have been populated. Another function coming up added as an action to the save_post hook.

As we saw before, security is paramount when dealing with WordPress plugins. Because we are dealing with something that can write to the WordPress database we need to ensure that this is handled securely.

There are two steps to take: firstly, adding a nonce (number used once) field to the meta box which will prevent improper access, and double-checking that the person trying to save the data is allowed to do so:

//create nonce

wp_nonce_field( basename( <strong>FILE</strong> ), 'landr_nonce' );

//check if nonce exists and is verified

if ( ! isset( $_POST[ 'landr_nonce'] ) ) {
    return;
}

if ( ! wp_verify_nonce( $_POST['landr_nonce'], basename( <strong>FILE</strong> ) ) ) {
    return;
}

//check permission to edit post

if ( ! current_user_can( 'edit_post', $post_id ) ) {
    return;
}

The 'liked' or 'reply' values can then be written to the postmeta table using add_post_meta() ready to be converted into webmention links.

if ( isset( $_POST['liked-url'] ) ) {
    $liked_url = sanitize_text_field( $_POST['liked-url'] );
    add_post_meta( $post_id, 'Liked', $liked_url );
}

if ( isset( $_POST['reply-url'] ) ) {
    $reply_url = sanitize_text_field( $_POST['reply-url'] );
    add_post_meta( $post_id, 'Reply', $reply_url );
}

Webmentions

Previously, the plugin used the content_save_pre hook to add the webmention link to the start of the post content. With the new code linked to save_post it wasn't going to work as it fires after content_save_pre.

The original code to write the link was modified slightly and moved into the same function so everything is processed at the same time.

It needs some tidying up but the updated plugin can be viewed in the GitHub repository.

Getting plugged in – part 4: meta boxes

After getting the directory page to display all replies, I thought:

"What if when a comment is received we immediately perform the linkbacks type check and, if true, rewrite the comment_type value back to the comments table in the database?"

How to do this?

WordPress has a 'comment_post' hook and actions added to this fire immediately after a comment is posted.

So, I just needed to hook in a function which checks comments as soon as they are posted and, if they are webmention replies, update their comment_type field in the database.

The below seems to work using the same linkbacks type check and then $wpdb->update to write back to the database:

function wm_comment_type( $comment_ID ) {

  $wmreply = get_comment_meta( $comment_ID, 'semantic_linkbacks_type', true );
  global $wpdb;

  if ( $wmreply == 'reply' ) {  

    $result = $wpdb->update(
      $wpdb->comments,

      array(
        'comment_type' => 'webmention' 
      ),

      array(
        'comment_ID' => $comment_ID
      ),

      array( 
        '%s'
      ) 
    );

  }
}

add_action( 'comment_post', 'wm_comment_type', 100, 1 );
Status

Improving the webmentions directoryComments

When creating my webmentions author directory I originally wanted to avoid any reliance on the #indieweb Semantic Linkbacks plugin, as not everyone will be using it, and I wanted to keep things fairly self-contained.

It also meant that if you are not using webmentions you could just remove that argument from the initial query and just use the template for any comments.

Unfortunately, WordPress doesn't provide proper support for comment types as it does post types.

So, when a webmention reply is received (as opposed to likes or RSVPs etc.) WordPress the plugin converts it to a normal comment, removing its 'comment_type' value. This means it doesn't get caught if filtering for only webmentions. (Thanks to Michael for the full lowdown.)

In order to catch these replies I am forced to add a check against the 'semantic_linkbacks_type' for each comment to see if it is actually a reply.

$wmreply = get_comment_meta( $comment->comment_ID, 'semantic_linkbacks_type', true );

  then check if

$wmreply == 'reply'

The original query included 'type' => 'webmention' but this, obviously, had to be removed so that the linkbacks check could be performed against all comments.

Combining the two type checks gives the desired result but, for my purposes, there is a caveat.

I wanted the directory to list those engaging via directly from their own sites but the above also lists replies send from Micro.blog as the service supports webmentions. I have, therefore, added it to the list of exceptions not to return authors for - I had already excluded my own domains and blank urls.

The current (sanitised) version of the directory page template is available on GitHub.

Lightbulb moment

After making these changes I realised there should actually be an easier way, or a more streamlined one at least.

What if when a comment is received we immediately perform the linkbacks type check and, if true, rewrite the comment_type value back to the comments table in the database?

But that's a project for another time.

Improving the webmentions directory

Getting plugged in – part 3: getting it wrongComments

A big part of learning by doing is getting things wrong. You're never going to get it right first time, every time and just have to accept that.

It's what prompted me to write that I needed a second WordPress installation to test against rather than keep breaking the blog.

But, as well as getting it wrong, we need to be willing to admit that we got it wrong.

And that's where I am at the moment.

The basic plugin is in and working, and hopefully a little bit more secure than it was, so I now I need to set priorities for the next steps.

The two main to-dos on my original list are:

  • register the 'liked' and 'reply' custom fields so always available
  • add a settings page

i think i was looking at the custom fields the wrong way as I haven't found a way to make an unpopulated entry be always available in the drop-down. It appears that items only exist in the custom fields drop-down if they are currently part of a key:value pair in the database.

As the plugin removes the target URL when the post is updated the options are no longer available.

I could be wrong. (Again.)

Custom field vs meta box

Off in search of another option I wondered about using meta boxes. Surprise, surprise, I was looking at them the wrong way.

I'm detecting a pattern here.

I initially thought that meta boxes were a separate, self contained entity and wouldn't play nice when posting from somewhere like WorkFlow.

But, a bit of research later, and it turns out that meta boxes are actually just a way of displaying the mechanism to add and store post metadata - a custom version of custom fields.

Creation of a metabox may require a little change to the base code but that'll be easy to manage and the next step in the process has essentially been decided for me.

So, no coding in this part, but there are advantages to being wrong.

Getting plugged in – part 3: getting it wrong

CSS needs full dependency selectors.

There, I've said it. Not that I'm the first and certainly won't be the last.

CSS should let you choose what to do with something based on what's inside it but this isn't possible. We can only style an element based on what contains it and not the other way around.

To introduce a bit of consistency with the look of the blog I needed to reduce the bottom margin of <p> tags that contained the "read more" link but there is nothing to distinguish these from any other <p> tag except what they contained.

As CSS does not provide a means to do this I needed another solution.

Step up jQuery which uses the .has selector allowing use to target an element if it has a conditional item within it.

Here is a quick jQuery function that apples the styling for the "shrink-p" class to a <p> tag if it contains a link with the "more-link" class:

<script>
  jQuery(function ($) {
    $( "p" ).has( "a.more-link" ).addClass( "shrink-p" );
  } );
</script>
Status

I added a filter to functions.php to truncate posts in the RSS feed of type 'status' that were longer than 280 characters, then insert a permalink at the end, so that they would play nicer with Micro.blog.

On the one hand it's good for it to be obvious they are longer posts without titles but, at the same time, it comes back to the issues of distribution and whether what we do on our own sites should be dictated by the means of distribution.

I'm torn between thinking "it's a tough call" and "what am I playing at?"

Status

Removing post titles from WordPress RSS feedsComments

I previously detailed a method of automatically replacing blank post titles so that I didn't have multiple items (posted from the Micro.blog app) listed as '(no title)' in the WordPress back end.

Micro.blog wants posts to not have titles but will treat certain text as blank, such as the date, hence the above.

If a status coming from a blog (via RSS) has a title, or one not in a recognised format, Micro.blog will treat it as a long form post and just display it as a title and link. Not ideal.

Jimmy Baum (@jimmymansaray) wanted to have just the date as status titles but this falls foul of these limitations.

So, how to allow different title formats on your own blog but then pass data to Micro.blog in such a way it displays those statuses correctly?

the_title_rss

WordPress allows you to customise your RSS feed on the fly by accessing different aspects of it and applying a filter. Just like modifying post contents prior to saving or display.

The post title in an RSS feed is logically called using the_title_rss() so we need to find a way to modify this based on the specific set of requirements.

Just as I do, Jimmy uses a category called 'microblog' with a dedicated feed so this is what we need to filter on. After a few tests it seemed the most reliable way was to check against the_category_rss() which lists the categories for the current post as would be displayed in the feed itself, like the below for my previous post:

<category><![CDATA[Podcast]]></category>
<category><![CDATA[Distribution]]></category>
<category><![CDATA[Microcast]]></category>
<category><![CDATA[Syndication]]></category>

We can then simply check for the presence of the required category within that string and return an empty title.

The code to do so should be as follows:

function remove_post_title_rss ( $title ) {
  $categories = get_the_category_rss();
  $pos = strpos($categories, 'microblog');
  if ( $pos != '' ) {
    $title = '';
  }
  return $title;
}

add_filter( 'the_title_rss', 'remove_post_title_rss');
Removing post titles from WordPress RSS feeds

Getting plugged in – part two: securityComments

The security of any code should be of the utmost importance but, if creating a plugin that might be distributed to other people's sites, it should be paramount.

It's one thing messing up your own site but another entirely breaking someone else's when they've put their trust in what you've written.

As we established last time, this series covers the process of improving my "Likes and Replies" WordPress plugin. This is a relatively simple piece of code that doesn't do much (yet) but it's still good to ensure it's as secure as it can be and get into good habits.

First steps

It is best practice to prevent direct access to plugin files meaning they can only be used within the context of a WordPress installation. This is done by adding the following to the start of any PHP files:

if (!defined('ABSPATH')) exit; // Don't run if accessed directly

ABSPATH is the absolute path to the WordPress installation directory and is defined by WordPress itself. If this is not available to the plugin it is not being run within the context of an installation.

We are relying on user input in the form of a URL to add likes or replies so should take steps to ensure that this is properly encoded and valid. We can use esc_url() to do this which removes invalid or dangerous characters.

Getting the address entered into the custom field then becomes:

$mentionurl = esc_url(get_post_meta($id, $type, true));

I had already made a change to the code replacing file_get_contents() with wp_remote_get() as the former was considered insecure.

A good start

With a more complex plugin additional protection may be required, like sanitising inputs which I, no doubt, will have to do later if I am able to meet my goals.

This is a good start to the learning process but I now need to work on my priorities for what to tackle next.

Getting plugged in – part two: security

Getting plugged in – part oneComments

I mentioned recently that I need to learn to code properly but it's more a case of learning the environment in which I'm working.

When I used to do a bit of VBA (Visual Basic for Applications) in a previous role the VB side of it wasn't an issue, the more complicated part was the A, the applications with their object models, and how you get them to do what you need.

It's the same with WordPress.

I can get by with PHP but learning the actions, hooks and filters that make WordPress do what you want takes time.

Saying that, it always amazes me just how easy it is to create a simple WordPress plugin - just add a comment to the start of a PHP file:

/*  
    Plugin Name: My plugin  
*/

You would normally add in more details like Description: and Version: but the only actual requirement for WordPress to recognise it as a valid plugin is a name.

How simple is that?

What's next?

Having moved my code for indieweb 'likes' and 'replies' from functions.php to a plugin I feel this is a perfect opportunity to learn by doing. While the plugin does what it's supposed to I would like to add enhancements to make it a more complete offering.

So, here's a quick 'back of an envelope' list of what I want to do:

  • add security to plugin to stop direct access (an easy one to start)
  • add a settings page to save plugin options
  • use these to determine the text added to a post
  • maybe add a choice to insert the text at the top or bottom of the post
  • register the 'Liked' and 'Reply' custom fields so always available in the drop-down
  • remove those entries when plugin is uninstalled

Getting to grips with each element of this is going to be a journey that I will document, for my own benefit if nothing else but if it helps others then that's great.

I hope you'll join me.

Getting plugged in – part one