News Sniffer Blog

News Sniffer Overhaul

johnl — Mon, 07 Jun 2010 00:01:54 +0000

I’ve given News Sniffer a huge overhaul – the culmination of over a year’s work (on and off part time). The biggest change is that we’re no longer checking for censored comments on Have Your Say. This was an intensive system to run and maintain and I just can’t dedicate the time and resources to it any more. The archive is currently removed but will be restored soon.

News Sniffer’s focus is now on monitoring changes to news articles on the web.

The whole news article version checking system has been overhauled making it much more efficient and easier to maintain. This makes it much easier to add features, particularly additional parsers for other news websites (we’re currently only watching the BBC). Watch this space over the coming months.

The check scheduling system is now much more reliable too, allowing more regular checks of more articles with less resources.

Revisionista broken, will be fixed soon

johnl — Wed, 15 Apr 2009 09:35:57 +0000

Due to a recent change at the BBC, Revisionista can no longer properly detect changes in articles. To fix this, I’ve brought forward plans for a new version of Revisionista. This should be up and running within a couple of weeks.

The new version will fix these problems, but also make it much easier to fix if things change again in the future. It will be much easier to add additional news sources too – so stay tuned for a great new Revisionista.

In the mean time, sorry for the inconvenience.

UPDATE: The BBC seem to have made more changes and now Revisionista is working again, thanks BBC!

The Register references News Sniffer

johnl — Fri, 14 Nov 2008 19:05:56 +0000

The Register used News Sniffer’s Revisionista to track the changes of a BBC article on the ID cards.

The report’s revision history, documented by News Sniffer, takes us on an impressive Odyssey from “Smith to unveil airport ID scheme”, through “Shops may take ID Card biometrics” to the final sales-pitch version, cunningly deniabled with quote marks.

http://www.theregister.co.uk/2008/11/14/bbc_stories_cant_wait/

Updates not working for the last week

johnl — Sat, 14 Jun 2008 01:51:01 +0000

Both Revisionista and Watch Your Mouth have not been updating correctly for about a week due. Working again now. Sorry for the outage.

Watch Your Mouth down for a week

johnl — Wed, 10 Oct 2007 18:06:59 +0000

The BBC upgraded their forum software on the Have Your Say site and it broke News Sniffer. If you’re interested, they’ve changed from using the “Thread ID” in their urls to using a “Forum ID”. I’ve done a quick patch up to fix things, but any comments censored in the last week or so are lost to News Sniffer. It’sl going to take a while whilst things get back up to speed, but it should be done by tomorrow morning.

Thanks to the News Sniffer reader who spotted the lack of updates and got in touch with me. It was a relatively quick fix once I knew about it.

Now including BBC health articles

johnl — Sun, 22 Apr 2007 15:27:12 +0000

I’ve just added the BBC health feed to Revisionista so we’ll start monitoring more health related articles. So far, the only health articles included were from the World or UK feeds.

Now excluding BBC sport articles

johnl — Sun, 22 Apr 2007 15:20:01 +0000

Revisionista now explicitly excludes BBC sport articles from it’s monitoring and I’ve deleted any from the last couple of months. This is because these articles generate a vast number of revisions mostly due to score updates during games. This is wasting News Sniffer resources that could be better spent monitoring other articles.

Upgrade: wym comment cleanup, downtime, and improved search

johnl — Fri, 20 Apr 2007 12:13:53 +0000

News Sniffer was down for a couple of hours yesterday afternoon whilst I upgraded the software that runs it. This fixes a few display bugs and more importantly patches up another problem where bbc comments were being misclassed as censored. The misclassification was due to a mistake when handling british summer time, so it’s only been prevalent since the end of March. We double checked all censored comment from the last few months and removed any misclassified ones.

Due to the downtime whilst we cleaned and reindexed the database, some news articles and bbc comments were not added to the database so will not be tracked. So, if you’re looking for a particular news article published yesterday afternoon, it’s likely that it is missing. Apologies for the inconvenience.

The biggest new feature is vastly improved search system.

It works like most search engines, so just type keywords in and you get results. You can be a bit more advanced too though. Say you want to find all censored bbc comments by the author ‘John Smith’. Do a search for:

author:"john smith"

You want all the comments mentioning the NHS but to exclude any mentioning Patricia Hewitt:

nhs -hewitt

Maybe you want to find all censored bbc comments with the words ‘iran’ and ‘bomb’ made in April 2007:

iran bomb created_at:200704*

To find a particular bbc comment, you can search on the bbc’s own comment id number:

bbcid:bbcid:2481215

To get all Revisionista news articles with the work ‘iraq’ in the title published today (19th April 2007):

title:iraq created_at:20070419

The BBC Have Your Say search is here, and the Revisionista news article search is here. Happy searching!

New domain name

johnl — Mon, 09 Apr 2007 15:36:01 +0000

News Sniffer now has its own domain name: www.newssniffer.co.uk
The old address (newssniffer.newworldodour.co.uk) will automatically redirect you, but update your bookmarks!

No updates for the last 2 days

johnl — Wed, 28 Feb 2007 23:23:01 +0000

Apologies but due to a VPN problem, News Sniffer has not been monitoring news articles or Have Your Say forums for the last 2 days. All working again now though. I’ll set up some alerting system to prevent this happening again.