Essentially, this seems to be: add the tag ‘geotagged‘, along with the tags ‘geo:lat=X.xxx‘ and ‘geo:lon=X.xxx‘, where the X.xxx’s are the latitude and longtitude numbers that are likely to come straight out of your GPS, in decimal degrees (WGS84).

This is all very nice, but the problem with tags in this format is that there is no easy or efficient way to use them to retrieve all items tagged for a particular locality. Sure, if I’m standing right on top of the Eureka Tower at -37.821362,144.964213, I can search for tags geo:lat=-37.821362 and geo:lon=144.964213 to find all the geotagged links for that exact location, but what if I’m standing 50 metres across the street looking up at the tower and want to search for links near my current location ?

Enter the geohash, a hash function for geolocation coordinates invented by Gustavo Niemeyer (not to be confused with the xkcd Spontaneous Adventure Generation algorithm of the same name). Wikipedia gives a reasonable explanation of how geohashes work … essentially the latitude and longitude are encoded as strings like r1r0fdzdwg. Geohashes have the useful property of having arbitrary precision … geohashes with the same prefix represent locations in the same vicinity. This means that the location across the street from the Eureka tower, at geohash r1r0fdy7sm, shares the prefix r1r0fd with the geohash closest to the top of the Eureka Tower, at r1r0fdzdwg.

My proposal for delicious geotaggers is that in addition to the geo:lat and geo:lon tags, several truncated geo:hash tags should also be used. If I were to bookmark something related to the Eureka Tower, I may tag it:

geotagged
geo:lat=-37.821362
geo:lon=144.964213
geo:hash=r1r0fdzdwg
geo:hash=r1r0fdz
geo:hash=r1r0f

Then, anyone searching for the tag geo:hash=r1r0f will find every item within the area that this geohash covers … this would include not only the Eureka Tower, but the Rialto Towers too.

For each bookmarked item, the number of truncated geohashes used as tags roughly determine the distance ranges (ie bounding boxes) that can be searched. Exactly which truncations, or how many geohash tags to use, is an existing problem that I haven’t yet decided the best solution for; is it best to ‘overload’ with every possible geohash truncation (eg include tags geo:hash=r1r0fdzdwg, geo:hash=r1r0fdzdwg, geo:hash=r1r0fdzdw, geo:hash=r1r0fdzd, geo:hash=r1r0fdz …etc… to geo:hash=r) ? This is probably overkill. A better approach would be to choose just a few key truncations that roughly correlate to a range of sensibly sized patches on the Earths surface, eg, bounding boxes with diagonal lengths of:

• geo:hash=r1r0fdzdwg ~60 cm ['exact']
• geo:hash=r1r0fdzd ~20 m
• geo:hash=r1r0fdz ~150 m
• geo:hash=r1r0fd ~600 m
• geo:hash=r1r0f ~4.8 km
• geo:hash=r1r0 ~19.5 km
• geo:hash=r1r ~150 km

These ranges map loosely to those deemed useful by Brightkite, which lets you search for events around you within 20 m, 200 m, 2 km, 4 km, 10 km, 50 km and 100 km. Maybe we only need a few of these. If only a few truncations were provided by the tagger, the user can always execute multiple searches, starting from the full geohash of their current location and truncating back, character by character, (effectively expanding the search radius) until they start to get hits. There may also be techniques whereby the last character(s) of the truncated hash can be incremented/decremented to search neighboring bounding boxes (eg for r1r0fd, also search for r1r0fc, r1r0fe tags), although I need to think about this a little more.

Of course, the best solution for more useful geotagging within delicious would be for delicious/Yahoo to explicitly support some style of geotagging and provide a geotag-aware search facility … but until that day, geohashes may well do the job well enough. Next step for me: write a proof of concept application that actually produces and makes use of these types of tags ….

For example, typing:

gg open science on friendfeed

into the YubNub search box searches Google for “open science on friendfeed“, via YubNub.

I thought I’d highlight a few life science- and bioinformatics-related YubNub commands I find myself using quite often in my day-to-day work. Some are commands I created, others someone else created. This is the beauty of YubNub … often someone has already made the ‘obvious’ command … it’s worth just trying to search with a command you expect to exist, since it often does.

Onward, with the list:

• pubmed — Searches PubMed
• hubmed — Searches HubMed (Alf Eatons featureful alternative interface to PubMed)
• gopubmed — Searches GoPubMed (an ontology enhanced PubMed search)
• doi — Redirects you based on a Digital Object Identifier (DOI), via http://dx.doi.org/
• pdb — Searches the Protein DataBank for 3D structures. Usually the search term should be a 4 letter pdb code.
• uniprot — Searches the Uniprot database (use an accession, id or keyword as the query).
• ihop — Searches iHOP, information Hyperlinked over Proteins, for views of the biomedical literature guided by gene networks. Nothing to do with pancakes (or prayer).

There is also a class of more general, non-biomedical commands which I often use:

• gg — The Google.
• gim — The Google Image Search.
• wp — Good ol’ Wikipedia.
• ucc – The universal currency converter at XE.com. Use it like ucc 399 aud usd, to convert $399 Australian dollars to US dollars. Then, if you have your cash in Australian dollars, weep about the recent drop in the exchange rate
• man — Like *nix man ‘manual pages’, but for YubNub commands. Eg, man ucc will give the manual page describing how to used the ucc command.
• ls — A bit like the *nix shell ls, this command lists existing YubNub commands that contain your query in their name, description or url. eg. searching ls protein gives you a short list of all the commands related to proteins.

I’ve installed the YubNub opensearch plugin so I can search directly from the search box (or location bar) in Firefox. Maybe one day Ubiquity will fulfill this purpose, since in many way it is the natural progression of the YubNub idea. But for the moment YubNub is the fastest, most streamlined way I’ve found to quickly fire off a search when I need to hunt down a reference, protein sequence or 3D structure. Nothing like instant gratification

I recently needed to make a simple, two dimensional figure of a beta-barrel membrane protein. I went hunting for programs that might take a sequence and/or structure and produce a pretty looking diagram to save me constructing everything by hand. Here are two I found and tried.

TMRPres2D

Ioannis C. Spyropoulos, Theodore D. Liakopoulos, Pantelis G. Bagos and Stavros J. Hamodrakas TMRPres2D: high quality visual representation of transmembrane protein models Bioinformatics. 2004; 20: 3258-3260. (link)

Pros:

• Cross-platform (Java)
• Simple interface, GUI (zero learning curve)
• Lots of input options (defines transmembrane regions directly from SwissProt or PIR annotations online, takes input from several transmembrane region predictors)
• Lots of output formats and options (Postscript, gif, jpg, png, svg, bmp)
• Various colouring options (hydrophobicity, charge, “printer friendly”)
• Makes reasonable looking diagrams of helical transmembrane proteins

Cons:

• Doesn’t handle beta-barrel membrane proteins gracefully (strand drawing is overlapped, messy).
• The membrane is assumed to be a eukaryotic plasma membrane, with labels “cytoplasmic/extracellular” (which should be, for instance, “extracellular/periplasm” for a bacterial outer membrane protein). This is easily changed on the diagram with external editing.

TeXtopo

Beitz, E. (2000), TeXtopo: shaded membrane protein topology plots in LaTeX2e. Bioinformatics 16: 1050-1051. (link). See the original publication or Professor Eric Beitz’s site for a better example than my image.

Pros:

• Beautiful, clean, publication quality diagrams, courtesy of LaTeX
• Multiple input options (Swissprot format, PHD, HMMTOP, user defined)
• Multiple sequence annotation options including colouring by various physiochemical properties (hydrophobicity, charge), sequence conservation or user defined schemes.
• Will depict membrane embedded half-loops and lipid anchors.
• Versatile output (Postscript, pdf, dvi, basically anything that LaTex can be rendered as)
• Also can generate attractive looking helical wheel plots
• Did I mention the output is clean and looks great … ?

Cons:

• Steep learning curve for the uninitiated, despite extensive documentation (ie LaTeX code, no GUI)
• No support for beta-barrel membrane proteins

If I ever need to make a 2D diagram of a helical membrane protein for a publication, TeXtopo would be my first choice. For quickly getting an overview of some transmembrane prediction results or a protein with defined tranmembrane regions in Uniprot, TMRPres2D is the quickest and easiest method.

In the end, since neither program would do a decent job at cleanly depicting the strands of a beta-barrel in a simple 2D plot, I ended up coding my own hackish solution (svg_barrel.tar.gz or svg_barrel_gui_win32.zip) using Python and a tweaked version of SVGdraw.py. This allowed me to generate some SVG graphics to use as a starting point, and then hand edit the result in Inkscape to align strands to loosely match the real hydrogen bonding patterns. I also added some simple beizer curves for the loops, since neat placement of loop residues was the tricky part that I decided I didn’t have time to tackle.

Here’s the end result, after hand editing:

And here is the 3D version, as a point of reference:

The 2D vector diagram could do with some work to aid in a more accurate representation (unfortunately ‘flat’ views of a 3D barrel always have to make some compromises), but it does the job. The goal was to keep it simple … simple it is. One day I may extend this code to actually use known structure coordinates to automatically align the strands (saving tedious manual alignment), and write some code that properly lays out the loops.

Anyone know any other programs of similar functionality I’ve missed ?

New features include:

• Suggest/autocomplete for journal title field, using the journal title lists provided by PubMed.
• A “Verify” button. Allows a ResolveRef URL to be constructed with the web form and verified as working and valid without actually forwarding the user to the article.
• Some bugfixes (handled the case where there is no DOI in the PubMed record, handled network timeouts to PubMed)
• Refreshed visuals
• Disqus comments box for feedback

In the interest of just getting something working quickly, I implemented the suggest feature in the laziest, possibly most RAM and CPU hungry way possible (the “JQuery Suggest” code queries the web app with substrings as you type each character. At the server side, the app uses a regex to scan a ~1.5 Mb list of journal titles held in RAM). I’ve already noticed a few “This request used a high amount of CPU” warnings in the logs, with the threat “High CPU requests have a small quota, and if you exceed this quota, your app will be temporarily disabled“. If my nasty hack starts heating up Google’s datacentre too much, I might have to disable the ’suggest’ feature until I can implement it “properly”.

Reflections, discoveries

This idea of implementing Openref-style article identifiers has been an fun experiment, and a nice way to learn more about the ins-and-outs of PubMed. When working on implementing the ’suggest’ feature, a major drawback became even more apparent … journal titles (the [TA] field) used by PubMed are not always easily guessable, and many common abbreviations used in reference lists do not appear to exist in PubMed’s downloadable flat-file journal title lists. This is the list that ResolveRef uses to make the ’suggestions’, so having ‘missing’ journal titles presents a problem if I want users to be able to painlessly construct ResolveRef URLs.

Proc. Natl. Acad. Sci. U.S.A. is a perfect example. Many article bibliographies use PNAS – that would be my guess if I were trying to create a ResolveRef URL for a PNAS paper – and yet this journal title does not exist as far as PubMed’s official journal list is concerned. Issues surrounding this problem were discussed on Noel’s original OpenRef post. The odd thing, is that if I search the PubMed Journals database, for “PNAS”, it finds it, and gives me a record where PNAS is listed under “Other titles(s)”. If someone could point me to where I can get these extra fields containing additional names for a journal that are not provided in the the downloadable flat-files, it would be much appreciated (I bet Alf knows the answer. Or maybe I should email the folks at PubMed). If I can get a better list of titles the ’suggest’ feature in ResolveRef would suddenly become a whole lot more useful. Another way around this may be to use CrossRef, and I’m looking into that, but I get the feeling that usage of the CrossRef API is more restricted, so I haven’t bothered with it so far.

Thoughts about the future of ResolveRef / OpenRef

At this stage, ResolveRef URLs are not actually identifiers. They simply act like a frontend to a single-hit PubMed search, and several different ResolveRef URLs can return the same DOI URL (and hence the same journal article). A proper identifier would have a one-to-one mapping between the human-readable ResolveRef URLs and a DOI. In the future, I may attempt to get ResolveRef to ‘normalize’ URLs by allowing only a single journal title for each journal and forcing the use of volume numbers if present. The user could use the web interface to enter the values, and ResolveRef will return a normalized URL. Only normalized URLs would successfully forward to the DOI URL, others will return an error with “Did you mean ..insert normalized URL ..?“. One drawback is that this would reduce the guessablity of ResolveRef URLs, but the advantage is that they could be treated like identifiers: one article would have one and only one valid ResolveRef URL. By requiring a tool (like the ResolveRef web form) to help users build a vaild URL, and removing some of the guessability, ResolveRef would move a little closer to a reinvention of OpenURL (although I think OpenRef/ResolveRef URLs are still more readable and cleaner than OpenURLs, and are much more guessable if you have a bibliography in front of you).

A key cosmetic (and philosophical) difference between OpenURL and OpenRef/ResolveRef URLs is that OpenURL uses HTTP GET fields, eg ?title=bla&issn=12345, while OpenRef/ResolveRef uses the URL path itself eg, somejournalname/2008/4/1996. It’s a bit like one scheme was designed in the age of CGI scripts, while the other was designed for web applications capable of more RESTful behaviour. In my mind OpenURL is more versatile but much uglier, while OpenRef is cleaner and simpler but can only reference journal articles. OpenRef-style URLs will never be able to reference the breadth of resources that an OpenURL can theoretically handle. Maybe hybrid solution could work … some kind of OpenURL server that could “speak OpenRef” … accepting OpenRef-style URLs where possible, while still dealing with regular OpenURL style “?bla=blarg&” query strings for everything else.

As far as I can tell OpenURLs are not identifiers with a one-to-one URL-to-article mapping – this is a drawback since you could not do a Google search to reliably find sites that reference an article via it’s OpenURL … you theoretically could do this with a normalized OpenRef/ResolveRef URL, since there will only be one unique string used to reference any one article (as Noel pointed out, OpenRef strings have some properites akin to InChi strings). Obviously to do this cleanly, ResolveRef would need a nicer domain (something akin to dx.doi.org).

Anyhow, I’m not expecting ResolveRef / OpenRef to make any impact on anything anywhere anytime soon. I’m not a librarian, I don’t sit on an NISO/ANSI committee, and I don’t see publishers seeing a need to adopt anything beyond the DOI. But it’s been nice to play around with, and I’m likely to continue doing so.

Firstly, there is the “just testing and don’t actually have a citation on hand to key-in” class of users, that tried requests something like:

/ref/xx/2007//

Not much sympathy here … it’s pretty much like dialing a random phone number and hoping it someone will pick up.

Then there is a class of users who appear to have sensible intentions, but provide incomplete ResolveRef URLs, eg:

/ref/Organic%20Letters/2000//

Maybe I poorly described ResolveRef in the initial announcement, maybe the documentation in the “About” box on the ResolveRef site is unclear or maybe these users just didn’t read the docs in the first place. When I described the service as “A RESTful way to do PubMed searches”, maybe it would have been more accurate to say “A simple, RESTful way to resolve a single journal article using only the human-readble citation information”. ResolveRef does not give a list of results to a PubMed search; it forwards to a single hit (ideally the requested article), or gives an error if it can’t be found. By the looks of it, many users seem to want to use ResolveRef as a way to retrieve a list of results. While this goes against the original spirit of ResolveRef being a resolver for an [almost] unique identifier for journal articles (akin to Noel’s OpenRef proposal), I may be tempted to update ResolveRef to return a list of hits in the future (or just forward to the HubMed or PubMed results page).

There are also some actual bugs which throw nasty python backtraces (I think this one was actually me trying to use ResolveRef to look up a reference at work ):

/ref/Protein%20Sci/1999/8/689

This threw an error since ResolveRef (stupidly) assumed that every PubMed record has an associated DOI … however for some reason this Protein Science article does not have a DOI recorded in PubMed, so it fails to resolve with ResolveRef. This is (yet another) drawback to using PubMed as a backend. I’m thinking I may need to make ResolveRef interface with CrossRef somehow too, since that may act as a backup (or complete replacement) for these cases.

There also seem to be occasional errors generated when the HTTP connection from the Google App servers to PubMed fails; my fault entirely … that type of exception should always be anticipated and caught in a networked application.

Apart from guessing how people may like to use the application by examining the logs, edoardo.marcora also suggested that autocomplete/suggest for the journal field would be nice. I agree … this was a feature I was working on prior to the initial release, but it was taking too long so I just launched ResolveRef without it.

There is a new version in the pipeline, and will be ready for release soon. I’ll also put it on Google Code, warts and all. I already have the “suggest” functionality working, and once I resolve the few bugs discussed above, I’ll push out an update. Stay tuned.

(Yes, this service is called Dapper, sharing a name with the popular Ubuntu GNU/Linux release. I’m a little suspicious that maybe this was a deliberate marketing trick to pull search traffic intended for Ubuntu ….).

Techcrunch describes Dapper well … ‘create an API for any site’. Essentially, Dapper is good at analyzing web pages which have a fixed format (eg Google search results) and will extract the content in a predictable fashion to provide the raw data in XML, RSS, JSON or CSV format. Depending on the type of data you extract from the page, Dapper can also display data as a Google Map, a Google Gadget or Netvibes module, send email alerts or output iCalendar format.

By making it simple to extract information from sites that do not provide their data as an RSS feed or other useful format, it is possible for non-programmers to use Dapper and ‘liberate’ these sites, producing feeds compatible with their favorite news feed reader from an otherwise dead and lifeless web page. Web application programmers can waste less time writing screen scrapers, and waste less time fixing broken code caused by slight changes in the page format. It may not be something you use for a mission critical component, but for a quick mashup or to get started quickly before writing your own more robust parser, I think Dapper will prove (and is proving) very valuable.

However, there is a downside. From playing with Dapper on-and-off for the past few months, I’ve established that Dapper works quite well extracting data from very uniform pages like Google search hits [however I didn't do that since it's against the terms of service. "No screen-scraping for you", says the Google-Nazi] or the front page of digg .. but usually fails on pages that don’t follow a strict pattern. Getting the wrong data (or junk) in the wrong field 5% of the time may be tolerable for the occasional frivolous RSS feed, but it is annoying enough that for more important applications it is a real show stopper.

One of the reasons it took me several months to get around to posting about Dapper is that I desperately wanted a killer example of extracting data from a bioinformatics database or web site. I’ve found that most decent projects already make their data available in some useful format like XML or CSV and don’t really require scraping with Dapper, while some of the less organized projects which only provide say, HTML tables, [I won't name names ... in fact I've forgotten about them already ... "no citation for you" says the citation-Soup-Nazi ] often failed to work well with Dapper’s page analysis unless the page formatting was strictly uniform.

Pedro Beltrão has given an example of using openKapow as a scraper for bioinformatics … Dapper and openKapow seem to be competing in the same space, however Dapper is entirely web-based and openKapow requires some free desktop software. I haven’t tried openKapow yet, but Techcrunch’s rundown on several scraping services lists the use case of Dapper as “Quickly Scrape Data” and openKapow as “Robustly Scrape Data”, which is telling.

I replicated Pedro’s openKapow Ensembl orthologue search in Dapper as an example. It’s not the best example since, as Pedro notes, Ensembl is one of the ‘good guys’ that already provide results in XML format.

First, I fed Dapper four URLs for Ensembl gene report pages , which contain a section with predicted orthologues. Apparently, giving Dapper several pages of the same format helps the analysis:

Then, I selected the Gene ID in the orthologues list .. Dapper colours fields it detects as the same type. There is a cryptic unlabeled slider which determines the ‘greediness’ of the selection:

After selecting “Save and continue”, Dapper asks for the newly defined field to be named. In this case, I chose the same name as Pedro (”ort_geneID”), just for the hell of it:

This process was repeated to create a field for the species name, which I named “ort_spp”. Dapper allows ‘Fields’ to be grouped into ‘Groups’, so I grouped the “ort_geneID” and “ort_spp” fields into a group called “orthologue”: (data not shown ).

Now, we save the Dapp. In “Advanced Options”, I changed the Ensembl gene ID part of the URL to {geneID}. This tells Dapper to make this part of the URL a query field, so that the user can provide any gene ID they like and have the orthologue results scraped:

Finally, we can test the saved Dapp, and retrieve XML formatted results for a particular gene ID:

The gene ID can be changed in the Dapper XML transform URL (http://www.dapper.net/RunDapp?dappName=EnsemblOrthologues&v=1&
variableArg_0=ENSG00000100347) to get XML results for orthologues of other human genes.
Various other transforms, like a cruft-free CSV version are also possible. Feel free to have a play yourself with my Ensembl Orthologues Dapp (like I can stop you now ! It’s public & live & irrevocable).

The programme (Career Development Session, Omics Session / Systems Biology, Scientific Communication 2.0 and Participant’s Contributions) and speakers list makes it look sort of like a “Biology 2.0″ conference.

Apart from the (possible) IRC sessions, hopefully the fact that everything is stored as video/audio + comments on their content managment system means the ‘inconvenient’ timezone in Australia won’t limit my participation too much.

(via the worldwide bioinformatics cabal , Neil via Pedro, Roland and Stew)