We’ve just updated the Pfam website again. This update comes fairly soon after the major, Pfam 24.0 release and it’s intended to fix some of the more annoying bugs and omissions that we’ve found in the last week or so.
There are various small changes and fixes all over the site, but there are several more significant changes that you might need to be aware of.

Documentation

We’ve started the painful process of updating our documentation for the HMMER3-derived data. You’ll find that most of the tabs in the help page are now up to date, though some do still carry a warning about their content. We’ll be working through these remaining sections and will update them as soon as possible.

Domain architectures search

We now have the domain architectures search working again. The submission form has been tidied up a little and we use the new domain graphics library to render the architecture graphics but, beyond that, the search should work as it did before.

Sequence search restrictions

The previous version of HMMER, v2, accepted “-” as a valid sequence character, but HMMER3 considers that to be an invalid character and returns an error. In the initial 24.0 website release, you could submit a search sequence with “-” and the search would fail with an unhelpful message. With this update, the validation procedure on the website now catches “-” before submission and tells you where the invalid character was found.

GI numbers

Putting a GI number into a jump box now sends you to a page about that NCBI sequence entry, rather than returning errors. The NCBI pages are still “under construction”, but they’re better than the gaping hole that previously existed in the site !

RESTful services

We’ve reinstated the “RESTful” services for the major parts of the site. You should now be able to use the API to get data about Pfam-A families and individual proteins, and, probably after some changes to your code, to submit single-sequence searches again.

The switch to HMMER3 has changed quite a few aspects of the data, as well as the way we run our searches, so there are changes to most of the XML schemas, mostly fairly minor. The documentation on the RESTful interface is now up to date, so check there for information on the new XML formats.

Sequence searches

It’s probably worth giving a little more detail on the changes that we’ve had to make to the RESTful interface to the sequence search system. If you’ve previously used the search interface, you will probably need to update your scripts accordingly.

No “estimated time”

In Pfam 23.0,  searches used HMMER2 and could take on the order of a minute for a Pfam-A search, so we used a “polling page” to give the user something to look at while they waited. It showed a progress bar and gave the user an idea of the estimated run time for the search. Now that we’re using HMMER3, most Pfam-A searches are so fast that it took longer to load the polling page than to run the search. Now we don’t bother calculating an estimated run time and we’ve ditched that intermediate page altogether. Results are now loaded into the results page as they appear on the search system.

When running a search using the API, you can simply add a short delay (a couple of seconds should be fine for most sequences) between submitting the search and retrieving the results. If your search is still running when you try to retrieve the results, you just won’t get results; wait a little longer and hit the same URL again.

When running a search against Pfam 23.0, the procedure was to submit the search and check the estimated run time in the XML that came back. After waiting for that period, you would then retrieve results from a URL in the XML.

Only one job ID

In Pfam 23.0, if you chose to run both a Pfam-A and a Pfam-B search, you would get two job identifiers and you would have to retrieve results for each search separately. Because we use HMMER3 to search for both Pfam-A and Pfam-B matches now, we run the jobs in the same queue, so there’s only a single job ID. Furthermore, the Pfam-A and Pfam-B hits are returned in the same result XML document. You can distinguish them using the “type” attribute on the match element.

Summary

There have been a lots of small changes and a few larger ones in this update, but hopefully nothing too disruptive. If you have any problems using any of the newly added or recently repaired features, do let us know.

Posted by John.

We have just released the latest update to Pfam. Release 24.0 contains a total of 11,912 families, with 1,808 new families and 236 families killed since the last release. 75.15% of all proteins in Pfamseq contain a match to at least one Pfam domain. 53.18% of all residues in the sequence database fall within Pfam domains.

As we’ve discussed previously, release 24.0 is the first to be generated using HMMER3. This migration has necessitated changes in our file formats and users should be aware that HMMER3 is not backward compatible with HMMER2. For a list of changes, please check the documentation on the Pfam website.

The new release is currently available from the Pfam site at the Sanger Institute and will be available from the other Pfam sites shortly. This release has been a massive undertaking for us and, although most features of the website are working well, some details remain to be addressed, notably the programmatic interface to the sequence search system. Please bear with us while we bring these back over the next few weeks.

Update: we’ve had one or two users reporting that the tabbed pages in the site (e.g. help, family, etc.) are broken. Although that’s eminently possible, if you see a broken page, please try doing a shift+reload before anything else. That should force your browser to download the latest versions of all of the javascript files, and hopefully fix the problem there and then.

We are now on the brink of releasing Pfam 24.0.  This release of Pfam, version 24.0, will be a landmark release as it will be the first to be built using the the new version of the HMMER package, HMMER3. We are well aware that we have been claiming this release as imminent for some time, but we are now at the point of flicking the big switch.  There are numerous changes that users need to know about and we will briefly summarise them here.

HMMER version change

The new version of HMMER is considerably faster and more sensitive than previous ones.  This has resulted in numerous families being merged and a significantly larger number of families being classified into clans (as we are now better able to detect similarities between families).  All families have had their significance thresholds (also known as GA or gathering thresholds) re-set, because of the new statistics underlying the HMMER3 implementation.   Thus, it is NOT possible to produce both a HMMER2 and HMMER3 set of HMMs with PFam defined cut-offs.  In addition, we only need a single HMM per family, so the previous ‘Pfam_ls’ and ‘Pfam_fs’ files are now replaced by a single file called ‘Pfam-A.hmm’.  As well as the Pfam-A.hmms, we will also be making a HMM library for the 20,000 largest Pfam-B families (‘Pfam-B.hmm’).  Some of you are already using pre-released versions of these HMMER3 HMM libraries as part of the new version of “pfam_scan.pl”.

Although we will not bother you with the details now, the change in HMMER version has also made us re-think many of our policies, file formats and even the philosophy behind the database.  We will try to address the most significant of these in our documentation and, of course, in this blog.

Sequence Database

In addition to migrating Pfam from HMMER2 to HMMER3, we have also updated the underlying protein sequence database.  Pfam 24.0 is based on UniProtKB release 15.6, dating from 28th July 2009, and this update has resulted in a near doubling of our underlying sequence database.  This has obviously changed the character of several families. For example, GP120 is no longer our largest Pfam family, having dropped to being the 3rd largest family, behind ABC_tran and RVT_1, with the latter two families each exceeding 100,000 sequence matches!

Relational Database

If you are not concerned about the MySQL database, then skip to the next section.  If you are, then you need to know that, along with all the other changes, we have also changed the type of table-engine used, from MyISAM to InnoDB.  There are both technical and managerial reasons for having done this, but one of the main benefits to the user is that InnoDB supports foreign key relationships. This improves data consistency and allows the tracing of table relationships.  However, this technical change, along with the data changes, has resulted in many changes to the database schema, including subtle column-name changes and more obvious data-type changes.

New Families

No Pfam release would be complete without the addition of new families.  We have had the obligatory summer vacation student here adding families this year.  As a result of his efforts, and our day-to-day work, there have been 1808 new families added to the database since 23.0. Release 24.0 will contain 11,912 families!!!  Want to know the coverage?…..well you will have to wait for that!

Website Changes

There is good news if you only use the Pfam website, since not a great deal has changed here. You will notice many small changes around the site, as well as a few larger ones, such as the new domain graphics.  There have been a few usability improvements, notably in the behaviour of the tabs and their interaction with your browser’s “back” button.  You should now find that pressing “back” will take you to the last tab that you viewed, rather than to the last complete web page.  This might take some getting used to, but we hope that it will prove more intuitive than the original mechanism.  You should also be able to bookmark any tab in the site directly, which was not previously easy to do.

Sequence searches

One of the most exciting changes to the website is the massive increase in the speed of interactive, single-sequence searches.  Because searches used to take upwards of thirty seconds to complete, we used to show a page with a progress bar and a status message telling you how your search was doing. Thanks to the dramatic speed increases in HMMER3, we’ve been able to do away with the progress bar and we now load the results page directly.  For all but the largest sequences, you should see your search results loading within just a few seconds.

Domain graphics

Visually, probably the most significant difference in the site is the new style domain graphics.  Our domain graphics are pictorial representations of sequences and the Pfam matches to them.  In the existing Pfam 23.0 site, every domain graphic is generated on our server as an image file, using the Perl GD module.  Each image is then served separately by our servers.  Obviously, when viewing many sequences in one page, your browser might have to make hundreds of requests to our server and, during periods of heavy load, this severely reduces the performance of the website.  Also, in the past we have had periods during which we were generating these temporary domain graphic images faster than we could delete them!  To circumvent these issues, we have completely changed the way that the domain graphics are drawn.  In the new site, all of the domain graphics in a page are described by a single chunk of data and then drawn in your browser using a javascript library.  This should make the page load more quickly and it allows us to add more information to the domain graphics; you should see new tooltips as you move your mouse over the images, giving information about each domain.

RESTful interface

One part of the site that has changed noticeably (and which will be broken in places at first) is the so-called “RESTful interface”.  If you’re not familiar with the RESTful interface, it’s essentially a programmatic interface to some parts of the Pfam website, such as family pages or the sequence search system.  Right now it’s looking as if the RESTful interface for much of the site will be released after the initial 24.0 data release, because we still have a lot to do there.  However, because the single-sequence search mechanism has changed so much, the RESTful interface to that sub-system will change as soon as the site is released.  Most importantly, there are some changes to the XML that is returned by this interface, so, if you use it, you will almost certainly need to update your scripts.  We’ll do our best to document all of the changes in time for the release.

In Summary

If you are waiting for Pfam 24.0, it will not be long now… And if you rely on Pfam as a source of data, then start thinking about scheduling time in October to go and revisit any code that uses it.

Back in May we wrote a blog post about the new version of pfam_scan.pl. We asked if there was anyone out there who was willing to help us test our new script, and we were pleasantly surprised at the number of people who got in contact with us – so a big thank you to all those who have helped. Since releasing the alpha version of pfam_scan.pl to our testers we have made some internal changes to the script that are worth mentioning:

Searching Pfam-B HMMs

We are frequently asked how to search sequences against Pfam-B families, and now we are providing this facility as part of pfam_scan.pl. Pfam-A families are the main entry point for scientists in their day-to-day use of Pfam. The alignments for each Pfam-A family have been carefully checked by one of our curators, and each one is accompanied by annotation. In the past we have made a library of all the curated entries in the database (the Pfam-A HMMs) available for users to search against.

Within Pfam, we have a second set of families  called Pfam-B families. These families are automatically generated alignments that have no accompanying HMMs or annotation, and their alignments have not passed any of the normal quality control that we would perform for a Pfam-A family. This means Pfam-B families are of much lower quality than our Pfam-A families, however they do help us to fill the sequence space that Pfam-A families do not cover.

In the last release there were 223,403 Pfam-B entries, with the number of sequences matching each Pfam-B entry following a long-tailed distribution. With HMMER3 being significantly faster than HMMER2, we have chosen to generate HMMs for the top 20,000 (largest automatic families) Pfam-B families, as these are the most relevant. Most entries below this cut-off contain less than 10 sequences. As multi-threaded versions of HMMER3 become available we may include more entires, but for the moment the search times against the Pfam-A and Pfam-B libraries are equivalent.

BUT….please remember that Pfam-B accessions are not stable between Pfam releases, so do not rely on them!

Additional formats

A further change adds the ability to write search results in JavaScript Object Notation (JSON). JSON is a compact, text-based data format, which is most commonly used in the context of the web and javascript applications. However, precisely because it’s so compact and portable, JSON can also be useful as a sort of lightweight XML replacement. Hopefully the JSON output options of pfam_scan.pl will be useful for those users who would otherwise have to parse the raw text output in order to do more processing.

We wanted the new version of pfam_scan.pl to be fast, but also much more maintainable than the previous version. When we started work on the new pfam_scan.pl, we made the decision to use a number of third party Perl modules from CPAN. The most important of these is definitely the Moose framework For those of you who are unfamiliar with Moose, it’s a complete object system that improves on the Perl 5 object system and makes object-oriented code simpler and more powerful.

A word or two about Moose

Some our testers noticed that using Moose contributed fairly significantly to the runtime of the script (excluding running HMMER3). The performance overhead of a system like Moose is definitely something that needs to be considered. After careful consideration, we decided that it was a price worth paying, but we also thought that it was worth explaining our reasoning a little.

One of the benefits of Moose is that Moose-based objects can be configured to perform extensive data validation, in a way that is both easy to implement and maintain. The Pfam production pipeline has been entirely re-written for the upcoming Pfam release, and it now relies heavily on Moose for data validation. Since the pipeline modules perform many of the operations that we need in pfam_scan.pl, such as parsing the output of HMMER3 programs, we’ve been able to use these new modules in the script, rather than having to rewrite the same functionality from scratch.

The validation checks that can be built into Moose objects avoid the need to write complex data validation procedures for ourselves. The code that implements the checks is part of the core Perl modules and is therefore maintained by the maintainers of the CPAN modules, not us. Our new production code was easier to write because it relies on Moose and the built-in validation checks are much tighter than they would otherwise have been, despite the fact that we’ve written less code overall. By using the same modules in pfam_scan.pl, we’ve further reduced the amount of code that we need to maintain overall and have significantly speeded up the development of pfam_scan.pl itself.

In short, we’re happy that we’ve struck an appropriate balance between all out performance and long term maintainability. We’ll keep up to date with changes to the Moose framework that might help improve performance, and we’ve already improved the performance of the production pipelines by leveraging some of the more esoteric Moose features. The final release version of the modules and scripts will include these under-the-hood tweaks.

And Finally…..

Once again, thank you if you tested out the alpha release of the HMMER3-enabled version of pfam_scan.pl.  Your feedback should reduce our pain, and the pain of the community, when it’s bundled with Pfam release 24.0. For those of you who can not wait, you can find the beta release of the script on our (well Rob’s) ftp site. Of course if you find any bugs, or have any feedback, please contact us.

Update 16/10/09

We released the first official release of pfam_scan.pl with Pfam 24.0, and it’s available for download (the tarball is called PfamScan.tar.gz) at ftp://ftp.sanger.ac.uk/pub/databases/Pfam/Tools/.

Domains of Unknown Function, or DUFs, is a large set of families found in the Pfam database. Examples would be “DUF26” or “DUF282“. The DUF naming scheme was introduced by Chris Ponting, through the addition of DUF1 and DUF2 to the SMART database. These two domains were found to be widely distributed in bacterial signalling proteins. Subsequently, the functions of these domains were identified and they have since been renamed as the GGDEF and EAL domains respectively (structures shown in Figures 1 and 2). These families were added to Pfam in 1997, and little did Chris know that he was starting a trend that would see thousands of uncharacterised families being added to the domain databases.

Structure of the GGDEF domain (in green), formerly known as DUF1, now known to function as a diguanylate cyclase enzyme. Structure of the EAL domain (in green), formerly known as DUF2, now known to function as a cyclic diguanylate-specific phosphodiesterase enzyme.

Structure of the EAL domain (in green), formerly known as DUF2, now known to function as a cyclic diguanylate-specific phosphodiesterase enzyme.

At least in Britain, the word “duff” conjures up something substandard, with the dictionary definition stating:

duff adj. Brit. slang 1. worthless 2. useless.

However, in reality, DUFs are treated with the same loving care as all other Pfam families. The only difference is that our curators are unable to identify any functional information from the scientific literature.

In Pfam release 23, the DUF number scheme has reached DUF2607, and the fraction of DUF families in Pfam has increased to about 22% of all families (shown in Figure 3).

Growth of DUFs in Pfam

It looks as though the number of DUFs is on the increase. Because DUFs require little annotation, they are often easy families to add to Pfam. We expect that soon the number of DUFs will outnumber the families of known function being added to Pfam.

Identifying functions for Domains of Unknown Function is extremely important if we are to understand biology at a systems level. Essentially there are two ways to find out the function of an uncharacterised domain: the first involves identifying a similarity to a domain of known function, either by sequence comparison or perhaps from a newly solved structure; the second way is good old fashioned molecular biology. Sir Rich Roberts put forward a proposal to stimulate experimentation on uncharacterized proteins [1].

Slowly, momentum is being gained and more functions of DUFs are being identified. Since we started adding DUFs nearly 10 years ago, over 270 of them have been renamed presumably when a function had been identified. Our curators have not yet had time to recheck all of the existing 2,000 or so DUFs to see if new functional information has been identified. Therefore, over the coming year, we hope to recheck all of them, and rename and re-annotate those where function is now known. If you know of any recently identified functions for these families, please do let us know.

In recent years, structural genomics initiatives have solved the structures of literally hundreds of proteins in uncharacterised families. In many cases, this has helped to narrow down the possible function of a family. For example, DUF442 was shown to be a non-classical phosphatase enzyme, see Figure 4 [2].

Active site of the DUF442 phosphatase enzyme in sequence and structure (From Krishna et al)

The DUFs remain a treasure trove of novel biology waiting to be plundered. So, why not get that pioneer spirit and join the gold rush!

Posted by Alex.

References

[1] R.J. Roberts (2004): Identifying protein function – A call for community action.
Plos. Biol. 2:e42.

[2] S.S. Krishna et al. (2007) Crystal structure of NMA1982 from Neisseria meningitidis at 1.5 Å resolution provides a structural scaffold for nonclassical, eukaryotic-like phosphatases. Proteins. 69:415-421

It has been a little quiet on the Pfam blog recently, but behind the scenes we’ve been working hard on the migration to HMMER3.

We have built HMMER3 models for all of the Pfam alignments, and searched them against the sequence database. This part was super quick, as HMMER3 is ~100 times faster than HMMER2. Due to the increased sensitivity of HMMER3, many of our Pfam families have grown in size, and we have found that ~80,000 sequences in the sequence database now have overlapping matches to more than one Pfam family.

Within Pfam we have a rule that states that our families should not overlap; this means that any one amino acid can belong to only a single Pfam family.  The exception to this rule applies to families within a clan – clans are Pfam’s collections of related families – where overlaps between clan members are allowed. Over the last few weeks we’ve been working through and resolving the list of 80,000 overlaps.

There are several methods we use for resolving overlaps.  Where families are related, we put them into the same clan, or merge them together if similarity is very high; sometimes families overlap by a few residues and here we trim the domain boundaries such that the two families no longer overlap; we also have cases where we think the sequence(s) that have the overlap are false positives in one or other of the families, and in these cases we raise the threshold in that family such that that sequence is excluded. Thus maintaining the high quality of Pfam data.

The overlaps generated by HMMER3 have allowed us to find new relationships between families, and to confirm relationships that we had an inkling about.  We’re using PRC,  SCOOP and structural data along with the HMMER3 overlap data to decide whether to put families into the same clan.  In addition to adding many families to existing Pfam clans, so far we have created approximately 60 new clans.

Overlap resolution has been quite a lengthy process, and we’ve still got ~10,000 to go. However, we are hoping that because more families are now in clans, we will have added a great deal of value to the Pfam database through indicating which families are related. Resolution of overlaps is undertaken, admittedly on a smaller scale, every time we update the sequence database for a new Pfam release. Future releases should benefit from the improved clan-infrastructure in terms of overlap resolution, and with each release we will hope to improve this even further.

We’ll keep you posted about progress towards the next release of Pfam (version 24.0).

I thought it would be useful to give a quick overview of some of the major things that have been going on behind the scenes at Pfam during 2008. Overall it may have seemed like a quiet year for our users as we only made one public release of data in July, release 23.0. However, like a paddling duck, the calmness viewed from above belies some furious paddling below.

In total 1326 new families were added to our curation database. These were derived from a number of different sources such as Pfam-B, our automatically generated clustering of sequence regions not already in Pfam. As well as adding new families, we also made a valiant attempt to go through our existing families and improve their scope through a process we call iteration.

Iteration takes the full sequence alignment for a family and attempts to make a new non-redundant seed alignment from it. When a new HMM is made from this seed alignment, we hope to find new even more distant homologues for the family. During 2008 we iterated every single family that was not in a Pfam clan. For about 50% of Pfam families we were able to find more homologues. While in most cases there were only modest improvements, in some cases we found many hundreds of new family members.  In some cases this process allowed us to realise that two families were actually related and merge them into a single entry. For example we found that the uncharacterised family DUF30 (accession PF01727) was actually related to Peptidase_S7 (PF00949).

We changed the underlying source of protein clusters used in Pfam-B, from PRODOM to the PairsDB, from Andreas Heger and Liisa Holm. Unfortunately, the PRODOM database was not able to keep up a frequent release schedule and has become somewhat out of date. The greatly improved coverage of Pfam-B has increased the overall comprehensiveness of Pfam.

The rapid growth in the number of protein sequences has meant that we have had to revisit many aspects of the software used to run Pfam. Large portions of the codebase were rewritten, in order to help us scale with the sequence deluge. In 2008 we started providing Pfam match data on NCBI GenPept and Metagenomics, in addition to UniProt sequences.  Thus, from just over 3 million UniProt sequences in 2007, this expanded to 17 million sequences. This sequence growth, combined with the increased number of families, has resulted in a dramatic increase in the amount of computer power required to produce a Pfam release. In order to produce a release within such a timescale as not to make it obsolete, our release pipeline was restructured from a linear procedure to one that is largely parallel. Pfam 23,0 release took about two months to produce, but consumed approximately 60 CPU years on the Sanger compute cluster. As part of this restructuring, we have also endeavoured to increase the level of quality control. The primary focus here was to ensure data consistency between the data stored on disk (that is used to represent a family during the curation process) and the information populated in the MySQL database (that is used to provide the website).

As the first post suggested, this blog will partly describe the progress and issues faced with the migration of Pfam to HMMER3.  We’ve been waiting for the mercurial HMMER3 for well over a year now, watching all the while its ever receding release date.  However, it has finally been released, albeit in alpha phase! Given Sean Eddy’s past record on HMMER2, particularly his attention to detail and his hatred of bugs in his software, we (Pfam) are already confident enough to be looking at migrating to HMMER3. This post will set out the rationale for moving Pfam to HMMER3 quickly and look at some of the issues that will inevitably follow such a move.

Speed!

First and foremost, we’re concerned about compute time: the last Pfam release, Pfam 23.0, took nearly 60 CPU years to produce, most of that time spent running hmmsearch against the various sequence databases (UniProt, NCBI genpept, metagenomic sequences). For us especially, the 100-fold speed increase promised by HMMER3 makes it well worth exploring ! Additionally, when it comes to searching Pfam with HMMER3, we can immediately cut the search time in half, since there will be only one HMM per family. Currently, we produce two HMMs per family, in order to ensure that the family is as comprehensive as possible, which attentive Pfam users will recognise as Pfam_ls and Pfam_fs. The ls models (“glocal”) find sequence matches that corresponded to the whole length of the HMM. The fs models (local) find partial matches with respect to the model; more often than not, partial matches would be shorter than the actual domains in the sequence. However, we understand that, with HMMER3, many of the multi-hit local alignment issues have been resolved, and partial matches can now be more readily extended to match the full length of the model, where appropriate. Consequently, with HMMER3 we will only need  one HMM per family (“local”) and this will remove a heck of a lot of confusion as to how a match was found.

Sensitivity

The second reason for migrating to HMMER3 is that it is much more sensitive than HMMER2, which is a very good thing! We now have well over 10,000 families in Pfam and many of these families belong to Pfam clans. Clans are collections of related families and are required to cover the situation where one HMM is insufficiently sensitive to find all members of the family. Furthermore, we tend to set rather conservative thresholds when we curate a family, in order to avoid the inclusion of false positives, but we do this even at the expense of missing real matches. Our hope is that the new HMMER3 software will enable our large, divergent families to become even larger and that we will be able to collect all known members of a family, without the inclusion of false negatives. Cases of wine have been bet on the increased sensitivity, and it’s about time Sean actually won one of these bets!

Issues

Personal experience with HMMER3 gives us confidence that switching will be ultimately beneficial to Pfam. Having played with both a pre-alpha release and the alpha release version of HMMER3, it certainly appears to be living up to expectation. There are still some issues with HMMER3, such as missing a few true positive matches that HMMER2 found (from what I can tell, these are due to the filtering-heuristics/bias composition) and HMMER3 failing miserably on a handful of very short (less than 10 amino acid) tandem repeats.  Despite these problems, HMMER3 is, overall, a great improvement, and we are now seriously looking at migrating Pfam to use this new version of HMMER.

What are the issues for Pfam users? Well, the good news is that if you use Pfam mainly via the website, there should be little change. Hopefully, you’ll find that sequence searches will run even faster and, once we’re fully operational with HMMER3, we hope to be able to provide some additional services/features that take advantage of the new features in HMMER3.

For those using the Pfam data files rather than the website, we don’t expect a huge number of changes in the Pfam-A.full and Pfam-A.seed flat files. However, if you use the Pfam HMMs then most stuff is (initially) going to break. There is no way to produce parallel sets of HMMs using both HMMER2 and HMMER3, so when we finally make the change, it is going to be fairly drastic!

Conclusions

We still have a lot of work to do before we can release a version of Pfam based on HMMER3. Every Pfam family will need to have its curated threshold reset, a process that is likely to be time-consuming, and we need to re-write large sections of our production pipelines to deal with the changes that HMMER3 requires. As we face issues and/or make changes, we intend to post to this blog.

These are exciting times, but the changes we all face (both the Pfam group and our user community) will undoubtedly cause some pain. As we get closer to a final switch over to HMMER3, we’ll use this blog to tell users more about the process, and hopefully as a way to get feedback from Pfam users and to find out their hopes and fears for HMMER3 ! Ultimately, we will produce a signifcantly more useful version of Pfam, where, for instance, a bacterial genome can be comfortably searched on a basic laptop within minutes, but it will be an interesting journey.

Welcome to the new blog for the Xfam databases ! Xfam is our shorthand for the combination of Pfam and Rfam databases, which we note will also future-proof us, in case we add any further databases to the brand.

We hope that this blog will become a useful point of reference, where our users can learn about what is going on behind the scenes at Xfam central. We will be announcing some important changes that are coming with the eagerly awaited release of HMMER 3. As well as announcing new releases of the data and website, we’ll also try to discuss our philosophy on protein/RNA domains and sequence classification. If there are other topics that you would like to hear more about, why not leave us a comment.