liftOver QA

• Quick version of testing protocol.
• Last updated: May 11, 2006
• NOTE: If this is a new assembly, make sure that there is a liftOver file from the old assembly to this one (#1 requested liftOver file after a new release).

Overview

hgwdev set up

1. Symbolic link to /hive/data/genomes/$db1/bed/liftOver/ in hgdownload area:
   
   /usr/local/apache/htdocs/goldenPath/$db1/liftOver/${db1}To${Db2}.over.chain.gz
   
   If file/link is named incorrectly, rename it.
   If file/link is missing, check in /usr/local/apache/htdocs/goldenPath/$db1/vs$Db2 for a file named like $db1.$db2.over.chain.gz. Move this file/link to correct location. Do not just copy or the RR will end up with two copies of the file. If file/link is not compressed, compress.
2. Symbolic link to /hive/data/genomes/$db1/bed/liftOver/ in /gbdb area:
   
   /gbdb/$db1/liftOver/${db1}To${Db2}.over.chain.gz
   
   If this is a real uncompressed file, find the copy of the file on the /hive/data/genomes filesystem and create a link to it, then delete the file. If the file does not exist on /hive/data/genomes, move it there.
   If this is a real compressed file, find the copy of the file on the /hive/data/genomes filesystem and create a link to it, then delete the file. If the file does not exist on /hive/data/genomes, move it there. Compress the version on /hive/data/genomes.
   
   If link is named incorrectly, rename it.
   If link is missing, add it using the same path as for hgdownloads into the /cluster/home area or to the hgdownloads file itself.
3. Metadata present in hgcentraltest:
   

   hgsql -Ne 'SELECT * FROM liftOverChain WHERE fromDb = "$db1" AND toDb = "$db2"' hgcentraltest
   
   If this is missing, copy an existing line from the table into a file. Edit the file to represent the new liftOver data. Upload the new line into the table.
   Confirm that the toDb, fromDb, and path are correct.
   Confirm that the defaults are correct. same species: multiple = N, cross species:multiple = Y

test on hgwdev

1. Test hgLiftOver

Link to tool: "Main menu", side blue bar "Utilities", first page link "Batch Conversion".

Configure pull down menus to match the toDB and fromDb being tested.

Test lifts using both positional and bed format.
Both will work for same species with defaults.
Many positional searches will fail for cross species (where multiple = N).
Most bed search will pass for cross species (where multiple = Y). If they fail, lower the minMatch (as low as 0.01) until some result for some region is obtained successfully. Compare the results to the net and chain tracks. Should be exactly the same coordinates results.


2. Test hgConvert

Link to tool: From inside of a genome's database browser, top blue bar link "Convert".

Center the "fromDb" browser window on genomic region of interest.

Click on "Convert" link. Configure pull down menu to point to the "toDb" being tested. Submit.

Test same lifts as for hgLiftOver.
Results that are present in hgConvert will be identical to those from hgLiftOver for the majority of cases, especially if hgLiftOver has minMatch = 0.01.
Results that are present in both hgLiftOver and hgConvert will always match exactly at the coordinate level and will also match the net and chain tracks exactly.
Results that are present in hgConvert but that are NOT in hgLiftOver will still match the net and chains exactly. If not, this might be worth confirming with Jim until we understand the hgConvert program better [edit this later].
Results that are present in hgLiftOver but that are NOT in hgConvert are a problem in all cases and should be reported to the developer and possibly Jim.

push and test on hgwbeta

1. Push /gbdb file from hgwdev to hgnfs1 via a push-request.
2. Copy metadata line from hgcentraltest.liftOverChain to hgcentralbeta.liftOverChain.

   hgsql -Ne 'SELECT * FROM liftOverChain WHERE fromDb = "$db1" AND toDb = "$db2"' hgcentraltest > liftOver.metadata

   hgsql -h hgwbeta -e "LOAD DATA LOCAL INFILE 'liftOver.metadata' INTO TABLE liftOverChain" hgcentralbeta
   
   

3. Test hgLiftOver and hgConvert on hgwbeta using same test cases as on hgwdev. Results should be identical.

push and test on RR

1. After push is complete and hgcentral.dbDb.active = 1, copy metadata line from hgcentralbeta.liftOverChain to hgcentral.liftOverChain.

hgsql -h genome-centdb -e "LOAD DATA LOCAL INFILE 'liftOver.metadata' INTO TABLE liftOverChain" hgcentral


2. Test hgLiftOver and hgConvert on RR using same test cases as on hgwdev and hgwbeta. Results should be identical.

push downloads

1. Check for pairwise alignment files and ask for a push of
   /usr/local/apache/htdocs/goldenPath/$db1/vs$Db2 (link)
   to hgdownload (actual file).
   If there is no md5sum.txt file in the directory, make it and push it.
2. Ask for a push of liftOver downloads files from
   /usr/local/apache/htdocs/goldenPath/$db1/liftOver (link)
   to hgdownload (actual file).
   If there is no md5sum.txt file in the directory, make it and push it.