From larrym at soe.ucsc.edu Mon Jun 2 10:42:39 2008 From: larrym at soe.ucsc.edu (Larry Meyer) Date: Mon, 2 Jun 2008 10:42:39 -0700 Subject: [Genome-mirror] v183 Genome Browser Available Message-ID: <20080602174239.GA9069@soe.ucsc.edu> Hello, The v183 source is now available at: http://hgdownload.cse.ucsc.edu/admin/jksrc.zip or labeled with source number: http://hgdownload.cse.ucsc.edu/admin/jksrc.v183.zip Please note that the installation file list below has a new entry: /usr/local/apache/htdocs/js/* These new files should be copied into a directory called "js" in the apache DocumentRoot. For more details, see notes about DOCUMENTROOT in src/product/README.building.source cartDump cartReset das hgc hgBlat hgConvert hgCustom hgGateway hgGene hgGenome hgLiftOver hgNear hgPcr hgSession hgTables hgTracks hgTrackUi hgVisiGene mkEncodeFrameset hgEncodeDataVersions pbGateway pbGlobal pbTracks phyloGif /usr/local/apache/cgi-bin/all.joiner /usr/local/apache/cgi-bin/galaAvail.tab /usr/local/apache/cgi-bin/hgNearData/* /usr/local/apache/cgi-bin/hgGeneData/* /usr/local/apache/cgi-bin/hgcData/* /usr/local/apache/cgi-bin/hgCgiData/* /usr/local/apache/cgi-bin/loader/* /usr/local/apache/cgi-bin/visiGeneData/* /usr/local/apache/htdocs/js/* Contents of v183 may be seen at: http://genome-test.cse.ucsc.edu/builds/versions.html Please check both v183-preview and v183-final to see all the code and data changes in this release. Larry Meyer UCSC CBSE From davidell at gmx.de Tue Jun 3 07:27:37 2008 From: davidell at gmx.de (davidell at gmx.de) Date: Tue, 03 Jun 2008 16:27:37 +0200 Subject: [Genome-mirror] problem with getting html files for $WEBROOT directory Message-ID: <20080603142737.209050@gmx.net> Hi, I am following instructions on http://genome.ucsc.edu/admin/mirror.html (http://genomewiki.ucsc.edu/index.php/Browser_Installation) for installing a browser mirror on my ubuntu 8.04, but I have problems getting all the html files and text files (for the $WEBROOT directory) using rsync. Testing the rsync connection results in: $ rsync -navz --progress rsync://hgdownload.cse.ucsc.edu rsync: failed to connect to hgdownload.cse.ucsc.edu: Connection refused (111) rsync error: error in socket IO (code 10) at clientserver.c(105) [receiver=2.6.9] When I browse ftp://hgdownload.cse.ucsc.edu/ I find directories and files totally different from the installation page http://genome.ucsc.edu/admin/mirror.html, step 2 Is there another way for getting the files listed in step2, e.g. from the ftp server ftp://hgdownload.cse.ucsc.edu/ ? Is the instruction site up-to-date for installing a browser mirror? thanx for your help, best regards David -- GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen! Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf at gmx From larrym at soe.ucsc.edu Tue Jun 3 16:04:34 2008 From: larrym at soe.ucsc.edu (Larry Meyer) Date: Tue, 3 Jun 2008 16:04:34 -0700 Subject: [Genome-mirror] Updated v183 Genome Browser Available Message-ID: <20080603230434.GG1614@soe.ucsc.edu> Dear Mirrors, Today we fixed a bug in hgTracks that was causing display problems with wiggle tracks. The problem was that the track labels would creep up slightly with each new wiggle track. This caused the labels to start overwriting track data when the user was viewing many wiggle tracks. We have rebuilt the v183 binaries and made them available in the usual place. Updated v183 source is now available at: http://hgdownload.cse.ucsc.edu/admin/jksrc.zip We are sorry to release CGIs again so soon after the usual bi-weekly release, but the bug was sufficiently serious that we felt you would prefer to have the new version. We're sorry for any inconvenience this may have caused, Larry Meyer UCSC CBSE From larrym at soe.ucsc.edu Tue Jun 3 16:22:55 2008 From: larrym at soe.ucsc.edu (Larry Meyer) Date: Tue, 3 Jun 2008 16:22:55 -0700 Subject: [Genome-mirror] problem with getting html files for $WEBROOT directory In-Reply-To: <20080603142737.209050@gmx.net> References: <20080603142737.209050@gmx.net> Message-ID: <20080603232255.GI1614@soe.ucsc.edu> Hi David, It appears that you may have hit a momentary network problem either on your or our side. I suggest you retry the rsync. The documentation is correct; i.e. the correct procedure is to get the files via rsync, not ftp. Larry Meyer UCSC CBSE On Tue, Jun 03, 2008 at 04:27:37PM +0200, davidell at gmx.de wrote: > Hi, > > I am following instructions on > http://genome.ucsc.edu/admin/mirror.html > (http://genomewiki.ucsc.edu/index.php/Browser_Installation) > for installing a browser mirror on my ubuntu 8.04, but I > have problems getting all the html files and text files > (for the $WEBROOT directory) using rsync. > Testing the rsync connection results in: > > $ rsync -navz --progress rsync://hgdownload.cse.ucsc.edu > rsync: failed to connect to hgdownload.cse.ucsc.edu: Connection refused (111) > rsync error: error in socket IO (code 10) at clientserver.c(105) [receiver=2.6.9] > > When I browse ftp://hgdownload.cse.ucsc.edu/ I find directories and files > totally different from the installation page > http://genome.ucsc.edu/admin/mirror.html, step 2 > > Is there another way for getting the files listed in step2, e.g. from > the ftp server ftp://hgdownload.cse.ucsc.edu/ ? > Is the instruction site up-to-date for installing a browser mirror? > > thanx for your help, > best regards > David > -- > GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen! > Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf at gmx > _______________________________________________ > Genome-mirror mailing list > Genome-mirror at soe.ucsc.edu > http://www.soe.ucsc.edu/mailman/listinfo/genome-mirror From kayla at soe.ucsc.edu Wed Jun 4 10:01:55 2008 From: kayla at soe.ucsc.edu (Kayla Smith) Date: Wed, 04 Jun 2008 10:01:55 -0700 Subject: [Genome-mirror] release of new assembly: cavPor3 Message-ID: <4846CA83.10901@cse.ucsc.edu> Hello Mirror sites, We are planning to release a new browser for guinea pig, cavPor3, soon. Please be prepared to host the following data: cavPor3 mySQL tables: 30 G /gbdb/cavPor3/* files: 12 G We will also be releasing net and chain tracks to cavPor3 in the following assembly databases. Here are the approximate sizes of the netCavPor3 and %chainCavPor3% tables in each database: galGal3: 1.1 G monDom4: 9.3 G rn4: 3.3 G mm9: 3.1 G hg18: 5.6 G _______ total: ~22.4 G Please let us know if you have any questions or concerns: Kayla Smith UCSC Genome Bioinformatics Group From ann at soe.ucsc.edu Thu Jun 5 16:14:09 2008 From: ann at soe.ucsc.edu (Ann Zweig) Date: Thu, 05 Jun 2008 16:14:09 -0700 Subject: [Genome-mirror] releasing two new assemblies Message-ID: <48487341.40504@soe.ucsc.edu> Hello mirror sites, Early next week, we are intending to release the new lancelet (braFlo1) and lamprey (petMar1) assemblies. Please be prepared to host about 8 GB of new data. braFlo1 (lancelet): braFlo1 MySQL database 2.3 GB files in /gbdb/braFlo1/* 0.7 GB petMar1 (lamprey): petMar1 MySQL database 2.8 GB files in /gbdb/petMar1/* 0.8 GB Additionally there are net and chain tables from other organisms that will be released to the respective annotation databases. These tables are named netBraFlo1 & netPetMar1 and *chainBraFlo1 & *chainPetMar1*. The size of these tables (and their associated /gbdb/ files) is as follows: nets and chains to braFlo1: galGal3 88 MB hg18 160 MB mm9 129 MB ====================== total: 0.4 GB nets and chains to petMar1: hg18 253 MB mm9 209 MB oryLat1 221 MB galGal3 220 MB =============== total: 0.9 GB Regards, ---------- Ann Zweig UCSC Genome Bioinformatics Group http://genome.ucsc.edu From hooverdm at helix.nih.gov Mon Jun 16 08:13:15 2008 From: hooverdm at helix.nih.gov (David Hoover) Date: Mon, 16 Jun 2008 11:13:15 -0400 Subject: [Genome-mirror] Deflecting cgi-bin hits Message-ID: <4856830B.5050609@helix.nih.gov> Besides the robots.txt file, is there any other mechanism for throttling or deflecting the massive numbers of automated hits on dynamically generated pages from cgi-bin, either from robots or otherwise? David Hoover, Helix Systems, CIT/NIH From James.Zhu at ARS.USDA.GOV Mon Jun 16 07:35:10 2008 From: James.Zhu at ARS.USDA.GOV (Zhu, James) Date: Mon, 16 Jun 2008 10:35:10 -0400 Subject: [Genome-mirror] Mirror Procedure Message-ID: <6B4AF69268EFAE4C80A786BA99F7A66702BC60C1@MD-MAIL-01.ARSNET.ARS.USDA.GOV> Regarding to my earlier question, how to run this command line "mysql -youraccountoptions hgcentral < hgcentral.sql" to setup "hgcentral" tables? James From James.Zhu at ARS.USDA.GOV Mon Jun 16 06:58:48 2008 From: James.Zhu at ARS.USDA.GOV (Zhu, James) Date: Mon, 16 Jun 2008 09:58:48 -0400 Subject: [Genome-mirror] Mirror Procedures Message-ID: <6B4AF69268EFAE4C80A786BA99F7A66702BC604D@MD-MAIL-01.ARSNET.ARS.USDA.GOV> I have installed an UCSC mirror for two genomes according to the instruction. When I used a internet browser to go to localhost, the browser displayed the home page of UCSC Genome Bioinformatics as expected. I clicked on Genome Browser in order to browse the genomes. The internet browser display "Please set backupcentral options in the hg.conf file.". What should I do to set the options? James From hiram at soe.ucsc.edu Mon Jun 16 09:26:16 2008 From: hiram at soe.ucsc.edu (Hiram Clawson) Date: Mon, 16 Jun 2008 09:26:16 -0700 Subject: [Genome-mirror] Mirror Procedures In-Reply-To: <6B4AF69268EFAE4C80A786BA99F7A66702BC604D@MD-MAIL-01.ARSNET.ARS.USDA.GOV> References: <6B4AF69268EFAE4C80A786BA99F7A66702BC604D@MD-MAIL-01.ARSNET.ARS.USDA.GOV> Message-ID: <48569428.8020206@soe.ucsc.edu> Good Morning James: Please note the documentation in the source tree directory src/product/README.* e.g.: http://genome-test.cse.ucsc.edu/~kent/src/unzipped/product/README.mysql.setup --Hiram Zhu, James wrote: > I have installed an UCSC mirror for two genomes according to the > instruction. When I used a internet browser to go to localhost, the > browser displayed the home page of UCSC Genome Bioinformatics as > expected. I clicked on Genome Browser in order to browse the genomes. > The internet browser display "Please set backupcentral options in the > hg.conf file.". What should I do to set the options? > > > James From rhead at soe.ucsc.edu Mon Jun 16 10:41:47 2008 From: rhead at soe.ucsc.edu (Brooke Rhead) Date: Mon, 16 Jun 2008 10:41:47 -0700 Subject: [Genome-mirror] Mirror Procedure In-Reply-To: <6B4AF69268EFAE4C80A786BA99F7A66702BC60C1@MD-MAIL-01.ARSNET.ARS.USDA.GOV> References: <6B4AF69268EFAE4C80A786BA99F7A66702BC60C1@MD-MAIL-01.ARSNET.ARS.USDA.GOV> Message-ID: <4856A5DB.5080506@soe.ucsc.edu> Hello James, Instructions for MySQL database setup for the Genome Browser are here: http://genome-test.cse.ucsc.edu/~kent/src/unzipped/product/README.mysql.setup Also see the install instructions: http://genome-test.cse.ucsc.edu/~kent/src/unzipped/product/README.install Finally, general help with MySQL commands can be found on the site www.mysql.com . Specifically, see: http://dev.mysql.com/doc/refman/4.1/en/mysql-commands.html I hope this is helpful. -- Brooke Rhead UCSC Genome Bioinformatics Group Zhu, James wrote: > Regarding to my earlier question, how to run this command line "mysql > -youraccountoptions hgcentral < hgcentral.sql" to setup "hgcentral" > tables? > > James > > > _______________________________________________ > Genome-mirror mailing list > Genome-mirror at soe.ucsc.edu > http://www.soe.ucsc.edu/mailman/listinfo/genome-mirror From forgetta at gmail.com Mon Jun 16 15:56:53 2008 From: forgetta at gmail.com (Vince Forgetta) Date: Mon, 16 Jun 2008 18:56:53 -0400 Subject: [Genome-mirror] wigMaf error: Allocated too much memory - more than 2, 147, 483, 647 bytes (2, 147, 490, 617) Message-ID: Hi all, I have successfully loaded a multiple alignment (11-way alignment of a 4Mb bacterial genomes). The genomes are each in one fasta record. I have successfully loaded the maf and mafSummary table. The tracks appear fine in the browser. However, when I click on the track to get a more detailed alignment I get the following error: Allocated too much memory - more than 2,147,483,647 bytes (2,147,490,617) The track definitions is below. Is there any documentation on how to load multiple alignments? All I found was the "Whole genome alignment howto" on the wiki. Thanks for the help. Vince track multiz11way shortLabel Alignment longLabel Multiz Alignment group cdiffCompGeno summary multiz11waySummary priority 104.2 visibility pack color 0, 10, 100 altColor 0,90,10 speciesOrder cdqad1 cdqaf1 cdqaa1 cduaa1 cdqac1 cdsbi1 cdeaa1 cdqab1 cdsst1 cduab1 cdqae1 type wigMaf 0.0 1.0 From rhead at soe.ucsc.edu Mon Jun 16 17:47:42 2008 From: rhead at soe.ucsc.edu (Brooke Rhead) Date: Mon, 16 Jun 2008 17:47:42 -0700 Subject: [Genome-mirror] Deflecting cgi-bin hits In-Reply-To: <4856830B.5050609@helix.nih.gov> References: <4856830B.5050609@helix.nih.gov> Message-ID: <485709AE.4050706@soe.ucsc.edu> Hi David, There is a bit of software in the kent source tree called "bottleneck". If you type the command with no arguments, you will get a usage message: ---- $ bottleneck bottleneck v2 - A server that helps slow down hyperactive web robots usage: bottleneck start Start up bottleneck server bottleneck query ip-address [count] Ask bottleneck server how long to wait to service ip-address bottleneck list List accessing sites bottleneck set ip-address milliseconds Set current delay for given ip-address options: -port=XXXX - Use specific tcp/ip port. Default 17776. -host=XXXXX - Use specific host. Default localhost. -subnet=WWW.XXX.YYY.ZZZ Restrict access to subnet (example 192.168.255.255) -penalty=N - Penalty (in milliseconds) for each access, default 150 -recovery=N - Amount to recover (in milliseconds) for each second between accesses. Default 10 Note penalty and recovery if moved from defaults should balance At the default settings an equilibrium will be achieved when queries are spaced 15 seconds apart. The maximum delay should thus be 15 seconds It will take 25 minutes of idleness for the maximum delay to decay back to zero. ---- You will also need to set up a few lines in your hg.conf file: # Bottleneck info for taming hyperactive web bots bottleneck.host=(your server) bottleneck.port=17776 -- Brooke Rhead UCSC Genome Bioinformatics Group David Hoover wrote: > Besides the robots.txt file, is there any other mechanism for throttling > or deflecting the massive numbers of automated hits on dynamically > generated pages from cgi-bin, either from robots or otherwise? > > David Hoover, Helix Systems, CIT/NIH > _______________________________________________ > Genome-mirror mailing list > Genome-mirror at soe.ucsc.edu > http://www.soe.ucsc.edu/mailman/listinfo/genome-mirror From rhead at soe.ucsc.edu Mon Jun 16 18:03:37 2008 From: rhead at soe.ucsc.edu (Brooke Rhead) Date: Mon, 16 Jun 2008 18:03:37 -0700 Subject: [Genome-mirror] wigMaf error: Allocated too much memory - more than 2, 147, 483, 647 bytes (2, 147, 490, 617) In-Reply-To: References: Message-ID: <48570D69.4040602@soe.ucsc.edu> Hello Vince, There are instructions in src/hg/makeDb/trackDb/README for setting up all types of trackDb entries. The instructions for "type wigMaf" are quite long, so I won't paste them here. You can see an example of the trackDb setup that we use in src/hg/makeDb/trackDb/human/hg18/trackDb.ra (look for "track multiz28way"). Also see the instructions in src/product/README.trackDb . I hope this information is helpful. If you have further questions, please feel free to contact us again at genome-mirror at soe.ucsc.edu. -- Brooke Rhead UCSC Genome Bioinformatics Group Vince Forgetta wrote: > Hi all, > > I have successfully loaded a multiple alignment (11-way alignment of a 4Mb > bacterial genomes). The genomes are each in one fasta record. I have > successfully loaded the maf and mafSummary table. The tracks appear fine in > the browser. However, when I click on the track to get a more detailed > alignment I get the following error: > > Allocated too much memory - more than 2,147,483,647 bytes (2,147,490,617) > > The track definitions is below. Is there any documentation on how to load > multiple alignments? All I found was the "Whole genome alignment howto" on > the wiki. > > Thanks for the help. > > Vince > > track multiz11way > shortLabel Alignment > longLabel Multiz Alignment > group cdiffCompGeno > summary multiz11waySummary > priority 104.2 > visibility pack > color 0, 10, 100 > altColor 0,90,10 > speciesOrder cdqad1 cdqaf1 cdqaa1 cduaa1 cdqac1 cdsbi1 cdeaa1 cdqab1 cdsst1 > cduab1 cdqae1 > type wigMaf 0.0 1.0 > _______________________________________________ > Genome-mirror mailing list > Genome-mirror at soe.ucsc.edu > http://www.soe.ucsc.edu/mailman/listinfo/genome-mirror From James.Zhu at ARS.USDA.GOV Tue Jun 17 09:22:40 2008 From: James.Zhu at ARS.USDA.GOV (Zhu, James) Date: Tue, 17 Jun 2008 12:22:40 -0400 Subject: [Genome-mirror] Compiling Errors Message-ID: <6B4AF69268EFAE4C80A786BA99F7A66702BC670B@MD-MAIL-01.ARSNET.ARS.USDA.GOV> I used CVS to download and compile the executable file and have the following errors: make[3]: *** [doInstall] Error 1 make[3]: Leaving directory `/var/www/cgi-bin/kent/src/hg/js' make[2]: *** [user] Error 2 make[2]: Leaving directory `/var/www/cgi-bin/kent/src/hg/js' make[1]: *** [cgi] Error 2 make[1]: Leaving directory `/var/www/cgi-bin/kent/src/hg' make: *** [cgi] Error 2 What is wrong? The OS is a Redhat Enterprise 4. James From aamp at ucsc.edu Tue Jun 17 09:51:21 2008 From: aamp at ucsc.edu (Andy Pohl) Date: Tue, 17 Jun 2008 09:51:21 -0700 Subject: [Genome-mirror] v184 Genome Browser available Message-ID: <9fa943760806170951gac4afeeyef4e7fcf06930a72@mail.gmail.com> Hello, The v184 source is now available at: http://hgdownload.cse.ucsc.edu/admin/jksrc.zip or labeled with source number: http://hgdownload.cse.ucsc.edu/admin/jksrc.v184.zip cartDump cartReset das hgc hgBlat hgConvert hgCustom hgGateway hgGene hgGenome hgLiftOver hgNear hgPcr hgSession hgTables hgTracks hgTrackUi hgVisiGene mkEncodeFrameset hgEncodeDataVersions pbGateway pbGlobal pbTracks phyloGif /usr/local/apache/cgi-bin/all.joiner /usr/local/apache/cgi-bin/galaAvail.tab /usr/local/apache/cgi-bin/hgNearData/* /usr/local/apache/cgi-bin/hgGeneData/* /usr/local/apache/cgi-bin/hgcData/* /usr/local/apache/cgi-bin/hgCgiData/* /usr/local/apache/cgi-bin/loader/* /usr/local/apache/cgi-bin/visiGeneData/* /usr/local/apache/htdocs/js/* Contents of v184 may be seen at: http://genome-test.cse.ucsc.edu/builds/versions.html Please check both v184-preview and v184-final to see all the code and data changes in this release. Andy Pohl UCSC CBSE From larrym at soe.ucsc.edu Tue Jun 17 09:52:30 2008 From: larrym at soe.ucsc.edu (Larry Meyer) Date: Tue, 17 Jun 2008 09:52:30 -0700 Subject: [Genome-mirror] Compiling Errors In-Reply-To: <6B4AF69268EFAE4C80A786BA99F7A66702BC670B@MD-MAIL-01.ARSNET.ARS.USDA.GOV> References: <6B4AF69268EFAE4C80A786BA99F7A66702BC670B@MD-MAIL-01.ARSNET.ARS.USDA.GOV> Message-ID: <20080617165230.GA29572@soe.ucsc.edu> I think you need to set DOCUMENTROOT to point to your apache DOCUMENTROOT; see: kent/src/product/README.building.source for details. Larry Meyer On Tue, Jun 17, 2008 at 12:22:40PM -0400, Zhu, James wrote: > I used CVS to download and compile the executable file and have the > following errors: > > make[3]: *** [doInstall] Error 1 > make[3]: Leaving directory `/var/www/cgi-bin/kent/src/hg/js' > make[2]: *** [user] Error 2 > make[2]: Leaving directory `/var/www/cgi-bin/kent/src/hg/js' > make[1]: *** [cgi] Error 2 > make[1]: Leaving directory `/var/www/cgi-bin/kent/src/hg' > make: *** [cgi] Error 2 > > What is wrong? The OS is a Redhat Enterprise 4. > > > James > _______________________________________________ > Genome-mirror mailing list > Genome-mirror at soe.ucsc.edu > http://www.soe.ucsc.edu/mailman/listinfo/genome-mirror From forgetta at gmail.com Wed Jun 18 07:53:32 2008 From: forgetta at gmail.com (Vince Forgetta) Date: Wed, 18 Jun 2008 10:53:32 -0400 Subject: [Genome-mirror] wigMaf error: Allocated too much memory - more than 2, 147, 483, 647 bytes (2, 147, 490, 617) In-Reply-To: <48570D69.4040602@soe.ucsc.edu> References: <48570D69.4040602@soe.ucsc.edu> Message-ID: Hi again, After reading the manuals and reloading the tables I still get the same problem ... specifically hgc seems to consume all the memory of the server and then crashes. From what I can tell the tables load correctly: $ hgLoadMaf cdqac1 multiz11way Advisory lock created Indexing and tabulating /gbdb/cdqac1/multiz11way/tba.maf Loading multiz11way into database Loaded 1800 mafs in 1 files from /gbdb/cdqac1/multiz11way Advisory lock has been released $ hgLoadMafSummary cdqac1 -minSize=30000 -mergeGap=1500 -maxSize=200000 multiz11waySummary cdqac1/multiz11way/tba.maf Indexing and tabulating cdqac1/multiz11way/tba.maf Created 1417 summary blocks from 13844 components and 1800 mafs from cdqac1/multiz11way/tba.maf Loading into cdqac1 table multiz11waySummary... Loading completeAdvisory lock has been released I am using multiz/TBA to generate the multiple alignment. Should I be using autoMZ? Should the maf file be post processed? ... apart from using maf_project and removing single "#" the comment lines (hgLoadMaf does not like comment lines. Thanks. Vince On Mon, Jun 16, 2008 at 9:03 PM, Brooke Rhead wrote: > Hello Vince, > > There are instructions in src/hg/makeDb/trackDb/README for setting up all > types of trackDb entries. The instructions for "type wigMaf" are quite > long, so I won't paste them here. You can see an example of the trackDb > setup that we use in src/hg/makeDb/trackDb/human/hg18/trackDb.ra (look for > "track multiz28way"). > > Also see the instructions in src/product/README.trackDb . > > I hope this information is helpful. If you have further questions, please > feel free to contact us again at genome-mirror at soe.ucsc.edu. > > -- > Brooke Rhead > UCSC Genome Bioinformatics Group > > > Vince Forgetta wrote: > >> Hi all, >> >> I have successfully loaded a multiple alignment (11-way alignment of a 4Mb >> bacterial genomes). The genomes are each in one fasta record. I have >> successfully loaded the maf and mafSummary table. The tracks appear fine >> in >> the browser. However, when I click on the track to get a more detailed >> alignment I get the following error: >> >> Allocated too much memory - more than 2,147,483,647 bytes (2,147,490,617) >> >> The track definitions is below. Is there any documentation on how to load >> multiple alignments? All I found was the "Whole genome alignment howto" >> on >> the wiki. >> >> Thanks for the help. >> >> Vince >> >> track multiz11way >> shortLabel Alignment >> longLabel Multiz Alignment >> group cdiffCompGeno >> summary multiz11waySummary >> priority 104.2 >> visibility pack >> color 0, 10, 100 >> altColor 0,90,10 >> speciesOrder cdqad1 cdqaf1 cdqaa1 cduaa1 cdqac1 cdsbi1 cdeaa1 cdqab1 >> cdsst1 >> cduab1 cdqae1 >> type wigMaf 0.0 1.0 >> _______________________________________________ >> Genome-mirror mailing list >> Genome-mirror at soe.ucsc.edu >> http://www.soe.ucsc.edu/mailman/listinfo/genome-mirror >> > From forgetta at gmail.com Wed Jun 18 11:17:59 2008 From: forgetta at gmail.com (Vince Forgetta) Date: Wed, 18 Jun 2008 14:17:59 -0400 Subject: [Genome-mirror] wigMaf error: Allocated too much memory - more than 2, 147, 483, 647 bytes (2, 147, 490, 617) In-Reply-To: References: <48570D69.4040602@soe.ucsc.edu> Message-ID: Hi again, Nailed the bug ... but not sure how to kill it. For some reason the reference sequence is appearing as a sub-track in the multiple alignment i.e. browser for genome A has a subtrack entry for genome A in the multiple alignment. When I remove this track using the setting page I am able to view the detailed alignment. Does anyone know how to suppress the alignment rows from the reference genome from being loaded into the database? Thanks. Vince On Wed, Jun 18, 2008 at 10:53 AM, Vince Forgetta wrote: > Hi again, > > After reading the manuals and reloading the tables I still get the same > problem ... specifically hgc seems to consume all the memory of the server > and then crashes. From what I can tell the tables load correctly: > > $ hgLoadMaf cdqac1 multiz11way > Advisory lock created > Indexing and tabulating /gbdb/cdqac1/multiz11way/tba.maf > Loading multiz11way into database > Loaded 1800 mafs in 1 files from /gbdb/cdqac1/multiz11way > Advisory lock has been released > > $ hgLoadMafSummary cdqac1 -minSize=30000 -mergeGap=1500 -maxSize=200000 > multiz11waySummary cdqac1/multiz11way/tba.maf > Indexing and tabulating cdqac1/multiz11way/tba.maf > Created 1417 summary blocks from 13844 components and 1800 mafs from > cdqac1/multiz11way/tba.maf > Loading into cdqac1 table multiz11waySummary... > Loading completeAdvisory lock has been released > > I am using multiz/TBA to generate the multiple alignment. Should I be > using autoMZ? > > Should the maf file be post processed? ... apart from using maf_project and > removing single "#" the comment lines (hgLoadMaf does not like comment > lines. > > Thanks. > > Vince > > > On Mon, Jun 16, 2008 at 9:03 PM, Brooke Rhead wrote: > >> Hello Vince, >> >> There are instructions in src/hg/makeDb/trackDb/README for setting up all >> types of trackDb entries. The instructions for "type wigMaf" are quite >> long, so I won't paste them here. You can see an example of the trackDb >> setup that we use in src/hg/makeDb/trackDb/human/hg18/trackDb.ra (look for >> "track multiz28way"). >> >> Also see the instructions in src/product/README.trackDb . >> >> I hope this information is helpful. If you have further questions, please >> feel free to contact us again at genome-mirror at soe.ucsc.edu. >> >> -- >> Brooke Rhead >> UCSC Genome Bioinformatics Group >> >> >> Vince Forgetta wrote: >> >>> Hi all, >>> >>> I have successfully loaded a multiple alignment (11-way alignment of a >>> 4Mb >>> bacterial genomes). The genomes are each in one fasta record. I have >>> successfully loaded the maf and mafSummary table. The tracks appear fine >>> in >>> the browser. However, when I click on the track to get a more detailed >>> alignment I get the following error: >>> >>> Allocated too much memory - more than 2,147,483,647 bytes (2,147,490,617) >>> >>> The track definitions is below. Is there any documentation on how to load >>> multiple alignments? All I found was the "Whole genome alignment howto" >>> on >>> the wiki. >>> >>> Thanks for the help. >>> >>> Vince >>> >>> track multiz11way >>> shortLabel Alignment >>> longLabel Multiz Alignment >>> group cdiffCompGeno >>> summary multiz11waySummary >>> priority 104.2 >>> visibility pack >>> color 0, 10, 100 >>> altColor 0,90,10 >>> speciesOrder cdqad1 cdqaf1 cdqaa1 cduaa1 cdqac1 cdsbi1 cdeaa1 cdqab1 >>> cdsst1 >>> cduab1 cdqae1 >>> type wigMaf 0.0 1.0 >>> _______________________________________________ >>> Genome-mirror mailing list >>> Genome-mirror at soe.ucsc.edu >>> http://www.soe.ucsc.edu/mailman/listinfo/genome-mirror >>> >> > From forgetta at gmail.com Wed Jun 18 11:52:55 2008 From: forgetta at gmail.com (Vince Forgetta) Date: Wed, 18 Jun 2008 14:52:55 -0400 Subject: [Genome-mirror] wigMaf error: Allocated too much memory - more than 2, 147, 483, 647 bytes (2, 147, 490, 617) In-Reply-To: References: <48570D69.4040602@soe.ucsc.edu> Message-ID: Doh! Problem solved. Can't include the reference species name in the speciesOrder track field. On Wed, Jun 18, 2008 at 2:17 PM, Vince Forgetta wrote: > Hi again, > > Nailed the bug ... but not sure how to kill it. For some reason the > reference sequence is appearing as a sub-track in the multiple alignment > i.e. browser for genome A has a subtrack entry for genome A in the multiple > alignment. When I remove this track using the setting page I am able to > view the detailed alignment. > > Does anyone know how to suppress the alignment rows from the reference > genome from being loaded into the database? > > Thanks. > > Vince > > > On Wed, Jun 18, 2008 at 10:53 AM, Vince Forgetta > wrote: > >> Hi again, >> >> After reading the manuals and reloading the tables I still get the same >> problem ... specifically hgc seems to consume all the memory of the server >> and then crashes. From what I can tell the tables load correctly: >> >> $ hgLoadMaf cdqac1 multiz11way >> Advisory lock created >> Indexing and tabulating /gbdb/cdqac1/multiz11way/tba.maf >> Loading multiz11way into database >> Loaded 1800 mafs in 1 files from /gbdb/cdqac1/multiz11way >> Advisory lock has been released >> >> $ hgLoadMafSummary cdqac1 -minSize=30000 -mergeGap=1500 -maxSize=200000 >> multiz11waySummary cdqac1/multiz11way/tba.maf >> Indexing and tabulating cdqac1/multiz11way/tba.maf >> Created 1417 summary blocks from 13844 components and 1800 mafs from >> cdqac1/multiz11way/tba.maf >> Loading into cdqac1 table multiz11waySummary... >> Loading completeAdvisory lock has been released >> >> I am using multiz/TBA to generate the multiple alignment. Should I be >> using autoMZ? >> >> Should the maf file be post processed? ... apart from using maf_project >> and removing single "#" the comment lines (hgLoadMaf does not like comment >> lines. >> >> Thanks. >> >> Vince >> >> >> On Mon, Jun 16, 2008 at 9:03 PM, Brooke Rhead wrote: >> >>> Hello Vince, >>> >>> There are instructions in src/hg/makeDb/trackDb/README for setting up all >>> types of trackDb entries. The instructions for "type wigMaf" are quite >>> long, so I won't paste them here. You can see an example of the trackDb >>> setup that we use in src/hg/makeDb/trackDb/human/hg18/trackDb.ra (look for >>> "track multiz28way"). >>> >>> Also see the instructions in src/product/README.trackDb . >>> >>> I hope this information is helpful. If you have further questions, >>> please feel free to contact us again at genome-mirror at soe.ucsc.edu. >>> >>> -- >>> Brooke Rhead >>> UCSC Genome Bioinformatics Group >>> >>> >>> Vince Forgetta wrote: >>> >>>> Hi all, >>>> >>>> I have successfully loaded a multiple alignment (11-way alignment of a >>>> 4Mb >>>> bacterial genomes). The genomes are each in one fasta record. I have >>>> successfully loaded the maf and mafSummary table. The tracks appear >>>> fine in >>>> the browser. However, when I click on the track to get a more detailed >>>> alignment I get the following error: >>>> >>>> Allocated too much memory - more than 2,147,483,647 bytes >>>> (2,147,490,617) >>>> >>>> The track definitions is below. Is there any documentation on how to >>>> load >>>> multiple alignments? All I found was the "Whole genome alignment howto" >>>> on >>>> the wiki. >>>> >>>> Thanks for the help. >>>> >>>> Vince >>>> >>>> track multiz11way >>>> shortLabel Alignment >>>> longLabel Multiz Alignment >>>> group cdiffCompGeno >>>> summary multiz11waySummary >>>> priority 104.2 >>>> visibility pack >>>> color 0, 10, 100 >>>> altColor 0,90,10 >>>> speciesOrder cdqad1 cdqaf1 cdqaa1 cduaa1 cdqac1 cdsbi1 cdeaa1 cdqab1 >>>> cdsst1 >>>> cduab1 cdqae1 >>>> type wigMaf 0.0 1.0 >>>> _______________________________________________ >>>> Genome-mirror mailing list >>>> Genome-mirror at soe.ucsc.edu >>>> http://www.soe.ucsc.edu/mailman/listinfo/genome-mirror >>>> >>> >> > From hooverdm at helix.nih.gov Mon Jun 23 14:44:42 2008 From: hooverdm at helix.nih.gov (David Hoover) Date: Mon, 23 Jun 2008 17:44:42 -0400 Subject: [Genome-mirror] tfbs conserved track Message-ID: <4860194A.5090907@helix.nih.gov> We have an in-house GB mirror, and we also subscribe to the Biobase TRANSFAC database. The track TFBS Conserved uses version 7.0, while we have access to version 12.1. Is there a simple way of generating the TFBS Conserved track using the latest TRANSFAC flat file database? Thanks, David Hoover, CIT/NIH