[Genome] needLargeMem: Out of memory

[Hiram Clawson]

Good Afternoon Helder:

The genome is already split by chromosome in fasta files at:
	http://hgdownload.cse.ucsc.edu/goldenPath/hg18/chromosomes/

Or, you can use the specification:
	file.2bit:seqid
to specify only one chromosome to be used from the 2bit file.

For example:
	blat hg18.2bit:chr1 multi_test.fa output.psl

Please note the output help message from the blat command when
used with no arguments.  You are running stand-alone blat by
doing what you are already trying to do.

--Hiram

Helder Nakaya wrote:
> Hello Galt and Hiram,
> 
> I am not trying to align two genomes. Here is what I have done:
> 
> I want to get the genomic coordinates from sequences of a multifasta
> file. This multifasta file contains only 2 sequences smaller than
> 2,000 nt (it's a test. Later, I will run with a multifasta containing
> 244k sequences with 60-mers).
> 
> So, I am trying to run blat using this command line:
> blat /home/database/hg18.2bit multi_test.fa output
> 
> I use "top" to see if gfServer was running and it was not.
> 
> I would like to know how do I split the genome by chromosome and run
> stand-alone blat.
> 
> All the best,
> 
> Helder
> 
> 2008/4/15, Galt Barber <galt at soe.ucsc.edu>:
>> I don't think Helder is trying to align two entire genomes
>> (human and mouse)
>> to each other.  If that were so, then he needs blastz anyways.
>>
>> Furthermore, though blat targets can be huge genomes,
>> blat queries cannot exceed a certain size, so the queries would have
>> to be broken up into chunks of 5-10k and then the final
>> results would need chaining with tools.  Somewhat complex.
>> As you pointed out, most users would be better off just
>> using our chains.
>>
>> Perhaps Helder can tell us more about what he's
>> really trying to do?
>>
>> -Galt
>>
>>
>> On Tue, 15 Apr 2008, Hiram Clawson wrote:
>>
>>> Good Morning Helder:
>>>
>>> I would guess trying to use both the mm9 and hg18 genomes
>>> together would require about 16 Gb of memory, or thereabouts.
>>>
>>> You *may* be able to do one chromosome vs. one chromosome
>>> at a time.  For example chr1 vs. chr1 which would be the
>>> worst case.  Even that may be too much for a 2 Gb memory
>>> machine, I don't know.  Try chrM vs chrM and work up from
>>> there.
>>>
>>> Your input to blat can be the chr1.fa fasta files.  You do
>>> not need to convert each fasta file to a 2bit file unless
>>> you are trying to save disk space.
>>>
>>> You can also use the genome browser chain and net tracks
>>> between hg18 and mm9.  Those are better alignments than blat
>>> can produce all by itself.
>>>
>>> --Hiram
>>>
>>> Helder Nakaya wrote:
>>>> Hey Galt, thanks.
>>>>
>>>> I'm trying to load hg18 and mm9. I have already converted all
>>>> chromosome .fa files to one single hg18.2bit file (with FaToTwoBit).
>>>>
>>>> I'm running stand-alone blat. But I have tried to load sequence genome
>>>> with gfServer before run blat.
>>>>
>>>> If I split the genome by chromosomes, will the database be the
>>>> directory containing the chr.2bit files?
>>>>
>>>> Best,
>>>>
>>>> Helder
>>>>
>>>> 2008/4/15, Galt Barber <galt at soe.ucsc.edu>:
>>>>> It means you are out of memory.
>>>>>  #define ENOMEM 12
>>>>>
>>>>> Which genome are you trying to load?
>>>>> How big is it?
>>>>> Are you running stand-alone blat? or gfServer/gfClient?
>>>>> What does your command-line look like?
>>>>>
>>>>> Typically when people don't have enough RAM,
>>>>> we advise them to split the database of the genome
>>>>> by chromosome, and do one blat run per chrom.
>>>>>
>>>>> One then needs to use utilities like
>>>>> pslReps or pslCDnaFilter
>>>>> to combine the resulting psl result files.
>>>>>
>>>>> -Galt
>>>>>
>>>>>
>>>>> On Tue, 15 Apr 2008, Helder Nakaya wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm new on this list and maybe this is not a novel question.
>>>>>>
>>>>>> I have instaled the blatSrc34 on my laptop (UBUNTU, system type:
>>>>>> 32-bit, processor: Intel(R) Core2 Duo CPU, T7250, 2GHz) with 2038MB
>>>>>> (RAM).
>>>>>>
>>>>>> When I try to run BLAT, it gives me this error message:
>>>>>> "needLargeMem: Out of memory - request size 158821425 bytes, errno: 12"
>>>>>>
>>>>>> I would like to know if someone has also had this problem and how do I solve it.
>>>>>>
>>>>>> I think I need to allocate dynamic memory in memalloc.c program but I
>>>>>> don't know how to do this.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Helder