[Genome] download browser graphics programmatically

Thu May 17 10:55:15 PDT 2007

wget --post-data or --post-file may help.  Using cookies (there's an 
example in the --post-data section of the wget man page) will help to 
save state -- for example, you could do a post request to set subtrack 
visibilities and --save-cookies.  Then you could do multiple 
--load-cookie requests where you just tweak the position, for example, 
and the use of cookies will recall your subtrack settings (and almost 
all other settings that have been set).

Better yet, in that first request where you --save-cookies, parse the 
hgsid integer from the output and save that off too; then pass it back 
in subsequent requests.  That is more efficient because we don't have 
to create a new user-session for each subsequent wget command.

But that's getting fairly complicated; if you're familiar with Perl, 
it might be just as easy to write a script using the libraries 
LWP::UserAgent, HTTP::Cookies and HTML::Form instead of scripting 
around wget.  

That's strange about wget -O causing the GIF to be embedded.  If the 
HTML is still intact enough so that you can parse out the .gif 
filename, you can fetch the .gif file separately with yet another wget 
command.  (but again, if getting that complex, may be better to write 
a perl/python/whatever script with finer control over the transfers)

On Thu, 17 May 2007, Hiram Clawson wrote:

> Good Morning Bob:
> 
> If you lose the --no-directories argument, the results will go
> to individual files in a created sub-directory hierarchy to
> perhaps avoid the long-name problem.  Perhaps, I don't know.
> There are so many arguments to wget, some combination should
> be able to work.
> 
> At this time there are no browser visibility lines for the sub-tracks.
> We're considering how to implement such a feature in the future.
> 
> --Hiram
> 
> Bob Thurman wrote:
> > Thanks, Hiram.  I tried the URL change you suggested for subtracks, but 
> > that just makes the wget-downloaded html file name too long for my 
> > operating system.  And if I use the -O option for wget, then the gifs 
> > get embedded in that output file, in such a way that I can't see how to 
> > extract them.
> > These subtrack select options have to go in the URL?  No way to use them 
> > in a "browser" line in the custom track file?
> > 
> > Thanks again,
> > 
> > Bob Thurman
> > 
> > Hiram Clawson wrote:
> >> Good Morning Bob:
> >>
> >> To set the width of your gif image, use a pix=N argument in your
> >> URL, for example: ... hgTracks?pix=1000&hgt.customText= ...
> >> to set size of 1000.  The maximum allowed width is 5000.
> >>
> >> Your later email indicated you found the bottom of your gif image.
> >>
> >> At this time we do not have a simple mechanism to set some sub-tracks
> >> on and some sub-tracks off for a single composite track.  You would have
> >> to laboriously set each sub-track on or off in your URL, it would be
> >> a lot of variables.  For example:
> >> encodeSangerChip=full  - to get any of the "on" sub-tracks in full mode
> >> then each individual track, is a 1 or 0 for on or off, here is an example
> >> with only three tracks on:
> >> encodeSangerChipH4acHeLa_sel=1
> >> encodeSangerChipH3K4me3HeLa_sel=1
> >> encodeSangerChipH3acHeLa_sel=1
> >> encodeSangerChipCTCF_sel=0
> >> encodeSangerChipH3K27me3_sel=0
> >> encodeSangerChipH3K36me3_sel=0
> >> encodeSangerChipH3K4me1HFL1_sel=0
> >> encodeSangerChipH3K4me1HeLa_sel=0
> >> encodeSangerChipH3K4me1Molt4_sel=0
> >> encodeSangerChipH3K4me1Ptr8_sel=0
> >> encodeSangerChipH3K4me1_sel=0
> >> encodeSangerChipH3K4me2HFL1_sel=0
> >> encodeSangerChipH3K4me2HeLa_sel=0
> >> encodeSangerChipH3K4me2K562_sel=0
> >> encodeSangerChipH3K4me2Molt4_sel=0
> >> encodeSangerChipH3K4me2Ptr8_sel=0
> >> encodeSangerChipH3K4me2_sel=0
> >> encodeSangerChipH3K4me3HFL1_sel=0
> >> encodeSangerChipH3K4me3K562_sel=0
> >> encodeSangerChipH3K4me3Molt4_sel=0
> >> encodeSangerChipH3K4me3Ptr8_sel=0
> >> encodeSangerChipH3K4me3_sel=0
> >> encodeSangerChipH3K79me3_sel=0
> >> encodeSangerChipH3K9me3_sel=0
> >> encodeSangerChipH3acHFL1_sel=0
> >> encodeSangerChipH3acK562_sel=0
> >> encodeSangerChipH3acMolt4_sel=0
> >> encodeSangerChipH3ac_sel=0
> >> encodeSangerChipH4acHFL1_sel=0
> >> encodeSangerChipH4acK562_sel=0
> >> encodeSangerChipH4acMolt4_sel=0
> >> encodeSangerChipH4ac_sel=0
> >>
> >> To find these variables for any particular track, set it up as you would
> >> like to see in the genome browser, then alter your WEB browser URL
> >> to read cartDump where it says hgTracks
> >>
> >> --Hiram
> >>
> >> Bob Thurman wrote:
> >>> Great, I can get this to work, but I'm still running into a couple of 
> >>> problems.
> >>>
> >>> 1) I would like to control the width of the display (and hence the 
> >>> gif).  The help on " Displaying Your Own Annotations in the Genome 
> >>> Browser*" *says "Browser lines allow you to configure such things as 
> >>> the genome position that the Genome Browser will initially open to, 
> >>> the width of the display, and the configuration of the other 
> >>> annotation tracks that are shown (or hidden) in the initial 
> >>> display."  But I don't see any attribute name in the "Browser Lines" 
> >>> section that will allow me to control the width.
> >>>
> >>> 2) My gif seems to be getting cut off at the bottom for all the 
> >>> tracks I want to display.  Is there a fixed height for the gif and, 
> >>> if so, is there a way to get around this limit?
> >>>
> >>> 3) I'd like to display individual subtracks from the ENCODE 
> >>> SangerChip tracks (there are over 30).  But the help on Browser Lines 
> >>> says "It is not possible to display only a subset of the subtracks at 
> >>> this time."  True?  Any way around this?
> >>>
> >>> Thanks for your help so far.
> >>>
> >>> Bob Thurman
> >>>
> >>> Hiram Clawson wrote:
> >>>> Good Morning Bob:
> >>>>
> >>>> You can use wget to fetch the browser page and its associated .gif
> >>>> graphic.  The resulting html file ends up with an unusual name,
> >>>> but the .gif file is there too.  For example:
> >>>>
> >>>> wget --no-directories --recursive --follow-tags=img --html-extension \
> >>>> --convert-links \
> >>>> 'http://genome.ucsc.edu/cgi-bin/hgTracks?hgt.customText=http://genome-test.cse.ucsc.edu/~hiram/ctDb/bedWigGtfPsl.txt' 
> >>>>
> >>>>
> >>>> This causes the loading of the custom tracks specified, which also 
> >>>> controls which tracks are
> >>>> displayed and position, note the browser specifications in the 
> >>>> custom track file:
> >>>>     http://genome-test.cse.ucsc.edu/~hiram/ctDb/bedWigGtfPsl.txt
> >>>>
> >>>> The resulting .gif file is named something like: 
> >>>> hgt_genome_34c6_9eae80.gif
> >>>> The hex ID numbers there will be different each time.
> >>>> For this particular example, as a bonus, you also get the ideogram 
> >>>> graphic:
> >>>>      hgtIdeo_genome_34c6_9eae80.gif
> >>>>
> >>>> --Hiram
> >>>>
> >>>> Bob Thurman wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I would like a way to generate from within a program or from a unix 
> >>>>> command line, a browser screen shot displaying tracks that I load 
> >>>>> myself and download that graphic (as with the "PDF/PS" link at the 
> >>>>> top of your page). Is this possible?  Is there a way to do this by 
> >>>>> piping output through a wget command or something?
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Bob Thurman
> >>>>>
> >>>>> Dept. of Genome Sciences
> >>>>> University of Washington
> >>>
> >>
> > 
> _______________________________________________
> Genome maillist  -  Genome at soe.ucsc.edu
> http://www.soe.ucsc.edu/mailman/listinfo/genome
> 

-- 
angie at soe.ucsc.edu
Software Developer, UCSC CBSE / Genome Bioinformatics Group