Table of Contents - spidercrawl-0.3.9 Documentation
Classes and Modules
- Spidercrawl
- Spidercrawl::Page
- Spidercrawl::ParallelRequest
- Spidercrawl::Request
- Spidercrawl::SpiderWorker
- Spiderman
- UserAgents
Methods
- ::new — Spidercrawl::Page
- ::new — Spidercrawl::Request
- ::new — Spidercrawl::ParallelRequest
- ::new — Spidercrawl::SpiderWorker
- ::random — UserAgents
- ::shoot — Spiderman
- #absolutify — Spidercrawl::Page
- #after_fetch — Spidercrawl::SpiderWorker
- #base_url — Spidercrawl::Page
- #before_fetch — Spidercrawl::SpiderWorker
- #content — Spidercrawl::Page
- #content_type — Spidercrawl::Page
- #crawl — Spidercrawl::SpiderWorker
- #css — Spidercrawl::Page
- #curl — Spidercrawl::Request
- #doc — Spidercrawl::Page
- #emails — Spidercrawl::Page
- #external_links — Spidercrawl::Page
- #fetch — Spidercrawl::Request
- #fetch — Spidercrawl::ParallelRequest
- #headers — Spidercrawl::Page
- #host — Spidercrawl::Page
- #images — Spidercrawl::Page
- #internal_links — Spidercrawl::Page
- #links — Spidercrawl::Page
- #meta_descriptions — Spidercrawl::Page
- #meta_keywords — Spidercrawl::Page
- #not_found? — Spidercrawl::Page
- #on_failure — Spidercrawl::SpiderWorker
- #on_redirect — Spidercrawl::SpiderWorker
- #on_success — Spidercrawl::SpiderWorker
- #parallel_crawl — Spidercrawl::SpiderWorker
- #redirect? — Spidercrawl::Page
- #response_code — Spidercrawl::Page
- #scheme — Spidercrawl::Page
- #setup_page — Spidercrawl::SpiderWorker
- #success? — Spidercrawl::Page
- #text — Spidercrawl::Page
- #title — Spidercrawl::Page
- #url — Spidercrawl::Page
- #words — Spidercrawl::Page