|
A powerful web crawler utility for webmasters to develop search directory, web site promotion. It extracts URL, meta tag (title, desc, keyword), body text, page size, last modified date value from web site, search results or list of URLs. High speed, multi-threaded, accurate
extraction - directly saves data to disk file. Program has numerous filters to restrict
session, like - URL filter, text filter, data filter, domain filter, date modified, etc.
It allows user-selectable recursion levels, retrieval threads, timeout, proxy support
and many other options.
|
