Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the photoshoot-lite domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home/n5nrlvvge9l2/public_html/wp-includes/functions.php on line 6121
Resources – The Dark Crawler

Resources

There are multiple ways to use The Dark Crawler. Based upon your needs, you can access it while still being hosted at its current location. However, if the system is to be installed in your facility, there are certain restrictions to consider.


SOFTWARE

Operating System Required: Windows

Database Required: MySQL

PERFORMANCE

Each instance of the crawler runs on threads (default: 25). We average downloading ~60,000 webpages per day, per crawler. Thus, each thread downloads ~2,400 webpages per day. Traffic includes images.


LICENSING OPTIONS

CLOUD-BASED

  • Licensing can be per thread: download ~2,400 webpages per day per thread
  • License can be per crawler computer: download ~60,000 webpages per day per thread
  • Crawls can be segregated per user so data remains private

PRIVATE HOSTING

  • Capacity depends on your infrastructure, use-case, and available bandwidth
  • Information stored in private database

DATA AS A SERVICE (DaaS)

  • Infrastructure stays with us
  • Licensed for monitoring per website
  • Small website = $ ? / month
  • Larger website = $ ?? / month