There are multiple ways to use The Dark Crawler. Based upon your needs, you can access it while still being hosted at its current location. However, if the system is to be installed in your facility, there are certain restrictions to consider.
SOFTWARE
Operating System Required: Windows
Database Required: MySQL
PERFORMANCE
Each instance of the crawler runs on threads (default: 25). We average downloading ~60,000 webpages per day, per crawler. Thus, each thread downloads ~2,400 webpages per day. Traffic includes images.
LICENSING OPTIONS
CLOUD-BASED
- Licensing can be per thread: download ~2,400 webpages per day per thread
- License can be per crawler computer: download ~60,000 webpages per day per thread
- Crawls can be segregated per user so data remains private
PRIVATE HOSTING
- Capacity depends on your infrastructure, use-case, and available bandwidth
- Information stored in private database
DATA AS A SERVICE (DaaS)
- Infrastructure stays with us
- Licensed for monitoring per website
- Small website = $ ? / month
- Larger website = $ ?? / month