There are multiple ways to use The Dark Crawler. Based upon your needs, you can access it while still being hosted at its current location. However, if the system is to be installed in your facility, there are certain restrictions to consider.
Operating System Required: Windows
Database Required: MySQL
Each instance of the crawler runs on threads (default: 25). We average downloading ~60,000 webpages per day, per crawler. Thus, each thread downloads ~2,400 webpages per day. Traffic includes images.
- Licensing can be per thread: download ~2,400 webpages per day per thread
- License can be per crawler computer: download ~60,000 webpages per day per thread
- Crawls can be segregated per user so data remains private
- Capacity depends on your infrastructure, use-case, and available bandwidth
- Information stored in private database
data as a service
- Infrastructure stays with us
- Licensed for monitoring per website
- Small website = $ ? / month
- Larger website = $ ?? / month