Skip to content
  • mail@thedarkcrawler.com

The Dark Crawler

Exploring the dark corners of the Internet

Menu
  • Homepage
  • Use Cases
    • Website Analysis
    • Network Exploration
    • Coordinating Reporting
    • Usage Scenarios
  • Features
    • Scalability
    • Maintenance
    • Licensing
    • Reporting Tools
  • Resources
  • Blog
  • Login

Homepage

The Dark Crawler is a customizable Web intelligence platform for cybercrime researchers, investigators and concerned organizations.

HOW DOES IT WORK?

AN AUTOMATED SYSTEM

The Dark Crawler is a web‐based application that, given a set of user‐specified starting websites, will recursively follow the links out of that site, both on the Clear-Web Internet and the hidden Dark-Web.


FOR RESEARCHERS AND INVESTIGATORS

Scalability, versatility and a breadth of available analytical tools make the Dark Crawler a valuable solution for collecting and analyzing digital evidence of online criminal activity.

HOW DID IT START OUT?

A DATA COLLECTION TOOL

Initiated by Mercur IT Solutions, Inc., the development of the Dark Crawler started in 2010 as a course project in the School of Criminology at Simon Fraser University.

The Crawler was designed to assist in data collection for a course project centered around child mapping child exploitation material (CEM), so the researcher would not have to manually visit and map each potential website.


WIDER APPLICATIONS

The tool has evolved through various research projects in close collaboration between many other participants, including support from the Government of Canada and law enforcement agencies.

Based on the initial successes of the research projects Mercur IT Solutions has continued to develop this tool into a practical, user-friendly product, that can produce tangible results in a variety of settings.

WHAT IS IT USED FOR?

STUDYING CYBERCRIME

Given its roots in academia, the system is still heavily used for academic research. So far, various applications of the Dark Crawler resulted in 15 academic publications and 21 presentations, including at the United Nations Office on Drugs and Crime in Doha, Qatar (2015).


POLICY AND PRACTICE

The Dark Crawler allows law enforcement to prioritize various online criminal activity within their jurisdiction based on which websites are either hosted or owned within their own jurisdictions.

Why Choose the Dark Crawler?

SCALABILITY

Scalable architecture allows to connect more computers, increasing performance linearly up to an order of magnitude.

LICENSING

Each client can select the licensing plan most suitable to their goals. The Dark Crawler can be hosted in the “cloud”, or on local hardware and networks, or accessed on a “data as a service” basis.

MAINTENANCE

Scalable architecture allows to connect more computers, increasing performance linearly up to an order of magnitude.

REPORTING TOOLS

Each client can select the licensing plan most suitable to their goals. The Dark Crawler can be hosted in the “cloud”, or on local hardware and networks, or accessed on a “data as a service” basis.

  • April 2021
  • March 2021
  • October 2020
  • August 2019
  • June 2019
  • December 2018
  • October 2018
  • July 2018
  • April 2018
  • September 2017
  • January 2017
  • December 2016
  • November 2016
  • September 2016
  • November 2015

The Dark Crawler 2023 . Powered by WordPress